Recent changes to bugs

#7 Loop in TokeniserWhitespace.tokenizeToArrayList

2014-12-07T17:12:31.851000Z

I have a fixed version here:

https://github.com/mpkorstanje/simmetrics/blob/master/src/main/java/uk/ac/shef/wit/simmetrics/tokenisers/TokeniserCSVBasic.java

But you'll have to build from source. I'm doing an overhaul of the whole thing.

Loop in TokeniserWhitespace.tokenizeToArrayList

2014-02-12T17:13:58.663000Z

Never ending loop with specific inputs in uk.ac.shef.wit.simmetrics.tokenisers.TokeniserWhitespace.tokenizeToArrayList

sample program:
public static void main(String[] args) {
System.out.println("start");
InterfaceStringMetric l_metric = new MongeElkan();
String l_address_a = "POST OFFICE HOLD & PHONE";
String l_address_b = "2665 - C N HIGHLAND AVE";
float l_score = l_metric.getSimilarity(l_address_a, l_address_b);
System.out.println("end, score=" + l_score);
}

stack trace:
"main" prio=10 tid=0x00007ff70800d800 nid=0x5451 runnable [0x00007ff70ead2000]
java.lang.Thread.State: RUNNABLE
at uk.ac.shef.wit.simmetrics.tokenisers.TokeniserWhitespace.tokenizeToArrayList(TokeniserWhitespace.java:121)
at uk.ac.shef.wit.simmetrics.similaritymetrics.MongeElkan.getSimilarity(MongeElkan.java:170)
at com.mm.server.inventory.app.MergeWorklistFunctions.main(MergeWorklistFunctions.java:213)

String tokensier break down method runs in deadlock

2011-01-07T11:02:39Z

The TokeniserWhitespace.tokenizeToArrayList(String input) runs into deadlock when the input string contains more than one whitespace simultaneously.
I've fixed it in the source code in version 1.6. But can't commit in your provided svn..

Thanks

TagLink constructor message massive performance impact

2010-04-20T13:47:48Z

The (misspelled) performance message "WARNING - this metric is not recomended for fast processing..." is causing a massive performance issue when used in a multithreaded environment.

If several threads each create TagLink algorithm instance, they create contention for the System.out PrintStream which queues all the threads behind each other until the performance message is output.

I have moved (and spellchecked!) the warning to a static block so it is only output when the class is loaded, and not on every construction and I have a 10 times performance improvement. I have also made the code stricter around the use of generics to avoid unneccessary casts.

The irony is not lost on me that a message warning of poor performance is such a massive bottleneck!

Non-breaking space causes infinite loop

2009-09-21T13:59:22Z

The method tokenizeToArrayList in TokeniserWhitespace.java uses two methods to look at whitespace characters: Character.isWhitespace() and the characters in its delimiters field: "\r\n\t \u00A0". Unfortunately, isWhitespace() does not regard a non-breaking space (\u00A0) as a whitespace character, which ends up causing an infinite loop when tokenising a string.

The fix is to test for a non-breaking space character when testing isWhitespace() on line 116:
if (Character.isWhitespace(ch) || (int)ch == 160) {
curPos++;
}

Bug with character encoding

2008-07-07T16:28:07Z

Some of the metrics (for example BlockDistance) fail if one of the strings has a unicode 160 (non-blocking space) in.

Euclidean Distance always returns 0.0

2007-06-28T21:18:19Z

I recently installed simmetrics v 1.6 and i'm having a strange result from euclidean distance function.

I just downloaded the jar and the source, and start playing with the SimpleExample.java file, when trying with the Euclidean distance, it always return 0.0, unless inputing two equal strings.

abb aba return 0.0

abc abd return 0.0

abc abc return 1.0

I tried with a lot of Strings of different sizes, and had the same result.

Luis Ibáñez
ldibanyez@gmail.com

Jaro impemetation

2007-01-02T16:18:14Z

I’ve found two thing wrong in the implementation of the jaro algorithm and since i am not familiar with the cvs i thought i should post‘em here.

1) In the computation of distance the line should be
this.Distance = Math.Min(string1.Length, string2.Length) / 2 + Math.Min(string1.Length, string2.Length) % 2; in order to have a proper rounding

2) and to avoid the left vs right distance difference that shows up sometimes we have to edit the following line:
//compare char with range of characters to either side
for (int j = Math.Max (0, i - distance); !foundIt && j <= Math.Min(i + distance, string2.Length - 1 ); j++)

Keep up the good work!