Recent changes to bugs

#7 Loop in TokeniserWhitespace.tokenizeToArrayList

mpkorstanje — Sun, 07 Dec 2014 17:12:31 -0000

I have a fixed version here:

https://github.com/mpkorstanje/simmetrics/blob/master/src/main/java/uk/ac/shef/wit/simmetrics/tokenisers/TokeniserCSVBasic.java

But you'll have to build from source. I'm doing an overhaul of the whole thing.

Loop in TokeniserWhitespace.tokenizeToArrayList

Mitch Claborn — Wed, 12 Feb 2014 17:13:58 -0000

Never ending loop with specific inputs in uk.ac.shef.wit.simmetrics.tokenisers.TokeniserWhitespace.tokenizeToArrayList

sample program:
public static void main(String[] args) {
System.out.println("start");
InterfaceStringMetric l_metric = new MongeElkan();
String l_address_a = "POST OFFICE HOLD & PHONE";
String l_address_b = "2665 - C N HIGHLAND AVE";
float l_score = l_metric.getSimilarity(l_address_a, l_address_b);
System.out.println("end, score=" + l_score);
}

stack trace:
"main" prio=10 tid=0x00007ff70800d800 nid=0x5451 runnable [0x00007ff70ead2000]
java.lang.Thread.State: RUNNABLE
at uk.ac.shef.wit.simmetrics.tokenisers.TokeniserWhitespace.tokenizeToArrayList(TokeniserWhitespace.java:121)
at uk.ac.shef.wit.simmetrics.similaritymetrics.MongeElkan.getSimilarity(MongeElkan.java:170)
at com.mm.server.inventory.app.MergeWorklistFunctions.main(MergeWorklistFunctions.java:213)

String tokensier break down method runs in deadlock

Zubair Ahmed — Fri, 07 Jan 2011 11:02:39 -0000

The TokeniserWhitespace.tokenizeToArrayList(String input) runs into deadlock when the input string contains more than one whitespace simultaneously.
I've fixed it in the source code in version 1.6. But can't commit in your provided svn..

Thanks

TagLink constructor message massive performance impact

Anonymous — Tue, 20 Apr 2010 13:47:48 -0000

The (misspelled) performance message "WARNING - this metric is not recomended for fast processing..." is causing a massive performance issue when used in a multithreaded environment.

If several threads each create TagLink algorithm instance, they create contention for the System.out PrintStream which queues all the threads behind each other until the performance message is output.

I have moved (and spellchecked!) the warning to a static block so it is only output when the class is loaded, and not on every construction and I have a 10 times performance improvement. I have also made the code stricter around the use of generics to avoid unneccessary casts.

The irony is not lost on me that a message warning of poor performance is such a massive bottleneck!

Non-breaking space causes infinite loop

Craig — Mon, 21 Sep 2009 13:59:22 -0000

The method tokenizeToArrayList in TokeniserWhitespace.java uses two methods to look at whitespace characters: Character.isWhitespace() and the characters in its delimiters field: "\r\n\t \u00A0". Unfortunately, isWhitespace() does not regard a non-breaking space (\u00A0) as a whitespace character, which ends up causing an infinite loop when tokenising a string.

The fix is to test for a non-breaking space character when testing isWhitespace() on line 116:
if (Character.isWhitespace(ch) || (int)ch == 160) {
curPos++;
}

Bug with character encoding

Anonymous — Mon, 07 Jul 2008 16:28:07 -0000

Some of the metrics (for example BlockDistance) fail if one of the strings has a unicode 160 (non-blocking space) in.

Euclidean Distance always returns 0.0

Anonymous — Thu, 28 Jun 2007 21:18:19 -0000

I recently installed simmetrics v 1.6 and i'm having a strange result from euclidean distance function.

I just downloaded the jar and the source, and start playing with the SimpleExample.java file, when trying with the Euclidean distance, it always return 0.0, unless inputing two equal strings.

abb aba return 0.0

abc abd return 0.0

abc abc return 1.0

I tried with a lot of Strings of different sizes, and had the same result.

Luis Ibáñez
ldibanyez@gmail.com

Jaro impemetation

Anonymous — Tue, 02 Jan 2007 16:18:14 -0000

I’ve found two thing wrong in the implementation of the jaro algorithm and since i am not familiar with the cvs i thought i should post‘em here.

1) In the computation of distance the line should be
this.Distance = Math.Min(string1.Length, string2.Length) / 2 + Math.Min(string1.Length, string2.Length) % 2; in order to have a proper rounding

2) and to avoid the left vs right distance difference that shows up sometimes we have to edit the following line:
//compare char with range of characters to either side
for (int j = Math.Max (0, i - distance); !foundIt && j <= Math.Min(i + distance, string2.Length - 1 ); j++)

Keep up the good work!