quality of keywords

the results from examination of LOs - current keywords vz previously assigned

Logged In: YES
user_id=131330
Originator: NO

Lukasz, could this somehow be related to the latest changes?

Logged In: YES
user_id=559872
Originator: NO

1. Claudia changed the English linguistic model (which POS tags qualify words/sequences as potential keyword candidates). The new model is more compact, so some candidates that qualified before the changes may be disqualified now. On the other hand, we hoped to filter out more junk. This is a matter for discussion with Claudia, as I just put her changes into the code.

2. A bug was corrected (for all languages) that made the qualifier ignore "CMLU" category (=POS tags that qualify a word as a potential part of a multi-word keyword candidate, but not as a single-word candidate, for instance VBP, VBD, VBZ, VBN in the current model). Before the bug was corrected, CMLUs were treated as MLUs, i.e. could form one-word keywords. This bugfix can only make the results better - if the linguistic model is sane; the only possible change caused by this bugfix is that some one-word keywords (assigned CMLU category) proposed in the old version are not proposed now. If this is the case, the linguistic model should be changed - appropriate POS should be assigned MLU, not CMLU category. CMLU means "I don't want words of this POS to be one-word keywords".

I can't find the comparison part ("current keywords vz previously assigned") in the file. Nonetheless, it's another thing to be analysed with the knowledge of English POS categories: what candidates disappeared, what was their "POS pattern" (e.g. "JJ NN"), is this pattern forbidden now, what candidates appeared, what was their POS pattern, was this pattern allowed before, ...

Another possibility is that Mirek did not build the current English language model from scratch (AFAIK he did, but he didn't answer Claudia's question openly yet).

Logged In: YES
user_id=131330
Originator: NO

Can we close this one?