Multiclass ROC

Anol Paisal — Mon, 06 Feb 2012 10:44:12 -0000

I see PyML.evaluators.roc module, it just has 1 by 1 lable.
Please include one vs rest roc.

Storage of trained kernel data

Anonymous — Tue, 26 Oct 2004 09:01:01 -0000

I use your lib for very large datasets. The training
time is sometimes very long. Is it possible to enhance
the SVM class with a read/write method ?

--
Thilo Wehrmann
thilo.wehrmann@mail.uni-wuerzburg.de

matthews correlation coefficient (mcc) et al.

ozi — Thu, 02 Sep 2004 00:05:53 -0000

matthews correlation coefficient (mcc) is a valuable
tool for model selection.

Especially if classes are very unevenly distributed you
can get acceptable values for ROC, ppv, balancedSuccess
etc. although the classifier assigns all examples to
the same class. mcc gives 0 in such cases. AFAIK the
extension to more than two classes ist straightforward
(i think the Baldi/Brunak book has a formula).

However pure mcc assigns 0 also to perfect classifiers,
so one needs probably a measure which is mcc derived.
The minimum and the standard deviation of the
performance measures obtained by stratifiedCV are also
important indicators and should be given together with
the average.

In summary I would like to see a bit more flexibility
in how the best model is selected from e.g. a ParamGrid.

Thank you.

Recent changes to feature-requests

Multiclass ROC

Storage of trained kernel data

matthews correlation coefficient (mcc) et al.