[
https://issues.apache.org/jira/browse/SPARK-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063155#comment-14063155
]
Xiangrui Meng commented on SPARK-2341:
--------------------------------------
[~srowen] Using enum or string sounds good. As you already knew, using string
may be better because of Python.
Rounding is used because people use either +1/-1 or 1/0 for binary
classification in LIBSVM and we require 1/0 in MLlib. Actually the +1/-1 is the
only corner case I wanted to cover when multiclass=false. We added LIBSVM
support because there are many commonly used datasets we can download from
LIBSVM/LIBLINEAR website and other places. It is easier for people to test
MLlib's algorithms.
It would be nice if you have free cycles to implement a method that convert
classes to numbers. For the long term, I'm thinking about for each dataset, we
can attach metadata that contains feature names, feature types, number of
non-zeros, and for every categorical feature we have a value <-> {0, 1, ...}
map.
> loadLibSVMFile doesn't handle regression datasets
> -------------------------------------------------
>
> Key: SPARK-2341
> URL: https://issues.apache.org/jira/browse/SPARK-2341
> Project: Spark
> Issue Type: Bug
> Components: MLlib
> Affects Versions: 1.0.0
> Reporter: Eustache
> Priority: Minor
> Labels: easyfix
>
> Many datasets exist in LibSVM format for regression tasks [1] but currently
> the loadLibSVMFile primitive doesn't handle regression datasets.
> More precisely, the LabelParser is either a MulticlassLabelParser or a
> BinaryLabelParser. What happens then is that the file is loaded but in
> multiclass mode : each target value is interpreted as a class name !
> The fix would be to write a RegressionLabelParser which converts target
> values to Double and plug it into the loadLibSVMFile routine.
> [1] http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/regression.html
--
This message was sent by Atlassian JIRA
(v6.2#6252)
