[
https://issues.apache.org/jira/browse/SPARK-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970596#comment-14970596
]
Xusen Yin commented on SPARK-11234:
-----------------------------------
[~mengxr] I add the cooking classification code here:
https://gist.github.com/yinxusen/ad4372b8c0af5ae54a4a
Here are what I find:
1. Currently, multi-line per record JSON file is hard to handle, I have to load
the data with JsonInputFormat in the json-pxf-ext package.
2. String indexer is easy to use. But it is hard to do beyond existing
transformers. Like in the code, when I want to add all vectors that belong to
the same id together, I have to write an aggregate function.
3. ParamGridBuilder accepts discrete parameter candidates, but I need to add
some parameters with guess like Array(1.0, 0.1, 0.01). I don't know which
parameter is suitable and how to fill in the array will get a better result.
How about giving a range of real numbers so that the ParamGridBuilder can
generate candidates for me like [0.0001, 1]?
4. The evaluator forces me to select a metric method. But sometimes I want to
see all the evaluation results, say F1, precision-recall, AUC, etc.
5. ML transformers will get stuck when facing with Int type. It's strange that
we have to transform all Int values to double values before hand. I think a
wise auto casting is helpful.
> What's cooking classification
> -----------------------------
>
> Key: SPARK-11234
> URL: https://issues.apache.org/jira/browse/SPARK-11234
> Project: Spark
> Issue Type: Sub-task
> Components: ML
> Reporter: Xusen Yin
>
> I add the subtask to post the work on this dataset:
> https://www.kaggle.com/c/whats-cooking
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
