[ https://issues.apache.org/jira/browse/SPARK-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364246#comment-14364246 ]
Kian Ho commented on SPARK-6340: -------------------------------- Hi Joseph, I initially considered that as a solution, however it was my understanding that you couldn't guarantee the same ordering between the instances pre- and post- transformations (since the transformations will be distributed across worker nodes). Is this correct? This question was also mentioned by a couple of users in that thread. Thanks > mllib.IDF for LabelPoints > ------------------------- > > Key: SPARK-6340 > URL: https://issues.apache.org/jira/browse/SPARK-6340 > Project: Spark > Issue Type: Improvement > Components: MLlib > Affects Versions: 1.3.0 > Environment: python 2.7.8 > pyspark > OS: Linux Mint 17 Qiana (Cinnamon 64-bit) > Reporter: Kian Ho > Priority: Minor > Labels: feature > > as per: > http://apache-spark-user-list.1001560.n3.nabble.com/Using-TF-IDF-from-MLlib-td19429.html#a19528 > Having the IDF.fit accept LabelPoints would be useful since, correct me if > i'm wrong, there currently isn't a way of keeping track of which labels > belong to which documents if one needs to apply a conventional tf-idf > transformation on labelled text data. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org