Re: Spark ML - LogisticRegression interpreting prediction

2017-10-23 Thread pun
Thanks a lot! You are right! -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Spark ML - LogisticRegression interpreting prediction

2017-10-22 Thread pun
Hello, I have a LogisticRegression model for predicting a binary label. Once I train the model, I run it to get some predictions. I get the following values for RawPrediction. How should I interpret these? Whdo they mean? ++|rawPrediction

Spark ML - LogisticRegression extract words with highest weights

2017-10-05 Thread pun
I am using Spark ML's pipeline to classify text documents with the following steps: Tokenizer -> CountVectorizer -> LogisticRegression I want to be able to print the words with the highest weights. Can this be done? So far I have been able to extract the LR coefficients, but can those be tied up t

Re: Spark ML - CountVectorizer + LogisticRegression unexpected behavior

2017-10-04 Thread pun
Nm. Rookie error. I wasn't caching the DF. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

mllib - CountVectorizer + LogisticRegression unexpected behavior

2017-10-04 Thread pun
Hello,I have a model, which uses CountVectorizer and LogisticRegression. *Everything seems to work fine, except that when I am running the last step to get results and predictions, the document ids (doc_id) are being changed completely. Do you know why that is? Am I doing something wrong?* import

Re: How to run MLlib's word2vec in CBOW mode?

2017-09-28 Thread pun
Thank you so much!Any sense to how long this may take to get released? TIA -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

How to run MLlib's word2vec in CBOW mode?

2017-09-28 Thread pun
Hello, My understanding is that word2vec can be ran in two modes: continuous bag-of-words (CBOW) (order of words does not matter) continuous skip-gram (order of words matters) I would like to run the *CBOW* implementation from Spark's MLlib, but it is not clear to me from the documentation and th