Re: [EXTERNAL] - Re: Spark ML / ALS question

2020-12-03 Thread Steve Pruitt
Thanks, I confused myself. I was looking at org.apache.spark.ml.recommendation.ALS Javadoc. Not sure why it shows up. I didn't notice the Developer API tag, so "fit" it is! -S From: Sean Owen Sent: Wednesday, December 2, 2020 3:51 PM To: Steve

Spark ML / ALS question

2020-12-02 Thread Steve Pruitt
I am having a little difficulty finding information on the ALS train(…) method in spark.ml. Its unclear when to use it. In the java doc, the parameters are undocumented. What is difference between train(..) and fit(..). When would do you use one or the other? -S

RE: [EXTERNAL] - Re: Problem with the ML ALS algorithm

2019-06-26 Thread Steve Pruitt
has not rated and the result would be the same essentially. On Wed, Jun 26, 2019 at 3:03 PM Steve Pruitt mailto:bpru...@opentext.com>> wrote: Number of users is 1055 Number of items is 4 Ratings values are either 120, 20, 0 From: Nick Pentreath mailto:nick.pentre...@gmail.com>> Sent: W

RE: [EXTERNAL] - Re: Problem with the ML ALS algorithm

2019-06-26 Thread Steve Pruitt
is not positive definite. Try increasing regParam (try 0.1, 1.0 for example). What does the data look like? e.g. number of users, number of items, number of ratings, etc? On Wed, Jun 26, 2019 at 12:06 AM Steve Pruitt mailto:bpru...@opentext.com>> wrote: I get an inexplicable exception when

Problem with the ML ALS algorithm

2019-06-25 Thread Steve Pruitt
I get an inexplicable exception when trying to build an ALSModel with the implicit set to true. I can’t find any help online. Thanks in advance. My code is: ALS als = new ALS() .setMaxIter(5) .setRegParam(0.01) .setUserCol("customer")

[Spark ML] [Pyspark] [Scenario Beginner] [Level Beginner]

2019-04-02 Thread Steve Pruitt
Weight values above? Thanks. -S From: Steve Pruitt Sent: Monday, April 01, 2019 12:39 PM To: user Subject: [EXTERNAL] - [Spark ML] [Pyspark] [Scenario Beginner] [Level Beginner] After following a tutorial on Recommender systems using Pyspark / Spark ML. I decided to jump in with my own

[Spark ML] [Pyspark] [Scenario Beginner] [Level Beginner]

2019-04-01 Thread Steve Pruitt
After following a tutorial on Recommender systems using Pyspark / Spark ML. I decided to jump in with my own dataset. I am specifically trying to predict video suggestions based on an implicit feature for the time a video was watched. I wrote a generator to produce my dataset. I have a

RE: [EXTERNAL] - Re: testing frameworks

2018-05-22 Thread Steve Pruitt
[mailto:hol...@pigscanfly.ca] Sent: Monday, May 21, 2018 11:32 AM To: Steve Pruitt <bpru...@opentext.com> Cc: user@spark.apache.org Subject: [EXTERNAL] - Re: testing frameworks So I’m biased as the author of spark-testing-base but I think it’s pretty ok. Are you looking for unit or integ

testing frameworks

2018-05-21 Thread Steve Pruitt
Hi, Can anyone recommend testing frameworks suitable for Spark jobs. Something that can be integrated into a CI tool would be great. Thanks.