Hi Joseph,
I think about converting IDS but there will be birthday problem. The
probability of a Hash Collision
http://preshing.com/20110504/hash-collision-probabilities/ is important
for me because of the user number. I don't know how can I modify ALS to use
Integer.
yasemin
2015-06-04 2:28 GMT+03:00 Joseph Bradley jos...@databricks.com:
Hi Yasemin,
If you can convert your user IDs to Integers in pre-processing (if you
have a couple billion users), that would work. Otherwise...
In Spark 1.3: You may need to modify ALS to use Long instead of Int.
In Spark 1.4: spark.ml.recommendation.ALS (in the Pipeline API) exposes
ALS.train as a DeveloperApi to allow users to use Long instead of Int.
We're also thinking about better ways to permit Long IDs.
Joseph
On Wed, Jun 3, 2015 at 5:04 AM, Yasemin Kaya godo...@gmail.com wrote:
Hi,
I want to use Spark's ALS in my project. I have the userid
like 30011397223227125563254 and Rating Object which is the Object of ALS
wants Integer as a userid so the id field does not fit into a 32 bit
Integer. How can I solve that ? Thanks.
Best,
yasemin
--
hiç ender hiç
--
hiç ender hiç
