MovieLensALS - Scala Pattern Magic

2014-08-04 Thread Steve Nunez
Can one of the Scala experts please explain this bit of pattern magic from the Spark ML tutorial: _._2.user ? As near as I can tell, this is applying the _2 function to the wildcard, and then applying the Œuser¹ function to that. In a similar way the Œproduct¹ function is applied in the next

Re: MovieLensALS - Scala Pattern Magic

2014-08-04 Thread Sean Owen
ratings is an RDD of Rating objects. You can see them created as the second element of the tuple. It's a simple case class: https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala#L66 This is just accessing the user and product field of

Re: MovieLensALS - Scala Pattern Magic

2014-08-04 Thread Holden Karau
Hi Steve, The _ notation can be a bit confusing when starting with Scala, we can rewrite it to avoid using it here. So instead of val numUsers = ratings.map(_._2.user) we can write val numUsers = ratings.map(x = x._2.user) ratings is an Key-Value RDD (which is an RDD comprised of tuples) and so