You can use .distinct.count on your user RDD.
What are you trying to achieve with the time group by? — Sent from Mailbox On Tue, Jul 15, 2014 at 8:14 PM, buntu <buntu...@gmail.com> wrote: > Hi -- > New to Spark and trying to figure out how to do a generate unique counts per > page by date given this raw data: > timestamp,page,userId > 1405377264,google,user1 > 1405378589,google,user2 > 1405380012,yahoo,user1 > .. > I can do a groupBy a field and get the count: > val lines=sc.textFile("data.csv") > val csv=lines.map(_.split(",")) > // group by page > csv.groupBy(_(1)).count > But not able to see how to do count distinct on userId and also apply > another groupBy on timestamp field. Please let me know how to handle such > cases. > Thanks! > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Count-distinct-with-groupBy-usage-tp9781.html > Sent from the Apache Spark User List mailing list archive at Nabble.com.