Re: word count (group by users) in spark

2015-09-21 Thread Aniket Bhatnagar
ry", >>> "1g") >>> val sc = new SparkContext(conf) >>> val sqlContext= new SQLContext(sc) >>> >>> val hc=new HiveContext(sc) >>> >>> hc.sql("CREATE EXTERNAL TABLE IF NOT EXISTS default.TEST (user_name &

Re: word count (group by users) in spark

2015-09-21 Thread Zhang, Jingyu
gt;>>> object HiveWordCount { >>>> >>>> def main(args: Array[String]): Unit = >>>> { >>>> Logger.getLogger("org").setLevel(Level.WARN) >>>> Logger.getLogger("akka").setLevel(Level.WARN) >>>>

Re: word count (group by users) in spark

2015-09-20 Thread Huy Banh
TS STRING )ROW FORMAT DELIMITED FIELDS TERMINATED BY '001' >> STORED AS TEXTFILE LOCATION '/data/kali/test' ") >> >> val op=hc.sql("select user_name,COLLECT_SET(text) from (select >> user_name,concat(sub,' ',count(comments)) as text from default.test >>

word count (group by users) in spark

2015-09-19 Thread kali.tumm...@gmail.com
from default.test LATERAL VIEW explode(split(comments,',')) subView AS sub group by user_name,sub)w group by user_name") op.collect.foreach(println) } Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/word-count-group-by-users

Re: word count (group by users) in spark

2015-09-19 Thread Aniket Bhatnagar
ION '/data/kali/test' ") > > val op=hc.sql("select user_name,COLLECT_SET(text) from (select > user_name,concat(sub,' ',count(comments)) as text from default.test > LATERAL > VIEW explode(split(comments,',')) subView AS sub group by user_name,sub)w > group by user_na