I might be wrong here but I don't believe it's discouraged. Maybe part of the reason there's not a lot of examples is that sql2rdd returns an RDD (TableRDD that is https://github.com/amplab/shark/blob/master/src/main/scala/shark/SharkContext.scala). I haven't done anything too complicated yet but my impression is that almost any Spark example of manipulating RDDs should applying from that line onwards.
Are you asking for samples what to do with the RDD once you get it or how to get a SharkContext from a standalone program? Also, my reading of a recent email on this list is that SharkAPI will be largely superceded by a more general SparkSQL API in 1.0 (http://people.apache.org/~pwendell/catalyst-docs/sql-programming-guide.html). So if you're just starting out and you don't have short term needs that might be a better place to start... On Mon, Apr 7, 2014 at 9:14 AM, Jerry Lam <chiling...@gmail.com> wrote: > Hi Shark, > > Should I assume that Shark users should not use the shark APIs since there > are no documentations for it? If there are documentations, can you point it > out? > > Best Regards, > > Jerry > > > On Thu, Apr 3, 2014 at 9:24 PM, Jerry Lam <chiling...@gmail.com> wrote: >> >> Hello everyone, >> >> I have successfully installed Shark 0.9 and Spark 0.9 in standalone mode >> in a cluster of 6 nodes for testing purposes. >> >> I would like to use Shark API in Spark programs. So far I could only find >> the following: >> >> $./bin/shark-shell >> scala> val youngUsers = sc.sql2rdd("SELECT * FROM users WHERE age < 20") >> scala> println(youngUsers.count) >> ... >> scala> val featureMatrix = youngUsers.map(extractFeatures(_)) >> scala> kmeans(featureMatrix) >> >> Is there a more complete sample code to start a program using Shark API in >> Spark? >> >> Thanks! >> >> Jerry > >
