Re: Q regarding thrift server to expose RDD via SQL

2015-02-17 Thread Marco
In the meantime I've played with Spark 'standalone' cluster. There it works. Is my assumption correct that the Thriftserver only works with the Spark standalone cluster ? Or would Mesos be an alternative? BR Marco 2015-02-16 16:36 GMT+01:00 Marco : > Hi, > > I've played with the feature to expo

Re: Q regarding thrift server to expose RDD via SQL

2015-02-17 Thread Nick Dimiduk
By "ThriftServer," you mean HiveServer2? Or are you talking about HBase's thrift gateway? Both should work, though neither are required. As my previous mail, you should have better scalable performance by using HBase's InputFormats instead of these intermediary services. I have no experience runni

Re: Q regarding thrift server to expose RDD via SQL

2015-02-16 Thread Nick Dimiduk
Using TableInputFormat directly will have better scalability than HS2. Better still, use TableSnapshotInputFormat to work from a snapshot (since RDDs are immutable anyway). -n On Monday, February 16, 2015, Marco wrote: > Hi, > > I've played with the feature to expose RDD via Thrift to enable JD

Q regarding thrift server to expose RDD via SQL

2015-02-16 Thread Marco
Hi, I've played with the feature to expose RDD via Thrift to enable JDBC access. (Spark 1.2) val eventsView = sqlContext.createSchemaRDD(eventSchemaRdd) eventsView.registerTempTable("Events") HiveThriftServer2.startWithContext(sqlContext) This works all fine. Now, my understanding is yo