Re: Nested DataFrames

2015-06-25 Thread pawan kumar
May be you could try something like this using sparkSQL 1.4 and dataframes student.join(Grade, Grade(student_id) === student(student_id), left) .groupBy(id) .agg(sum(grade(Marks)), avg(grade(Marks))) You could refer to the following document :

Re: Code review - Spark SQL command-line client for Cassandra

2015-06-22 Thread pawan kumar
Hi, Zeppelin has a cassandra-spark-connector built into the build. I have not tried it yet may be you could let us know. https://github.com/apache/incubator-zeppelin/pull/79 To build a Zeppelin version with the *Datastax Spark/Cassandra connector

Re: Code review - Spark SQL command-line client for Cassandra

2015-06-22 Thread pawan kumar
-cassandra-connector_2.11%7C1.4.0-M1%7Cjar)? What exactly is it that does not currently exist for Spark 1.4? Thanks, Matthew *From:* pawan kumar [mailto:pkv...@gmail.com] *Sent:* 22 June 2015 17:19 *To:* Silvio Fiorito *Cc:* Mohammed Guller; Matthew Johnson; shahid ashraf; user

Re: Spark 1.4.0 build Error on Windows

2015-06-03 Thread pawan kumar
I got the same error message when using maven 3.3 . On Jun 3, 2015 8:58 AM, Ted Yu yuzhih...@gmail.com wrote: I used the same command on Linux but didn't reproduce the error. Can you include -X switch on your command line ? Also consider upgrading maven to 3.3.x Cheers On Wed, Jun 3,

RE: Tableau + Spark SQL Thrift Server + Cassandra

2015-04-03 Thread pawan kumar
* is pretty straight forward, especially if you are using DSE. Create a new DSN in Tableau using the ODBC driver that comes with DSE. Once you connect, Tableau allows to use C* keyspace as schema and column families as tables. Mohammed *From:* pawan kumar [mailto:pkv...@gmail.com] *Sent

Re: Tableau + Spark SQL Thrift Server + Cassandra

2015-04-03 Thread pawan kumar
Hi Todd, Thanks for the link. I would be interested in this solution. I am using DSE for cassandra. Would you provide me with info on connecting with DSE either through Tableau or zeppelin. The goal here is query cassandra through spark sql so that I could perform joins and groupby on my queries.

Re: Tableau + Spark SQL Thrift Server + Cassandra

2015-04-03 Thread pawan kumar
be available. Another possibility is to perhaps use the tuplejump cash project, https://github.com/tuplejump/cash. HTH. -Todd On Fri, Apr 3, 2015 at 11:11 AM, pawan kumar pkv...@gmail.com wrote: Thanks mohammed. Will give it a try today. We would also need the sparksSQL piece as we

Re: Tableau + Spark SQL Thrift Server + Cassandra

2015-04-03 Thread pawan kumar
, pawan kumar pkv...@gmail.com wrote: Hi Todd, Thanks for the help. So i was able to get the DSE working with tableau as per the link provided by Mohammed. Now i trying to figure out if i could write sparksql queries from tableau and get data from DSE. My end goal is to get a web based tool where i

Re: Creating a SparkR standalone job

2014-04-07 Thread pawan kumar
haven't tried it out though, but do let us know if you get it to work. Thanks Shivaram On Mon, Apr 7, 2014 at 2:21 PM, pawan kumar pkv...@gmail.com wrote: Hi, Is it possible to create a standalone job in scala using sparkR? If possible can you provide me with the information