RE: Spark SQL DataFrame to Kafka Topic

2017-05-16 Thread Revin Chalil
Thanks Michael, that worked, appreciate your help. From: Michael Armbrust [mailto:mich...@databricks.com] Sent: Monday, May 15, 2017 11:45 AM To: Revin Chalil <rcha...@expedia.com> Cc: User <user@spark.apache.org> Subject: Re: Spark SQL DataFrame to Kafka Topic The foreach sink fr

Re: Spark SQL DataFrame to Kafka Topic

2017-05-15 Thread Michael Armbrust
elp is much appreciated. Thank you. > > > > > > *From:* Tathagata Das [mailto:tathagata.das1...@gmail.com] > *Sent:* Friday, January 13, 2017 3:31 PM > *To:* Koert Kuipers <ko...@tresata.com> > *Cc:* Peyman Mohajerian <mohaj...@gmail.com>; Senthil Kumar < >

RE: Spark SQL DataFrame to Kafka Topic

2017-05-15 Thread Revin Chalil
rs <ko...@tresata.com>; silvio.fior...@granturing.com Subject: RE: Spark SQL DataFrame to Kafka Topic Hi TD / Michael, I am trying to use the foreach sink to write to Kafka and followed this<https://databricks.com/blog/2017/04/04/real-time-end-to-end-integration-with-apache-kafka-in-apache-sparks-

RE: Spark SQL DataFrame to Kafka Topic

2017-05-14 Thread Revin Chalil
athagata.das1...@gmail.com] Sent: Friday, January 13, 2017 3:31 PM To: Koert Kuipers <ko...@tresata.com> Cc: Peyman Mohajerian <mohaj...@gmail.com>; Senthil Kumar <senthilec...@gmail.com>; User <user@spark.apache.org>; senthilec...@apache.org Subject: Re: Spark SQL DataFrame to Kafka

Re: Spark SQL DataFrame to Kafka Topic

2017-01-24 Thread ayan guha
Kumar <senthilec...@gmail.com> > wrote: > > Hi Team , > > Sorry if this question already asked in this forum.. > > Can we ingest data to Apache Kafka Topic from Spark SQL DataFrame ?? > > Here is my Code which Reads Parquet File : > > *val sqlContext = new org.apache.s

Re: Spark SQL DataFrame to Kafka Topic

2017-01-24 Thread Koert Kuipers
reaming-kafka.html >>> http://spark.apache.org/docs/latest/structured-streaming-pro >>> gramming-guide.html >>> >>> On Fri, Jan 13, 2017 at 3:32 AM, Senthil Kumar <senthilec...@gmail.com> >>> wrote: >>> >>>> Hi Team , >>&g

Re: Spark SQL DataFrame to Kafka Topic

2017-01-13 Thread Tathagata Das
t;> >>> Hi Team , >>> >>> Sorry if this question already asked in this forum.. >>> >>> Can we ingest data to Apache Kafka Topic from Spark SQL DataFrame ?? >>> >>> Here is my Code which Reads Parquet File : >>> >>>

Re: Spark SQL DataFrame to Kafka Topic

2017-01-13 Thread Koert Kuipers
.html > http://spark.apache.org/docs/latest/structured-streaming- > programming-guide.html > > On Fri, Jan 13, 2017 at 3:32 AM, Senthil Kumar <senthilec...@gmail.com> > wrote: > >> Hi Team , >> >> Sorry if this question already asked in this forum.. >>

Re: Spark SQL DataFrame to Kafka Topic

2017-01-13 Thread Peyman Mohajerian
; Hi Team , > > Sorry if this question already asked in this forum.. > > Can we ingest data to Apache Kafka Topic from Spark SQL DataFrame ?? > > Here is my Code which Reads Parquet File : > > *val sqlContext = new org.apache.spark.sql.SQLContext(sc);* > > *val df = sqlCo

Spark SQL DataFrame to Kafka Topic

2017-01-13 Thread Senthil Kumar
Hi Team , Sorry if this question already asked in this forum.. Can we ingest data to Apache Kafka Topic from Spark SQL DataFrame ?? Here is my Code which Reads Parquet File : *val sqlContext = new org.apache.spark.sql.SQLContext(sc);* *val df = sqlContext.read.parquet("