Thanks Jacek for the pointer. Any idea which package can be used in .format(). The test cases seem to work out of the DefaultSource class defined within the DataFrameReaderWriterSuite [ org.apache.spark.sql.streaming.test.DefaultSource]
Thanking You --------------------------------------------------------------------------------- Praveen Devarao Spark Technology Centre IBM India Software Labs --------------------------------------------------------------------------------- "Courage doesn't always roar. Sometimes courage is the quiet voice at the end of the day saying I will try again" From: Jacek Laskowski <ja...@japila.pl> To: Praveen Devarao/India/IBM@IBMIN Cc: user <user@spark.apache.org>, dev <d...@spark.apache.org> Date: 08/03/2016 04:17 pm Subject: Re: Spark structured streaming Hi Praveen, I've spent few hours on the changes related to streaming dataframes (included in the SPARK-8360) and concluded that it's currently only possible to read.stream(), but not write.stream() since there are no streaming Sinks yet. Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Tue, Mar 8, 2016 at 10:38 AM, Praveen Devarao <praveen...@in.ibm.com> wrote: > Hi, > > I would like to get my hands on the structured streaming feature > coming out in Spark 2.0. I have tried looking around for code samples to get > started but am not able to find any. Only few things I could look into is > the test cases that have been committed under the JIRA umbrella > https://issues.apache.org/jira/browse/SPARK-8360but the test cases don't > lead to building a example code as they seem to be working out of internal > classes. > > Could anyone point me to some resources or pointers in code that I > can start with to understand structured streaming from a consumability > angle. > > Thanking You > --------------------------------------------------------------------------------- > Praveen Devarao > Spark Technology Centre > IBM India Software Labs > --------------------------------------------------------------------------------- > "Courage doesn't always roar. Sometimes courage is the quiet voice at the > end of the day saying I will try again" --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
