This is in active development, so there is not much that can be done from an end user perspective. In particular the only sink that is available in apache/master is a testing sink that just stores the data in memory. We are working on a parquet based file sink and will eventually support all the of Data Source API file formats (text, json, csv, orc, parquet).
On Tue, Mar 8, 2016 at 7:38 AM, Jacek Laskowski <ja...@japila.pl> wrote: > Hi Praveen, > > I don't really know. I think TD or Michael should know as they > personally involved in the task (as far as I could figure it out from > the JIRA and the changes). Ping people on the JIRA so they notice your > question(s). > > Pozdrawiam, > Jacek Laskowski > ---- > https://medium.com/@jaceklaskowski/ > Mastering Apache Spark http://bit.ly/mastering-apache-spark > Follow me at https://twitter.com/jaceklaskowski > > > On Tue, Mar 8, 2016 at 12:32 PM, Praveen Devarao <praveen...@in.ibm.com> > wrote: > > Thanks Jacek for the pointer. > > > > Any idea which package can be used in .format(). The test cases seem to > work > > out of the DefaultSource class defined within the > DataFrameReaderWriterSuite > > [org.apache.spark.sql.streaming.test.DefaultSource] > > > > Thanking You > > > --------------------------------------------------------------------------------- > > Praveen Devarao > > Spark Technology Centre > > IBM India Software Labs > > > --------------------------------------------------------------------------------- > > "Courage doesn't always roar. Sometimes courage is the quiet voice at the > > end of the day saying I will try again" > > > > > > > > From: Jacek Laskowski <ja...@japila.pl> > > To: Praveen Devarao/India/IBM@IBMIN > > Cc: user <user@spark.apache.org>, dev <d...@spark.apache.org> > > Date: 08/03/2016 04:17 pm > > Subject: Re: Spark structured streaming > > ________________________________ > > > > > > > > Hi Praveen, > > > > I've spent few hours on the changes related to streaming dataframes > > (included in the SPARK-8360) and concluded that it's currently only > > possible to read.stream(), but not write.stream() since there are no > > streaming Sinks yet. > > > > Pozdrawiam, > > Jacek Laskowski > > ---- > > https://medium.com/@jaceklaskowski/ > > Mastering Apache Spark http://bit.ly/mastering-apache-spark > > Follow me at https://twitter.com/jaceklaskowski > > > > > > On Tue, Mar 8, 2016 at 10:38 AM, Praveen Devarao <praveen...@in.ibm.com> > > wrote: > >> Hi, > >> > >> I would like to get my hands on the structured streaming feature > >> coming out in Spark 2.0. I have tried looking around for code samples to > >> get > >> started but am not able to find any. Only few things I could look into > is > >> the test cases that have been committed under the JIRA umbrella > >> https://issues.apache.org/jira/browse/SPARK-8360butthe test cases don't > >> lead to building a example code as they seem to be working out of > internal > >> classes. > >> > >> Could anyone point me to some resources or pointers in code > that I > >> can start with to understand structured streaming from a consumability > >> angle. > >> > >> Thanking You > >> > >> > --------------------------------------------------------------------------------- > >> Praveen Devarao > >> Spark Technology Centre > >> IBM India Software Labs > >> > >> > --------------------------------------------------------------------------------- > >> "Courage doesn't always roar. Sometimes courage is the quiet voice at > the > >> end of the day saying I will try again" > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > > For additional commands, e-mail: user-h...@spark.apache.org > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
