Did you plan to modify dstream interface in order to work with dataframe ? It would be nice handle dstreams without generics
Paolo Inviata dal mio Windows Phone ________________________________ Da: Tathagata Das<mailto:[email protected]> Inviato: 10/09/2015 07:42 A: N B<mailto:[email protected]> Cc: user<mailto:[email protected]> Oggetto: Re: Tungsten and Spark Streaming Rewriting is necessary. You will have to convert RDD/DStream operations to DataFrame operations. So get the RDDs in DStream, using transform/foreachRDD, convert to DataFrames and then do DataFrame operations. On Wed, Sep 9, 2015 at 9:23 PM, N B <[email protected]<mailto:[email protected]>> wrote: Hello, How can we start taking advantage of the performance gains made under Project Tungsten in Spark 1.5 for a Spark Streaming program? >From what I understand, this is available by default for Dataframes. But for a >program written using Spark Streaming, would we see any potential gains "out >of the box" in 1.5 or will we have to rewrite some portions of the application >code to realize that benefit? Any insight/documentation links etc in this regard will be appreciated. Thanks Nikunj
