Re: Spark 2 or Spark 1.6.x?

Hi, You need to describe more. For example, in Spark 2.0.2, you can't use stable versions of Apache Kafka. In general, I would say start with 2.0.2- On Mon, Dec 12, 2016 at 7:34 AM, Lohith Samaga M wrote: > Hi, > > I am new to Spark. I would like to

Re: I coded an example to use Twitter stream as a data source for Spark

org/docs/latest/streaming-custom-receivers.html> and > generate the data from there itself instead of spark connecting to a socket > server which will be fed by another twitter client? > > Thanks > Best Regards > > On Sat, Dec 19, 2015 at 5:47 PM, Amir Rahnama <amirrahn.

I coded an example to use Twitter stream as a data source for Spark

Hi guys, Thought someone would need this: https://github.com/ambodi/realtime-spark-twitter-stream-mining you can use this approach to feed twitter stream to your spark job. So far, PySpark does not have a twitter dstream source. -- Thanks and Regards, Amir Hossein Rahnama *Tel: +46 (0)

storing result of aggregation of spark streaming

Hi, I am gonna store the results of my stream job into a db, which one of databases has the native support (if any)? -- Thanks and Regards, Amir Hossein Rahnama *Tel: +46 (0) 761 681 102* Website: www.ambodi.com Twitter: @_ambodi

Re: How can you sort wordcounts by counts in stateful_network_wordcount.py example

PM, Young, Matthew T <matthew.t.yo...@intel.com > wrote: > You can use foreachRDD > <http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations-on-dstreams> > to get access to the batch API in streaming jobs. > > > > > > *From:* Ami

Re: How can you sort wordcounts by counts in stateful_network_wordcount.py example

ly swap key and value of the RDD and then sort? > > On Thu, Nov 12, 2015 at 8:53 AM, Amir Rahnama <amirrahn...@gmail.com> > wrote: > >> Hey, >> >> Anybody knows how can one sort the result in the stateful example? >> >> Py

How can you sort wordcounts by counts in stateful_network_wordcount.py example

Hey, Anybody knows how can one sort the result in the stateful example? Python would be prefered. https://github.com/apache/spark/blob/859dff56eb0f8c63c86e7e900a12340c199e6247/examples/src/main/python/streaming/stateful_network_wordcount.py -- Thanks and Regards, Amir Hossein Rahnama *Tel: