Re: why generateJob is a private API?

2015-03-17 Thread madhu phatak
Hi, Thank you for the response. Regards, Madhukara Phatak http://datamantra.io/ On Tue, Mar 17, 2015 at 5:50 AM, Tathagata Das t...@databricks.com wrote: It was not really meant to be pubic and overridden. Because anything you want to do to generate jobs from RDDs can be done using

Re: why generateJob is a private API?

2015-03-16 Thread Tathagata Das
It was not really meant to be pubic and overridden. Because anything you want to do to generate jobs from RDDs can be done using DStream.foreachRDD On Sun, Mar 15, 2015 at 11:14 PM, madhu phatak phatak@gmail.com wrote: Hi, I am trying to create a simple subclass of DStream. If I

why generateJob is a private API?

2015-03-16 Thread madhu phatak
Hi, I am trying to create a simple subclass of DStream. If I understand correctly, I should override *compute *lazy operations and *generateJob* for actions. But when I try to override, generateJob it gives error saying method is private to the streaming package. Is my approach is correct or am