Re: why generateJob is a private API?

2015-03-17 Thread madhu phatak 
Hi,
 Thank you for the response.

Regards,
Madhukara Phatak
http://datamantra.io/

On Tue, Mar 17, 2015 at 5:50 AM, Tathagata Das t...@databricks.com wrote:

 It was not really meant to be pubic and overridden. Because anything you
 want to do to generate jobs from RDDs can be done using DStream.foreachRDD

 On Sun, Mar 15, 2015 at 11:14 PM, madhu phatak phatak@gmail.com
 wrote:

 Hi,
  I am trying to create a simple subclass of DStream.  If I understand
 correctly, I should override *compute *lazy operations and *generateJob*
 for actions. But when I try to override, generateJob it gives error saying
 method is private to the streaming package. Is my approach is correct or am
 I  missing something?


 Regards,
 Madhukara Phatak
 http://datamantra.io/

Re: why generateJob is a private API?

2015-03-16 Thread Tathagata Das 
It was not really meant to be pubic and overridden. Because anything you
want to do to generate jobs from RDDs can be done using DStream.foreachRDD

On Sun, Mar 15, 2015 at 11:14 PM, madhu phatak phatak@gmail.com wrote:

 Hi,
  I am trying to create a simple subclass of DStream.  If I understand
 correctly, I should override *compute *lazy operations and *generateJob*
 for actions. But when I try to override, generateJob it gives error saying
 method is private to the streaming package. Is my approach is correct or am
 I  missing something?


 Regards,
 Madhukara Phatak
 http://datamantra.io/