Re: does KafkaCluster can be public ?

If anyone is interested in keeping tabs on it, the jira for this is https://issues.apache.org/jira/browse/SPARK-10963 On Wed, Oct 7, 2015 at 3:16 AM, Erwan ALLAIN wrote: > Thanks guys ! > > On Wed, Oct 7, 2015 at 1:41 AM, Cody Koeninger wrote: > >>

Re: does KafkaCluster can be public ?

Thanks guys ! On Wed, Oct 7, 2015 at 1:41 AM, Cody Koeninger wrote: > Sure no prob. > > On Tue, Oct 6, 2015 at 6:35 PM, Tathagata Das wrote: > >> Given the interest, I am also inclining towards making it a public >> developer API. Maybe even

Re: does KafkaCluster can be public ?

Or maybe annotate with @DeveloperApi Cheers On Tue, Oct 6, 2015 at 7:24 AM, Cody Koeninger wrote: > I personally think KafkaCluster (or the equivalent) should be made > public. When I'm deploying spark I just sed out the private[spark] and > rebuild. > > There's a general

Re: does KafkaCluster can be public ?

For what it's worth, I also use this class in an app, but it happens to be from Java code where it acts as if it's public. So no problem for my use case, but I suppose, another small vote for the usefulness of this class to the caller. I end up using getLatestLeaderOffsets to figure out how to

Re: does KafkaCluster can be public ?

I personally think KafkaCluster (or the equivalent) should be made public. When I'm deploying spark I just sed out the private[spark] and rebuild. There's a general reluctance to make things public due to backwards compatibility, but if enough people ask for it... ? On Tue, Oct 6, 2015 at 6:51

does KafkaCluster can be public ?

Hello, I'm currently testing spark streaming with kafka. I'm creating DirectStream with KafkaUtils and everything's fine. However I would like to use the signature where I can specify my own message handler (to play with partition and offset). In this case, I need to manage offset/partition by

Re: does KafkaCluster can be public ?

Given the interest, I am also inclining towards making it a public developer API. Maybe even experimental. Cody, mind submitting a patch? On Tue, Oct 6, 2015 at 7:45 AM, Sean Owen wrote: > For what it's worth, I also use this class in an app, but it happens > to be from

Re: does KafkaCluster can be public ?

Sure no prob. On Tue, Oct 6, 2015 at 6:35 PM, Tathagata Das wrote: > Given the interest, I am also inclining towards making it a public > developer API. Maybe even experimental. Cody, mind submitting a patch? > > > On Tue, Oct 6, 2015 at 7:45 AM, Sean Owen

Re: does KafkaCluster can be public ?

You can put a class in the org.apache.spark namespace to access anything that is private[spark]. You can then make enrichments there to access whatever you need. Just beware upgrade pain :) El martes, 6 de octubre de 2015, Erwan ALLAIN escribió: > Hello, > > I'm