Thanks guys ! On Wed, Oct 7, 2015 at 1:41 AM, Cody Koeninger <c...@koeninger.org> wrote:
> Sure no prob. > > On Tue, Oct 6, 2015 at 6:35 PM, Tathagata Das <t...@databricks.com> wrote: > >> Given the interest, I am also inclining towards making it a public >> developer API. Maybe even experimental. Cody, mind submitting a patch? >> >> >> On Tue, Oct 6, 2015 at 7:45 AM, Sean Owen <so...@cloudera.com> wrote: >> >>> For what it's worth, I also use this class in an app, but it happens >>> to be from Java code where it acts as if it's public. So no problem >>> for my use case, but I suppose, another small vote for the usefulness >>> of this class to the caller. I end up using getLatestLeaderOffsets to >>> figure out how to initialize initial offsets. >>> >>> On Tue, Oct 6, 2015 at 3:24 PM, Cody Koeninger <c...@koeninger.org> >>> wrote: >>> > I personally think KafkaCluster (or the equivalent) should be made >>> public. >>> > When I'm deploying spark I just sed out the private[spark] and rebuild. >>> > >>> > There's a general reluctance to make things public due to backwards >>> > compatibility, but if enough people ask for it... ? >>> > >>> > On Tue, Oct 6, 2015 at 6:51 AM, Jonathan Coveney <jcove...@gmail.com> >>> wrote: >>> >> >>> >> You can put a class in the org.apache.spark namespace to access >>> anything >>> >> that is private[spark]. You can then make enrichments there to access >>> >> whatever you need. Just beware upgrade pain :) >>> >> >>> >> >>> >> El martes, 6 de octubre de 2015, Erwan ALLAIN < >>> eallain.po...@gmail.com> >>> >> escribió: >>> >>> >>> >>> Hello, >>> >>> >>> >>> I'm currently testing spark streaming with kafka. >>> >>> I'm creating DirectStream with KafkaUtils and everything's fine. >>> However >>> >>> I would like to use the signature where I can specify my own message >>> handler >>> >>> (to play with partition and offset). In this case, I need to manage >>> >>> offset/partition by myself to fill fromOffsets argument. >>> >>> I have found a Jira on this usecase >>> >>> https://issues.apache.org/jira/browse/SPARK-6714 but it has been >>> closed >>> >>> telling that it's too specific. >>> >>> I'm aware that it can be done using kafka api (TopicMetaDataRequest >>> and >>> >>> OffsetRequest) but what I have to do is almost the same as the >>> KafkaCluster >>> >>> which is private. >>> >>> >>> >>> is it possible to : >>> >>> - add another signature in KafkaUtils ? >>> >>> - make KafkaCluster public ? >>> >>> >>> >>> or do you have any other srmart solution where I don't need to >>> copy/paste >>> >>> KafkaCluster ? >>> >>> >>> >>> Thanks. >>> >>> >>> >>> Regards, >>> >>> Erwan ALLAIN >>> > >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >> >
