Only interested in certain partitions

[Jens Rantil]

Hi,

Background: I am using Kafka 0.9 using the Java client. I have a consumer
with a fixed set of keys that it is interested in from a given topic.
Assuming I have many partitions, I could manually assign my consumer to
only listen to the relevant partitions, given my keys.

Question: Given a key and a topic, when using the new Java consumer, how
can I figure out which partition the key will be written to? If not
possible, I will file a JIRA.

Thanks,
Jens

-- 
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook  Linkedin

 Twitter 

Re: Only interested in certain partitions

[Ewen Cheslack-Postava]

One option is to instantiate and invoke the DefaultPartitioner yourself (or
whatever partitioner you've specified for partitioner.class). However, that
will require passing in a Cluster object, which you'll need to construct
yourself. This is just used to get the number of partitions for the topic
so it shouldn't be a problem to fill it in with basic, dummy data as long
as you have the right number of partitions.

-Ewen

On Wed, Jan 27, 2016 at 9:05 AM, Jens Rantil  wrote:

> Hi,
>
> Background: I am using Kafka 0.9 using the Java client. I have a consumer
> with a fixed set of keys that it is interested in from a given topic.
> Assuming I have many partitions, I could manually assign my consumer to
> only listen to the relevant partitions, given my keys.
>
> Question: Given a key and a topic, when using the new Java consumer, how
> can I figure out which partition the key will be written to? If not
> possible, I will file a JIRA.
>
> Thanks,
> Jens
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: jens.ran...@tink.se
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
> Facebook  Linkedin
> <
> http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary
> >
>  Twitter 
>



-- 
Thanks,
Ewen