Hi All,
This might be too late about partitioning strategy and use cases to cover.
I have had experience with both producer and consumer side. I have
different use case on this partition selection strategy.
Problem:
We have heterogeneous environment of producers (by that I mean we have
Hi Bhavesh,
I realized I didn’t send any vote closed message on this thread.
Although the changes you are proposing related to partitioning but the change
proposed in this KIP is minor one and will benefit the users . I appreciate if
you can open up another thread for the discussion
Gianmarco,
I’ll send the patch soon.
Thanks,
Harsha
On May 18, 2015 at 1:34:50 AM, Gianmarco De Francisci Morales (g...@apache.org)
wrote:
Hi,
If everything is in order, can we proceed to implement it?
Cheers,
--
Gianmarco
On 13 May 2015 at 03:06, Jiangjie Qin
Hi,
If everything is in order, can we proceed to implement it?
Cheers,
--
Gianmarco
On 13 May 2015 at 03:06, Jiangjie Qin j...@linkedin.com.invalid wrote:
Hi Harsha,
If you open this link
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposa
ls
All the KIPs are
Hey Harsha,
It looks you created the KIP page at wrong place. . . Can you move the
page to a child page of
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposa
ls
Thanks.
Jiangjie (Becket) Qin
On 5/6/15, 6:12 PM, Harsha ka...@harsha.io wrote:
Thanks for the review Joel.
Hi Harsha,
If you open this link
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposa
ls
All the KIPs are the child page of this page which you can see from the
left bar. Only KIP-22 is missing. It looks you created it as a child page
of
Hey Harsha,
The link in your last email gives page not found. . .
Jiangjie (Becket) Qin
On 5/12/15, 3:12 PM, Sriharsha Chintalapani ka...@harsha.io wrote:
Hi Jiangjie,
Its under
https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22++Expose+a+Partit
Hi Jiangjie,
Its under
https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22++Expose+a+Partitioner+interface+in+the+new+producer
I checked other KIPS they are under /KAFKA as well.
Thanks,
Harsha
On May 12, 2015 at 2:12:30 PM, Jiangjie Qin (j...@linkedin.com.invalid) wrote:
Hey
Thanks Jay. I removed partitioner.metadata from KIP. I’ll send an updated patch.
--
Harsha
Sent with Airmail
On May 5, 2015 at 6:31:47 AM, Sriharsha Chintalapani (harsh...@fastmail.fm)
wrote:
Thanks for the comments everyone.
Hi Jay,
I do have a question regarding configurable interface
+1 with a minor comment: do we need an init method given it extends
Configurable?
Also, can you move this wiki out of drafts and add it to the table in
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals?
Thanks,
Joel
On Wed, May 06, 2015 at 07:46:46AM -0700,
Thanks for the review Joel. I agree don't need a init method we can use
configure. I'll update the KIP.
-Harsha
On Wed, May 6, 2015, at 04:45 PM, Joel Koshy wrote:
+1 with a minor comment: do we need an init method given it extends
Configurable?
Also, can you move this wiki out of drafts and
Thanks for the comments everyone.
Hi Jay,
I do have a question regarding configurable interface on how to pass a
MapString, ? properties. I couldn’t find any other classes using it. JMX
reporter overrides it but doesn’t implement it. So with configurable
partitioner how can a user pass in
Thanks Harsha,
I think you addressed all the points raised so far.
LGTM +1.
Cheers,
--
Gianmarco
On 4 May 2015 at 03:57, Harsha ka...@harsha.io wrote:
Thanks Jay Gianmarco for the comments. I picked the option A, if user
sends a partition id than it will applied and partitioner.class
I'd pick option A too. Rest LGTM. +1
On Mon, May 4, 2015 at 10:03 AM, Gianmarco De Francisci Morales
g...@apache.org wrote:
Thanks Harsha,
I think you addressed all the points raised so far.
LGTM +1.
Cheers,
--
Gianmarco
On 4 May 2015 at 03:57, Harsha ka...@harsha.io wrote:
Thanks
Hey Harsha,
That proposal sounds good. One minor thing--I don't think we need to have
the partitioner.metadata property. Our reason for using string properties
is exactly to make config extensible at runtime. So a given partitioner can
add whatever properties make sense using the configure() api
Hi,
Here are the questions I think we should consider:
1. Do we need this at all given that we have the partition argument in
ProducerRecord which gives full control? I think we do need it because this
is a way to plug in a different partitioning strategy at run time and do it
in a fairly
Hey Harsha,
A few comments:
Can you finish up the KIP there are some unfinished sentences and odd
whitespace things going on.
Here are the questions I think we should consider:
1. Do we need this at all given that we have the partition argument in
ProducerRecord which gives full control? I
Hi Jay,
Sorry about the KIP formatting . I fixed those in the KIP.
2. We certainly need to add both the serialized and unserialized form for
the key as both are useful.
I added those to the interface.
3. Do we need to add the value? I suspect people will have uses for
computing
Hey Sriharsha,
Great, thanks!
For 4:
Yeah the use case for init and close is making use of any kind of metadata.
An example of this would be if you are trying to do range partitioning you
need to map lexicographic ranges to numeric partitions. You might do this
by adding a new property to the
Hi,
Here is the KIP for adding a partitioner interface for producer.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22+-+Expose+a+Partitioner+interface+in+the+new+producer
There is one open question about how interface should look like. Please take a
look and let me know if you
20 matches
Mail list logo
