Hey Becket,
What are the next steps on this KIP. As per your comment earlier on the
thread -
I do agree it makes more sense
to avoid duplicate effort and plan based on new consumer. I’ll modify the
KIP.
Did you get a chance to think about the simplified design that we proposed
earlier? Do
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30482/#review71562
---
Ship it!
Ship It!
- Jay Kreps
On Feb. 6, 2015, 11:02 p.m.,
I don't think we need a KIP/vote here, this is just an internal
refactoring. We had said previously and noted in the document that the KIPs
were just for big new features or public api changes.
I am a big +1 on the idea. We'll have to be careful in the code review
since it would really easy to
I closed about 350 redundant or obsolete issues. If I closed an issue you
think is not obsolete, my apologies, just reopen.
-Jay
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30403/#review71557
---
config/server.properties
Ahmet AKYOL created KAFKA-1932:
--
Summary: kafka topic (creation) templates
Key: KAFKA-1932
URL: https://issues.apache.org/jira/browse/KAFKA-1932
Project: Kafka
Issue Type: Wish
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30763/
---
Review request for kafka.
Bugs: KAFKA-1865
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29468/#review71572
---
[
https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311252#comment-14311252
]
Manikumar Reddy commented on KAFKA-1884:
[~guozhang] [~jkreps]
KAFKA-1919 solves
Apparently I joined this list at the right time :P
On Sat, Feb 7, 2015 at 4:40 PM, Jay Kreps jay.kr...@gmail.com wrote:
I closed about 350 redundant or obsolete issues. If I closed an issue you
think is not obsolete, my apologies, just reopen.
-Jay
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29831/#review71555
---
Ship it!
Thanks for attaching the latest run of the tool. I
On Feb. 1, 2015, 8:46 p.m., Guozhang Wang wrote:
Not sure this stuff is actually here for review...may still be a work in
progress. Overall this structure of code makes a ton of sense to me. Left
some minor comments.
Yes this is more of a WIP patch, but the scope of this JIRA does not
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30196/#review71556
---
core/src/test/scala/unit/kafka/integration/PrimitiveApiTest.scala
I¹m a little bit concerned about the request routers among brokers.
Typically we have a dominant percentage of produce and fetch
request/response. Routing them from one broker to another seems not wanted.
Also I think we generally have two types of requests/responses: data
related and admin
[
https://issues.apache.org/jira/browse/KAFKA-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311314#comment-14311314
]
Mark Payne commented on KAFKA-1831:
---
[~jkreps]: that sounds perfect! I appreciate you
[
https://issues.apache.org/jira/browse/KAFKA-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmet AKYOL updated KAFKA-1932:
---
Description:
AFAIK, the only way to create a Kafka topic (without using the default
settings) is
A problem I am having is actually understanding which KIPs are intended to
be complete proposals and which are works in progress. Joe you seem to have
a bunch of these. Can you move them elsewhere until they are really fully
done and ready for review and discussion?
-Jay
On Fri, Feb 6, 2015 at
[
https://issues.apache.org/jira/browse/KAFKA-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311274#comment-14311274
]
Manikumar Reddy commented on KAFKA-1877:
[~jkreps] [~junrao] Currently Kafka
Hey Joe,
I think this is proposing several things:
1. A new command line utility. This isn't really fully specified here.
There is sample usage but I actually don't really understand what all the
commands will be. Also, presumably this will replace the existing shell
scripts, right? We obviously
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30482/
---
(Updated Feb. 6, 2015, 11:02 p.m.)
Review request for kafka.
Bugs:
[
https://issues.apache.org/jira/browse/KAFKA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-1758:
-
Component/s: log
corrupt recovery file prevents startup
--
Argh, I just realized that the producer and consumer have already almost
removed that so it wouldn't be in common but just something for the
broker. Maybe later this year 0.9/1.0 item to crack into.
On Sun, Feb 8, 2015 at 11:34 AM, Joe Stein joe.st...@stealth.ly wrote:
Jay,
Can we add
[
https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Stein updated KAFKA-1856:
-
Attachment: KAFKA-1845.result.txt
really cool, just tried this out
{code}
python
Following up on our previous thread on making batch send a little easier,
here is a concrete proposal to add a flush() method to the producer:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-8+-+Add+a+flush+method+to+the+producer+API
A proposed implementation is here:
[
https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira reassigned KAFKA-1928:
---
Assignee: Gwen Shapira
Move kafka.network over to using the network classes in
Hey Jiangjie,
Re routing support doesn't force clients to use it. Java and all existing
clients would work as now where request are intelligently routed by the
client, but this would lower the bar for new clients. That said I agree the
case for reroute get admin commands is much stronger than
[
https://issues.apache.org/jira/browse/KAFKA-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311387#comment-14311387
]
Evan Huus commented on KAFKA-1718:
--
[~guozhang], [~jkreps] my understanding is that while
[
https://issues.apache.org/jira/browse/KAFKA-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Holoman reassigned KAFKA-1929:
---
Assignee: Jeff Holoman
Convert core kafka module to use the errors in
[
https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311445#comment-14311445
]
Neha Narkhede commented on KAFKA-313:
-
bq. If KAFKA-1476 will be committed soon, it
Looks good to me.
I like the idea of not blocking additional sends but not guaranteeing that
flush() will deliver them.
I assume that with linger.ms = 0, flush will just be a noop (since the
queue will be empty). Is that correct?
Gwen
On Sun, Feb 8, 2015 at 10:25 AM, Jay Kreps
GitHub user redbaron opened a pull request:
https://github.com/apache/kafka/pull/43
Finer locking in log append
This patch adds finer locking when appending to log. It breaks
global append lock into 2 sequential and 1 parallel phase.
Basic idea is to allow every thread
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30763/#review71573
---
Minor issue with cleaning an InterruptionException, but otherwise
Thanks for the background.
I picked the Network classes portion of it, since I was already looking at
how to refactor send/receive and friends to support extending with TLS and
SASL. Having to do this in just one place will be really nice :)
Gwen
On Sun, Feb 8, 2015 at 7:26 AM, Jay Kreps
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29893/#review71574
---
dev-utils/test-patch.py
[
https://issues.apache.org/jira/browse/KAFKA-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-1486.
-
Resolution: Duplicate
I think this duplicated KAFKA-1927. Re-open and explain if I got it wrong
Jay,
Can we add another package (or two) to org.apache.kafka.common for metadata
and consensus. We can call them something else but the idea would be to
have 1 common layer for meta data information (right now we put the json
into zookeeper) and 1 common layer for asynchronous watches (which we
This was awesome :)
Peak rate of 3 per minute was reported around 3:30pm PST ;)
On Sat, Feb 7, 2015 at 4:40 PM, Jay Kreps jay.kr...@gmail.com wrote:
I closed about 350 redundant or obsolete issues. If I closed an issue you
think is not obsolete, my apologies, just reopen.
-Jay
I think the new tickets can be done in parallel, and are not an actual
dependency for KAFKA-1845. Is that correct?
On Sat, Feb 7, 2015 at 1:44 PM, Jay Kreps jay.kr...@gmail.com wrote:
I don't think we need a KIP/vote here, this is just an internal
refactoring. We had said previously and noted
Yeah totally, all the cleanups should be independent, this thread just
reminded me to file tickets for them.
-jay
On Sunday, February 8, 2015, Gwen Shapira gshap...@cloudera.com wrote:
I think the new tickets can be done in parallel, and are not an actual
dependency for KAFKA-1845. Is that
[
https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311353#comment-14311353
]
Andrii Biletskyi commented on KAFKA-1845:
-
Updated reviewboard
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30126/
---
(Updated Feb. 8, 2015, 3:05 p.m.)
Review request for kafka.
Bugs: KAFKA-1845
[
https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrii Biletskyi updated KAFKA-1845:
Attachment: KAFKA-1845_2015-02-08_17:05:22.patch
KafkaConfig should use ConfigDef
Hey all,
Someone asked about why there is code duplication between org.apache.common
and core. The answer seemed like it might be useful to others, so including
it here:
Originally Kafka was more of a proof of concept and we didn't separate the
clients from the server. LinkedIn was much smaller
Hi Neha,
Yes, I’ve updated the KIP so the entire KIP is based on new consumer now.
I’ve put both designs with and without data channel in the KIP as I still
feel we might need the data channel to provide more flexibility,
especially after message handler is introduced. I’ve put my thinking of
the
Maxim Ivanov created KAFKA-1933:
---
Summary: Fine-grained locking in log append
Key: KAFKA-1933
URL: https://issues.apache.org/jira/browse/KAFKA-1933
Project: Kafka
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/KAFKA-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311527#comment-14311527
]
Maxim Ivanov commented on KAFKA-1933:
-
Created reviewboard
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30775/
---
Review request for kafka.
Bugs: KAFKA-1933
[
https://issues.apache.org/jira/browse/KAFKA-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Ivanov updated KAFKA-1933:
Attachment: KAFKA-1933.patch
Fine-grained locking in log append
Well actually in the case of linger.ms = 0 the send is still asynchronous
so calling flush() blocks until all the previously sent records have
completed. It doesn't speed anything up in that case, though, since they
are already available to send.
-Jay
On Sun, Feb 8, 2015 at 10:36 AM, Gwen
Github user redbaron closed the pull request at:
https://github.com/apache/kafka/pull/43
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Jiangjie Qin created KAFKA-1934:
---
Summary: Add a shutdownNow() call to new producer
Key: KAFKA-1934
URL: https://issues.apache.org/jira/browse/KAFKA-1934
Project: Kafka
Issue Type: Bug
[
https://issues.apache.org/jira/browse/KAFKA-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311599#comment-14311599
]
Jiangjie Qin commented on KAFKA-1447:
-
I think KAFKA-1305 solved this issue.
Few comments -
1. Why do we need the message handler? Do you have concrete use cases in
mind? If not, we should consider adding it in the future when/if we do have
use cases for it. The purpose of the mirror maker is a simple tool for
setting up Kafka cluster replicas. I don't see why we need to
[
https://issues.apache.org/jira/browse/KAFKA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311617#comment-14311617
]
Jiangjie Qin commented on KAFKA-1908:
-
[~aozeritsky] It looks the scenario should not
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30777/
---
Review request for kafka.
Bugs: KAFKA-1919
[
https://issues.apache.org/jira/browse/KAFKA-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps resolved KAFKA-1934.
--
Resolution: Duplicate
Add a shutdownNow() call to new producer
[
https://issues.apache.org/jira/browse/KAFKA-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps updated KAFKA-1934:
-
Issue Type: New Feature (was: Bug)
Add a shutdownNow() call to new producer
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29467/#review71595
---
I would vote for the name
close(long timeout, TimeUnit unit)
I
Yeah, I second Neha's comments. The current mm code has taken something
pretty simple and made it pretty scary with callbacks and wait/notify
stuff. Do we believe this works? I can't tell by looking at it which is
kind of bad for something important like this. I don't mean this as
criticism, I
[
https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311630#comment-14311630
]
Guozhang Wang commented on KAFKA-1919:
--
Comments for the patch: instead of calling
[
https://issues.apache.org/jira/browse/KAFKA-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311689#comment-14311689
]
Ewen Cheslack-Postava commented on KAFKA-1934:
--
There was previous discussion
On Feb. 9, 2015, 1:27 a.m., Jay Kreps wrote:
clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java,
line 371
https://reviews.apache.org/r/29467/diff/1/?file=802888#file802888line371
This approach will actually leak the sender thread if there are still
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311699#comment-14311699
]
Jay Kreps commented on KAFKA-1660:
--
This is very similar to KAFKA-1659 and KAFKA-1934.
Hi Jay, thanks a lot for the comments.
I think this solution is better. We probably don’t need data channel
anymore. It can be replaced with a list of producer if we need more sender
thread.
I’ll update the KIP page.
The reasoning about message handler is mainly for efficiency purpose. I’m
Thanks for the feedback, Neha. Please see inline replies.
―Jiangjie (Becket) Qin
On 2/8/15, 2:40 PM, Neha Narkhede n...@confluent.io wrote:
Few comments -
1. Why do we need the message handler? Do you have concrete use cases in
mind? If not, we should consider adding it in the future when/if
[
https://issues.apache.org/jira/browse/KAFKA-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311684#comment-14311684
]
Jay Kreps commented on KAFKA-1934:
--
This is a public interface change so we should do a
[
https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311632#comment-14311632
]
Guozhang Wang commented on KAFKA-1884:
--
[~omkreddy] I think this is a valid point, we
[
https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311683#comment-14311683
]
Jay Kreps commented on KAFKA-1919:
--
That's reasonable. Patch here:
[
https://issues.apache.org/jira/browse/KAFKA-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311690#comment-14311690
]
Jay Kreps commented on KAFKA-1933:
--
This is very interesting but also kind of scary. My
69 matches
Mail list logo
