Kafka 0.8

2012-09-19 Thread Vito Sabella
Is there a quick place where I can get up to speed on the client protocol and other changes in 0.8? Thanks Vito

Re: Kafka 0.8

2012-09-19 Thread Neha Narkhede
The design of Kafka 0.8 is here - https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Replication The client protocol changes are here - https://cwiki.apache.org/confluence/display/KAFKA/New+Wire+Format+Proposal Thanks, Neah On Wed, Sep 19, 2012 at 11:16 AM, Vito Sabella wrote: > Is ther

Re: Kafka 0.8

2012-09-19 Thread Joel Koshy
ign of Kafka 0.8 is here - > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Replication > The client protocol changes are here - > https://cwiki.apache.org/confluence/display/KAFKA/New+Wire+Format+Proposal > > Thanks, > Neah > > On Wed, Sep 19, 2012 at 11:16 AM, Vito S

RE: Kafka 0.8

2012-09-19 Thread Vito Sabella
Thanks! Are there any plans to move to Thrift/Avro/similar/... to make writing and maintaining new clients in other languages easier?  Would that be a desired contribution? -Vito > Date: Wed, 19 Sep 2012 11:26:39 -0700 > Subject: Re: Kafka 0.8

Re: Kafka 0.8

2012-09-19 Thread Jun Rao
> > Date: Wed, 19 Sep 2012 11:26:39 -0700 > > Subject: Re: Kafka 0.8 > > From: neha.narkh...@gmail.com > > To: kafka-dev@incubator.apache.org > > > > The design of Kafka 0.8 is here - > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Rep

Re: Kafka 0.8

2012-09-20 Thread Jay Kreps
s to move to Thrift/Avro/similar/... to make > > writing and maintaining new clients in other languages easier? Would > that > > be a desired contribution? > > > > -Vito > > > > -------- > > > Date: Wed, 19 Sep 2012 11:26:3

Kafka 0.8 trial version

2012-11-26 Thread Jun Rao
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.8+Quick+Start The code is expected to be stable and we are still looking into some performance issues with hundreds of topic/partitions. If you see any issues, please let us know. Thanks, Jun

Producer request in kafka-0.8

2012-08-23 Thread Rakesh Nair
The current project i am doing requires producer requests to be acknowledged by kafka. Since this fix seems to have gone into kafka0.8 i thought of trying it out. I have a question on whether to use Producer class or SyncProducer class for sending the request. I am using Producer class for sending

Re: SyncProducer on Kafka 0.8

2012-08-29 Thread Jun Rao
In 0.8, topics need to be created first before you can produce to them. Try using Producer instead, which has the logic to trigger the auto creation of a topic. Thanks, Jun On Wed, Aug 29, 2012 at 1:27 PM, Rakesh Nair wrote: > I am trying to use the SyncProducer in Kafka 0.8. Here is a sim

Re: Kafka 0.8 trial version

2012-11-27 Thread S Ahmed
to give it a try. > Please check out revision 1411070 from > https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8 and follow > the following wiki. > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.8+Quick+Start > > The code is expected to be stable and we are still

Re: Kafka 0.8 trial version

2012-11-27 Thread Neha Narkhede
we'd like people to give it a try. >> Please check out revision 1411070 from >> https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8 and follow >> the following wiki. >> >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.8+Quick+Start >>

Re: Producer request in kafka-0.8

2012-08-23 Thread Neha Narkhede
In addition to this, you want to ensure that "producer.type" is set to "sync". That will ensure the send is synchronous with acks. Thanks, Neha On Thu, Aug 23, 2012 at 4:36 PM, Rakesh Nair wrote: > The current project i am doing requires producer requests to be > acknowledged by kafka. Since thi

end-to-end latency in kafka 0.8

2012-10-25 Thread Jay Kreps
I ran the end-to-end latency test on 0.8 branch and it actually looks pretty good even with the remaining 0.8 perf bugs. I get an average 0.29 ms for the producer=>broker=>consumer roundtrip over localhost. Some background for those not following along: End-to-end latency is the time it takes for

Re: Build failed in Jenkins: Kafka-0.8 #7

2012-07-19 Thread Jay Kreps
Does anyone know what this scaladoc failure is? It is totally impossible to understand that error -Jay On Thu, Jul 19, 2012 at 3:04 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/Kafka-0.8/7/changes> > > Changes: &

Re: Build failed in Jenkins: Kafka-0.8 #7

2012-07-20 Thread Neha Narkhede
.org> wrote: > > > See <https://builds.apache.org/job/Kafka-0.8/7/changes> > > > > Changes: > > > > [jkreps] KAFKA-371 Refactoring of LogManager. Reviewed by Neha. > > > > -- > > [...truncated 2641 lines...] > &

Re: Build failed in Jenkins: Kafka-0.8 #7

2012-07-20 Thread Joel Koshy
> Until we know the cause, looks like deleting the doc on the Producer's > constructor might be ok. Thoughts ? > +1 . This shouldn't hold up jenkins and it does look like a bug.

Re: Build failed in Jenkins: Kafka-0.8 #16

2012-08-01 Thread Neha Narkhede
] [0mError running kafka.server.ServerShutdownTest: Test FAILED [0m Thanks, Neha On Wed, Aug 1, 2012 at 10:27 AM, Apache Jenkins Server wrote: > See <https://builds.apache.org/job/Kafka-0.8/16/changes> > > Changes: > > [junrao] recommit: revisit the become leader and become

Re: Build failed in Jenkins: Kafka-0.8 #19

2012-08-14 Thread Neha Narkhede
Same issue where one unit test fails to load the snappy libraries On Tue, Aug 14, 2012 at 1:08 PM, Apache Jenkins Server wrote: > See <https://builds.apache.org/job/Kafka-0.8/19/changes> > > Changes: > > [jkreps] KAFKA-424. Fix command line usage description. Patch fro

Re: Build failed in Jenkins: Kafka-0.8 #23

2012-08-20 Thread Jay Kreps
wrote: > See <https://builds.apache.org/job/Kafka-0.8/23/changes> > > Changes: > > [junrao] enforce broker.id to be a non-negative integer; patched by > Swapnil Ghike; reviewed by Jun Rao, Neha Narkhede; KAFKA-424 > > -- > [.

Re: Build failed in Jenkins: Kafka-0.8 #36

2012-09-10 Thread Prashanth Menon
Hmmm, the Jenkins build fails to load the native snappy libraries. I thought this issue was resolved? On Mon, Sep 10, 2012 at 10:37 AM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/Kafka-0.8/36/changes> > > Changes: > &

Re: end-to-end latency in kafka 0.8

2012-10-25 Thread Taylor Gautier
And I believe you added group-commit also, yes? On Thu, Oct 25, 2012 at 5:08 PM, Jay Kreps wrote: > I ran the end-to-end latency test on 0.8 branch and it actually looks > pretty good even with the remaining 0.8 perf bugs. I get an average 0.29 ms > for the producer=>broker=>consumer roundtrip o

Re: end-to-end latency in kafka 0.8

2012-10-25 Thread Neha Narkhede
Single machine performance looks great. I'm assuming this was done with replication factor = 1, right ? In reality, the only factor that can affect this number, is the time it takes for the followers to fetch the data. This is important for end-to-end latency since until the follower replicas fetc

Re: end-to-end latency in kafka 0.8

2012-10-25 Thread Jay Kreps
Yes, this is single machine, I think that is the most directly comparable case to 0.7. It would be good to rerun with a replicated topic too. -Jay On Thu, Oct 25, 2012 at 5:44 PM, Neha Narkhede wrote: > Single machine performance looks great. I'm assuming this was done > with replication factor

Re: Build failed in Jenkins: Kafka-0.8 #113

2012-11-20 Thread Jay Kreps
Hey Jun, I think AdminTest.testShutdownBroker started failing consistently after this checkin. -Jay On Sun, Nov 18, 2012 at 10:09 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/Kafka-0.8/113/changes> > > Changes: >