Hmm..yeah some error logs would be nice like Gwen pointed out. Do any of your brokers fall out of the ISR when sending messages? It seems like your setup should be fine, so I'm not entirely sure.
On Tue, Jul 14, 2015 at 1:31 PM, Yuheng Du <[email protected]> wrote: > Jiefu, > > I am performing these tests on a 6 nodes cluster in cloudlab (a > infrastructure built for scientific research). I use 2 nodes as producers, > 2 as brokers only, and 2 as consumers. I have tested for each individual > machines and they work well. I did not use AWS. Thank you! > > On Tue, Jul 14, 2015 at 4:20 PM, JIEFU GONG <[email protected]> wrote: > > > Yuheng, are you performing these tests locally or using a service such as > > AWS? I'd try using each separate machine individually first, connecting > to > > the ZK/Kafka servers and ensuring that each is able to first log and > > consume messages independently. > > > > On Tue, Jul 14, 2015 at 1:17 PM, Gwen Shapira <[email protected]> > > wrote: > > > > > Are there any errors on the broker logs? > > > > > > On Tue, Jul 14, 2015 at 11:57 AM, Yuheng Du <[email protected]> > > > wrote: > > > > Jiefu, > > > > > > > > Thank you. The three producers can run at the same time. I mean > should > > > they > > > > be started at exactly the same time? (I have three consoles from each > > of > > > > the three machines and I just start the console command manually one > by > > > > one) Or a small variation of the starting time won't matter? > > > > > > > > Gwen and Jiefu, > > > > > > > > I have started the three producers at three machines, after a while, > > all > > > of > > > > them gives a java.net.ConnectException: > > > > > > > > [2015-07-14 12:56:46,352] WARN Error in I/O with producer0-link-0/ > > > > 192.168.1.1 (org.apache.kafka.common.network.Selector) > > > > > > > > java.net.ConnectException: Connection refused...... > > > > > > > > [2015-07-14 12:56:48,056] WARN Error in I/O with producer1-link-0/ > > > > 192.168.1.2 (org.apache.kafka.common.network.Selector) > > > > > > > > java.net.ConnectException: Connection refused..... > > > > > > > > What could be the cause? > > > > > > > > Thank you guys! > > > > > > > > > > > > > > > > > > > > On Tue, Jul 14, 2015 at 2:47 PM, JIEFU GONG <[email protected]> > > wrote: > > > > > > > >> Yuheng, > > > >> > > > >> Yes, if you read the blog post it specifies that he's using three > > > separate > > > >> machines. There's no reason the producers cannot be started at the > > same > > > >> time, I believe. > > > >> > > > >> On Tue, Jul 14, 2015 at 11:42 AM, Yuheng Du < > [email protected] > > > > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > > >> > I am running the performance test for kafka. > > > >> > https://gist.github.com/jkreps > > > >> > /c7ddb4041ef62a900e6c > > > >> > > > > >> > For the "Three Producers, 3x async replication" scenario, the > > command > > > is > > > >> > the same as one producer: > > > >> > > > > >> > bin/kafka-run-class.sh > > > org.apache.kafka.clients.tools.ProducerPerformance > > > >> > test 50000000 100 -1 acks=1 > > > >> > bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 > > > >> > buffer.memory=67108864 batch.size=8196 > > > >> > > > > >> > So How to I run the test for three producers? Do I just run them > on > > > three > > > >> > separate servers at the same time? Will there be some error in > this > > > way > > > >> > since the three producers can't be started at the same time? > > > >> > > > > >> > Thanks. > > > >> > > > > >> > best, > > > >> > Yuheng > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> > > > >> Jiefu Gong > > > >> University of California, Berkeley | Class of 2017 > > > >> B.A Computer Science | College of Letters and Sciences > > > >> > > > >> [email protected] <[email protected]> | (925) 400-3427 > > > >> > > > > > > > > > > > -- > > > > Jiefu Gong > > University of California, Berkeley | Class of 2017 > > B.A Computer Science | College of Letters and Sciences > > > > [email protected] <[email protected]> | (925) 400-3427 > > > -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences [email protected] <[email protected]> | (925) 400-3427
