Re: Handling 2 to 3 Million Events before Kafka

2017-06-22 Thread SenthilKumar K
..@gmail.com] >>> Sent: Wednesday, June 21, 2017 10:28 AM >>> To: Tauzell, Dave >>> Cc: users@kafka.apache.org; senthilec...@apache.org; >>> d...@kafka.apache.org >>> Subject: Re: Handling 2 to 3 Million Events before Kafka >>> >>> So netty wou

Re: Handling 2 to 3 Million Events before Kafka

2017-06-22 Thread SenthilKumar K
r K [mailto:senthilec...@gmail.com] >> Sent: Wednesday, June 21, 2017 10:28 AM >> To: Tauzell, Dave >> Cc: users@kafka.apache.org; senthilec...@apache.org; d...@kafka.apache.org >> Subject: Re: Handling 2 to 3 Million Events before Kafka >> >> So netty would

Re: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Garrett Barton
Netty forum to see what they think? > -Dave > > From: SenthilKumar K [mailto:senthilec...@gmail.com] > Sent: Wednesday, June 21, 2017 10:28 AM > To: Tauzell, Dave > Cc: users@kafka.apache.org; senthilec...@apache.org; d...@kafka.apache.org > Subject: Re: Handling 2 to 3 Millio

RE: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Tauzell, Dave
...@kafka.apache.org Subject: Re: Handling 2 to 3 Million Events before Kafka So netty would work for this case ? I do have netty server and seems to be i'm not getting the expected results .. here is the git https://github.com/senthilec566/netty4-server , is this right implementation ? C

Re: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread SenthilKumar K
* users@kafka.apache.org; senthilec...@apache.org; > d...@kafka.apache.org; Senthil kumar > *Subject:* Re: Handling 2 to 3 Million Events before Kafka > > > > Thanks Jeyhun. Yes http server would be problematic here w.r.t network , > memory .. > > > > Hi Dave , Th

RE: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Tauzell, Dave
enthilec...@gmail.com<mailto:senthilec...@gmail.com>] Sent: Wednesday, June 21, 2017 7:58 AM To: users@kafka.apache.org<mailto:users@kafka.apache.org> Cc: senthilec...@apache.org<mailto:senthilec...@apache.org>; Senthil kumar; d...@kafka.apache.org<mailto:d...@kafka.apache.org> S

Re: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread SenthilKumar K
@gmail.com] > Sent: Wednesday, June 21, 2017 7:58 AM > To: users@kafka.apache.org > Cc: senthilec...@apache.org; Senthil kumar; d...@kafka.apache.org > Subject: Handling 2 to 3 Million Events before Kafka > > Hi Team , Sorry if this question is irrelevant to Kafka Group ... > &g

RE: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Tauzell, Dave
K [mailto:senthilec...@gmail.com] Sent: Wednesday, June 21, 2017 7:58 AM To: users@kafka.apache.org Cc: senthilec...@apache.org; Senthil kumar; d...@kafka.apache.org Subject: Handling 2 to 3 Million Events before Kafka Hi Team , Sorry if this question is irrelevant to Kafka Group ... I have

Re: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Jeyhun Karimov
Hi, With kafka you can increase overall throughput by increasing the number of nodes in a cluster. I had a similar issue, where we needed to ingest vast amounts of data to streaming system. In our case, kafka was a bottleneck, because of disk I/O. To solve it, we implemented (simple) distributed

Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread SenthilKumar K
Hi Team , Sorry if this question is irrelevant to Kafka Group ... I have been trying to solve problem of handling 5 GB/sec ingestion. Kafka is really good candidate for us to handle this ingestion rate .. 100K machines > { Http Server (Jetty/Netty) } --> Kafka Cluster.. I see the problem