Re: Kafka or Flume

2017-07-02 Thread Sidharth Kumar
t; On Sat, Jul 1, 2017 at 7:26 AM, Sidharth Kumar < > sidharthkumar2...@gmail.com> wrote: > >> Thanks for your suggestions. I feel kafka will be better but need some >> extra like either kafka with flume or kafka with spark streaming. Can you >> kindly suggest which wi

Re: Kafka or Flume

2017-07-02 Thread Gagan Brahmi
low should be simpler since you might need some basic > verification of the data. You may want to include NiFi in the mix which > should do the job. > > It can look something like this: > > > Regards, > Gagan Brahmi > > On Sat, Jul 1, 2017 at 7:26 AM, Sidharth Kumar <

Re: Kafka or Flume

2017-07-01 Thread Sidharth Kumar
gt; Thanks for your suggestions. I feel kafka will be better but need some > extra like either kafka with flume or kafka with spark streaming. Can you > kindly suggest which will be better and in which situation which > combination will perform best. > > Thanks in advance for your hel

Re: Kafka or Flume

2017-07-01 Thread Gagan Brahmi
Regards, Gagan Brahmi On Sat, Jul 1, 2017 at 7:26 AM, Sidharth Kumar <sidharthkumar2...@gmail.com> wrote: > Thanks for your suggestions. I feel kafka will be better but need some > extra like either kafka with flume or kafka with spark streaming. Can you > kindly suggest which wil

Re: Kafka or Flume

2017-07-01 Thread Sidharth Kumar
Thanks for your suggestions. I feel kafka will be better but need some extra like either kafka with flume or kafka with spark streaming. Can you kindly suggest which will be better and in which situation which combination will perform best. Thanks in advance for your help. Warm Regards Sidharth

Re: Kafka or Flume

2017-06-29 Thread daemeon reiydelle
complexity issues which will exceed the capability of Flume. ​There are git repos that have everything you need, which include the kafka adapter, hdfs writer, etc. A lot of this is built into flume. ​ I assume this might be a bit off topic, so googling flume & kafka will help you? On Thu, Ju

RE: Kafka or Flume

2017-06-29 Thread Mallanagouda Patil
combination of these steps to process over 10 million events/second. I hope it helps.. Thanks Mallan On Jun 30, 2017 10:31 AM, "Sidharth Kumar" <sidharthkumar2...@gmail.com> wrote: > Thanks! What about Kafka with Flume? And also I would like to tell that > everyday data

RE: Kafka or Flume

2017-06-29 Thread Sidharth Kumar
Thanks! What about Kafka with Flume? And also I would like to tell that everyday data intake is in millions and can't afford to loose even a single piece of data. Which makes a need of high availablity. Warm Regards Sidharth Kumar | Mob: +91 8197 555 599/7892 192 367 | LinkedIn

RE: Kafka or Flume

2017-06-29 Thread JP gupta
Singh Thakur [mailto:sudeepthaku...@gmail.com] Sent: 30 June 2017 09:26 To: Sidharth Kumar Cc: Maggy; common-u...@hadoop.apache.org Subject: Re: Kafka or Flume In your use Kafka would be better because you want some transformations and validations. Kind regards, Sudeep Singh Thakur On Jun 30, 2017 8:57 A

Re: Kafka or Flume

2017-06-29 Thread Sudeep Singh Thakur
my question is which > injestion tool will be best for this Kafka or Flume? > > Any suggestions will be a great help for me. > > > Warm Regards > > Sidharth Kumar | Mob: +91 8197 555 599/7892 192 367 | LinkedIn: > www.linkedin.com/in/sidharthkumar2792 > > > > > >

Kafka or Flume

2017-06-29 Thread Sidharth Kumar
of historical data which is stored in hadoop. So, my question is which injestion tool will be best for this Kafka or Flume? Any suggestions will be a great help for me. Warm Regards Sidharth Kumar | Mob: +91 8197 555 599/7892 192 367 | LinkedIn: www.linkedin.com/in/sidharthkumar2792