Hi Snehal, Thanks for your input. They already have their own Java APIs to access data. But why do I create rest API? What is the benefits?
Say, if there are 500 data providers, do I need 500 producers at my end to collect data? At least, the number of producers should be proportional to number of data providers. In addition, I also need maybe 500 or more kafka servers. You see, I think the system has too many producers. It is waste of money. Best, Chao -----Original Message----- From: Snehal Nagmote [mailto:nagmote.sne...@gmail.com] Sent: Wednesday, July 13, 2016 3:38 PM To: users@kafka.apache.org Subject: Re: Role of Producer Hi Chao , To solve this problem , I can think of creating rest api . Your end point can have one of the parameter as data provider if you want to send it to different topics based on data provider . On backend , when you get data , you can send it to Kafka Topics, using Kafka Producer at the end. Thanks, Snehal On 13 July 2016 at 13:31, Luo, Chao <chao....@here.com> wrote: > Dear Kafka guys, > > I just started to build up a Kafka system two weeks ago. Here I have a > question about how to design/implement the producer. > > In my system, there are many data providers. I need to collect > real-time data from them and store it in a NoSQL database. The problem > is that different data providers have their own Java APIs, and they > will not use Kafka-client to send data directly to my Kafka servers. > So I need to first collect data from them and feed it to the Kafka > servers. I guess I need to finish data acquisition in the Producers. > My question is that there are a great number of data providers so I > also need a lot of producers??? Or is there any more efficient ways to deal > with it? > > Best, > Chao >
