Prabhjot, The answer changes slightly for the Producer and Consumer and depends on your timeline and comfort with using new APIs
Today and in the future, for the Producer, you should be using the "new" producer, which isn't all that new anymore: org.apache.kafka.clients.producer.KafkaProducer; Today with 0.9 yet to be released you'd likely want to use the High-Level Consumer. This is covered in the official docs here: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example and in this blog post http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/ along with most of the other examples that you'll find. After .9 is released, I'd encourage you to take a look at the new Consumer API. This has a lot of advantages in terms of offset management and will be the only consumer client that fully supports security features like SSL that are slated to be released into the platform. Your choice of development language is entirely up to you. Note that the only version of clients that will be maintained in the project going forward are being implemented in Java, so Scala or Java shouldn't matter too much for you. Hope this helps Jeff On Thu, Nov 5, 2015 at 12:14 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hello Folks, > > Requesting your expertise on this. > I see that under core/src/main/scala/kafka/producer/, there are many > implementations - Producer.scala and SyncProducer.scala > > Also, going via the producerPerformance.scala, there are 2 implementations > - NewShinyProducer (which points to KafkaProducer.java) and the OldProducer > > Similar might be the case with Consumers, but I have not seen that yet. > > Please let me know which producer and consumer is supposed to be used and > which ones will be phased out in future releases, so I can focus on only 1 > type of producer and consumer (high level as well as simple) > > Thanks, > Prabhjot > > Thanks, > Prabhjot > > On Thu, Nov 5, 2015 at 3:55 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Adding users as well > > > > On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > > wrote: > > > >> Hi, > >> > >> I'm using the latest update: 0.8.2.2 > >> I would like to use the latest producer and consumer apis > >> over the past few weeks, I have tried to do some performance > benchmarking > >> using the producer and consumer scripts provided in the bin directory. > It > >> was a fun activity and I have learnt a lot about kafka. > >> > >> But, I have also experienced that sometimes the implementation of the > >> performance scripts was not up-to-date or some items were different than > >> the documentation > >> > >> Now, I would like to develop my application with kafka. I'm comfortable > >> using scala/java > >> > >> Please let me know which producer and consumer (both high level and > >> simple) class/object should I be using > >> > >> Thanks a lot, > >> Prabhjot > >> > > > > > > > > -- > > --------------------------------------------------------- > > "There are only 10 types of people in the world: Those who understand > > binary, and those who don't" > > > > > > -- > --------------------------------------------------------- > "There are only 10 types of people in the world: Those who understand > binary, and those who don't" > -- Jeff Holoman Systems Engineer
