Re: Using Kafka to replace local .net message queues

Hi Akshay, In regards to your 3rd question (and indirectly to your 2nd question), instead of having different consumer groups, why not just multiple consumers in the same group? That would ensure that each consumer only reads from one partition in the topic. You can even assign the partition if

Re: Prioritized Topics for Kafka

Use case: we process documents from a variety of sources. We want to process some of these sources in a priority order, but we don’t want to necessarily finish all the higher priority sources before going to lower priority because the volume of higher priority sources can be extremely high. We

Re: SQL Server CDC

You can read more here: https://docs.confluent.io/current/connect/connect-jdbc/docs/source_connector.html Hope that helps. Thanks, Subhash Sent from my iPhone > On Jun 19, 2018, at 11:12 AM, Pranav Shah wrote: > > Hello, > > Is there connector available for Kafka SQL Server CDC? > >

Re: SQL Server CDC

Hi Pranav, Yes, there is a JDBC source connector that you can use to achieve CDC from SQL into Kafka using connect. Thanks, Subhash Sent from my iPhone > On Jun 19, 2018, at 11:12 AM, Pranav Shah wrote: > > Hello, > > Is there connector available for Kafka SQL Server CDC? > > Thanks, >

Re: one topic, one partition, many consumer instances

duplicates. Hope that helps. Thanks, Subhash On Tue, Feb 13, 2018 at 10:44 AM, Xavier Noria <f...@hashref.com> wrote: > On Tue, Feb 13, 2018 at 2:59 PM, Subhash Sriram <subhash.sri...@gmail.com> > wrote: > > Hey Xavier, > > > > Within a consumer group, you ca

Re: one topic, one partition, many consumer instances

Hey Xavier, Within a consumer group, you can only have as many consumers as you have partitions in a topic. If you start more consumers than partitions within the same group, they will just be idle. Thanks, Subhash Sent from my iPhone > On Feb 13, 2018, at 8:30 AM, Xavier Noria

Re: Training and Cert course recommendation

Hi Brian, Maybe this will be of help: https://www.confluent.io/certification/ Thanks, Subhash Sent from my iPhone > On Jan 19, 2018, at 1:15 PM, brian spallholtz > wrote: > > I am a SA/SE on several kafka clusters, I was wondering if there was a > training and

Re: Authorization in Kafka

Hi, I am not an expert, but from looking at the ACL documentation, you can't control read authorization at the partition level, only at the topic level. If it is possible to control access at the partition level, maybe you could have a dedicated partition for each customerID? Thanks, Subhash

Re: Kafka Monitoring

Hi Irtiza, Have you looked at jmxtrans? It has multiple output writers for the metrics and one of them is the KeyOutWriter which just writes to disk. https://github.com/jmxtrans/jmxtrans/wiki Hope that helps! Thanks, Subhash Sent from my iPhone > On Dec 6, 2017, at 5:36 AM, Irtiza Ali

Re: Querying consumer groups programmatically (from Golang)

Hi Jens, Have you looked at Burrow? https://github.com/linkedin/Burrow/blob/master/README.md Thanks, Subhash Sent from my iPhone > On Aug 12, 2017, at 8:55 AM, Jens Rantil wrote: > > Hi, > > I am one of the maintainers of prometheus-kafka-consumer-group-exporter[1],

Re: Kafka 0.10 - kafka console consumer not reading the data in order that it was published

by publishing to a single partition topic. When you consume, you should see that it is all in order. I hope that helps. Thanks, Subhash On Thu, Jun 22, 2017 at 4:37 PM, karan alang <karan.al...@gmail.com> wrote: > Hi Subhash, > > number of partitions - 3 > > On Thu, Jun 22, 2017

Re: Kafka 0.10 - kafka console consumer not reading the data in order that it was published

How many partitions are in your topic? On Thu, Jun 22, 2017 at 3:33 PM, karan alang wrote: > Hi All - > > version - kafka 0.10 > I'm publishing data into Kafka topic using command line, > and reading the data using kafka console consumer > > *Publish command ->* > >

Re: Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh

hould be listed again in the output, but it should get fresh > offsets (with `--describe` for example), since the old offsets were > removed once it became inactive. > > --Vahid > > > > > From: Subhash Sriram <subhash.sri...@gmail.com> > To: users@kafka

Re: Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh

ved from the internal topic offset and it > will not be listed in the consumer group command output. > > But the consumer group in your case should be alive, since it did not > become inactive. > > Did the command use to list the group in the output before? > > --Vahid

Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh

Hey everyone, I am a little bit confused about how the kafka-consumer-groups.sh/ ConsumerGroupCommand works, and was hoping someone could shed some light on this for me. We are running Kafka 0.10.1.0, and using the new Consumer API with the Confluent.Kafka C# library (v0.9.5) that uses