Reset offset of consumer to 0

2016-12-14 Thread Jagat Singh
Hi, Is there any easy way to reset offset of consumer group back to 0 for given topic. I am using below command to check offset bin/kafka-consumer-offset-checker.sh --zookeeper zookeeper-host.com:2181 --group consumer-group --topic nameoftopic --security SASL_PLAINTEXT Thanks, J

Error Failed to update metadata after 60000 ms: Kafka 0.10 + Kerberos + HDP 2.5

2016-10-20 Thread Jagat Singh
We are trying to run a simple Kafka code with new api to test Kerberos integration. The code is present at https://pastebin.ubuntu.com/23356675/ We are using ticket based mechanism to test first and then we will use Keytabs java -Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login

Re: Kafka (Streams) scalability

2016-07-23 Thread Jagat Singh
My post if not directly referring to KS. The new free book by Orielly has very good explanation about Kafka Topic counts. You can download it from below link ( See Chapter 4) http://shop.oreilly.com/product/0636920049463.do In short quoting from there >>> These problems are likely substantiall

Re: Trying to get kafka data to Hadoop

2015-03-04 Thread Jagat Singh
Also see the related tool http://confluent.io/downloads/ Confluent is bringing the glue together for Kafta , Avro , Camus Though there is no clarity around support (e.g update of Kafta) around it at this moment. On Thu, Mar 5, 2015 at 8:57 AM, Joel Koshy wrote: > I think the camus mailing l

Re: Large Messages

2014-11-17 Thread Jagat Singh
You can see the compression options. To change the size the property to look for is related to message.max.bytes. On Tue, Nov 18, 2014 at 9:51 AM, Parag Shah wrote: > Hi all, > > I am fairly new to Kafka and need some general advice on handling > large messages. My message sizes range from

Re: Is AWS Kinesis Kafka?

2014-11-13 Thread Jagat Singh
Do you think things will change now , with Kafta makers setting up own company and will provide commercial support? On Fri, Nov 14, 2014 at 10:28 AM, cac...@gmail.com wrote: > Yeah the real question is really are the products built on top of Kafka > (Kafka with a hat on). The last place I worke

Re: One of two consumers is always Idle though I have 2 partitions

2014-11-13 Thread Jagat Singh
It would be worth reading once the consumer section from the documentation. https://kafka.apache.org/documentation.html On Fri, Nov 14, 2014 at 10:09 AM, Palur Sandeep wrote: > Yes, they are on the same consumer group, but I have two partitions. > > On Thu, Nov 13, 2014 at 5:04

Re: One of two consumers is always Idle though I have 2 partitions

2014-11-13 Thread Jagat Singh
Are both of them in same Consumer Group? On Fri, Nov 14, 2014 at 9:12 AM, Palur Sandeep wrote: > Dear Developers, > > I am 2nd year masters student at IIT. I am using Kafka for one of my > research projects.My question is the following: > > 1. I have a producer, consumer and a broker(that contai