Re: Create topic, then fetch offset

2015-01-14 Thread Guozhang Wang
About the misleading error code, it is covered in the following ticket: KAFKA-1852 https://issues.apache.org/jira/browse/KAFKA-1852 On Wed, Jan 14, 2015 at 12:11 AM, Henri Pihkala henri.pihk...@streamr.com wrote: Thanks Harsha, Looks like ZkStringSerializer was the key to creating topics via

Re: Create topic, then fetch offset

2015-01-14 Thread Henri Pihkala
Thanks Harsha, Looks like ZkStringSerializer was the key to creating topics via AdminUtils. That’s quite an easy mistake to make. I now noticed that the issue is already being addressed at https://issues.apache.org/jira/browse/KAFKA-1737 https://issues.apache.org/jira/browse/KAFKA-1737, great

Re: Create topic, then fetch offset

2015-01-13 Thread Harsha
It looks like you are not adding a ZkStringSerializer to your zkClient. ZkClient zkClient = new ZkClient(ZK_CONN_STRING); in Kafka TopicCommand uses new ZkClient(opts.options.valueOf(opts.zkConnectOpt), 3, 3, ZKStringSerializer) because of this although your topic is getting created and

Re: Create topic, then fetch offset

2015-01-13 Thread Mayuresh Gharat
Also I don’t think you need to try the TopicMetadataRequest when auto.create.topics.enable=true, explicitly. I suppose if you try to produce using a topic name Kafka should automatically create the topic for you. Thanks, Mayuresh On Jan 13, 2015, at 2:28 PM, Henri Pihkala

Re: Create topic, then fetch offset

2015-01-13 Thread Guozhang Wang
Henri, FetchOffsetRequest is used for retrieving the last committed offset of a certain consumer group; for your use case you would send OffsetRequest. That said the error code is misleading, it may be a bug in not indicating the right error code. Guozhang On Fri, Jan 9, 2015 at 4:10 AM,

Re: Create topic, then fetch offset

2015-01-13 Thread Harsha
You can import ZKStringSerializer from kafka.utils.ZkClient or write your own similar string serializer like this https://gist.github.com/harshach/7b5447c39168eb6062e0 On Tue, Jan 13, 2015, at 04:31 PM, Harsha wrote: It looks like you are not adding a ZkStringSerializer to your zkClient.

Re: Create topic, then fetch offset

2015-01-13 Thread Harsha
Internally producer sends a TopicMetadataRequest which creates the topic if auto.create.topics.enable is true. On Tue, Jan 13, 2015, at 04:49 PM, Harsha wrote: You can import ZKStringSerializer from kafka.utils.ZkClient or write your own similar string serializer like this

Re: Create topic, then fetch offset

2015-01-13 Thread Henri Pihkala
Hi, Thanks for your reply, Guozhang, you are right. I eventually got it working, but it required TWO changes to what I initially tried: 1) Use OffsetRequest, not OffsetFetchRequest, like Guozhang pointed out. 2) Create topics by sending a TopicMetadataRequest and having