I have found out what was the issue -- it was bug in the library.
Thanks everyone for help.

On Tue, Apr 21, 2015 at 9:02 PM, Andrei <[email protected]> wrote:
> @Piotr I can confirm that in native (Java/Scala) library returned message
> offsets start with the one specified in request, not the next one. Note
> sure about underlying protocol, though.
>
> As long as you also know Python, I would suggest you looking at
> kafka-python [1] library that is pretty readable and has already solved
> most issues you will encounter.
>
> [1]: https://github.com/mumrah/kafka-python
>
>
>
> On Tue, Apr 21, 2015 at 6:46 PM, Piotr Husiatyński <[email protected]> wrote:
>
>> I'm using Go with kafka library** and sending fetch request as
>> described in wire protocol documentation returns messages off by one.
>> Are you sure that code that you wrote is not sending fetch request
>> with offset being one less than you pass?
>>
>> ** https://github.com/optiopay/kafka
>>
>> On Tue, Apr 21, 2015 at 4:54 PM, Manoj Khangaonkar
>> <[email protected]> wrote:
>> > Hi,
>> >
>> > I have used code like
>> >
>> > FetchRequest req = new FetchRequestBuilder()
>> >           .clientId(clientname)
>> >           .addFetch(topic, partition, offset_in_partition,
>> 100000).build();
>> > FetchResponse fetchResponse = consumer.fetch(req);
>> >
>> > And it returns the message that I was looking for.
>> >
>> > The offset_in_partion is a value I get by calling
>> >
>> > offset_in_partition = messageAndOffset.nextOffset(); // call this on a
>> > previous message
>> >
>> > and saving it somewhere for future use.
>> >
>> > The only caveat is that fetch returns a block of messages which means I
>> > might
>> > get messages with lower offset that I just need to ignore.
>> >
>> > regards
>> >
>> >
>> > On Tue, Apr 21, 2015 at 6:47 AM, Piotr Husiatyński <[email protected]>
>> wrote:
>> >
>> >> What do you mean that message has expired?
>> >>
>> >> If I will do fetch request with offset 4, I will receive first message
>> >> with offset 5. I think fetch is always excluding requested offset,
>> >> while documentation is saying it will be included in response.
>> >>
>> >> On Tue, Apr 21, 2015 at 3:35 PM, Manoj Khangaonkar
>> >> <[email protected]> wrote:
>> >> > Hi,
>> >> >
>> >> > I suspect If some message from the given offset have expired, then
>> they
>> >> > will not be returned.
>> >> >
>> >> > regards
>> >> >
>> >> > On Tue, Apr 21, 2015 at 5:14 AM, Piotr Husiatyński <[email protected]>
>> >> wrote:
>> >> >
>> >> >> According to documentation, sending fetch request with offset value
>> >> >> result in messages starting with given offset (including) and
>> greater.
>> >> >>
>> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-FetchAPI
>> >> >>
>> >> >> > Logically one specifies the topics, partitions, and starting
>> offset at
>> >> >> which to begin the fetch and gets back a chunk of messages. In
>> general,
>> >> the
>> >> >> return messages will have offsets larger than or equal to the
>> starting
>> >> >> offset
>> >> >>
>> >> >> I'm sending fetch request with offset 5 and first message I'm getting
>> >> >> is offset 6. Am I doing something wrong or documentation has to be
>> >> >> fixed?
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > http://khangaonkar.blogspot.com/
>> >>
>> >
>> >
>> >
>> > --
>> > http://khangaonkar.blogspot.com/
>>

Reply via email to