Thanks, i eventually figured that out, until points to the next offset that
would be available when a message arrives and thus from == until means no
message is available.
On Mon, Nov 30, 2015 at 5:07 PM, Cody Koeninger <c...@koeninger.org> wrote:
> If you had exactly 1 message in the 0th topicpartition, to read it you
> would use
>
> OffsetRange("topicname", 0, 0, 1)
>
> Kafka's simple shell consumer in that case would print
>
> next offset = 1
>
>
> So instead trying to consume
>
> OffsetRange("topicname", 0, 1, 2)
> shouldn't be expected to work
>
>
>
> On Sat, Nov 28, 2015 at 8:35 AM, Nikos Viorres <nvior...@gmail.com> wrote:
>
>> Hi,
>>
>> I am using KafkaUtils.createRDD to retrieve data from Kafka for batch
>> processing and
>> when Invoking KafkaUtils.createRDD with an OffsetRange where
>> OffsetRange.fromOffset == OffsetRange.untilOffset for a particular
>> partition, i get an empy RDD.
>> Documentation is clear that until is exclusive and from inclusive, but if
>> i use OffsetRange.untilOffset + 1 i get an invalid OffsetRange exception
>> during the check.
>> Sinve this should also apply in general (if untilOffset is exculsive you
>> cannot fetch it ), does it mean that untilOffset is also non-existent in
>> Kafka (and thus always exlcusive) or i am missing something?
>>
>> regards
>>
>> p.s. by manually using the kafka protocol to query the offsets i see
>> that kafka.api.OffsetRequest.EarliestTime()
>> == kafka.api.OffsetRequest.LatestTime() and set to a poisitive value
>>
>
>
