Actually I have not tried this. But I do not think it will work. Because the next offset is the offset you set + 1. There is no mechanism inside Samza that treats -1 as the latest (though in Kafka it does). If it works, let me know. :) I may miss something.
Another way of doing this is to set the latest offset using the checkpoint tool. You can get the latest offset of the topic from Kafka using bin/kafka-run-class.sh kafka.tools.GetOffsetShell <https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-ConsumerOffsetChecker> Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jul 14, 2015 at 10:33 AM, Bae, Jae Hyeon <metac...@gmail.com> wrote: > Hi Yan > > Thanks for your response. I want to reset the offset just once now. If I > use systems.system-name.streams.stream-name.samza.reset.offset in the > configuration file, before restarting the job, I need to remove those > properties again. That's why I don't want to use that property. > > Is it OK setting invalid offset such as -1 or Long.MAX_VALUE? > > On Tue, Jul 14, 2015 at 10:16 AM, Yan Fang <yanfang...@gmail.com> wrote: > > > Hi Jae, > > > > Do you want to reset the offset to the latest one when you *start *the > job > > or when the just is *running*? > > > > If it's the former one, you can use > > > > systems.system-name.samza.offset.default=upcoming > > systems.system-name.streams.stream-name.samza.reset.offset=true > > > > What is the reason that you do not want to use > > systems.system-name.streams.stream-name.samza.reset.offset > > ? > > > > Thanks, > > > > Fang, Yan > > yanfang...@gmail.com > > > > On Tue, Jul 14, 2015 at 12:49 AM, Bae, Jae Hyeon <metac...@gmail.com> > > wrote: > > > > > Hi > > > > > > I want to reset the offset for the job to the latest one, which means I > > > want to ignore them without using > > > systems.system-name.streams.stream-name.samza.reset.offset option. > > > > > > If I use checkpoint tool and reset the offset as -1 or Long.MAX_VALUE, > in > > > my theory, kafka consumer will throw an exception and it will reset > them > > as > > > the latest one, am I right? Otherwise, please let me know how to reset > > > offsets to the latest one. > > > > > > Thank you > > > Best, Jae > > > > > >
