I didn't know I have to manually set the checkpoint topic as log-compaction. Is there any reason why log-compaction is not being set by default on the checkpoint topic?
I will try and see how it works. Thank you Best, Jae On Tue, Nov 3, 2015 at 8:33 PM, Yi Pan <nickpa...@gmail.com> wrote: > Hi, Bae, > > Where did you see this log? Is it in JobRunner? Or AppMaster? Or > SamzaContainer? > > There are a few factors that may have the impact: > 1. How many system stream partitions you have as the input? And how many > tasks are there? > 2. Did you set your checkpoint topic as log-compact topic in Kafka? The > topic size would be much smaller if log compaction is turned on. > > Regards > > -Yi > > On Tue, Nov 3, 2015 at 3:59 PM, Bae, Jae Hyeon <metac...@gmail.com> wrote: > > > Hi Samza Dev > > > > Do you know why the following job is taking too long? > > > > 2015-11-03 23:58:17 KafkaCheckpointManager [INFO] Get latest offset > 3386930 > > for topic __samza_checkpoint_ver_1_for_xxx_1 and partition 0. > > > > This is seriously slowing down development. How can I fix this problem? > > > > Thank you > > Best, Jae > > >
