Re: Review Request 52476: Do not load task store which are older than delete tombstones.

2017-02-07 Thread Shanthoosh Venkataraman
> On Oct. 24, 2016, 9:43 p.m., Prateek Maheshwari wrote: > > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala, > > line 138 > > > > > > s/is greater than/is older than. Fixed. > On Oct.

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

2017-02-07 Thread Shanthoosh Venkataraman
> On Jan. 25, 2017, 10:28 p.m., Prateek Maheshwari wrote: > > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala, > > line 97 > > > > > > "Got default storage ..." Changed. > On Jan. 25, 2

Re: Can't fetch messages from kafka

2017-02-07 Thread 舒琦
Hi, Thanks for your replying. I did not make a thread dump, if it happen again I’ll dump the thread stack. 舒琦 地址:长沙市岳麓区文轩路27号麓谷企业广场A4栋1单元6F 网址:http://www.eefung.com 微博:http://weibo.com/eefung 邮编:410013 电话:400-677-0986 传真:0731-88519609 > 在 2017年2月7日,23:52,Jagadish Venkatraman 写

Re: Questions about checkpointing and Kinesis

2017-02-07 Thread Boris S
If you are implementing your own consumer, on start Samza will call register method of the consumer and will pass the offsets. If you are using CheckpointListener (like Jagadish mentioned) you will get a callback on each checkpoint (so you can ignore the register call), otherwise you can use checkp

Re: Questions about checkpointing and Kinesis

2017-02-07 Thread Chad Greenberg
Thanks Jagadish for the reply. A few comments. I was under the impression that using the KCL would mean recording the offsets in Kinesis/DynanoDB and therefore not Samza. Avoiding the KCL so that I can save state in Samza. I assumed that the checkpoint values were based on the values of the p

Re: Can't fetch messages from kafka

2017-02-07 Thread Jagadish Venkatraman
>> all the task can’t fetch any messages any more, even after restarted the samza job Some questions: Are you observing an exception in the logs? Are you able to consume the messages using a simple consumer? Is Samza hanging on the call to fetch? A thread dump of the impacted container is helpfu