Re: Kinesis Operator Help

2016-02-17 Thread Tushar Gosavi
Hi Jim, Take a look at OffsetManager interface implemented for Kafka operator. There is something similar in kinesis (I guess its ShardManager), which you can override and provide your method to keep offset information at a fixed location. The newly launched application calls ShardManager loadInit

Re: Kinesis Operator Help

2016-02-17 Thread Sandesh Hegde
Jim, You can put the following line in a shell script, dtcli - e "launch -originalAppId apppid apa_path" Even you can shutdown the app from the shell script. On Tue, Feb 16, 2016, 10:22 PM Munagala Ramanath wrote: > By "relaunch option" I'm assuming you mean "launch -originalAppId ...". > Loo

RE: Kinesis Operator Help

2016-02-17 Thread Jim
Yes, Ram was right, I do not want a command line fix, because at some point, an operator is going to launch one without doing that, and the system will be messed up. Thanks, Jim From: Sandesh Hegde [mailto:sand...@datatorrent.com] Sent: Wednesday, February 17, 2016 9:05 AM To: users@apex.incub

Re: Kinesis Operator Help

2016-02-17 Thread Pramod Immaneni
Jim, Is there state in the rest of the operators in your application? If that cases storing the offsets only the input operator(s) is not sufficient as you will lose the state from the other operators in your DAG. Second modifying idempotent storage manager to store in an external location, while