[External] Regarding kinesis data analytics for flink

2019-06-27 Thread Vishal Sharma
configure task slots Just wanted to know, If you can share your experience with this platform. Thanks, Vishal Sharma -- *_Grab is hiring. Learn more at _**https://grab.careers <https://grab.careers/>* By communicating with Grab Inc and/or its subsidiaries, associate companies and joi

[External] Using scala parallel collection with ForkJoinPool

2019-06-20 Thread Vishal Sharma
, ForkJoinPool by default uses 1 thread per core, can it cause flink task slots to starve for resources ? Also, are there any potential concurrency issues with this approach ? Thanks, Vishal Sharma -- *_Grab is hiring. Learn more at _**https://grab.careers <https://grab.careers/>* By communi

Re: [External] checkpoint metadata not in configured directory state.checkpoints.dir

2019-06-20 Thread Vishal Sharma
Hi Congxian, I am not sure how can I track the checkpoint path. Can you suggestion regarding this ? Thanks, Vishal Sharma On Thu, Jun 20, 2019 at 11:17 AM Congxian Qiu wrote: > Hi, Vishal > If you want to restart from the last competed external checkpoint of the > previous stoped

Re: [External] checkpoint metadata not in configured directory state.checkpoints.dir

2019-06-19 Thread Vishal Sharma
Hi Chesnay, Can you suggest, How should I go about automating job restart from last completed externalised checkpoint in case of failure ? I am not sure about the path for the latest completed checkpoint. Thanks, Vishal Sharma On Wed, Jun 19, 2019 at 11:11 PM Chesnay Schepler wrote

[External] checkpoint metadata not in configured directory state.checkpoints.dir

2019-06-19 Thread Vishal Sharma
xternalised checkpoint in case of failure. For this to happen, I need to able to figure out path for the metadata of latest checkpoint. Thanks, Vishal Sharma -- *_Grab is hiring. Learn more at _**https://grab.careers <https://grab.careers/>* By communicating with Grab Inc and/or its subsidiar

[External] Flink StreamingFileSink not ingesting to S3 when checkpointing is disabled

2019-05-27 Thread Vishal Sharma
Hello everyone, I want to use aws s3 as sink for a data stream in flink. I am using StreamingFileSink class to create a sink. I don't need checkpointing for my job, but when I disable checkpointing, data is no longer written to S3. case 1 : checkpointing enabled When checkpointing is enabled,