Re: Running flink in a Local Execution Environment for Production Workloads

2020-10-30 Thread David Anderson
Another factor to be aware of is that there's no cluster configuration file to load (the mini-cluster doesn't know where to find flink-conf.yaml), and by default you won't have a REST API endpoint or web UI available. But you can do the configuration in the code, and/or provide configuration on

Re: Running flink in a Local Execution Environment for Production Workloads

2020-10-29 Thread Matthias Pohl
Hi Joseph, thanks for reaching out to us. There shouldn't be any downsides other than the one you already mentioned as far as I know. Best, Matthias On Fri, Oct 23, 2020 at 1:27 PM Joseph Lorenzini wrote: > Hi all, > > > > I plan to run flink jobs as docker containers in a AWS Elastic

Running flink in a Local Execution Environment for Production Workloads

2020-10-23 Thread Joseph Lorenzini
Hi all,   I plan to run flink jobs as docker containers in a AWS Elastic Container Service. I will have checkpointing enabled where state is stored in a s3 bucket. Each deployment will run in a per-job mode.  Are there any non-obvious downsides to running these jobs with a local