Re: Schema Evolution on Dynamic Schema

2019-03-09 Thread Rong Rong
Hi Shahar, >From my understanding, if you use "groupby" withAggregateFunctions, they save the accumulators to SQL internal states: which are invariant from your input schema. Based on what you described I think that's why it is fine for recovering from existing state. I think one confusion you mig

Re: StandAlone job on k8s fails with "Unknown method truncate" on restore

2019-03-09 Thread Vishal Santoshi
We reverted back to BucketingSink and that works as expected. In conclusion RollingFileSink needs hadoop 2.7 for for hadoop File System. On Sat, Feb 23, 2019 at 2:30 PM Vishal Santoshi wrote: > Any one ? I am sure there are hadoop 2.6 integrations with 1.7.1 OR I am > overlooking something... >

Mapping of CLI arguments to flink-conf.yaml

2019-03-09 Thread Sergey Zhemzhitsky
Hi there I mostly use flink for stream-processing jobs and deploy them as independent ones [1]. So I'd like every job to have its own configuration and don't like to remember the values of all the command line arguments used to start the job every time. In spark for most of the CLI arguments ther