customize class path using on yarn

2016-07-24 Thread Dong-iL, Kim
I’m sorry. I’ve asked but there is no reply. I wanna add config file to class path.(eg. hibernate.cfg.xml) I’ve put it in FLINK_LIB_DIR. It was shipped but not in class path. I’ve used the option(-C, —class path) of flink common but not work. what shall I do?

Supporting Multi-tenancy in a Flink

2016-07-24 Thread Aparup Banerjee (apbanerj)
We are building a Stream processing system using Apache beam on top of Flink using the Flink Runner. Our pipelines take Kafka streams as sources , and can write to multiple sinks. The system needs to be tenant aware. Tenants can share same Kafka topic. Tenants can write their own pipelines. We

Re: State in external db (dynamodb)

2016-07-24 Thread Chen Qin
> On Jul 22, 2016, at 2:54 AM, Josh wrote: > > Hi all, > > >(1) Only write to the DB upon a checkpoint, at which point it is known that > >no replay of that data will occur any more. Values from partially successful > >writes will be overwritten >with correct value. I