Stream processing into single sink to multiple DB Schemas

2021-06-06 Thread Tamir Sagi
Hey Community Assuming there are 3 groups, A, B, C Each group represents a set of data about employees and salaries. Group A ( 0-20K $) Group B (20K$ - 50K$) Group C ( > 50K$) Is it possible to process stream data from single source containing information about employees and salaries and split

Re: Stream processing into single sink to multiple DB Schemas

2021-06-07 Thread Nicolaus Weidner
Hi Tamir, I assume you want to use the Jdbc connector? You can use three filters on your input stream to separate it into three separate streams, then add a sink to each of those (see e.g. [1]). Then you can have a different SQL statement for each of the three sinks. If you specify the driver name

Re: Stream processing into single sink to multiple DB Schemas

2021-06-07 Thread Maciej Obuchowski
Hey, We had similar problem, but with 1000s of tables. I've created issue [1] and PR with internally used solution [2], but unfortunately, there seems to be no interest in upstreaming this feature. Thanks, Maciej [1] https://issues.apache.org/jira/browse/FLINK-21643 [2] https://github.com/apache/

Re: Stream processing into single sink to multiple DB Schemas

2021-07-11 Thread Tamir Sagi
; user@flink.apache.org Subject: Re: Stream processing into single sink to multiple DB Schemas EXTERNAL EMAIL Hey, We had similar problem, but with 1000s of tables. I've created issue [1] and PR with internally used solution [2], but unfortunately, there seems to be no interest in upstr