I researched a bit more and another suggested solution is to build a custom source function that somehow waits for each operator to load it's configuration which is infact set in the open method of the source itself. I'm not sure if that's a good idea as that just exposes entire job configuration to an operator.
Can we leverage watermarks/idle sources somehow? Basically set the timestamp of configuration stream to a very low number at the start and then force it to be read before data from other sources start flowing in. As configurations aren't going to change frequently we can idle these sources. 1. Is the above approach even possible? 2. Can an idle source resume once configuration changes? A rough sketch of timestamp assignment, re-activating an idle source would help! -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
