Fwd: Global connection open and close

2024-03-22 Thread Alex Cruise
[previous didn't cc list, sorry for dupes] The classic connection pool pattern, where expensive connections are created relatively few times and used by lots of transient short-lived tasks, each of which borrows a connection from the pool and returns it when done, would still be usable here, but a

Re: Global connection open and close

2024-03-22 Thread Alex Cruise
The classic connection pool pattern, where expensive connections are created relatively few times and used by lots of transient short-lived tasks, each of which borrows a connection from the pool and returns it when done, would still be usable here, but as Péter points out, you can't rely on a sing

Re: Global connection open and close

2024-03-21 Thread Péter Váry
Hi Jacob, Flink jobs, tasks typically run on multiple nodes/servers. This means that it is not possible to have a connection shared on job level. You can read about the architecture in more detail in the docs. [1] I hope this helps, Péter [1] - https://nightlies.apache.org/flink/flink-docs-rele

Global connection open and close

2024-03-21 Thread Jacob Rollings
Hello, Is there a way in Flink to instantiate or open connections (to cache/db) at global level, so that it can be reused across many process functions rather than doing it in each operator's open()?Along with opening, also wanted to know if there is a way to close them at job level stop, such tha