Hi Konstantinos, This sounds like a useful extension to me. Would you like to create a Jira issue and contribute the improvement?
In the meantime, you can just fork the code of JDBCInputFormat and adjust it to your needs. Best, Fabian Am Mo., 15. Apr. 2019 um 08:53 Uhr schrieb Papadopoulos, Konstantinos < konstantinos.papadopou...@iriworldwide.com>: > Hi Rong, > > > > We have already tried to set the fetch size with no success. According to > PG documentation we have to set both configuration parameters (i.e., > auto-commit to false and limit fetch) to achieve our purpose. > > > > Thanks, > > Konstantinos > > > > *From:* Rong Rong <walter...@gmail.com> > *Sent:* Παρασκευή, 12 Απριλίου 2019 6:50 μμ > *To:* Papadopoulos, Konstantinos > <konstantinos.papadopou...@iriworldwide.com> > *Cc:* user <user@flink.apache.org> > *Subject:* Re: Flink JDBC: Disable auto-commit mode > > > > Hi Konstantinos, > > > > Seems like setting for auto commit is not directly possible in the current > JDBCInputFormatBuilder. > > However there's a way to specify the fetch size [1] for your DB > round-trip, doesn't that resolve your issue? > > > > Similarly in JDBCOutputFormat, a batching mode was also used to stash > upload rows before flushing to DB. > > > > -- > > Rong > > > > [1] > https://docs.oracle.com/cd/E18283_01/java.112/e16548/resltset.htm#insertedID4 > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fcd%2FE18283_01%2Fjava.112%2Fe16548%2Fresltset.htm%23insertedID4&data=02%7C01%7C%7C44d9fa279b674725483008d6bf5e7b3e%7C43728c2044474b27ac2e4bdabb3c0121%7C0%7C1%7C636906809861700178&sdata=XE%2FjZhh%2F8dcBe679%2FccdGq0rutlAiXHBHTj45eoXM70%3D&reserved=0> > > > > On Fri, Apr 12, 2019 at 6:23 AM Papadopoulos, Konstantinos < > konstantinos.papadopou...@iriworldwide.com> wrote: > > Hi all, > > We are facing an issue when trying to integrate PostgreSQL with Flink > JDBC. When you establish a connection to the PostgreSQL database, it is in > auto-commit mode. It means that each SQL statement is treated as a > transaction and is automatically committed, but this functionality results > in unexpected behavior (e.g., out-of-memory errors) when executed for large > result sets. In order to bypass such issues, we must disable the > auto-commit mode. To do this, in a simple Java application, we call the > setAutoCommit() method of the Connection object. > > So, my question is: How can we achieve this by using JDBCInputFormat of > Flink? > > Thanks in advance, > > Konstantinos > >
