[
https://issues.apache.org/jira/browse/BEAM-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839302#comment-16839302
]
Ismaël Mejía commented on BEAM-7230:
------------------------------------
[~brachi_packter] the fix was merged, now it does not try to close the
datasource so it can be shared by multiple threads in he same VM. This is the
core of the issue. However the SNAPSHOTS are generated every day so you have to
wait until tomorrow to have the updated JAR.
You can continue using Hikari if you prefer but you have to pay attention that
the DataSource instance you provide in the function is created only once and
shared by all the users of the function by making it static and a singleton,
something like this:
{code}
private static class DataSourceProviderFn extends SerializableFunction<Void,
DataSource> {
private static HikariDataSource ds;
public DataSourceProviderFn(String dataBaseName, String userName, String
project, String region, String instanceName) {
if (ds == null)
{ ds = new HikariDataSource( ... ); }
}
@Override
public DataSource apply(Void input)
{ return ds; }
}
{code}
Note that `PoolableDataSourceProvider` guarantees this I will probably improve
the documentation on this in subsequent PR later on.
> Using JdbcIO creates huge amount of connections
> -----------------------------------------------
>
> Key: BEAM-7230
> URL: https://issues.apache.org/jira/browse/BEAM-7230
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.11.0
> Reporter: Brachi Packter
> Assignee: Ismaël Mejía
> Priority: Major
>
> I want to write form DataFlow to GCP cloud SQL, I'm using connection pool,
> and still I see huge amount of connections in GCP SQL (4k while I set
> connection pool to 300), and most of them in sleep.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
