[
https://issues.apache.org/jira/browse/BEAM-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622986#comment-16622986
]
Byambasuren Ganbaatar commented on BEAM-5305:
---------------------------------------------
[~jbonofre]Could respond this issue here? thanks.
> Timeout handling in JDBCIO
> --------------------------
>
> Key: BEAM-5305
> URL: https://issues.apache.org/jira/browse/BEAM-5305
> Project: Beam
> Issue Type: Bug
> Components: io-java-jdbc
> Affects Versions: 2.5.0
> Reporter: Byambasuren Ganbaatar
> Assignee: Jean-Baptiste Onofré
> Priority: Critical
> Labels: JdbcIO, oracle
>
> Hello, we have a batch pipeline that is reading data from Oracle, writing it
> to BigQuery.
> Using JdbcIO, BigQueryIO and Oracle driver [3].
> We run this job 2 times in a day, the pipeline usually works fine, it
> completed within 10 minutes.
> But sometimes we have a [1] warning from this job, it never finished until we
> cancelling it.
> (sometimes means 4 times in last month)
> Even we set timeout configuration for our code [2], warning log says [1]
> Processing stuck in step Read JDBC.
>
> Would you explain and correct us this kind of problem?
> We already contacted Google Support Team, they said our configuration code is
> looked fine them. Then they instructed us to ask here.
> Thanks,
> Bya
> [1] Warning Log
> {code:java}
> Processing stuck in step Read JDBC/JdbcIO.Read/JdbcIO.ReadAll/ParDo(Read) for
> at least 13h40m00s without outputting or completing in state process at
> sun.nio.ch.FileDispatcherImpl.read0(Native Method) at
> sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at
> sun.nio.ch.IOUtil.read(IOUtil.java:197) at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at
> oracle.net.nt.TimeoutSocketChannel.read(TimeoutSocketChannel.java:144) at
> oracle.net.ns.NIOHeader.readHeaderBuffer(NIOHeader.java:82) at
> oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:139) at
> oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:101) at
> oracle.net.ns.NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:80)
> at
> oracle.jdbc.driver.T4CMAREngineNIO.prepareForReading(T4CMAREngineNIO.java:98)
> at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:534)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:485) at
> oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252) at
> oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612) at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:226)
> at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:59)
> at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:747)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082)
> at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3780)
> at
> oracle.jdbc.driver.T4CPreparedStatement.executeInternal(T4CPreparedStatement.java:1343)
> at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3822)
> at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1165)
> at
> org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
> at
> org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
> at
> org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
> at
> org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
> at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.processElement(JdbcIO.java:601)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeProcessElement(Unknown
> Source)
> {code}
>
> [2] Jdbc configure
> {code:java}
> import org.apache.beam.sdk.io.jdbc.JdbcIO;
> import oracle.jdbc.driver.OracleConnection;
> JdbcIO.DataSourceConfiguration dataSource = JdbcIO
>
> .DataSourceConfiguration.create("oracle.jdbc.driver.OracleDriver", oracleUrl)
> .withUsername(oracleUser)
> .withPassword(oraclePassword)
> .withConnectionProperties(String.format("%s=%s",
>
> OracleConnection.CONNECTION_PROPERTY_THIN_NET_CONNECT_TIMEOUT, 10000))
> .withConnectionProperties(String.format("%s=%s",
>
> OracleConnection.CONNECTION_PROPERTY_THIN_READ_TIMEOUT, 600000))
> .withConnectionProperties(String.format("%s=%s",
>
> OracleConnection.CONNECTION_PROPERTY_TIMEZONE_AS_REGION, false))
> .withConnectionProperties(String.format("%s=%s",
> OracleConnection.CONNECTION_PROPERTY_NET_KEEPALIVE,
> false))
> ;
> pipeline.apply(JdbcIO.<TableRow>read()
> .withDataSourceConfiguration(dataSource)
> .withQuery(readQuery)
> .withCoder(TableRowJsonCoder.of())
> .withRowMapper(new TableRowRowMapper())
> );
> {code}
>
> [3] Oracle DB driver: ojdbc8 12.2.0.1
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
