[
https://issues.apache.org/jira/browse/FLINK-30960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685948#comment-17685948
]
Bo Shen edited comment on FLINK-30960 at 2/8/23 3:27 PM:
---------------------------------------------------------
[~martijnvisser] OK, will keep that in mind.
By looking at the code in the main branch of
[https://github.com/apache/flink-connector-jdbc] "JdbcOutputFormat.java", the
code concerned are mostly the same as the version in v1.13.6
The same issue is reproduced using Flink 1.16. I have uploaded a full run log.
was (Author: JIRAUSER298909):
[~martijnvisser] OK, will keep that in mind.
By looking at the code in the main branch of
[https://github.com/apache/flink-connector-jdbc] "JdbcOutputFormat.java", the
code concerned are mostly the same on main branch in the JDBC connector
repository.
The same issue is reproduced using Flink 1.16. I have uploaded a full run log.
> OutOfMemory error using jdbc sink
> ---------------------------------
>
> Key: FLINK-30960
> URL: https://issues.apache.org/jira/browse/FLINK-30960
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC
> Affects Versions: 1.16.0
> Environment: Using SQL-Client + TableAPI
> Processing Kafka messages to MySQL databases.
> Reporter: Bo Shen
> Priority: Major
> Attachments: heap-memory-metrics.png,
> image-2023-02-08-16-58-27-765.png, taskmanager.log
>
>
> Here in production I have an environment consisting of a kafka source, to
> which third-party providers push a few thousand messages every hour, and I
> use flink jdbc sink to store those messages into a mysql database.
> Recently there are a few taskmanager process crashes. My investigation
> suggests that the cause boils down to the way exceptions are handled in jdbc
> batched mode.
>
> When writing to JDBC failed in batched mode due to some error like
> DataTuncation, the exception is stored in field "flushException" waiting to
> be processed by the task main thread.
> This exception is only checked on the next call to "writeRecords". In my
> case, if the exception happens to occur when processing the last batch of
> records, and there are no further record comming from source for the next
> hour, the flushing thread simply repeatly throws a new RuntimeException
> wrapping the last "flushException" as cause. The new RuntimeException is
> stored as the new "flushException".
> Hence, every time the RuntimeException is thrown, the exception hierachy gets
> bigger, and eventually before the exception is processed on the main thread,
> the jvm runs out of memory, which causes an unexpected process crash.
> !image-2023-02-08-16-58-27-765.png!
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
