So I know specifically, it's the indexing and I put setQueryTimeout. So the job fails. And goes into retry. That's fine.
But just wondering is there a way to pause the stream at a specified time/checkpoint and then resume after a specified time? On Wed, May 4, 2022 at 10:23 AM Martijn Visser <martijnvis...@apache.org> wrote: > Hi John, > > It is generic, but each database has its own dialect implementation > because they all have their differences unfortunately :) > > I wish I knew how I could help you out here. Perhaps some of the JDBC > maintainers could chip in. > > Best regards, > > Martijn > > On Sun, 1 May 2022 at 04:06, John Smith <java.dev....@gmail.com> wrote: > >> Plus in a way isn't the flink-jdbc connector kinda generic? At least the >> older one didn't seem to be server specific. >> >> On Sat, Apr 30, 2022 at 10:04 PM John Smith <java.dev....@gmail.com> >> wrote: >> >>> Hi Martin, is there anything I need to check for? >>> >>> On Tue, Apr 26, 2022 at 9:50 PM John Smith <java.dev....@gmail.com> >>> wrote: >>> >>>> Yeah based off the flink JDBC output format... >>>> >>>> >>>> On Tue, Apr 26, 2022 at 10:05 AM Martijn Visser < >>>> martijnvis...@apache.org> wrote: >>>> >>>>> Hi John, >>>>> >>>>> Have you built your own JDBC MSSQL source or sink or perhaps a CDC >>>>> driver? Because I'm not aware of a Flink Microsoft SQL Server JDBC driver. >>>>> >>>>> Best regards, >>>>> >>>>> Martijn Visser >>>>> https://twitter.com/MartijnVisser82 >>>>> https://github.com/MartijnVisser >>>>> >>>>> >>>>> On Tue, 26 Apr 2022 at 16:01, John Smith <java.dev....@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi running 1.14.4 >>>>>> >>>>>> Logs included: >>>>>> https://www.dropbox.com/s/8zjndt5rzd9o80f/flink-flink-taskexecutor-138-task-0002.log?dl=0 >>>>>> >>>>>> 1- My task managers shut down with: Terminating TaskManagerRunner >>>>>> with exit code 1. >>>>>> 2- It seems to happen at the same time every day. Which leads me to >>>>>> believe it's our database indexing (See below for reasoning of this). >>>>>> 3- Most of our jobs are ETL from Kafka to SQL Server. >>>>>> 4- We see the following exceptions in the logs: >>>>>> - Task 'Sink: jdbc (1/1)#10' did not react to cancelling signal >>>>>> - interrupting; it is stuck for 30 seconds in method: >>>>>> ... com.microsoft.sqlserver.jdbc.TDSChannel ... >>>>>> - Sink: jdbc (1/1)#9 (3aaf6d8a45df6c43198bc8297b42354c) >>>>>> switched from RUNNING to FAILED with failure cause: >>>>>> org.apache.flink.util.FlinkException: Disconnect from JobManager >>>>>> responsible for ... >>>>>> 5- Also seeing this: Failed to close consumer network client with >>>>>> type org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient >>>>>> java.lang.NoClassDefFoundError: >>>>>> org/apache/kafka/common/network/Selector$CloseMode >>>>>> >>>>>> So what I'm guessing is happening is the indexing is blocking the job >>>>>> and the task manager cannot cleanly remove the job and finally after a >>>>>> while it decides to shut down completely? >>>>>> >>>>>> Is there a way to pause the stream and restart at a later time >>>>>> knowing that this happens always at the same wall clock time? Or maybe >>>>>> allow the JDBC to cleanly shutdown with a timeout? >>>>>> >>>>>> >>>>>>
