[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784541#comment-17784541 ] Jorick Caberio commented on FLINK-33251: I was able to replicate this issue on my Mac Mini M2 running Flink 1.17.1 {code} $ uname -a Darwin Joricks-Mini.bbrouter 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 {code} {code:sql} CREATE TABLE total_amount_table ( `transaction_id` STRING, `transaction_datetime` STRING, `amount` DOUBLE ) WITH ( 'connector' = 'kafka', 'topic' = 'total_amount_table', 'properties.bootstrap.servers' = 'localhost:9092', 'properties.group.id' = 'total_amount_table', 'scan.startup.mode' = 'latest-offset', 'format' = 'json' ); {code} {code:sql} INSERT INTO total_amount_table VALUES ('txn1', '2023-11-09T16:10:25Z', 125125.125); {code} {code:sql} SELECT * FROM total_amount_table; {code} {code:sql} [ERROR] Could not execute SQL statement. Reason: org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: connection timed out: localhost/127.0.0.1:50358 {code} > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784715#comment-17784715 ] Martijn Visser commented on FLINK-33251: [~jcaberio] Are you also on a VPN by any chance? > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784905#comment-17784905 ] Jorick Caberio commented on FLINK-33251: No > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17785160#comment-17785160 ] Jorick Caberio commented on FLINK-33251: FWIW I tried replicating this in 1.16 and 1.18. 1.16 works as expected while 1.18 has the same issue. > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17785738#comment-17785738 ] Shengkai Fang commented on FLINK-33251: --- I can not reproduce this problem in my local machine with the steps above. {code:java} Darwin B-QB5MMD6M-0305.local 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT 2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64 {code} Could you modify the log level of FLINK_HOME/conf/log4j-cli.properties to TRACE and then upload the log file? {code:java} rootLogger.level = TRACE {code} > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840061#comment-17840061 ] Robert Metzger commented on FLINK-33251: I'm having this problem with 1.19.0 as well, on a M1 MBP. The problem is tricky to reproduce (e.g. it doesn't happen all the times) {code} 2024-04-23 12:51:14,317 DEBUG org.apache.flink.runtime.rest.RestClient [] - Shutting down rest endpoint. 2024-04-23 12:51:14,317 DEBUG org.apache.flink.shaded.netty4.io.netty.buffer.PoolThreadCache [] - Freed 2 thread-local buffer(s) from thread: flink-rest-client-netty-thread-1 2024-04-23 12:51:14,318 DEBUG org.apache.flink.runtime.rest.RestClient [] - Rest endpoint shutdown complete. 2024-04-23 12:51:14,318 TRACE org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop [] - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@c1d225b 2024-04-23 12:51:14,318 DEBUG org.apache.flink.runtime.rest.RestClient [] - Rest client endpoint started. 2024-04-23 12:51:14,318 DEBUG org.apache.flink.runtime.rest.RestClient [] - Sending request of class class org.apache.flink.runtime.rest.messages.job.coordination.ClientCoordinationRequestBody to localhost:8081/v1/jobs/5d8d1b8ef7dc49381a3855ae10a18ec5/coordinators/b728d985904d42b0fdd945a9e3253fca 2024-04-23 12:51:14,320 DEBUG org.apache.flink.runtime.rest.RestClient [] - Received response {"serializedCoordinationResult":"rO0ABXNyAExvcmcuYXBhY2hlLmZsaW5rLnN0cmVhbWluZy5hcGkub3BlcmF0b3JzLmNvbGxlY3QuQ29sbGVjdENvb3JkaW5hdGlvblJlc3BvbnNlAAECAANKABZsYXN0Q2hlY2twb2ludGVkT2Zmc2V0TAARc2VyaWFsaXplZFJlc3VsdHN0ABBMamF2YS91dGlsL0xpc3Q7TAAHdmVyc2lvbnQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAABzcgATamF2YS51dGlsLkFycmF5TGlzdHiB0h2Zx2GdAwABSQAEc2l6ZXhwAHcEAHh0ACQ3MzgyMjA5Ni0wODE2LTQ5NTMtODA4NC1kMDJhZTg0ZjNhNWU="}. 2024-04-23 12:51:14,321 DEBUG org.apache.flink.runtime.rest.RestClient [] - Shutting down rest endpoint. 2024-04-23 12:51:14,321 DEBUG org.apache.flink.shaded.netty4.io.netty.buffer.PoolThreadCache [] - Freed 3 thread-local buffer(s) from thread: flink-rest-client-netty-thread-1 2024-04-23 12:51:14,321 DEBUG org.apache.flink.runtime.rest.RestClient [] - Rest endpoint shutdown complete. 2024-04-23 12:51:14,390 DEBUG org.apache.flink.runtime.rest.RestClient [] - Sending request of class class org.apache.flink.runtime.rest.messages.EmptyRequestBody to localhost:62113/v2/sessions/88b98272-be33-4303-a649-942acd213e84/heartbeat 2024-04-23 12:51:14,391 TRACE org.apache.flink.runtime.rest.FileUploadHandler [] - Received request. URL:/v2/sessions/88b98272-be33-4303-a649-942acd213e84/heartbeat Method:POST 2024-04-23 12:51:14,391 TRACE org.apache.flink.table.gateway.rest.handler.session.TriggerSessionHeartbeatHandler [] - Received request /v2/sessions/88b98272-be33-4303-a649-942acd213e84/heartbeat. 2024-04-23 12:51:14,391 TRACE org.apache.flink.table.gateway.rest.handler.session.TriggerSessionHeartbeatHandler [] - Starting request processing. 2024-04-23 12:51:14,391 DEBUG org.apache.flink.runtime.rest.RestClient [] - Received response {}. 2024-04-23 12:51:14,425 TRACE org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop [] - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@1436813 2024-04-23 12:51:14,425 DEBUG org.apache.flink.runtime.rest.RestClient [] - Rest client endpoint started. 2024-04-23 12:51:14,426 DEBUG org.apache.flink.runtime.rest.RestClient [] - Sending request of class class org.apache.flink.runtime.rest.messages.EmptyRequestBody to localhost:8081/v1/jobs/5d8d1b8ef7dc49381a3855ae10a18ec5/status 2024-04-23 12:51:14,434 DEBUG org.apache.flink.runtime.rest.RestClient [] - Received response {"status":"RUNNING"}. 2024-04-23 12:51:14,435 DEBUG org.apache.flink.runtime.rest.RestClient [] - Shutting down rest endpoint. 2024-04-23 12:51:14,435 DEBUG org.apache.flink.shaded.netty4.io.netty.buffer.PoolThreadCache [] - Freed 2 thread-local buffer(s) from thread: flink-rest-client-netty-thread-1 2024-04-23 12:51:14,435 DEBUG org.apache.flink.runtime.rest.RestClient [] - Rest endpoint shutdown complete. 2024-04-23 12:51:14,435 TRACE org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop [] - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@64f0cb77 2024-04-23 12:51:14,435 DEBUG org.apache.flink.runtime.rest.RestClient [] - Rest client endpoint started. 2024-04-23 12:51:14,436 DEBUG org.apache.flink.runtime.rest.RestClient [] - Sending request of class class org.apache.flink.runtime.rest.messages.job.coordination.Clie
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847791#comment-17847791 ] Keith Lee commented on FLINK-33251: --- I've also experience similar issue so adding my observation here. Similar to Robert's observation, it does not happen all the time. Exception seen: ``` [ERROR] Could not execute SQL statement. Reason: org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: connection timed out: localhost/127.0.0.1:54658 ``` Flink environment: ``` Flink SQL Client Version: 1.18.1 Command used to start SQL Client: bin/sql-client.sh --init sqlClientDemoInit Flink Version: 1.18.1 Command used to start JM: bin/jobmanager.sh start-foreground Command used to start TM: bin/taskmanager.sh start-foreground ``` ``` Model Name: MacBook Pro Model Identifier: MacBookPro18,1 Model Number: Z14V000GDB/A Chip: Apple M1 Pro Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 32 GB System Firmware Version: 10151.101.3 OS Loader Version: 8422.141.2.700.1 Serial Number (system): DW1GQ06D7F Hardware UUID: E41F9063-CF2B-539A-89D4-05E36577A809 Provisioning UDID: 6000-000E69EC3602401E Activation Lock Status: Disabled ``` > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848610#comment-17848610 ] Keith Wall commented on FLINK-33251: I see this issue too on flink-1.19.0. uname -a Darwin Oslo.local 22.6.0 Darwin Kernel Version 22.6.0: Mon Feb 19 19:48:53 PST 2024; root:xnu-8796.141.3.704.6~1/RELEASE_X86_64 x86_64 > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849440#comment-17849440 ] Keith Lee commented on FLINK-33251: --- It seems that reproducing this on M1 Mac is not difficult, also can confirm that Amazon Linux 2023 does NOT run into same issue. To reproduce, just start sqlClient and run the following. You may or may not get timeout when the DQL initialises, however, if you wait long enough while viewing the results, the issue occurs eventually. ``` CREATE TABLE Orders (order_number BIGINT,price DECIMAL(32,2), buyer ROW, order_time TIMESTAMP(3)) WITH ('connector' = 'datagen'); SELECT * FROM Orders; ``` > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849467#comment-17849467 ] Keith Lee commented on FLINK-33251: --- Adding IntelliJ scratch file that can reliably reproduce: [^reproduce_FLINK-33251.java] To use this, your run configuration 1. should use flink-sql-client as classpath module. 2. should define environment variable: FLINK_CONF_DIR=//flink-1.18.1/conf/ > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip, reproduce_FLINK-33251.java > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849961#comment-17849961 ] Keith Lee commented on FLINK-33251: --- The timeout could come from any of: 1. SQL Gateway not responding/in time to connection 2. ExecutorImpl 3. Flink's RESTClient implementation issue 4. Netty's Bootstrap (unlikely as it is). I've ran the following cURL commands to rule out SQL Gateway and can confirm that connection timeouts are NOT seen. {quote}{{SESSION_HANDLE=$(curl --request POST http://localhost:8083/v1/sessions | jq -r .sessionHandle)}} {{OPERATION_HANDLE=$(curl --request POST http://localhost:8083/v1/sessions/${SESSION_HANDLE}/statements/ --data '\{"statement": "CREATE TABLE Orders (order_number BIGINT, order_time TIMESTAMP(3)) WITH ('"'connector'"' = '"'datagen'"');"}' | jq -r .operationHandle)}} {{{}curl --request GET http://localhost:8083/v1/sessions/${SESSION_HANDLE}/operations/${OPERATION_HANDLE}/result/0 | jq .{}}}{{{}OPERATION_HANDLE=$(curl --request POST http://localhost:8083/v1/sessions/${SESSION_HANDLE}/statements/ --data '\{"statement": "SELECT * FROM Orders;"}' | jq -r .operationHandle){}}} {{{}NEXT_RESULT_URI=$(curl --request GET http://localhost:8083/v1/sessions/${SESSION_HANDLE}/operations/${OPERATION_HANDLE}/result/0 | jq -r .nextResultUri){}}}{{{}while true; do echo $NEXT_RESULT_URI; NEXT_RESULT_URI_TEMP=$(curl --request GET http://localhost:8083/${NEXT_RESULT_URI} | jq -r .nextResultUri); NEXT_RESULT_URI=$NEXT_RESULT_URI_TEMP; done{}}}{{{}{}}} {quote} {{So I think the bug is in either ExecutorImpl or RestClient or Netty's bootstrap class.}} {{One interesting observation I have is the }}{{{}fetchResultWithInterval{}}}{{{}: [https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/ExecutorImpl.java#L469-L471]{}}} {quote}{{ private FetchResultsResponseBody getFetchResultResponse(}} {{ OperationHandle operationHandle,}} {{ long token,}} {{ boolean fetchResultWithInterval,}} {{ Function interruptedExceptionHandler) {}} {{ try {}} {{ if (fetchResultWithInterval) {}} {{ Thread.sleep(100);}} {{ }}} {quote} {{[~fsk119] can you elaborate on why the interval is necessary? As the other observation that I have is that the ConnectTimeoutException seems to correlate with how quickly we call getFetchResultResponse.}} > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip, reproduce_FLINK-33251.java > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute st
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774431#comment-17774431 ] Sergey Nuyanzin commented on FLINK-33251: - I followed the steps on linux (Fedora 38) with jdk8 for flink master (1.19-SNAPSHOT) and it works I tend to think that this is your env related issues like Mac M1 or something like that > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33251) SQL Client query execution aborts after a few seconds: ConnectTimeoutException
[ https://issues.apache.org/jira/browse/FLINK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774475#comment-17774475 ] Martijn Visser commented on FLINK-33251: [~rmoff] Are you on VPN by any chance? > SQL Client query execution aborts after a few seconds: ConnectTimeoutException > -- > > Key: FLINK-33251 > URL: https://issues.apache.org/jira/browse/FLINK-33251 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.18.0, 1.17.1 > Environment: Macbook Pro > Apple M1 Max > > {code:java} > $ uname -a > Darwin asgard08 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT > 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64 > {code} > {code:bash} > $ java --version > openjdk 11.0.20.1 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > $ mvn --version > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) > Maven home: /opt/homebrew/Cellar/maven/3.9.5/libexec > Java version: 11.0.20.1, vendor: Homebrew, runtime: > /opt/homebrew/Cellar/openjdk@11/11.0.20.1/libexec/openjdk.jdk/Contents/Home > Default locale: en_GB, platform encoding: UTF-8 > OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac" > {code} >Reporter: Robin Moffatt >Priority: Major > Attachments: log.zip > > > If I run a streaming query from an unbounded connector from the SQL Client, > it bombs out after ~15 seconds. > {code:java} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:52596 > {code} > This *doesn't* happen on 1.16.2. It *does* happen on *1.17.1* and *1.18* that > I have just built locally (git repo hash `9b837727b6d`). > The corresponding task's status in the Web UI shows as `CANCELED`. > --- > h2. To reproduce > Launch local cluster and SQL client > {code} > ➜ flink-1.18-SNAPSHOT ./bin/start-cluster.sh > Starting cluster. > Starting standalonesession daemon on host asgard08. > Starting taskexecutor daemon on host asgard08. > ➜ flink-1.18-SNAPSHOT ./bin/sql-client.sh > […] > Flink SQL> > {code} > Set streaming mode and result mode > {code:sql} > Flink SQL> SET 'execution.runtime-mode' = 'STREAMING'; > [INFO] Execute statement succeed. > Flink SQL> SET 'sql-client.execution.result-mode' = 'changelog'; > [INFO] Execute statement succeed. > {code} > Define a table to read data from CSV files in a folder > {code:sql} > CREATE TABLE firewall ( > event_time STRING, > source_ip STRING, > dest_ipSTRING, > source_prt INT, > dest_prt INT > ) WITH ( > 'connector' = 'filesystem', > 'path' = 'file:///tmp/firewall/', > 'format' = 'csv', > 'source.monitor-interval' = '1' -- unclear from the docs what the unit is > here > ); > {code} > Create a CSV file to read in > {code:bash} > $ mkdir /tmp/firewall > $ cat > /tmp/firewall/data.csv < 2018-05-11 00:19:34,151.35.34.162,125.26.20.222,2014,68 > 2018-05-11 22:20:43,114.24.126.190,21.68.21.69,379,1619 > EOF > {code} > Run a streaming query > {code} > SELECT * FROM firewall; > {code} > You will get results showing (and if you add another data file it will show > up) - but after ~30 seconds the query aborts and throws an error back to the > user at the SQL Client prompt > {code} > [ERROR] Could not execute SQL statement. Reason: > org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: localhost/127.0.0.1:58470 > Flink SQL> > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)