[jira] [Created] (FLINK-36613) RescaleCheckpointManuallyITCase.testCheckpointRescalingInKeyedState fails on AZP
Sergey Nuyanzin created FLINK-36613:
---
Summary:
RescaleCheckpointManuallyITCase.testCheckpointRescalingInKeyedState fails on AZP
Key: FLINK-36613
URL: https://issues.apache.org/jira/browse/FLINK-36613
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing, Tests
Affects Versions: 2.0-preview
Reporter: Sergey Nuyanzin
{{RescaleCheckpointManuallyITCase.testCheckpointRescalingInKeyedState}} fails
like
{noformat}
Oct 28 04:42:03 04:42:03.884 [ERROR]
org.apache.flink.test.checkpointing.RescaleCheckpointManuallyITCase.testCheckpointRescalingInKeyedState
-- Time elapsed: 6.107 s <<< FAILURE!
Oct 28 04:42:03 java.lang.AssertionError: expected:<[(0,24000), (1,22500),
(0,34500), (1,33000), (0,21000), (0,45000), (2,31500), (2,42000), (1,6000),
(0,28500), (0,52500), (2,15000), (1,3000), (1,51000), (0,1500), (0,49500),
(2,12000), (2,6), (0,36000), (1,10500), (1,58500), (0,46500), (0,9000),
(0,57000), (2,19500), (2,43500), (1,7500), (1,55500), (2,3), (1,18000),
(0,54000), (2,40500), (1,4500), (0,16500), (2,27000), (1,39000), (2,13500),
(1,25500), (0,37500), (0,61500), (2,0), (2,48000)]> but was:<[(1,22500),
(0,34500), (1,33000), (0,45000), (0,21000), (2,31500), (0,28500), (0,52500),
(2,15000), (1,3000), (1,51000), (0,49500), (0,1500), (1,58500), (1,10500),
(0,46500), (0,9000), (0,57000), (2,43500), (2,19500), (1,7500), (1,55500),
(2,40500), (1,4500), (0,16500), (2,27000), (1,39000), (2,13500), (1,25500),
(0,37500), (0,61500)]>
Oct 28 04:42:03 at org.junit.Assert.fail(Assert.java:89)
Oct 28 04:42:03 at org.junit.Assert.failNotEquals(Assert.java:835)
Oct 28 04:42:03 at org.junit.Assert.assertEquals(Assert.java:120)
Oct 28 04:42:03 at org.junit.Assert.assertEquals(Assert.java:146)
Oct 28 04:42:03 at
org.apache.flink.test.checkpointing.RescaleCheckpointManuallyITCase.restoreAndAssert(RescaleCheckpointManuallyITCase.java:216)
Oct 28 04:42:03 at
org.apache.flink.test.checkpointing.RescaleCheckpointManuallyITCase.testCheckpointRescalingKeyedState(RescaleCheckpointManuallyITCase.java:138)
Oct 28 04:42:03 at
org.apache.flink.test.checkpointing.RescaleCheckpointManuallyITCase.testCheckpointRescalingInKeyedState(RescaleCheckpointManuallyITCase.java:111)
Oct 28 04:42:03 at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
Oct 28 04:42:03 at
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
{noformat}
for instance
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=63345&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba
the issue is floating however could be reproduced locally after several
repetitions like
{code}
cd flink-tests
for i in $(seq 1 100); do ../mvnw -Dtest=RescaleCheckpointManuallyITCase test
|| break ; done
{code}
there is a similar issue FLINK-32483 however it seems fixed (not closed for
some reason and not related)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36606) TimestampITCase generates test artifacts outside of tmp folder
Sergey Nuyanzin created FLINK-36606:
---
Summary: TimestampITCase generates test artifacts outside of tmp
folder
Key: FLINK-36606
URL: https://issues.apache.org/jira/browse/FLINK-36606
Project: Flink
Issue Type: Technical Debt
Components: Tests
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
steps to reproduce
{noformat}
cd flink-tests && ../mvnw -Dtest=TimestampITCase test && git status
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36470) Bump checkstyle to 10+
Sergey Nuyanzin created FLINK-36470: --- Summary: Bump checkstyle to 10+ Key: FLINK-36470 URL: https://issues.apache.org/jira/browse/FLINK-36470 Project: Flink Issue Type: Technical Debt Components: Build System Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Since java was bumped to 11 checkstyle could be bumped to 10+ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36417) STATE_TTL doesn't work with WatermarkAssigner
Sergey Nuyanzin created FLINK-36417:
---
Summary: STATE_TTL doesn't work with WatermarkAssigner
Key: FLINK-36417
URL: https://issues.apache.org/jira/browse/FLINK-36417
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
A query to reproduce
{code:sql}
SELECT /*+ STATE_TTL('st'='1d', 'vt' = '3d') */ st.* FROM SimpleTable st
LEFT JOIN(SELECT DISTINCT b FROM VirtualTable) vt
ON st.b = vt.b WHERE vt.b IS NOT NULL
{code}
It fails with not being able to find table with name {{st}}.
The reason is that if we look into a plan there is WatermarkAssigner which
doesn't support hints
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36415) ProjectWatermarkAssignerTransposeRule can generate extra casts which do not make sense
Sergey Nuyanzin created FLINK-36415:
---
Summary: ProjectWatermarkAssignerTransposeRule can generate extra
casts which do not make sense
Key: FLINK-36415
URL: https://issues.apache.org/jira/browse/FLINK-36415
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
For instance
if there is a table containing deeply nested array and rows like this
{code:sql}
`chart` row<`result` array not null>
not null> not null>
{code}
the optimized plan will look like
{noformat}
LogicalProject(EXPR$0=[CAST(CAST(CAST(CAST(ITEM($0.result,
1).meta):RecordType:peek_no_expand(VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
NOT NULL symbol)):RecordType:peek_no_expand(VARCHAR(2147483647) CHARACTER SET
"UTF-16LE" NOT NULL symbol).symbol):VARCHAR(2147483647) CHARACTER SET
"UTF-16LE"):VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+- LogicalTableScan(table=[[default_catalog, default_database, ItemTable,
project=[chart], metadata=[]]])
{noformat}
while it should contain only 2 casts (one to cast to Varchar and another for
RecordType:peek_no_expand), to others are useless
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36367) Use RexNormalize in EquivalentExprShuttle
Sergey Nuyanzin created FLINK-36367: --- Summary: Use RexNormalize in EquivalentExprShuttle Key: FLINK-36367 URL: https://issues.apache.org/jira/browse/FLINK-36367 Project: Flink Issue Type: Bug Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin This is a follow up for the discussions in comments https://github.com/apache/flink/pull/25388#discussion_r1774177868 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36361) Do not use StringBuilder for EquivalentExprShuttle
Sergey Nuyanzin created FLINK-36361:
---
Summary: Do not use StringBuilder for EquivalentExprShuttle
Key: FLINK-36361
URL: https://issues.apache.org/jira/browse/FLINK-36361
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
Currently there is {{EquivalentExprShuttle}} where there is a map of
{{toString}} node to {{RelNode}}. In fact there is no need for toString which
is calculated each time with usage of {{StringBuilder}} under the hood. And
even more we faced one weird case where it consumes memory.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36330) Session Window TVFs with named parameters don't support column expansion
Sergey Nuyanzin created FLINK-36330:
---
Summary: Session Window TVFs with named parameters don't support
column expansion
Key: FLINK-36330
URL: https://issues.apache.org/jira/browse/FLINK-36330
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API, Table SQL / Planner
Reporter: Sergey Nuyanzin
The issue is very similar to FLINK-33169
a query to reproduce
{code:sql}
SELECT t3_s, SUM(t3_i) AS agg
FROM
TABLE(
SESSION(
TABLE t3 PARTITION BY t3_s, DESCRIPTOR(t3_m_virtual), INTERVAL
'1' MINUTE))
GROUP BY t3_s, window_start, window_end
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36266) Insert into as select * behaves incorrect
Sergey Nuyanzin created FLINK-36266: --- Summary: Insert into as select * behaves incorrect Key: FLINK-36266 URL: https://issues.apache.org/jira/browse/FLINK-36266 Project: Flink Issue Type: Bug Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36262) Port fix for CALCITE-5965
Sergey Nuyanzin created FLINK-36262: --- Summary: Port fix for CALCITE-5965 Key: FLINK-36262 URL: https://issues.apache.org/jira/browse/FLINK-36262 Project: Flink Issue Type: Bug Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin In case of tables with deeply nested structures this issue with unnecessary String concatenations in the RexFieldAccess constructor started to play significant role and we faced it already. Since the fix is only in Calcite 1.36.0 it can take a long time before it comes with Calcite upgrade (currently there are 1.33 and 1.34 in ready for review only) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36217) Remove powermock usage
Sergey Nuyanzin created FLINK-36217: --- Summary: Remove powermock usage Key: FLINK-36217 URL: https://issues.apache.org/jira/browse/FLINK-36217 Project: Flink Issue Type: Technical Debt Components: Tests Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Most of the tests are either moved to a different repo like connectors or rewritten in powermock free way. Powermock itself became unmaintained (latest release was in 2020 https://github.com/powermock/powermock/releases/tag/powermock-2.0.9) and latest commit 2 years ago https://github.com/powermock/powermock also there is no support for junit5 (the request to support it and even PR from junit5 maintainers is ready for review since Feb 2023 https://github.com/powermock/powermock/pull/1146) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36191) FsMergingCheckpointStorageLocationTest generates test data not in tmp folder
Sergey Nuyanzin created FLINK-36191:
---
Summary: FsMergingCheckpointStorageLocationTest generates test
data not in tmp folder
Key: FLINK-36191
URL: https://issues.apache.org/jira/browse/FLINK-36191
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing, Tests
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
to reproduce this command could be used
{code:bash}
cd flink-runtime && ./../mvnw -Dtest=FsMergingCheckpointStorageLocationTest
test && git status
{code}
the output will contain something like
{noformat}
Untracked files:
(use "git add ..." to include in what will be committed)
org.junit.rules.TemporaryFolder@3cc1435c/
org.junit.rules.TemporaryFolder@625732/
org.junit.rules.TemporaryFolder@bef2d72/
{noformat}
the reason is that in {{@Before}} there *not* absolute path is in use
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36126) Qualification of current catalog and current database should happen during execution, not parsing
Sergey Nuyanzin created FLINK-36126: --- Summary: Qualification of current catalog and current database should happen during execution, not parsing Key: FLINK-36126 URL: https://issues.apache.org/jira/browse/FLINK-36126 Project: Flink Issue Type: Sub-task Components: Table SQL / API, Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36085) Refeactor operation with LIKE/ILIKE
Sergey Nuyanzin created FLINK-36085:
---
Summary: Refeactor operation with LIKE/ILIKE
Key: FLINK-36085
URL: https://issues.apache.org/jira/browse/FLINK-36085
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
Current implementation has several issues
1. every operation should implement its own support of {{LIKE}}, {{ILIKE}}
while common logic could be extracted and reused
2. Some operation generate invalid SQL for asSummaryString and the format is
different between show operations
so the logic could be generalized for this {{SQL SHOW}} operations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36038) SHOW CREATE VIEW returns invalid query
Sergey Nuyanzin created FLINK-36038:
---
Summary: SHOW CREATE VIEW returns invalid query
Key: FLINK-36038
URL: https://issues.apache.org/jira/browse/FLINK-36038
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affects Versions: 1.20.0
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
especially for views with comments
for instance
1. create view
{code:sql}
CREATE VIEW v1 COMMENT 'test view' AS SELECT 1, 'a';
{code}
2. show create
{code:sql}
SHOW CREATE VIEW v1;
{code}
it returns query
{code:sql}
CREATE VIEW `default_catalog`.`default_database`.`v1`(`EXPR$0`, `EXPR$1`) as
SELECT 1, 'a' COMMENT 'test view'
{code}
now if we try to execute it fails as
{noformat}
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.sql.parser.impl.ParseException: Incorrect syntax near the
keyword 'COMMENT' at line 2, column 15.
Was expecting one of:
"AS" ...
"EXCEPT" ...
{noformat}
the reason is that {{COMMENT}} should be before the query according to syntax
mentioned at
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/#create-view
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36026) Options from OPTIONS hint are not present in compile plan
Sergey Nuyanzin created FLINK-36026:
---
Summary: Options from OPTIONS hint are not present in compile plan
Key: FLINK-36026
URL: https://issues.apache.org/jira/browse/FLINK-36026
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
Could be reproduced with any query containing \{{OPTIONS}} hint
Options are reflected in node's description however not under options section
of compiled plan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-35423) ARRAY_EXCEPT should support set semantics
Sergey Nuyanzin created FLINK-35423:
---
Summary: ARRAY_EXCEPT should support set semantics
Key: FLINK-35423
URL: https://issues.apache.org/jira/browse/FLINK-35423
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Reporter: Sergey Nuyanzin
After a number of discussions e.g. here [1]
It was decided to follow set semantics for {{ARRAY_EXCEPT}} and
{{ARRAY_INTERSECT}}.
It is marked as a blocker since {{ARRAY_EXCEPT}} was added in 1.20 only and has
not been released yet, so it the change should be done before release to avoid
inconsistencies.
[1] https://github.com/apache/flink/pull/24526
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-35094) SinkTestSuiteBase.testScaleDown is hanging for 1.20-SNAPSHOT
Sergey Nuyanzin created FLINK-35094:
---
Summary: SinkTestSuiteBase.testScaleDown is hanging for
1.20-SNAPSHOT
Key: FLINK-35094
URL: https://issues.apache.org/jira/browse/FLINK-35094
Project: Flink
Issue Type: Bug
Components: Connectors / ElasticSearch
Affects Versions: elasticsearch-3.1.0, 1.20.0
Reporter: Sergey Nuyanzin
Currently it is reproduced for elastic search connector
all the ci jobs (for all jdks) against 1.20-SNAPSHOT are hanging on
{noformat}
2024-04-12T05:56:50.6179284Z "main" #1 prio=5 os_prio=0 cpu=18726.96ms
elapsed=2522.03s tid=0x7f670c025a50 nid=0x3c6d waiting on condition
[0x7f6712513000]
2024-04-12T05:56:50.6180667Zjava.lang.Thread.State: TIMED_WAITING (sleeping)
2024-04-12T05:56:50.6181497Zat
java.lang.Thread.sleep(java.base@17.0.10/Native Method)
2024-04-12T05:56:50.6182762Zat
org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:152)
2024-04-12T05:56:50.6184456Zat
org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:145)
2024-04-12T05:56:50.6186346Zat
org.apache.flink.connector.testframe.testsuites.SinkTestSuiteBase.checkResultWithSemantic(SinkTestSuiteBase.java:504)
2024-04-12T05:56:50.6188474Zat
org.apache.flink.connector.testframe.testsuites.SinkTestSuiteBase.restartFromSavepoint(SinkTestSuiteBase.java:327)
2024-04-12T05:56:50.6190145Zat
org.apache.flink.connector.testframe.testsuites.SinkTestSuiteBase.testScaleDown(SinkTestSuiteBase.java:224)
2024-04-12T05:56:50.6191247Zat
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@17.0.10/Native
Method)
2024-04-12T05:56:50.6192806Zat
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@17.0.10/NativeMethodAccessorImpl.java:77)
2024-04-12T05:56:50.6193863Zat
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.10/DelegatingMethodAccessorImpl.java:43)
2024-04-12T05:56:50.6194834Zat
java.lang.reflect.Method.invoke(java.base@17.0.10/Method.java:568)
{noformat}
for 1.17, 1.18, 1.19 there is no such issue and everything is ok
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-35059) Bump org.postgresql:postgresql from 42.5.1 to 42.5.5 in flink-connector-jdbc
Sergey Nuyanzin created FLINK-35059: --- Summary: Bump org.postgresql:postgresql from 42.5.1 to 42.5.5 in flink-connector-jdbc Key: FLINK-35059 URL: https://issues.apache.org/jira/browse/FLINK-35059 Project: Flink Issue Type: Technical Debt Components: Connectors / JDBC Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35057) Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink jdbc connector
Sergey Nuyanzin created FLINK-35057: --- Summary: Bump org.apache.commons:commons-compress from 1.25.0 to 1.26.1 for Flink jdbc connector Key: FLINK-35057 URL: https://issues.apache.org/jira/browse/FLINK-35057 Project: Flink Issue Type: Technical Debt Components: Connectors / JDBC Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34951) Flink-ci-mirror stopped running for commits after 22nd of March
Sergey Nuyanzin created FLINK-34951:
---
Summary: Flink-ci-mirror stopped running for commits after 22nd of
March
Key: FLINK-34951
URL: https://issues.apache.org/jira/browse/FLINK-34951
Project: Flink
Issue Type: Bug
Components: Build System / CI
Reporter: Sergey Nuyanzin
Blocker since it impacts all branches
if we look at {{flink-ci.flink-master-mirror}}
https://dev.azure.com/apache-flink/apache-flink/_build?definitionId=1
we could see that ci was run against commit
https://github.com/flink-ci/flink-mirror/commit/4edafcc8b0b96920036a1afaaa37ae87b77668ed
The problem is that this commit was done on 22nd of March and after that there
were a number of other commits. At the same time there no any ci at
{{flink-ci.flink-master-mirror}} for commits after that
Same check could be done for other branches
and it doesn't run for newer commits
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34950) Disable spotless on Java 21 for connector-shared-utils
Sergey Nuyanzin created FLINK-34950: --- Summary: Disable spotless on Java 21 for connector-shared-utils Key: FLINK-34950 URL: https://issues.apache.org/jira/browse/FLINK-34950 Project: Flink Issue Type: Bug Components: Connectors / Parent Affects Versions: connector-parent-1.1.0 Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin after https://github.com/apache/flink-connector-shared-utils/pull/19 spotless was stopped being skipped for java17+ in parent pom however we still need to skip it for java21+ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34942) Support Flink 1.19, 1.20-SNAPSHOT for OpenSearch connector
Sergey Nuyanzin created FLINK-34942: --- Summary: Support Flink 1.19, 1.20-SNAPSHOT for OpenSearch connector Key: FLINK-34942 URL: https://issues.apache.org/jira/browse/FLINK-34942 Project: Flink Issue Type: Bug Components: Connectors / Opensearch Affects Versions: 3.1.0 Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Currently it fails with similar issue as FLINK-33493 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34941) Cannot convert org.apache.flink.streaming.api.CheckpointingMode to org.apache.flink.core.execution.CheckpointingMode
Sergey Nuyanzin created FLINK-34941:
---
Summary: Cannot convert
org.apache.flink.streaming.api.CheckpointingMode to
org.apache.flink.core.execution.CheckpointingMode
Key: FLINK-34941
URL: https://issues.apache.org/jira/browse/FLINK-34941
Project: Flink
Issue Type: Bug
Components: API / Core, Connectors / ElasticSearch, Runtime /
Checkpointing
Affects Versions: 1.20.0
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
After this change FLINK-34516 elasticsearch connector for 1.20-SNAPSHOT starts
failing with
{noformat}
Error:
/home/runner/work/flink-connector-elasticsearch/flink-connector-elasticsearch/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchSinkE2ECaseBase.java:[75,5]
method does not override or implement a method from a supertype
Error:
/home/runner/work/flink-connector-elasticsearch/flink-connector-elasticsearch/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchSinkE2ECaseBase.java:[85,84]
incompatible types: org.apache.flink.streaming.api.CheckpointingMode cannot be
converted to org.apache.flink.core.execution.CheckpointingMode
{noformat}
https://github.com/apache/flink-connector-elasticsearch/actions/runs/8436631571/job/23104522666#step:15:12668
set blocker since now every build of elasticsearch connector against
1.20-SNAPSHOT is failing
probably same issue is for opensearch connector
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34896) Mi
Sergey Nuyanzin created FLINK-34896: --- Summary: Mi Key: FLINK-34896 URL: https://issues.apache.org/jira/browse/FLINK-34896 Project: Flink Issue Type: Sub-task Reporter: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34895) Migrate FlinkRewriteSubQueryRule
Sergey Nuyanzin created FLINK-34895: --- Summary: Migrate FlinkRewriteSubQueryRule Key: FLINK-34895 URL: https://issues.apache.org/jira/browse/FLINK-34895 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34894) Migrate JoinDependentConditionDerivationRule
Sergey Nuyanzin created FLINK-34894: --- Summary: Migrate JoinDependentConditionDerivationRule Key: FLINK-34894 URL: https://issues.apache.org/jira/browse/FLINK-34894 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34893) Bump Checkstyle to 9+
Sergey Nuyanzin created FLINK-34893: --- Summary: Bump Checkstyle to 9+ Key: FLINK-34893 URL: https://issues.apache.org/jira/browse/FLINK-34893 Project: Flink Issue Type: Bug Components: Build System Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin The issue with current checkstyle is that there is checkstyle IntellijIdea plugin And recently it dropped checkstyle 8 support [1] At the same time we can not move to Checkstyle 10 since 10.x requires java 11+ [1] https://github.com/jshiell/checkstyle-idea/blob/main/CHANGELOG.md -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34676) Migrate ConvertToNotInOrInRule
Sergey Nuyanzin created FLINK-34676: --- Summary: Migrate ConvertToNotInOrInRule Key: FLINK-34676 URL: https://issues.apache.org/jira/browse/FLINK-34676 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34675) Migrate AggregateReduceGroupingRule
Sergey Nuyanzin created FLINK-34675: --- Summary: Migrate AggregateReduceGroupingRule Key: FLINK-34675 URL: https://issues.apache.org/jira/browse/FLINK-34675 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34674) Migrate CalcSnapshotTransposeRule
Sergey Nuyanzin created FLINK-34674: --- Summary: Migrate CalcSnapshotTransposeRule Key: FLINK-34674 URL: https://issues.apache.org/jira/browse/FLINK-34674 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34637) Migrate JoinConditionEqualityTransferRule
Sergey Nuyanzin created FLINK-34637: --- Summary: Migrate JoinConditionEqualityTransferRule Key: FLINK-34637 URL: https://issues.apache.org/jira/browse/FLINK-34637 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34602) Migrate StreamPhysicalWindowJoinRule
Sergey Nuyanzin created FLINK-34602: --- Summary: Migrate StreamPhysicalWindowJoinRule Key: FLINK-34602 URL: https://issues.apache.org/jira/browse/FLINK-34602 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34601) Migrate StreamPhysicalConstantTableFunctionScanRule
Sergey Nuyanzin created FLINK-34601: --- Summary: Migrate StreamPhysicalConstantTableFunctionScanRule Key: FLINK-34601 URL: https://issues.apache.org/jira/browse/FLINK-34601 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34600) Migrate PushLimitIntoLegacyTableSourceScanRule
Sergey Nuyanzin created FLINK-34600: --- Summary: Migrate PushLimitIntoLegacyTableSourceScanRule Key: FLINK-34600 URL: https://issues.apache.org/jira/browse/FLINK-34600 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34599) Migrate BatchPhysicalConstantTableFunctionScanRule
Sergey Nuyanzin created FLINK-34599: --- Summary: Migrate BatchPhysicalConstantTableFunctionScanRule Key: FLINK-34599 URL: https://issues.apache.org/jira/browse/FLINK-34599 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34598) Migrate RemoveRedundantLocalRankRule
Sergey Nuyanzin created FLINK-34598: --- Summary: Migrate RemoveRedundantLocalRankRule Key: FLINK-34598 URL: https://issues.apache.org/jira/browse/FLINK-34598 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34597) Migrate SimplifyFilterConditionRule
Sergey Nuyanzin created FLINK-34597: --- Summary: Migrate SimplifyFilterConditionRule Key: FLINK-34597 URL: https://issues.apache.org/jira/browse/FLINK-34597 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34596) Migrate RemoveRedundantLocalHashAggRule
Sergey Nuyanzin created FLINK-34596: --- Summary: Migrate RemoveRedundantLocalHashAggRule Key: FLINK-34596 URL: https://issues.apache.org/jira/browse/FLINK-34596 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34587) Suport MODE aggregate function
Sergey Nuyanzin created FLINK-34587:
---
Summary: Suport MODE aggregate function
Key: FLINK-34587
URL: https://issues.apache.org/jira/browse/FLINK-34587
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
There is {{AVG}} function however sometimes it is required to find the most
repeatable item and {{MODE}} could be helpful to solve this
it is already supported by a number of vendors like DuckDB[1], Snowflake[2],
Spark [3], PostgreSQL[4]
[1] https://duckdb.org/docs/sql/aggregates.html
[2] https://docs.snowflake.com/en/sql-reference/functions/mode
[3] https://spark.apache.org/docs/latest/sql-ref-functions-builtin.html
[4] https://www.postgresql.org/docs/16/functions-aggregate.html
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34269) Sync maven-shade-plugin across modules in flink-shaded
Sergey Nuyanzin created FLINK-34269: --- Summary: Sync maven-shade-plugin across modules in flink-shaded Key: FLINK-34269 URL: https://issues.apache.org/jira/browse/FLINK-34269 Project: Flink Issue Type: Technical Debt Components: BuildSystem / Shaded Affects Versions: shaded-18.0 Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34234) Apply ShadeOptionalChecker for flink-shaded
Sergey Nuyanzin created FLINK-34234:
---
Summary: Apply ShadeOptionalChecker for flink-shaded
Key: FLINK-34234
URL: https://issues.apache.org/jira/browse/FLINK-34234
Project: Flink
Issue Type: Bug
Components: BuildSystem / Shaded
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
As it was found within FLINK-34148
that newer version of shade plugin breaks previous behavior and non shaded
artifacts are started being added to flink-shaded deps.
The tasks is to apply same check for flink-shaded with help of
{{ShadeOptionalChecker}} which is already applied for Flink
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34163) Migrate SimplifyJoinConditionRule
Sergey Nuyanzin created FLINK-34163: --- Summary: Migrate SimplifyJoinConditionRule Key: FLINK-34163 URL: https://issues.apache.org/jira/browse/FLINK-34163 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34162) Migrate LogicalUnnestRule
Sergey Nuyanzin created FLINK-34162: --- Summary: Migrate LogicalUnnestRule Key: FLINK-34162 URL: https://issues.apache.org/jira/browse/FLINK-34162 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34161) Migrate RewriteMinusAllRule
Sergey Nuyanzin created FLINK-34161: --- Summary: Migrate RewriteMinusAllRule Key: FLINK-34161 URL: https://issues.apache.org/jira/browse/FLINK-34161 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34160) Migrate FlinkCalcMergeRule
Sergey Nuyanzin created FLINK-34160: --- Summary: Migrate FlinkCalcMergeRule Key: FLINK-34160 URL: https://issues.apache.org/jira/browse/FLINK-34160 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34159) Migrate ConstantRankNumberColumnRemoveRule
Sergey Nuyanzin created FLINK-34159: --- Summary: Migrate ConstantRankNumberColumnRemoveRule Key: FLINK-34159 URL: https://issues.apache.org/jira/browse/FLINK-34159 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34158) Migrate WindowAggregateReduceFunctionsRule
Sergey Nuyanzin created FLINK-34158: --- Summary: Migrate WindowAggregateReduceFunctionsRule Key: FLINK-34158 URL: https://issues.apache.org/jira/browse/FLINK-34158 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34157) Migrate FlinkLimit0RemoveRule
Sergey Nuyanzin created FLINK-34157: --- Summary: Migrate FlinkLimit0RemoveRule Key: FLINK-34157 URL: https://issues.apache.org/jira/browse/FLINK-34157 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34156) Move Flink Calcite rules from Scala to Java
Sergey Nuyanzin created FLINK-34156: --- Summary: Move Flink Calcite rules from Scala to Java Key: FLINK-34156 URL: https://issues.apache.org/jira/browse/FLINK-34156 Project: Flink Issue Type: Technical Debt Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Fix For: 2.0.0 This is an umbrella task for migration of Calcite rules from Scala to Java mentioned at https://cwiki.apache.org/confluence/display/FLINK/2.0+Release The reason is that since 1.28.0 ( CALCITE-4787 - Move core to use Immutables instead of ImmutableBeans ) Calcite started to use Immutables (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( CALCITE-4839 - Remove remnants of ImmutableBeans post 1.28 release ). All rule configuration related api which is not Immutables based is marked as deprecated. Since Immutables implies code generation while java compilation it is seems impossible to use for rules in Scala code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34135) A number of ci failures with Access to the path '.../_work/_temp/containerHandlerInvoker.js' is denied.
Sergey Nuyanzin created FLINK-34135:
---
Summary: A number of ci failures with Access to the path
'.../_work/_temp/containerHandlerInvoker.js' is denied.
Key: FLINK-34135
URL: https://issues.apache.org/jira/browse/FLINK-34135
Project: Flink
Issue Type: Bug
Components: Build System / CI
Reporter: Sergey Nuyanzin
There is a number of builds failing with something like
{noformat}
##[error]Access to the path
'/home/agent03/myagent/_work/_temp/containerHandlerInvoker.js' is denied.
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34110) Bump janino to 3.1.11
Sergey Nuyanzin created FLINK-34110:
---
Summary: Bump janino to 3.1.11
Key: FLINK-34110
URL: https://issues.apache.org/jira/browse/FLINK-34110
Project: Flink
Issue Type: Technical Debt
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
Among others it fixes {{ArrayIndexOutOfBoundsException}} for unconditional
loops which connot complete normally
https://github.com/janino-compiler/janino/issues/208
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34013) ProfilingServiceTest.testRollingDeletion is unstable on AZP
Sergey Nuyanzin created FLINK-34013:
---
Summary: ProfilingServiceTest.testRollingDeletion is unstable on
AZP
Key: FLINK-34013
URL: https://issues.apache.org/jira/browse/FLINK-34013
Project: Flink
Issue Type: Bug
Components: API / Core
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56073&view=logs&j=0e7be18f-84f2-53f0-a32d-4a5e4a174679&t=7c1d86e3-35bd-5fd5-3b7c-30c126a78702&l=8258
fails as
{noformat}
Jan 06 02:09:28 org.opentest4j.AssertionFailedError: expected: <2> but was: <3>
Jan 06 02:09:28 at
org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
Jan 06 02:09:28 at
org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
Jan 06 02:09:28 at
org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
Jan 06 02:09:28 at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
Jan 06 02:09:28 at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
Jan 06 02:09:28 at
org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
Jan 06 02:09:28 at
org.apache.flink.runtime.util.profiler.ProfilingServiceTest.verifyRollingDeletionWorks(ProfilingServiceTest.java:167)
Jan 06 02:09:28 at
org.apache.flink.runtime.util.profiler.ProfilingServiceTest.testRollingDeletion(ProfilingServiceTest.java:117)
Jan 06 02:09:28 at java.lang.reflect.Method.invoke(Method.java:498)
Jan 06 02:09:28 at
java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
Jan 06 02:09:28 at
java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
Jan 06 02:09:28 at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
Jan 06 02:09:28 at
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
Jan 06 02:09:28 at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34012) Flink python fails with can't read file '/__w/2/s/flink-python/dev/.conda/lib/python3.10/site-packages//google
Sergey Nuyanzin created FLINK-34012:
---
Summary: Flink python fails with can't read file
'/__w/2/s/flink-python/dev/.conda/lib/python3.10/site-packages//google
Key: FLINK-34012
URL: https://issues.apache.org/jira/browse/FLINK-34012
Project: Flink
Issue Type: Bug
Components: API / Python
Reporter: Sergey Nuyanzin
Fix For: 1.19.0
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56073&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=c67e71ed-6451-5d26-8920-5a8cf9651901&l=20755
{noformat}
Jan 06 03:02:43 Installing collected packages: types-pytz,
types-python-dateutil, types-protobuf
Jan 06 03:02:43 Successfully installed types-protobuf-4.24.0.20240106
types-python-dateutil-2.8.19.20240106 types-pytz-2023.3.1.1
Jan 06 03:02:44 mypy: can't read file
'/__w/2/s/flink-python/dev/.conda/lib/python3.10/site-packages//google': No
such file or directory
Jan 06 03:02:44 Installing missing stub packages:
Jan 06 03:02:44 /__w/2/s/flink-python/dev/.conda/bin/python -m pip install
types-protobuf types-python-dateutil types-pytz
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34011) EmbeddedThreadDependencyTests.test_add_python_file fails with This function should not be called!
Sergey Nuyanzin created FLINK-34011:
---
Summary: EmbeddedThreadDependencyTests.test_add_python_file fails
with This function should not be called!
Key: FLINK-34011
URL: https://issues.apache.org/jira/browse/FLINK-34011
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.18.2
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55966&view=logs&j=bf5e383b-9fd3-5f02-ca1c-8f788e2e76d3&t=85189c57-d8a0-5c9c-b61d-fc05cfac62cf&l=27052
fails as
{noformat}
Jan 02 03:27:17 E Caused by: pemja.core.PythonException:
: This function should not be called!
Jan 02 03:27:17 E at
/__w/1/s/flink-python/pyflink/fn_execution/embedded/operation_utils.process_element(operation_utils.py:72)
Jan 02 03:27:17 E at
/__w/1/s/flink-python/pyflink/fn_execution/table/operations.process_element(operations.py:102)
Jan 02 03:27:17 E at .(:1)
Jan 02 03:27:17 E at
/__w/1/s/flink-python/pyflink/table/tests/test_dependency.plus_two(test_dependency.py:52)
Jan 02 03:27:17 E at
/tmp/python-dist-98ae39ca-f087-4122-8e67-f358a13f57c7/python-files/blob_p-bedc014754a86159784fd7cebff77ab534802643-43ac971c5ca669babbfe4ee1d5a1ad0e/test_dependency_manage_lib.add_two(test_dependency_manage_lib.py:2)
Jan 02 03:27:17 E at
pemja.core.PythonInterpreter.invokeMethodOneArgLong(Native Method)
Jan 02 03:27:17 E at
pemja.core.PythonInterpreter.invokeMethodOneArg(PythonInterpreter.java:222)
Jan 02 03:27:17 E at
pemja.core.PythonInterpreter.invokeMethod(PythonInterpreter.java:116)
Jan 02 03:27:17 E at
org.apache.flink.table.runtime.operators.python.scalar.EmbeddedPythonScalarFunctionOperator.processElement(EmbeddedPythonScalarFunctionOperator.java:170)
Jan 02 03:27:17 E at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75)
Jan 02 03:27:17 E at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50)
Jan 02 03:27:17 E at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
Jan 02 03:27:17 E at
org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:425)
Jan 02 03:27:17 E at
org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:520)
Jan 02 03:27:17 E at
org.apache.flink.streaming.api.operators.StreamSourceContexts$SwitchingOnClose.collect(StreamSourceContexts.java:110)
Jan 02 03:27:17 E at
org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:99)
Jan 02 03:27:17 E at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:114)
Jan 02 03:27:17 E at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:71)
Jan 02 03:27:17 E at
org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:338)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34010) KafkaTableITCase.testPerPartitionWatermarkWithIdleSource is unstable on AZP
Sergey Nuyanzin created FLINK-34010:
---
Summary: KafkaTableITCase.testPerPartitionWatermarkWithIdleSource
is unstable on AZP
Key: FLINK-34010
URL: https://issues.apache.org/jira/browse/FLINK-34010
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: 1.17.3
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55965&view=logs&j=9c5a5fe6-2f39-545e-1630-feb3d8d0a1ba&t=99b23320-1d05-5741-d63f-9e78473da39e&l=32434
fails as
{noformat}
Jan 02 05:24:56 [ERROR] Tests run: 39, Failures: 0, Errors: 1, Skipped: 0, Time
elapsed: 97.948 s <<< FAILURE! - in
org.apache.flink.streaming.connectors.kafka.table.KafkaTableITCase
Jan 02 05:24:56 [ERROR]
KafkaTableITCase.testPerPartitionWatermarkWithIdleSource Time elapsed: 5.494 s
<<< ERROR!
Jan 02 05:24:56 java.util.concurrent.TimeoutException: Can not get the expected
result.
Jan 02 05:24:56 at
org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:214)
Jan 02 05:24:56 at
org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:230)
Jan 02 05:24:56 at
org.apache.flink.streaming.connectors.kafka.table.KafkaTableTestUtils.waitingExpectedResults(KafkaTableTestUtils.java:82)
Jan 02 05:24:56 at
org.apache.flink.streaming.connectors.kafka.table.KafkaTableITCase.testPerPartitionWatermarkWithIdleSource(KafkaTableITCase.java:951)
Jan 02 05:24:56 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
Jan 02 05:24:56 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jan 02 05:24:56 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33919) AutoRescalingITCase hangs on AZP
Sergey Nuyanzin created FLINK-33919:
---
Summary: AutoRescalingITCase hangs on AZP
Key: FLINK-33919
URL: https://issues.apache.org/jira/browse/FLINK-33919
Project: Flink
Issue Type: Improvement
Components: Runtime / Checkpointing
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
This build fails on AZP
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55700&view=logs&j=a657ddbf-d986-5381-9649-342d9c92e7fb&t=dc085d4a-05c8-580e-06ab-21f5624dab16&l=8608
because of waiting
{noformat}
Dec 20 02:07:46 "main" #1 [14299] prio=5 os_prio=0 cpu=12675.70ms
elapsed=3115.94s tid=0x7f3f71481600 nid=14299 waiting on condition
[0x7f3f74913000]
Dec 20 02:07:46java.lang.Thread.State: TIMED_WAITING (sleeping)
Dec 20 02:07:46 at java.lang.Thread.sleep0(java.base@21.0.1/Native
Method)
Dec 20 02:07:46 at
java.lang.Thread.sleep(java.base@21.0.1/Thread.java:509)
Dec 20 02:07:46 at
org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:152)
Dec 20 02:07:46 at
org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:145)
Dec 20 02:07:46 at
org.apache.flink.runtime.testutils.CommonTestUtils.waitForOneMoreCheckpoint(CommonTestUtils.java:374)
Dec 20 02:07:46 at
org.apache.flink.test.checkpointing.AutoRescalingITCase.testCheckpointRescalingKeyedState(AutoRescalingITCase.java:265)
Dec 20 02:07:46 at
org.apache.flink.test.checkpointing.AutoRescalingITCase.testCheckpointRescalingInKeyedState(AutoRescalingITCase.java:196)
Dec 20 02:07:46 at
java.lang.invoke.LambdaForm$DMH/0x7f3f0f201400.invokeVirtual(java.base@21.0.1/LambdaForm$DMH)
Dec 20 02:07:46 at
java.lang.invoke.LambdaForm$MH/0x7f3f0f20c000.invoke(java.base@21.0.1/LambdaForm$MH)
Dec 20 02:07:46 at
java.lang.invoke.Invokers$Holder.invokeExact_MT(java.base@21.0.1/Invokers$Holder)
Dec 20 02:07:46 at
jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.base@21.0.1/DirectMethodHandleAccessor.java:153)
Dec 20 02:07:46 at
jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.base@21.0.1/DirectMethodHandleAccessor.java:103)
Dec 20 02:07:46 at
java.lang.reflect.Method.invoke(java.base@21.0.1/Method.java:580)
Dec 20 02:07:46 at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
Dec 20 02:07:46 at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Dec 20 02:07:46 at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
Dec 20 02:07:46 at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
Dec 20 02:07:46 at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
Dec 20 02:07:46 at
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
Dec 20 02:07:46 at
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
Dec 20 02:07:46 at
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33859) Support OpenSearch v2
Sergey Nuyanzin created FLINK-33859: --- Summary: Support OpenSearch v2 Key: FLINK-33859 URL: https://issues.apache.org/jira/browse/FLINK-33859 Project: Flink Issue Type: Improvement Components: Connectors / Opensearch Affects Versions: opensearch-1.2.0 Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin The main issue is that in OpenSearch v2 there were several breaking changes like [https://github.com/opensearch-project/OpenSearch/pull/9082] [https://github.com/opensearch-project/OpenSearch/pull/5902] which made current connector version failing while communicating with v2 Also it would make sense to add integration and e2e tests to test against v2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33858) CI fails with No space left on device
Sergey Nuyanzin created FLINK-33858: --- Summary: CI fails with No space left on device Key: FLINK-33858 URL: https://issues.apache.org/jira/browse/FLINK-33858 Project: Flink Issue Type: Bug Components: Build System / CI Reporter: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33815) Add tests against jdk17 for pulsar connector
Sergey Nuyanzin created FLINK-33815: --- Summary: Add tests against jdk17 for pulsar connector Key: FLINK-33815 URL: https://issues.apache.org/jira/browse/FLINK-33815 Project: Flink Issue Type: Technical Debt Components: Connectors / Pulsar Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33807) Update junit to 5.10.1
Sergey Nuyanzin created FLINK-33807:
---
Summary: Update junit to 5.10.1
Key: FLINK-33807
URL: https://issues.apache.org/jira/browse/FLINK-33807
Project: Flink
Issue Type: Technical Debt
Components: Test Infrastructure
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
There is logging improved for
>Exceptions thrown for files that cannot be deleted when cleaning up a
>temporary directory created via {{@TempDir}} now include the root cause.
[https://junit.org/junit5/docs/current/release-notes/index.html#bug-fixes-2]
which could help with debugging of FLINK-33641
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33800) Allow passing parameters to database via jdbc url
Sergey Nuyanzin created FLINK-33800:
---
Summary: Allow passing parameters to database via jdbc url
Key: FLINK-33800
URL: https://issues.apache.org/jira/browse/FLINK-33800
Project: Flink
Issue Type: Improvement
Components: Connectors / JDBC
Affects Versions: jdbc-3.1.1
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
Currently it does not allow to pass extra properties e.g.
an attempt to connect to
{{jdbc:postgresql://...?sslmode=require}}
fails with
{noformat}
Caused by: org.apache.flink.table.gateway.api.utils.SqlGatewayException: Failed
to fetchResults.
at
org.apache.flink.table.gateway.service.SqlGatewayServiceImpl.fetchResults(SqlGatewayServiceImpl.java:229)
at
org.apache.flink.table.gateway.rest.handler.statement.FetchResultsHandler.handleRequest(FetchResultsHandler.java:83)
... 48 more
Caused by: org.apache.flink.table.gateway.service.utils.SqlExecutionException:
Failed to execute the operation b70b5cf7-7068-4eb6-83a4-78aed36dbd35.
at
org.apache.flink.table.gateway.service.operation.OperationManager$Operation.processThrowable(OperationManager.java:414)
at
org.apache.flink.table.gateway.service.operation.OperationManager$Operation.lambda$run$0(OperationManager.java:267)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
{noformat}
because of of a logic at
{{org.apache.flink.connector.jdbc.catalog.JdbcCatalogUtils#validateJdbcUrl}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33796) Add ability to customize java version for python ci in connectors
Sergey Nuyanzin created FLINK-33796: --- Summary: Add ability to customize java version for python ci in connectors Key: FLINK-33796 URL: https://issues.apache.org/jira/browse/FLINK-33796 Project: Flink Issue Type: Improvement Components: Connectors / Parent Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin In FLINK-33556 there was added ci for python however currently it is sticked to jdk8 It would be great to have ability to customize java version -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33787) Java 17 support for jdbc connector
Sergey Nuyanzin created FLINK-33787: --- Summary: Java 17 support for jdbc connector Key: FLINK-33787 URL: https://issues.apache.org/jira/browse/FLINK-33787 Project: Flink Issue Type: Bug Components: Connectors / JDBC Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33786) Remove Hive connector from master branch
Sergey Nuyanzin created FLINK-33786: --- Summary: Remove Hive connector from master branch Key: FLINK-33786 URL: https://issues.apache.org/jira/browse/FLINK-33786 Project: Flink Issue Type: Sub-task Components: Connectors / Hive Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin The connector is going to be externalized at FLINK-30064 Once it is done it would make sense to remove it from master branch -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33776) Allow to specify optional profile for connectors
Sergey Nuyanzin created FLINK-33776: --- Summary: Allow to specify optional profile for connectors Key: FLINK-33776 URL: https://issues.apache.org/jira/browse/FLINK-33776 Project: Flink Issue Type: Bug Components: Connectors / Parent Reporter: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33755) Cleanup usage of deprecated StreamExecutionEnvironment#generateSequence
Sergey Nuyanzin created FLINK-33755: --- Summary: Cleanup usage of deprecated StreamExecutionEnvironment#generateSequence Key: FLINK-33755 URL: https://issues.apache.org/jira/browse/FLINK-33755 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33727) JoinRestoreTest is failing on AZP
Sergey Nuyanzin created FLINK-33727:
---
Summary: JoinRestoreTest is failing on AZP
Key: FLINK-33727
URL: https://issues.apache.org/jira/browse/FLINK-33727
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
Since it was introduced in FLINK-33470 it seems to be a reason
{noformat}
Dec 02 04:42:26 04:42:26.408 [ERROR] Failures:
Dec 02 04:42:26 04:42:26.408 [ERROR]
JoinRestoreTest>RestoreTestBase.testRestore:283
Dec 02 04:42:26 Expecting actual:
Dec 02 04:42:26 ["+I[9, carol, apple, 9000]",
Dec 02 04:42:26 "+I[8, bill, banana, 8000]",
Dec 02 04:42:26 "+I[6, jerry, pen, 6000]"]
Dec 02 04:42:26 to contain exactly in any order:
Dec 02 04:42:26 ["+I[Adam, null]",
Dec 02 04:42:26 "+I[Baker, Research]",
Dec 02 04:42:26 "+I[Charlie, Human Resources]",
Dec 02 04:42:26 "+I[Charlie, HR]",
Dec 02 04:42:26 "+I[Don, Sales]",
Dec 02 04:42:26 "+I[Victor, null]",
Dec 02 04:42:26 "+I[Helena, Engineering]",
Dec 02 04:42:26 "+I[Juliet, Engineering]",
Dec 02 04:42:26 "+I[Ivana, Research]",
Dec 02 04:42:26 "+I[Charlie, People Operations]"]
{noformat}
examples of failures
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55120&view=logs&j=0c940707-2659-5648-cbe6-a1ad63045f0a&t=075c2716-8010-5565-fe08-3c4bb45824a4&l=12099
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55129&view=logs&j=a9db68b9-a7e0-54b6-0f98-010e0aff39e2&t=cdd32e0b-6047-565b-c58f-14054472f1be&l=11786
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55136&view=logs&j=0c940707-2659-5648-cbe6-a1ad63045f0a&t=075c2716-8010-5565-fe08-3c4bb45824a4&l=12099
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55137&view=logs&j=0c940707-2659-5648-cbe6-a1ad63045f0a&t=075c2716-8010-5565-fe08-3c4bb45824a4&l=11779
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33706) Build_wheels_on_macos fails on AZP
Sergey Nuyanzin created FLINK-33706:
---
Summary: Build_wheels_on_macos fails on AZP
Key: FLINK-33706
URL: https://issues.apache.org/jira/browse/FLINK-33706
Project: Flink
Issue Type: Bug
Components: API / Python, Test Infrastructure
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55044&view=logs&j=f73b5736-8355-5390-ec71-4dfdec0ce6c5&t=90f7230e-bf5a-531b-8566-ad48d3e03bbb&l=427
fails on AZP as
{noformat}
note: This error originates from a subprocess, and is likely not a
problem with pip.
ERROR: Failed cleaning build dir for crcmod
Building wheel for dill (setup.py): started
Building wheel for dill (setup.py): finished with status 'done'
Created wheel for dill: filename=dill-0.0.0-py3-none-any.whl size=899
sha256=39d0b4b66ce11f42313482f4ad825029e861fd6dab87a743a95d75a44a1fedd6
Stored in directory:
/Users/runner/Library/Caches/pip/wheels/07/35/78/e9004fa30578734db7f10e7a211605f3f0778d2bdde38a239d
Building wheel for hdfs (setup.py): started
Building wheel for hdfs (setup.py): finished with status 'done'
Created wheel for hdfs: filename=UNKNOWN-0.0.0-py3-none-any.whl
size=928 sha256=cb3fd7d8c71b52bbc27cfb75842f9d4d9c6f3b847f3f4fe50323c945a0e38ccc
Stored in directory:
/Users/runner/Library/Caches/pip/wheels/68/dd/29/c1a590238f9ebbe4f7ee9b3583f5185d0b9577e23f05c990eb
WARNING: Built wheel for hdfs is invalid: Wheel has unexpected file
name: expected 'hdfs', got 'UNKNOWN'
Building wheel for pymongo (pyproject.toml): started
Building wheel for pymongo (pyproject.toml): finished with status 'done'
Created wheel for pymongo:
filename=pymongo-4.6.1-cp38-cp38-macosx_10_9_x86_64.whl size=478012
sha256=5dfc6fdb6a8a399f8f9da44e28bae19be244b15c8000cd3b2d7d6ff513cc6277
Stored in directory:
/Users/runner/Library/Caches/pip/wheels/54/d8/0e/2a61e90bb3872d903b15eb3c94cb70f438fb8792a28fee7bb1
Building wheel for docopt (setup.py): started
Building wheel for docopt (setup.py): finished with status 'done'
Created wheel for docopt: filename=UNKNOWN-0.0.0-py3-none-any.whl
size=920 sha256=612c56cd1a6344b8def6c4d3c3c1c8bb10e1f2b0d978fee0fc8b9281026e8288
Stored in directory:
/Users/runner/Library/Caches/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c
WARNING: Built wheel for docopt is invalid: Wheel has unexpected file
name: expected 'docopt', got 'UNKNOWN'
Successfully built dill pymongo
Failed to build fastavro crcmod hdfs docopt
ERROR: Could not build wheels for fastavro, which is required to install
pyproject.toml-based projects
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33705) Upgrade flink-shaded to 18.0
Sergey Nuyanzin created FLINK-33705: --- Summary: Upgrade flink-shaded to 18.0 Key: FLINK-33705 URL: https://issues.apache.org/jira/browse/FLINK-33705 Project: Flink Issue Type: Technical Debt Components: BuildSystem / Shaded Affects Versions: 1.19.0 Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Currently flink-shaded is in a process of releasing Once it is released it would make sense to upgrade the dependency in Flink -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33703) Use maven 3.8.6 for releasing of flink-shaded
Sergey Nuyanzin created FLINK-33703:
---
Summary: Use maven 3.8.6 for releasing of flink-shaded
Key: FLINK-33703
URL: https://issues.apache.org/jira/browse/FLINK-33703
Project: Flink
Issue Type: Technical Debt
Components: BuildSystem / Shaded
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
Currently there is maven-enforcer-plugin configuration (for release only)
{noformat}
(,3.3)
{noformat}
which seems to be outdated and for ci 3.8.6 is used
We should keep them in sync and use 3.8.6 for both
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33690) build_wheels_on_macos sometimes fails with read timed out on AZP
Sergey Nuyanzin created FLINK-33690:
---
Summary: build_wheels_on_macos sometimes fails with read timed out
on AZP
Key: FLINK-33690
URL: https://issues.apache.org/jira/browse/FLINK-33690
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.18.0
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55013&view=logs&j=f73b5736-8355-5390-ec71-4dfdec0ce6c5&t=90f7230e-bf5a-531b-8566-ad48d3e03bbb&l=217
failed as
{noformat}
...
File
"/private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7cgn/T/pip-standalone-pip-3jt0u_hl/__env_pip__.zip/pip/_vendor/urllib3/response.py",
line 573, in stream
data = self.read(amt=amt, decode_content=decode_content)
File
"/private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7cgn/T/pip-standalone-pip-3jt0u_hl/__env_pip__.zip/pip/_vendor/urllib3/response.py",
line 538, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py",
line 131, in __exit__
self.gen.throw(type, value, traceback)
File
"/private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7cgn/T/pip-standalone-pip-3jt0u_hl/__env_pip__.zip/pip/_vendor/urllib3/response.py",
line 440, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip._vendor.urllib3.exceptions.ReadTimeoutError:
HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33664) Setup cron build for java 21
Sergey Nuyanzin created FLINK-33664: --- Summary: Setup cron build for java 21 Key: FLINK-33664 URL: https://issues.apache.org/jira/browse/FLINK-33664 Project: Flink Issue Type: Sub-task Components: Build System / CI Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33655) ScalaSerializersMigrationTest#testStableAnonymousClassnameGeneration fails with jdk21
Sergey Nuyanzin created FLINK-33655:
---
Summary:
ScalaSerializersMigrationTest#testStableAnonymousClassnameGeneration fails with
jdk21
Key: FLINK-33655
URL: https://issues.apache.org/jira/browse/FLINK-33655
Project: Flink
Issue Type: Sub-task
Components: Tests
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
{noformat}
23:21:01.875 [ERROR]
org.apache.flink.api.scala.migration.ScalaSerializersMigrationTest.testStableAnonymousClassnameGeneration
Time elapsed: 0.209 s <<< FAILURE!
org.junit.ComparisonFailure: expected:<...MigrationTest$$anon$[8]> but
was:<...MigrationTest$$anon$[1]>
at org.junit.Assert.assertEquals(Assert.java:117)
at org.junit.Assert.assertEquals(Assert.java:146)
at
org.apache.flink.api.scala.migration.ScalaSerializersMigrationTest.testStableAnonymousClassnameGeneration(ScalaSerializersMigrationTest.scala:60)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
{noformat}
it could/should be skipped in same way it is skipped for java 17
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33646) Cleanup the rest usage of deprecated TableEnvironment#registerFunction and in docs
Sergey Nuyanzin created FLINK-33646:
---
Summary: Cleanup the rest usage of deprecated
TableEnvironment#registerFunction and in docs
Key: FLINK-33646
URL: https://issues.apache.org/jira/browse/FLINK-33646
Project: Flink
Issue Type: Sub-task
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
It seems there are still few places where it is not cleaned up , also docs
e.g.
{noformat}
docs/content.zh/docs/dev/table/functions/udfs.md
docs/content.zh/docs/dev/table/tableApi.md
docs/content/docs/dev/table/tableApi.md
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/AggregatedTable.java
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/FlatAggregateTable.java
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/GroupedTable.java
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/harness/TableAggregateHarnessTest.scala
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/AggregateITCase.scala
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/GroupWindowITCase.scala
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/MatchRecognizeITCase.scala
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/OverAggregateITCase.scala
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/utils/UserDefinedFunctionTestUtils.scala
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33642) Use docker image with jdk21 in CI
Sergey Nuyanzin created FLINK-33642: --- Summary: Use docker image with jdk21 in CI Key: FLINK-33642 URL: https://issues.apache.org/jira/browse/FLINK-33642 Project: Flink Issue Type: Sub-task Components: Build System / CI Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33641) RankITCase.testMultipleUnaryTopNAfterAgg fails on AZP
Sergey Nuyanzin created FLINK-33641:
---
Summary: RankITCase.testMultipleUnaryTopNAfterAgg fails on AZP
Key: FLINK-33641
URL: https://issues.apache.org/jira/browse/FLINK-33641
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
this build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=54856&view=logs&j=a9db68b9-a7e0-54b6-0f98-010e0aff39e2&t=cdd32e0b-6047-565b-c58f-14054472f1be&l=11289
fails with
{noformat}
Nov 24 02:21:53 Suppressed: java.nio.file.DirectoryNotEmptyException:
/tmp/junit1727687356898183357/junit4798298549994985259/1ac07a5866d81240870d5a2982531508
Nov 24 02:21:53 at
sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
Nov 24 02:21:53 at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
Nov 24 02:21:53 at java.nio.file.Files.delete(Files.java:1126)
Nov 24 02:21:53 at
org.junit.jupiter.engine.extension.TempDirectory$CloseablePath$1.deleteAndContinue(TempDirectory.java:293)
Nov 24 02:21:53 at
org.junit.jupiter.engine.extension.TempDirectory$CloseablePath$1.postVisitDirectory(TempDirectory.java:288)
Nov 24 02:21:53 at
org.junit.jupiter.engine.extension.TempDirectory$CloseablePath$1.postVisitDirectory(TempDirectory.java:264)
Nov 24 02:21:53 at
java.nio.file.Files.walkFileTree(Files.java:2688)
Nov 24 02:21:53 at
java.nio.file.Files.walkFileTree(Files.java:2742)
Nov 24 02:21:53 at
org.junit.jupiter.engine.extension.TempDirectory$CloseablePath.deleteAllFilesAndDirectories(TempDirectory.java:264)
Nov 24 02:21:53 at
org.junit.jupiter.engine.extension.TempDirectory$CloseablePath.close(TempDirectory.java:249)
Nov 24 02:21:53 ... 92 more
{noformat}
not sure however this might be related to recent JUnit4 => JUnit5 upgrade
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33639) Support zookeeper 3.9.0+
Sergey Nuyanzin created FLINK-33639: --- Summary: Support zookeeper 3.9.0+ Key: FLINK-33639 URL: https://issues.apache.org/jira/browse/FLINK-33639 Project: Flink Issue Type: Technical Debt Components: BuildSystem / Shaded, Runtime / Coordination Reporter: Sergey Nuyanzin This is a follow up task for Flink once Curator's CURATOR-687 task to support ZooKeeper 3.9.0+ is done -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33620) Add Java 21 to docker image
Sergey Nuyanzin created FLINK-33620: --- Summary: Add Java 21 to docker image Key: FLINK-33620 URL: https://issues.apache.org/jira/browse/FLINK-33620 Project: Flink Issue Type: Sub-task Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin currently there is only jdk 8, 11, 17 at https://github.com/zentol/flink-ci-docker -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33618) Upgrade swagger for flink-shaded to 2.2.19
Sergey Nuyanzin created FLINK-33618: --- Summary: Upgrade swagger for flink-shaded to 2.2.19 Key: FLINK-33618 URL: https://issues.apache.org/jira/browse/FLINK-33618 Project: Flink Issue Type: Technical Debt Components: BuildSystem / Shaded Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33617) Upgrade tcnative for flink-shaded
Sergey Nuyanzin created FLINK-33617: --- Summary: Upgrade tcnative for flink-shaded Key: FLINK-33617 URL: https://issues.apache.org/jira/browse/FLINK-33617 Project: Flink Issue Type: Technical Debt Components: BuildSystem / Shaded Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33591) Cleanup ysage of deprecated TableTestBase#addFunction
Sergey Nuyanzin created FLINK-33591:
---
Summary: Cleanup ysage of deprecated TableTestBase#addFunction
Key: FLINK-33591
URL: https://issues.apache.org/jira/browse/FLINK-33591
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin
I would say it is a subtask of cleaning up for
{{TableEnvironment#registerFunction}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33531) Nightly Python fails with NPE at metadataHandlerProvider on AZP
Sergey Nuyanzin created FLINK-33531:
---
Summary: Nightly Python fails with NPE at metadataHandlerProvider
on AZP
Key: FLINK-33531
URL: https://issues.apache.org/jira/browse/FLINK-33531
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
It seems starting 02.11.2023 every master nightly fails with this (that's why
it is a blocker)
for instance
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=54512&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=c67e71ed-6451-5d26-8920-5a8cf9651901]
{noformat}
2023-11-12T02:10:24.5082784Z Nov 12 02:10:24 if is_error(answer)[0]:
2023-11-12T02:10:24.5083620Z Nov 12 02:10:24 if len(answer) > 1:
2023-11-12T02:10:24.5084326Z Nov 12 02:10:24 type = answer[1]
2023-11-12T02:10:24.5085164Z Nov 12 02:10:24 value =
OUTPUT_CONVERTER[type](answer[2:], gateway_client) 2023-11-12T02:10:24.5086061Z
Nov 12 02:10:24 if answer[1] == REFERENCE_TYPE: 2023-11-12T02:10:24.5086850Z
Nov 12 02:10:24 > raise Py4JJavaError( 2023-11-12T02:10:24.5087677Z Nov 12
02:10:24 "An error occurred while calling \{0}{1}\{2}.\n".
2023-11-12T02:10:24.5088538Z Nov 12 02:10:24 format(target_id, ".", name),
value) 2023-11-12T02:10:24.5089551Z Nov 12 02:10:24 E
py4j.protocol.Py4JJavaError: An error occurred while calling
o3371.executeInsert. 2023-11-12T02:10:24.5090832Z Nov 12 02:10:24 E :
java.lang.NullPointerException: metadataHandlerProvider
2023-11-12T02:10:24.5091832Z Nov 12 02:10:24 E at
java.util.Objects.requireNonNull(Objects.java:228) 2023-11-12T02:10:24.5093399Z
Nov 12 02:10:24 E at
org.apache.calcite.rel.metadata.RelMetadataQueryBase.getMetadataHandlerProvider(RelMetadataQueryBase.java:122)
2023-11-12T02:10:24.5094480Z Nov 12 02:10:24 E at
org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118)
2023-11-12T02:10:24.5095365Z Nov 12 02:10:24 E at
org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844)
2023-11-12T02:10:24.5096306Z Nov 12 02:10:24 E at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307)
2023-11-12T02:10:24.5097238Z Nov 12 02:10:24 E at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
2023-11-12T02:10:24.5098014Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556)
2023-11-12T02:10:24.5098753Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:420)
2023-11-12T02:10:24.5099517Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.executeRuleInstance(HepPlanner.java:243)
2023-11-12T02:10:24.5100373Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance$State.execute(HepInstruction.java:178)
2023-11-12T02:10:24.5101313Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.lambda$executeProgram$0(HepPlanner.java:211)
2023-11-12T02:10:24.5102410Z Nov 12 02:10:24 E at
org.apache.flink.calcite.shaded.com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
2023-11-12T02:10:24.5103343Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210)
2023-11-12T02:10:24.5104105Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepProgram$State.execute(HepProgram.java:118)
2023-11-12T02:10:24.5104868Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:205)
2023-11-12T02:10:24.5105616Z Nov 12 02:10:24 E at
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:191)
2023-11-12T02:10:24.5106421Z Nov 12 02:10:24 E at
org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64)
2023-11-12T02:10:24.5107359Z Nov 12 02:10:24 E at
org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78)
2023-11-12T02:10:24.5108346Z Nov 12 02:10:24 E at
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59)
2023-11-12T02:10:24.5109407Z Nov 12 02:10:24 E at
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156)
2023-11-12T02:10:24.5110241Z Nov 12 02:10:24 E at
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)
2023-11-12T02:10:24.5111078Z Nov 12 02:10:24 E at
scala.collection.Iterator.foreach(Iterator.scala:937)
2023-11-12T02:10:24.5111734Z Nov 12 02:10:24 E at
scala.collection.Iterator.foreach$(Iterator.scala:937)
2023-11-12T02:10:24.5112410Z Nov 12 02:10:24 E at
scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
2023-11-12T02:10:24.5113145Z Nov 12 02:10:24 E at
scala.collection.IterableLike.foreach(IterableLike.scala:70)
2023-11-
[jira] [Created] (FLINK-33506) Make AWS connectors compilable with jdk17
Sergey Nuyanzin created FLINK-33506: --- Summary: Make AWS connectors compilable with jdk17 Key: FLINK-33506 URL: https://issues.apache.org/jira/browse/FLINK-33506 Project: Flink Issue Type: Improvement Components: Connectors / AWS Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Since 1.18 Flink with jdk 17 support is released it would make sense to add such support for connectors -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33485) Optimize the EXISTS sub-query by Metadata RowCount
Sergey Nuyanzin created FLINK-33485:
---
Summary: Optimize the EXISTS sub-query by Metadata RowCount
Key: FLINK-33485
URL: https://issues.apache.org/jira/browse/FLINK-33485
Project: Flink
Issue Type: Improvement
Components: Table SQL / API
Affects Versions: 1.18.0
Reporter: Sergey Nuyanzin
If the sub-query is guaranteed to produce at least one row, just return TRUE.
If the sub-query is guaranteed to produce no row, just return FALSE.
inspired by CALCITE-5117 however since there is {{FlinkSubQueryRemoveRule}}
then it shold be adopted accordingly
examples
{code:sql}
SELECT * FROM T2 WHERE EXISTS (SELECT SUM(a1), COUNT(*) FROM T1 WHERE 1=2)
{code}
aggregation functions always return 1 row even if there is an empty table then
we could just replace this query with
{code:sql}
SELECT * FROM T2
{code}
another example
{code:sql}
SELECT * FROM MyTable WHERE NOT EXISTS (SELECT a FROM MyTable LIMIT 0)
{code}
{{LIMIT 0}} means no rows so it cold be optimized to
{code:sql}
SELECT * FROM MyTable
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33471) Kubernetes operator supports compiling with Java 21
Sergey Nuyanzin created FLINK-33471: --- Summary: Kubernetes operator supports compiling with Java 21 Key: FLINK-33471 URL: https://issues.apache.org/jira/browse/FLINK-33471 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Sergey Nuyanzin Since there is a new Java LTS version available (21) it would make sense to support it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33457) FlinkImageBuilder checks for Java 21
Sergey Nuyanzin created FLINK-33457:
---
Summary: FlinkImageBuilder checks for Java 21
Key: FLINK-33457
URL: https://issues.apache.org/jira/browse/FLINK-33457
Project: Flink
Issue Type: Sub-task
Components: Tests
Reporter: Sergey Nuyanzin
Currently for java 21 it fails like
{noformat}
Nov 04 03:03:08 Caused by:
org.apache.flink.connector.testframe.container.ImageBuildException: Failed to
build image "flink-configured-jobmanager"
Nov 04 03:03:08 at
org.apache.flink.connector.testframe.container.FlinkImageBuilder.build(FlinkImageBuilder.java:234)
Nov 04 03:03:08 at
org.apache.flink.connector.testframe.container.FlinkTestcontainersConfigurator.configureJobManagerContainer(FlinkTestcontainersConfigurator.java:65)
Nov 04 03:03:08 ... 61 more
Nov 04 03:03:08 Caused by: java.lang.IllegalStateException: Unexpected Java
version: 21
Nov 04 03:03:08 at
org.apache.flink.connector.testframe.container.FlinkImageBuilder.getJavaVersionSuffix(FlinkImageBuilder.java:284)
Nov 04 03:03:08 at
org.apache.flink.connector.testframe.container.FlinkImageBuilder.lambda$buildBaseImage$3(FlinkImageBuilder.java:250)
Nov 04 03:03:08 at
org.testcontainers.images.builder.traits.DockerfileTrait.withDockerfileFromBuilder(DockerfileTrait.java:19)
Nov 04 03:03:08 at
org.apache.flink.connector.testframe.container.FlinkImageBuilder.buildBaseImage(FlinkImageBuilder.java:246)
Nov 04 03:03:08 at
org.apache.flink.connector.testframe.container.FlinkImageBuilder.build(FlinkImageBuilder.java:206)
Nov 04 03:03:08 ... 62 more
Nov 04 03:03:08
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33456) Upgrade maven-shade-plugin to 3.5.1
Sergey Nuyanzin created FLINK-33456: --- Summary: Upgrade maven-shade-plugin to 3.5.1 Key: FLINK-33456 URL: https://issues.apache.org/jira/browse/FLINK-33456 Project: Flink Issue Type: Sub-task Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Current plugin version does not support jdk20+ The support was added within MSHADE-454 and released with 3.5.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33453) Add java21-target profile
Sergey Nuyanzin created FLINK-33453: --- Summary: Add java21-target profile Key: FLINK-33453 URL: https://issues.apache.org/jira/browse/FLINK-33453 Project: Flink Issue Type: Sub-task Components: Build System Reporter: Sergey Nuyanzin Add a new profile analogous to the java11-target and java17-target profiles. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33394) DataGeneratorSourceITCase.testGatedRateLimiter fails on AZP
Sergey Nuyanzin created FLINK-33394:
---
Summary: DataGeneratorSourceITCase.testGatedRateLimiter fails on
AZP
Key: FLINK-33394
URL: https://issues.apache.org/jira/browse/FLINK-33394
Project: Flink
Issue Type: Bug
Components: Connectors / Common
Affects Versions: 1.17.2
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=54054&view=logs&j=5cae8624-c7eb-5c51-92d3-4d2dacedd221&t=5acec1b4-945b-59ca-34f8-168928ce5199&l=22927
fails on AZP as
{noformat}
Oct 26 07:37:41 [1L, 1L, 1L, 1L, 1L, 1L]
Oct 26 07:37:41 at
org.apache.flink.connector.datagen.source.DataGeneratorSourceITCase.testGatedRateLimiter(DataGeneratorSourceITCase.java:200)
Oct 26 07:37:41 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
Oct 26 07:37:41 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Oct 26 07:37:41 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Oct 26 07:37:41 at java.lang.reflect.Method.invoke(Method.java:498)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33369) Wordcount is failing on AZP with jdk17
Sergey Nuyanzin created FLINK-33369:
---
Summary: Wordcount is failing on AZP with jdk17
Key: FLINK-33369
URL: https://issues.apache.org/jira/browse/FLINK-33369
Project: Flink
Issue Type: Bug
Reporter: Sergey Nuyanzin
a number of builds start failing with that
since 25/10/2023 e.g.
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=54014&view=logs&j=64debf87-ecdb-5aef-788d-8720d341b5cb&t=2302fb98-0839-5df2-3354-bbae636f81a7&l=5021]
is failing as
{noformat}
Oct 25 03:36:24 job-cluster_1 | Starting Job Manager
Oct 25 03:36:24 job-cluster_1 | [ERROR] The execution result is empty.
Oct 25 03:36:24 job-cluster_1 | [ERROR] Could not get JVM parameters and
dynamic configurations properly.
Oct 25 03:36:24 job-cluster_1 | [ERROR] Raw output from BashJavaUtils:
Oct 25 03:36:24 job-cluster_1 | Error: LinkageError occurred while loading
main class org.apache.flink.runtime.util.bash.BashJavaUtils
Oct 25 03:36:24 job-cluster_1 |java.lang.UnsupportedClassVersionError:
org/apache/flink/runtime/util/bash/BashJavaUtils has been compiled by a more
recent version of the Java Runtime (class file version 61.0), this version of
the Java Runtime only recognizes class file versions up to 55.0
Oct 25 03:36:24 Attaching to container-scripts_taskmanager_1
Oct 25 03:36:24 taskmanager_1 | Starting Task Manager
Oct 25 03:36:24 taskmanager_1 | [ERROR] The execution result is empty.
Oct 25 03:36:24 taskmanager_1 | [ERROR] Could not get JVM parameters and
dynamic configurations properly.
Oct 25 03:36:24 taskmanager_1 | [ERROR] Raw output from BashJavaUtils:
Oct 25 03:36:24 taskmanager_1 | Error: LinkageError occurred while loading
main class org.apache.flink.runtime.util.bash.BashJavaUtils
Oct 25 03:36:24 taskmanager_1 |java.lang.UnsupportedClassVersionError:
org/apache/flink/runtime/util/bash/BashJavaUtils has been compiled by a more
recent version of the Java Runtime (class file version 61.0), this version of
the Java Runtime only recognizes class file versions up to 55.0
Oct 25 03:36:24 Checking for exceptions...
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33346) DispatcherResourceCleanupTest.testFatalErrorIfJobCannotBeMarkedDirtyInJobResultStore fails on AZP
Sergey Nuyanzin created FLINK-33346:
---
Summary:
DispatcherResourceCleanupTest.testFatalErrorIfJobCannotBeMarkedDirtyInJobResultStore
fails on AZP
Key: FLINK-33346
URL: https://issues.apache.org/jira/browse/FLINK-33346
Project: Flink
Issue Type: Technical Debt
Components: Runtime / Coordination
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53905&view=logs&j=0da23115-68bb-5dcd-192c-bd4c8adebde1&t=24c3384f-1bcb-57b3-224f-51bf973bbee8&l=6800
failed with
{noformat}
Oct 22 00:59:32 Caused by: java.io.IOException: Expected IOException.
Oct 22 00:59:32 at
org.apache.flink.runtime.dispatcher.DispatcherResourceCleanupTest.lambda$testFatalErrorIfJobCannotBeMarkedDirtyInJobResultStore$6(DispatcherResourceCleanupTest.java:558)
Oct 22 00:59:32 at
org.apache.flink.runtime.testutils.TestingJobResultStore.createDirtyResultAsync(TestingJobResultStore.java:81)
Oct 22 00:59:32 at
org.apache.flink.runtime.dispatcher.Dispatcher.createDirtyJobResultEntryAsync(Dispatcher.java:1441)
Oct 22 00:59:32 at
org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createDirtyJobResultEntryIfMissingAsync$45(Dispatcher.java:1422)
Oct 22 00:59:32 at
java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995)
Oct 22 00:59:32 ... 39 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33340) Bump Jackson to 2.15.3
Sergey Nuyanzin created FLINK-33340: --- Summary: Bump Jackson to 2.15.3 Key: FLINK-33340 URL: https://issues.apache.org/jira/browse/FLINK-33340 Project: Flink Issue Type: Technical Debt Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin Among others there is a number of improvements regarding parsing of numbers (jackson-core) https://github.com/FasterXML/jackson-core/blob/2.16/release-notes/VERSION-2.x -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33334) A number of json plan tests fail with comparisonfailure
Sergey Nuyanzin created FLINK-4:
---
Summary: A number of json plan tests fail with comparisonfailure
Key: FLINK-4
URL: https://issues.apache.org/jira/browse/FLINK-4
Project: Flink
Issue Type: Sub-task
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
for instance
{noformat}
[ERROR]
org.apache.flink.table.planner.plan.nodes.exec.stream.SortJsonPlanTest.testSort
Time elapsed: 0.037 s <<< FAILURE!
org.junit.ComparisonFailure:
expected:<...alse",
"[table-sink-class" : "DEFAULT",
"connector" : "values]"
}
...> but was:<...alse",
"[connector" : "values",
"table-sink-class" : "DEFAULT]"
}
...>
at org.junit.Assert.assertEquals(Assert.java:117)
at org.junit.Assert.assertEquals(Assert.java:146)
at
org.apache.flink.table.planner.utils.TableTestUtilBase.doVerifyJsonPlan(TableTestBase.scala:846)
at
org.apache.flink.table.planner.utils.TableTestUtilBase.verifyJsonPlan(TableTestBase.scala:813)
at
org.apache.flink.table.planner.plan.nodes.exec.stream.SortJsonPlanTest.testSort(SortJsonPlanTest.java:64)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33333) TableSinkJsonPlanTest fails as JsonParseException: Unexpected character ('"' (code 34)): was expecting comma
Sergey Nuyanzin created FLINK-3:
---
Summary: TableSinkJsonPlanTest fails as JsonParseException:
Unexpected character ('"' (code 34)): was expecting comma
Key: FLINK-3
URL: https://issues.apache.org/jira/browse/FLINK-3
Project: Flink
Issue Type: Sub-task
Components: Table SQL / Planner
Reporter: Sergey Nuyanzin
{{org.apache.flink.table.planner.plan.nodes.exec.stream.TableSinkJsonPlanTest#testCdcWithNonDeterministicFuncSinkWithDifferentPk}}
fails as
{noformat}
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParseException:
Unexpected character ('"' (code 34)): was expecting comma to separate Object
entries
at [Source: (String)"{
"flinkVersion" : "",
"nodes" : [ {
"id" : 1,
"type" : "stream-exec-table-source-scan_1",
"scanTableSource" : {
"table" : {
"identifier" : "`default_catalog`.`default_database`.`users`",
"resolvedTable" : {
"schema" : {
"columns" : [ {
"name" : "user_id",
"dataType" : "VARCHAR(2147483647) NOT NULL"
}, {
"name" : "user_name",
"dataType" : "VARCHAR(2147483647)"
"[truncated 4287 chars]; line: 34, column: 14]
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2418)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:749)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:673)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipComma(ReaderBasedJsonParser.java:2459)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextFieldName(ReaderBasedJsonParser.java:950)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:537)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:98)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:23)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4706)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2989)
at
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser.readValueAsTree(JsonParser.java:2392)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33332) AssertionError: Exception in test: sun.misc.Unsafe or java.nio.DirectByteBuffer.(long, int) not available
Sergey Nuyanzin created FLINK-2:
---
Summary: AssertionError: Exception in test: sun.misc.Unsafe or
java.nio.DirectByteBuffer.(long, int) not available
Key: FLINK-2
URL: https://issues.apache.org/jira/browse/FLINK-2
Project: Flink
Issue Type: Sub-task
Components: API / Python
Reporter: Sergey Nuyanzin
A couple of tests fail like
{noformat}
java.lang.AssertionError: Exception in test: sun.misc.Unsafe or
java.nio.DirectByteBuffer.(long, int) not available
at
org.apache.flink.table.runtime.arrow.ArrowReaderWriterTestBase.testBasicFunctionality(ArrowReaderWriterTestBase.java:86)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33331) NoClassDefFoundError: Could not initialize class org.apache.flink.table.runtime.arrow.serializers.ArrowSerializer
Sergey Nuyanzin created FLINK-1:
---
Summary: NoClassDefFoundError: Could not initialize class
org.apache.flink.table.runtime.arrow.serializers.ArrowSerializer
Key: FLINK-1
URL: https://issues.apache.org/jira/browse/FLINK-1
Project: Flink
Issue Type: Sub-task
Components: API / Python
Reporter: Sergey Nuyanzin
A number of tests fails like e.g.for
{{org.apache.flink.table.runtime.operators.python.scalar.arrow.ArrowPythonScalarFunctionOpe}}
{noformat}
[ERROR] ratorTest.testFinishBundleTriggeredByTime Time elapsed: 0.031 s <<<
ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.flink.table.runtime.arrow.serializers.ArrowSerializer
at
org.apache.flink.table.runtime.operators.python.scalar.arrow.ArrowPythonScalarFunctionOperator.open(ArrowPythonScalarFunctionOperator.java:72)
at
org.apache.flink.streaming.util.AbstractStreamOperatorTestHarness.open(AbstractStreamOperatorTestHarness.java:681)
at
org.apache.flink.table.runtime.operators.python.scalar.PythonScalarFunctionOperatorTestBase.testFinishBundleTriggeredByTime(PythonScalarFunctionOperatorTestBase.java:156)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-33330) Bump zookeeper to address CVE-2023-44981
Sergey Nuyanzin created FLINK-0: --- Summary: Bump zookeeper to address CVE-2023-44981 Key: FLINK-0 URL: https://issues.apache.org/jira/browse/FLINK-0 Project: Flink Issue Type: Technical Debt Reporter: Sergey Nuyanzin There is a [CVE-2023-44981|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-44981] which is fixed in 3.7.2, 3.8.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33329) Bump commons-compress to address CVEs
Sergey Nuyanzin created FLINK-33329: --- Summary: Bump commons-compress to address CVEs Key: FLINK-33329 URL: https://issues.apache.org/jira/browse/FLINK-33329 Project: Flink Issue Type: Technical Debt Reporter: Sergey Nuyanzin current version has CVE-2023-42503 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33323) HybridShuffleITCase fails with produced an uncaught exception in FatalExitExceptionHandler
Sergey Nuyanzin created FLINK-33323:
---
Summary: HybridShuffleITCase fails with produced an uncaught
exception in FatalExitExceptionHandler
Key: FLINK-33323
URL: https://issues.apache.org/jira/browse/FLINK-33323
Project: Flink
Issue Type: Bug
Components: Runtime / Network
Affects Versions: 1.19.0
Reporter: Sergey Nuyanzin
This build
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53853&view=logs&j=a596f69e-60d2-5a4b-7d39-dc69e4cdaed3&t=712ade8c-ca16-5b76-3acd-14df33bc1cb1&l=9166
fails with
{noformat}
01:15:38,516 [blocking-shuffle-io-thread-4] ERROR
org.apache.flink.util.FatalExitExceptionHandler [] - FATAL: Thread
'blocking-shuffle-io-thread-4' produced an uncaught exception. Stopping the
process...
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@4275bb45[Not
completed, task =
java.util.concurrent.Executors$RunnableAdapter@488dd035[Wrapped task =
org.apache.fl
ink.runtime.io.network.partition.hybrid.tiered.tier.disk.DiskIOScheduler$$Lambda$2561/0x000801a2f728@464a3754]]
rejected from
java.util.concurrent.ScheduledThreadPoolExecutor@22747816[Shutting down, pool
size = 10, active threads = 9,
queued tasks = 1, completed tasks = 1]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
~[?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340)
~[?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562)
~[?:?]
at
org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.disk.DiskIOScheduler.run(DiskIOScheduler.java:151)
~[flink-runtime-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
at
org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.disk.DiskIOScheduler.lambda$triggerScheduling$0(DiskIOScheduler.java:308)
~[flink-runtime-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
{noformat}
also logs are attached
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
