[jira] [Created] (FLINK-35507) Support For Individual Job Level Resource Allocation in Session Cluster in k8s
Amarjeet created FLINK-35507: Summary: Support For Individual Job Level Resource Allocation in Session Cluster in k8s Key: FLINK-35507 URL: https://issues.apache.org/jira/browse/FLINK-35507 Project: Flink Issue Type: New Feature Components: Deployment / Kubernetes Reporter: Amarjeet We can have a setup like Spark where in Spark Cluster we can set individual job level setting in a spark cluster to access the resouces from memory to core. Also Support Dynamic memory allocation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35506) disable kafka auto-commit and rely on flink’s checkpointing if both are enabled
elon_X created FLINK-35506:
--
Summary: disable kafka auto-commit and rely on flink’s
checkpointing if both are enabled
Key: FLINK-35506
URL: https://issues.apache.org/jira/browse/FLINK-35506
Project: Flink
Issue Type: Improvement
Components: Connectors / Kafka
Affects Versions: 1.16.1
Reporter: elon_X
When I use KafkaSource for consuming topics and set the Kafka parameter
{{{}enable.auto.commit=true{}}}, while also enabling checkpointing for the
task, I notice that both will commit offsets. Should Kafka's auto-commit be
disabled when enabling Flink checkpointing, similar to how it's done with
FlinkKafkaConsumer?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (FLINK-35323) Only the schema of the first hit table is recorded when the source-table of the transformer hits multiple tables
[
https://issues.apache.org/jira/browse/FLINK-35323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851498#comment-17851498
]
Leonard Xu edited comment on FLINK-35323 at 6/3/24 4:06 AM:
master: 88afc5f18d051b52da86b5017dc896a4a66e6e10
3.1: f4045fb8e78d905c0c7494a0485b1aca40a12f62
was (Author: leonard xu):
master: 88afc5f18d051b52da86b5017dc896a4a66e6e10
> Only the schema of the first hit table is recorded when the source-table of
> the transformer hits multiple tables
>
>
> Key: FLINK-35323
> URL: https://issues.apache.org/jira/browse/FLINK-35323
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
>Affects Versions: cdc-3.1.0
>Reporter: Wenkai Qi
>Assignee: yux
>Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.2.0, cdc-3.1.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> {code:java}
> transform:
> - source-table: mydb.web_\.*
> projection: \*, localtimestamp as new_timestamp
> description: project fields from source table {code}
> Table mydb.web_order: col1, col2, col3
> Table mydb.web_info: col1, col4
> If transform data operator processes `mydb.web_info` first and then
> `mydb.web_order`, its schema will always be `col1, col4`.
> Cause by: TransformDataOperator.java
> {code:java}
> private transient Map
> transformProjectionProcessorMap;
> private transient Map
> transformFilterProcessorMap; {code}
> The relationship of `TableId` is missing here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (FLINK-35323) Only the schema of the first hit table is recorded when the source-table of the transformer hits multiple tables
[
https://issues.apache.org/jira/browse/FLINK-35323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu resolved FLINK-35323.
Resolution: Fixed
> Only the schema of the first hit table is recorded when the source-table of
> the transformer hits multiple tables
>
>
> Key: FLINK-35323
> URL: https://issues.apache.org/jira/browse/FLINK-35323
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
>Affects Versions: cdc-3.1.0
>Reporter: Wenkai Qi
>Assignee: yux
>Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.2.0, cdc-3.1.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> {code:java}
> transform:
> - source-table: mydb.web_\.*
> projection: \*, localtimestamp as new_timestamp
> description: project fields from source table {code}
> Table mydb.web_order: col1, col2, col3
> Table mydb.web_info: col1, col4
> If transform data operator processes `mydb.web_info` first and then
> `mydb.web_order`, its schema will always be `col1, col4`.
> Cause by: TransformDataOperator.java
> {code:java}
> private transient Map
> transformProjectionProcessorMap;
> private transient Map
> transformFilterProcessorMap; {code}
> The relationship of `TableId` is missing here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35323) Only the schema of the first hit table is recorded when the source-table of the transformer hits multiple tables
[
https://issues.apache.org/jira/browse/FLINK-35323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851498#comment-17851498
]
Leonard Xu commented on FLINK-35323:
master: 88afc5f18d051b52da86b5017dc896a4a66e6e10
> Only the schema of the first hit table is recorded when the source-table of
> the transformer hits multiple tables
>
>
> Key: FLINK-35323
> URL: https://issues.apache.org/jira/browse/FLINK-35323
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
>Affects Versions: cdc-3.1.0
>Reporter: Wenkai Qi
>Assignee: yux
>Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.2.0, cdc-3.1.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> {code:java}
> transform:
> - source-table: mydb.web_\.*
> projection: \*, localtimestamp as new_timestamp
> description: project fields from source table {code}
> Table mydb.web_order: col1, col2, col3
> Table mydb.web_info: col1, col4
> If transform data operator processes `mydb.web_info` first and then
> `mydb.web_order`, its schema will always be `col1, col4`.
> Cause by: TransformDataOperator.java
> {code:java}
> private transient Map
> transformProjectionProcessorMap;
> private transient Map
> transformFilterProcessorMap; {code}
> The relationship of `TableId` is missing here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [BP-3.1][FLINK-35323][runtime] Fix transform failure when one rule matches multiple schemas [flink-cdc]
leonardBang merged PR #3312: URL: https://github.com/apache/flink-cdc/pull/3312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-35323][runtime] Fix transform failure when one rule matches multiple tables with incompatible schema [flink-cdc]
leonardBang merged PR #3313: URL: https://github.com/apache/flink-cdc/pull/3313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-35200][table] Support the execution of suspend, resume materialized table in full refresh mode [flink]
lsyldliu commented on code in PR #24877:
URL: https://github.com/apache/flink/pull/24877#discussion_r1623739592
##
flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/service/materializedtable/MaterializedTableManager.java:
##
@@ -302,21 +315,47 @@ private ResultFetcher callAlterMaterializedTableSuspend(
refreshHandler.getJobId(),
savepointPath);
-CatalogMaterializedTable updatedMaterializedTable =
-materializedTable.copy(
-CatalogMaterializedTable.RefreshStatus.SUSPENDED,
-
materializedTable.getRefreshHandlerDescription().orElse(null),
-serializeContinuousHandler(updateRefreshHandler));
-List tableChanges = new ArrayList<>();
-tableChanges.add(
-
TableChange.modifyRefreshStatus(CatalogMaterializedTable.RefreshStatus.ACTIVATED));
-AlterMaterializedTableChangeOperation
alterMaterializedTableChangeOperation =
-new AlterMaterializedTableChangeOperation(
-tableIdentifier, tableChanges,
updatedMaterializedTable);
+updateRefreshHandler(
+operationExecutor,
+handle,
+tableIdentifier,
+materializedTable,
+CatalogMaterializedTable.RefreshStatus.SUSPENDED,
+updateRefreshHandler.asSummaryString(),
+serializeContinuousHandler(updateRefreshHandler));
+}
-operationExecutor.callExecutableOperation(handle,
alterMaterializedTableChangeOperation);
+private void suspendRefreshWorkflow(
+OperationExecutor operationExecutor,
+OperationHandle handle,
+ObjectIdentifier tableIdentifier,
+CatalogMaterializedTable materializedTable) {
-return ResultFetcher.fromTableResult(handle, TABLE_RESULT_OK, false);
+try {
+RefreshHandlerSerializer refreshHandlerSerializer =
+workflowScheduler.getRefreshHandlerSerializer();
Review Comment:
You should check the `workflowScheduler` is not null before suspending.
##
flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/service/materializedtable/MaterializedTableManager.java:
##
@@ -328,24 +367,81 @@ private ResultFetcher callAlterMaterializedTableResume(
getCatalogMaterializedTable(operationExecutor,
tableIdentifier);
if (CatalogMaterializedTable.RefreshMode.CONTINUOUS
-!= catalogMaterializedTable.getRefreshMode()) {
-throw new SqlExecutionException(
-"Only support resume continuous refresh job currently.");
+== catalogMaterializedTable.getRefreshMode()) {
+resumeContinuousRefreshJob(
+operationExecutor,
+handle,
+tableIdentifier,
+catalogMaterializedTable,
+op.getDynamicOptions());
+} else {
+resumeRefreshWorkflow(
+operationExecutor,
+handle,
+tableIdentifier,
+catalogMaterializedTable,
+op.getDynamicOptions());
}
-ContinuousRefreshHandler continuousRefreshHandler =
+return ResultFetcher.fromTableResult(handle, TABLE_RESULT_OK, false);
+}
+
+private void resumeContinuousRefreshJob(
+OperationExecutor operationExecutor,
+OperationHandle handle,
+ObjectIdentifier tableIdentifier,
+CatalogMaterializedTable catalogMaterializedTable,
+Map dynamicOptions) {
+ContinuousRefreshHandler refreshHandler =
deserializeContinuousHandler(
catalogMaterializedTable.getSerializedRefreshHandler());
-Optional restorePath =
continuousRefreshHandler.getRestorePath();
+
+Optional restorePath = refreshHandler.getRestorePath();
executeContinuousRefreshJob(
operationExecutor,
handle,
catalogMaterializedTable,
tableIdentifier,
-op.getDynamicOptions(),
+dynamicOptions,
restorePath);
+}
-return ResultFetcher.fromTableResult(handle, TABLE_RESULT_OK, false);
+private void resumeRefreshWorkflow(
+OperationExecutor operationExecutor,
+OperationHandle handle,
+ObjectIdentifier tableIdentifier,
+CatalogMaterializedTable catalogMaterializedTable,
+Map dynamicOptions) {
+try {
+RefreshHandlerSerializer refreshHandlerSerializer =
Review Comment:
```suggestion
RefreshHandlerSerializer refreshHandlerSerializer =
```
##
Re: [PR] [FLINK-34482][checkpoint] Rename checkpointing options [flink]
flinkbot commented on PR #24878: URL: https://github.com/apache/flink/pull/24878#issuecomment-2144225118 ## CI report: * 965dcaeede05ed6a4f55369fa5c0227377505a22 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [FLINK-34482][checkpoint] Rename checkpointing options [flink]
masteryhx opened a new pull request, #24878: URL: https://github.com/apache/flink/pull/24878 ## What is the purpose of the change Rename checkpoint related options ## Brief change log - Rename checkpoint related options with prefix 'execution.checkpointing' ## Verifying this change This change is a trivial rework already covered by existing tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34482) Rename options for checkpointing
[ https://issues.apache.org/jira/browse/FLINK-34482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34482: --- Labels: pull-request-available (was: ) > Rename options for checkpointing > > > Key: FLINK-34482 > URL: https://issues.apache.org/jira/browse/FLINK-34482 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Checkpointing >Reporter: Zakelly Lan >Assignee: Hangxiang Yu >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-35129][cdc][postgres] Add checkpoint cycle option for commit offsets [flink-cdc]
leonardBang commented on code in PR #3349: URL: https://github.com/apache/flink-cdc/pull/3349#discussion_r1623742652 ## docs/content/docs/connectors/flink-sources/postgres-cdc.md: ## @@ -236,6 +236,15 @@ Connector Options so it does not need to be explicitly configured 'execution.checkpointing.checkpoints-after-tasks-finish.enabled' = 'true' + + scan.lsn-commit.checkpoints-num-delay + optional + 3 + Integer + The number of checkpoint delays before starting to commit the LSN offsets. + The checkpoint LSN offsets will be committed in rolling fashion, the earliest checkpoint identifier will be committed first from the delayed checkpoints. Review Comment: Could you explain to users why we need delay the offset commit? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-35129][cdc][postgres] Add checkpoint cycle option for commit offsets [flink-cdc]
leonardBang commented on PR #3349: URL: https://github.com/apache/flink-cdc/pull/3349#issuecomment-2144201121 adding note on the docs makes sense to me as the old API is marked as @Deprecated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-35503) OracleE2eITCase fails with error ORA-12528 on Mac M2
[
https://issues.apache.org/jira/browse/FLINK-35503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851486#comment-17851486
]
Zhongqiang Gong commented on FLINK-35503:
-
[~xiqian_yu] Let me take this issue. Because It have some change to orginal.
> OracleE2eITCase fails with error ORA-12528 on Mac M2
>
>
> Key: FLINK-35503
> URL: https://issues.apache.org/jira/browse/FLINK-35503
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
>Affects Versions: cdc-3.1.0
> Environment:
> * Mac M2 (Apple Silicon)
> * using docker desktop with Rosetta enabled for amd64 emulation
>
>Reporter: Saketh Kurnool
>Priority: Blocker
> Attachments: com.ververica.cdc.connectors.tests.OracleE2eITCase.txt,
> oracle-docker-setup-logs.txt
>
>
> Hello Flink CDC community,
> I am attempting to run `OracleE2eITCase` (in the cdc source connector e2e
> tests), and I am running into the following runtime exception:
> {code:java}
> java.sql.SQLException:
> Listener refused the connection with the following error:
> ORA-12528, TNS:listener: all appropriate instances are blocking new
> connections
>
> at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:854)
> at
> oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793)
> at
> oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57)
> at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747)
> at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
> at
> com.ververica.cdc.connectors.tests.OracleE2eITCase.getOracleJdbcConnection(OracleE2eITCase.java:197)
> at
> com.ververica.cdc.connectors.tests.OracleE2eITCase.testOracleCDC(OracleE2eITCase.java:149)
> at java.base/java.lang.reflect.Method.invoke(Method.java:567)
> at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> at
> org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
> Caused by: oracle.net.ns.NetException: Listener refused the connection with
> the following error:
> ORA-12528, TNS:listener: all appropriate instances are blocking new
> connections
>
> at oracle.net.ns.NSProtocolNIO.negotiateConnection(NSProtocolNIO.java:284)
> at oracle.net.ns.NSProtocol.connect(NSProtocol.java:340)
> at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1596)
> at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:588)
> ... 11 more{code}
> I have attached the test results to this issue.
> `OracleE2eITCase` runs the `goodboy008/oracle-19.3.0-ee:non-cdb` docker
> image. I am able to reproduce the same issue when I run this docker image
> locally - my observation is that dockerized Oracle DB instance is not being
> set up properly, as I notice another ORA in the setup logs (`ORA-03113:
> end-of-file on communication channel`). I have also attached the logs from
> the docker image setup to this issue. To reproduce the ORA-12528 issue
> locally, I:
> * ran: `docker run goodboy008/oracle-19.3.0-ee:non-cdb`
> * ssh'ed into the db pod
> * ran: `sqlplus sys/top_secret@//localhost:1521/ORCLCDB as sysdba`
> Any insight/workaround on getting this e2e test and the docker image running
> on my machine would be much appreciated. I'm also happy to provide any other
> information regarding my setup in the comments. Thank you!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-32081][checkpoint] Compatibility between file-merging on and off across job runs [flink]
Zakelly commented on code in PR #24873:
URL: https://github.com/apache/flink/pull/24873#discussion_r1623734579
##
flink-tests/src/test/java/org/apache/flink/test/checkpointing/SnapshotFileMergingCompatibilityITCase.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.test.checkpointing;
+
+import org.apache.flink.configuration.CheckpointingOptions;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend;
+import org.apache.flink.core.execution.RestoreMode;
+import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
+import org.apache.flink.test.util.MiniClusterWithClientResource;
+import org.apache.flink.util.TestLogger;
+
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.nio.file.Path;
+import java.util.Arrays;
+import java.util.Collection;
+
+import static
org.apache.flink.test.checkpointing.ResumeCheckpointManuallyITCase.runJobAndGetExternalizedCheckpoint;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * FileMerging Compatibility IT case which tests recovery from a checkpoint
created in different
+ * fileMerging mode (i.e. fileMerging enabled/disabled).
+ */
+public class SnapshotFileMergingCompatibilityITCase extends TestLogger {
+
+public static Collection parameters() {
+return Arrays.asList(
+new Object[][] {
+{RestoreMode.CLAIM, true},
+{RestoreMode.CLAIM, false},
+{RestoreMode.NO_CLAIM, true},
+{RestoreMode.NO_CLAIM, false}
+});
+}
+
+@ParameterizedTest(name = "RestoreMode = {0}, fileMergingAcrossBoundary =
{1}")
+@MethodSource("parameters")
+public void testSwitchFromDisablingToEnablingFileMerging(
+RestoreMode restoreMode, boolean fileMergingAcrossBoundary,
@TempDir Path checkpointDir)
+throws Exception {
+testSwitchingFileMerging(
+checkpointDir, false, true, restoreMode,
fileMergingAcrossBoundary);
+}
+
+@ParameterizedTest(name = "RestoreMode = {0}, fileMergingAcrossBoundary =
{1}")
+@MethodSource("parameters")
+public void testSwitchFromEnablingToDisablingFileMerging(
+RestoreMode restoreMode, boolean fileMergingAcrossBoundary,
@TempDir Path checkpointDir)
+throws Exception {
+testSwitchingFileMerging(
+checkpointDir, true, false, restoreMode,
fileMergingAcrossBoundary);
+}
+
+private void testSwitchingFileMerging(
+Path checkpointDir,
+boolean firstFileMergingSwitch,
+boolean secondFileMergingSwitch,
+RestoreMode restoreMode,
+boolean fileMergingAcrossBoundary)
+throws Exception {
+final Configuration config = new Configuration();
+config.set(CheckpointingOptions.CHECKPOINTS_DIRECTORY,
checkpointDir.toUri().toString());
+config.set(CheckpointingOptions.INCREMENTAL_CHECKPOINTS, true);
+config.set(CheckpointingOptions.FILE_MERGING_ACROSS_BOUNDARY,
fileMergingAcrossBoundary);
+config.set(CheckpointingOptions.FILE_MERGING_ENABLED,
firstFileMergingSwitch);
+MiniClusterWithClientResource firstCluster =
+new MiniClusterWithClientResource(
+new MiniClusterResourceConfiguration.Builder()
+.setConfiguration(config)
+.setNumberTaskManagers(2)
+.setNumberSlotsPerTaskManager(2)
+.build());
+EmbeddedRocksDBStateBackend stateBackend1 = new
EmbeddedRocksDBStateBackend();
+stateBackend1.configure(config,
Thread.currentThread().getContextClassLoader());
+firstCluster.before();
+String externalCheckpoint;
+try {
+externalCheckpoint =
+runJobAndGetExternalizedCheckpoint(
+stateBackend1, null, firstCluster,
Re: [PR] [FLINK-25537] [JUnit5 Migration] Module: flink-core with,Package: core [flink]
1996fanrui commented on PR #24685: URL: https://github.com/apache/flink/pull/24685#issuecomment-2144183912 > @flinkbot run azure Hi @GOODBOY008 , the CI cannot be triggered for this PR. Creating a new PR may be a workaround for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-35325][cdc-connector][paimon]Support for specifying column order. [flink-cdc]
yuxiqian commented on code in PR #3323:
URL: https://github.com/apache/flink-cdc/pull/3323#discussion_r1623725867
##
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-paimon/src/main/java/org/apache/flink/cdc/connectors/paimon/sink/PaimonMetadataApplier.java:
##
@@ -129,25 +133,113 @@ private void applyCreateTable(CreateTableEvent event)
private void applyAddColumn(AddColumnEvent event)
throws Catalog.TableNotExistException,
Catalog.ColumnAlreadyExistException,
Catalog.ColumnNotExistException {
-List tableChangeList = new ArrayList<>();
-event.getAddedColumns()
-.forEach(
-(column) -> {
-SchemaChange tableChange =
-SchemaChange.addColumn(
-column.getAddColumn().getName(),
-LogicalTypeConversion.toDataType(
-
DataTypeUtils.toFlinkDataType(
-
column.getAddColumn().getType())
-
.getLogicalType()));
-tableChangeList.add(tableChange);
-});
+List tableChangeList =
applyAddColumnEventWithPosition(event);
catalog.alterTable(
new Identifier(event.tableId().getSchemaName(),
event.tableId().getTableName()),
tableChangeList,
true);
}
+private List applyAddColumnEventWithPosition(AddColumnEvent
event)
+throws Catalog.TableNotExistException {
+List tableChangeList = new ArrayList<>();
+for (AddColumnEvent.ColumnWithPosition columnWithPosition :
event.getAddedColumns()) {
+SchemaChange tableChange;
+switch (columnWithPosition.getPosition()) {
+case FIRST:
+tableChange =
+SchemaChange.addColumn(
Review Comment:
What about declaring `applyAddColumnWithFirstPosition` and
`applyAddColumnWithAfterPosition` too for consistency? And maybe we can extract
some sharing logic (like column name, data conversion, and comments) to reduce
code duplication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-35503) OracleE2eITCase fails with error ORA-12528 on Mac M2
[
https://issues.apache.org/jira/browse/FLINK-35503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851479#comment-17851479
]
yux commented on FLINK-35503:
-
Hi [~sakkurn],
Currently [~gongzhongqiang] tweaks Oracle EE docker image for running E2e tests
and repackaged it as goodboy008/oracle-19.3.0-ee:non-cdb. However it supports
amd64 only, and requires Rosetta (or something similar) to run.
Oracle community discussions[2] revealed that ORA-03113 also appears when
running cross-architecture emulation, perhaps caused by a bug in Rosetta 2, but
is hard to investigate and debug since they're not open sourced.
As this PR [3] has brought official arm64 architecture support to Oracle docker
image, I think we can repackage Oracle E2e docker image to support both amd64
and arm64 architecture, which could help Mac developers run E2e tests easier.
I'd like to help implementing this if needed.
[1] https://hub.docker.com/r/gvenzl/oracle-xe/tags
[2] https://github.com/oracle/docker-images/discussions/1951
[3] https://github.com/oracle/docker-images/pull/2659
> OracleE2eITCase fails with error ORA-12528 on Mac M2
>
>
> Key: FLINK-35503
> URL: https://issues.apache.org/jira/browse/FLINK-35503
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
>Affects Versions: cdc-3.1.0
> Environment:
> * Mac M2 (Apple Silicon)
> * using docker desktop with Rosetta enabled for amd64 emulation
>
>Reporter: Saketh Kurnool
>Priority: Blocker
> Attachments: com.ververica.cdc.connectors.tests.OracleE2eITCase.txt,
> oracle-docker-setup-logs.txt
>
>
> Hello Flink CDC community,
> I am attempting to run `OracleE2eITCase` (in the cdc source connector e2e
> tests), and I am running into the following runtime exception:
> {code:java}
> java.sql.SQLException:
> Listener refused the connection with the following error:
> ORA-12528, TNS:listener: all appropriate instances are blocking new
> connections
>
> at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:854)
> at
> oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793)
> at
> oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57)
> at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747)
> at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
> at
> com.ververica.cdc.connectors.tests.OracleE2eITCase.getOracleJdbcConnection(OracleE2eITCase.java:197)
> at
> com.ververica.cdc.connectors.tests.OracleE2eITCase.testOracleCDC(OracleE2eITCase.java:149)
> at java.base/java.lang.reflect.Method.invoke(Method.java:567)
> at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> at
> org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
> Caused by: oracle.net.ns.NetException: Listener refused the connection with
> the following error:
> ORA-12528, TNS:listener: all appropriate instances are blocking new
> connections
>
> at oracle.net.ns.NSProtocolNIO.negotiateConnection(NSProtocolNIO.java:284)
> at oracle.net.ns.NSProtocol.connect(NSProtocol.java:340)
> at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1596)
> at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:588)
> ... 11 more{code}
> I have attached the test results to this issue.
> `OracleE2eITCase` runs the `goodboy008/oracle-19.3.0-ee:non-cdb` docker
> image. I am able to reproduce the same issue when I run this docker image
> locally - my observation is that dockerized Oracle DB instance is not being
> set up properly, as I notice another ORA in the setup logs (`ORA-03113:
> end-of-file on communication channel`). I have also attached the logs from
> the docker image setup to this issue. To reproduce the ORA-12528 issue
> locally, I:
> * ran: `docker run goodboy008/oracle-19.3.0-ee:non-cdb`
> * ssh'ed into the db pod
> * ran: `sqlplus sys/top_secret@//localhost:1521/ORCLCDB as sysdba`
> Any insight/workaround on getting this e2e test and the docker image running
> on my machine would be much appreciated. I'm also happy to provide any other
> information regarding my setup in the comments. Thank you!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (FLINK-35496) The annotations of the new JDBC connector should be changed to non-Public/non-PublicEvolving
[
https://issues.apache.org/jira/browse/FLINK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851478#comment-17851478
]
Rui Fan edited comment on FLINK-35496 at 6/3/24 2:00 AM:
-
Merged to main(jdbc-3.2.0) via: b955e8b5479c65512f94ee7270eb2868a4a14ebe
was (Author: fanrui):
Merged to main(jdbc-3.3.0) via: b955e8b5479c65512f94ee7270eb2868a4a14ebe
> The annotations of the new JDBC connector should be changed to
> non-Public/non-PublicEvolving
>
>
> Key: FLINK-35496
> URL: https://issues.apache.org/jira/browse/FLINK-35496
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / JDBC
>Reporter: RocMarshal
>Assignee: João Boto
>Priority: Major
> Labels: pull-request-available
> Fix For: jdbc-3.2.0
>
>
> In general, we use the Experimental annotation instead of {{PublicEvolving}}
> or {{Public}} for new features or new apis. But {{JdbcSink}} and
> JdbcSource(merged ) was marked as {{PublicEvolving}} in the first version.
> [~fanrui] commented it to the original PR[1].[1]
> [https://github.com/apache/flink-connector-jdbc/pull/2#discussion_r1621857589]
> CC [~eskabetxe] [~Sergey Nuyanzin] [~fanrui]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (FLINK-35496) The annotations of the new JDBC connector should be changed to non-Public/non-PublicEvolving
[
https://issues.apache.org/jira/browse/FLINK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan resolved FLINK-35496.
-
Fix Version/s: jdbc-3.2.0
Resolution: Fixed
> The annotations of the new JDBC connector should be changed to
> non-Public/non-PublicEvolving
>
>
> Key: FLINK-35496
> URL: https://issues.apache.org/jira/browse/FLINK-35496
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / JDBC
>Reporter: RocMarshal
>Assignee: João Boto
>Priority: Major
> Labels: pull-request-available
> Fix For: jdbc-3.2.0
>
>
> In general, we use the Experimental annotation instead of {{PublicEvolving}}
> or {{Public}} for new features or new apis. But {{JdbcSink}} and
> JdbcSource(merged ) was marked as {{PublicEvolving}} in the first version.
> [~fanrui] commented it to the original PR[1].[1]
> [https://github.com/apache/flink-connector-jdbc/pull/2#discussion_r1621857589]
> CC [~eskabetxe] [~Sergey Nuyanzin] [~fanrui]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35496) The annotations of the new JDBC connector should be changed to non-Public/non-PublicEvolving
[
https://issues.apache.org/jira/browse/FLINK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851478#comment-17851478
]
Rui Fan commented on FLINK-35496:
-
Merged to main(jdbc-3.3.0) via: b955e8b5479c65512f94ee7270eb2868a4a14ebe
> The annotations of the new JDBC connector should be changed to
> non-Public/non-PublicEvolving
>
>
> Key: FLINK-35496
> URL: https://issues.apache.org/jira/browse/FLINK-35496
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / JDBC
>Reporter: RocMarshal
>Assignee: João Boto
>Priority: Major
> Labels: pull-request-available
>
> In general, we use the Experimental annotation instead of {{PublicEvolving}}
> or {{Public}} for new features or new apis. But {{JdbcSink}} and
> JdbcSource(merged ) was marked as {{PublicEvolving}} in the first version.
> [~fanrui] commented it to the original PR[1].[1]
> [https://github.com/apache/flink-connector-jdbc/pull/2#discussion_r1621857589]
> CC [~eskabetxe] [~Sergey Nuyanzin] [~fanrui]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-35496] Fix annotations on new JdbcSink and JdbcSource [flink-connector-jdbc]
1996fanrui merged PR #127: URL: https://github.com/apache/flink-connector-jdbc/pull/127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-34273) git fetch fails
[
https://issues.apache.org/jira/browse/FLINK-34273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851472#comment-17851472
]
Weijie Guo commented on FLINK-34273:
test_cron_hadoop313
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=60011=logs=fc5181b0-e452-5c8f-68de-1097947f6483=10163a1a-ea71-5414-a832-7701bff37ba3=971
> git fetch fails
> ---
>
> Key: FLINK-34273
> URL: https://issues.apache.org/jira/browse/FLINK-34273
> Project: Flink
> Issue Type: Bug
> Components: Build System / CI, Test Infrastructure
>Affects Versions: 1.19.0, 1.18.1, 1.20.0
>Reporter: Matthias Pohl
>Priority: Major
> Labels: test-stability
>
> We've seen multiple {{git fetch}} failures. I assume this to be an
> infrastructure issue. This Jira issue is for documentation purposes.
> {code:java}
> error: RPC failed; curl 18 transfer closed with outstanding read data
> remaining
> error: 5211 bytes of body are still expected
> fetch-pack: unexpected disconnect while reading sideband packet
> fatal: early EOF
> fatal: fetch-pack: invalid index-pack output {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57080=logs=0e7be18f-84f2-53f0-a32d-4a5e4a174679=5d6dc3d3-393d-5111-3a40-c6a5a36202e6=667
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34513) GroupAggregateRestoreTest.testRestore fails
[
https://issues.apache.org/jira/browse/FLINK-34513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851471#comment-17851471
]
Weijie Guo commented on FLINK-34513:
Hi [~bvarghese], Is there any progress about this?
> GroupAggregateRestoreTest.testRestore fails
> ---
>
> Key: FLINK-34513
> URL: https://issues.apache.org/jira/browse/FLINK-34513
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.20.0
>Reporter: Matthias Pohl
>Assignee: Bonnie Varghese
>Priority: Critical
> Labels: pull-request-available, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57828=logs=26b84117-e436-5720-913e-3e280ce55cae=77cc7e77-39a0-5007-6d65-4137ac13a471=10881
> {code}
> Feb 24 01:12:01 01:12:01.384 [ERROR] Tests run: 10, Failures: 1, Errors: 0,
> Skipped: 1, Time elapsed: 2.957 s <<< FAILURE! -- in
> org.apache.flink.table.planner.plan.nodes.exec.stream.GroupAggregateRestoreTest
> Feb 24 01:12:01 01:12:01.384 [ERROR]
> org.apache.flink.table.planner.plan.nodes.exec.stream.GroupAggregateRestoreTest.testRestore(TableTestProgram,
> ExecNodeMetadata)[4] -- Time elapsed: 0.653 s <<< FAILURE!
> Feb 24 01:12:01 java.lang.AssertionError:
> Feb 24 01:12:01
> Feb 24 01:12:01 Expecting actual:
> Feb 24 01:12:01 ["+I[3, 1, 2, 8, 31, 10.0, 3]",
> Feb 24 01:12:01 "+I[2, 1, 4, 14, 42, 7.0, 6]",
> Feb 24 01:12:01 "+I[1, 1, 4, 12, 24, 6.0, 4]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 8.0, 7]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 6.0, 5]",
> Feb 24 01:12:01 "+I[7, 0, 1, 7, 7, 7.0, 1]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 7.0, 7]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 5.0, 5]",
> Feb 24 01:12:01 "+U[3, 1, 2, 8, 31, 9.0, 3]",
> Feb 24 01:12:01 "+U[7, 0, 1, 7, 7, 7.0, 2]"]
> Feb 24 01:12:01 to contain exactly in any order:
> Feb 24 01:12:01 ["+I[3, 1, 2, 8, 31, 10.0, 3]",
> Feb 24 01:12:01 "+I[2, 1, 4, 14, 42, 7.0, 6]",
> Feb 24 01:12:01 "+I[1, 1, 4, 12, 24, 6.0, 4]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 8.0, 7]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 6.0, 5]",
> Feb 24 01:12:01 "+U[3, 1, 2, 8, 31, 9.0, 3]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 7.0, 7]",
> Feb 24 01:12:01 "+I[7, 0, 1, 7, 7, 7.0, 2]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 5.0, 5]"]
> Feb 24 01:12:01 elements not found:
> Feb 24 01:12:01 ["+I[7, 0, 1, 7, 7, 7.0, 2]"]
> Feb 24 01:12:01 and elements not expected:
> Feb 24 01:12:01 ["+I[7, 0, 1, 7, 7, 7.0, 1]", "+U[7, 0, 1, 7, 7, 7.0, 2]"]
> Feb 24 01:12:01
> Feb 24 01:12:01 at
> org.apache.flink.table.planner.plan.nodes.exec.testutils.RestoreTestBase.testRestore(RestoreTestBase.java:313)
> Feb 24 01:12:01 at
> java.base/java.lang.reflect.Method.invoke(Method.java:580)
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34513) GroupAggregateRestoreTest.testRestore fails
[
https://issues.apache.org/jira/browse/FLINK-34513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851470#comment-17851470
]
Weijie Guo commented on FLINK-34513:
1.20 test_cron_jdk21 table
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=60008=logs=26b84117-e436-5720-913e-3e280ce55cae=77cc7e77-39a0-5007-6d65-4137ac13a471=11513
> GroupAggregateRestoreTest.testRestore fails
> ---
>
> Key: FLINK-34513
> URL: https://issues.apache.org/jira/browse/FLINK-34513
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.20.0
>Reporter: Matthias Pohl
>Assignee: Bonnie Varghese
>Priority: Critical
> Labels: pull-request-available, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57828=logs=26b84117-e436-5720-913e-3e280ce55cae=77cc7e77-39a0-5007-6d65-4137ac13a471=10881
> {code}
> Feb 24 01:12:01 01:12:01.384 [ERROR] Tests run: 10, Failures: 1, Errors: 0,
> Skipped: 1, Time elapsed: 2.957 s <<< FAILURE! -- in
> org.apache.flink.table.planner.plan.nodes.exec.stream.GroupAggregateRestoreTest
> Feb 24 01:12:01 01:12:01.384 [ERROR]
> org.apache.flink.table.planner.plan.nodes.exec.stream.GroupAggregateRestoreTest.testRestore(TableTestProgram,
> ExecNodeMetadata)[4] -- Time elapsed: 0.653 s <<< FAILURE!
> Feb 24 01:12:01 java.lang.AssertionError:
> Feb 24 01:12:01
> Feb 24 01:12:01 Expecting actual:
> Feb 24 01:12:01 ["+I[3, 1, 2, 8, 31, 10.0, 3]",
> Feb 24 01:12:01 "+I[2, 1, 4, 14, 42, 7.0, 6]",
> Feb 24 01:12:01 "+I[1, 1, 4, 12, 24, 6.0, 4]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 8.0, 7]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 6.0, 5]",
> Feb 24 01:12:01 "+I[7, 0, 1, 7, 7, 7.0, 1]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 7.0, 7]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 5.0, 5]",
> Feb 24 01:12:01 "+U[3, 1, 2, 8, 31, 9.0, 3]",
> Feb 24 01:12:01 "+U[7, 0, 1, 7, 7, 7.0, 2]"]
> Feb 24 01:12:01 to contain exactly in any order:
> Feb 24 01:12:01 ["+I[3, 1, 2, 8, 31, 10.0, 3]",
> Feb 24 01:12:01 "+I[2, 1, 4, 14, 42, 7.0, 6]",
> Feb 24 01:12:01 "+I[1, 1, 4, 12, 24, 6.0, 4]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 8.0, 7]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 6.0, 5]",
> Feb 24 01:12:01 "+U[3, 1, 2, 8, 31, 9.0, 3]",
> Feb 24 01:12:01 "+U[2, 1, 4, 14, 57, 7.0, 7]",
> Feb 24 01:12:01 "+I[7, 0, 1, 7, 7, 7.0, 2]",
> Feb 24 01:12:01 "+U[1, 1, 4, 12, 32, 5.0, 5]"]
> Feb 24 01:12:01 elements not found:
> Feb 24 01:12:01 ["+I[7, 0, 1, 7, 7, 7.0, 2]"]
> Feb 24 01:12:01 and elements not expected:
> Feb 24 01:12:01 ["+I[7, 0, 1, 7, 7, 7.0, 1]", "+U[7, 0, 1, 7, 7, 7.0, 2]"]
> Feb 24 01:12:01
> Feb 24 01:12:01 at
> org.apache.flink.table.planner.plan.nodes.exec.testutils.RestoreTestBase.testRestore(RestoreTestBase.java:313)
> Feb 24 01:12:01 at
> java.base/java.lang.reflect.Method.invoke(Method.java:580)
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-32523) NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted fails with timeout on AZP
[
https://issues.apache.org/jira/browse/FLINK-32523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851469#comment-17851469
]
Weijie Guo commented on FLINK-32523:
1.20 test_cron_hadoop313 tests
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=60006=logs=baf26b34-3c6a-54e8-f93f-cf269b32f802=8c9d126d-57d2-5a9e-a8c8-ff53f7b35cd9=9465
> NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted fails with timeout
> on AZP
> ---
>
> Key: FLINK-32523
> URL: https://issues.apache.org/jira/browse/FLINK-32523
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.16.2, 1.18.0, 1.17.1, 1.19.0, 1.20.0
>Reporter: Sergey Nuyanzin
>Assignee: Hangxiang Yu
>Priority: Critical
> Labels: pull-request-available, stale-assigned, test-stability
> Attachments: failure.log
>
>
> This build
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=50795=logs=39d5b1d5-3b41-54dc-6458-1e2ddd1cdcf3=0c010d0c-3dec-5bf1-d408-7b18988b1b2b=8638
> fails with timeout
> {noformat}
> Jul 03 01:26:35 org.junit.runners.model.TestTimedOutException: test timed out
> after 10 milliseconds
> Jul 03 01:26:35 at java.lang.Object.wait(Native Method)
> Jul 03 01:26:35 at java.lang.Object.wait(Object.java:502)
> Jul 03 01:26:35 at
> org.apache.flink.core.testutils.OneShotLatch.await(OneShotLatch.java:61)
> Jul 03 01:26:35 at
> org.apache.flink.test.checkpointing.NotifyCheckpointAbortedITCase.verifyAllOperatorsNotifyAborted(NotifyCheckpointAbortedITCase.java:198)
> Jul 03 01:26:35 at
> org.apache.flink.test.checkpointing.NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted(NotifyCheckpointAbortedITCase.java:189)
> Jul 03 01:26:35 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jul 03 01:26:35 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jul 03 01:26:35 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Jul 03 01:26:35 at java.lang.reflect.Method.invoke(Method.java:498)
> Jul 03 01:26:35 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> Jul 03 01:26:35 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Jul 03 01:26:35 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> Jul 03 01:26:35 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Jul 03 01:26:35 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> Jul 03 01:26:35 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> Jul 03 01:26:35 at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Jul 03 01:26:35 at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35413) VertexFinishedStateCheckerTest causes exit 239
[
https://issues.apache.org/jira/browse/FLINK-35413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851468#comment-17851468
]
Weijie Guo commented on FLINK-35413:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=60006=logs=77a9d8e1-d610-59b3-fc2a-4766541e0e33=125e07e7-8de0-5c6c-a541-a567415af3ef=11070
> VertexFinishedStateCheckerTest causes exit 239
> --
>
> Key: FLINK-35413
> URL: https://issues.apache.org/jira/browse/FLINK-35413
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.20.0
>Reporter: Ryan Skraba
>Priority: Critical
> Labels: test-stability
>
> 1.20 test_cron_azure core
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=59676=logs=77a9d8e1-d610-59b3-fc2a-4766541e0e33=125e07e7-8de0-5c6c-a541-a567415af3ef=9429
> {code}
> May 21 01:31:42 01:31:42.160 [ERROR]
> org.apache.flink.runtime.checkpoint.VertexFinishedStateCheckerTest
> May 21 01:31:42 01:31:42.160 [ERROR]
> org.apache.maven.surefire.booter.SurefireBooterForkException:
> ExecutionException The forked VM terminated without properly saying goodbye.
> VM crash or System.exit called?
> May 21 01:31:42 01:31:42.160 [ERROR] Command was /bin/sh -c cd
> '/__w/1/s/flink-runtime' && '/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java'
> '-XX:+UseG1GC' '-Xms256m' '-XX:+IgnoreUnrecognizedVMOptions'
> '--add-opens=java.base/java.util=ALL-UNNAMED'
> '--add-opens=java.base/java.lang=ALL-UNNAMED'
> '--add-opens=java.base/java.net=ALL-UNNAMED'
> '--add-opens=java.base/java.io=ALL-UNNAMED'
> '--add-opens=java.base/java.util.concurrent=ALL-UNNAMED' '-Xmx768m' '-jar'
> '/__w/1/s/flink-runtime/target/surefire/surefirebooter-20240521011847857_99.jar'
> '/__w/1/s/flink-runtime/target/surefire' '2024-05-21T01-15-09_325-jvmRun1'
> 'surefire-20240521011847857_97tmp' 'surefire_29-20240521011847857_98tmp'
> May 21 01:31:42 01:31:42.160 [ERROR] Error occurred in starting fork, check
> output in log
> May 21 01:31:42 01:31:42.160 [ERROR] Process Exit Code: 239
> May 21 01:31:42 01:31:42.160 [ERROR] Crashed tests:
> May 21 01:31:42 01:31:42.160 [ERROR]
> org.apache.flink.runtime.checkpoint.VertexFinishedStateCheckerTest
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:456)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkOnceMultiple(ForkStarter.java:358)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:296)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:250)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1240)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1089)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:905)
> May 21 01:31:42 01:31:42.160 [ERROR] at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
> {code}
> In the build artifact {{mvn-1.log}} the following FATAL error is found:
> {code}
> 01:19:08,584 [ pool-9-thread-1] ERROR
> org.apache.flink.util.FatalExitExceptionHandler [] - FATAL:
> Thread 'pool-9-thread-1' produced an uncaught exception. Stopping the
> process...
> java.util.concurrent.CompletionException:
> java.util.concurrent.RejectedExecutionException: Task
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@5ead9062
> rejected from
> java.util.concurrent.ScheduledThreadPoolExecutor@4d0e55ac[Shutting down, pool
> size = 1, active threads = 1, queued tasks = 1, completed tasks = 194]
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> ~[?:1.8.0_292]
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> ~[?:1.8.0_292]
> at
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:838)
> ~[?:1.8.0_292]
> at
> java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)
> ~[?:1.8.0_292]
> at
> java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:851)
> ~[?:1.8.0_292]
> at
> java.util.concurrent.CompletableFuture.handleAsync(CompletableFuture.java:2178)
> ~[?:1.8.0_292]
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer.allocateSlot(DefaultSlotStatusSyncer.java:138)
>
[jira] [Updated] (FLINK-35505) RegionFailoverITCase.testMultiRegionFailover has never ever restored state
[
https://issues.apache.org/jira/browse/FLINK-35505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weijie Guo updated FLINK-35505:
---
Description:
{code:java}
May 31 17:17:59 17:17:59.555 [ERROR]
org.apache.flink.test.checkpointing.RegionFailoverITCase.testMultiRegionFailover
-- Time elapsed: 27.92 s <<< FAILURE!
May 31 17:17:59 java.lang.AssertionError: The test multi-region job has never
ever restored state.
May 31 17:17:59 at org.junit.Assert.fail(Assert.java:89)
May 31 17:17:59 at org.junit.Assert.assertTrue(Assert.java:42)
May 31 17:17:59 at
org.apache.flink.test.checkpointing.RegionFailoverITCase.verifyAfterJobExecuted(RegionFailoverITCase.java:157)
May 31 17:17:59 at
org.apache.flink.test.checkpointing.RegionFailoverITCase.testMultiRegionFailover(RegionFailoverITCase.java:149)
May 31 17:17:59 at java.lang.reflect.Method.invoke(Method.java:498)
May 31 17:17:59 at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
May 31 17:17:59 at java.lang.Thread.run(Thread.java:748)
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=60001=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba=9222
> RegionFailoverITCase.testMultiRegionFailover has never ever restored state
> --
>
> Key: FLINK-35505
> URL: https://issues.apache.org/jira/browse/FLINK-35505
> Project: Flink
> Issue Type: Bug
> Components: Build System / CI
>Affects Versions: 1.20.0
>Reporter: Weijie Guo
>Priority: Major
>
> {code:java}
> May 31 17:17:59 17:17:59.555 [ERROR]
> org.apache.flink.test.checkpointing.RegionFailoverITCase.testMultiRegionFailover
> -- Time elapsed: 27.92 s <<< FAILURE!
> May 31 17:17:59 java.lang.AssertionError: The test multi-region job has never
> ever restored state.
> May 31 17:17:59 at org.junit.Assert.fail(Assert.java:89)
> May 31 17:17:59 at org.junit.Assert.assertTrue(Assert.java:42)
> May 31 17:17:59 at
> org.apache.flink.test.checkpointing.RegionFailoverITCase.verifyAfterJobExecuted(RegionFailoverITCase.java:157)
> May 31 17:17:59 at
> org.apache.flink.test.checkpointing.RegionFailoverITCase.testMultiRegionFailover(RegionFailoverITCase.java:149)
> May 31 17:17:59 at java.lang.reflect.Method.invoke(Method.java:498)
> May 31 17:17:59 at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> May 31 17:17:59 at java.lang.Thread.run(Thread.java:748)
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=60001=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba=9222
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-35505) RegionFailoverITCase.testMultiRegionFailover has never ever restored state
Weijie Guo created FLINK-35505: -- Summary: RegionFailoverITCase.testMultiRegionFailover has never ever restored state Key: FLINK-35505 URL: https://issues.apache.org/jira/browse/FLINK-35505 Project: Flink Issue Type: Bug Components: Build System / CI Affects Versions: 1.20.0 Reporter: Weijie Guo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-26808) [flink v1.14.2] Submit jobs via REST API not working after set web.submit.enable: false
[
https://issues.apache.org/jira/browse/FLINK-26808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851449#comment-17851449
]
Robert Metzger commented on FLINK-26808:
Closing as the PRs have been merged ...
> [flink v1.14.2] Submit jobs via REST API not working after set
> web.submit.enable: false
> ---
>
> Key: FLINK-26808
> URL: https://issues.apache.org/jira/browse/FLINK-26808
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission
>Affects Versions: 1.14.2
>Reporter: Luís Costa
>Assignee: Ufuk Celebi
>Priority: Minor
> Labels: pull-request-available
>
> Greetings,
> I am using flink version 1.14.2 and after changing web.submit.enable to
> false, job submission via REST API is no longer working.
> The app that uses flink receives a 404 with "Not found: /jars/upload"
> Looking into
> [documentation|[https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/]]
> saw that web.upload.dir is only used if {{web.submit.enable}} is true, if
> not it will be used JOB_MANAGER_WEB_TMPDIR_KEY
> Doing a curl to /jars it returns:
> {code:java}
> curl -X GET http://localhost:8081/jars
> HTTP/1.1 404 Not Found
> {"errors":["Unable to load requested file /jars."]} {code}
> Found this issue related to option web.submit.enable
> https://issues.apache.org/jira/browse/FLINK-13799
> Could you please let me know if this is an issue that you are already aware?
> Thanks in advance
> Best regards,
> Luís Costa
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (FLINK-26808) [flink v1.14.2] Submit jobs via REST API not working after set web.submit.enable: false
[
https://issues.apache.org/jira/browse/FLINK-26808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger resolved FLINK-26808.
Fix Version/s: 1.20.0
1.19.1
Resolution: Fixed
> [flink v1.14.2] Submit jobs via REST API not working after set
> web.submit.enable: false
> ---
>
> Key: FLINK-26808
> URL: https://issues.apache.org/jira/browse/FLINK-26808
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission
>Affects Versions: 1.14.2
>Reporter: Luís Costa
>Assignee: Ufuk Celebi
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.20.0, 1.19.1
>
>
> Greetings,
> I am using flink version 1.14.2 and after changing web.submit.enable to
> false, job submission via REST API is no longer working.
> The app that uses flink receives a 404 with "Not found: /jars/upload"
> Looking into
> [documentation|[https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/]]
> saw that web.upload.dir is only used if {{web.submit.enable}} is true, if
> not it will be used JOB_MANAGER_WEB_TMPDIR_KEY
> Doing a curl to /jars it returns:
> {code:java}
> curl -X GET http://localhost:8081/jars
> HTTP/1.1 404 Not Found
> {"errors":["Unable to load requested file /jars."]} {code}
> Found this issue related to option web.submit.enable
> https://issues.apache.org/jira/browse/FLINK-13799
> Could you please let me know if this is an issue that you are already aware?
> Thanks in advance
> Best regards,
> Luís Costa
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (FLINK-26808) [flink v1.14.2] Submit jobs via REST API not working after set web.submit.enable: false
[
https://issues.apache.org/jira/browse/FLINK-26808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger reassigned FLINK-26808:
--
Assignee: Ufuk Celebi (was: Anupam Aggarwal)
> [flink v1.14.2] Submit jobs via REST API not working after set
> web.submit.enable: false
> ---
>
> Key: FLINK-26808
> URL: https://issues.apache.org/jira/browse/FLINK-26808
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission
>Affects Versions: 1.14.2
>Reporter: Luís Costa
>Assignee: Ufuk Celebi
>Priority: Minor
> Labels: pull-request-available
>
> Greetings,
> I am using flink version 1.14.2 and after changing web.submit.enable to
> false, job submission via REST API is no longer working.
> The app that uses flink receives a 404 with "Not found: /jars/upload"
> Looking into
> [documentation|[https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/]]
> saw that web.upload.dir is only used if {{web.submit.enable}} is true, if
> not it will be used JOB_MANAGER_WEB_TMPDIR_KEY
> Doing a curl to /jars it returns:
> {code:java}
> curl -X GET http://localhost:8081/jars
> HTTP/1.1 404 Not Found
> {"errors":["Unable to load requested file /jars."]} {code}
> Found this issue related to option web.submit.enable
> https://issues.apache.org/jira/browse/FLINK-13799
> Could you please let me know if this is an issue that you are already aware?
> Thanks in advance
> Best regards,
> Luís Costa
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-35498) Unexpected argument name conflict error when do extract method params from udf
[
https://issues.apache.org/jira/browse/FLINK-35498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851430#comment-17851430
]
lincoln lee commented on FLINK-35498:
-
Add some inputs:
For the test class TestXyz, if the local variable `localV1` without an
initialization when declare, the `LocalVariableTable` of the method may include
several lines for it, as the following code example and class files' snippet
comparison
{code}
// will not fail if add initializationString localV1;
// with initialization
String localV1 = "";
{code}
!image-2024-06-02-23-09-17-768.png!
this will cause different analysis results via asm tool, and we should consider
these duplicate vars when extract the method params.
> Unexpected argument name conflict error when do extract method params from udf
> --
>
> Key: FLINK-35498
> URL: https://issues.apache.org/jira/browse/FLINK-35498
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.19.0, 1.20.0
>Reporter: lincoln lee
>Assignee: xuyang
>Priority: Major
> Attachments: image-2024-06-02-23-09-17-768.png
>
>
> Follow the steps to reproduce the error:
> test case:
> {code:java}
> util.addTemporarySystemFunction("myudf", new TestXyz)
> util.tableEnv.explainSql("select myudf(f1, f2) from t")
> {code}
>
> udf: TestXyz
> {code:java}
> public class TestXyz extends ScalarFunction {
> public String eval(String s1, String s2) {
> // will not fail if add initialization
> String localV1;
> if (s1 == null) {
> if (s2 != null) {
> localV1 = s2;
> } else {
> localV1 = s2 + s1;
> }
> } else {
> if ("xx".equals(s2)) {
> localV1 = s1.length() >= s2.length() ? s1 : s2;
> } else {
> localV1 = s1;
> }
> }
> if (s1 == null) {
> return s2 + localV1;
> }
> if (s2 == null) {
> return s1;
> }
> return s1.length() >= s2.length() ? s1 + localV1 : s2;
> }
> }
> {code}
>
> error stack:
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Unable to extract
> a type inference from method:
> public java.lang.String
> org.apache.flink.table.planner.runtime.utils.TestXyz.eval(java.lang.String,java.lang.String)
> at
> org.apache.flink.table.types.extraction.ExtractionUtils.extractionError(ExtractionUtils.java:362)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.extractResultMappings(BaseMappingExtractor.java:154)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.extractOutputMapping(BaseMappingExtractor.java:100)
> ... 53 more
> Caused by: org.apache.flink.table.api.ValidationException: Argument name
> conflict, there are at least two argument names that are the same.
> at
> org.apache.flink.table.types.extraction.ExtractionUtils.extractionError(ExtractionUtils.java:362)
> at
> org.apache.flink.table.types.extraction.ExtractionUtils.extractionError(ExtractionUtils.java:357)
> at
> org.apache.flink.table.types.extraction.FunctionSignatureTemplate.of(FunctionSignatureTemplate.java:73)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.lambda$createParameterSignatureExtraction$9(BaseMappingExtractor.java:381)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.putExtractedResultMappings(BaseMappingExtractor.java:298)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.collectMethodMappings(BaseMappingExtractor.java:244)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.extractResultMappings(BaseMappingExtractor.java:137)
> ... 54 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35498) Unexpected argument name conflict error when do extract method params from udf
[
https://issues.apache.org/jira/browse/FLINK-35498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lincoln lee updated FLINK-35498:
Attachment: image-2024-06-02-23-09-17-768.png
> Unexpected argument name conflict error when do extract method params from udf
> --
>
> Key: FLINK-35498
> URL: https://issues.apache.org/jira/browse/FLINK-35498
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.19.0, 1.20.0
>Reporter: lincoln lee
>Assignee: xuyang
>Priority: Major
> Attachments: image-2024-06-02-23-09-17-768.png
>
>
> Follow the steps to reproduce the error:
> test case:
> {code:java}
> util.addTemporarySystemFunction("myudf", new TestXyz)
> util.tableEnv.explainSql("select myudf(f1, f2) from t")
> {code}
>
> udf: TestXyz
> {code:java}
> public class TestXyz extends ScalarFunction {
> public String eval(String s1, String s2) {
> // will not fail if add initialization
> String localV1;
> if (s1 == null) {
> if (s2 != null) {
> localV1 = s2;
> } else {
> localV1 = s2 + s1;
> }
> } else {
> if ("xx".equals(s2)) {
> localV1 = s1.length() >= s2.length() ? s1 : s2;
> } else {
> localV1 = s1;
> }
> }
> if (s1 == null) {
> return s2 + localV1;
> }
> if (s2 == null) {
> return s1;
> }
> return s1.length() >= s2.length() ? s1 + localV1 : s2;
> }
> }
> {code}
>
> error stack:
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Unable to extract
> a type inference from method:
> public java.lang.String
> org.apache.flink.table.planner.runtime.utils.TestXyz.eval(java.lang.String,java.lang.String)
> at
> org.apache.flink.table.types.extraction.ExtractionUtils.extractionError(ExtractionUtils.java:362)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.extractResultMappings(BaseMappingExtractor.java:154)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.extractOutputMapping(BaseMappingExtractor.java:100)
> ... 53 more
> Caused by: org.apache.flink.table.api.ValidationException: Argument name
> conflict, there are at least two argument names that are the same.
> at
> org.apache.flink.table.types.extraction.ExtractionUtils.extractionError(ExtractionUtils.java:362)
> at
> org.apache.flink.table.types.extraction.ExtractionUtils.extractionError(ExtractionUtils.java:357)
> at
> org.apache.flink.table.types.extraction.FunctionSignatureTemplate.of(FunctionSignatureTemplate.java:73)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.lambda$createParameterSignatureExtraction$9(BaseMappingExtractor.java:381)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.putExtractedResultMappings(BaseMappingExtractor.java:298)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.collectMethodMappings(BaseMappingExtractor.java:244)
> at
> org.apache.flink.table.types.extraction.BaseMappingExtractor.extractResultMappings(BaseMappingExtractor.java:137)
> ... 54 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-21373) Port RabbitMQ Sink to FLIP-143 API
[ https://issues.apache.org/jira/browse/FLINK-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851428#comment-17851428 ] Ahmed Hamdy commented on FLINK-21373: - [~martijnvisser] Do you have a nomination for a reviewer? otherwise this will get stale > Port RabbitMQ Sink to FLIP-143 API > -- > > Key: FLINK-21373 > URL: https://issues.apache.org/jira/browse/FLINK-21373 > Project: Flink > Issue Type: Improvement > Components: Connectors/ RabbitMQ >Reporter: Jan Westphal >Assignee: Ahmed Hamdy >Priority: Minor > Labels: auto-unassigned, pull-request-available > > *Structure* > The unified Sink API provides a Writer, a Committer and a GlobalCommitter. > Right now we don’t see the need to use the Committer and GlobalCommitter as > the Writer is sufficient to hold up to the consistencies. Since we are in the > need of asynchronous RabbitMQ callbacks to know whether or not a message was > published successfully and have to store unacknowledged messages in the > checkpoint, there would be a large bidirectional communication and state > exchange overhead between the Writer and the Committer. > *At-most-once* > The Writer receives a message from Flink and simply publishes it to RabbitMQ. > The current RabbitMQ Sink only provides this mode. > *At-least-once* > The objective here is, to receive an acknowledgement by RabbitMQ for > published messages. Therefore, before publishing a message, we store the > message in a Map with the sequence number as its key. If the message is > acknowledged by RabbitMQ we can remove it from the Map. If we don’t receive > an acknowledgement for a certain amount of time (or a RabbitMQ specific so > called negative acknowledgement) we will try to resend the message when > doing a checkpoint. > *Exactly-once* > On checkpointing we send all messages by Flink in transaction mode to > RabbitMQ. This way, all the messages get sent or are rolled back on failure. > All messages that are not sent successfully are written to the checkpoint and > are tried to be sent with the next checkpoint. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-35200][table] Support the execution of suspend, resume materialized table in full refresh mode [flink]
hackergin commented on PR #24877: URL: https://github.com/apache/flink/pull/24877#issuecomment-2143766184 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
