MOBIN created FLINK-37348:
-----------------------------
Summary: [pipeline-paimon]fix paimon
ArrayIndexOutOfBoundsException when add column before first_column
Key: FLINK-37348
URL: https://issues.apache.org/jira/browse/FLINK-37348
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.3.0
Reporter: MOBIN
Repeat step：
# ALTER TABLE table_name add COLUMN new_first_column bigint first
# throw ArrayIndexOutOfBoundsException
log:
{code:java}
2025-02-19 11:32:16,586 INFO
org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator [] -
All sink subtask have flushed for table dw_app.cdc_sink19. Start to apply
schema change request:
SchemaChangeRequest{tableId=dw_app.cdc_sink19,
schemaChangeEvent=AddColumnEvent{tableId=dw_app.cdc_sink19,
addedColumns=[ColumnWithPosition{column=`new_first_column` BIGINT,
position=BEFORE, existedColumnName=first_column}]}, subTaskId=0}
that extracts to:
AddColumnEvent{tableId=rt_ods.cdc_sink19_add_column_EVOLVE4,
addedColumns=[ColumnWithPosition{column=`new_first_column` BIGINT,
position=BEFORE, existedColumnName=first_column}]}
2025-02-19 11:32:16,716 ERROR
org.apache.flink.cdc.runtime.operators.schema.common.SchemaRegistry [] - An
exception was triggered from Schema change applying task. Job will fail now.
org.apache.flink.util.FlinkRuntimeException: Failed to apply schema change
event.
at
org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$0(SchemaCoordinator.java:290)
~[flink-cdc-dist-3.3.0.jar:3.3.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_65]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_65]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_65]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:418) ~[?:1.8.0_65]
at java.util.ArrayList.get(ArrayList.java:431) ~[?:1.8.0_65]
at
org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumnWithBeforePosition(PaimonMetadataApplier.java:275)
~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumnEventWithPosition(PaimonMetadataApplier.java:237)
~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumn(PaimonMetadataApplier.java:206)
~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.lambda$applySchemaChange$0(PaimonMetadataApplier.java:127)
~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.common.event.visitor.SchemaChangeEventVisitor.visit(SchemaChangeEventVisitor.java:47)
~[flink-cdc-dist-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applySchemaChange(PaimonMetadataApplier.java:124)
~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applyAndUpdateEvolvedSchemaChange(SchemaCoordinator.java:434)
~[flink-cdc-dist-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applySchemaChange(SchemaCoordinator.java:401)
~[flink-cdc-dist-3.3.0.jar:3.3.0]
at
org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$0(SchemaCoordinator.java:288)
~[flink-cdc-dist-3.3.0.jar:3.3.0]
... 5 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
