[jira] [Commented] (FLINK-33666) MergeTableLikeUtil uses different constraint name than Schema
[
https://issues.apache.org/jira/browse/FLINK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796882#comment-17796882
]
Jim Hughes commented on FLINK-33666:
[~twalthr] Yes, the issue has been resolved.
The issue was a change in expectations around PK. From offline conversations,
you assured me that the change is ok and will not impact folks using compiled
plans. I think we are good.
An example stack trace is:
```
Dec 07 13:49:11 13:49:11.037 [ERROR]
org.apache.flink.table.planner.plan.nodes.exec.stream.GroupAggregateRestoreTest.testRestore(TableTestProgram,
ExecNodeMetadata)[1] Time elapsed: 0.17 s <<< ERROR!
Dec 07 13:49:11 org.apache.flink.table.api.TableException: Cannot load Plan
from file
'/__w/1/s/flink-table/flink-table-planner/src/test/resources/restore-tests/stream-exec-group-aggregate_1/group-aggregate-simple/plan/group-aggregate-simple.json'.
Dec 07 13:49:11 at
org.apache.flink.table.api.internal.TableEnvironmentImpl.loadPlan(TableEnvironmentImpl.java:760)
Dec 07 13:49:11 at
org.apache.flink.table.planner.plan.nodes.exec.testutils.RestoreTestBase.testRestore(RestoreTestBase.java:279)
Dec 07 13:49:11 Caused by:
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException:
The schema of table 'default_catalog.default_database.sink_t' from the
persisted plan does not match the schema loaded from the catalog: '(
Dec 07 13:49:11 `b` BIGINT NOT NULL,
Dec 07 13:49:11 `cnt` BIGINT,
Dec 07 13:49:11 `avg_a` DOUBLE,
Dec 07 13:49:11 `min_c` STRING,
Dec 07 13:49:11 CONSTRAINT `PK_129` PRIMARY KEY (`b`) NOT ENFORCED
Dec 07 13:49:11 )' != '(
Dec 07 13:49:11 `b` BIGINT NOT NULL,
Dec 07 13:49:11 `cnt` BIGINT,
Dec 07 13:49:11 `avg_a` DOUBLE,
Dec 07 13:49:11 `min_c` STRING,
Dec 07 13:49:11 CONSTRAINT `PK_b` PRIMARY KEY (`b`) NOT ENFORCED
Dec 07 13:49:11 )'. Make sure the table schema in the catalog is still
identical. (through reference chain:
org.apache.flink.table.planner.plan.nodes.exec.serde.JsonPlanGraph["nodes"]->java.util.ArrayList[5]->org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecSink["dynamicTableSink"]->org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSinkSpec["table"])
...
```
> MergeTableLikeUtil uses different constraint name than Schema
> -
>
> Key: FLINK-33666
> URL: https://issues.apache.org/jira/browse/FLINK-33666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: Timo Walther
>Assignee: Jeyhun Karimov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.0
>
>
> {{MergeTableLikeUtil}} uses a different algorithm to name constraints than
> {{Schema}}.
> {{Schema}} includes the column names.
> {{MergeTableLikeUtil}} uses a hashCode which means it might depend on JVM
> specifics.
> For consistency we should use the same algorithm. I propose to use
> {{Schema}}'s logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-33666) MergeTableLikeUtil uses different constraint name than Schema
[
https://issues.apache.org/jira/browse/FLINK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796862#comment-17796862
]
Timo Walther commented on FLINK-33666:
--
[~jhughes] was this issue resolved? Was it just a problem of the test base?
What was the complete exception stack trace?
> MergeTableLikeUtil uses different constraint name than Schema
> -
>
> Key: FLINK-33666
> URL: https://issues.apache.org/jira/browse/FLINK-33666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: Timo Walther
>Assignee: Jeyhun Karimov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.0
>
>
> {{MergeTableLikeUtil}} uses a different algorithm to name constraints than
> {{Schema}}.
> {{Schema}} includes the column names.
> {{MergeTableLikeUtil}} uses a hashCode which means it might depend on JVM
> specifics.
> For consistency we should use the same algorithm. I propose to use
> {{Schema}}'s logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-33666) MergeTableLikeUtil uses different constraint name than Schema
[
https://issues.apache.org/jira/browse/FLINK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794460#comment-17794460
]
Jim Hughes commented on FLINK-33666:
This change seems to prevent
`org.apache.flink.table.api.internal.TableEnvironmentImpl.loadPlan` from
working on Compiled Plans which used the previous method.
The PR changes restore test plans which should never be updated.
I was able to see this since [~bvarghese]'s separate PR was merged to master
today and it failed CI here:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55321=logs=0c940707-2659-5648-cbe6-a1ad63045f0a=075c2716-8010-5565-fe08-3c4bb45824a4
> MergeTableLikeUtil uses different constraint name than Schema
> -
>
> Key: FLINK-33666
> URL: https://issues.apache.org/jira/browse/FLINK-33666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: Timo Walther
>Assignee: Jeyhun Karimov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.0
>
>
> {{MergeTableLikeUtil}} uses a different algorithm to name constraints than
> {{Schema}}.
> {{Schema}} includes the column names.
> {{MergeTableLikeUtil}} uses a hashCode which means it might depend on JVM
> specifics.
> For consistency we should use the same algorithm. I propose to use
> {{Schema}}'s logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-33666) MergeTableLikeUtil uses different constraint name than Schema
[
https://issues.apache.org/jira/browse/FLINK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794086#comment-17794086
]
Timo Walther commented on FLINK-33666:
--
Thanks [~jeyhunkarimov] for working on this.
> MergeTableLikeUtil uses different constraint name than Schema
> -
>
> Key: FLINK-33666
> URL: https://issues.apache.org/jira/browse/FLINK-33666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: Timo Walther
>Assignee: Jeyhun Karimov
>Priority: Major
> Labels: pull-request-available
>
> {{MergeTableLikeUtil}} uses a different algorithm to name constraints than
> {{Schema}}.
> {{Schema}} includes the column names.
> {{MergeTableLikeUtil}} uses a hashCode which means it might depend on JVM
> specifics.
> For consistency we should use the same algorithm. I propose to use
> {{Schema}}'s logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-33666) MergeTableLikeUtil uses different constraint name than Schema
[
https://issues.apache.org/jira/browse/FLINK-33666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793590#comment-17793590
]
Jeyhun Karimov commented on FLINK-33666:
Hi [~twalthr] can you please assign this task to me or give me access to
self-assign tasks?
> MergeTableLikeUtil uses different constraint name than Schema
> -
>
> Key: FLINK-33666
> URL: https://issues.apache.org/jira/browse/FLINK-33666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: Timo Walther
>Priority: Major
> Labels: pull-request-available
>
> {{MergeTableLikeUtil}} uses a different algorithm to name constraints than
> {{Schema}}.
> {{Schema}} includes the column names.
> {{MergeTableLikeUtil}} uses a hashCode which means it might depend on JVM
> specifics.
> For consistency we should use the same algorithm. I propose to use
> {{Schema}}'s logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
