Doris-Breakwater commented on issue #67488: URL: https://github.com/apache/doris/issues/67488#issuecomment-5524094570
## Preliminary analysis This is partly confirmed from the `4.1.3` source, but the report does not yet contain enough information to classify the column-comment symptom. ### Verified facts - **Table comments are not copied when the CDC job auto-creates a target table.** The source metadata path builds the target `CreateTableInfo` with a literal empty table comment ([`StreamingJobUtils.java`, 4.1.3](https://github.com/apache/doris/blob/4.1.3/fe/fe-core/src/main/java/org/apache/doris/job/util/StreamingJobUtils.java#L439-L453)). Although the MySQL JDBC client has `getTableComment(...)`, this creation path never calls it. The checked current `branch-4.1` and `master` source retain this behavior. - **Initial column comments are intended to be copied.** Doris maps JDBC `REMARKS` into `Column.comment` ([`JdbcClient.java`, 4.1.3](https://github.com/apache/doris/blob/4.1.3/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClient.java#L411-L419)) and then passes `col.getComment()` into each target `ColumnDefinition` ([`StreamingJobUtils.java`, 4.1.3](https://github.com/apache/doris/blob/4.1.3/fe/fe-core/src/main/java/org/apache/doris/job/util/StreamingJobUtils.java#L418-L429)). Connector/J 8.0.33's `useInformationSchema=true` metadata implementation selects `COLUMN_COMMENT AS REMARKS` ([Connector/J source](https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/user-impl/java/com/mysql/cj/jdbc/DatabaseMetaDataUsingInfoSchema.java#L120-L220)). Therefore, the supplied SQL alone does not explain missing comments on columns created during initial target-table creation. - **Existing target tables are not updated.** The generated `CREATE TABLE` command only runs when the target table does not exist ([`StreamingInsertJob.java`, 4.1.3](https://github.com/apache/doris/blob/4.1.3/fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java#L298-L316)). No comment reconciliation is performed for an existing table. - **Comments added or changed after job creation are not synchronized by 4.1.3.** In that release, MySQL schema-change events are explicitly discarded with a TODO ([`MySqlDebeziumJsonDeserializer.java`, 4.1.3](https://github.com/apache/doris/blob/4.1.3/fs_brokers/cdc_client/src/main/java/org/apache/doris/cdcclient/source/deserialize/MySqlDebeziumJsonDeserializer.java#L51-L65)). Thus an `ALTER TABLE ... COMMENT` or a column-comment change after `CREATE JOB` cannot reach Doris through this path. - The issue currently has no labels and provides no minimal source-table DDL, target metadata output, timing information, or reproduction steps. ### Information needed to confirm the column-comment symptom Please provide a one-table reproduction and redact credentials/hosts: 1. The MySQL version and exact source `CREATE TABLE` statement, including a primary key, one column comment, and the table comment. Confirm that all comments existed **before** `CREATE JOB`. 2. Whether the corresponding Doris target table existed before `CREATE JOB`. Ideally reproduce after dropping only the test target table. 3. The following source metadata results, queried as the same MySQL user used by the job: ```sql SELECT TABLE_NAME, TABLE_COMMENT FROM information_schema.TABLES WHERE TABLE_SCHEMA = '<source_db>' AND TABLE_NAME = '<source_table>'; SELECT COLUMN_NAME, COLUMN_COMMENT FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = '<source_db>' AND TABLE_NAME = '<source_table>' ORDER BY ORDINAL_POSITION; ``` 4. `SHOW CREATE TABLE <target_db>.<target_table>` immediately after job creation, plus the exact Doris build/version string and relevant FE log lines around `CREATE JOB` if any warning or metadata exception appears. 5. Clarify whether "synchronize" means comments present before the initial snapshot, comment DDL issued after the job started, or both. ### Suggested next steps for maintainers 1. Treat initial table-comment propagation as a confirmed missing implementation: fetch the source table comment and supply it to `CreateTableInfo`, with coverage for both table and column comments. 2. Reproduce initial column-comment propagation with Connector/J 8.0.33 and a clean target table. If it fails, inspect `JdbcFieldSchema.remarks` immediately after `DatabaseMetaData.getColumns(...)` to determine whether the loss occurs in Connector/J metadata retrieval or later in Doris table creation. 3. Decide and document separate semantics for pre-created target tables and for comment-only DDL after a job starts. Updating those comments automatically may overwrite user-managed target metadata, so it should not be implicitly bundled with initial auto-creation behavior. Breakwater-GitHub-Analysis-Slot: slot_c14326365c8f -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
