zhengxiang378928908-code opened a new issue, #12753: URL: https://github.com/apache/seatunnel/issues/12753
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened The Doris connector maps `LARGEINT` to `DecimalType(20, 0)` (`AbstractDorisTypeConverter#convert`). Doris `LARGEINT` is a 128-bit signed integer, so its values range from -2^127 to 2^127-1, which is up to 39 digits. The declared precision of 20 is too small. The source reader itself returns the full value; only the declared schema is wrong. Any downstream that trusts the schema therefore creates a column that is too narrow. When a value with more than 20 digits arrives, the job fails. I reproduced this end to end with the **3.0.0 release** against **Doris 4.0.6**. The mapping code is unchanged since #6354, so `dev` behaves the same. The source table has 5 rows: | id | value | digits | |---|---|---| | 1 | 12345 | 5 | | 2 | 99999999999999999999 | 20 | | 3 | 100000000000000000000 | 21 | | 4 | 170141183460469231731687303715884105727 (max) | 39 | | 5 | -170141183460469231731687303715884105728 (min) | 39 + sign | | Job | Result | |---|---| | Doris → Console | All 5 values printed intact. The reader is fine. | | Doris → Doris, auto-create | OK. `reconvert()` special-cases `sourceType = largeint` and recreates a `largeint` column. | | **Doris → MySQL 8 (JDBC), auto-create** | Target column created as **`decimal(20,0)`**. The job fails on row 3: `Out of range value for column 'v'`, and 0 rows are written. Non-strict `sql_mode` fails the same way, because Connector/J reports the truncation. | Doris → Doris only works because of the `sourceType` special case in `reconvert()`. Every other sink gets `DECIMAL(20,0)`. `ARRAY<largeint>` has the same problem (`DorisTypeConverterV2`, `new DecimalArrayType(new DecimalType(20, 0))`). `MAX_DORIS_LARGEINT_TO_VARCHAR_LENGTH = 39` is defined in the converter but is not used. **Fix options.** The full `LARGEINT` range needs 39 digits. `DecimalType` itself has no cap, but sink converters do: `MySqlTypeConverter.MAX_PRECISION = 65`, whereas Doris and Oracle cap at 38. 1. Map to `DECIMAL(39,0)`. This is exact and keeps the numeric type. Sinks that allow 39 digits, such as MySQL, are correct. How sinks capped at 38 handle a 39-digit decimal still needs checking for each one. 2. Map to `STRING` with length 39. This is lossless everywhere, but downstream gets a string column instead of a number. Either way, the existing `sourceType = largeint` special case for Doris → Doris would stay. I lean towards option 1, since it fixes the reproduced failure without changing the logical type, but I'd appreciate a maintainer's preference before I open a PR. ### SeaTunnel Version 3.0.0 (also present on dev) ### SeaTunnel Config ```sql -- Doris 4.0.6 CREATE TABLE seatunnel_probe_tmp.li_src (id INT, v LARGEINT) DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES('replication_num'='1'); INSERT INTO seatunnel_probe_tmp.li_src VALUES (1, 12345), (2, 99999999999999999999), (3, 100000000000000000000), (4, 170141183460469231731687303715884105727), (5, -170141183460469231731687303715884105728); ``` ```conf env { parallelism = 1 job.mode = "BATCH" } source { Doris { fenodes = "doris-fe:8030" query-port = 9030 username = "root" password = "" database = "seatunnel_probe_tmp" table = "li_src" } } sink { Jdbc { url = "jdbc:mysql://mysql:3306/probe?useSSL=false&allowPublicKeyRetrieval=true" driver = "com.mysql.cj.jdbc.Driver" user = "root" password = "***" generate_sink_sql = true database = "probe" table = "li_dst" schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST" } } ``` ### Running Command ```shell ./bin/seatunnel.sh --config doris_to_mysql.conf -m local ``` ### Error Exception ```log -- auto-created target table CREATE TABLE `li_dst` (`id` int DEFAULT NULL, `v` decimal(20,0) DEFAULT NULL) Caused by: java.sql.BatchUpdateException: Data truncation: Out of range value for column 'v' at row 3 Caused by: com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Out of range value for column 'v' at row 3 ErrorCode:[COMMON-10], ErrorDescription:[Flush data operation that in sink connector failed] ``` ### Zeta or Flink or Spark Version Zeta (local mode) ### Java or Scala Version Java 8 (1.8.0_412) ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
