xy720 opened a new issue, #26964: URL: https://github.com/apache/doris/issues/26964
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? Insert/load may causes modify column failed when table properties has `light_scheme_change = true` ### What You Expected? Power! ### How to Reproduce? 1、create table with `light_schema_change` properties ``` CREATE TABLE `part_bak` ( `p_partkey` int(11) NOT NULL, `p_name` varchar(55) NOT NULL, `p_mfgr` varchar(25) NOT NULL, `p_brand` varchar(10) NOT NULL, `p_type` varchar(25) NOT NULL, `p_size` int(11) NOT NULL, `p_container` varchar(10) NOT NULL, `p_retailprice` decimal(15, 2) NOT NULL, `p_comment` varchar(23) NOT NULL, INDEX bitmap_test (`p_retailprice`) USING BITMAP COMMENT '' ) ENGINE=OLAP DUPLICATE KEY(`p_partkey`) COMMENT 'OLAP' DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 24 PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "in_memory" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false" ); ``` 2、start high-frequency writing to the table ``` #!/bin/bash max=2000 for (( i=1; i <= $max; ++i )) do curl --location-trusted -u root: -T data.txt -H "label:$(date +%Y%m%d%H%M%S%N)" http://127.0.0.1:8030/api/test/part_bak/_stream_load done ``` 3、alter table to modify column ``` alter table part_bak modify column p_retailprice DECIMAL(15, 8) NOT NULL; ``` 4、then alter job will failed ``` MySQL [tpch_100g_decimalv3]> show alter table column; +-------+-----------+-------------------------+-------------------------+-----------+---------+---------------+---------------+---------------+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+ | JobId | TableName | CreateTime | FinishTime | IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State | Msg | Progress | Timeout | +-------+-----------+-------------------------+-------------------------+-----------+---------+---------------+---------------+---------------+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+ | 48480 | part_bak | 2023-11-13 21:14:07.904 | 2023-11-13 21:14:31.906 | part_bak | 48481 | 48272 | 2:1532702836 | 45928 | CANCELLED | errCode = 2, detailMessage = schema change tasks failed on same tablet reach threshold [[ALTER], signature: 17318, backendId: 10004, tablet id: 17318, [ALTER], signature: 17318, backendId: 10005, tablet id: 48542] | NULL | 2592000 | +-------+-----------+-------------------------+-------------------------+-----------+---------+---------------+---------------+---------------+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+ 4 rows in set (0.00 sec) ``` ### Anything Else? And here is the WARNING log in be.INFO in backend 10004: <img width="1386" alt="image" src="https://github.com/apache/doris/assets/22125576/104f5174-8c14-4b57-99b4-cf8a0db5f223";> An Internal Error was thrown: ``` a shadow column is encountered __doris_shadow_p_retailprice [INTERNAL_ERROR]failed due to operate on shadow column ``` ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
