This is an automated email from the ASF dual-hosted git repository.
morrySnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 9ec9d5ffdec [Fix](fold_const) MAKE_SET constant folding should clear
(#64907)
9ec9d5ffdec is described below
commit 9ec9d5ffdec5e5c52748d67f5cedf9faaaf778ac
Author: linrrarity <[email protected]>
AuthorDate: Mon Jun 29 10:39:29 2026 +0800
[Fix](fold_const) MAKE_SET constant folding should clear (#64907)
related PR: #56367
Problem Summary:
`MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions
in the FE constant folding path, which leads to incorrect clearing of
high bits when `pos >= 32` due to integer shift modulo.
For Java `int` shifts, the shift distance is masked with 0x1F, which
means only the low 5 bits are used:
- `pos = 0..31` -> normal
- `pos = 32` -> treated as `0`
- `pos = 33` -> treated as `1`
- `pos = 64` -> treated as `0` again
For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32)
expect: `bit &= ~(1 << 32)`
got: `bit &= ~(1 << 0)`
That does **not** clear bit 32 at all. So:
- `bit` stays unchanged and `pos` stays 32
- the loop never makes progress
- the same string is appended again and again
- Java throws OutOfMemoryError: Java heap space
before(FE constant folding failed or FE OOM):
```text
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)
|
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0
|
| OUTPUT EXPRS:
|
| ms[#0]
|
| PARTITION: UNPARTITIONED
|
|
|
| HAS_COLO_PLAN_NODE: false
|
|
|
| VRESULT SINK
|
| MYSQL_PROTOCOL
|
|
|
| 0:VUNION(11)
|
| constant exprs:
|
| make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05',
'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16',
'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27',
'a28', 'a29', 'a30', 'a31', 'a32', 'a33') |
|
|
|
|
|
|
| ========== STATISTICS ==========
|
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
17 rows in set (28.822 sec)
```
```text
2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303)
[StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT
MAKE_SET(4294967296,
'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09',
'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a
18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27',
'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail
ed.
org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too
much time (32s > 30s). You should increment timeout by set 'nerei
ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout'
to false. Time consuming details, parse time: 6ms, plan time:
{"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra
nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1}
```
now:
```text
Doris> SET debug_skip_fold_constant = 0;
Doris> EXPLAIN SELECT MAKE_SET(4294967296,
-> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07',
-> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15',
-> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23',
-> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31',
-> 'a32', 'a33') AS ms;
+----------------------------------+
| Explain String(Nereids Planner) |
+----------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| ms[#0] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 0:VUNION(12) |
| constant exprs: |
| 'a32' |
| |
| |
| |
| ========== STATISTICS ========== |
+----------------------------------+
```
---
.../trees/expressions/functions/executable/StringArithmetic.java | 2 +-
.../sql_functions/string_functions/test_string_function.groovy | 5 +++++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/executable/StringArithmetic.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/executable/StringArithmetic.java
index 3bdf1aded66..d5cd59cfe47 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/executable/StringArithmetic.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/executable/StringArithmetic.java
@@ -1293,7 +1293,7 @@ public class StringArithmetic {
while (pos != 64 && pos < args.length && bit != 0) {
sb.append(args[pos].getValue());
sb.append(',');
- bit &= ~(1 << pos);
+ bit &= ~(1L << pos);
pos = Long.numberOfTrailingZeros(bit);
}
if (sb.length() != 0) {
diff --git
a/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
b/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
index 5819ea3c8db..9db9777b1c8 100644
---
a/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
+++
b/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
@@ -619,6 +619,11 @@ suite("test_string_function", "arrow_flight_sql") {
testFoldConst("SELECT MAKE_SET(4611686018427387903, 'a', 'b', 'c');")
testFoldConst("SELECT MAKE_SET(BIT_SHIFT_LEFT(1, 50) - 3, 'first',
'second', 'third');")
testFoldConst("SELECT MAKE_SET(3, '', 'a');")
+ testFoldConst("SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03',
'a04', 'a05', 'a06', 'a07', " +
+ "'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', " +
+ "'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', " +
+ "'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', " +
+ "'a32', 'a33');")
test {
sql"""SELECT MAKE_SET(184467440737095516156, 'a', 'b', 'c');"""
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
