felixwluo opened a new pull request, #61961:
URL: https://github.com/apache/doris/pull/61961
Problem Summary:
## compaction core
```
#0 0x0000000000000000 in ?? ()
#1 0x00005561c333fa74 in doris::parse_to_predicate (column=..., index=6,
condition=..., arena=0x7ff95ae1b710, opposite=true)
at /root/be/src/olap/predicate_creator.h:329
#2 0x00005561c333f351 in
doris::DeleteHandler::_parse_column_pred<doris::DeleteSubPredicatePB>
(this=this@entry=0x7ff89bc0bb08,
complete_schema=..., delete_pred_related_schema=..., sub_pred_list=...,
delete_conditions=delete_conditions@entry=0x7ff89bc0b330)
at /root/be/src/olap/delete_handler.cpp:375
#3 0x00005561c333c57f in doris::DeleteHandler::init (this=0x7ff89bc0bb08,
tablet_schema=..., delete_preds=..., version=15)
at /root/be/src/olap/delete_handler.cpp:410
#4 0x00005561c3c504d7 in doris::TabletReader::_init_delete_condition
(this=this@entry=0x7ff89bc0b848, read_params=...)
at /root/be/src/olap/tablet_reader.cpp:637
#5 0x00005561c3c4e1fb in doris::TabletReader::_init_params
(this=this@entry=0x7ff89bc0b848, read_params=...)
at /root/be/src/olap/tablet_reader.cpp:297
#6 0x00005561c3c4dda6 in doris::TabletReader::init (this=0x7ff89bc0b848,
read_params=...) at /root/be/src/olap/tablet_reader.cpp:126
#7 0x00005561cd62a27a in doris::vectorized::VerticalBlockReader::init
(this=0x7ff89bc0b848, read_params=..., sample_info=0x7ff89bc0c5e0)
at /root/be/src/vec/olap/vertical_block_reader.cpp:224
#8 0x00005561c3ac6f0f in doris::Merger::vertical_compact_one_group
(tablet=...,
reader_type=reader_type@entry=doris::ReaderType::READER_BASE_COMPACTION,
tablet_schema=..., is_key=true, column_group=...,
row_source_buf=0x7ff89bc0c590, src_rowset_readers=...,
dst_rowset_writer=0x7ff928685500, max_rows_per_segment=460833,
stats_output=0x7ff89bc0cb70, key_group_cluster_key_idxes=...,
batch_size=992, sample_info=0x7ff89bc0c5e0) at /root/be/src/olap/merger.cpp:252
#9 0x00005561c3ac8e02 in doris::Merger::vertical_merge_rowsets (tablet=...,
reader_type=<optimized out>, tablet_schema=...,
src_rowset_readers=..., dst_rowset_writer=0x7ff928685500,
max_rows_per_segment=460833, merge_way_num=1, stats_output=0x7ff89bc0cb70)
at /root/be/src/olap/merger.cpp:427
#10 0x00005561c3aacade in doris::Compaction::do_compaction_impl
(this=this@entry=0x7ff92e8a9290, permits=permits@entry=3)
at /root/be/src/olap/compaction.cpp:392
#11 0x00005561c3aabcac in doris::Compaction::do_compaction
(this=0x7ff92e8a9290, permits=3) at /root/be/src/olap/compaction.cpp:140
#12 0x00005561c3c07d92 in doris::BaseCompaction::execute_compact_impl
(this=0x7ff92e8a9290) at /root/be/src/olap/base_compaction.cpp:79
#13 0x00005561c3aab8d3 in doris::Compaction::execute_compact (this=0x6) at
/root/be/src/olap/compaction.cpp:122
#14 0x00005561c3bddf1b in
doris::Tablet::execute_compaction(doris::Compaction&)::$_0::operator()() const
(this=<optimized out>)
at /root/be/src/olap/tablet.cpp:2093
#15 doris::Tablet::execute_compaction (this=0x7ff8e54f5610, compaction=...)
at /root/be/src/olap/tablet.cpp:2093
#16 0x00005561c3a7e336 in
doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>,
doris::CompactionType, bool)::$_0::operator()() const (this=0x7ff92bc351c0) at
/root/be/src/olap/olap_server.cpp:1068
#17 std::__invoke_impl<void,
doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>,
doris::CompactionType, bool)::$_0&>(std::__invoke_other,
doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>,
doris::CompactionType, bool)::$_0&) (__f=...)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<void,
doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>,
doris::CompactionType,
bool)::$_0&>(doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>,
doris::CompactionType, bool)::$_0&) (__fn=...)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#19 std::_Function_handler<void (),
doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>,
doris::CompactionType, bool)::$_0>::_M_invoke(std::_Any_data const&)
(__functor=...)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#20 0x00005561c4109fb8 in doris::ThreadPool::dispatch_thread
(this=0x7ff92b66da00) at /root/be/src/util/threadpool.cpp:544
#21 0x00005561c40ff341 in std::function<void ()>::operator()() const
(this=0x7ff9e3e65e10)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#22 doris::Thread::supervise_thread (arg=0x7ff9dd49b860) at
/root/be/src/util/thread.cpp:498
#23 0x00007ffa0a7a2215 in start_thread () from /lib64/libc.so.6
#24 0x00007ffa0a824bdc in clone3 () from /lib64/libc.so.6
```
## query core
```
#0 0x0000000000000000 in ?? ()
#1 0x000055c0b109da74 in doris::parse_to_predicate (column=..., index=6,
condition=..., arena=0x7f8357530270, opposite=true)
at /root/be/src/olap/predicate_creator.h:329
#2 0x000055c0b109d351 in
doris::DeleteHandler::_parse_column_pred<doris::DeleteSubPredicatePB>
(this=this@entry=0x7f837ab178c0,
complete_schema=..., delete_pred_related_schema=..., sub_pred_list=...,
delete_conditions=delete_conditions@entry=0x7f7f53dc5b00)
at /root/be/src/olap/delete_handler.cpp:375
#3 0x000055c0b109a57f in doris::DeleteHandler::init (this=0x7f837ab178c0,
tablet_schema=..., delete_preds=..., version=16)
at /root/be/src/olap/delete_handler.cpp:410
#4 0x000055c0b19ae4d7 in doris::TabletReader::_init_delete_condition
(this=this@entry=0x7f837ab17600, read_params=...)
at /root/be/src/olap/tablet_reader.cpp:637
#5 0x000055c0b19ac1fb in doris::TabletReader::_init_params
(this=this@entry=0x7f837ab17600, read_params=...)
at /root/be/src/olap/tablet_reader.cpp:297
#6 0x000055c0b19abda6 in doris::TabletReader::init (this=0x7f837ab17600,
read_params=...) at /root/be/src/olap/tablet_reader.cpp:126
#7 0x000055c0bb36da9f in doris::vectorized::BlockReader::init (this=0x6,
read_params=...) at /root/be/src/vec/olap/block_reader.cpp:201
#8 0x000055c0b61ddffc in doris::vectorized::NewOlapScanner::open
(this=0x7f7fac09a810, state=<optimized out>)
at /root/be/src/vec/exec/scan/new_olap_scanner.cpp:228
#9 0x000055c0b61eb5e0 in doris::vectorized::ScannerScheduler::_scanner_scan
(ctx=..., scan_task=...)
at /root/be/src/vec/exec/scan/scanner_scheduler.cpp:249
#10 0x000055c0b61ec2cd in
doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const
(this=<optimized out>)
at /root/be/src/vec/exec/scan/scanner_scheduler.cpp:170
#11
doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}::operator()() const (this=0x7f802ccdef20) at
/root/be/src/vec/exec/scan/scanner_scheduler.cpp:169
#12 std::__invoke_impl<void,
doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}&>(std::__invoke_other,
doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}&) (__f=...)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#13 std::__invoke_r<void,
doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}&) (__fn=...)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#14 std::_Function_handler<void (),
doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>,
std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()()
const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#15 0x000055c0b1e67fb8 in doris::ThreadPool::dispatch_thread
(this=0x7f8368f68200) at /root/be/src/util/threadpool.cpp:544
#16 0x000055c0b1e5d341 in std::function<void ()>::operator()() const
(this=0x7f837ab06f90)
at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#17 doris::Thread::supervise_thread (arg=0x7f8247529280) at
/root/be/src/util/thread.cpp:498
#18 0x00007f837d5a2215 in start_thread () from /lib64/libc.so.6
#19 0x00007f837d624bdc in clone3 () from /lib64/libc.so.6
```
### Summary
1. This PR fixes a BE coredump caused by legacy DeleteSubPredicatePB.op
encoding during delete predicate parsing
2. The issue affects both query scan and compaction paths and is rooted in
inconsistent operator encoding in historical rowset metadata
### Root Cause
1. DeleteSubPredicatePB read path did not normalize op before predicate
creation
2. Historical metadata may contain legacy operators such as
- >
- <
- *=
3. But predicate creation logic expects normalized internal operators
- >>
- <<
- =
4. When condition_op is not recognized, parse_to_predicate() leaves creator
function pointer null and then calls it, causing coredump
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
