Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


morningman merged PR #67442:
URL: https://github.com/apache/doris/pull/67442


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


morningman commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5519791112

   skip buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


morningman commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5519769196

   
   ### Local pipeline review — ✅ PASS
   
   ```yaml
   schema: doris-repo-review/v1
   status: PASS
   pr: apache/doris#67442
   commit: de5a2dd2b4d998e8515de555e80a3897c7a561ee
   base: 049410596f4d8c0a5c23bb5519e6b8f09dc63ab4
   reviewed_at: 2026-09-03T11:12+08:00
   reviewer: morningman
   model: claude-fable-5-1
   effort: max
   findings: {blocker: 0, major: 0, minor: 1, nit: 6}
   rounds: 1
   converged: true
   ```
   
   **Notes for maintainers**
   
   - `be/test/load/memtable/memtable_flush_executor_test.cpp:624` — the new 
tests wait for task-object destruction with fixed sleeps and unbounded `while 
(cnt == 0) sleep` loops; `pool->wait()` is deterministic here because 
`dispatch_thread` resets the runnable (`threadpool.cpp:628`) before it 
decrements `_active_threads` (`threadpool.cpp:667`).
   - `be/src/load/memtable/memtable_flush_executor.cpp:124` — the map entry is 
erased before the memtable tracker scopes below it; on the not-consumed paths 
the node allocated under `memtable->mem_tracker()` is freed under another 
tracker (few-byte residue, fatal only with 
`crash_in_memory_tracker_inaccurate`). Moving the two `SCOPED_*` guards above 
the `if` fixes it.
   - `be/src/load/memtable/memtable_flush_executor.cpp:190` — `_memtable2block` 
now treats "map != nullptr" as `need_allocated_lsn()`; true today (single 
producer `GroupRowsetWriter::init`, both builders pass a fresh data-writer 
ctx), worth pinning with a `DCHECK_EQ` here.
   - `be/src/load/memtable/memtable_flush_executor.h:31` — a forward 
declaration of `segment_v2::SegmentAllocatedLsnMap` would do instead of 
including `storage/binlog.h`; the old `struct RowsetWriterContext;` forward 
declaration is now unused.
   
   Reviewed locally with the `doris-repo-review` pipeline (a local port of 
`.github/workflows/code-review-runner.yml`). This is not a CI status 
check.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


hello-stephen commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5509886722

   # BE Regression && UT Coverage Report
   Increment line coverage `100% (0/0)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/67442_b63e944be1957bdd909ba34ae87dfa3672d14c14_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/67442_b63e944be1957bdd909ba34ae87dfa3672d14c14_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 76.22% (34681/45503) |
   | Line Coverage | 61.25% (391258/638804) |
   | Region Coverage   | 57.46% (328939/572500) |
   | Branch Coverage   | 58.23% (150017/257637) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


bosswnx commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5509634546

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


hello-stephen commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5508390793

   
   
   ClickBench: Total hot run time: 14.58 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit b63e944be1957bdd909ba34ae87dfa3672d14c14, 
data reload: false
   
   query1   0.010.010.01
   query2   0.080.040.03
   query3   0.250.120.10
   query4   1.600.090.10
   query5   0.170.150.16
   query6   1.270.730.69
   query7   0.030.010.00
   query8   0.050.030.03
   query9   0.290.210.22
   query10  0.340.360.34
   query11  0.160.120.12
   query12  0.140.120.11
   query13  0.300.320.32
   query14  0.450.440.44
   query15  0.370.360.35
   query16  0.200.240.22
   query17  0.710.690.66
   query18  0.180.150.16
   query19  1.201.131.16
   query20  0.010.010.01
   query21  15.45   0.160.13
   query22  5.030.040.05
   query23  16.19   0.250.10
   query24  3.060.330.27
   query25  0.110.050.04
   query26  0.700.160.11
   query27  0.040.030.02
   query28  3.710.540.25
   query29  12.43   3.172.56
   query30  0.250.120.13
   query31  2.750.360.18
   query32  3.560.310.23
   query33  1.401.461.41
   query34  15.39   2.161.74
   query35  1.761.731.72
   query36  0.460.280.29
   query37  0.060.040.04
   query38  0.050.040.03
   query39  0.040.020.03
   query40  0.110.070.08
   query41  0.080.020.02
   query42  0.040.020.02
   query43  0.030.020.03
   Total cold run time: 90.51 s
   Total hot run time: 14.58 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


hello-stephen commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5508342093

   
   
   TPC-DS: Total hot run time: 82483 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit b63e944be1957bdd909ba34ae87dfa3672d14c14, 
data reload: false
   
   query5   4260397 340 340
   query6   391 145 129 129
   query7   4950425 231 231
   query8   292 127 133 127
   query9   8711284028672840
   query10  403 238 183 183
   query11  53691058932 932
   query12  118 68  68  68
   query13  1204426 327 327
   query14  6117215720642064
   query14_11957194119281928
   query15  176 122 111 111
   query16  941 369 360 360
   query17  810 444 375 375
   query18  2336326 244 244
   query19  168 145 116 116
   query20  69  69  74  69
   query21  198 101 90  90
   query22  5534530453885304
   query23  6895619261246124
   query23_15949601060836010
   query24  72741105790 790
   query24_1792 787 798 787
   query25  442 310 275 275
   query26  1241242 137 137
   query27  2755393 250 250
   query28  4728150415051504
   query29  940 444 368 368
   query30  258 160 133 133
   query31  823 404 339 339
   query32  131 90  72  72
   query33  466 227 194 194
   query34  982 824 499 499
   query35  424 421 347 347
   query36  578 559 531 531
   query37  122 82  75  75
   query38  1017854 820 820
   query39  493 484 486 484
   query39_1453 497 478 478
   query40  206 92  90  90
   query41  59  57  56  56
   query42  76  74  77  74
   query43  239 237 218 218
   query44  1027558 576 558
   query45  108 109 101 101
   query46  774 838 530 530
   query47  783 767 714 714
   query48  320 313 235 235
   query49  549 242 193 193
   query50  778 268 206 206
   query51  8269810481138104
   query52  75  67  61  61
   query53  191 200 147 147
   query54  228 167 176 167
   query55  70  58  66  58
   query56  312 182 170 170
   query57  710 657 638 638
   query58  195 166 156 156
   query59  1205121911071107
   query60  228 187 178 178
   query61  116 111 125 111
   query62  353 202 187 187
   query63  171 139 140 139
   query64  2767722 600 600
   query65  1646161516881615
   query66  1877250 200 200
   query67  10023   980096009600
   query68  30111247737 737
   query69  337 220 199 199
   query70  678 617 636 617
   query71  252 173 172 172
   query72  2399172015961596
   query73  640 590 364 364
   query74  2003121411401140
   query75  11661093955 955
   query76  2359728 533 533
   query77  243 261 221 221
   query78  4032364832733273
   query79  2781870 603 603
   query80  1614338 286 286
   query81  514 157 131 131
   query82  615 136 96  96
   query83  284 208 188 188
   query84  299 112 93  93
   query85  840 364 304 304
   query86  480 175 180 175
   query87  1005983 902 902
   query88  3061212621202120
   query89  284 197 178 178
   query90  2176130 125 125
   query91  133 113 105 105
   query92  98  70  72  70
   query93  22881115687 687
   query94  662 276 218 218
   query95  508 320 230 230
   query96  812 591 290 290
   query97  10581053986 986
   query98  180 135 136 135
   query99  417 353 307 307
   Total cold run time: 180787 ms
   Total hot run time: 82483 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


hello-stephen commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5508267152

   
   
   TPC-H: Total hot run time: 16915 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit b63e944be1957bdd909ba34ae87dfa3672d14c14, 
data reload: false
   
   -- Round 1 --
   
   q1   17587   302330423023
   q2   2109258 220 220
   q3   10221   892 507 507
   q4   4675243 202 202
   q5   7680582 386 386
   q6   140 114 94  94
   q7   522 507 391 391
   q8   9236942 928 928
   q9   3492237323832373
   q10  6526848 718 718
   q11  393 193 180 180
   q12  623 252 203 203
   q13  18130   153111501150
   q14  162 153 142 142
   q15  q16 445 397 371 371
   q17  1404815 836 815
   q18  3065223922432239
   q19  1282870 739 739
   q20  391 284 200 200
   q21  5596180818591808
   q22  327 271 226 226
   Total cold run time: 94006 ms
   Total hot run time: 16915 ms
   
   - Round 2, with runtime_filter_mode=off -
   
   q1   3431335133233323
   q2   508 398 368 368
   q3   2212230321172117
   q4   11831166901 901
   q5   2181210321252103
   q6   165 122 90  90
   q7   1024948 847 847
   q8   1600141114131411
   q9   3120311231023102
   q10  1855179216131613
   q11  364 271 249 249
   q12  449 425 346 346
   q13  1461151711451145
   q14  185 173 167 167
   q15  q16 385 393 357 357
   q17  3691333732213221
   q18  4781441246984412
   q19  863 913 898 898
   q20  999 969 812 812
   q21  3859317131573157
   q22  384 357 314 314
   Total cold run time: 34700 ms
   Total hot run time: 30953 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


bosswnx commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5506414485

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](be) Own the allocated LSN map in SharedMemtable to fix UAF on flush task teardown [doris]

2026-09-02 Thread via GitHub 


hello-stephen commented on PR #67442:
URL: https://github.com/apache/doris/pull/67442#issuecomment-5506390937

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]