Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-06-02 Thread via GitHub 


morningman merged PR #51185:
URL: https://github.com/apache/doris/pull/51185


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-06-01 Thread via GitHub 


github-actions[bot] commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2927326956

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-06-01 Thread via GitHub 


github-actions[bot] commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2927326906

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-30 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2922031024

   
   
   ClickBench: Total hot run time: 29.01 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 3784bd9f23885ca49a79fe2b8d6801692747bd4d, 
data reload: false
   
   query1   0.040.030.03
   query2   0.120.100.11
   query3   0.250.190.19
   query4   1.590.190.11
   query5   0.460.420.42
   query6   1.170.670.65
   query7   0.030.020.01
   query8   0.040.040.03
   query9   0.580.530.51
   query10  0.570.600.57
   query11  0.150.110.10
   query12  0.150.110.11
   query13  0.610.600.59
   query14  0.780.810.83
   query15  0.870.870.85
   query16  0.390.390.39
   query17  1.061.031.07
   query18  0.230.210.21
   query19  1.901.821.85
   query20  0.010.000.01
   query21  15.39   0.890.56
   query22  0.751.190.65
   query23  14.99   1.390.65
   query24  7.480.960.89
   query25  0.520.260.09
   query26  0.670.160.13
   query27  0.040.050.05
   query28  9.270.920.45
   query29  12.55   4.083.35
   query30  0.250.090.06
   query31  2.820.590.38
   query32  3.230.550.47
   query33  3.073.023.11
   query34  15.79   5.194.46
   query35  4.504.564.48
   query36  0.680.490.48
   query37  0.090.060.06
   query38  0.060.040.04
   query39  0.030.020.02
   query40  0.170.150.13
   query41  0.080.020.02
   query42  0.040.020.02
   query43  0.040.030.02
   Total cold run time: 103.51 s
   Total hot run time: 29.01 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-30 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2922015899

   
   
   TPC-DS: Total hot run time: 191375 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 3784bd9f23885ca49a79fe2b8d6801692747bd4d, 
data reload: false
   
   query1   1421111810521052
   query2   6099179517621762
   query3   11022   443444784434
   query4   54109   25827   22913   22913
   query5   5170516 485 485
   query6   373 209 213 209
   query7   5023516 287 287
   query8   318 233 213 213
   query9   6183267626972676
   query10  426 341 279 279
   query11  15153   15035   14885   14885
   query12  158 113 115 113
   query13  1114517 410 410
   query14  10229   630563856305
   query15  192 207 183 183
   query16  7047624 472 472
   query17  1082714 595 595
   query18  1555412 312 312
   query19  202 196 186 186
   query20  131 122 124 122
   query21  206 123 103 103
   query22  4301436942184218
   query23  34257   33241   33640   33241
   query24  6738244824622448
   query25  487 492 409 409
   query26  694 269 154 154
   query27  2294523 344 344
   query28  3245216321662163
   query29  589 563 426 426
   query30  274 218 198 198
   query31  839 866 787 787
   query32  75  64  64  64
   query33  497 358 309 309
   query34  776 866 551 551
   query35  785 847 793 793
   query36  956 975 883 883
   query37  118 105 78  78
   query38  4277432441604160
   query39  1533143914551439
   query40  213 118 112 112
   query41  62  57  56  56
   query42  136 116 106 106
   query43  506 509 486 486
   query44  1390850 850 850
   query45  195 187 178 178
   query46  868 1051648 648
   query47  1873181117731773
   query48  387 428 346 346
   query49  655 493 410 410
   query50  652 703 416 416
   query51  4191425142404240
   query52  113 114 98  98
   query53  241 267 191 191
   query54  591 590 523 523
   query55  82  83  87  83
   query56  318 304 301 301
   query57  1157118810971097
   query58  264 268 258 258
   query59  2696274526712671
   query60  335 341 312 312
   query61  142 128 125 125
   query62  749 716 672 672
   query63  229 185 183 183
   query64  14461000669 669
   query65  4207414141484141
   query66  709 399 296 296
   query67  16002   15479   15458   15458
   query68  5480848 548 548
   query69  529 299 331 299
   query70  1220113410551055
   query71  427 317 295 295
   query72  5709477548514775
   query73  1209689 350 350
   query74  9133892186578657
   query75  3331324426992699
   query76  38891186731 731
   query77  555 368 276 276
   query78  994710046   92159215
   query79  1952823 585 585
   query80  643 512 447 447
   query81  494 276 214 214
   query82  354 123 98  98
   query83  262 247 236 236
   query84  291 109 95  95
   query85  778 406 316 316
   query86  340 313 308 308
   query87  4317439342834283
   query88  3297229523102295
   query89  401 310 279 279
   query90  1827212 212 212
   query91  138 142 113 113
   query92  72  68  58  58
   query93  2655943 578 578
   query94  672 419 309 309
   query95  363 295 289 289
   query96  495 572 283 283
   query97  2686277026332633
   query98  239 205 203 203
   query99  1320137712811281
   Total cold run time: 295382 ms
   Total hot run time: 191375 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-30 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2921982651

   
   
   TPC-H: Total hot run time: 33647 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 3784bd9f23885ca49a79fe2b8d6801692747bd4d, 
data reload: false
   
   -- Round 1 --
   q1   26380   510049554955
   q2   1960272 172 172
   q3   10313   1234724 724
   q4   10225   1000519 519
   q5   7508239623232323
   q6   179 168 132 132
   q7   909 734 619 619
   q8   9311124211071107
   q9   6738507550205020
   q10  6855231018711871
   q11  483 292 272 272
   q12  342 356 220 220
   q13  17788   371831283128
   q14  251 231 221 221
   q15  561 486 500 486
   q16  428 438 384 384
   q17  589 849 351 351
   q18  7548715669836983
   q19  1926964 571 571
   q20  324 335 221 221
   q21  3767256223672367
   q22  1054101010011001
   Total cold run time: 115439 ms
   Total hot run time: 33647 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5193503050735030
   q2   238 308 222 222
   q3   2168265523282328
   q4   1377177912921292
   q5   4431442043914391
   q6   224 167 131 131
   q7   2008196018541854
   q8   2603254525242524
   q9   7168722568756875
   q10  2991322128092809
   q11  576 516 514 514
   q12  700 790 599 599
   q13  3585398433773377
   q14  280 301 272 272
   q15  524 499 488 488
   q16  475 504 435 435
   q17  1125149113971397
   q18  7704754674227422
   q19  842 854 883 854
   q20  2024206719271927
   q21  4718454345204520
   q22  1086105710261026
   Total cold run time: 52040 ms
   Total hot run time: 50287 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-30 Thread via GitHub 


morningman commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2921737200

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-30 Thread via GitHub 


wuwenchi commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2921535463

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-29 Thread via GitHub 


morningman commented on code in PR #51185:
URL: https://github.com/apache/doris/pull/51185#discussion_r2113607866


##
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##
@@ -229,19 +231,21 @@ public void startSplit(int numBackends) throws 
UserException {
 public void doStartSplit() {
 TableScan scan = createTableScan();
 CompletableFuture.runAsync(() -> {
+AtomicReference> taskRef = new 
AtomicReference<>();
 try {
 preExecutionAuthenticator.execute(
 () -> {
 CloseableIterable fileScanTasks = 
planFileScanTask(scan);
-
-// 1. this task should stop when all splits are 
assigned
-// 2. if we want to stop this plan, we can close 
the fileScanTasks to stop
-splitAssignment.addCloseable(fileScanTasks);
-
-fileScanTasks.forEach(fileScanTask ->
-
splitAssignment.addToQueue(Lists.newArrayList(createIcebergSplit(fileScanTask;
-
-return null;
+taskRef.set(planFileScanTask(scan));

Review Comment:
   BTW, the `threadPoolWithPreAuth` in `ExternalCatalog.java` is only used in 



##
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##
@@ -229,19 +231,21 @@ public void startSplit(int numBackends) throws 
UserException {
 public void doStartSplit() {
 TableScan scan = createTableScan();
 CompletableFuture.runAsync(() -> {
+AtomicReference> taskRef = new 
AtomicReference<>();
 try {
 preExecutionAuthenticator.execute(
 () -> {
 CloseableIterable fileScanTasks = 
planFileScanTask(scan);
-
-// 1. this task should stop when all splits are 
assigned
-// 2. if we want to stop this plan, we can close 
the fileScanTasks to stop
-splitAssignment.addCloseable(fileScanTasks);
-
-fileScanTasks.forEach(fileScanTask ->
-
splitAssignment.addToQueue(Lists.newArrayList(createIcebergSplit(fileScanTask;
-
-return null;
+taskRef.set(planFileScanTask(scan));

Review Comment:
   You call `planFileScanTask()` twice?



##
fe/fe-core/src/main/java/org/apache/doris/datasource/SplitAssignment.java:
##
@@ -100,10 +108,22 @@ private void appendBatch(Multimap batch) 
throws UserException {
 for (Split split : splits) {
 locations.add(splitToScanRange.getScanRange(backend, 
locationProperties, split, pathPartitionKeys));
 }
-try {
-assignment.computeIfAbsent(backend, be -> new 
LinkedBlockingQueue<>(1)).put(locations);
-} catch (Exception e) {
-throw new UserException("Failed to offer batch split", e);
+while (true) {
+BlockingQueue> queue =
+assignment.computeIfAbsent(backend, be -> new 
LinkedBlockingQueue<>(1));
+try {
+if (queue.offer(locations, 100, TimeUnit.MILLISECONDS)) {
+return;
+}
+} catch (InterruptedException e) {
+throw new UserException("Failed to offer batch split by 
interrupted", e);
+}
+if (needMoreSplit()) {

Review Comment:
   This logic is wrong



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2918262475

   
   
   TPC-H: Total hot run time: 33752 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 78f32380563be2bb4fad78d54c41414f304fac00, 
data reload: false
   
   -- Round 1 --
   q1   26134   509150725072
   q2   2088282 191 191
   q3   10512   1242701 701
   q4   10233   998 514 514
   q5   7732233623242324
   q6   181 162 128 128
   q7   886 737 593 593
   q8   9302121810321032
   q9   6782509851115098
   q10  6820230119001900
   q11  499 286 270 270
   q12  343 345 218 218
   q13  17761   362131233123
   q14  228 226 208 208
   q15  536 480 499 480
   q16  442 445 370 370
   q17  589 832 378 378
   q18  7808717871037103
   q19  1649949 543 543
   q20  334 348 230 230
   q21  3603315323262326
   q22  10511016950 950
   Total cold run time: 115513 ms
   Total hot run time: 33752 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5082505350035003
   q2   247 325 232 232
   q3   2154265622442244
   q4   1379174814431443
   q5   4557440343664366
   q6   212 168 128 128
   q7   1964189217541754
   q8   2551255324552455
   q9   7124709671567096
   q10  3017317427412741
   q11  560 522 500 500
   q12  682 769 627 627
   q13  3541385633223322
   q14  285 291 282 282
   q15  516 483 475 475
   q16  447 494 450 450
   q17  1134150413961396
   q18  7721758774697469
   q19  762 762 769 762
   q20  1993209118491849
   q21  4685429042864286
   q22  1063103210131013
   Total cold run time: 51676 ms
   Total hot run time: 49893 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2918280870

   
   
   ClickBench: Total hot run time: 29.04 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 78f32380563be2bb4fad78d54c41414f304fac00, 
data reload: false
   
   query1   0.040.040.03
   query2   0.120.110.11
   query3   0.260.190.19
   query4   1.600.190.12
   query5   0.430.430.41
   query6   1.160.650.66
   query7   0.030.010.02
   query8   0.040.040.03
   query9   0.590.510.51
   query10  0.550.590.57
   query11  0.150.110.10
   query12  0.150.110.12
   query13  0.620.600.60
   query14  0.790.810.81
   query15  0.870.850.85
   query16  0.390.390.38
   query17  1.031.031.01
   query18  0.230.210.21
   query19  1.921.791.84
   query20  0.010.010.02
   query21  15.40   0.900.58
   query22  0.781.160.77
   query23  14.78   1.410.64
   query24  6.871.600.65
   query25  0.510.120.27
   query26  0.730.180.14
   query27  0.040.060.05
   query28  9.250.900.47
   query29  12.61   4.003.34
   query30  0.250.100.06
   query31  2.830.590.39
   query32  3.230.550.46
   query33  3.033.093.09
   query34  15.84   5.084.50
   query35  4.534.504.50
   query36  0.660.500.47
   query37  0.080.060.06
   query38  0.050.040.04
   query39  0.020.020.02
   query40  0.170.140.13
   query41  0.080.030.02
   query42  0.030.020.02
   query43  0.040.030.02
   Total cold run time: 102.79 s
   Total hot run time: 29.04 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2918274968

   
   
   TPC-DS: Total hot run time: 185650 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 78f32380563be2bb4fad78d54c41414f304fac00, 
data reload: false
   
   query1   1029492 488 488
   query2   6549178818191788
   query3   6749224 218 218
   query4   26645   23320   23062   23062
   query5   4390666 478 478
   query6   306 217 194 194
   query7   4622526 304 304
   query8   276 233 205 205
   query9   8628262026512620
   query10  490 326 276 276
   query11  15623   15155   14834   14834
   query12  157 108 107 107
   query13  1656534 410 410
   query14  8721598460735984
   query15  201 198 168 168
   query16  7110647 417 417
   query17  939 742 551 551
   query18  1955406 285 285
   query19  192 182 194 182
   query20  119 123 114 114
   query21  211 138 111 111
   query22  4195418738963896
   query23  34111   33023   33030   33023
   query24  8539238723862386
   query25  549 468 415 415
   query26  1254286 157 157
   query27  2743526 347 347
   query28  4337213721082108
   query29  791 593 446 446
   query30  292 219 191 191
   query31  924 883 776 776
   query32  74  73  65  65
   query33  576 374 320 320
   query34  818 920 554 554
   query35  809 850 728 728
   query36  10021007883 883
   query37  127 106 83  83
   query38  4023417540354035
   query39  1483144214001400
   query40  222 130 122 122
   query41  73  64  92  64
   query42  123 112 117 112
   query43  490 517 491 491
   query44  1356823 825 823
   query45  194 175 168 168
   query46  904 1088641 641
   query47  1756179617591759
   query48  399 437 307 307
   query49  750 522 404 404
   query50  667 725 413 413
   query51  4151419341294129
   query52  109 111 101 101
   query53  231 279 183 183
   query54  587 581 508 508
   query55  89  88  82  82
   query56  301 312 326 312
   query57  1148113710831083
   query58  265 255 250 250
   query59  2688263825862586
   query60  339 320 304 304
   query61  131 144 131 131
   query62  811 776 677 677
   query63  238 196 186 186
   query64  4369987 669 669
   query65  4380420142084201
   query66  1144458 316 316
   query67  15885   15406   15352   15352
   query68  7791909 578 578
   query69  482 313 273 273
   query70  1248112010831083
   query71  438 323 296 296
   query72  5697476548764765
   query73  684 654 362 362
   query74  8955905691319056
   query75  3402331026692669
   query76  32401304758 758
   query77  536 386 295 295
   query78  10015   10168   93989398
   query79  1458862 586 586
   query80  601 527 448 448
   query81  489 266 222 222
   query82  206 131 100 100
   query83  255 256 240 240
   query84  295 109 91  91
   query85  756 363 309 309
   query86  377 334 277 277
   query87  4362449542774277
   query88  3004231723052305
   query89  420 341 283 283
   query90  1977213 205 205
   query91  144 143 121 121
   query92  75  65  62  62
   query93  1600976 593 593
   query94  676 433 307 307
   query95  377 296 278 278
   query96  507 620 288 288
   query97  2726282526662666
   query98  246 210 207 207
   query99  1396148312691269
   Total cold run time: 271450 ms
   Total hot run time: 185650 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


wuwenchi commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2918077457

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2915472833

   
   
   ClickBench: Total hot run time: 28.96 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 69003e5128d0fc68883a2f8d39b434934689fa2f, 
data reload: false
   
   query1   0.040.040.04
   query2   0.130.100.11
   query3   0.260.200.20
   query4   1.600.200.20
   query5   0.480.450.45
   query6   1.180.670.66
   query7   0.020.020.02
   query8   0.040.040.03
   query9   0.600.520.52
   query10  0.570.580.57
   query11  0.160.100.11
   query12  0.150.120.11
   query13  0.610.600.60
   query14  0.790.820.81
   query15  0.880.850.86
   query16  0.390.390.39
   query17  1.081.091.04
   query18  0.230.210.22
   query19  1.901.781.82
   query20  0.020.010.01
   query21  15.41   0.900.55
   query22  0.761.240.89
   query23  14.75   1.400.65
   query24  7.521.820.37
   query25  0.310.260.11
   query26  0.630.160.14
   query27  0.050.040.05
   query28  8.760.900.44
   query29  12.53   3.983.28
   query30  0.240.090.07
   query31  2.820.610.39
   query32  3.230.550.47
   query33  3.103.063.10
   query34  15.78   5.104.51
   query35  4.544.584.49
   query36  0.660.500.48
   query37  0.090.060.06
   query38  0.060.040.04
   query39  0.030.030.03
   query40  0.170.140.14
   query41  0.080.030.02
   query42  0.030.020.03
   query43  0.040.030.02
   Total cold run time: 102.72 s
   Total hot run time: 28.96 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2915423257

   
   
   TPC-H: Total hot run time: 34151 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 69003e5128d0fc68883a2f8d39b434934689fa2f, 
data reload: false
   
   -- Round 1 --
   q1   26121   515050125012
   q2   2065278 189 189
   q3   10458   1240702 702
   q4   10233   981 530 530
   q5   7788239723092309
   q6   187 161 132 132
   q7   890 733 615 615
   q8   9324131811101110
   q9   6850508950895089
   q10  6841232119441944
   q11  501 304 284 284
   q12  346 356 218 218
   q13  17779   369531603160
   q14  258 233 223 223
   q15  533 487 476 476
   q16  431 444 376 376
   q17  620 859 383 383
   q18  7922721272047204
   q19  1732959 560 560
   q20  349 352 228 228
   q21  3774324424422442
   q22  10751025965 965
   Total cold run time: 116077 ms
   Total hot run time: 34151 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5213506251165062
   q2   237 325 231 231
   q3   2204281923212321
   q4   1355177714471447
   q5   4565443744174417
   q6   216 167 131 131
   q7   2037194417811781
   q8   2539268925442544
   q9   7188705571717055
   q10  3028316527492749
   q11  586 532 511 511
   q12  692 783 616 616
   q13  3607390632533253
   q14  297 296 265 265
   q15  533 485 464 464
   q16  456 494 431 431
   q17  1167158113581358
   q18  7852750973737373
   q19  807 801 911 801
   q20  1972201618631863
   q21  4842431243814312
   q22  10831034998 998
   Total cold run time: 52476 ms
   Total hot run time: 49983 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2915456983

   
   
   TPC-DS: Total hot run time: 185903 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 69003e5128d0fc68883a2f8d39b434934689fa2f, 
data reload: false
   
   query1   1029500 516 500
   query2   6559181017901790
   query3   6747220 225 220
   query4   26670   23752   22914   22914
   query5   4337618 460 460
   query6   300 214 200 200
   query7   4621483 293 293
   query8   298 257 249 249
   query9   8608263026322630
   query10  497 353 275 275
   query11  15362   15085   14881   14881
   query12  164 114 107 107
   query13  1654548 411 411
   query14  8877630763326307
   query15  210 194 168 168
   query16  7194636 483 483
   query17  1209726 599 599
   query18  1982393 332 332
   query19  196 195 165 165
   query20  121 122 116 116
   query21  215 131 109 109
   query22  4073426539613961
   query23  34170   33169   33138   33138
   query24  8355240824652408
   query25  554 466 402 402
   query26  1236272 157 157
   query27  2729502 340 340
   query28  4325211421042104
   query29  775 563 447 447
   query30  287 217 209 209
   query31  941 862 746 746
   query32  76  69  66  66
   query33  549 375 315 315
   query34  804 872 538 538
   query35  814 804 732 732
   query36  947 980 900 900
   query37  119 104 79  79
   query38  4067402140434021
   query39  1466141114071407
   query40  211 130 114 114
   query41  69  62  56  56
   query42  122 111 113 111
   query43  486 501 463 463
   query44  1358814 818 814
   query45  182 173 166 166
   query46  870 1027650 650
   query47  1754177517231723
   query48  385 413 329 329
   query49  780 511 448 448
   query50  668 686 417 417
   query51  4183411340704070
   query52  106 112 97  97
   query53  227 260 208 208
   query54  584 581 510 510
   query55  89  88  86  86
   query56  315 295 309 295
   query57  1132115910801080
   query58  282 280 271 271
   query59  2494260525762576
   query60  331 320 318 318
   query61  130 129 128 128
   query62  813 725 682 682
   query63  246 195 194 194
   query64  44181006689 689
   query65  4302428342704270
   query66  1148418 321 321
   query67  15924   15535   15238   15238
   query68  8491889 572 572
   query69  481 304 278 278
   query70  1251108110351035
   query71  466 336 310 310
   query72  5561479746814681
   query73  735 648 357 357
   query74  9141915687358735
   query75  3916322427162716
   query76  36501202787 787
   query77  781 396 304 304
   query78  10156   10075   93249324
   query79  2914823 585 585
   query80  636 533 460 460
   query81  488 253 226 226
   query82  501 130 93  93
   query83  281 328 235 235
   query84  295 112 96  96
   query85  806 376 315 315
   query86  391 309 289 289
   query87  4489447043564356
   query88  3627234022942294
   query89  405 316 314 314
   query90  1846220 222 220
   query91  145 158 116 116
   query92  74  68  56  56
   query93  2171968 586 586
   query94  676 421 307 307
   query95  365 292 291 291
   query96  564 566 282 282
   query97  2799272527052705
   query98  238 211 208 208
   query99  1451139812671267
   Total cold run time: 276537 ms
   Total hot run time: 185903 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-28 Thread via GitHub 


wuwenchi commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2915333179

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-23 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2903803476

   
   
   ClickBench: Total hot run time: 28.82 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 73f3c1357d65a9d3157b106cfad000e60b0771e6, 
data reload: false
   
   query1   0.040.040.04
   query2   0.130.110.11
   query3   0.240.200.19
   query4   1.600.200.11
   query5   0.430.420.41
   query6   1.150.660.65
   query7   0.030.010.02
   query8   0.040.040.04
   query9   0.590.520.52
   query10  0.550.570.57
   query11  0.160.110.11
   query12  0.140.120.11
   query13  0.620.600.60
   query14  0.790.800.81
   query15  0.890.840.83
   query16  0.370.400.38
   query17  1.061.021.05
   query18  0.230.210.21
   query19  1.931.801.82
   query20  0.010.010.02
   query21  15.43   0.890.52
   query22  0.781.120.74
   query23  14.90   1.360.64
   query24  6.791.110.57
   query25  0.550.120.07
   query26  0.560.160.13
   query27  0.050.050.06
   query28  9.050.930.45
   query29  12.56   3.993.35
   query30  0.250.090.07
   query31  2.800.610.39
   query32  3.230.540.47
   query33  3.023.043.06
   query34  15.63   5.034.49
   query35  4.564.524.51
   query36  0.660.490.48
   query37  0.080.060.07
   query38  0.050.040.04
   query39  0.030.030.02
   query40  0.170.140.13
   query41  0.080.030.03
   query42  0.030.020.02
   query43  0.040.030.03
   Total cold run time: 102.3 s
   Total hot run time: 28.82 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-23 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2903790183

   
   
   TPC-DS: Total hot run time: 192355 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 73f3c1357d65a9d3157b106cfad000e60b0771e6, 
data reload: false
   
   query1   1438109810981098
   query2   6320181718021802
   query3   10974   456444234423
   query4   53535   24336   23029   23029
   query5   5065583 478 478
   query6   331 213 202 202
   query7   4873503 290 290
   query8   315 258 246 246
   query9   5405269526612661
   query10  426 331 279 279
   query11  15053   15062   14857   14857
   query12  160 109 102 102
   query13  1030552 411 411
   query14  10357   658265226522
   query15  212 196 189 189
   query16  7141677 476 476
   query17  1100777 605 605
   query18  1573420 345 345
   query19  201 214 173 173
   query20  128 127 118 118
   query21  211 129 108 108
   query22  4399446444304430
   query23  34354   33525   33360   33360
   query24  6576245824072407
   query25  477 481 413 413
   query26  721 281 154 154
   query27  2306520 351 351
   query28  2999213921382138
   query29  593 560 455 455
   query30  272 217 197 197
   query31  892 880 789 789
   query32  74  64  63  63
   query33  457 360 311 311
   query34  787 859 534 534
   query35  827 839 742 742
   query36  932 1014870 870
   query37  112 101 76  76
   query38  4246427542094209
   query39  1545144714561447
   query40  219 132 111 111
   query41  55  55  53  53
   query42  132 126 109 109
   query43  499 511 479 479
   query44  1368867 850 850
   query45  187 181 165 165
   query46  882 1045664 664
   query47  1822184417641764
   query48  402 436 327 327
   query49  718 524 450 450
   query50  653 707 422 422
   query51  4219424042294229
   query52  108 110 98  98
   query53  238 268 185 185
   query54  584 590 513 513
   query55  87  89  85  85
   query56  298 320 314 314
   query57  1211121311371137
   query58  270 254 259 254
   query59  2664278526402640
   query60  358 335 324 324
   query61  133 134 132 132
   query62  734 755 714 714
   query63  232 194 193 193
   query64  18751110699 699
   query65  4340417742894177
   query66  729 406 308 308
   query67  15869   15623   15348   15348
   query68  7150845 532 532
   query69  541 309 276 276
   query70  1264110710941094
   query71  503 332 297 297
   query72  5897482948144814
   query73  1521652 362 362
   query74  8934921386978697
   query75  4002321227152715
   query76  42281212791 791
   query77  717 392 289 289
   query78  10149   10218   93009300
   query79  2716815 574 574
   query80  705 507 439 439
   query81  486 264 219 219
   query82  446 126 96  96
   query83  306 247 244 244
   query84  296 118 94  94
   query85  791 362 319 319
   query86  365 290 271 271
   query87  4413449742764276
   query88  3622230123022301
   query89  399 376 278 278
   query90  1882210 206 206
   query91  144 145 108 108
   query92  81  68  65  65
   query93  1979967 601 601
   query94  663 416 304 304
   query95  373 294 322 294
   query96  509 567 280 280
   query97  2750276026282628
   query98  228 210 200 200
   query99  1482138712871287
   Total cold run time: 298845 ms
   Total hot run time: 192355 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-23 Thread via GitHub 


doris-robot commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2903757796

   
   
   TPC-H: Total hot run time: 33586 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 73f3c1357d65a9d3157b106cfad000e60b0771e6, 
data reload: false
   
   -- Round 1 --
   q1   26010   509950065006
   q2   2083278 178 178
   q3   10410   1253725 725
   q4   10221   1011526 526
   q5   7522236522762276
   q6   174 161 131 131
   q7   908 737 618 618
   q8   9313123810721072
   q9   6784511250485048
   q10  6863228918931893
   q11  487 279 277 277
   q12  347 344 220 220
   q13  17799   364930653065
   q14  231 226 218 218
   q15  528 493 491 491
   q16  419 430 394 394
   q17  598 845 354 354
   q18  7623725170677067
   q19  1732939 554 554
   q20  332 327 222 222
   q21  3584255622882288
   q22  10691010963 963
   Total cold run time: 115037 ms
   Total hot run time: 33586 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5165515251115111
   q2   243 325 233 233
   q3   2158264523052305
   q4   1368175013501350
   q5   4340432443474324
   q6   227 169 129 129
   q7   1979194217751775
   q8   2557256124532453
   q9   7208721270137013
   q10  3084318827512751
   q11  573 517 505 505
   q12  675 756 596 596
   q13  3513391332633263
   q14  287 306 303 303
   q15  534 487 483 483
   q16  462 498 457 457
   q17  1127154613851385
   q18  7822762374077407
   q19  807 835 1055835
   q20  1973211618691869
   q21  4774448043794379
   q22  1080106410281028
   Total cold run time: 51956 ms
   Total hot run time: 49954 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-23 Thread via GitHub 


wuwenchi commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2903644790

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](iceberg) Fix some issues in batch mode [doris]

2025-05-23 Thread via GitHub 


hello-stephen commented on PR #51185:
URL: https://github.com/apache/doris/pull/51185#issuecomment-2903495970

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]