Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-03 Thread via GitHub


morningman merged PR #44267:
URL: https://github.com/apache/doris/pull/44267


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-03 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2515905504

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511525108

   
   
   ClickBench: Total hot run time: 33.71 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 093f49e8380bd1193739fa32d9cf6332f1e61cec, 
data reload: false
   
   query1   0.030.030.03
   query2   0.080.050.05
   query3   0.230.060.05
   query4   1.650.090.08
   query5   0.410.390.41
   query6   1.160.670.65
   query7   0.020.020.02
   query8   0.050.050.04
   query9   0.540.510.49
   query10  0.550.550.56
   query11  0.170.120.12
   query12  0.160.130.12
   query13  0.620.600.59
   query14  2.702.852.76
   query15  0.930.840.84
   query16  0.380.380.38
   query17  1.081.061.08
   query18  0.180.190.18
   query19  1.981.772.03
   query20  0.020.010.01
   query21  15.36   0.680.68
   query22  3.987.522.19
   query23  18.28   1.411.33
   query24  2.060.220.24
   query25  0.150.090.08
   query26  0.260.180.18
   query27  0.090.080.08
   query28  13.28   1.161.16
   query29  12.62   3.433.41
   query30  0.250.070.06
   query31  2.860.420.41
   query32  3.230.490.48
   query33  3.033.033.14
   query34  16.73   4.534.59
   query35  4.604.594.61
   query36  0.690.480.48
   query37  0.190.160.17
   query38  0.160.150.15
   query39  0.060.040.04
   query40  0.170.140.13
   query41  0.100.050.05
   query42  0.060.050.05
   query43  0.050.040.05
   Total cold run time: 111.2 s
   Total hot run time: 33.71 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511511977

   
   
   TPC-DS: Total hot run time: 196867 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 093f49e8380bd1193739fa32d9cf6332f1e61cec, 
data reload: false
   
   query1   1252991 962 962
   query2   6264201520832015
   query3   11081   457344144414
   query4   66809   29165   23490   23490
   query5   4998465 453 453
   query6   419 198 182 182
   query7   5517310 316 310
   query8   333 243 248 243
   query9   8840268526762676
   query10  442 245 250 245
   query11  17176   15209   15851   15209
   query12  156 103 100 100
   query13  1429418 406 406
   query14  9831677079566770
   query15  225 188 193 188
   query16  7127507 493 493
   query17  1065589 607 589
   query18  1694311 324 311
   query19  216 162 166 162
   query20  119 114 111 111
   query21  216 109 103 103
   query22  4667459545774577
   query23  34721   34215   34474   34215
   query24  5560248625902486
   query25  497 386 375 375
   query26  666 148 154 148
   query27  1872282 288 282
   query28  4283249224462446
   query29  711 413 411 411
   query30  205 149 148 148
   query31  1006813 877 813
   query32  64  54  57  54
   query33  431 284 289 284
   query34  914 522 529 522
   query35  888 761 780 761
   query36  1109976 993 976
   query37  131 70  71  70
   query38  4596450344854485
   query39  1518144815001448
   query40  207 97  102 97
   query41  44  41  44  41
   query42  105 103 103 103
   query43  538 483 502 483
   query44  1211825 823 823
   query45  198 167 169 167
   query46  1172719 704 704
   query47  2033201619251925
   query48  406 306 333 306
   query49  722 395 383 383
   query50  878 407 408 407
   query51  7455717471357135
   query52  98  86  86  86
   query53  246 182 177 177
   query54  518 392 408 392
   query55  86  73  80  73
   query56  266 251 235 235
   query57  1257113511041104
   query58  214 207 213 207
   query59  3108302329512951
   query60  257 238 232 232
   query61  102 110 103 103
   query62  828 672 666 666
   query63  213 215 198 198
   query64  1334658 629 629
   query65  3355318432293184
   query66  699 302 305 302
   query67  16074   15699   15776   15699
   query68  3860565 551 551
   query69  430 255 251 251
   query70  1204107211441072
   query71  372 250 248 248
   query72  6366415140544054
   query73  772 365 361 361
   query74  10272   901089978997
   query75  3372280227262726
   query76  2010104311371043
   query77  532 272 267 267
   query78  10389   950694729472
   query79  2137604 602 602
   query80  1355415 424 415
   query81  516 229 241 229
   query82  1290120 128 120
   query83  200 140 137 137
   query84  282 72  73  72
   query85  1008291 293 291
   query86  417 296 297 296
   query87  4686457145464546
   query88  3828222921952195
   query89  417 309 291 291
   query90  1936189 188 188
   query91  131 100 102 100
   query92  68  48  58  48
   query93  2753548 550 548
   query94  841 283 331 283
   query95  356 248 249 248
   query96  625 278 279 278
   query97  2871268826792679
   query98  213 197 198 197
   query99  1613131713491317
   Total cold run time: 319794 ms
   Total hot run time: 196867 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511480009

   
   
   TPC-H: Total hot run time: 40078 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 093f49e8380bd1193739fa32d9cf6332f1e61cec, 
data reload: false
   
   -- Round 1 --
   q1   17912   778573427342
   q2   2044183 188 183
   q3   10818   109311951093
   q4   10468   738 781 738
   q5   7606272127422721
   q6   241 149 147 147
   q7   1020617 614 614
   q8   9241185019201850
   q9   6658641565286415
   q10  7040229423372294
   q11  477 257 270 257
   q12  431 233 235 233
   q13  17770   304230683042
   q14  238 212 209 209
   q15  600 540 518 518
   q16  664 588 600 588
   q17  982 559 494 494
   q18  7276675266536653
   q19  1344909 964 909
   q20  468 184 181 181
   q21  4027332132863286
   q22  385 311 324 311
   Total cold run time: 107710 ms
   Total hot run time: 40078 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7395729672897289
   q2   324 239 225 225
   q3   2925293329612933
   q4   2167183818611838
   q5   5653567856775677
   q6   227 144 139 139
   q7   2262181818281818
   q8   3413354035153515
   q9   8873894089748940
   q10  3618354736263547
   q11  586 513 518 513
   q12  859 595 645 595
   q13  12184   329932593259
   q14  300 269 295 269
   q15  577 519 538 519
   q16  709 670 669 669
   q17  1881161416151614
   q18  8257779075727572
   q19  1688153316221533
   q20  2123190118651865
   q21  5854552055975520
   q22  639 594 567 567
   Total cold run time: 72514 ms
   Total hot run time: 60416 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511454692

   TeamCity be ut coverage result:
Function Coverage: 38.39% (9978/25989) 
Line Coverage: 29.46% (83688/284059)
Region Coverage: 28.58% (43049/150617)
Branch Coverage: 25.21% (21888/86834)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/093f49e8380bd1193739fa32d9cf6332f1e61cec_093f49e8380bd1193739fa32d9cf6332f1e61cec/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511369124

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511137654

   
   
   TPC-DS: Total hot run time: 196512 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 45b432847a6f82cdd301080f845ef7fbb778b79e, 
data reload: false
   
   query1   1250933 945 933
   query2   6238210320422042
   query3   10987   423645064236
   query4   66676   28539   23578   23578
   query5   4969455 442 442
   query6   430 189 196 189
   query7   5473305 293 293
   query8   327 244 234 234
   query9   8454272226832683
   query10  455 266 248 248
   query11  17249   15294   15861   15294
   query12  178 104 103 103
   query13  1438435 413 413
   query14  10177   713474447134
   query15  223 190 191 190
   query16  7067465 491 465
   query17  1018583 563 563
   query18  1847305 299 299
   query19  196 152 153 152
   query20  123 113 109 109
   query21  210 108 102 102
   query22  4864448044264426
   query23  34759   34540   34102   34102
   query24  5421246925282469
   query25  519 383 377 377
   query26  639 154 148 148
   query27  1856283 276 276
   query28  4751247724452445
   query29  679 414 416 414
   query30  205 153 153 153
   query31  1036862 846 846
   query32  66  56  55  55
   query33  429 303 301 301
   query34  913 500 539 500
   query35  877 782 744 744
   query36  1080983 965 965
   query37  132 73  73  73
   query38  4463439444294394
   query39  1532150814591459
   query40  201 97  101 97
   query41  43  42  44  42
   query42  112 96  98  96
   query43  532 503 497 497
   query44  1165815 803 803
   query45  191 174 167 167
   query46  1183727 724 724
   query47  2022193119181918
   query48  407 315 316 315
   query49  733 394 370 370
   query50  845 387 398 387
   query51  7267722770277027
   query52  107 89  89  89
   query53  255 181 183 181
   query54  521 414 407 407
   query55  82  74  76  74
   query56  255 242 249 242
   query57  12761142
   query58  234 216 217 216
   query59  3226316130673067
   query60  271 266 249 249
   query61  111 104 104 104
   query62  773 670 653 653
   query63  207 182 195 182
   query64  1375669 647 647
   query65  3231320931653165
   query66  719 304 303 303
   query67  16074   15562   15476   15476
   query68  3919557 538 538
   query69  427 247 238 238
   query70  1191112311401123
   query71  327 245 253 245
   query72  6411404541614045
   query73  753 360 363 360
   query74  10279   914590599059
   query75  3408268126632663
   query76  1888107110261026
   query77  499 259 272 259
   query78  10530   951794539453
   query79  1482611 592 592
   query80  897 448 438 438
   query81  482 244 227 227
   query82  1276120 116 116
   query83  256 147 139 139
   query84  290 71  71  71
   query85  915 393 298 298
   query86  351 291 301 291
   query87  4857454145684541
   query88  3760221621782178
   query89  427 293 292 292
   query90  1999195 184 184
   query91  131 106 101 101
   query92  62  49  52  49
   query93  1995541 541 541
   query94  770 283 276 276
   query95  345 242 247 242
   query96  628 276 282 276
   query97  2870270226962696
   query98  225 198 195 195
   query99  1622132613021302
   Total cold run time: 317896 ms
   Total hot run time: 196512 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2511107892

   
   
   TPC-H: Total hot run time: 40571 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 45b432847a6f82cdd301080f845ef7fbb778b79e, 
data reload: false
   
   -- Round 1 --
   q1   17577   773574357435
   q2   2059182 163 163
   q3   10661   113412471134
   q4   10560   756 703 703
   q5   7649279627362736
   q6   251 152 152 152
   q7   1006633 603 603
   q8   9249188519631885
   q9   6646654565526545
   q10  7019232923792329
   q11  468 264 267 264
   q12  431 229 226 226
   q13  17780   304430493044
   q14  229 214 217 214
   q15  580 541 532 532
   q16  645 594 599 594
   q17  987 599 573 573
   q18  7543663167726631
   q19  1331106310671063
   q20  474 180 180 180
   q21  4507333532523252
   q22  462 313 319 313
   Total cold run time: 108114 ms
   Total hot run time: 40571 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7374737572967296
   q2   323 231 228 228
   q3   2953298530252985
   q4   2112186018591859
   q5   5663573256775677
   q6   231 147 147 147
   q7   2265181218361812
   q8   3369356336203563
   q9   8996906590439043
   q10  3616362235643564
   q11  618 502 494 494
   q12  853 676 608 608
   q13  12596   328332113211
   q14  316 291 288 288
   q15  580 533 541 533
   q16  678 644 631 631
   q17  1890166916111611
   q18  8222784376347634
   q19  1740166514551455
   q20  2125190918771877
   q21  5851561555525552
   q22  637 591 572 572
   Total cold run time: 73008 ms
   Total hot run time: 60640 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-02 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510870017

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


morningman commented on code in PR #44267:
URL: https://github.com/apache/doris/pull/44267#discussion_r1865305204


##
fe/be-java-extensions/hudi-scanner/src/main/scala/org/apache/doris/hudi/BaseSplitReader.scala:
##
@@ -497,8 +499,11 @@ abstract class BaseSplitReader(val split: HoodieSplit) {
 options: Map[String, String],
 hadoopConf: Configuration): PartitionedFile => 
Iterator[InternalRow] = {
 partitionedFile => {
-  val reader = new HoodieAvroHFileReader(
-hadoopConf, partitionedFile.filePath.toPath, new 
CacheConfig(hadoopConf))
+  var hadoopStorageConfiguration = new 
HadoopStorageConfiguration(hadoopConf);
+  var sotragePath = new StoragePath(partitionedFile.toPath.toUri.getPath);

Review Comment:
   ```suggestion
 var storagePath = new 
StoragePath(partitionedFile.toPath.toUri.getPath);
   ```



##
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##
@@ -142,6 +141,19 @@ public static  String encodeObjectToString(T t) {
 @Override
 protected void setScanParams(TFileRangeDesc rangeDesc, Split split) {
 if (split instanceof PaimonSplit) {
+PaimonSplit paimonSplit = (PaimonSplit) split;
+if (rangeDesc.getFormatType() == TFileFormatType.FORMAT_JNI
+&& paimonSplit.getSplit() == null) {
+// fall back to JNI reader

Review Comment:
   ```suggestion
   // fall back to native reader
   ```



##
be/src/vec/exec/scan/vfile_scanner.cpp:
##
@@ -754,28 +750,10 @@ Status VFileScanner::_get_next_reader() {
 
 // create reader for specific format
 Status init_status;
-TFileFormatType::type format_type = _params->format_type;
+TFileFormatType::type format_type = range.format_type;
 // JNI reader can only push down column value range
 bool push_down_predicates =
 !_is_load && _params->format_type != 
TFileFormatType::FORMAT_JNI;
-if (format_type == TFileFormatType::FORMAT_JNI && 
range.__isset.table_format_params) {

Review Comment:
   This need to be reserved.
   Because when upgrading, the BE will be upgrade, and the old FE will send old 
format request to new BE.
   You can reserve this logic but mark it as deprecated in 3.1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510519107

   
   
   ClickBench: Total hot run time: 33.25 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 07cb8434938075bd10736b2672dbe6f3318b1e74, 
data reload: false
   
   query1   0.030.030.03
   query2   0.080.050.05
   query3   0.220.060.06
   query4   1.660.080.08
   query5   0.420.390.41
   query6   1.150.660.67
   query7   0.020.010.01
   query8   0.050.050.04
   query9   0.560.510.50
   query10  0.550.550.55
   query11  0.160.130.12
   query12  0.150.140.13
   query13  0.620.610.60
   query14  2.862.732.73
   query15  0.900.840.84
   query16  0.390.370.37
   query17  1.061.001.00
   query18  0.200.180.19
   query19  1.921.781.95
   query20  0.020.020.01
   query21  15.35   0.700.67
   query22  4.827.451.80
   query23  18.20   1.421.36
   query24  2.140.220.22
   query25  0.150.090.09
   query26  0.260.180.18
   query27  0.070.080.08
   query28  13.25   1.161.14
   query29  12.64   3.363.37
   query30  0.250.060.06
   query31  2.880.420.41
   query32  3.230.490.49
   query33  3.083.033.09
   query34  16.88   4.574.56
   query35  4.574.614.59
   query36  0.660.470.48
   query37  0.200.160.15
   query38  0.160.150.15
   query39  0.060.040.04
   query40  0.160.130.12
   query41  0.100.060.06
   query42  0.060.050.05
   query43  0.040.040.04
   Total cold run time: 112.23 s
   Total hot run time: 33.25 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510514535

   
   
   TPC-DS: Total hot run time: 197569 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 07cb8434938075bd10736b2672dbe6f3318b1e74, 
data reload: false
   
   query1   1246976 955 955
   query2   6232211020872087
   query3   11053   446444754464
   query4   67382   29089   23530   23530
   query5   4887465 452 452
   query6   434 189 178 178
   query7   5506315 301 301
   query8   324 233 224 224
   query9   8645272527302725
   query10  443 246 241 241
   query11  17008   15317   15830   15317
   query12  154 101 100 100
   query13  1413434 418 418
   query14  10719   712267486748
   query15  222 214 201 201
   query16  7252494 533 494
   query17  1074605 594 594
   query18  1922340 310 310
   query19  230 159 178 159
   query20  129 114 115 114
   query21  202 110 104 104
   query22  4576456846364568
   query23  35462   35385   34557   34557
   query24  5690253025312530
   query25  503 375 399 375
   query26  650 163 152 152
   query27  1955282 292 282
   query28  4559249225062492
   query29  650 420 411 411
   query30  211 158 146 146
   query31  998 843 873 843
   query32  62  53  97  53
   query33  422 312 286 286
   query34  944 519 526 519
   query35  871 768 751 751
   query36  1098949 934 934
   query37  118 75  75  75
   query38  4612446943114311
   query39  1511149014601460
   query40  207 105 104 104
   query41  45  42  41  41
   query42  110 100 99  99
   query43  532 496 508 496
   query44  1213828 820 820
   query45  189 169 171 169
   query46  1158741 752 741
   query47  2059193719621937
   query48  421 312 327 312
   query49  744 383 400 383
   query50  844 400 395 395
   query51  7339722071937193
   query52  99  85  84  84
   query53  248 183 179 179
   query54  500 410 398 398
   query55  77  89  79  79
   query56  246 251 236 236
   query57  1246111311351113
   query58  220 202 222 202
   query59  3228309229572957
   query60  267 253 239 239
   query61  109 119 109 109
   query62  795 682 660 660
   query63  212 195 201 195
   query64  1456659 637 637
   query65  3305325532553255
   query66  721 303 303 303
   query67  16193   15680   15667   15667
   query68  3740572 575 572
   query69  471 247 248 247
   query70  1203109111521091
   query71  394 246 255 246
   query72  6092415840554055
   query73  772 360 371 360
   query74  10188   908891049088
   query75  3400267526672667
   query76  1938111311061106
   query77  493 274 275 274
   query78  10543   944692779277
   query79  2123595 613 595
   query80  1351421 432 421
   query81  524 248 231 231
   query82  1283118 113 113
   query83  259 144 140 140
   query84  282 69  70  69
   query85  1004343 314 314
   query86  420 300 302 300
   query87  4664460246194602
   query88  3720222021942194
   query89  419 301 285 285
   query90  1936187 180 180
   query91  135 102 102 102
   query92  70  50  50  50
   query93  3150533 549 533
   query94  818 292 298 292
   query95  358 249 240 240
   query96  644 275 282 275
   query97  2879271126892689
   query98  221 204 202 202
   query99  1610130512951295
   Total cold run time: 322256 ms
   Total hot run time: 197569 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510491661

   
   
   TPC-H: Total hot run time: 39878 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 07cb8434938075bd10736b2672dbe6f3318b1e74, 
data reload: false
   
   -- Round 1 --
   q1   17986   803572507250
   q2   2056178 167 167
   q3   10751   110411671104
   q4   10533   701 761 701
   q5   7612272526432643
   q6   239 150 150 150
   q7   1008639 602 602
   q8   9246183819521838
   q9   6705649965156499
   q10  7011233023162316
   q11  469 262 259 259
   q12  425 228 231 228
   q13  17782   301830773018
   q14  249 214 209 209
   q15  584 528 520 520
   q16  667 587 575 575
   q17  986 578 512 512
   q18  7371678367666766
   q19  13441043952 952
   q20  457 201 178 178
   q21  4042316930893089
   q22  388 320 302 302
   Total cold run time: 107911 ms
   Total hot run time: 39878 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7372725972207220
   q2   322 226 230 226
   q3   2945296430472964
   q4   2079187519621875
   q5   5629567757295677
   q6   220 143 141 141
   q7   2331185418581854
   q8   3353348235153482
   q9   8978900990449009
   q10  3627361535633563
   q11  603 518 494 494
   q12  816 624 647 624
   q13  12102   331932413241
   q14  305 272 280 272
   q15  584 537 523 523
   q16  677 665 640 640
   q17  1848162615801580
   q18  8438763476367634
   q19  1693164316051605
   q20  2164191418741874
   q21  5769562454365436
   q22  644 576 600 576
   Total cold run time: 72499 ms
   Total hot run time: 60510 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510440109

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510075041

   
   
   ClickBench: Total hot run time: 33.4 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 68a9ea1d1a387cc158f225927fac2f422dfc6815, 
data reload: false
   
   query1   0.040.030.02
   query2   0.070.030.03
   query3   0.240.070.07
   query4   1.620.100.11
   query5   0.420.420.42
   query6   1.140.680.67
   query7   0.020.020.02
   query8   0.040.040.03
   query9   0.570.530.50
   query10  0.540.570.54
   query11  0.140.110.10
   query12  0.140.110.12
   query13  0.600.600.59
   query14  2.702.782.83
   query15  0.900.820.82
   query16  0.380.380.38
   query17  1.081.061.07
   query18  0.230.210.20
   query19  1.971.881.97
   query20  0.010.010.01
   query21  15.36   0.620.60
   query22  2.832.492.32
   query23  16.86   0.960.90
   query24  3.121.121.39
   query25  0.310.120.14
   query26  0.380.150.13
   query27  0.040.040.04
   query28  10.30   1.091.07
   query29  12.56   3.173.16
   query30  0.250.060.06
   query31  2.840.390.38
   query32  3.270.460.47
   query33  2.963.073.00
   query34  17.12   4.444.46
   query35  4.504.534.48
   query36  0.670.480.48
   query37  0.090.060.06
   query38  0.050.030.03
   query39  0.030.020.02
   query40  0.170.130.13
   query41  0.070.020.02
   query42  0.030.020.02
   query43  0.040.030.03
   Total cold run time: 106.7 s
   Total hot run time: 33.4 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510060776

   
   
   TPC-DS: Total hot run time: 191399 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 68a9ea1d1a387cc158f225927fac2f422dfc6815, 
data reload: false
   
   query1   971 381 370 370
   query2   6534212220542054
   query3   6713205 213 205
   query4   34205   23584   23592   23584
   query5   4371463 442 442
   query6   270 179 172 172
   query7   4629301 304 301
   query8   299 250 231 231
   query9   9737270727192707
   query10  478 253 257 253
   query11  18395   15311   15223   15223
   query12  147 101 101 101
   query13  1646406 408 406
   query14  9425717669676967
   query15  258 171 179 171
   query16  7972444 485 444
   query17  1689581 562 562
   query18  1928335 279 279
   query19  357 150 141 141
   query20  113 108 108 108
   query21  203 101 98  98
   query22  4605431342864286
   query23  34715   35484   34808   34808
   query24  11489   243423532353
   query25  544 372 369 369
   query26  1215155 151 151
   query27  2803283 274 274
   query28  8306243224372432
   query29  657 401 396 396
   query30  303 162 149 149
   query31  1025801 818 801
   query32  96  55  56  55
   query33  786 291 281 281
   query34  970 504 524 504
   query35  872 735 720 720
   query36  1107940 950 940
   query37  191 78  72  72
   query38  4412424943214249
   query39  1481142014231420
   query40  285 99  98  98
   query41  51  47  45  45
   query42  113 100 97  97
   query43  526 488 491 488
   query44  1226795 787 787
   query45  180 164 161 161
   query46  1170693 720 693
   query47  1978184718811847
   query48  395 313 316 313
   query49  1262379 403 379
   query50  802 392 368 368
   query51  7164705470677054
   query52  100 91  87  87
   query53  256 180 183 180
   query54  1104383 386 383
   query55  77  76  81  76
   query56  260 229 238 229
   query57  1238113211031103
   query58  219 225 211 211
   query59  3178291929652919
   query60  266 232 244 232
   query61  110 106 107 106
   query62  883 668 684 668
   query63  213 184 178 178
   query64  5009690 640 640
   query65  3286326732343234
   query66  1319309 304 304
   query67  16031   15508   15695   15508
   query68  5500540 541 540
   query69  405 254 248 248
   query70  1218114611241124
   query71  351 254 247 247
   query72  6063403840334033
   query73  772 363 365 363
   query74  10369   900890669008
   query75  3370263926062606
   query76  3285111410941094
   query77  550 270 270 270
   query78  10425   936493539353
   query79  1699596 603 596
   query80  1113432 423 423
   query81  524 233 233 233
   query82  927 113 117 113
   query83  255 147 139 139
   query84  229 121 69  69
   query85  1379297 288 288
   query86  369 305 312 305
   query87  4812457545944575
   query88  3390224021842184
   query89  398 292 299 292
   query90  2111183 180 180
   query91  138 99  101 99
   query92  57  50  50  50
   query93  1093548 548 548
   query94  1132285 287 285
   query95  364 243 246 243
   query96  612 274 282 274
   query97  2862265927202659
   query98  228 196 188 188
   query99  1546130313091303
   Total cold run time: 302164 ms
   Total hot run time: 191399 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2510030669

   
   
   TPC-H: Total hot run time: 40361 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 68a9ea1d1a387cc158f225927fac2f422dfc6815, 
data reload: false
   
   -- Round 1 --
   q1   17617   795572457245
   q2   2054188 188 188
   q3   10596   116312341163
   q4   10246   755 669 669
   q5   7600279826992699
   q6   238 145 146 145
   q7   1015628 602 602
   q8   9254193019351930
   q9   6715658365366536
   q10  7003227323352273
   q11  453 260 263 260
   q12  414 229 229 229
   q13  17769   299830082998
   q14  254 217 210 210
   q15  570 534 526 526
   q16  628 589 620 589
   q17  994 573 576 573
   q18  7502676767876767
   q19  1340995 1014995
   q20  469 185 187 185
   q21  4150327633563276
   q22  375 303 314 303
   Total cold run time: 107256 ms
   Total hot run time: 40361 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7238725272257225
   q2   328 227 235 227
   q3   2925287430142874
   q4   2075185118621851
   q5   5656570056555655
   q6   218 138 139 138
   q7   2168176118511761
   q8   3353352235083508
   q9   8948907690679067
   q10  3593354835703548
   q11  606 504 510 504
   q12  814 640 603 603
   q13  10661   318332083183
   q14  303 270 270 270
   q15  570 547 528 528
   q16  666 642 617 617
   q17  1799159215701570
   q18  7969750875377508
   q19  1661155215761552
   q20  2074182718161816
   q21  5430518253365182
   q22  661 559 571 559
   Total cold run time: 69716 ms
   Total hot run time: 59746 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-12-01 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2509816140

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-30 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2508978514

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-30 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2508978506

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2507123288

   TeamCity be ut coverage result:
Function Coverage: 38.33% (9976/26029) 
Line Coverage: 29.41% (83552/284058)
Region Coverage: 28.53% (42996/150693)
Branch Coverage: 25.15% (21840/86822)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/b1d46693ee6d59f851f18ad679586e5bd558d2d5_b1d46693ee6d59f851f18ad679586e5bd558d2d5/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2507100810

   
   
   ClickBench: Total hot run time: 32.94 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit b1d46693ee6d59f851f18ad679586e5bd558d2d5, 
data reload: false
   
   query1   0.030.030.03
   query2   0.060.030.03
   query3   0.230.060.06
   query4   1.640.100.11
   query5   0.420.400.39
   query6   1.150.650.64
   query7   0.020.010.01
   query8   0.040.030.02
   query9   0.560.520.51
   query10  0.550.560.56
   query11  0.140.100.11
   query12  0.140.120.12
   query13  0.610.590.60
   query14  2.812.842.74
   query15  0.890.820.82
   query16  0.380.380.39
   query17  1.001.051.01
   query18  0.230.220.22
   query19  1.851.811.88
   query20  0.020.010.01
   query21  15.36   0.630.57
   query22  2.233.061.75
   query23  17.01   0.860.89
   query24  2.931.401.47
   query25  0.160.030.21
   query26  0.590.150.14
   query27  0.070.040.04
   query28  10.22   1.101.07
   query29  12.52   3.343.34
   query30  0.250.060.05
   query31  2.880.370.38
   query32  3.280.470.46
   query33  2.972.993.01
   query34  17.06   4.454.45
   query35  4.534.504.49
   query36  0.680.480.49
   query37  0.090.060.06
   query38  0.050.030.04
   query39  0.030.030.02
   query40  0.160.120.12
   query41  0.080.030.02
   query42  0.040.020.02
   query43  0.040.030.04
   Total cold run time: 106 s
   Total hot run time: 32.94 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2507096253

   
   
   TPC-DS: Total hot run time: 191002 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit b1d46693ee6d59f851f18ad679586e5bd558d2d5, 
data reload: false
   
   query1   988 385 366 366
   query2   6509220821242124
   query3   6706208 210 208
   query4   34292   23508   23824   23508
   query5   4379455 440 440
   query6   294 184 187 184
   query7   4634303 300 300
   query8   298 235 230 230
   query9   9722274527272727
   query10  479 261 259 259
   query11  18122   15371   15245   15245
   query12  155 105 99  99
   query13  1677426 416 416
   query14  10174   653467446534
   query15  282 186 186 186
   query16  8148455 491 455
   query17  1682583 575 575
   query18  2143290 303 290
   query19  365 147 151 147
   query20  115 113 108 108
   query21  204 101 98  98
   query22  4446417141934171
   query23  35108   34254   34311   34254
   query24  10772   249925302499
   query25  658 408 378 378
   query26  1639153 154 153
   query27  2894277 285 277
   query28  8248244424232423
   query29  988 415 404 404
   query30  296 151 143 143
   query31  1043813 819 813
   query32  122 56  63  56
   query33  779 296 311 296
   query34  956 509 556 509
   query35  905 777 741 741
   query36  1101936 972 936
   query37  145 77  82  77
   query38  4280421641564156
   query39  1482144715301447
   query40  286 103 100 100
   query41  50  48  46  46
   query42  115 98  100 98
   query43  554 520 498 498
   query44  1242845 804 804
   query45  186 164 168 164
   query46  1155706 688 688
   query47  1952183418401834
   query48  406 304 320 304
   query49  1221382 409 382
   query50  809 388 388 388
   query51  7165697370946973
   query52  97  92  91  91
   query53  251 180 188 180
   query54  1234401 411 401
   query55  88  77  84  77
   query56  261 226 237 226
   query57  1245112011401120
   query58  240 212 214 212
   query59  3264317432873174
   query60  293 262 238 238
   query61  109 110 112 110
   query62  886 722 693 693
   query63  218 187 186 186
   query64  5109655 635 635
   query65  3313320132263201
   query66  1435314 312 312
   query67  16238   15574   15678   15574
   query68  5340554 554 554
   query69  415 250 249 249
   query70  1212111711061106
   query71  351 246 266 246
   query72  6435414741934147
   query73  774 361 369 361
   query74  10535   914389268926
   query75  3454266627792666
   query76  3173112210601060
   query77  501 266 285 266
   query78  10406   933393239323
   query79  2546608 632 608
   query80  1187425 426 425
   query81  524 236 238 236
   query82  732 121 121 121
   query83  244 151 143 143
   query84  233 74  77  74
   query85  1566310 303 303
   query86  457 307 302 302
   query87  4683457645244524
   query88  4093222422112211
   query89  408 303 299 299
   query90  2167190 187 187
   query91  138 107 104 104
   query92  65  55  52  52
   query93  1630550 538 538
   query94  1078294 296 294
   query95  363 252 257 252
   query96  623 280 284 280
   query97  2849267926902679
   query98  226 192 193 192
   query99  1538133313471333
   Total cold run time: 306598 ms
   Total hot run time: 191002 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2507087683

   
   
   TPC-H: Total hot run time: 40067 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit b1d46693ee6d59f851f18ad679586e5bd558d2d5, 
data reload: false
   
   -- Round 1 --
   q1   17596   802173497349
   q2   2049179 166 166
   q3   10584   116711241124
   q4   10214   724 780 724
   q5   7612277227302730
   q6   244 148 146 146
   q7   990 621 595 595
   q8   9231193119281928
   q9   6663647966196479
   q10  7031234923082308
   q11  461 259 268 259
   q12  429 222 224 222
   q13  17754   298430102984
   q14  250 207 212 207
   q15  570 540 518 518
   q16  682 581 603 581
   q17  1009544 539 539
   q18  7412670867876708
   q19  1342974 973 973
   q20  471 183 179 179
   q21  4008319030283028
   q22  388 328 320 320
   Total cold run time: 106990 ms
   Total hot run time: 40067 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7304729272667266
   q2   333 242 241 241
   q3   2942281428412814
   q4   1971171016871687
   q5   5414542854155415
   q6   223 143 139 139
   q7   2136170017041700
   q8   3256345734763457
   q9   8712863786798637
   q10  3507350434613461
   q11  590 490 476 476
   q12  811 572 590 572
   q13  12136   303830253025
   q14  299 253 277 253
   q15  567 510 514 510
   q16  691 636 646 636
   q17  1812161016111610
   q18  7983759874977497
   q19  1693159514441444
   q20  2089182318071807
   q21  5468530552205220
   q22  661 596 566 566
   Total cold run time: 70598 ms
   Total hot run time: 58433 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2507041951

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2507024751

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-28 Thread via GitHub


morningman commented on code in PR #44267:
URL: https://github.com/apache/doris/pull/44267#discussion_r1862327637


##
be/src/vec/exec/format/table/hudi_jni_reader.cpp:
##
@@ -65,16 +65,21 @@ HudiJniReader::HudiJniReader(const TFileScanRangeParams& 
scan_params,
 {"input_format", _hudi_params.input_format}};
 
 // Use compatible hadoop client to read data
-for (auto& kv : _scan_params.properties) {
+for (const auto& kv : _scan_params.properties) {
 if (kv.first.starts_with(HOODIE_CONF_PREFIX)) {
 params[kv.first] = kv.second;
 } else {
 params[HADOOP_CONF_PREFIX + kv.first] = kv.second;
 }
 }
 
-_jni_connector = 
std::make_unique("org/apache/doris/hudi/HudiJniScanner", params,
-required_fields);
+if (config::use_old_hudi_jni_reader) [[unlikely]] {

Review Comment:
   Better use a session variable
   
   set hudi_jni_scanner = "spark" or "hadoop".
   and default is hadoop



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-26 Thread via GitHub


wuwenchi commented on code in PR #44267:
URL: https://github.com/apache/doris/pull/44267#discussion_r1859641277


##
gensrc/thrift/PlanNodes.thrift:
##
@@ -448,6 +447,8 @@ struct TFileScanRangeParams {
 22: optional TTextSerdeType  text_serde_type 
 // used by flexible partial update
 23: optional string sequence_map_col
+// if set true, be will be forced to use jni reader
+24: bool force_jni_reader; 

Review Comment:
   Should this behavior be specified in FE? Otherwise, when checking the plan, 
we won't know how many JNIs have been used.



##
fe/be-java-extensions/hadoop-hudi-scanner/src/main/java/org/apache/doris/hudi/HadoopHudiJniScanner.java:
##
@@ -0,0 +1,272 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.hudi;
+
+import org.apache.doris.common.classloader.ThreadClassLoaderContext;
+import org.apache.doris.common.jni.JniScanner;
+import org.apache.doris.common.jni.vec.ColumnType;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.Maps;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.common.JavaUtils;
+import org.apache.hadoop.hive.serde2.Deserializer;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.StructField;
+import org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector;
+import org.apache.hadoop.io.ArrayWritable;
+import org.apache.hadoop.io.NullWritable;
+import org.apache.hadoop.mapred.FileSplit;
+import org.apache.hadoop.mapred.InputFormat;
+import org.apache.hadoop.mapred.JobConf;
+import org.apache.hadoop.mapred.RecordReader;
+import org.apache.hadoop.mapred.Reporter;
+import org.apache.hadoop.util.ReflectionUtils;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.time.ZoneId;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+/**
+ * HadoopHudiJniScanner is a JniScanner implementation that reads Hudi data 
using hudi-hadoop-mr.
+ */
+public class HadoopHudiJniScanner extends JniScanner {
+private static final Logger LOG = 
LoggerFactory.getLogger(HadoopHudiJniScanner.class);
+
+private static final String HADOOP_CONF_PREFIX = "hadoop_conf.";
+
+// Hudi data info
+private final String basePath;
+private final String dataFilePath;
+private final long dataFileLength;
+private final String[] deltaFilePaths;
+private final String instantTime;
+private final String serde;
+private final String inputFormat;
+
+// schema info
+private final String hudiColumnNames;
+private final String[] hudiColumnTypes;
+private final String[] requiredFields;
+private List requiredColumnIds;
+private ColumnType[] requiredTypes;
+// private final String[] nestedFields;

Review Comment:
   delete useless variable



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-25 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2498388906

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-25 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2497793065

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-25 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2497730771

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-24 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2497131793

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-22 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2493362005

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2492991571

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2492815038

   
   
   TPC-H: Total hot run time: 40408 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 01413781a887846e4a866acc572e23fc4a456004, 
data reload: false
   
   -- Round 1 --
   q1   17633   761772987298
   q2   2059174 186 174
   q3   10706   111611621116
   q4   10447   818 743 743
   q5   7605286427412741
   q6   244 145 144 144
   q7   992 637 624 624
   q8   9251192019591920
   q9   6639646464736464
   q10  7053233123302330
   q11  472 259 260 259
   q12  436 219 211 211
   q13  17790   306430523052
   q14  241 208 211 208
   q15  570 523 522 522
   q16  674 587 589 587
   q17  1002583 599 583
   q18  7401673867266726
   q19  13631058975 975
   q20  469 182 182 182
   q21  3966323332483233
   q22  383 321 316 316
   Total cold run time: 107396 ms
   Total hot run time: 40408 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7313781772677267
   q2   326 228 231 228
   q3   2920294729942947
   q4   2066187418441844
   q5   5695571957405719
   q6   231 140 141 140
   q7   2265186318411841
   q8   3449360935503550
   q9   8957900090549000
   q10  3618356435933564
   q11  598 523 499 499
   q12  827 614 598 598
   q13  10428   324332653243
   q14  307 263 292 263
   q15  592 509 522 509
   q16  721 661 662 661
   q17  1887165416131613
   q18  8537788477257725
   q19  1809156916121569
   q20  2106190219101902
   q21  5719551853795379
   q22  658 565 549 549
   Total cold run time: 71029 ms
   Total hot run time: 60610 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2492727481

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2492720597

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490546504

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490486337

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490417180

   
   
   ClickBench: Total hot run time: 32.65 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit da2c6d4220c27610dc537441dfcc887ff06a08ed, 
data reload: false
   
   query1   0.030.040.03
   query2   0.060.040.03
   query3   0.240.070.07
   query4   1.630.100.11
   query5   0.410.410.41
   query6   1.140.660.66
   query7   0.020.020.01
   query8   0.040.020.03
   query9   0.580.510.51
   query10  0.560.560.56
   query11  0.150.100.11
   query12  0.130.100.11
   query13  0.610.600.60
   query14  2.772.832.75
   query15  0.910.820.83
   query16  0.370.380.40
   query17  1.071.051.06
   query18  0.220.210.20
   query19  1.961.781.96
   query20  0.020.010.01
   query21  15.37   0.590.58
   query22  2.492.572.21
   query23  17.11   0.930.75
   query24  2.630.781.17
   query25  0.270.060.06
   query26  0.400.130.14
   query27  0.040.040.05
   query28  11.35   1.091.07
   query29  12.52   3.243.24
   query30  0.250.060.06
   query31  2.870.380.37
   query32  3.310.460.46
   query33  2.993.033.06
   query34  16.63   4.504.43
   query35  4.564.464.55
   query36  0.680.490.48
   query37  0.090.060.07
   query38  0.050.040.04
   query39  0.030.020.03
   query40  0.170.120.12
   query41  0.090.020.02
   query42  0.040.030.02
   query43  0.040.030.03
   Total cold run time: 106.9 s
   Total hot run time: 32.65 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490405100

   
   
   TPC-DS: Total hot run time: 197412 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit da2c6d4220c27610dc537441dfcc887ff06a08ed, 
data reload: false
   
   query1   1271944 928 928
   query2   6247210820472047
   query3   10775   404139703970
   query4   67474   29435   23721   23721
   query5   4933463 461 461
   query6   421 192 180 180
   query7   5542307 290 290
   query8   316 229 229 229
   query9   8625268626572657
   query10  467 272 288 272
   query11  17288   15341   15929   15341
   query12  161 110 109 109
   query13  1477414 430 414
   query14  10837   744475507444
   query15  226 198 197 197
   query16  7099468 488 468
   query17  1034563 574 563
   query18  1833307 312 307
   query19  213 160 149 149
   query20  126 108 114 108
   query21  205 108 114 108
   query22  4710449246244492
   query23  34812   34207   34442   34207
   query24  5490249925832499
   query25  494 400 415 400
   query26  682 153 157 153
   query27  1902287 296 287
   query28  4332245724442444
   query29  685 428 427 427
   query30  223 152 150 150
   query31  997 817 840 817
   query32  65  59  64  59
   query33  405 296 276 276
   query34  928 528 511 511
   query35  893 739 738 738
   query36  1123988 997 988
   query37  117 69  77  69
   query38  4458445744224422
   query39  1514148314511451
   query40  202 100 105 100
   query41  48  43  45  43
   query42  113 99  102 99
   query43  541 516 501 501
   query44  1206855 832 832
   query45  189 178 167 167
   query46  1174717 701 701
   query47  2027191519501915
   query48  437 340 334 334
   query49  748 419 422 419
   query50  844 397 396 396
   query51  7406722970287028
   query52  103 89  92  89
   query53  268 181 178 178
   query54  539 410 405 405
   query55  81  78  89  78
   query56  263 260 264 260
   query57  1337120011381138
   query58  219 208 222 208
   query59  3401306429612961
   query60  266 249 239 239
   query61  113 108 110 108
   query62  802 669 664 664
   query63  217 189 185 185
   query64  1370681 724 681
   query65  3309324232343234
   query66  720 287 308 287
   query67  16004   15678   15764   15678
   query68  4018585 569 569
   query69  422 255 262 255
   query70  1270113210971097
   query71  363 243 257 243
   query72  6358404040464040
   query73  758 359 364 359
   query74  10389   916289038903
   query75  3430272826882688
   query76  1880996 1156996
   query77  484 275 279 275
   query78  10511   947693809380
   query79  2152611 605 605
   query80  1390427 440 427
   query81  531 227 225 225
   query82  1277187 117 117
   query83  269 163 149 149
   query84  285 79  72  72
   query85  1014316 298 298
   query86  420 312 303 303
   query87  4781472146814681
   query88  3563226723092267
   query89  412 303 297 297
   query90  1956191 189 189
   query91  135 106 101 101
   query92  68  49  51  49
   query93  2695543 548 543
   query94  826 301 286 286
   query95  360 253 250 250
   query96  634 285 276 276
   query97  2869269827082698
   query98  226 192 198 192
   query99  1601132113011301
   Total cold run time: 321694 ms
   Total hot run time: 197412 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490379157

   
   
   TPC-H: Total hot run time: 40093 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit da2c6d4220c27610dc537441dfcc887ff06a08ed, 
data reload: false
   
   -- Round 1 --
   q1   17587   751473067306
   q2   2049181 173 173
   q3   10545   109712091097
   q4   10451   755 834 755
   q5   7597271827192718
   q6   244 150 147 147
   q7   998 635 609 609
   q8   9452188719031887
   q9   6609647365206473
   q10  7055232923242324
   q11  463 262 253 253
   q12  413 229 232 229
   q13  19417   307031043070
   q14  247 213 218 213
   q15  583 541 516 516
   q16  656 598 598 598
   q17  983 614 515 515
   q18  7305683267116711
   q19  13291034990 990
   q20  471 186 183 183
   q21  3987322230083008
   q22  378 319 318 318
   Total cold run time: 108819 ms
   Total hot run time: 40093 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7362749073687368
   q2   340 233 241 233
   q3   3057313630413041
   q4   2183204319501950
   q5   5825583159005831
   q6   242 161 153 153
   q7   2326185718361836
   q8   3471365435253525
   q9   9015898889958988
   q10  3607359435453545
   q11  600 523 516 516
   q12  831 615 599 599
   q13  16572   337131943194
   q14  326 268 288 268
   q15  571 536 529 529
   q16  688 647 639 639
   q17  1858163416391634
   q18  8514767878107678
   q19  2404157115081508
   q20  2131187618921876
   q21  5593553853655365
   q22  621 551 582 551
   Total cold run time: 78137 ms
   Total hot run time: 60827 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-21 Thread via GitHub


github-actions[bot] commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490312876

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-20 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2490307865

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-20 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2489066742

   
   
   ClickBench: Total hot run time: 33.32 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 550a2f47ed646fd15758baf12332be335ddfdbf8, 
data reload: false
   
   query1   0.040.040.03
   query2   0.060.030.03
   query3   0.230.080.07
   query4   1.620.100.11
   query5   0.440.380.41
   query6   1.150.670.66
   query7   0.020.020.01
   query8   0.050.030.04
   query9   0.570.520.50
   query10  0.560.570.57
   query11  0.150.100.10
   query12  0.130.120.11
   query13  0.620.610.61
   query14  2.702.842.82
   query15  0.890.830.84
   query16  0.390.390.40
   query17  1.061.071.04
   query18  0.200.210.20
   query19  1.961.842.04
   query20  0.010.010.01
   query21  15.38   0.590.58
   query22  2.592.702.32
   query23  17.04   0.900.93
   query24  3.190.981.27
   query25  0.190.260.11
   query26  0.500.140.14
   query27  0.040.050.04
   query28  10.69   1.091.09
   query29  12.56   3.203.21
   query30  0.250.060.06
   query31  2.870.380.38
   query32  3.280.460.46
   query33  3.003.083.00
   query34  16.97   4.504.52
   query35  4.524.454.50
   query36  0.640.480.47
   query37  0.100.060.06
   query38  0.050.040.04
   query39  0.040.030.03
   query40  0.150.120.12
   query41  0.080.020.02
   query42  0.030.030.02
   query43  0.040.030.02
   Total cold run time: 107.05 s
   Total hot run time: 33.32 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-20 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2489053705

   
   
   TPC-DS: Total hot run time: 196370 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 550a2f47ed646fd15758baf12332be335ddfdbf8, 
data reload: false
   
   query1   1242947 916 916
   query2   6267227421692169
   query3   10766   401239613961
   query4   67925   29270   23590   23590
   query5   4986457 445 445
   query6   415 183 171 171
   query7   5482296 284 284
   query8   317 231 231 231
   query9   8730268026692669
   query10  455 252 254 252
   query11  17173   15278   15848   15278
   query12  154 103 102 102
   query13  1434433 426 426
   query14  10904   696870036968
   query15  211 192 185 185
   query16  7103473 445 445
   query17  1033556 558 556
   query18  1771282 284 282
   query19  185 159 141 141
   query20  115 109 110 109
   query21  221 99  99  99
   query22  4416448845234488
   query23  34436   34043   34024   34024
   query24  5305251024242424
   query25  483 379 384 379
   query26  638 156 146 146
   query27  1692284 285 284
   query28  4334243424182418
   query29  654 396 400 396
   query30  210 145 153 145
   query31  1000783 841 783
   query32  65  54  55  54
   query33  427 280 283 280
   query34  913 512 519 512
   query35  828 712 727 712
   query36  1067962 981 962
   query37  125 76  78  76
   query38  4378420243124202
   query39  1458146114361436
   query40  199 98  100 98
   query41  48  48  46  46
   query42  115 102 99  99
   query43  553 515 502 502
   query44  1165809 814 809
   query45  188 165 168 165
   query46  1127695 687 687
   query47  1998187118891871
   query48  411 325 326 325
   query49  746 406 405 405
   query50  860 384 400 384
   query51  7306703671177036
   query52  102 90  87  87
   query53  247 174 179 174
   query54  513 394 431 394
   query55  76  74  77  74
   query56  252 227 251 227
   query57  1289120611491149
   query58  221 215 217 215
   query59  3378314233763142
   query60  270 257 250 250
   query61  109 109 107 107
   query62  816 672 658 658
   query63  218 198 189 189
   query64  1401691 659 659
   query65  3273324132213221
   query66  701 306 321 306
   query67  16059   15813   15820   15813
   query68  3978561 568 561
   query69  424 252 267 252
   query70  1186116211501150
   query71  344 253 255 253
   query72  6378410140564056
   query73  757 359 352 352
   query74  10261   903290709032
   query75  3409268326572657
   query76  1762106210151015
   query77  488 274 284 274
   query78  10603   946194799461
   query79  1153600 622 600
   query80  786 442 441 441
   query81  531 243 221 221
   query82  240 120 119 119
   query83  180 153 149 149
   query84  284 72  69  69
   query85  836 294 332 294
   query86  354 304 294 294
   query87  4628464646794646
   query88  3712219721522152
   query89  428 306 295 295
   query90  2009188 188 188
   query91  131 104 100 100
   query92  61  48  54  48
   query93  1409542 544 542
   query94  775 294 289 289
   query95  347 249 248 248
   query96  606 281 278 278
   query97  2848273327202720
   query98  211 196 197 196
   query99  1753136513261326
   Total cold run time: 315831 ms
   Total hot run time: 196370 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to 

Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-20 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2489019142

   
   
   TPC-H: Total hot run time: 39897 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 550a2f47ed646fd15758baf12332be335ddfdbf8, 
data reload: false
   
   -- Round 1 --
   q1   17600   746372667266
   q2   2052167 164 164
   q3   10616   110012381100
   q4   10552   761 679 679
   q5   7624273927452739
   q6   248 151 150 150
   q7   1000669 628 628
   q8   9246186119111861
   q9   6553640663776377
   q10  7054230822882288
   q11  464 256 278 256
   q12  415 216 225 216
   q13  17778   303430283028
   q14  238 216 210 210
   q15  583 525 528 525
   q16  675 583 576 576
   q17  983 548 550 548
   q18  7307663367166633
   q19  1332104010661040
   q20  485 174 182 174
   q21  4287339231263126
   q22  371 315 313 313
   Total cold run time: 107463 ms
   Total hot run time: 39897 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   7224728272687268
   q2   339 228 223 223
   q3   2958286730062867
   q4   2055179517971795
   q5   5664572257065706
   q6   236 141 140 140
   q7   2270186218581858
   q8   3451350535513505
   q9   8879892188998899
   q10  3624357635743574
   q11  598 501 483 483
   q12  838 634 612 612
   q13  10980   324532323232
   q14  311 270 286 270
   q15  582 515 531 515
   q16  697 653 652 652
   q17  1863164916231623
   q18  8216773376397639
   q19  1729161715291529
   q20  2113188618671867
   q21  5518558154225422
   q22  648 587 609 587
   Total cold run time: 70793 ms
   Total hot run time: 60266 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-20 Thread via GitHub


suxiaogang223 commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2488918977

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hudi) upgrade hudi to 0.15.0 [doris]

2024-11-20 Thread via GitHub


doris-robot commented on PR #44267:
URL: https://github.com/apache/doris/pull/44267#issuecomment-2485050955

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting 
information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it 
now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and 
after the optimization?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org