Re: [PR] [optimize](util) Faster bit_pack [doris]
HappenLee commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2196389831
##
be/src/util/frame_of_reference_coding.cpp:
##
@@ -87,21 +246,14 @@ void ForEncoder::bit_pack(const T* input, uint8_t
in_num, int bit_width, uint
return;
}
-T in_mask = 0;
-int bit_index = 0;
-*output = 0;
-for (int i = 0; i < in_num; i++) {
-in_mask = ((T)1) << (bit_width - 1);
-for (int k = 0; k < bit_width; k++) {
-if (bit_index > 7) {
-bit_index = 0;
-output++;
-*output = 0;
-}
-*output |= (((input[i] & in_mask) >> (bit_width - k - 1)) << (7 -
bit_index));
-in_mask >>= 1;
-bit_index++;
-}
+if (bit_width <= 8) {
+bit_pack_8(input, in_num, bit_width, output);
+} else if (bit_width <= 16) {
+bit_pack_32(input, in_num, bit_width, output);
+} else if (bit_width <= 32) {
+bit_pack_32<__int128_t>(input, in_num, bit_width, output);
+} else {
+bit_pack_128(input, in_num, bit_width, output);
Review Comment:
bit_width <= 64
`bit_pack_4` which is better `bit_pack_1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
HappenLee commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2196383562
##
be/src/util/frame_of_reference_coding.cpp:
##
@@ -87,21 +246,14 @@ void ForEncoder::bit_pack(const T* input, uint8_t
in_num, int bit_width, uint
return;
}
-T in_mask = 0;
-int bit_index = 0;
-*output = 0;
-for (int i = 0; i < in_num; i++) {
-in_mask = ((T)1) << (bit_width - 1);
-for (int k = 0; k < bit_width; k++) {
-if (bit_index > 7) {
-bit_index = 0;
-output++;
-*output = 0;
-}
-*output |= (((input[i] & in_mask) >> (bit_width - k - 1)) << (7 -
bit_index));
-in_mask >>= 1;
-bit_index++;
-}
+if (bit_width <= 8) {
+bit_pack_8(input, in_num, bit_width, output);
Review Comment:
bit_pack_8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
HappenLee commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2196383562
##
be/src/util/frame_of_reference_coding.cpp:
##
@@ -87,21 +246,14 @@ void ForEncoder::bit_pack(const T* input, uint8_t
in_num, int bit_width, uint
return;
}
-T in_mask = 0;
-int bit_index = 0;
-*output = 0;
-for (int i = 0; i < in_num; i++) {
-in_mask = ((T)1) << (bit_width - 1);
-for (int k = 0; k < bit_width; k++) {
-if (bit_index > 7) {
-bit_index = 0;
-output++;
-*output = 0;
-}
-*output |= (((input[i] & in_mask) >> (bit_width - k - 1)) << (7 -
bit_index));
-in_mask >>= 1;
-bit_index++;
-}
+if (bit_width <= 8) {
+bit_pack_8(input, in_num, bit_width, output);
Review Comment:
`bit_pack_8` enqual `bit_pack_32` which is better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
BiteThet merged PR #52829: URL: https://github.com/apache/doris/pull/52829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
wumeibanfa commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2194401454
##
be/benchmark/benchmark_main.cpp:
##
@@ -46,7 +47,8 @@ static void Example1(benchmark::State& state) {
}
// could BENCHMARK many functions to compare them together.
BENCHMARK(Example1);
-
+BENCHMARK(BM_BitPack)->DenseRange(1, 127)->Unit(benchmark::kNanosecond);
+BENCHMARK(BM_BitPackOptimized)->DenseRange(1,
127)->Unit(benchmark::kNanosecond);
Review Comment:
Just delete them？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
BiteThet commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2194393052
##
be/benchmark/benchmark_main.cpp:
##
@@ -46,7 +47,8 @@ static void Example1(benchmark::State& state) {
}
// could BENCHMARK many functions to compare them together.
BENCHMARK(Example1);
-
+BENCHMARK(BM_BitPack)->DenseRange(1, 127)->Unit(benchmark::kNanosecond);
+BENCHMARK(BM_BitPackOptimized)->DenseRange(1,
127)->Unit(benchmark::kNanosecond);
Review Comment:
Maybe we shouldn't add benchmark code to doris' codebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
github-actions[bot] commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3051643125 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
hello-stephen commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3051525601 # BE UT Coverage Report Increment line coverage `100.00% (102/102)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/2c517bf26945abd5a3f255916a9215a78174675b_2c517bf26945abd5a3f255916a9215a78174675b/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/2c517bf26945abd5a3f255916a9215a78174675b_2c517bf26945abd5a3f255916a9215a78174675b/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.34% (15513/27054) | | Line Coverage | 46.35% (141099/304436) | | Region Coverage | 45.60% (71327/156426) | | Branch Coverage | 40.33% (37600/93220) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3051357877 ClickBench: Total hot run time: 29.36 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2c517bf26945abd5a3f255916a9215a78174675b, data reload: false query1 0.050.040.03 query2 0.080.040.04 query3 0.250.080.07 query4 1.610.110.11 query5 0.440.420.41 query6 1.180.660.67 query7 0.020.020.01 query8 0.050.040.04 query9 0.610.510.52 query10 0.570.560.60 query11 0.160.100.11 query12 0.160.110.11 query13 0.640.600.62 query14 0.780.840.79 query15 0.920.870.88 query16 0.410.390.40 query17 1.061.071.10 query18 0.240.210.21 query19 1.961.801.85 query20 0.010.010.02 query21 15.37 0.880.58 query22 0.741.290.95 query23 14.68 1.360.66 query24 6.811.580.35 query25 0.410.160.10 query26 0.690.170.14 query27 0.080.060.05 query28 9.210.910.46 query29 12.56 4.023.24 query30 0.250.100.07 query31 2.830.600.39 query32 3.230.560.49 query33 3.243.103.16 query34 16.06 5.354.77 query35 4.844.804.86 query36 0.710.510.48 query37 0.090.060.07 query38 0.050.040.04 query39 0.030.030.03 query40 0.180.150.14 query41 0.090.020.02 query42 0.030.030.02 query43 0.040.040.03 Total cold run time: 103.42 s Total hot run time: 29.36 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3051326697 TPC-DS: Total hot run time: 186658 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2c517bf26945abd5a3f255916a9215a78174675b, data reload: false query1 971 385 387 385 query2 6524168017091680 query3 6739210 208 208 query4 27032 23393 23488 23393 query5 4360575 458 458 query6 317 220 207 207 query7 4623506 297 297 query8 296 225 219 219 query9 8610267826572657 query10 455 305 270 270 query11 15384 15005 14861 14861 query12 153 102 101 101 query13 1631533 404 404 query14 8497591260205912 query15 206 212 179 179 query16 7154436 268 268 query17 1172738 597 597 query18 1980413 313 313 query19 234 185 152 152 query20 120 114 112 112 query21 217 116 101 101 query22 4300414746554147 query23 35047 33553 33718 33553 query24 8445240923962396 query25 541 455 385 385 query26 1227258 147 147 query27 2758506 375 375 query28 4271213321092109 query29 745 572 426 426 query30 287 216 185 185 query31 903 842 759 759 query32 73 59 58 58 query33 551 329 283 283 query34 831 839 525 525 query35 604 655 557 557 query36 935 966 882 882 query37 114 97 74 74 query38 4155409240224022 query39 1473141514461415 query40 210 112 105 105 query41 55 51 50 50 query42 118 110 110 110 query43 485 496 469 469 query44 1373831 836 831 query45 176 174 162 162 query46 836 1010641 641 query47 1722176617091709 query48 407 414 333 333 query49 745 492 379 379 query50 640 677 414 414 query51 4281424640184018 query52 112 108 101 101 query53 238 258 209 209 query54 582 563 509 509 query55 81 84 81 81 query56 291 306 281 281 query57 1170116911321132 query58 273 250 257 250 query59 2503261424992499 query60 343 316 313 313 query61 123 119 118 118 query62 799 724 672 672 query63 235 196 190 190 query64 43651282932 932 query65 4241416041714160 query66 1130408 304 304 query67 15821 15589 15547 15547 query68 7909901 531 531 query69 509 314 272 272 query70 1219112911651129 query71 423 330 307 307 query72 5613466248474662 query73 658 600 353 353 query74 8909893587238723 query75 3353321427012701 query76 33691154766 766 query77 526 369 363 363 query78 10938 11035 10189 10189 query79 2005820 625 625 query80 699 511 432 432 query81 508 260 222 222 query82 392 122 94 94 query83 248 250 229 229 query84 277 98 84 84 query85 755 356 314 314 query86 394 307 281 281 query87 4359442844214421 query88 2877226822612261 query89 393 310 283 283 query90 1960206 198 198 query91 137 137 106 106 query92 67 55 57 55 query93 2635931 589 589 query94 705 308 197 197 query95 375 282 276 276 query96 480 573 288 288 query97 2673276926222622 query98 246 212 211 211 query99 1304139212831283 Total cold run time: 274205 ms Total hot run time: 186658 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3051290106 TPC-H: Total hot run time: 33406 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2c517bf26945abd5a3f255916a9215a78174675b, data reload: false -- Round 1 -- q1 17575 511151175111 q2 1944276 194 194 q3 10374 1295744 744 q4 10292 1024546 546 q5 8501244123782378 q6 181 162 128 128 q7 910 744 599 599 q8 9305131311031103 q9 6873513850675067 q10 6879240719861986 q11 505 301 284 284 q12 346 351 224 224 q13 17791 363130873087 q14 241 233 211 211 q15 536 499 479 479 q16 422 427 383 383 q17 593 864 360 360 q18 7633715270627062 q19 1226954 585 585 q20 343 356 217 217 q21 4060324723592359 q22 359 324 299 299 Total cold run time: 106889 ms Total hot run time: 33406 ms - Round 2, with runtime_filter_mode=off - q1 5134513851305130 q2 241 328 225 225 q3 2194266322432243 q4 1338176713241324 q5 4223444245704442 q6 206 167 126 126 q7 2067197218101810 q8 2667256826202568 q9 7128721472637214 q10 3109344529112911 q11 592 541 494 494 q12 688 790 607 607 q13 3824392232263226 q14 314 313 270 270 q15 511 489 496 489 q16 444 485 447 447 q17 1217160413511351 q18 806176037603 q19 893 973 1005973 q20 2050210619021902 q21 5192462745474547 q22 643 614 564 564 Total cold run time: 52736 ms Total hot run time: 50466 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
wumeibanfa commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3051206995 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
github-actions[bot] commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3045658587 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
github-actions[bot] commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3045658479 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
hello-stephen commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3045273853 # BE UT Coverage Report Increment line coverage `100.00% (102/102)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/38a96c4ca9f61513254c4ab3b7ed2cec992e1846_38a96c4ca9f61513254c4ab3b7ed2cec992e1846/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/38a96c4ca9f61513254c4ab3b7ed2cec992e1846_38a96c4ca9f61513254c4ab3b7ed2cec992e1846/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.32% (15501/27043) | | Line Coverage | 46.30% (140920/304338) | | Region Coverage | 45.56% (71252/156398) | | Branch Coverage | 40.27% (37546/93226) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3044845973 ClickBench: Total hot run time: 29.12 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 38a96c4ca9f61513254c4ab3b7ed2cec992e1846, data reload: false query1 0.050.040.04 query2 0.090.050.04 query3 0.240.070.07 query4 1.620.110.11 query5 0.430.420.43 query6 1.170.660.66 query7 0.030.020.02 query8 0.040.040.04 query9 0.600.510.51 query10 0.570.570.56 query11 0.160.110.11 query12 0.150.110.11 query13 0.620.610.62 query14 0.800.840.84 query15 0.910.870.87 query16 0.380.380.40 query17 1.061.071.08 query18 0.220.210.21 query19 1.921.871.86 query20 0.020.010.02 query21 15.39 0.850.54 query22 0.741.160.64 query23 15.01 1.400.63 query24 6.942.010.35 query25 0.420.210.12 query26 0.610.160.13 query27 0.060.050.06 query28 9.160.900.44 query29 12.52 3.913.30 query30 0.250.100.06 query31 2.840.600.39 query32 3.230.580.47 query33 3.073.133.11 query34 16.13 5.374.73 query35 4.774.904.81 query36 0.690.500.48 query37 0.090.060.07 query38 0.060.040.04 query39 0.040.020.03 query40 0.160.140.14 query41 0.080.020.03 query42 0.040.020.03 query43 0.040.030.03 Total cold run time: 103.42 s Total hot run time: 29.12 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3044823778 TPC-DS: Total hot run time: 185801 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 38a96c4ca9f61513254c4ab3b7ed2cec992e1846, data reload: false query1 1011388 390 388 query2 6527170116681668 query3 6745212 208 208 query4 25872 23494 22942 22942 query5 4330581 422 422 query6 307 211 205 205 query7 4637505 290 290 query8 284 217 223 217 query9 8594263826342634 query10 450 308 278 278 query11 15680 15036 14909 14909 query12 146 111 103 103 query13 1629504 414 414 query14 9248582558235823 query15 197 185 173 173 query16 7490427 255 255 query17 1334709 561 561 query18 1993392 302 302 query19 193 193 166 166 query20 123 124 112 112 query21 209 121 104 104 query22 4144399846723998 query23 34602 34004 33385 33385 query24 8457235223502350 query25 510 466 379 379 query26 1234262 139 139 query27 2776509 346 346 query28 4266213221312131 query29 717 536 415 415 query30 284 220 191 191 query31 917 860 757 757 query32 71 63 67 63 query33 564 340 309 309 query34 770 855 510 510 query35 586 656 562 562 query36 952 978 881 881 query37 112 109 79 79 query38 4150413341064106 query39 1491141613931393 query40 209 115 101 101 query41 53 52 51 51 query42 121 111 105 105 query43 495 498 472 472 query44 1281837 828 828 query45 215 172 162 162 query46 846 1065614 614 query47 1760179017081708 query48 382 417 307 307 query49 709 479 386 386 query50 625 690 419 419 query51 4253423341504150 query52 107 107 97 97 query53 218 251 187 187 query54 580 559 492 492 query55 81 76 77 76 query56 291 307 282 282 query57 1184118910981098 query58 255 248 254 248 query59 2531256625232523 query60 326 321 320 320 query61 122 118 115 115 query62 778 726 661 661 query63 225 186 185 185 query64 4369137510441044 query65 4309413942014139 query66 1090414 301 301 query67 15974 15492 15475 15475 query68 8583934 552 552 query69 487 305 257 257 query70 1219109211061092 query71 466 320 310 310 query72 5592480551064805 query73 714 634 347 347 query74 9347916186778677 query75 3990322326712671 query76 36841158727 727 query77 795 381 295 295 query78 10815 11141 10258 10258 query79 2853810 568 568 query80 661 527 440 440 query81 458 249 224 224 query82 452 126 102 102 query83 271 258 246 246 query84 286 113 84 84 query85 788 350 307 307 query86 341 312 283 283 query87 4413451744624462 query88 3059226722502250 query89 399 314 290 290 query90 1938209 212 209 query91 144 138 114 114 query92 73 61 58 58 query93 2047955 595 595 query94 671 312 195 195 query95 377 289 288 288 query96 493 563 281 281 query97 2740276326302630 query98 220 208 216 208 query99 1455140112621262 Total cold run time: 276822 ms Total hot run time: 185801 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3044768461 TPC-H: Total hot run time: 33202 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 38a96c4ca9f61513254c4ab3b7ed2cec992e1846, data reload: false -- Round 1 -- q1 17631 522450215021 q2 1959281 193 193 q3 10314 1304713 713 q4 10228 1059517 517 q5 7475240223742374 q6 186 159 129 129 q7 898 742 614 614 q8 9316129910691069 q9 6793516950785078 q10 6866238819441944 q11 484 282 283 282 q12 341 347 207 207 q13 17751 369031013101 q14 226 232 212 212 q15 556 480 482 480 q16 420 433 391 391 q17 606 877 353 353 q18 7596717871197119 q19 1205966 547 547 q20 317 333 212 212 q21 3651313223422342 q22 358 310 304 304 Total cold run time: 105177 ms Total hot run time: 33202 ms - Round 2, with runtime_filter_mode=off - q1 5085503451975034 q2 242 329 219 219 q3 2162269422552255 q4 1368174813211321 q5 4204418546734185 q6 219 171 126 126 q7 2035195218211821 q8 2644258825402540 q9 7576716573177165 q10 3072333928512851 q11 575 530 499 499 q12 781 784 650 650 q13 3681396233793379 q14 290 307 277 277 q15 551 499 477 477 q16 461 502 448 448 q17 1287161813581358 q18 7848773476257625 q19 819 811 1032811 q20 2078205119351935 q21 4997457745264526 q22 626 593 558 558 Total cold run time: 52601 ms Total hot run time: 50060 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
zclllyybb commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3044692744 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
wumeibanfa commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2188968210
##
be/src/util/frame_of_reference_coding.cpp:
##
@@ -72,6 +72,168 @@ void ForEncoder::put_batch(const T* in_data, size_t
count) {
// todo(kks): improve this method by SIMD instructions
+template
+void ForEncoder::bit_pack_8(const T* input, uint8_t in_num, int bit_width,
uint8_t* output) {
+int64_t s = 0;
+uint8_t output_mask = 255;
+int tail_count = in_num & 7;
+int full_batch_size = (in_num >> 3) << 3;
+
+for (int i = 0; i < full_batch_size; i += 8) {
+s |= static_cast(input[i + 7]);
+s |= (static_cast(input[i + 6])) << bit_width;
+s |= (static_cast(input[i + 5])) << (2 * bit_width);
+s |= (static_cast(input[i + 4])) << (3 * bit_width);
+s |= (static_cast(input[i + 3])) << (4 * bit_width);
+s |= (static_cast(input[i + 2])) << (5 * bit_width);
+s |= (static_cast(input[i + 1])) << (6 * bit_width);
+s |= (static_cast(input[i])) << (7 * bit_width);
+
+for (int j = 0; j < bit_width; j++) {
+output[j] = (s >> ((bit_width - j - 1) << 3)) & output_mask;
+}
+output += bit_width;
+s = 0;
+}
+
+// remainder
+int byte = tail_count * bit_width;
+int bytes = (byte + 7) >> 3;
+
+for (int i = 0; i < tail_count; i++) {
+s |= (static_cast(input[i + full_batch_size]))
+ << ((tail_count - i - 1) * bit_width);
+}
+
+s <<= (bytes << 3) - byte;
+
+for (int i = 0; i < bytes; i++) {
+output[i] = (s >> ((bytes - i - 1) << 3)) & output_mask;
+}
+}
+
+template
+void ForEncoder::bit_pack_16(const T* input, uint8_t in_num, int bit_width,
uint8_t* output) {
+int64_t s = 0;
+uint8_t output_mask = 255;
+int tail_count = in_num & 3;
+int full_batch_size = (in_num >> 2) << 2;
+int output_size = 0; // How many outputs can be processed at a time
+int bit_width_remainder =
+(bit_width << 2) & 7; // How many bits will be left after
processing 4 numbers at a time
+int extra_bit = 0;// Extra bits after each process
+
+for (int i = 0; i < full_batch_size; i += 4) {
+s <<= bit_width;
+s |= (static_cast(input[i]));
+s <<= bit_width;
+s |= (static_cast(input[i + 1]));
+s <<= bit_width;
+s |= (static_cast(input[i + 2]));
+s <<= bit_width;
+s |= (static_cast(input[i + 3]));
+
+output_size = (bit_width * 4 + extra_bit) >> 3;
Review Comment:
Explain why calculated it this way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
zclllyybb commented on code in PR #52829:
URL: https://github.com/apache/doris/pull/52829#discussion_r2188954637
##
be/src/util/frame_of_reference_coding.cpp:
##
@@ -72,6 +72,168 @@ void ForEncoder::put_batch(const T* in_data, size_t
count) {
// todo(kks): improve this method by SIMD instructions
+template
+void ForEncoder::bit_pack_8(const T* input, uint8_t in_num, int bit_width,
uint8_t* output) {
+int64_t s = 0;
+uint8_t output_mask = 255;
+int tail_count = in_num & 7;
+int full_batch_size = (in_num >> 3) << 3;
+
+for (int i = 0; i < full_batch_size; i += 8) {
+s |= static_cast(input[i + 7]);
+s |= (static_cast(input[i + 6])) << bit_width;
+s |= (static_cast(input[i + 5])) << (2 * bit_width);
+s |= (static_cast(input[i + 4])) << (3 * bit_width);
+s |= (static_cast(input[i + 3])) << (4 * bit_width);
+s |= (static_cast(input[i + 2])) << (5 * bit_width);
+s |= (static_cast(input[i + 1])) << (6 * bit_width);
+s |= (static_cast(input[i])) << (7 * bit_width);
+
+for (int j = 0; j < bit_width; j++) {
+output[j] = (s >> ((bit_width - j - 1) << 3)) & output_mask;
+}
+output += bit_width;
+s = 0;
+}
+
+// remainder
+int byte = tail_count * bit_width;
+int bytes = (byte + 7) >> 3;
+
+for (int i = 0; i < tail_count; i++) {
+s |= (static_cast(input[i + full_batch_size]))
+ << ((tail_count - i - 1) * bit_width);
+}
+
+s <<= (bytes << 3) - byte;
+
+for (int i = 0; i < bytes; i++) {
+output[i] = (s >> ((bytes - i - 1) << 3)) & output_mask;
+}
+}
+
+template
+void ForEncoder::bit_pack_16(const T* input, uint8_t in_num, int bit_width,
uint8_t* output) {
+int64_t s = 0;
+uint8_t output_mask = 255;
+int tail_count = in_num & 3;
+int full_batch_size = (in_num >> 2) << 2;
+int output_size = 0; // How many outputs can be processed at a time
+int bit_width_remainder =
+(bit_width << 2) & 7; // How many bits will be left after
processing 4 numbers at a time
+int extra_bit = 0;// Extra bits after each process
+
+for (int i = 0; i < full_batch_size; i += 4) {
+s <<= bit_width;
+s |= (static_cast(input[i]));
+s <<= bit_width;
+s |= (static_cast(input[i + 1]));
+s <<= bit_width;
+s |= (static_cast(input[i + 2]));
+s <<= bit_width;
+s |= (static_cast(input[i + 3]));
+
+output_size = (bit_width * 4 + extra_bit) >> 3;
Review Comment:
for expressions like this(L136, L137, L139) and those in your other
implementations, please add some comment.
##
be/benchmark/benchmark_bit_pack.cpp:
##
@@ -0,0 +1,102 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "util/frame_of_reference_coding.h"
+
+namespace doris {
+
+// original bit_pack function
+template
+void bit_pack(const T* input, uint8_t in_num, int bit_width, uint8_t*
output) {
+if (in_num == 0 || bit_width == 0) {
+return;
+}
+
+T in_mask = 0;
+int bit_index = 0;
+*output = 0;
+for (int i = 0; i < in_num; i++) {
+in_mask = ((T)1) << (bit_width - 1);
+for (int k = 0; k < bit_width; k++) {
+if (bit_index > 7) {
+bit_index = 0;
+output++;
+*output = 0;
+}
+*output |= (((input[i] & in_mask) >> (bit_width - k - 1)) <<
(7 - bit_index));
+in_mask >>= 1;
+bit_index++;
+}
+}
+}
+
+static void BM_BitPack(benchmark::State& state) {
+int w = state.range(0); // 获取参数 w（bit_width）
Review Comment:
dont use Chinese comment
##
be/src/util/frame_of_reference_coding.cpp:
##
@@ -72,6 +72,168 @@ void ForEncoder::put_batch(const T* in_data, size_t
count) {
// todo(kks): improve this method by SIMD instructions
+template
+void ForEncoder::bit_pack_8(const T* input, uint8_t in_num, int bit_width,
uint8_t* output) {
Review Comment:
in`bit_p
Re: [PR] [optimize](util) Faster bit_pack [doris]
hello-stephen commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3041377615 # BE UT Coverage Report Increment line coverage `100.00% (143/143)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/e34c13b01b9f8edddf4aa88df66e12d1f541f643_e34c13b01b9f8edddf4aa88df66e12d1f541f643/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/e34c13b01b9f8edddf4aa88df66e12d1f541f643_e34c13b01b9f8edddf4aa88df66e12d1f541f643/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.33% (15487/27016) | | Line Coverage | 46.31% (140764/303950) | | Region Coverage | 45.59% (71223/156224) | | Branch Coverage | 40.31% (37529/93100) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3041308074 ClickBench: Total hot run time: 29.29 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e34c13b01b9f8edddf4aa88df66e12d1f541f643, data reload: false query1 0.050.040.04 query2 0.080.050.04 query3 0.250.080.07 query4 1.600.110.11 query5 0.450.440.41 query6 1.160.660.66 query7 0.020.010.02 query8 0.040.040.04 query9 0.620.530.52 query10 0.570.580.57 query11 0.160.110.11 query12 0.150.110.12 query13 0.630.610.62 query14 0.790.800.85 query15 0.900.860.90 query16 0.390.380.39 query17 1.091.071.12 query18 0.230.210.21 query19 2.001.831.84 query20 0.010.010.01 query21 15.39 0.920.53 query22 0.751.240.79 query23 14.78 1.350.66 query24 7.102.430.45 query25 0.560.230.10 query26 0.620.170.13 query27 0.050.050.05 query28 9.330.910.44 query29 12.58 3.993.29 query30 0.250.090.07 query31 2.840.590.39 query32 3.230.550.47 query33 3.093.113.08 query34 16.24 5.454.76 query35 4.844.824.86 query36 0.690.510.49 query37 0.090.070.06 query38 0.060.040.03 query39 0.030.030.02 query40 0.170.140.14 query41 0.080.030.02 query42 0.030.030.02 query43 0.040.030.02 Total cold run time: 104.03 s Total hot run time: 29.29 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3041304576 TPC-DS: Total hot run time: 184871 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e34c13b01b9f8edddf4aa88df66e12d1f541f643, data reload: false query1 1004405 380 380 query2 6546169016301630 query3 6733207 208 207 query4 26796 23988 23467 23467 query5 4478558 431 431 query6 300 210 202 202 query7 4625504 286 286 query8 278 222 212 212 query9 8626263226332632 query10 487 327 265 265 query11 15217 15208 14717 14717 query12 158 103 104 103 query13 1644526 421 421 query14 8473582458355824 query15 216 193 183 183 query16 7489659 490 490 query17 1209715 578 578 query18 1994414 365 365 query19 190 191 155 155 query20 118 115 117 115 query21 207 119 105 105 query22 4224401141504011 query23 34149 33189 33129 33129 query24 8377233823522338 query25 549 475 376 376 query26 1240267 143 143 query27 2748508 357 357 query28 4298212721102110 query29 748 582 428 428 query30 279 213 185 185 query31 906 798 760 760 query32 68 60 61 60 query33 547 398 303 303 query34 802 829 523 523 query35 789 800 760 760 query36 977 973 886 886 query37 110 98 74 74 query38 4116418740554055 query39 1523141013841384 query40 203 116 103 103 query41 61 51 51 51 query42 122 108 107 107 query43 499 516 468 468 query44 1321812 815 812 query45 183 166 161 161 query46 851 992 623 623 query47 1723181416811681 query48 386 408 303 303 query49 740 470 402 402 query50 629 684 406 406 query51 4038417241314131 query52 106 112 100 100 query53 220 247 187 187 query54 572 579 503 503 query55 83 76 82 76 query56 308 303 284 284 query57 1170116211421142 query58 265 251 275 251 query59 2578261324602460 query60 323 316 297 297 query61 124 123 139 123 query62 825 711 651 651 query63 222 186 188 186 query64 4307978 675 675 query65 4333416441624162 query66 1137406 301 301 query67 15667 15674 15550 15550 query68 7856897 527 527 query69 467 356 263 263 query70 1205112911091109 query71 463 319 318 318 query72 5624475347814753 query73 702 611 354 354 query74 9090914286728672 query75 3820320326362636 query76 35861139709 709 query77 778 403 282 282 query78 10008 10345 92749274 query79 1834835 585 585 query80 650 522 450 450 query81 495 260 224 224 query82 189 125 96 96 query83 258 244 232 232 query84 258 116 90 90 query85 872 356 314 314 query86 366 308 305 305 query87 4503452043124312 query88 2898227122552255 query89 382 307 275 275 query90 1828202 201 201 query91 143 142 109 109 query92 66 58 55 55 query93 1320950 580 580 query94 591 399 319 319 query95 365 286 280 280 query96 497 545 280 280 query97 2740275026512651 query98 221 206 216 206 query99 1336140413091309 Total cold run time: 271645 ms Total hot run time: 184871 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3041298711 TPC-H: Total hot run time: 33747 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e34c13b01b9f8edddf4aa88df66e12d1f541f643, data reload: false -- Round 1 -- q1 17576 516150365036 q2 1927270 184 184 q3 10316 1250694 694 q4 10226 976 521 521 q5 7526236323262326 q6 172 159 126 126 q7 880 732 594 594 q8 9308124010461046 q9 6741503850515038 q10 6861236519671967 q11 483 296 273 273 q12 341 342 207 207 q13 17765 363630753075 q14 240 223 208 208 q15 541 482 479 479 q16 421 415 383 383 q17 588 833 354 354 q18 7983733971387138 q19 1202949 523 523 q20 327 355 231 231 q21 3743325323202320 q22 1080103210241024 Total cold run time: 106247 ms Total hot run time: 33747 ms - Round 2, with runtime_filter_mode=off - q1 5101507850745074 q2 239 325 221 221 q3 2155264122752275 q4 1375174913231323 q5 4348449644864486 q6 245 168 125 125 q7 2038202418271827 q8 2655259225662566 q9 7380727272577257 q10 308028952895 q11 568 530 487 487 q12 730 812 717 717 q13 3557399333363336 q14 303 315 284 284 q15 529 470 512 470 q16 473 496 457 457 q17 1155158913861386 q18 7933798874517451 q19 797 813 974 813 q20 1926197218961896 q21 4894451844674467 q22 1090108710041004 Total cold run time: 52571 ms Total hot run time: 50817 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
wumeibanfa commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3041261446 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
hello-stephen commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3039749987 # BE UT Coverage Report Increment line coverage `100.00% (143/143)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/e2b344682b3ef71aa7ed40745f8047bd2348d2cb_e2b344682b3ef71aa7ed40745f8047bd2348d2cb/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/e2b344682b3ef71aa7ed40745f8047bd2348d2cb_e2b344682b3ef71aa7ed40745f8047bd2348d2cb/report/index.html) | Category | Coverage | |---|| | Function Coverage | 57.33% (15489/27016) | | Line Coverage | 46.32% (140782/303949) | | Region Coverage | 45.60% (71235/156224) | | Branch Coverage | 40.32% (37536/93100) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3039485358 ClickBench: Total hot run time: 29.03 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e2b344682b3ef71aa7ed40745f8047bd2348d2cb, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.240.090.08 query4 1.610.120.11 query5 0.420.450.43 query6 1.160.660.66 query7 0.020.020.01 query8 0.040.040.04 query9 0.600.510.51 query10 0.570.560.56 query11 0.160.110.11 query12 0.140.120.11 query13 0.620.610.61 query14 0.790.820.80 query15 0.890.860.87 query16 0.390.410.39 query17 1.081.031.03 query18 0.230.220.21 query19 1.921.881.86 query20 0.010.010.02 query21 15.40 0.900.56 query22 0.761.260.67 query23 14.89 1.360.66 query24 6.761.540.37 query25 0.300.210.06 query26 0.700.170.15 query27 0.060.060.05 query28 8.510.930.43 query29 12.56 3.973.28 query30 0.240.090.06 query31 2.840.590.39 query32 3.230.570.46 query33 3.013.043.07 query34 16.05 5.374.76 query35 4.844.864.79 query36 0.690.500.50 query37 0.090.070.07 query38 0.040.040.04 query39 0.030.030.02 query40 0.170.140.14 query41 0.080.030.02 query42 0.030.020.03 query43 0.040.030.03 Total cold run time: 102.32 s Total hot run time: 29.03 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3039476889 TPC-DS: Total hot run time: 184681 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e2b344682b3ef71aa7ed40745f8047bd2348d2cb, data reload: false query1 1003395 391 391 query2 6539168717231687 query3 6750208 218 208 query4 26751 23651 22969 22969 query5 4340572 426 426 query6 314 216 203 203 query7 4625495 283 283 query8 302 217 204 204 query9 8600263426432634 query10 472 303 285 285 query11 15616 15034 14770 14770 query12 158 103 99 99 query13 1633510 386 386 query14 8850559456085594 query15 202 185 180 180 query16 7201620 461 461 query17 1184698 554 554 query18 1986393 293 293 query19 186 185 155 155 query20 118 118 119 118 query21 209 119 110 110 query22 4074410746074107 query23 34756 33669 33790 33669 query24 8404232023292320 query25 519 438 382 382 query26 1230258 141 141 query27 2773503 343 343 query28 4347212521052105 query29 772 566 477 477 query30 275 209 183 183 query31 917 815 730 730 query32 79 60 57 57 query33 548 368 292 292 query34 791 890 509 509 query35 763 816 751 751 query36 943 961 877 877 query37 109 94 72 72 query38 4153414441354135 query39 1473155014141414 query40 207 114 100 100 query41 53 51 48 48 query42 126 103 106 103 query43 496 500 497 497 query44 1281809 831 809 query45 185 167 163 163 query46 839 989 610 610 query47 1702178717221722 query48 389 412 308 308 query49 741 464 384 384 query50 614 680 414 414 query51 4113414540644064 query52 113 106 104 104 query53 224 244 191 191 query54 569 555 497 497 query55 78 76 78 76 query56 301 283 284 283 query57 1171118711241124 query58 269 259 250 250 query59 2584275225052505 query60 348 343 329 329 query61 148 147 157 147 query62 826 723 667 667 query63 216 179 178 178 query64 4314990 629 629 query65 4255413941584139 query66 1135451 307 307 query67 15573 15473 15311 15311 query68 7761869 535 535 query69 467 309 268 268 query70 1178103910731039 query71 411 311 309 309 query72 5570477047144714 query73 655 658 344 344 query74 9221884786668666 query75 3430314426592659 query76 33001119714 714 query77 515 373 297 297 query78 990910307 93589358 query79 1319782 568 568 query80 586 516 446 446 query81 495 259 214 214 query82 194 134 109 109 query83 257 253 232 232 query84 236 95 86 86 query85 805 420 386 386 query86 400 314 286 286 query87 4466446544404440 query88 2907229523002295 query89 458 328 284 284 query90 1980208 211 208 query91 133 136 109 109 query92 80 57 54 54 query93 1570933 584 584 query94 663 407 297 297 query95 372 301 281 281 query96 490 560 279 279 query97 2690273526252625 query98 234 212 203 203 query99 1294142112591259 Total cold run time: 270881 ms Total hot run time: 184681 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [optimize](util) Faster bit_pack [doris]
doris-robot commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3039459890 TPC-H: Total hot run time: 35142 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e2b344682b3ef71aa7ed40745f8047bd2348d2cb, data reload: false -- Round 1 -- q1 17621 547152995299 q2 1933284 189 189 q3 10444 1431733 733 q4 10248 1100560 560 q5 7560268026512651 q6 203 168 130 130 q7 991 780 620 620 q8 9303145812701270 q9 6803522452285224 q10 6978245520002000 q11 522 295 278 278 q12 347 388 209 209 q13 17765 384231153115 q14 234 235 229 229 q15 568 497 485 485 q16 425 432 382 382 q17 602 914 341 341 q18 7595727272207220 q19 12161074582 582 q20 340 352 216 216 q21 4144337424802480 q22 10511022929 929 Total cold run time: 106893 ms Total hot run time: 35142 ms - Round 2, with runtime_filter_mode=off - q1 5499541054755410 q2 246 336 228 228 q3 2248275623182318 q4 1463190114211421 q5 4838466945424542 q6 255 180 124 124 q7 2175208119511951 q8 2986296030712960 q9 7209713371957133 q10 3223338529812981 q11 652 529 513 513 q12 697 859 635 635 q13 3693422034323432 q14 291 298 290 290 q15 571 491 479 479 q16 475 512 450 450 q17 1242175215171517 q18 8004768677067686 q19 878 906 902 902 q20 2089212819341934 q21 5457493548844884 q22 1095105910411041 Total cold run time: 55286 ms Total hot run time: 52831 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
wumeibanfa commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3039414838 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [optimize](util) Faster bit_pack [doris]
Thearas commented on PR #52829: URL: https://github.com/apache/doris/pull/52829#issuecomment-3039412951 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
