Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen merged PR #63228: URL: https://github.com/apache/doris/pull/63228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4646813493 skip buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4630244168 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
github-actions[bot] commented on code in PR #63228:
URL: https://github.com/apache/doris/pull/63228#discussion_r3361523943
##
regression-test/pipeline/common/doris-utils.sh:
##
@@ -662,6 +662,14 @@ _monitor_regression_log() {
}
+_redact_creds() {
+local expr="" v
+for v in "${hwYunAk:-}" "${hwYunSk:-}" "${s3SourceAk:-}" "${s3SourceSk:-}"
"${txYunAk:-}" "${txYunSk:-}"; do
+[[ -n "${v}" ]] && expr+="s/${v}//g;"
Review Comment:
This still leaves a credential leak path for valid secret values that
contain sed syntax. For example, an S3 secret key can contain `/`; with
`s3SourceSk='abc/def'` this builds `s/abc/def//g;`, `sed` fails, and the helper
suppresses the failure with `|| true`, so the logs/configs are copied into the
archive unredacted. Regex metacharacters such as `[` can also make the pattern
invalid or overmatch. Please escape the sed pattern/delimiter for each value,
or use a fixed-string/quoted redaction approach, before suppressing redaction
command failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4629735104 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
github-actions[bot] commented on code in PR #63228:
URL: https://github.com/apache/doris/pull/63228#discussion_r3340168890
##
regression-test/pipeline/common/doris-utils.sh:
##
@@ -674,17 +674,19 @@ archive_doris_logs() {
(
cd "${DORIS_HOME}" || return 1
cp --parents -rf "fe/conf" "${archive_dir}"/
+if sed -i
"s/${hwYunAk:-}//g;s/${hwYunSk:-}//g;s/${s3SourceAk:-}//g;s/${s3SourceSk:-}//g;s/${txYunAk:-}//g;s/${txYunSk:-}//g"
fe/log/* &>/dev/null; then :; fi
Review Comment:
This redaction can be skipped in normal cloud/vault runs when the optional
Huawei credentials are unset. `cloud_p0/run.sh` and `vault_p0/run.sh` require
`s3SourceAk/s3SourceSk` but only warn for missing `hwYunAk/hwYunSk`; with
`hwYunAk` empty this expands to an invalid leading `sed` expression
(`s//g...`), `sed` exits before processing the later required S3 credentials,
and the failure is suppressed by `&>/dev/null`, so the archive can still
contain `s3SourceAk/s3SourceSk`. Please build the redaction script only from
non-empty values (and apply the same fix to the duplicated BE/regression/conf
redaction lines) instead of interpolating empty variables directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
github-actions[bot] commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4600860120 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
github-actions[bot] commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4600859183 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4600575628 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4441627487 TPC-DS: Total hot run time: 169982 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 433c4e87e03fc44fd53bf7d8e27bd494ac0903d8, data reload: false query5 4296662 506 506 query6 342 226 205 205 query7 4208562 318 318 query8 334 234 224 224 query9 8841398739933987 query10 449 343 303 303 query11 5730239322062206 query12 186 125 122 122 query13 1360618 468 468 query14 5992534050275027 query14_14369435042944294 query15 207 202 179 179 query16 993 446 400 400 query17 926 744 616 616 query18 2427469 350 350 query19 211 194 163 163 query20 142 130 130 130 query21 206 135 117 117 query22 13616 13657 14616 13657 query23 17294 16539 16084 16084 query23_116322 16205 16250 16205 query24 7390175313171317 query24_11340134413421342 query25 583 527 446 446 query26 1347308 171 171 query27 2698590 357 357 query28 4523193119521931 query29 977 630 549 549 query30 302 240 199 199 query31 11071058942 942 query32 85 75 69 69 query33 544 354 302 302 query34 11781161653 653 query35 780 790 680 680 query36 1356136511521152 query37 152 103 88 88 query38 3205308630483048 query39 916 931 887 887 query39_1863 886 904 886 query40 238 155 140 140 query41 69 66 66 66 query42 109 111 109 109 query43 321 325 279 279 query44 query45 212 207 195 195 query46 10591234711 711 query47 2279225122022202 query48 405 410 303 303 query49 661 576 435 435 query50 686 282 219 219 query51 4321425841814181 query52 103 102 92 92 query53 249 271 208 208 query54 306 272 248 248 query55 89 87 81 81 query56 303 301 282 282 query57 1402137912951295 query58 279 266 257 257 query59 1547161214091409 query60 329 330 319 319 query61 159 160 156 156 query62 660 610 559 559 query63 234 191 203 191 query64 2398819 689 689 query65 query66 1738507 375 375 query67 29969 29953 29790 29790 query68 query69 456 329 300 300 query70 10441008973 973 query71 305 264 272 264 query72 3039269723802380 query73 839 791 439 439 query74 5067487846874687 query75 2743265523122312 query76 22571117765 765 query77 399 438 341 341 query78 12819 12918 12337 12337 query79 1539968 762 762 query80 1346563 475 475 query81 522 276 240 240 query82 999 150 125 125 query83 324 268 248 248 query84 314 143 109 109 query85 923 516 441 441 query86 452 337 301 301 query87 3442331231823182 query88 3543269826832683 query89 433 372 331 331 query90 1935171 178 171 query91 180 171 138 138 query92 77 75 68 68 query93 1172933 579 579 query94 727 343 254 254 query95 668 370 334 334 query96 1065765 321 321 query97 2687266025932593 query98 230 228 233 228 query99 1082111510101010 Total cold run time: 253360 ms Total hot run time: 169982 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4441538651 TPC-H: Total hot run time: 29701 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 433c4e87e03fc44fd53bf7d8e27bd494ac0903d8, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17654 399738263826 q2 q3 10720 875 598 598 q4 4665461 337 337 q5 7436131911521152 q6 187 171 135 135 q7 913 935 756 756 q8 9306142713061306 q9 5531532253565322 q10 6257206818101810 q11 456 264 262 262 q12 629 412 287 287 q13 18089 327727282728 q14 295 278 262 262 q15 q16 862 860 787 787 q17 993 1063752 752 q18 6400566556775665 q19 1140121811531153 q20 504 393 262 262 q21 4620240119711971 q22 476 415 330 330 Total cold run time: 97133 ms Total hot run time: 29701 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4846470647614706 q2 q3 4658477741754175 q4 2112217913611361 q5 4957500652595006 q6 200 169 133 133 q7 2090179016321632 q8 308231143082 q9 8360834483788344 q10 4484449842534253 q11 616 423 393 393 q12 680 739 527 527 q13 3279369329152915 q14 313 314 269 269 q15 q16 773 801 706 706 q17 1462128412561256 q18 8020701669876987 q19 1152114711651147 q20 2253223419881988 q21 6087533947894789 q22 536 495 414 414 Total cold run time: 60211 ms Total hot run time: 54083 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4441341274 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](ci) rm some infos [doris]
hello-stephen commented on PR #63228: URL: https://github.com/apache/doris/pull/63228#issuecomment-4441336273 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
