adoroszlai commented on code in PR #3629: URL: https://github.com/apache/ozone/pull/3629#discussion_r947006838
########## hadoop-ozone/dist/src/main/compose/ozone/docker-config: ########## @@ -48,3 +48,5 @@ OZONE_CONF_DIR=/etc/hadoop OZONE_LOG_DIR=/var/log/hadoop no_proxy=om,scm,s3g,recon,kdc,localhost,127.0.0.1 + +export SECURITY_ENABLED=false Review Comment: `SECURITY_ENABLED` is already defined in `test.sh`. ```suggestion ``` ########## hadoop-ozone/dist/src/main/compose/ozonesecure/docker-config: ########## @@ -124,6 +124,7 @@ KMS-SITE.XML_hadoop.kms.proxyuser.s3g.hosts=* #BYTEMAN_SCRIPT_URL=https://raw.githubusercontent.com/apache/hadoop/trunk/dev-support/byteman/hadooprpc.btm OZONE_DATANODE_SECURE_USER=root +export SECURITY_ENABLED=true Review Comment: `SECURITY_ENABLED` is already defined in `test.sh`. ```suggestion ``` ########## hadoop-ozone/dist/src/main/compose/ozonesecure-ha/docker-config: ########## @@ -130,6 +130,7 @@ HDFS-SITE.XML_rpc.metrics.percentiles.intervals=60,300 #BYTEMAN_SCRIPT_URL=https://raw.githubusercontent.com/apache/hadoop/trunk/dev-support/byteman/hadooprpc.btm OZONE_DATANODE_SECURE_USER=root +export SECURITY_ENABLED=true Review Comment: ```suggestion ``` ########## hadoop-ozone/dist/src/main/smoketest/s3/freon.robot: ########## @@ -18,26 +18,35 @@ Documentation S3 gateway test with aws cli Library OperatingSystem Library String Library BuiltIn -Resource ../commonlib.robot -Resource ../ozone-lib/freon.robot -Test Timeout 5 minutes +Resource ./commonawslib.robot +Suite Setup Setup v4 headers + *** Variables *** -${testuser} testuser +${ENDPOINT_URL} http://s3g:9878 *** Keywords *** -Set Access Key and Id - ${result} = Execute ozone s3 getsecret - ${accessKey} = Get Regexp Matches ${result} (?<=awsAccessKey=).* - ${secret} = Get Regexp Matches ${result} (?<=awsSecret=).* - ${accessKey} = Set Variable ${accessKey[0]} - ${secret} = Set Variable ${secret[0]} - Set Environment Variable AWS_SECRET_ACCESS_KEY ${secret} - Set Environment Variable AWS_ACCESS_KEY_ID ${accessKey} +# Export access key and secret to the environment +Setup aws credentials + ${accessKey} = Execute aws configure get aws_access_key_id + ${secret} = Execute aws configure get aws_secret_access_key + Set Environment Variable AWS_SECRET_ACCESS_KEY ${secret} + Set Environment Variable AWS_ACCESS_KEY_ID ${accessKey} + +Access key id + ${env_var}= Get Environment Variable AWS_ACCESS_KEY_ID + +Freon S3BG + [arguments] ${prefix}=s3bg ${n}=100 ${threads}=10 ${args}=${EMPTY} + ${result} = Execute ozone freon s3bg -e ${ENDPOINT_URL} -t ${threads} -n ${n} -p ${prefix} ${args} + Should contain ${result} Successful executions: ${n} *** Test Cases *** -S3g bucket creation performance - Kinit test user ${testuser} ${testuser}.keytab - Set Access Key and Id - Freon S3BG +Export AWS credentials + Setup aws credentials + +Check access key id + Access key id Review Comment: This seems to be unnecessary. ########## hadoop-ozone/dist/src/main/compose/ozone-ha/docker-config: ########## @@ -48,3 +48,5 @@ OZONE_CONF_DIR=/etc/hadoop OZONE_LOG_DIR=/var/log/hadoop no_proxy=om1,om2,om3,scm,s3g,recon,kdc,localhost,127.0.0.1 + +export SECURITY_ENABLED=false Review Comment: ```suggestion ``` ########## hadoop-ozone/dist/src/main/smoketest/s3/freon.robot: ########## @@ -18,26 +18,35 @@ Documentation S3 gateway test with aws cli Library OperatingSystem Library String Library BuiltIn -Resource ../commonlib.robot -Resource ../ozone-lib/freon.robot -Test Timeout 5 minutes +Resource ./commonawslib.robot +Suite Setup Setup v4 headers Review Comment: ```suggestion Suite Setup Setup s3 tests ``` ########## hadoop-ozone/dist/src/main/smoketest/s3/freon.robot: ########## @@ -18,26 +18,35 @@ Documentation S3 gateway test with aws cli Library OperatingSystem Library String Library BuiltIn -Resource ../commonlib.robot -Resource ../ozone-lib/freon.robot -Test Timeout 5 minutes +Resource ./commonawslib.robot +Suite Setup Setup v4 headers + *** Variables *** -${testuser} testuser +${ENDPOINT_URL} http://s3g:9878 *** Keywords *** -Set Access Key and Id - ${result} = Execute ozone s3 getsecret - ${accessKey} = Get Regexp Matches ${result} (?<=awsAccessKey=).* - ${secret} = Get Regexp Matches ${result} (?<=awsSecret=).* - ${accessKey} = Set Variable ${accessKey[0]} - ${secret} = Set Variable ${secret[0]} - Set Environment Variable AWS_SECRET_ACCESS_KEY ${secret} - Set Environment Variable AWS_ACCESS_KEY_ID ${accessKey} +# Export access key and secret to the environment +Setup aws credentials + ${accessKey} = Execute aws configure get aws_access_key_id + ${secret} = Execute aws configure get aws_secret_access_key + Set Environment Variable AWS_SECRET_ACCESS_KEY ${secret} + Set Environment Variable AWS_ACCESS_KEY_ID ${accessKey} + +Access key id + ${env_var}= Get Environment Variable AWS_ACCESS_KEY_ID + +Freon S3BG + [arguments] ${prefix}=s3bg ${n}=100 ${threads}=10 ${args}=${EMPTY} + ${result} = Execute ozone freon s3bg -e ${ENDPOINT_URL} -t ${threads} -n ${n} -p ${prefix} ${args} + Should contain ${result} Successful executions: ${n} *** Test Cases *** -S3g bucket creation performance - Kinit test user ${testuser} ${testuser}.keytab - Set Access Key and Id - Freon S3BG +Export AWS credentials + Setup aws credentials Review Comment: This should be a `[setup]` step for `Run Freon S3BG`, not a separate test case. ########## hadoop-ozone/dist/src/main/smoketest/s3/commonawslib.robot: ########## @@ -58,6 +58,7 @@ Setup v2 headers Set Environment Variable AWS_SECRET_ACCESS_KEY ANYKEY Setup v4 headers + ${SECURITY_ENABLED} = Get Environment Variable SECURITY_ENABLED Review Comment: `SECURITY_ENABLED` is defined as a Robot variable, propagated from the environment variable by: https://github.com/apache/ozone/blob/ec3d55f6b5101476ef0d889a26b8fb9cf6e4d243/hadoop-ozone/dist/src/main/compose/testlib.sh#L193 It does not need to be looked up from the environment again. ```suggestion ``` ########## hadoop-ozone/dist/src/main/smoketest/s3/freon.robot: ########## @@ -18,26 +18,35 @@ Documentation S3 gateway test with aws cli Library OperatingSystem Library String Library BuiltIn -Resource ../commonlib.robot -Resource ../ozone-lib/freon.robot -Test Timeout 5 minutes +Resource ./commonawslib.robot +Suite Setup Setup v4 headers + *** Variables *** -${testuser} testuser +${ENDPOINT_URL} http://s3g:9878 *** Keywords *** -Set Access Key and Id - ${result} = Execute ozone s3 getsecret - ${accessKey} = Get Regexp Matches ${result} (?<=awsAccessKey=).* - ${secret} = Get Regexp Matches ${result} (?<=awsSecret=).* - ${accessKey} = Set Variable ${accessKey[0]} - ${secret} = Set Variable ${secret[0]} - Set Environment Variable AWS_SECRET_ACCESS_KEY ${secret} - Set Environment Variable AWS_ACCESS_KEY_ID ${accessKey} +# Export access key and secret to the environment +Setup aws credentials + ${accessKey} = Execute aws configure get aws_access_key_id + ${secret} = Execute aws configure get aws_secret_access_key + Set Environment Variable AWS_SECRET_ACCESS_KEY ${secret} + Set Environment Variable AWS_ACCESS_KEY_ID ${accessKey} + +Access key id + ${env_var}= Get Environment Variable AWS_ACCESS_KEY_ID Review Comment: This seems to be unnecessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
