[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=648583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648583 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 09/Sep/21 13:04 Start Date: 09/Sep/21 13:04 Worklog Time Spent: 10m Work Description: steveloughran merged pull request #3273: URL: https://github.com/apache/hadoop/pull/3273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648583) Time Spent: 11h 40m (was: 11.5h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 11h 40m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=648083=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648083
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 08/Sep/21 16:55
Start Date: 08/Sep/21 16:55
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3273:
URL: https://github.com/apache/hadoop/pull/3273#issuecomment-915406453
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 13s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 35s | | branch-3.3 passed |
| +1 :green_heart: | compile | 0m 34s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 0m 24s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 0m 38s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 0m 29s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 0m 58s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 18m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 58s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 30s | | The patch does not
generate ASF License warnings. |
| | | 91m 50s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3273/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3273 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux adc64f7444ce 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29
06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / af4adc2cbc66412de4fe602fd71dcb724f557f44 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3273/1/testReport/ |
| Max. process+thread count | 541 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3273/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 648083)
Time Spent: 11.5h (was: 11h 20m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
>
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=639460=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-639460
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 18/Aug/21 16:59
Start Date: 18/Aug/21 16:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-811242106
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 7s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 57s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 75m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 38e03d0eae61 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8e3ea6ef829b4d3391b986d1de118907369adcff |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/testReport/ |
| Max. process+thread count | 735 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=638874=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-638874
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 17/Aug/21 20:29
Start Date: 17/Aug/21 20:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-811242106
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 7s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 57s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 75m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 38e03d0eae61 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8e3ea6ef829b4d3391b986d1de118907369adcff |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/testReport/ |
| Max. process+thread count | 735 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=635059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635059 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 11:00 Start Date: 06/Aug/21 11:00 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #3273: URL: https://github.com/apache/hadoop/pull/3273#issuecomment-894180995 +1 pending yetus being happy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 635059) Time Spent: 11h (was: 10h 50m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 11h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=635058=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635058 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 10:59 Start Date: 06/Aug/21 10:59 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892181247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 635058) Time Spent: 10h 50m (was: 10h 40m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 10h 50m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=635057=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635057
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 06/Aug/21 10:58
Start Date: 06/Aug/21 10:58
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825696671
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 25s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 30s | | The patch does not
generate ASF License warnings. |
| | | 81m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 524ec6161dfd 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6a755632b83621204acdd6e29d0a10042690f1fb |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/16/testReport/ |
| Max. process+thread count | 600 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/16/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=635056=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635056 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 10:58 Start Date: 06/Aug/21 10:58 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-822958936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 635056) Time Spent: 10.5h (was: 10h 20m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 10.5h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=635055=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635055 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 10:58 Start Date: 06/Aug/21 10:58 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816469810 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 635055) Time Spent: 10h 20m (was: 10h 10m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 10h 20m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634980=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634980 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 07:35 Start Date: 06/Aug/21 07:35 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #3273: URL: https://github.com/apache/hadoop/pull/3273#issuecomment-894066970 ``` AppendBlob HNS-OAuth [INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0 [ERROR] Failures: [ERROR] ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value [ERROR] Errors: [ERROR] ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendNoInfiniteLease:173->twoWriters:161 » IO [ERROR] Tests run: 556, Failures: 1, Errors: 1, Skipped: 98 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 76 HNS-OAuth [INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 556, Failures: 0, Errors: 0, Skipped: 98 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 52 HNS-SharedKey [ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:171->fuzzyValidate:49 The actual value 9 is not within the expected range: [5.60, 8.40]. [ERROR] Tests run: 102, Failures: 1, Errors: 0, Skipped: 0 [WARNING] Tests run: 556, Failures: 0, Errors: 0, Skipped: 54 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 40 NonHNS-SharedKey [INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 556, Failures: 0, Errors: 0, Skipped: 276 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 40 ``` JIRAs to track failures: [TestAbfsClientThrottlingAnalyzer](https://issues.apache.org/jira/browse/HADOOP-17826), Appendblob [streamOps](https://issues.apache.org/jira/browse/HADOOP-17716), [Appendblob lease](https://issues.apache.org/jira/browse/HADOOP-17781) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634980) Time Spent: 10h (was: 9h 50m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 10h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634981=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634981 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 07:35 Start Date: 06/Aug/21 07:35 Worklog Time Spent: 10m Work Description: sumangala-patki edited a comment on pull request #3273: URL: https://github.com/apache/hadoop/pull/3273#issuecomment-894066970 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` AppendBlob HNS-OAuth [INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0 [ERROR] Failures: [ERROR] ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value [ERROR] Errors: [ERROR] ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendNoInfiniteLease:173->twoWriters:161 » IO [ERROR] Tests run: 556, Failures: 1, Errors: 1, Skipped: 98 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 76 HNS-OAuth [INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 556, Failures: 0, Errors: 0, Skipped: 98 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 52 HNS-SharedKey [ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:171->fuzzyValidate:49 The actual value 9 is not within the expected range: [5.60, 8.40]. [ERROR] Tests run: 102, Failures: 1, Errors: 0, Skipped: 0 [WARNING] Tests run: 556, Failures: 0, Errors: 0, Skipped: 54 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 40 NonHNS-SharedKey [INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 556, Failures: 0, Errors: 0, Skipped: 276 [WARNING] Tests run: 255, Failures: 0, Errors: 0, Skipped: 40 ``` JIRAs to track failures: [TestAbfsClientThrottlingAnalyzer](https://issues.apache.org/jira/browse/HADOOP-17826), Appendblob [streamOps](https://issues.apache.org/jira/browse/HADOOP-17716), [Appendblob lease](https://issues.apache.org/jira/browse/HADOOP-17781) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634981) Time Spent: 10h 10m (was: 10h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 10h 10m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634976
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 06/Aug/21 07:28
Start Date: 06/Aug/21 07:28
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3273:
URL: https://github.com/apache/hadoop/pull/3273#issuecomment-894063057
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 13s | | branch-3.3 passed |
| +1 :green_heart: | compile | 0m 37s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 0m 28s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 0m 41s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 0m 32s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 1m 1s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 16m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 59s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 72m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3273/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3273 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 610b7e95d601 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / af4adc2cbc66412de4fe602fd71dcb724f557f44 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3273/1/testReport/ |
| Max. process+thread count | 718 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3273/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 634976)
Time Spent: 9h 50m (was: 9h 40m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
>
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634954=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634954 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 06/Aug/21 06:14 Start Date: 06/Aug/21 06:14 Worklog Time Spent: 10m Work Description: sumangala-patki opened a new pull request #3273: URL: https://github.com/apache/hadoop/pull/3273 Contributed by Sumangala Patki (cherry picked from commit 3450522c2f5a4cf9b54dce4c25a71f1b4b98c446) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634954) Time Spent: 9h 40m (was: 9.5h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 9h 40m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634435 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 05/Aug/21 11:50 Start Date: 05/Aug/21 11:50 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825691761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634435) Time Spent: 9.5h (was: 9h 20m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 9.5h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634421=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634421 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 05/Aug/21 11:48 Start Date: 05/Aug/21 11:48 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892704557 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` AppendBlob-HNS-OAuth [WARNING] Tests run: 105, Failures: 0, Errors: 0, Skipped: 1 [ERROR] Failures: [ERROR] ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value [ERROR] Errors: [ERROR] ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendNoInfiniteLease:178->twoWriters:166 » IO [ERROR] Tests run: 558, Failures: 1, Errors: 1, Skipped: 98 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 76 HNS-OAuth [WARNING] Tests run: 105, Failures: 0, Errors: 0, Skipped: 1 [WARNING] Tests run: 558, Failures: 0, Errors: 0, Skipped: 98 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 52 HNS-SharedKey [WARNING] Tests run: 105, Failures: 0, Errors: 0, Skipped: 2 [ERROR] Failures: [ERROR] ITestAbfsRestOperationException.testCustomTokenFetchRetryCount:93->testWithDifferentCustomTokenFetchRetry:122->Assert.assertTrue:42->Assert.fail:89 Number of token fetch retries (4) done, does not match with fs.azure.custom.token.fetch.retry.count configured (0) [ERROR] Tests run: 558, Failures: 1, Errors: 0, Skipped: 54 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 40 NonHNS-SharedKey [ERROR] Failures: [ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:171->fuzzyValidate:49 The actual value 16 is not within the expected range: [5.60, 8.40]. [ERROR] Tests run: 105, Failures: 1, Errors: 0, Skipped: 2 [WARNING] Tests run: 558, Failures: 0, Errors: 0, Skipped: 276 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 40 ``` JIRAs to track failures: [Appendblob lease](https://issues.apache.org/jira/browse/HADOOP-17781), [TestAbfsClientThrottlingAnalyzer](https://issues.apache.org/jira/browse/HADOOP-17826), [StreamOps](https://issues.apache.org/jira/browse/HADOOP-17716) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634421) Time Spent: 9h 20m (was: 9h 10m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 9h 20m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634355=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634355 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 05/Aug/21 11:41 Start Date: 05/Aug/21 11:41 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892888668 thanks, merged to trunk. Do a test run after cherrypicking to 3.3 and we can merge it there too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634355) Time Spent: 9h 10m (was: 9h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 9h 10m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634293=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634293
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 05/Aug/21 11:35
Start Date: 05/Aug/21 11:35
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892765945
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 58s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 2s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 59s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 75m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 9afdd382b76b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2cd7519d6d5c0aa4923aefffedf45ed31b5dd5ad |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/2/testReport/ |
| Max. process+thread count | 715 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=634195=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634195 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 05/Aug/21 11:24 Start Date: 05/Aug/21 11:24 Worklog Time Spent: 10m Work Description: steveloughran merged pull request #2845: URL: https://github.com/apache/hadoop/pull/2845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634195) Time Spent: 8h 50m (was: 8h 40m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 8h 50m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633755=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633755 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 04/Aug/21 18:46 Start Date: 04/Aug/21 18:46 Worklog Time Spent: 10m Work Description: steveloughran merged pull request #2845: URL: https://github.com/apache/hadoop/pull/2845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 633755) Time Spent: 8.5h (was: 8h 20m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 8.5h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633756=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633756 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 04/Aug/21 18:46 Start Date: 04/Aug/21 18:46 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892888668 thanks, merged to trunk. Do a test run after cherrypicking to 3.3 and we can merge it there too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 633756) Time Spent: 8h 40m (was: 8.5h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 8h 40m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633664=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633664
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 04/Aug/21 15:43
Start Date: 04/Aug/21 15:43
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892765945
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 58s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 2s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 59s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 75m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 9afdd382b76b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2cd7519d6d5c0aa4923aefffedf45ed31b5dd5ad |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/2/testReport/ |
| Max. process+thread count | 715 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633579=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633579 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 04/Aug/21 14:26 Start Date: 04/Aug/21 14:26 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892704557 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` AppendBlob-HNS-OAuth [WARNING] Tests run: 105, Failures: 0, Errors: 0, Skipped: 1 [ERROR] Failures: [ERROR] ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value [ERROR] Errors: [ERROR] ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendNoInfiniteLease:178->twoWriters:166 » IO [ERROR] Tests run: 558, Failures: 1, Errors: 1, Skipped: 98 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 76 HNS-OAuth [WARNING] Tests run: 105, Failures: 0, Errors: 0, Skipped: 1 [WARNING] Tests run: 558, Failures: 0, Errors: 0, Skipped: 98 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 52 HNS-SharedKey [WARNING] Tests run: 105, Failures: 0, Errors: 0, Skipped: 2 [ERROR] Failures: [ERROR] ITestAbfsRestOperationException.testCustomTokenFetchRetryCount:93->testWithDifferentCustomTokenFetchRetry:122->Assert.assertTrue:42->Assert.fail:89 Number of token fetch retries (4) done, does not match with fs.azure.custom.token.fetch.retry.count configured (0) [ERROR] Tests run: 558, Failures: 1, Errors: 0, Skipped: 54 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 40 NonHNS-SharedKey [ERROR] Failures: [ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:171->fuzzyValidate:49 The actual value 16 is not within the expected range: [5.60, 8.40]. [ERROR] Tests run: 105, Failures: 1, Errors: 0, Skipped: 2 [WARNING] Tests run: 558, Failures: 0, Errors: 0, Skipped: 276 [WARNING] Tests run: 259, Failures: 0, Errors: 0, Skipped: 40 ``` JIRAs to track failures: [Appendblob lease](https://issues.apache.org/jira/browse/HADOOP-17781), [TestAbfsClientThrottlingAnalyzer](https://issues.apache.org/jira/browse/HADOOP-17826), [StreamOps](https://issues.apache.org/jira/browse/HADOOP-17716) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 633579) Time Spent: 8h 10m (was: 8h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 8h 10m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633435
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 04/Aug/21 10:55
Start Date: 04/Aug/21 10:55
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-824633241
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 4m 48s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/12/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 2m 17s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 4s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/12/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0
fixed = 4 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 2s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 52m 22s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e110be4fc283 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 43597189aa88707ae0cac43951a35b4c5df4e201 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633433
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 04/Aug/21 10:54
Start Date: 04/Aug/21 10:54
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825635266
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 16s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 1s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 79m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 63c61097c366 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c69b3922cb447c0ab7184c912db410b77c0e0d53 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/13/testReport/ |
| Max. process+thread count | 583 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/13/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633432=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633432 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 04/Aug/21 10:54 Start Date: 04/Aug/21 10:54 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825691761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 633432) Time Spent: 7h 40m (was: 7.5h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 7h 40m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=633177=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633177 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 03/Aug/21 21:33 Start Date: 03/Aug/21 21:33 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-892181247 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | patch | 0m 21s | | https://github.com/apache/hadoop/pull/2845 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/2845 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 633177) Time Spent: 7.5h (was: 7h 20m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 7.5h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=590853=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590853 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 29/Apr/21 06:30 Start Date: 29/Apr/21 06:30 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-828973979 Hi @steveloughran, thanks for the review. Have addressed the comments, please take a look. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 590853) Time Spent: 7h 20m (was: 7h 10m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 7h 20m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=589649=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589649 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 27/Apr/21 10:09 Start Date: 27/Apr/21 10:09 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-89174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589649) Time Spent: 7h 10m (was: 7h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 7h 10m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=588001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-588001 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 23/Apr/21 17:05 Start Date: 23/Apr/21 17:05 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825793853 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 99, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 519, Failures: 0, Errors: 0, Skipped: 70 [WARNING] Tests run: 261, Failures: 0, Errors: 0, Skipped: 50 HNS SharedKey [INFO] Tests run: 99, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 507, Failures: 0, Errors: 0, Skipped: 26 [WARNING] Tests run: 251, Failures: 0, Errors: 0, Skipped: 40 Non-HNS SharedKey [INFO] Tests run: 99, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 519, Failures: 0, Errors: 0, Skipped: 248 [ERROR] Errors: [ERROR] ITestAbfsReadWriteAndSeek.testReadAndWriteWithDifferentBufferSizesAndSeek:62->testReadWriteAndSeek:78 ? TestTimedOut [ERROR] Tests run: 261, Failures: 0, Errors: 1, Skipped: 40 ``` JIRA to track timeout failure: [ITestAbfsReadWriteAndSeek](https://issues.apache.org/jira/browse/HADOOP-15702) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 588001) Time Spent: 7h (was: 6h 50m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 7h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587869=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587869
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 14:26
Start Date: 23/Apr/21 14:26
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825696671
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 25s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 30s | | The patch does not
generate ASF License warnings. |
| | | 81m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 524ec6161dfd 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6a755632b83621204acdd6e29d0a10042690f1fb |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/16/testReport/ |
| Max. process+thread count | 600 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/16/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587865=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587865
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 14:19
Start Date: 23/Apr/21 14:19
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825691761
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 48s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 2s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 83m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e6e07728c4f6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6283044912e7b82381ede642d6c860547820a98f |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/15/testReport/ |
| Max. process+thread count | 671 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/15/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587863=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587863
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 14:14
Start Date: 23/Apr/21 14:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825688366
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 55s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 4s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 59s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 84m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7e88b338881a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cb464328d51f1e2640d7e3eb2db2ece893f959ca |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/14/testReport/ |
| Max. process+thread count | 542 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/14/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587830=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587830
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 13:01
Start Date: 23/Apr/21 13:01
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r619201004
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +78,36 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String encodedUrlStr(String url) {
+try {
+ return URLEncoder.encode(url, "UTF-8");
+} catch (UnsupportedEncodingException e) {
+ return "https%3A%2F%2Ffailed%2Fto%2Fencode%2Furl";
+}
+ }
+
+ public static String getMaskedUrl(String url) {
+for (String qpKey : MASK_PARAM_KEYS) {
Review comment:
I think this comment has got posted from an older review; using std
methods now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587830)
Time Spent: 6h 20m (was: 6h 10m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587827=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587827
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 13:00
Start Date: 23/Apr/21 13:00
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r619200089
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +98,63 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587827)
Time Spent: 6h 10m (was: 6h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587825=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587825
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 12:59
Start Date: 23/Apr/21 12:59
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r619199531
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +98,63 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
+ HashSet queryParamsForPartialMask) {
+return maskUrlQueryParameters(keyValueList, queryParamsForFullMask,
+queryParamsForPartialMask, 256);
+ }
+
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
+ HashSet queryParamsForPartialMask, int queryLen) {
+StringBuilder maskedUrl = new StringBuilder(queryLen);
+for (NameValuePair keyValuePair : keyValueList) {
+ String key = keyValuePair.getName();
+ if (key.isEmpty()) {
+throw new IllegalArgumentException("Query param key should not be
empty");
+ }
+ String value = keyValuePair.getValue();
+ maskedUrl.append(key);
+ maskedUrl.append(EQUAL);
+ if (value != null && !value.isEmpty()) { //no mask
+if (queryParamsForFullMask.contains(key)) {
+ maskedUrl.append(FULL_MASK);
+} else if (queryParamsForPartialMask.contains(key)) {
+ int valueLen = value.length();
+ int maskedLen = valueLen - Math.min(4, valueLen);
+ maskedUrl.append(StringUtils.repeat(CHAR_MASK, maskedLen));
+ maskedUrl.append(value, maskedLen, valueLen);
+} else {
+ maskedUrl.append(value);
+}
+ }
+ maskedUrl.append(AND_MARK);
+}
+maskedUrl.deleteCharAt(maskedUrl.length() - 1);
+return maskedUrl.toString();
+ }
+
+ public static String encodedUrlStr(String url) {
+try {
+ return URLEncoder.encode(url, "UTF-8");
+} catch (UnsupportedEncodingException e) {
+ return "https%3A%2F%2Ffailed%2Fto%2Fencode%2Furl";
Review comment:
Yes :D this function was a part of AbfsHttpOp earlier
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587825)
Time Spent: 6h (was: 5h 50m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587824=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587824
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 12:57
Start Date: 23/Apr/21 12:57
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r619198007
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +98,63 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
+ HashSet queryParamsForPartialMask) {
+return maskUrlQueryParameters(keyValueList, queryParamsForFullMask,
+queryParamsForPartialMask, 256);
+ }
+
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
+ HashSet queryParamsForPartialMask, int queryLen) {
+StringBuilder maskedUrl = new StringBuilder(queryLen);
+for (NameValuePair keyValuePair : keyValueList) {
+ String key = keyValuePair.getName();
+ if (key.isEmpty()) {
+throw new IllegalArgumentException("Query param key should not be
empty");
+ }
+ String value = keyValuePair.getValue();
+ maskedUrl.append(key);
+ maskedUrl.append(EQUAL);
+ if (value != null && !value.isEmpty()) { //no mask
+if (queryParamsForFullMask.contains(key)) {
+ maskedUrl.append(FULL_MASK);
+} else if (queryParamsForPartialMask.contains(key)) {
+ int valueLen = value.length();
+ int maskedLen = valueLen - Math.min(4, valueLen);
Review comment:
done [have modified const value to align with server logs masking logic]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587824)
Time Spent: 5h 50m (was: 5h 40m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587823=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587823
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 12:55
Start Date: 23/Apr/21 12:55
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r619197177
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -18,14 +18,39 @@
package org.apache.hadoop.fs.azurebfs.utils;
+import java.io.UnsupportedEncodingException;
+import java.net.URL;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
import java.util.regex.Pattern;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.NameValuePair;
+import org.apache.http.client.utils.URLEncodedUtils;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.AND_MARK;
+import static org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.EQUAL;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SAOID;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SIGNATURE;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SKOID;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SUOID;
+
/**
* Utility class to help with Abfs url transformation to blob urls.
*/
public final class UriUtils {
private static final String ABFS_URI_REGEX =
"[^.]+\\.dfs\\.(preprod\\.){0,1}core\\.windows\\.net";
private static final Pattern ABFS_URI_PATTERN =
Pattern.compile(ABFS_URI_REGEX);
+ private static final HashSet FULL_MASK_PARAM_KEYS = new HashSet<>(
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587823)
Time Spent: 5h 40m (was: 5.5h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587822=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587822
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 12:55
Start Date: 23/Apr/21 12:55
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r619196996
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -227,11 +229,30 @@ public String getLogString() {
.append(" m=")
.append(method)
.append(" u=")
- .append(getSignatureMaskedEncodedUrl());
+ .append(getMaskedEncodedUrl());
return sb.toString();
}
+ public String getMaskedUrl() {
+if (!shouldMask) {
+ return url.toString();
+}
+if (this.maskedUrl != null) {
Review comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587822)
Time Spent: 5.5h (was: 5h 20m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587821=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587821
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 12:49
Start Date: 23/Apr/21 12:49
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-825635266
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 16s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 1s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 79m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 63c61097c366 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c69b3922cb447c0ab7184c912db410b77c0e0d53 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/13/testReport/ |
| Max. process+thread count | 583 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/13/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587798=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587798
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 23/Apr/21 12:02
Start Date: 23/Apr/21 12:02
Worklog Time Spent: 10m
Work Description: vinaysbadami commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r611322958
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +78,36 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String encodedUrlStr(String url) {
+try {
+ return URLEncoder.encode(url, "UTF-8");
+} catch (UnsupportedEncodingException e) {
+ return "https%3A%2F%2Ffailed%2Fto%2Fencode%2Furl";
+}
+ }
+
+ public static String getMaskedUrl(String url) {
+for (String qpKey : MASK_PARAM_KEYS) {
Review comment:
can we use standard url methods for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 587798)
Time Spent: 5h 10m (was: 5h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587315=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587315
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 22/Apr/21 15:33
Start Date: 22/Apr/21 15:33
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r618504576
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -227,11 +229,30 @@ public String getLogString() {
.append(" m=")
.append(method)
.append(" u=")
- .append(getSignatureMaskedEncodedUrl());
+ .append(getMaskedEncodedUrl());
return sb.toString();
}
+ public String getMaskedUrl() {
+if (!shouldMask) {
+ return url.toString();
+}
+if (this.maskedUrl != null) {
Review comment:
remove the this. prefix; not needed
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +98,63 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
+ HashSet queryParamsForPartialMask) {
+return maskUrlQueryParameters(keyValueList, queryParamsForFullMask,
+queryParamsForPartialMask, 256);
+ }
+
+ public static String maskUrlQueryParameters(List keyValueList,
+ HashSet queryParamsForFullMask,
+ HashSet queryParamsForPartialMask, int queryLen) {
+StringBuilder maskedUrl = new StringBuilder(queryLen);
+for (NameValuePair keyValuePair : keyValueList) {
+ String key = keyValuePair.getName();
+ if (key.isEmpty()) {
+throw new IllegalArgumentException("Query param key should not be
empty");
+ }
+ String value = keyValuePair.getValue();
+ maskedUrl.append(key);
+ maskedUrl.append(EQUAL);
+ if (value != null && !value.isEmpty()) { //no mask
+if (queryParamsForFullMask.contains(key)) {
+ maskedUrl.append(FULL_MASK);
+} else if (queryParamsForPartialMask.contains(key)) {
+ int valueLen = value.length();
+ int maskedLen = valueLen - Math.min(4, valueLen);
+ maskedUrl.append(StringUtils.repeat(CHAR_MASK, maskedLen));
+ maskedUrl.append(value, maskedLen, valueLen);
+} else {
+ maskedUrl.append(value);
+}
+ }
+ maskedUrl.append(AND_MARK);
+}
+maskedUrl.deleteCharAt(maskedUrl.length() - 1);
+return maskedUrl.toString();
+ }
+
+ public static String encodedUrlStr(String url) {
+try {
+ return URLEncoder.encode(url, "UTF-8");
+} catch (UnsupportedEncodingException e) {
+ return "https%3A%2F%2Ffailed%2Fto%2Fencode%2Furl";
Review comment:
heh. nice.
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -18,14 +18,39 @@
package org.apache.hadoop.fs.azurebfs.utils;
+import java.io.UnsupportedEncodingException;
+import java.net.URL;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
import java.util.regex.Pattern;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.NameValuePair;
+import org.apache.http.client.utils.URLEncodedUtils;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.AND_MARK;
+import static org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.EQUAL;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SAOID;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SIGNATURE;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SKOID;
+import static
org.apache.hadoop.fs.azurebfs.constants.HttpQueryParams.QUERY_PARAM_SUOID;
+
/**
* Utility class to help with Abfs url transformation to blob urls.
*/
public final class UriUtils {
private static final String ABFS_URI_REGEX =
"[^.]+\\.dfs\\.(preprod\\.){0,1}core\\.windows\\.net";
private static final Pattern ABFS_URI_PATTERN =
Pattern.compile(ABFS_URI_REGEX);
+ private static final HashSet FULL_MASK_PARAM_KEYS = new HashSet<>(
Review comment:
use `Set` as the type of variable
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +98,63 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587297=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587297
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 22/Apr/21 15:23
Start Date: 22/Apr/21 15:23
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-821037219
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 13s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 46s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/8/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 5s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/8/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 94m 40s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azurebfs.services.TestAbfsPerfTracker |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 99f0d8107016 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 42d76d46af4fd505846fe291880ac92122f3d71d |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587298=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587298
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 22/Apr/21 15:23
Start Date: 22/Apr/21 15:23
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816990399
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 30s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 74m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 47403bc6a3e0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d72d490b27e5db691d2bcd3af4869b6d18f23295 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/7/testReport/ |
| Max. process+thread count | 670 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587296=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587296
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 22/Apr/21 15:23
Start Date: 22/Apr/21 15:23
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-822809327
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 18s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 3s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 6s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/9/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 86m 38s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azurebfs.services.TestAbfsPerfTracker |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e657838fbf56 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3a95de32d14c0c9b5b00a91edd200cb1aa9991ef |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/9/testReport/ |
| Max. process+thread count | 719 (vs. ulimit of 5500) |
| modules | C:
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=587115=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-587115
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 22/Apr/21 08:06
Start Date: 22/Apr/21 08:06
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-824633241
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 4m 48s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/12/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 2m 17s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 4s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/12/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0
fixed = 4 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 2s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 52m 22s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e110be4fc283 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 43597189aa88707ae0cac43951a35b4c5df4e201 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=586356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-586356
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 21/Apr/21 06:28
Start Date: 21/Apr/21 06:28
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r617235044
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +93,55 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ ArrayList queryParamsForFullMask,
+ ArrayList queryParamsForPartialMask) {
+StringBuilder maskedUrl = new StringBuilder();
+for (NameValuePair keyValuePair : keyValueList) {
+ String key = keyValuePair.getName();
+ if (key.isEmpty()) {
+throw new IllegalArgumentException("Query param key can not be empty");
+ }
+ String value = keyValuePair.getValue();
+ maskedUrl.append(key);
+ maskedUrl.append(EQUAL);
+ if (value != null && !value.isEmpty()) { //no mask
+if (queryParamsForFullMask.contains(key)) {
+ maskedUrl.append(MASK);
+} else if (queryParamsForPartialMask.contains(key)) {
+ int visibleLen = Math.min(4, value.length());
+ maskedUrl.append(value, 0, visibleLen);
+ maskedUrl.append(MASK);
+} else {
+ maskedUrl.append(value);
+}
+ }
+ maskedUrl.append(AND_MARK);
+}
+maskedUrl.deleteCharAt(maskedUrl.length() - 1);
+return maskedUrl.toString();
+ }
+
+ public static String encodedUrlStr(String url) {
+try {
+ return URLEncoder.encode(url, "UTF-8");
+} catch (UnsupportedEncodingException e) {
+ return "https%3A%2F%2Ffailed%2Fto%2Fencode%2Furl";
+}
+ }
+
+ public static String getMaskedUrl(URL url) {
+String queryString = url.getQuery();
+if (queryString == null) {
Review comment:
check for SAS auth type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 586356)
Time Spent: 4h 10m (was: 4h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=586354=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-586354
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 21/Apr/21 06:20
Start Date: 21/Apr/21 06:20
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r617231047
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +93,55 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ ArrayList queryParamsForFullMask,
+ ArrayList queryParamsForPartialMask) {
+StringBuilder maskedUrl = new StringBuilder();
+for (NameValuePair keyValuePair : keyValueList) {
+ String key = keyValuePair.getName();
+ if (key.isEmpty()) {
+throw new IllegalArgumentException("Query param key can not be empty");
+ }
+ String value = keyValuePair.getValue();
+ maskedUrl.append(key);
+ maskedUrl.append(EQUAL);
+ if (value != null && !value.isEmpty()) { //no mask
+if (queryParamsForFullMask.contains(key)) {
+ maskedUrl.append(MASK);
+} else if (queryParamsForPartialMask.contains(key)) {
+ int visibleLen = Math.min(4, value.length());
+ maskedUrl.append(value, 0, visibleLen);
Review comment:
make last 4 visible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 586354)
Time Spent: 4h (was: 3h 50m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=586351=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-586351
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 21/Apr/21 06:13
Start Date: 21/Apr/21 06:13
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r617228128
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -73,6 +93,55 @@ public static String generateUniqueTestPath() {
return testUniqueForkId == null ? "/test" : "/" + testUniqueForkId +
"/test";
}
+ public static String maskUrlQueryParameters(List keyValueList,
+ ArrayList queryParamsForFullMask,
+ ArrayList queryParamsForPartialMask) {
+StringBuilder maskedUrl = new StringBuilder();
Review comment:
set a size for stringbuilder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 586351)
Time Spent: 3h 50m (was: 3h 40m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=586349=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-586349
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 21/Apr/21 06:11
Start Date: 21/Apr/21 06:11
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r617227165
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
##
@@ -18,14 +18,34 @@
package org.apache.hadoop.fs.azurebfs.utils;
+import java.io.UnsupportedEncodingException;
+import java.net.URL;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
import java.util.regex.Pattern;
+import org.apache.http.NameValuePair;
+import org.apache.http.client.utils.URLEncodedUtils;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.AND_MARK;
+import static org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.EQUAL;
+
/**
* Utility class to help with Abfs url transformation to blob urls.
*/
public final class UriUtils {
private static final String ABFS_URI_REGEX =
"[^.]+\\.dfs\\.(preprod\\.){0,1}core\\.windows\\.net";
private static final Pattern ABFS_URI_PATTERN =
Pattern.compile(ABFS_URI_REGEX);
+ private static final ArrayList FULL_MASK_PARAM_KEYS = new
ArrayList<>(
+ Collections.singleton("sig"));
+ private static final ArrayList PARTIAL_MASK_PARAM_KEYS =
+ new ArrayList<>(
+ Arrays.asList("skoid", "saoid", "suoid"));
Review comment:
move to const file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 586349)
Time Spent: 3h 40m (was: 3.5h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=585573=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-585573 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 20/Apr/21 05:56 Start Date: 20/Apr/21 05:56 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-822994686 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS-OAuth [INFO] Tests run: 95, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 494, Failures: 0, Errors: 0, Skipped: 71 [WARNING] Tests run: 261, Failures: 0, Errors: 0, Skipped: 50 HNS-SharedKey [INFO] Tests run: 95, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 503, Failures: 0, Errors: 0, Skipped: 26 [WARNING] Tests run: 261, Failures: 0, Errors: 0, Skipped: 40 NonHNS-SharedKey [INFO] Tests run: 95, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 506, Failures: 0, Errors: 0, Skipped: 249 [WARNING] Tests run: 261, Failures: 0, Errors: 0, Skipped: 40 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 585573) Time Spent: 3.5h (was: 3h 20m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=58=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-58
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 20/Apr/21 05:04
Start Date: 20/Apr/21 05:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-822975575
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 36s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 1s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 79m 44s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 2332844ad30c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1e391437c058f691918877ee06ed5a6cc3e4d499 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/11/testReport/ |
| Max. process+thread count | 730 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=585545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-585545
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 20/Apr/21 04:07
Start Date: 20/Apr/21 04:07
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-822958936
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 49s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 1s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 79m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux c993feda8e9e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d635f073fa800b0062ac027078de09b3a3aa805b |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/10/testReport/ |
| Max. process+thread count | 671 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=585543=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-585543
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 20/Apr/21 03:53
Start Date: 20/Apr/21 03:53
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r616323639
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -227,11 +224,26 @@ public String getLogString() {
.append(" m=")
.append(method)
.append(" u=")
- .append(getSignatureMaskedEncodedUrl());
+ .append(getMaskedEncodedUrl());
return sb.toString();
}
+ public String getMaskedUrl() {
+if (maskedUrl != null) {
+ return maskedUrl;
Review comment:
done
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -227,11 +224,26 @@ public String getLogString() {
.append(" m=")
.append(method)
.append(" u=")
- .append(getSignatureMaskedEncodedUrl());
+ .append(getMaskedEncodedUrl());
return sb.toString();
}
+ public String getMaskedUrl() {
+if (maskedUrl != null) {
+ return maskedUrl;
+}
+maskedUrl = UriUtils.getMaskedUrl(url.toString());
+return maskedUrl;
+ }
+
+ public String getMaskedEncodedUrl() {
+if (maskedEncodedUrl != null) {
+ return maskedEncodedUrl;
+}
+return UriUtils.encodedUrlStr(getMaskedUrl());
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 585543)
Time Spent: 3h (was: 2h 50m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=585387=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-585387
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 19/Apr/21 21:49
Start Date: 19/Apr/21 21:49
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-822809327
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 18s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 3s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 6s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/9/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 86m 38s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azurebfs.services.TestAbfsPerfTracker |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e657838fbf56 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3a95de32d14c0c9b5b00a91edd200cb1aa9991ef |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/9/testReport/ |
| Max. process+thread count | 719 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U:
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=584156=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-584156
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 16/Apr/21 09:10
Start Date: 16/Apr/21 09:10
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-821037219
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 13s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 46s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/8/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 5s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/8/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 94m 40s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azurebfs.services.TestAbfsPerfTracker |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 99f0d8107016 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 42d76d46af4fd505846fe291880ac92122f3d71d |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580777=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580777
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 12/Apr/21 05:05
Start Date: 12/Apr/21 05:05
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r611329081
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -227,11 +224,26 @@ public String getLogString() {
.append(" m=")
.append(method)
.append(" u=")
- .append(getSignatureMaskedEncodedUrl());
+ .append(getMaskedEncodedUrl());
return sb.toString();
}
+ public String getMaskedUrl() {
+if (maskedUrl != null) {
+ return maskedUrl;
+}
+maskedUrl = UriUtils.getMaskedUrl(url.toString());
+return maskedUrl;
+ }
+
+ public String getMaskedEncodedUrl() {
+if (maskedEncodedUrl != null) {
+ return maskedEncodedUrl;
+}
+return UriUtils.encodedUrlStr(getMaskedUrl());
Review comment:
assign the value to this.maskedEncodedUrl, then return the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 580777)
Time Spent: 2.5h (was: 2h 20m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580776=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580776
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 12/Apr/21 05:04
Start Date: 12/Apr/21 05:04
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r611329008
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -227,11 +224,26 @@ public String getLogString() {
.append(" m=")
.append(method)
.append(" u=")
- .append(getSignatureMaskedEncodedUrl());
+ .append(getMaskedEncodedUrl());
return sb.toString();
}
+ public String getMaskedUrl() {
+if (maskedUrl != null) {
+ return maskedUrl;
Review comment:
this. , for consistency reasons
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 580776)
Time Spent: 2h 20m (was: 2h 10m)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580481=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580481
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 10/Apr/21 05:02
Start Date: 10/Apr/21 05:02
Worklog Time Spent: 10m
Work Description: sumangala-patki commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r610997227
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -558,6 +560,24 @@ public String getSignatureMaskedEncodedUrl() {
return this.maskedEncodedUrl;
}
+ public void maskSASObjectIDs() {
+int oidStartIdx, ampIdx, oidEndIndex, qpStrIdx;
Review comment:
done
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -558,6 +560,24 @@ public String getSignatureMaskedEncodedUrl() {
return this.maskedEncodedUrl;
}
+ public void maskSASObjectIDs() {
+int oidStartIdx, ampIdx, oidEndIndex, qpStrIdx;
+for (String qpKey : SAS_OID_PARAM_KEYS) {
+ qpStrIdx = maskedUrl.indexOf('&' + qpKey);
+ if (qpStrIdx == -1) {
+qpStrIdx = maskedUrl.indexOf('?' + qpKey);
+if (qpStrIdx == -1) {
+ continue;
+}
+ }
+ oidStartIdx = qpStrIdx + qpKey.length() + 1;
+ ampIdx = maskedUrl.indexOf("&", oidStartIdx);
+ oidEndIndex = (ampIdx != -1) ? ampIdx : maskedUrl.length();
+ maskedUrl = maskedUrl.substring(0, oidStartIdx + 5) + "" + maskedUrl
+ .substring(oidEndIndex);
+}
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 580481)
Time Spent: 2h 10m (was: 2h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580362=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580362
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 09/Apr/21 21:50
Start Date: 09/Apr/21 21:50
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816990399
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 30s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 74m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 47403bc6a3e0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d72d490b27e5db691d2bcd3af4869b6d18f23295 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/7/testReport/ |
| Max. process+thread count | 670 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580321=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580321 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 09/Apr/21 21:00 Start Date: 09/Apr/21 21:00 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816969123 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 512, Failures: 0, Errors: 0, Skipped: 71 [WARNING] Tests run: 261, Failures: 0, Errors: 0, Skipped: 50 HNS SharedKey [INFO] Tests run: 93, Failures: 1, Errors: 0, Skipped: 0 [WARNING] Tests run: 506, Failures: 0, Errors: 0, Skipped: 26 [ERROR] Errors: ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 ? TestTimedOut [INFO] [ERROR] Tests run: 251, Failures: 0, Errors: 1, Skipped: 40 Non-HNS SharedKey [INFO] Tests run: 93, Failures: 1, Errors: 0, Skipped: 0 [WARNING] Tests run: 506, Failures: 0, Errors: 0, Skipped: 249 [WARNING] Tests run: 261, Failures: 0, Errors: 0, Skipped: 40 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 580321) Time Spent: 1h 50m (was: 1h 40m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580310=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580310
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 09/Apr/21 20:31
Start Date: 09/Apr/21 20:31
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816955565
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 9s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 59s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 1m 57s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/6/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 75m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azurebfs.services.TestAbfsHttpOperation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 5ce49daba023 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7af9f80120f64b089e12f1c9d9feb4292d61dd38 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/6/testReport/ |
| Max. process+thread count | 596 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U:
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=580073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-580073
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 09/Apr/21 15:12
Start Date: 09/Apr/21 15:12
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816753156
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 58s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/5/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 2 new + 0 unchanged - 0
fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 1m 57s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/5/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 74m 57s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azurebfs.services.TestAbfsHttpOperation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 999115d44da1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 407143ff232959ada459122237b563478aaa3e8e |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=579813=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-579813
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 09/Apr/21 07:16
Start Date: 09/Apr/21 07:16
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-816469810
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 11s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/4/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 2 new + 0 unchanged - 0
fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 59s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 81m 0s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 71d36e92d1d1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 57c437e76e4efbc4198b158f8508cf8bca9f9b77 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/4/testReport/ |
| Max. process+thread count | 539 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=575356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-575356
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 01/Apr/21 07:15
Start Date: 01/Apr/21 07:15
Worklog Time Spent: 10m
Work Description: vinaysbadami commented on a change in pull request
#2845:
URL: https://github.com/apache/hadoop/pull/2845#discussion_r605424939
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -558,6 +560,24 @@ public String getSignatureMaskedEncodedUrl() {
return this.maskedEncodedUrl;
}
+ public void maskSASObjectIDs() {
+int oidStartIdx, ampIdx, oidEndIndex, qpStrIdx;
+for (String qpKey : SAS_OID_PARAM_KEYS) {
+ qpStrIdx = maskedUrl.indexOf('&' + qpKey);
Review comment:
this.maskedUrl to be consistent with rest of file
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -558,6 +560,24 @@ public String getSignatureMaskedEncodedUrl() {
return this.maskedEncodedUrl;
}
+ public void maskSASObjectIDs() {
+int oidStartIdx, ampIdx, oidEndIndex, qpStrIdx;
+for (String qpKey : SAS_OID_PARAM_KEYS) {
+ qpStrIdx = maskedUrl.indexOf('&' + qpKey);
+ if (qpStrIdx == -1) {
+qpStrIdx = maskedUrl.indexOf('?' + qpKey);
+if (qpStrIdx == -1) {
+ continue;
+}
+ }
+ oidStartIdx = qpStrIdx + qpKey.length() + 1;
+ ampIdx = maskedUrl.indexOf("&", oidStartIdx);
+ oidEndIndex = (ampIdx != -1) ? ampIdx : maskedUrl.length();
+ maskedUrl = maskedUrl.substring(0, oidStartIdx + 5) + "" + maskedUrl
+ .substring(oidEndIndex);
+}
Review comment:
should we move all the masking logic to a single static method that
takes a string and returns a masked string
that will make testing easier.
Also should we look at this method in a utils class to keep this class
cleaner
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
##
@@ -558,6 +560,24 @@ public String getSignatureMaskedEncodedUrl() {
return this.maskedEncodedUrl;
}
+ public void maskSASObjectIDs() {
+int oidStartIdx, ampIdx, oidEndIndex, qpStrIdx;
Review comment:
move to point of first use
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 575356)
Time Spent: 1h 10m (was: 1h)
> ABFS: Partially obfuscate SAS object IDs in Logs
>
>
> Key: HADOOP-17618
> URL: https://issues.apache.org/jira/browse/HADOOP-17618
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Delegation SAS tokens are created using various parameters for specifying
> details such as permissions and validity. The requests are logged, along with
> values of all the query parameters. This change will partially mask values
> logged for the following object IDs representing the security principal:
> skoid, saoid, suoid
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=575333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-575333 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 01/Apr/21 05:57 Start Date: 01/Apr/21 05:57 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-811664415 Hi @steveloughran this is for privacy (no issue yet); the values masked identify the security principal (user/app) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 575333) Time Spent: 1h (was: 50m) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=574985=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-574985
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 31/Mar/21 16:46
Start Date: 31/Mar/21 16:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-811242106
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 7s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 57s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 75m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 38e03d0eae61 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8e3ea6ef829b4d3391b986d1de118907369adcff |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/testReport/ |
| Max. process+thread count | 735 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=574937=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-574937 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 31/Mar/21 16:06 Start Date: 31/Mar/21 16:06 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-811176186 this security/privacy issues, or just due diligence? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 574937) Time Spent: 40m (was: 0.5h) > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask values > logged for the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=574896=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-574896
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 31/Mar/21 15:04
Start Date: 31/Mar/21 15:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-811134933
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 8s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | spotbugs | 1m 2s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/2/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 14m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 79m 13s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.SAS_OID_PARAM_KEYS
should be package protected At AbfsHttpOperation.java: At
AbfsHttpOperation.java:[line 55] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 99c675fb5a63 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc641124c903e8650f8512f91ed0b31636b78012 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[
https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=574886=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-574886
]
ASF GitHub Bot logged work on HADOOP-17618:
---
Author: ASF GitHub Bot
Created on: 31/Mar/21 14:41
Start Date: 31/Mar/21 14:41
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2845:
URL: https://github.com/apache/hadoop/pull/2845#issuecomment-89174
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 22m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | spotbugs | 1m 7s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/1/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 14m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 54s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 98m 47s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.SAS_OID_PARAM_KEYS
should be package protected At AbfsHttpOperation.java: At
AbfsHttpOperation.java:[line 55] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2845/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2845 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 43b865b6a2ef 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e0b34b31c682c24cd049910e0c4e8ba63213e4ea |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
[jira] [Work logged] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs
[ https://issues.apache.org/jira/browse/HADOOP-17618?focusedWorklogId=574817=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-574817 ] ASF GitHub Bot logged work on HADOOP-17618: --- Author: ASF GitHub Bot Created on: 31/Mar/21 12:57 Start Date: 31/Mar/21 12:57 Worklog Time Spent: 10m Work Description: sumangala-patki opened a new pull request #2845: URL: https://github.com/apache/hadoop/pull/2845 Delegation SAS tokens are created using various parameters for specifying details such as permissions and validity. The requests are logged, along with values of all the query parameters. This change will partially mask the values of the following object IDs representing the security principal: `skoid`, `saoid`, `suoid` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 574817) Remaining Estimate: 0h Time Spent: 10m > ABFS: Partially obfuscate SAS object IDs in Logs > > > Key: HADOOP-17618 > URL: https://issues.apache.org/jira/browse/HADOOP-17618 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Delegation SAS tokens are created using various parameters for specifying > details such as permissions and validity. The requests are logged, along with > values of all the query parameters. This change will partially mask the > values of the following object IDs representing the security principal: > skoid, saoid, suoid -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
