[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647504#comment-17647504
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1351504800
ok, merged in a different branch, backporting with retesting. closing this
one as done.
thanks!
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647505#comment-17647505
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran closed pull request #5110: HADOOP-18183. s3a audit logs to
publish range start/end of GET requests in audit header
URL: https://github.com/apache/hadoop/pull/5110
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647503#comment-17647503
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on PR #5218:
URL: https://github.com/apache/hadoop/pull/5218#issuecomment-1351503498
merged, backporting.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647114#comment-17647114
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran merged PR #5218:
URL: https://github.com/apache/hadoop/pull/5218
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647113#comment-17647113
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on PR #5218:
URL: https://github.com/apache/hadoop/pull/5218#issuecomment-1351408553
build was happy locally; test s3 london @ scale, and verified in logs that
things worked
```
183c9826b45486e485693808f38e2c4071004bf5dfd4c3ab210f0a21a4235ef8
stevel-london [14/Dec/2022:12:59:35 +] 109.157.193.147
arn:aws:iam::152813717728:user/stevel-dev 446CNGJCGD6DCZ4D REST.GET.OBJECT
fork-0007/test/testReadSingleByteAcrossReadahead-sequential "GET
/fork-0007/test/testReadSingleByteAcrossReadahead-sequential?versionId=uWXlWeVE5kCrTG.TRmC4gNekp5yWrELr
HTTP/1.1" 206 - 2048 2048 28 14
"https://audit.example.org/hadoop/1/op_open/be0d9c4a-964e-423d-b22b-7fa5b56db268-0029/?op=op_open&p1=fork-0007/test/testReadSingleByteAcrossReadahead-sequential&pr=stevel&ps=d12bd202-8831-4faf-965e-f3d7f23c3782&rg=0-2047&id=be0d9c4a-964e-423d-b22b-7fa5b56db268-0029&t0=53&fs=be0d9c4a-964e-423d-b22b-7fa5b56db268&t1=53&ts=1671022775232";
here rg=0-2047; a single byte was read in a stream with sequential read and
readahead.
I am happy with this and doing the merge, as it is a yetus issue and have
just done the testing locally.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646695#comment-17646695
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on PR #5218:
URL: https://github.com/apache/hadoop/pull/5218#issuecomment-1348771609
this is just #5110 with two changes
* newline in logging message (checkstyle)
* changed audit.md to say GET not GetObjectRequest
if yetus is happy will merge
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646693#comment-17646693
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran opened a new pull request, #5218:
URL: https://github.com/apache/hadoop/pull/5218
The start and end of the range is set in a new audit param "rg", e.g
"?rg=100-200"
Contributed by Ankit Saurabh
### How was this patch tested?
s3 london
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [X] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646661#comment-17646661
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1047224496
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -251,11 +251,10 @@ private void
attachRangeFromRequest(AmazonWebServiceRequest request) {
return;
}
if (rangeValue.length != 2) {
- WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
- + rangeValue.length + " elements. Ignoring");
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got {} elements. Ignoring.", rangeValue.length);
Review Comment:
ok, checkstyle is unhappy. needs splitting.
i will do that myself locally as i do a retest and merge.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646657#comment-17646657
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1348676320
yeah, looking at it. got distracted by the various crises...
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644086#comment-17644086
]
ASF GitHub Bot commented on HADOOP-18183:
-
dannycjones commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1340246393
hey @steveloughran - do you have time to give this another pass over the
next week?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637975#comment-17637975
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1325712422
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 15s | | trunk passed |
| +1 :green_heart: | compile | 26m 18s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 41s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 0s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 10s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 52s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 40s | | the patch passed |
| +1 :green_heart: | compile | 25m 31s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 25m 31s | | the patch passed |
| +1 :green_heart: | compile | 30m 53s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 30m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 59s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/10/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | mvnsite | 3m 5s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 1s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 49s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 7s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 262m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 52c3270ba72d 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f24228897b4a7c3ac87985ea46587c53738ce987 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637970#comment-17637970
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1325704121
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 39s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 17s | | trunk passed |
| +1 :green_heart: | compile | 25m 28s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 10s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 41s | | the patch passed |
| +1 :green_heart: | compile | 24m 46s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 24m 46s | | the patch passed |
| +1 :green_heart: | compile | 22m 12s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 19s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/9/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | mvnsite | 2m 53s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 48s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 37s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 32s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 0s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 250m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux fcc52656a409 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f24228897b4a7c3ac87985ea46587c53738ce987 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637966#comment-17637966
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1325694271
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 8s | | trunk passed |
| +1 :green_heart: | compile | 23m 35s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 20m 53s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 16s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 43s | | the patch passed |
| +1 :green_heart: | compile | 22m 37s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 22m 37s | | the patch passed |
| +1 :green_heart: | compile | 20m 53s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 1s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/11/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | mvnsite | 3m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 15s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 13s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 44s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 10s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 21s | | The patch does not
generate ASF License warnings. |
| | | 237m 32s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 05cfc293c5ce 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f24228897b4a7c3ac87985ea46587c53738ce987 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637959#comment-17637959
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1325671729
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 53s | | trunk passed |
| +1 :green_heart: | compile | 25m 40s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 16s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 8s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 51s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 42s | | the patch passed |
| +1 :green_heart: | compile | 24m 47s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 24m 47s | | the patch passed |
| +1 :green_heart: | compile | 22m 14s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/8/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | mvnsite | 2m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 51s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 51s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 40s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 1s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 250m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 7b50dac4e0cd 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2122eafcd622c30cc0dc8ceeccdd7793743a67e6 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637923#comment-17637923
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-132393
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 2s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 33m 24s | | trunk passed |
| +1 :green_heart: | compile | 31m 25s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 24m 47s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 12s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 18s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 13s | | the patch passed |
| +1 :green_heart: | compile | 30m 37s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 30m 37s | | the patch passed |
| +1 :green_heart: | compile | 25m 51s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 25m 51s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 42s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/7/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | mvnsite | 3m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 21s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 55s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 21m 0s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 44s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 24s | | The patch does not
generate ASF License warnings. |
| | | 283m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux c7f547021cba 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c17dc7729e0ec5c4f721baa12439365dfb6c5aad |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637916#comment-17637916
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1325486633
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 22s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 8s | | trunk passed |
| +1 :green_heart: | compile | 25m 45s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 26s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 7s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 51s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 4s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 37s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 44s | | the patch passed |
| +1 :green_heart: | compile | 30m 54s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 30m 54s | | the patch passed |
| +1 :green_heart: | compile | 22m 9s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 22s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/6/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | mvnsite | 2m 55s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 58s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 32s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 15s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 9s | | The patch does not
generate ASF License warnings. |
| | | 263m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 9ce692e89833 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 46d371cbe90436cc16f52f2da74ffda10dd8cb3a |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634396#comment-17634396
]
ASF GitHub Bot commented on HADOOP-18183:
-
dannycjones commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1022901285
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,26 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request.
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + " elements. Ignoring");
+ return;
+}
+String combinedRangeValue = String.format("bytes=%d-%d",
rangeValue[0], rangeValue[1]);
Review Comment:
Yes, I think that makes sense.
The only risk I see is if the object store uses a range specified in another
unit from bytes. In V1 SDK, it's hard coded to `bytes` - no issue there. In V2,
I believe you specify the header itself so we can warn once if we see something
other than `bytes=` prefix and ignore the header if so. We are the ones
specifying the header in the first place anyway, right? (FYI @ahmarsuhail
@passaro)
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634378#comment-17634378
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1022845937
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,26 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request.
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + " elements. Ignoring");
+ return;
+}
+String combinedRangeValue = String.format("bytes=%d-%d",
rangeValue[0], rangeValue[1]);
Review Comment:
sorry, i'd missed that. yes, you are right. just include it without the
bytes= stuff.
i guess we do have to consider what happens if/when s3 supports multiple
ranges, but that would be just something like "1-3,4-7", wouldn't it?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633643#comment-17633643
]
ASF GitHub Bot commented on HADOOP-18183:
-
dannycjones commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1021284548
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,26 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request.
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + " elements. Ignoring");
+ return;
+}
+String combinedRangeValue = String.format("bytes=%d-%d",
rangeValue[0], rangeValue[1]);
Review Comment:
@steveloughran, can you elaborate on the `rg` prefix? Is it for the query
parameter? If so, we already have that as `r` (`PARAM_RANGE`).
`bytes=` is part of the range header value which - following the SDK upgrade
- can just be dumped in place, without needing reconstruction like this.
That being said, it does seem a bit of a waste. Maybe we drop it entirely
here (and for all connectors) and just include whatever the value of the range
header was minus `bytes=` prefix.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632678#comment-17632678
]
ASF GitHub Bot commented on HADOOP-18183:
-
steveloughran commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1020764834
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,26 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request.
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + " elements. Ignoring");
+ return;
+}
+String combinedRangeValue = String.format("bytes=%d-%d",
rangeValue[0], rangeValue[1]);
Review Comment:
can you make a two byte prefix ("rg"?) and put in AuditConstants?
* shorter as we don't know the upper limit on referrer strings and would
like to keep it down
* in the file for other connectors to use, and to make it easy to look at
what to expect
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,26 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request.
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + " elements. Ignoring");
Review Comment:
nit: use the slf4j {} expansion for on demand expansion and resilience to
NPEs in toString(). not that important here, but good to keep to the same code
style everywhere
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630323#comment-17630323
]
ASF GitHub Bot commented on HADOOP-18183:
-
sauraank commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1306971560
Thanks @dannycjones ! Hey @steveloughran can you please review this PR?
Thank you!
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629856#comment-17629856
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1305718495
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 56s | | trunk passed |
| +1 :green_heart: | compile | 25m 38s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 9s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 0s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 48s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 42s | | the patch passed |
| +1 :green_heart: | compile | 24m 46s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 24m 46s | | the patch passed |
| +1 :green_heart: | compile | 22m 3s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 3s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 50s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 39s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 5s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 9s | | The patch does not
generate ASF License warnings. |
| | | 249m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 0d6757a82444 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / deddad4394ac39c818e6ce3c1a82b35080071afd |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/5/testReport/ |
| Max. process+thread count | 1248 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629795#comment-17629795
]
ASF GitHub Bot commented on HADOOP-18183:
-
sauraank commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1305551465
Thanks @dannycjones @ashutoshcipher for the review. I have fixed the
checkstyle recommendation. I have also added the integration test result for
hadoop-aws.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629752#comment-17629752
]
ASF GitHub Bot commented on HADOOP-18183:
-
dannycjones commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1305440067
>Please fix the style check
Oops, didn't realise this doesn't fail CI. Thanks for calling that out,
Ashutosh.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629174#comment-17629174
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1304062597
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 37s | | trunk passed |
| +1 :green_heart: | compile | 23m 24s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 20m 44s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 10s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 16s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 12s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 45s | | the patch passed |
| +1 :green_heart: | compile | 22m 38s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 22m 38s | | the patch passed |
| +1 :green_heart: | compile | 20m 39s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 9s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/4/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) |
| +1 :green_heart: | mvnsite | 3m 5s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 8s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 15s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 42s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 7s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
| | | 236m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux db12b8f0438b 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 732fd000f69f3d241a41a92433f9beaf2fbe7151 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/4/testReport/ |
| Max. process
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629127#comment-17629127
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1303936275
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 39s | | trunk passed |
| +1 :green_heart: | compile | 28m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 24m 10s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 10s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 3s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 39s | | the patch passed |
| +1 :green_heart: | compile | 24m 44s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 24m 44s | | the patch passed |
| +1 :green_heart: | compile | 21m 58s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 58s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 17s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/3/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) |
| +1 :green_heart: | mvnsite | 2m 54s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 34s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 2s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 259m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7e8991217d06 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 620059c814fc910af75f27d4a2870b6d8e3e656e |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/3/testReport/ |
| Max. process
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629072#comment-17629072
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1303823879
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 19s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 32s | | trunk passed |
| +1 :green_heart: | compile | 25m 36s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 2s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 0s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 6s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 48s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 39s | | the patch passed |
| +1 :green_heart: | compile | 24m 44s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 24m 44s | | the patch passed |
| +1 :green_heart: | compile | 22m 8s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) |
| +1 :green_heart: | mvnsite | 2m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 59s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 49s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 37s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 31s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 33s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 4s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not
generate ASF License warnings. |
| | | 248m 14s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e8de1c61d80d 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2be27c95a1d7111acd61750d7bb755e88f4eba69 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/2/testReport/ |
| Max. process
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629053#comment-17629053
]
ASF GitHub Bot commented on HADOOP-18183:
-
hadoop-yetus commented on PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#issuecomment-1303779631
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 48s | | trunk passed |
| +1 :green_heart: | compile | 25m 21s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 5s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 54s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 23s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 39s | | the patch passed |
| +1 :green_heart: | compile | 27m 37s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 27m 37s | | the patch passed |
| +1 :green_heart: | compile | 25m 32s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 25m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 43s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) |
| +1 :green_heart: | mvnsite | 3m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 13s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 21s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 0s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 260m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 674f516116b7 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6837be9fc5ae7b4001fe782a1e24cee82a17a53c |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5110/1/testReport/ |
| Max. process
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628918#comment-17628918
]
ASF GitHub Bot commented on HADOOP-18183:
-
passaro commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1013926432
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,25 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + ". Ignoring");
+}
+String combinedRangeValue = String.format("bytes=%d-%d",
rangeValue[0], rangeValue[1]);
Review Comment:
Edit: Duplicate of Danny's comment.
These 2 lines should only be executed when `length == 2`.
Maybe you're missing a `return` in the `if` branch above?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628914#comment-17628914
]
ASF GitHub Bot commented on HADOOP-18183:
-
passaro commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1013926432
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,25 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + ". Ignoring");
+}
+String combinedRangeValue = String.format("bytes=%d-%d",
rangeValue[0], rangeValue[1]);
Review Comment:
These 2 lines should only be executed when `length == 2`.
Maybe you're missing a `return` in the `if` branch above?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628913#comment-17628913
]
ASF GitHub Bot commented on HADOOP-18183:
-
dannycjones commented on code in PR #5110:
URL: https://github.com/apache/hadoop/pull/5110#discussion_r1013919563
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/LoggingAuditor.java:
##
@@ -230,6 +240,25 @@ private class LoggingAuditSpan extends
AbstractAuditSpanImpl {
private final HttpReferrerAuditHeader referrer;
+/**
+ * Attach Range of data for GetObject Request
+ * @param request given get object request
+ */
+private void attachRangeFromRequest(AmazonWebServiceRequest request) {
+ if (request instanceof GetObjectRequest) {
+long[] rangeValue = ((GetObjectRequest) request).getRange();
+if (rangeValue == null || rangeValue.length == 0) {
+ return;
+}
+if (rangeValue.length != 2) {
+ WARN_INCORRECT_RANGE.warn("Expected range to contain 0 or 2
elements. Got "
+ + rangeValue.length + ". Ignoring");
+}
Review Comment:
We don't actually ignore the value here. We should return or put the next
few lines in an `else` block.
We'd get an
[IndexOutOfBoundsException](https://docs.oracle.com/javase/8/docs/api/java/lang/IndexOutOfBoundsException.html)
when we access `rangeValue[1]` when there's only one element, for example.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
> Labels: pull-request-available
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628894#comment-17628894
]
ASF GitHub Bot commented on HADOOP-18183:
-
sauraank opened a new pull request, #5110:
URL: https://github.com/apache/hadoop/pull/5110
### Description of PR
Added the Range in the referer header for GetObjectRequest. It is of the
format "bytes=%d-%d".
### How was this patch tested?
Added the unit tests for it. Tested by running the unit tests and
integration test on hadoop-aws successfully.
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619658#comment-17619658
]
Alessandro Passaro commented on HADOOP-18183:
-
+1 for adding the whole range header. This is also how the AWS SDK v2 handles
ranges (rather than explicitly exposing start & end), so it will simplify the
upgrade.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Ankit Saurabh
>Priority: Minor
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607665#comment-17607665
]
Steve Loughran commented on HADOOP-18183:
-
of the two-path operations, copy is the only one which exists, and that is all
done in the xfer manager. even if we did do it ourselves, it is only in the
source path where the range is read, isn't it?
so how about the p1rs and p1re are used, for now...if ever we needed more than
one, then that could go in later using the same scheme.
Using short names is important as we still don't know what that range limit in
the log entries areldl
actually, maybe we should have the range header in the audit log, rather than
start and end? that way if multiple ranges in a GET are supported, vectorIO can
take advantage of it and there is no need to make any changes in the auditing.
similarly, a range like "100-" to the EOF should be recordable.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Priority: Minor
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603028#comment-17603028
]
Daniel Carl Jones commented on HADOOP-18183:
We may need to consider the use cases that have two paths. Do both need ranges
associated? - probably not, not sure though. But maybe we ought to be clear
that the range start & end belong to path 1. Should there be something like
"p1rs","p2rs"?
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Priority: Minor
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header
[
https://issues.apache.org/jira/browse/HADOOP-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603027#comment-17603027
]
Daniel Carl Jones commented on HADOOP-18183:
We may be able to avoid modifying {{buildHttpReferrer()}} since we have
{{set(k, v)}} public.
> s3a audit logs to publish range start/end of GET requests in audit header
> -
>
> Key: HADOOP-18183
> URL: https://issues.apache.org/jira/browse/HADOOP-18183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Priority: Minor
>
> we don't get the range of ranged get requests in s3 server logs, because the
> AWS s3 log doesn't record that information. we can see it's a partial get
> from the 206 response, but the length of data retrieved is lost.
> LoggingAuditor.beforeExecution() would need to recognise a ranged GET and
> determine the extra key-val pairs for range start and end (rs & re?)
> we might need to modify {{HttpReferrerAuditHeader.buildHttpReferrer()}} to
> take a map of so it can dynamically create a header for each
> request; currently that is not in there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
