EnricoMi commented on PR #8306:
URL: https://github.com/apache/hadoop/pull/8306#issuecomment-4223930708
Adding this to `hadoop-tools/hadoop-aws/src/test/resources/log4j.properties`
```
log4j.logger.software.amazon.awssdk.thirdparty.org.apache.http.headers=DEBUG
```
shows request headers:
```
2026-04-10 10:43:27,384 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(133)) -
http-outgoing-0 >> GET
/vast-bucket?list-type=2&delimiter=%2F&max-keys=2&prefix=test%2F HTTP/1.1
2026-04-10 10:43:27,384 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> Host: vast.company.org
2026-04-10 10:43:27,384 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> amz-sdk-request: attempt=1; max=2
2026-04-10 10:43:27,385 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> User-Agent: Hadoop 3.6.0-SNAPSHOT aws-sdk-java/2.35.4
md/io#sync md/http#Apache ua/2.1 api/S3#2.35.x os/Linux#5.15.0-91-generic
lang/java#17.0.18 md/OpenJDK_64-Bit_Server_VM#17.0.18+8-Ubuntu-122.04.1
md/vendor#Ubuntu md/en_GB m/F,N,N,G hll/cross-region
2026-04-10 10:43:27,385 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> x-amz-content-sha256: UNSIGNED-PAYLOAD
2026-04-10 10:43:27,385 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> X-Amz-Date: 20260410T094327Z
2026-04-10 10:43:27,385 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> x-amz-delete-contents: true
2026-04-10 10:43:27,385 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onRequestSubmitted(136)) -
http-outgoing-0 >> Connection: Keep-Alive
2026-04-10 10:43:28,389 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onResponseReceived(122)) -
http-outgoing-0 << HTTP/1.1 400 Bad Request
2026-04-10 10:43:28,390 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onResponseReceived(125)) -
http-outgoing-0 << x-amz-bucket-region: vast
2026-04-10 10:43:28,390 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onResponseReceived(125)) -
http-outgoing-0 << Content-Type: application/xml
2026-04-10 10:43:28,390 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onResponseReceived(125)) -
http-outgoing-0 << Transfer-Encoding: chunked
2026-04-10 10:43:28,391 [setup] DEBUG http.headers
(LoggingManagedHttpClientConnection.java:onResponseReceived(125)) -
http-outgoing-0 << Server: vast 5.4.3.1
2026-04-10 10:43:29,402 [setup] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(474)) - List vast-bucket:/test/ delimiter=/
keys=2 requester pays=null ;
software.amazon.awssdk.services.s3.model.S3Exception: Invalid Argument
(Service: S3, Status Code: 400, Request ID: 30d1310001aac2, Extended Request
ID: 30d1310001aac2) (SDK Attempt Count: 1),
2026-04-10 10:43:29,402 [setup] DEBUG s3a.S3ARetryPolicy
(S3ARetryPolicy.java:shouldRetry(318)) - Retry probe for AWSBadRequestException
with 0 retries and 0 failovers, idempotent=true, due to
org.apache.hadoop.fs.s3a.AWSBadRequestException: List
playpen-enricmin222:/test/ delimiter=/ keys=2 requester pays=null on /:
software.amazon.awssdk.services.s3.model.S3Exception: Invalid Argument
(Service: S3, Status Code: 400, Request ID: 30d1310001aac2, Extended Request
ID: 30d1310001aac2) (SDK Attempt Count: 1):InvalidArgument: Invalid Argument
(Service: S3, Status Code: 400, Request ID: 30d1310001aac2, Extended Request
ID: 30d1310001aac2) (SDK Attempt Count: 1)
org.apache.hadoop.fs.s3a.AWSBadRequestException: List vast-bucket:/test/
delimiter=/ keys=2 requester pays=null on /:
software.amazon.awssdk.services.s3.model.S3Exception: Invalid Argument
(Service: S3, Status Code: 400, Request ID: 30d1310001aac2, Extended Request
ID: 30d1310001aac2) (SDK Attempt Count: 1):InvalidArgument: Invalid Argument
(Service: S3, Status Code: 400, Request ID: 30d1310001aac2, Extended Request
ID: 30d1310001aac2) (SDK Attempt Count: 1)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:271)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:481)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.listObjects(S3AFileSystem.java:3106)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4096)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3987)
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
