adoroszlai opened a new pull request, #10097:
URL: https://github.com/apache/ozone/pull/10097

   ## What changes were proposed in this pull request?
   
   Audit log shows only the last two elements of stack traces.  For several 
failed S3 Gateway requests it is always the same:
   
   ```
   ... | ERROR | S3GAudit | ... | op=GET_BUCKET ... | ret=FAILURE | 
org.apache.hadoop.ozone.s3.exception.OS3Exception
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:192)
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:179)
   ```
   
   due to two levels of `newError` calls.
   
   This PR changes code to avod chained `newError` calls by creating 
`OS3Exception` in various `newError` implementations.  Related changes to avoid 
duplication:
   
   - Simplify `OS3Exception` creation:
     - pass error code (`S3ErrorTable`) instead of separate variables
     - also pass cause exception and `resource`
     - remove unused constructors
     - log from constructor instead of `newError`
   - Create few more variants of `newError` to reduce `null` arguments in 
callers.
   - Move `BucketEndpoint#translateException` to `S3ErrorTable` (renamed to 
`translateResultCode`), add more cases to `switch`.  Change `ObjectEndpoint` to 
rely on result code translation in `newError`.
   
   Related minor fix:
   
   - Deleting non-existent keys returns success response (204), not an error.  
However, the audit log shows `ERROR` instead of `SUCCESS`, because `NOT_FOUND` 
error is handled in the outer `delete` method, not `handleDeleteRequest`.  This 
is now moved to let audit log reflect the correct result.
   
   https://issues.apache.org/jira/browse/HDDS-15000
   
   ## How was this patch tested?
   
   Before:
   
   ```
   | ERROR | S3GAudit | user=dlfknslnfslf | ip=172.18.0.6 | op=GET_BUCKET 
{"bucket":"[bucket-etqvtcowkf]","list-type":"[2]","max-keys":"[-1]","encoding-type":"[url]","x-amz-request-id":"1f772aef-ed39-41a6-b470-5fff0962422e","x-amz-id-2":"bn6THRxxaQso"}
 | ret=FAILURE | org.apache.hadoop.ozone.s3.exception.OS3Exception
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:192)
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:179)
   ```
   
   After:
   
   ```
   | ERROR | S3GAudit | user=dlfknslnfslf | ip=172.18.0.6 | op=GET_BUCKET 
{"bucket":"[bucket-mpqrnkuavm]","list-type":"[2]","max-keys":"[-1]","encoding-type":"[url]","x-amz-request-id":"28a04087-212e-4e82-bc3b-60976b690f86","x-amz-id-2":"0mVf1nwDd"}
 | ret=FAILURE | org.apache.hadoop.ozone.s3.exception.OS3Exception: Invalid 
Argument
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:261)
        at 
org.apache.hadoop.ozone.s3.endpoint.BucketEndpoint.validateMaxKeys(BucketEndpoint.java:257)
   ```
   
   Before:
   
   ```
   | ERROR | S3GAudit | user=dlfknslnfslf | ip=172.18.0.6 | 
op=INIT_MULTIPART_UPLOAD 
{"bucket":"[bucket-etqvtcowkf]","path":"[ozone-test-6052266241/mpuWithLargeMetadata]","x-amz-request-id":"0e1c28c9-3451-498b-933e-2b668991216f","x-amz-id-2":"5kEMzckEG19","uploads":"[]"}
 | ret=FAILURE | org.apache.hadoop.ozone.s3.exception.OS3Exception
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:192)
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:179)
   ```
   
   After:
   
   ```
   | ERROR | S3GAudit | user=dlfknslnfslf | ip=172.18.0.6 | 
op=INIT_MULTIPART_UPLOAD 
{"bucket":"[bucket-mpqrnkuavm]","path":"[ozone-test-5385731147/mpuWithLargeMetadata]","x-amz-request-id":"41329a9a-7ab3-4fcb-8f41-06e130631a2e","x-amz-id-2":"S6IY69aIGOpGP","uploads":"[]"}
 | ret=FAILURE | org.apache.hadoop.ozone.s3.exception.OS3Exception: Illegal 
user defined metadata. Combined size exceeds the maximum allowed metadata size 
of 2KB
        at 
org.apache.hadoop.ozone.s3.exception.S3ErrorTable.newError(S3ErrorTable.java:261)
        at 
org.apache.hadoop.ozone.s3.endpoint.EndpointBase.getCustomMetadataFromHeaders(EndpointBase.java:315)
   ```
   
   Before:
   
   ```
   | ERROR | S3GAudit | user=dlfknslnfslf | ip=172.18.0.6 | op=DELETE_KEY 
{"bucket":"[bucket-etqvtcowkf]","path":"[thereisnosuchfile]",...} | ret=FAILURE 
| KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Key not found
        at 
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:766)
        at 
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.deleteKey(OzoneManagerProtocolClientSideTranslatorPB.java:965)
   ```
   
   After:
   
   ```
   | INFO  | S3GAudit | user=dlfknslnfslf | ip=172.18.0.6 | op=DELETE_KEY 
{"bucket":"[bucket-mpqrnkuavm]","path":"[thereisnosuchfile]",...} | ret=SUCCESS 
|  
   ```
   
   https://github.com/adoroszlai/ozone/actions/runs/24665673476


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to