[ https://issues.apache.org/jira/browse/NIFI-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15156281#comment-15156281 ]
ASF subversion and git services commented on NIFI-1542: ------------------------------------------------------- Commit 4c5263220c5b45ca82a2d5119311f4a481d9bbba in nifi's branch refs/heads/master from [~jskora] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4c52632 ] NIFI-1542: Update how PutS3Object handles a 403 AccessDenied response to a S3 ListMultipartUploads request. * Change log message from error to warning and added note about S3 permissions. * Advance the ageoff check time by the check interval as if the request succeeded to prevent re-checking on every upload when permission does not exist. Reviewed by Tony Kurc (tk...@apache.org) (with minor amendments for whitespace and error message consistency). This closes #240 > PutS3Object Processor Logs Errors Without ListBucketMultipartUploads > Permission > ------------------------------------------------------------------------------- > > Key: NIFI-1542 > URL: https://issues.apache.org/jira/browse/NIFI-1542 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Affects Versions: 0.5.0 > Environment: Linux, Chrome > Reporter: James Wing > Assignee: Joe Skora > Priority: Minor > Fix For: 0.5.1 > > Attachments: error-checking-s3-multipart-upload-list.png > > > In 0.5.0, I am seeing an error putting objects to Amazon S3 using the > PutS3Object processor. The *put operation itself succeeds*, but a nasty-gram > is displayed in the flow and logged as a side effect. The error text is > "Error checking S3 Multipart Upload list for <my s3 bucket>", from line 698 > of PutS3Object.java. This code was introduced in NIFI-1107, expanding the > PutS3Object processor with support for S3 multi-part uploads of larger files. > Logged in nifi-app.log: > {code} > 2016-02-19 20:56:24,312 ERROR [Timer-Driven Process Thread-3] > o.a.nifi.processors.aws.s3.PutS3Object PutS > 3Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Error checking S3 Multipart > Upload list for batchiq-scra > tch: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: > AccessDenied; Request ID: 7483E76EA > 52E33C4) > 2016-02-19 20:56:24,533 INFO [Timer-Driven Process Thread-3] > o.a.nifi.processors.aws.s3.PutS3Object PutS3 > Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Successfully put > StandardFlowFileRecord[uuid=a4cc19f5-cd2 > 4-4be7-8157-21f2609fa8cd,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1455912275117-1, container=default, > section=1], offset=3072, > length=1024],offset=0,name=273670477119454,size=1024] to Amazon S3 in 327 > milliseconds > {code} > The error-but-success behavior appears to happen because the exception is > caught, logged, but not re-thrown. The approximate code path is as follows: > # PutS3Object::onTrigger method calls ageoffS3Uploads regardless of > single-part/multi-part code path > # ageoffS3Uploads calls getS3AgeoffListAndAgeoffLocalState > # getS3AgeoffListAndAgeoffLocalState calls s3.listMultipartUploads > # After the exception, onTrigger proceeds with the single-part upload path > *Troubleshooting Notes* > Listing multi-part S3 uploads requires an additional S3 permission, > s3:ListBucketMultipartUploads. I would expect the PutS3Object processor to > only require the s3:PutObject permission for single-part uploads. > I tested that this behavior did not occur in NiFi 0.4.2. I also tested that > adding the s3:ListBucketMultipartUploads permission to my AWS credential > circumvents the error messages from the S3PutObject processor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)