Thanks Joe, I was about to request a feature like this (ListS3Objects). From: Joe Skora <jsk...@gmail.com<mailto:jsk...@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Thursday, October 8, 2015 at 11:39 AM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: FetchS3Object processor
There is already a ticket (NIFI-840<https://issues.apache.org/jira/browse/NIFI-840>) in the hopper to create a ListS3Objects processor that can track bucket contents and trigger FetchS3Object. In the meantime, if you want it triggered automatically (as opposed to generate flow file) you may be able to accomplish something similar with a combination of HTTP and XML processors. On Thu, Oct 8, 2015 at 2:32 PM, Chakrader Dewaragatla <chakrader.dewaraga...@lifelock.com<mailto:chakrader.dewaraga...@lifelock.com>> wrote: Thanks Aldrin. GenerateFlowFile helped to trigger. Thanks, -Chakri From: Aldrin Piri <aldrinp...@gmail.com<mailto:aldrinp...@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Thursday, October 8, 2015 at 11:22 AM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: FetchS3Object processor Chakri, The FetchS3Object processor does not act as a source directly and is instead driven by incoming FlowFiles. The use case was that one could utilize Expression Language to interact with buckets and paths in a dynamic fashion. Alternatively, if you are looking to just grab a specific file, you can use GenerateFlowFile to "trigger" the FetchS3Object with its static configuration. On Thu, Oct 8, 2015 at 1:45 PM, Chakrader Dewaragatla <chakrader.dewaraga...@lifelock.com<mailto:chakrader.dewaraga...@lifelock.com>> wrote: Thanks Joe. I have S3fetch and putfile processors setup, files are not fetching despite files exist on s3. Am I missing anything ? Does S3fetch processor need any additional processor to feed data ? Thanks, -Chakri From: Joe Skora <jsk...@gmail.com<mailto:jsk...@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Thursday, October 8, 2015 at 3:38 AM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: FetchS3Object processor Chakri, The Amazon docs explain here<http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html> that S3 doesn't support pathes within a bucket, but you can embed path-like naming into the "Object Key" parameter to FetchS3Object. For your example, you would use bucket="s3://<bucket>" and key="<path>/<objectfile>" to incorporate the path as you store the object. I hope that helps. Regards, Joe On Thu, Oct 8, 2015 at 2:39 AM, Chakrader Dewaragatla <chakrader.dewaraga...@lifelock.com<mailto:chakrader.dewaraga...@lifelock.com>> wrote: Nifi users - FetchS3Object processor do not have option to specify Object file path, how do you set that ? Eg : s3://<bucket>/<path>/<objectfile> Thanks, -Chakri ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________
