Re: PutS3Object error

[James Wing]

You should not need to modify your hosts file.  Can you try using the AWS
CLI tools to list the bucket from the same machine running NiFi?  Something
like:

aws s3 ls s3://nifibucket/

If that is working, you should have all the required network connectivity.
Also, would you please describe any other property configuration you are
using on PutS3Object like endpoint override, proxy configuration, anything
like that?

Thanks,

James

On Tue, Nov 1, 2016 at 2:48 PM, John Burns  wrote:

> Hi James
>
> Actually I have already configured it as you mention and the error
> persists.
>
> The key and secret key are set correctly too. Do I need to modify my
> /etc/hosts perhaps?
>
> Thanks
>
> John
>
> On Nov 1, 2016 20:53, "James Wing"  wrote:
>
>> John,
>>
>> When you configure the PutS3Object processor, you should configure the
>> Bucket Name as just 'nifibucket' (without quotes), and set the Region to
>> 'eu-west-1'.  You do not need the long-form '
>> nifibucket.s3-eu-west-1.amazonaws.com', I think that might be what is
>> confusing it.
>>
>> Thanks,
>>
>> James
>>
>> On Tue, Nov 1, 2016 at 1:31 PM, John Burns  wrote:
>>
>>> Hi,
>>>
>>> I am getting a strange error when writing a png to a public S3 bucket
>>> nifibucket.s3-eu-west-1.amazonaws.com: It seems to be an
>>> UnknownHostException but I can access the bucket XML fine in the browser.
>>>
>>> The processor flow is simply GetFile -> PutS3Object
>>>
>>> Any info welcome.
>>>
>>> Thanks
>>>
>>> John
>>>
>>>
>>> com.amazonaws.http.AmazonHttpClient Unable to execute HTTP request:
>>> nifibucket.s3-eu-west-1.amazonaws.com: unknown error
>>> java.net.UnknownHostException: nifibucket.s3-eu-west-1.amazonaws.com:
>>> unknown error
>>> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
>>> ~[na:1.8.0_77]
>>> at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
>>> ~[na:1.8.0_77]
>>> at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
>>> ~[na:1.8.0_77]
>>> at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
>>> ~[na:1.8.0_77]
>>> at java.net.InetAddress.getAllByName(InetAddress.java:1192)
>>> ~[na:1.8.0_77]
>>> at java.net.InetAddress.getAllByName(InetAddress.java:1126)
>>> ~[na:1.8.0_77]
>>> at 
>>> com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27)
>>> ~[aws-java-sdk-core-1.11.8.jar:na]
>>> at 
>>> com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38)
>>> ~[aws-java-sdk-core-1.11.8.jar:na]
>>> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperato
>>> r.connect(DefaultHttpClientConnectionOperator.java:111)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>>> .connect(PoolingHttpClientConnectionManager.java:353)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at sun.reflect.GeneratedMethodAccessor319.invoke(Unknown Source)
>>> ~[na:na]
>>> at 
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> ~[na:1.8.0_77]
>>> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
>>> at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handl
>>> er.invoke(ClientConnectionManagerFactory.java:76)
>>> ~[aws-java-sdk-core-1.11.8.jar:na]
>>> at com.amazonaws.http.conn.$Proxy121.connect(Unknown Source) ~[na:na]
>>> at 
>>> org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at 
>>> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at 
>>> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at 
>>> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at 
>>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at 
>>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
>>> ~[httpclient-4.4.1.jar:4.4.1]
>>> at 
>>> com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
>>> ~[aws-java-sdk-core-1.11.8.jar:na]
>>> at 
>>> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:852)
>>> [aws-java-sdk-core-1.11.8.jar:na]
>>> at 
>>> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:695)
>>> [aws-java-sdk-core-1.11.8.jar:na]
>>> at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:447)
>>> [aws-java-sdk-core-1.11.8.jar:na]
>>> at 
>>> com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:409)
>>> [aws-java-sdk-core-1.11.8.jar:na]
>>> at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:358)
>>> [aws-java-sdk-core-1.11.8.jar:na]
>>> at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3787)
>>> 

Re: PutS3Object error

[James Wing]

John,

When you configure the PutS3Object processor, you should configure the
Bucket Name as just 'nifibucket' (without quotes), and set the Region to
'eu-west-1'.  You do not need the long-form '
nifibucket.s3-eu-west-1.amazonaws.com', I think that might be what is
confusing it.

Thanks,

James

On Tue, Nov 1, 2016 at 1:31 PM, John Burns  wrote:

> Hi,
>
> I am getting a strange error when writing a png to a public S3 bucket
> nifibucket.s3-eu-west-1.amazonaws.com: It seems to be an
> UnknownHostException but I can access the bucket XML fine in the browser.
>
> The processor flow is simply GetFile -> PutS3Object
>
> Any info welcome.
>
> Thanks
>
> John
>
>
> com.amazonaws.http.AmazonHttpClient Unable to execute HTTP request:
> nifibucket.s3-eu-west-1.amazonaws.com: unknown error
> java.net.UnknownHostException: nifibucket.s3-eu-west-1.amazonaws.com:
> unknown error
> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> ~[na:1.8.0_77]
> at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
> ~[na:1.8.0_77]
> at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
> ~[na:1.8.0_77]
> at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
> ~[na:1.8.0_77]
> at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[na:1.8.0_77]
> at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[na:1.8.0_77]
> at com.amazonaws.SystemDefaultDnsResolver.resolve(
> SystemDefaultDnsResolver.java:27) ~[aws-java-sdk-core-1.11.8.jar:na]
> at 
> com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38)
> ~[aws-java-sdk-core-1.11.8.jar:na]
> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(
> DefaultHttpClientConnectionOperator.java:111)
> ~[httpclient-4.4.1.jar:4.4.1]
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(
> PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.4.1.jar:4.4.1]
> at sun.reflect.GeneratedMethodAccessor319.invoke(Unknown Source) ~[na:na]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
> at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(
> ClientConnectionManagerFactory.java:76) ~[aws-java-sdk-core-1.11.8.jar:na]
> at com.amazonaws.http.conn.$Proxy121.connect(Unknown Source) ~[na:na]
> at org.apache.http.impl.execchain.MainClientExec.
> establishRoute(MainClientExec.java:380) ~[httpclient-4.4.1.jar:4.4.1]
> at org.apache.http.impl.execchain.MainClientExec.
> execute(MainClientExec.java:236) ~[httpclient-4.4.1.jar:4.4.1]
> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
> ~[httpclient-4.4.1.jar:4.4.1]
> at 
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
> ~[httpclient-4.4.1.jar:4.4.1]
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> ~[httpclient-4.4.1.jar:4.4.1]
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> ~[httpclient-4.4.1.jar:4.4.1]
> at 
> com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
> ~[aws-java-sdk-core-1.11.8.jar:na]
> at 
> com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:852)
> [aws-java-sdk-core-1.11.8.jar:na]
> at 
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:695)
> [aws-java-sdk-core-1.11.8.jar:na]
> at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:447)
> [aws-java-sdk-core-1.11.8.jar:na]
> at 
> com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:409)
> [aws-java-sdk-core-1.11.8.jar:na]
> at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:358)
> [aws-java-sdk-core-1.11.8.jar:na]
> at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3787)
> [aws-java-sdk-s3-1.11.8.jar:na]
> at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3729)
> [aws-java-sdk-s3-1.11.8.jar:na]
> at 
> com.amazonaws.services.s3.AmazonS3Client.listMultipartUploads(AmazonS3Client.java:2727)
> [aws-java-sdk-s3-1.11.8.jar:na]
> at org.apache.nifi.processors.aws.s3.PutS3Object.
> getS3AgeoffListAndAgeoffLocalState(PutS3Object.java:715)
> [nifi-aws-processors-1.0.0.jar:1.0.0]
> at org.apache.nifi.processors.aws.s3.PutS3Object.
> ageoffS3Uploads(PutS3Object.java:698) [nifi-aws-processors-1.0.0.
> jar:1.0.0]
> at 
> org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:395)
> [nifi-aws-processors-1.0.0.jar:1.0.0]
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-1.0.0.jar:1.0.0]
> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(
> StandardProcessorNode.java:1064) [nifi-framework-core-1.0.0.jar:1.0.0]
> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(
> 

Re: Does NiFi ConsumeJMS Processor supports OpenJMS?

[Oleg Zhurakousky]

Nirmal

Did some digging and the NPE is due to the fact that we are invoking a default 
constructor for whatever connection factory class is provided (i.e., 
‘org.exolab.jms.client.JmsConnectionFactory’). And that works for most major 
JMS providers, however for OpenJMS after looking at the code it is rather clear 
that the default constructor that is exposed was not exposed to be actually 
used by the developers but rather internal serialization use.

/**
 * Default constructor required for serialization
 */
public JmsConnectionFactory() {
}

Not sure why it was done this way . . ., but the NPE is due to the fact that 
the server proxy class is null so when it attempts to do 
Class.forName(proxyClassName) in getProxy() method it fails with NPE.

One other thing I noticed is that all OpenJMS examples are based on obtaining 
ConnectionFactory from JNDI for which we currently do not have support.

Once again, please raise the JIRA if you believe that it’s important to add 
such feature.

Cheers
Oleg
On Nov 1, 2016, at 9:29 AM, Oleg Zhurakousky 
> wrote:

Nirmal

While ConsumeJMS was developed and tested with the idea of supporting multiple 
providers, ‘openjms’ was not the one that it was tested with.
I will look at the error and will follow up, but the fact that it already shows 
NPE means we have a bug somewhere, so please raise the JIRA 
(https://issues.apache.org/jira/browse/NIFI) if you don’t mind or let m know 
and I’ll do it.

Thank you for reporting it.
Cheers
Oleg


On Nov 1, 2016, at 8:35 AM, Nirmal Kumar 
> wrote:

Hi All,

I am trying to read messages from openjms-0.7.7 using the ConsumeJMS Processor 
but getting following exception:

2016-11-01 14:43:17,260 ERROR [Timer-Driven Process Thread-4] 
o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - 
JMSConsumer[destination:queue1; pub-sub:false;] ConsumeJMS - 
JMSConsumer[destination:queue1; pub-sub:false;] failed to process session due 
to org.springframework.jms.UncategorizedJmsException: Uncategorized exception 
occured during JMS processing; nested exception is javax.jms.JMSException: 
Failed to create proxy: java.lang.NullPointerException: 
org.springframework.jms.UncategorizedJmsException: Uncategorized exception 
occured during JMS processing; nested exception is javax.jms.JMSException: 
Failed to create proxy: java.lang.NullPointerException
2016-11-01 14:43:17,265 ERROR [Timer-Driven Process Thread-4] 
o.apache.nifi.jms.processors.ConsumeJMS
org.springframework.jms.UncategorizedJmsException: Uncategorized exception 
occured during JMS processing; nested exception is javax.jms.JMSException: 
Failed to create proxy: java.lang.NullPointerException
at 
org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
 ~[na:na]
at 
org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:169)
 ~[na:na]
at 
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:497) ~[na:na]
at 
org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:764) 
~[na:na]
at 
org.springframework.jms.core.JmsTemplate.receive(JmsTemplate.java:738) ~[na:na]
at 
org.springframework.jms.core.JmsTemplate.receive(JmsTemplate.java:727) ~[na:na]
at 
org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) ~[na:na]
at 
org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:79) 
~[na:na]
at 
org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136)
 ~[na:na]
at 
org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:50) ~[na:na]
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
 ~[nifi-api-1.0.0.jar:1.0.0]
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064)
 ~[nifi-framework-core-1.0.0.jar:1.0.0]
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
 [nifi-framework-core-1.0.0.jar:1.0.0]
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
 [nifi-framework-core-1.0.0.jar:1.0.0]
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
 [nifi-framework-core-1.0.0.jar:1.0.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
[na:1.8.0_45]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 [na:1.8.0_45]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 

Re: Processors on the fly for many sensor devices

[Andrew Psaltis]

Good morning Davy,
I have all (barring tests that fail) code in place to fulfill NIFI-2615
[1]. I plan on wrapping up the testing of it today and submitting a P/R.
One thing to call out is that at present it is it's own module. However, I
plan on adding it to the NiFi Standard Processor bundle before submitting
the P/R. Let me know if you want me to add in the processing for the
demarcator splitting.

Thanks,
Andrew

On Mon, Oct 31, 2016 at 4:37 PM, Andrew Psaltis 
wrote:

> Hi Davy,
> Thanks for the P/R - I merged that in just a little bit ago.
>
> I think the feature you are describing makes total sense, I'm sure this
> would be useful in many different use cases. I am in the process of
> wrapping up the changes to fulfill JIRA NIFI-2615 [1]. Do you want to add
> the "delimiter based spliter" feature to that JIRA? Are you planning on
> implementing it or are you asking if I would?
>
> I should have the updated bits up in GH this evening/very early am.
>
> Thanks your kinda words regarding our loss.
>
> Thanks,
> Andrew
>
> On Sun, Oct 30, 2016 at 7:26 PM, Davy De Waele  wrote:
>
>> Hi Andrew,
>>
>> I've looked at your code and submitted a pull request as I needed to do
>> some code fixes to get it working
>> Also posted an issue / question on how you see this GetTCP processor
>> evolve.
>>
>> We're using Nifi to capture sensor data over TCP so a good GetTCP is
>> definitely something we need in our Nifi tool-belt
>>
>> We would like to use it in a slightly different manner as per the current
>> implementation. We would like to be able to specify a delimiter (byte
>> sequence) in order to generate flow files out of this processor.
>> Our sensor devices use a delimiter (end of transmission byte) to mark the
>> end of a message in the tcp stream.
>>
>> That way, the GetTCP processor would already "split" the tcp stream into
>> separate FlowFiles where each FlowFile would correspond to a response
>> received from the tcp stream. I've seen something similar in the ListenTCP
>> processor.
>>
>> Sorry to hear about your loss. My condolences.
>>
>> Regards,
>> Davy
>>
>>
>>
>> On Tue, Oct 18, 2016 at 10:16 AM, Andrew Psaltis <
>> psaltis.and...@gmail.com> wrote:
>>
>>> Davey,
>>> Sorry for the delay in getting this done, I was away from the keyboard
>>> for almost 1.5 weeks after a death in the family. I have moved the GetTCP
>>> processor code to it's own bundle and separate repo [1] with NiFi 1.0.0 as
>>> the target. I'm not sure if that is the long term home for it. However, it
>>> does feel a little cleaner then mucking with the NiFi Standard Bundle or me
>>> making the assumption it would be adopted as part of the standard bundle.
>>>
>>> I know of at least one bug in the processor that has been reported -- it
>>> fails to reconnect if the connection is severed. I will be working on this,
>>> however, I will not be able to get to it till Wednesday night or most
>>> likely Thursday morning.
>>>
>>> Thanks again for your patience and interest.
>>>
>>> Andrew
>>>
>>> [1] https://github.com/apsaltis/nifi-gettcp-bundle
>>>
>>> On Mon, Oct 3, 2016 at 10:56 AM, Davy De Waele 
>>> wrote:
>>>
 Hi Andrew,

 We'd be happy to test and provide feedback.
 We have a use-case now with Nifi that we're in the process of
 implementing.

 If you have sources or binaries feel free to let me know.

 Regards,
 Davy

 On Fri, Sep 30, 2016 at 9:24 PM, Andrew Psaltis <
 psaltis.and...@gmail.com> wrote:

> Hi Davy,
> Sorry for the slow response, I am traveling today. However, I should
> be able to get a build of it for 1.0.0 out later today or tomorrow.
>
> Thanks,
> Andrew
>
> On Fri, Sep 30, 2016 at 6:54 PM, Davy De Waele 
> wrote:
>
>> Hi Andrew,
>>
>> We tested your 0.0.6 binary and it seemed to do what we want.
>> Unfortunately it is not compatible with the current Nifi version.
>> Do you have a binary available that would work with nifi 1.0.0 ?
>>
>> Thx.
>>
>> On Wed, Sep 28, 2016 at 11:39 PM, Andrew Psaltis <
>> psaltis.and...@gmail.com> wrote:
>>
>>> Davy,
>>> The processor I have been working on may meet your needs. You are
>>> correct, at this time I have not pushed the source for it, still working
>>> through some hurdles. The one thing to work out is how you would
>>> dynamically add the processors -- suppose you may be able to use the 
>>> REST
>>> API for NiFi. I would imagine there are quite a number of these devices
>>> that you would need to have processors for. In the use case I have been
>>> working on, there may be 600 or so endpoints that I need to connect to 
>>> and
>>> I'm trying to figure out does it make sense to do it this way.
>>>
>>> I'll hopefully be in a place soon that I can push the code I have
>>> for the GetTCP 

Re: nifi is running out of memory

[Gop Krr]

Thanks Joe for checking. Yes, I got past it and I was successfully able to
demo it to the team :) Now, the next challenge is to drive the performance
out of nifi for the high throughput.

On Mon, Oct 31, 2016 at 7:08 PM, Joe Witt  wrote:

> Krish,
>
> Did you ever get past this?
>
> Thanks
> Joe
>
> On Fri, Oct 28, 2016 at 2:36 PM, Gop Krr  wrote:
> > James, permission issue got resolved. I still don't see any write.
> >
> > On Fri, Oct 28, 2016 at 10:34 AM, Gop Krr  wrote:
> >>
> >> Thanks James.. I am looking into permission issue and update the
> thread. I
> >> will also make the changes as you per your recommendation.
> >>
> >> On Fri, Oct 28, 2016 at 10:23 AM, James Wing  wrote:
> >>>
> >>> From the screenshot and the error message, I interpret the sequence of
> >>> events to be something like this:
> >>>
> >>> 1.) ListS3 succeeds and generates flowfiles with attributes referencing
> >>> S3 objects, but no content (0 bytes)
> >>> 2.) FetchS3Object fails to pull the S3 object content with an Access
> >>> Denied error, but the failed flowfiles are routed on to PutS3Object
> (35,179
> >>> files / 0 bytes in the "putconnector" queue)
> >>> 3.) PutS3Object is succeeding, writing the 0 byte content from ListS3
> >>>
> >>> I recommend a couple thing for FetchS3Object:
> >>>
> >>> * Only allow the "success" relationship to continue to PutS3Object.
> >>> Separate the "failure" relationship to either loop back to
> FetchS3Object or
> >>> go to a LogAttibute processor, or other handling path.
> >>> * It looks like the permissions aren't working, you might want to
> >>> double-check the access keys or try a sample file with the AWS CLI.
> >>>
> >>> Thanks,
> >>>
> >>> James
> >>>
> >>>
> >>> On Fri, Oct 28, 2016 at 10:01 AM, Gop Krr  wrote:
> 
>  This is how my nifi flow looks like.
> 
>  On Fri, Oct 28, 2016 at 9:57 AM, Gop Krr  wrote:
> >
> > Thanks Bryan, Joe, Adam and Pierre. I went past this issue by
> switching
> > to 0.71.  Now it is able to list the files from buckets and create
> those
> > files in the another bucket. But write is not happening and I am
> getting the
> > permission issue ( I have attached below for the reference) Could
> this be
> > the setting of the buckets or it has more to do with the access key.
> All the
> > files which are creaetd in the new bucket are of 0 byte.
> > Thanks
> > Rai
> >
> > 2016-10-28 16:45:25,438 ERROR [Timer-Driven Process Thread-3]
> > o.a.nifi.processors.aws.s3.FetchS3Object FetchS3Object[id=x]
> Failed to
> > retrieve S3 Object for
> > StandardFlowFileRecord[uuid=y,claim=,offset=0,name=
> x.gz,size=0];
> > routing to failure: com.amazonaws.services.s3.
> model.AmazonS3Exception:
> > Access Denied (Service: Amazon S3; Status Code: 403; Error Code:
> > AccessDenied; Request ID: xxx), S3 Extended Request ID:
> > lu8tAqRxu+ouinnVvJleHkUUyK6J6rIQCTw0G8G6
> DB6NOPGec0D1KB6cfUPsj08IQXI8idtiTp4=
> >
> > 2016-10-28 16:45:25,438 ERROR [Timer-Driven Process Thread-3]
> > o.a.nifi.processors.aws.s3.FetchS3Object
> >
> > com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied
> > (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied;
> Request ID:
> > 0F34E71C0697B1D8)
> >
> > at
> > com.amazonaws.http.AmazonHttpClient.handleErrorResponse(
> AmazonHttpClient.java:1219)
> > ~[aws-java-sdk-core-1.10.32.jar:na]
> >
> > at
> > com.amazonaws.http.AmazonHttpClient.executeOneRequest(
> AmazonHttpClient.java:803)
> > ~[aws-java-sdk-core-1.10.32.jar:na]
> >
> > at
> > com.amazonaws.http.AmazonHttpClient.executeHelper(
> AmazonHttpClient.java:505)
> > ~[aws-java-sdk-core-1.10.32.jar:na]
> >
> > at
> > com.amazonaws.http.AmazonHttpClient.execute(
> AmazonHttpClient.java:317)
> > ~[aws-java-sdk-core-1.10.32.jar:na]
> >
> > at
> > com.amazonaws.services.s3.AmazonS3Client.invoke(
> AmazonS3Client.java:3595)
> > ~[aws-java-sdk-s3-1.10.32.jar:na]
> >
> > at
> > com.amazonaws.services.s3.AmazonS3Client.getObject(
> AmazonS3Client.java:1116)
> > ~[aws-java-sdk-s3-1.10.32.jar:na]
> >
> > at
> > org.apache.nifi.processors.aws.s3.FetchS3Object.
> onTrigger(FetchS3Object.java:106)
> > ~[nifi-aws-processors-0.7.1.jar:0.7.1]
> >
> > at
> > org.apache.nifi.processor.AbstractProcessor.onTrigger(
> AbstractProcessor.java:27)
> > [nifi-api-0.7.1.jar:0.7.1]
> >
> > at
> > org.apache.nifi.controller.StandardProcessorNode.onTrigger(
> StandardProcessorNode.java:1054)
> > [nifi-framework-core-0.7.1.jar:0.7.1]
> >
> > at
> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(
>