moving dev to bcc Yes I believe the issue here is that FetchS3 doesn't do chunked transfers and so is loading all into memory. I've not verified this in the code yet but it seems quite likely. Krish if you can verify that going with a larger heap gets you in the game can you please file a JIRA.
Thanks Joe On Thu, Oct 27, 2016 at 9:34 PM, Bryan Bende <bbe...@gmail.com> wrote: > Hello, > > Are you running with all of the default settings? > > If so you would probably want to try increasing the memory settings in > conf/bootstrap.conf. > > They default to 512mb, you may want to try bumping it up to 1024mb. > > -Bryan > > On Thu, Oct 27, 2016 at 5:46 PM, Gop Krr <gop....@gmail.com> wrote: >> >> Hi All, >> >> I have very simple data flow, where I need to move s3 data from one bucket >> in one account to another bucket under another account. I have attached my >> processor configuration. >> >> >> 2016-10-27 20:09:57,626 ERROR [Flow Service Tasks Thread-2] >> org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[Flow Service >> Tasks Thread-2,5,main]: java.lang.OutOfMemoryError: Java heap space >> >> I am very new to NiFi and trying ot get few of the use cases going. I need >> help from the community. >> >> Thanks again >> >> Rai >> >> >> >
