Lorenzo Without seeing the code and logs it would be very difficult to help. nifi has no trouble by design writing large files (GBs) to many things including hdfs so the issue is probably in how this client library interacts with the data stream.
On Sep 2, 2016 4:19 PM, "Lorenzo Peder" <lorenzo.pe...@ds-iq.com> wrote: > Hi All, > > > > We’ve run into an issue uploading a larger file (~50Mb) into an Azure Data > Lake using a custom processor in nifi 0.7-1.0. This custom processor has > worked consistently for smaller files, but once encountered with this > larger file, it spits http error 404 (file not found). Eventually a minor > portion of the file wrote to the data lake. > > We used fiddler to capture network traffic between Nifi and the Azure data > lake while the processor was running and captured http error 204 (no > contents). > > Then we wrote a java sdk script to upload this same file without Nifi into > the data lake and it worked successfully. > > These findings lead us to believe that this issue is occurring within > Nifi, if someone could please point us in the right direction in resolving > this issue it would be greatly appreciated. > > > > Thank you, > > > > Lorenzo Peder > > Operations Analyst, Campaign Operations & Services > > > > 425.974.1363 : Office > > 425.260.5027 : Mobile > > www.ds-iq.com > > > > Dynamic Shopper Intelligence > > > > This e-mail may contain confidential or privileged information. > > If you are not the intended recipient, please notify the sender > immediately and then delete this message. > > > > >
