Hello, I noticed I maybe was not configuring the processor correctly but nothing went wrong until now.
I had the following configured: *Container Name*: "base_container/A/B" *Blob*: ${filename} I changed it to this and it works: *Container Name*: "base_container" *Blob*: "/A/B/"${filename} Thanks, Eric On Wed, Sep 16, 2020 at 1:12 PM Paul Kelly <pkelly.n...@gmail.com> wrote: > Hi Eric, > > We also noticed an issue with PutAzureBlobStorage when upgrading to NiFi > 1.12.0, and I believe it has to do with changes made for > https://issues.apache.org/jira/browse/NIFI-6913, where the default > behavior changed to check to see if a container exists before writing a > file, and creating the container if it doesn't exist. The SAS tokens we > are using within our flows do not have permission to list containers, so > this check is failing, and therefore the overall Put operation is failing. > We are seeing a different error, but I suspect it is related since this in > your stack trace: > > at > com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:354) > at > com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:301) > > which indicates that yours is also trying to create the container and > failing due to this new check. Do you have permission to list and create > containers? > > I reported this issue in https://issues.apache.org/jira/browse/NIFI-7794. > I plan to submit a fix for it by adding a property to the processor to > determine if a container should be created if it doesn't exist, but if > someone else beats me to it, that'd be great. > > Paul > > On Wed, Sep 16, 2020 at 7:10 PM Joey Frazee <joey.fra...@icloud.com> > wrote: > >> Eric, can you share any details about your config (e.g., what do you have >> in the Blob property)? I tried the following scenarios in an upgrade to >> 1.12.0 and main and they seem to work. >> >> Pre-existing object: A/B/test.json >> >> New object: A/B/${filename}.json >> >> New object with new pseudo-dirs: >> ${random():mod(10):plus(1)}/${random():mod(10):plus(1)}/${filename}.json >> >> -joey >> >> On Sep 16, 2020, 11:02 AM -0700, Eric Secules <esecu...@gmail.com>, >> wrote: >> >> Hello everyone, >> >> >> I was able to see why this is an issue. It's an issue that the blob is >> stored several layers deep at "my-container/A/B/my_test_blob.json" >> >> >> -Eric >> >> >> On Wed, Sep 16, 2020 at 10:49 AM Eric Secules <esecu...@gmail.com> wrote: >> >>> Hello everyone, >>> >>> I tried upgrading to 1.12.0 and right away noticed that >>> PutAzureBlobStorage is failing due to the following error. I don't think >>> it's an issue with access because I can use a ListAzureBlobStorage on the >>> same container and I haven't changed the permissions of the container. And >>> I didn't change any parameters during the upgrade and it was working on >>> 1.11.4. >>> >>> I am writing the blob to a container path that already exists: >>> "my-container/A/B/my_test_blob.json:" >>> >>> 2020-09-16 00:59:17,283 ERROR [Timer-Driven Process Thread-6] >>> o.a.n.p.a.storage.PutAzureBlobStorage >>> PutAzureBlobStorage[id=15404d54-bc14-350c-7847-521b765dd57f] Failed to put >>> Azure blob my_test_blob.json: com.microsoft.azure.storage.StorageException: >>> The requested URI does not represent any resource on the server. >>> com.microsoft.azure.storage.StorageException: The requested URI does not >>> represent any resource on the server. >>> at >>> com.microsoft.azure.storage.StorageException.translateException(StorageException.java:87) >>> at >>> com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:220) >>> at >>> com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:744) >>> at >>> com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:354) >>> at >>> com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:301) >>> at >>> org.apache.nifi.processors.azure.storage.PutAzureBlobStorage.onTrigger(PutAzureBlobStorage.java:100) >>> at >>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) >>> at >>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174) >>> at >>> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) >>> at >>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) >>> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> at java.lang.Thread.run(Thread.java:748) >>> Caused by: java.lang.NullPointerException: null >>> at >>> com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:202) >>> ... 16 common frames omitted >>> >>> Thanks, >>> Eric >>> >>