Hello,

I noticed I maybe was not configuring the processor correctly but nothing
went wrong until now.

I had the following configured:
*Container Name*: "base_container/A/B"
*Blob*: ${filename}

I changed it to this and it works:
*Container Name*: "base_container"
*Blob*: "/A/B/"${filename}

Thanks,
Eric

On Wed, Sep 16, 2020 at 1:12 PM Paul Kelly <pkelly.n...@gmail.com> wrote:

> Hi Eric,
>
> We also noticed an issue with PutAzureBlobStorage when upgrading to NiFi
> 1.12.0, and I believe it has to do with changes made for
> https://issues.apache.org/jira/browse/NIFI-6913, where the default
> behavior changed to check to see if a container exists before writing a
> file, and creating the container if it doesn't exist.  The SAS tokens we
> are using within our flows do not have permission to list containers, so
> this check is failing, and therefore the overall Put operation is failing.
> We are seeing a different error, but I suspect it is related since this in
> your stack trace:
>
>         at
> com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:354)
>         at
> com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:301)
>
> which indicates that yours is also trying to create the container and
> failing due to this new check.  Do you have permission to list and create
> containers?
>
> I reported this issue in https://issues.apache.org/jira/browse/NIFI-7794.
> I plan to submit a fix for it by adding a property to the processor to
> determine if a container should be created if it doesn't exist, but if
> someone else beats me to it, that'd be great.
>
> Paul
>
> On Wed, Sep 16, 2020 at 7:10 PM Joey Frazee <joey.fra...@icloud.com>
> wrote:
>
>> Eric, can you share any details about your config (e.g., what do you have
>> in the Blob property)? I tried the following scenarios in an upgrade to
>> 1.12.0 and main and they seem to work.
>>
>> Pre-existing object: A/B/test.json
>>
>> New object: A/B/${filename}.json
>>
>> New object with new pseudo-dirs:
>> ${random():mod(10):plus(1)}/${random():mod(10):plus(1)}/${filename}.json
>>
>> -joey
>>
>> On Sep 16, 2020, 11:02 AM -0700, Eric Secules <esecu...@gmail.com>,
>> wrote:
>>
>> Hello everyone,
>>
>>
>> I was able to see why this is an issue. It's an issue that the blob is
>> stored several layers deep at "my-container/A/B/my_test_blob.json"
>>
>>
>> -Eric
>>
>>
>> On Wed, Sep 16, 2020 at 10:49 AM Eric Secules <esecu...@gmail.com> wrote:
>>
>>> Hello everyone,
>>>
>>> I tried upgrading to 1.12.0 and right away noticed that
>>> PutAzureBlobStorage is failing due to the following error. I don't think
>>> it's an issue with access because I can use a ListAzureBlobStorage on the
>>> same container and I haven't changed the permissions of the container. And
>>> I didn't change any parameters during the upgrade and it was working on
>>> 1.11.4.
>>>
>>> I am writing the blob to a container path that already exists:
>>> "my-container/A/B/my_test_blob.json:"
>>>
>>> 2020-09-16 00:59:17,283 ERROR [Timer-Driven Process Thread-6]
>>> o.a.n.p.a.storage.PutAzureBlobStorage
>>> PutAzureBlobStorage[id=15404d54-bc14-350c-7847-521b765dd57f] Failed to put
>>> Azure blob my_test_blob.json: com.microsoft.azure.storage.StorageException:
>>> The requested URI does not represent any resource on the server.
>>> com.microsoft.azure.storage.StorageException: The requested URI does not
>>> represent any resource on the server.
>>>         at
>>> com.microsoft.azure.storage.StorageException.translateException(StorageException.java:87)
>>>         at
>>> com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:220)
>>>         at
>>> com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:744)
>>>         at
>>> com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:354)
>>>         at
>>> com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:301)
>>>         at
>>> org.apache.nifi.processors.azure.storage.PutAzureBlobStorage.onTrigger(PutAzureBlobStorage.java:100)
>>>         at
>>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>>>         at
>>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
>>>         at
>>> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
>>>         at
>>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
>>>         at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
>>>         at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>         at
>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>>>         at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>>>         at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>>>         at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>         at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>         at java.lang.Thread.run(Thread.java:748)
>>> Caused by: java.lang.NullPointerException: null
>>>         at
>>> com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:202)
>>>         ... 16 common frames omitted
>>>
>>> Thanks,
>>> Eric
>>>
>>

Reply via email to