[jira] [Closed] (NIFI-4819) Add support to delete blob from Azure Storage container
[ https://issues.apache.org/jira/browse/NIFI-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zenfenan closed NIFI-4819. -- Assignee: zenfenan > Add support to delete blob from Azure Storage container > --- > > Key: NIFI-4819 > URL: https://issues.apache.org/jira/browse/NIFI-4819 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: zenfenan >Assignee: zenfenan >Priority: Major > Fix For: 1.6.0 > > > Implement a delete processor that handles deleting blob from Azure Storage > container. This should be an extension of nifi-azure-nar bundle. Currently, > the azure bundle's storage processors has support to list, fetch, put Azure > Storage blobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4864) Additional Resources property pointing at a directory won't find new JARs
[ https://issues.apache.org/jira/browse/NIFI-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zenfenan updated NIFI-4864: --- Description: If you have a processor/Controller Service/Reporting Task that has a property with dynamicallyModifiesClasspath(true) and you set the value to a directory, the resources in that directory will only be calculated when that property changes. This means if you added JARs to the directory later, and stopped and started your processor, those new JARs still won't be available. You would have to change the property to a new directory, or back and forth to some other directory, to force a recalculation. The setProperties method in AbstractConfiguredComponent is where it looks at incoming property changes and determines if any were for classpath related properties and then calls reload accordingly. We would need to consider the case where setProperties is never even being called, someone just stops and starts the processor and would want to pick up any new JARs added. A possible solution might be to computer some kind of hash/fingerprint of the URLs each time reload is called, and then when starting the processor we could recompute the fingerprint and compare it to the previous one. If they are different then we call reload before starting the component. was: If you have a processor that has a property with dynamicallyModifiesClasspath(true) and you set the value to a directory, the resources in that directory will only be calculated when that property changes. This means if you added JARs to the directory later, and stopped and started your processor, those new JARs still won't be available. You would have to change the property to a new directory, or back and forth to some other directory, to force a recalculation. The setProperties method in AbstractConfiguredComponent is where it looks at incoming property changes and determines if any were for classpath related properties and then calls reload accordingly. We would need to consider the case where setProperties is never even being called, someone just stops and starts the processor and would want to pick up any new JARs added. A possible solution might be to computer some kind of hash/fingerprint of the URLs each time reload is called, and then when starting the processor we could recompute the fingerprint and compare it to the previous one. If they are different then we call reload before starting the component. > Additional Resources property pointing at a directory won't find new JARs > - > > Key: NIFI-4864 > URL: https://issues.apache.org/jira/browse/NIFI-4864 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0 >Reporter: Bryan Bende >Assignee: zenfenan >Priority: Minor > > If you have a processor/Controller Service/Reporting Task that has a property > with dynamicallyModifiesClasspath(true) and you set the value to a directory, > the resources in that directory will only be calculated when that property > changes. This means if you added JARs to the directory later, and stopped and > started your processor, those new JARs still won't be available. You would > have to change the property to a new directory, or back and forth to some > other directory, to force a recalculation. > The setProperties method in AbstractConfiguredComponent is where it looks at > incoming property changes and determines if any were for classpath related > properties and then calls reload accordingly. > We would need to consider the case where setProperties is never even being > called, someone just stops and starts the processor and would want to pick up > any new JARs added. > A possible solution might be to computer some kind of hash/fingerprint of the > URLs each time reload is called, and then when starting the processor we > could recompute the fingerprint and compare it to the previous one. If they > are different then we call reload before starting the component. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4826) ListAzureBlobStorage doesn't write azure.blobname properly
[
https://issues.apache.org/jira/browse/NIFI-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zenfenan updated NIFI-4826:
---
Status: Resolved (was: Closed)
> ListAzureBlobStorage doesn't write azure.blobname properly
> --
>
> Key: NIFI-4826
> URL: https://issues.apache.org/jira/browse/NIFI-4826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>Reporter: zenfenan
>Priority: Minor
> Fix For: 1.6.0
>
>
> ListAzureBlobStorage as of now takes the substring from the blob's primary
> URI i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname.
> For ex, if the blob is in the path
> "mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
> It will write azure.blobname as "blob". So if we have the blob located under
> a multiple hierarchy directory structure such as the above one, it will be
> troublesome in the downstream processors like FetchAzureBlobStorage which
> expects the full blob name to be given i.e. "path/to/the/blob". Giving just
> "blob" here will fail.
> A workaround that can be followed right now, is to use "ExecuteScript" and
> get the substring from primary URI i.e. everything after the
> "https://"+storageAccountName+"/"+containerName+"/;. A better approach would
> be to make use of the CloudBlob.getName() API provided in Azure SDK. It
> should be a minor change since we are already using this SDK and the said
> class in our processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (NIFI-4826) ListAzureBlobStorage doesn't write azure.blobname properly
[
https://issues.apache.org/jira/browse/NIFI-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zenfenan closed NIFI-4826.
--
> ListAzureBlobStorage doesn't write azure.blobname properly
> --
>
> Key: NIFI-4826
> URL: https://issues.apache.org/jira/browse/NIFI-4826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>Reporter: zenfenan
>Priority: Minor
> Fix For: 1.6.0
>
>
> ListAzureBlobStorage as of now takes the substring from the blob's primary
> URI i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname.
> For ex, if the blob is in the path
> "mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
> It will write azure.blobname as "blob". So if we have the blob located under
> a multiple hierarchy directory structure such as the above one, it will be
> troublesome in the downstream processors like FetchAzureBlobStorage which
> expects the full blob name to be given i.e. "path/to/the/blob". Giving just
> "blob" here will fail.
> A workaround that can be followed right now, is to use "ExecuteScript" and
> get the substring from primary URI i.e. everything after the
> "https://"+storageAccountName+"/"+containerName+"/;. A better approach would
> be to make use of the CloudBlob.getName() API provided in Azure SDK. It
> should be a minor change since we are already using this SDK and the said
> class in our processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (NIFI-4826) ListAzureBlobStorage doesn't write azure.blobname properly
[
https://issues.apache.org/jira/browse/NIFI-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zenfenan closed NIFI-4826.
--
> ListAzureBlobStorage doesn't write azure.blobname properly
> --
>
> Key: NIFI-4826
> URL: https://issues.apache.org/jira/browse/NIFI-4826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>Reporter: zenfenan
>Priority: Minor
> Fix For: 1.6.0
>
>
> ListAzureBlobStorage as of now takes the substring from the blob's primary
> URI i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname.
> For ex, if the blob is in the path
> "mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
> It will write azure.blobname as "blob". So if we have the blob located under
> a multiple hierarchy directory structure such as the above one, it will be
> troublesome in the downstream processors like FetchAzureBlobStorage which
> expects the full blob name to be given i.e. "path/to/the/blob". Giving just
> "blob" here will fail.
> A workaround that can be followed right now, is to use "ExecuteScript" and
> get the substring from primary URI i.e. everything after the
> "https://"+storageAccountName+"/"+containerName+"/;. A better approach would
> be to make use of the CloudBlob.getName() API provided in Azure SDK. It
> should be a minor change since we are already using this SDK and the said
> class in our processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4831) InvokeHTTP Processor overwrites filename attribute when doing a POST
[ https://issues.apache.org/jira/browse/NIFI-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347014#comment-16347014 ] zenfenan edited comment on NIFI-4831 at 1/31/18 3:31 PM: - Which NiFi version are you using? Can you be a bit more specific on how it happens? I can't seem to reproduce the issue on my end. I tried with GenerateFlowFile as well as GetFile and made a POST request and when the request is success, it goes to "Response" relationship and the filename is intact. It is not overwritten. was (Author: sivaprasanna): Can you be a bit more specific on how it happens? I can't seem to reproduce the issue on my end. I tried with GenerateFlowFile as well as GetFile and made a POST request and when the request is success, it goes to "Response" relationship and the filename is intact. It is not overwritten. > InvokeHTTP Processor overwrites filename attribute when doing a POST > > > Key: NIFI-4831 > URL: https://issues.apache.org/jira/browse/NIFI-4831 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tyler Bryant >Priority: Minor > > The 'filename' attribute gets overwritten when doing a POST in an InvokeHTTP > Processor. I can seem to get around it by putting a 'fileName' attribute in > before the InvokeHTTP Processor, and then it correctly keeps the 'filename'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4831) InvokeHTTP Processor overwrites filename attribute when doing a POST
[ https://issues.apache.org/jira/browse/NIFI-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347014#comment-16347014 ] zenfenan commented on NIFI-4831: Can you be a bit more specific on how it happens? I can't seem to reproduce the issue on my end. I tried with GenerateFlowFile as well as GetFile and made a POST request and when the request is success, it goes to "Response" relationship and the filename is intact. It is not overwritten. > InvokeHTTP Processor overwrites filename attribute when doing a POST > > > Key: NIFI-4831 > URL: https://issues.apache.org/jira/browse/NIFI-4831 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tyler Bryant >Priority: Minor > > The 'filename' attribute gets overwritten when doing a POST in an InvokeHTTP > Processor. I can seem to get around it by putting a 'fileName' attribute in > before the InvokeHTTP Processor, and then it correctly keeps the 'filename'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4826) ListAzureBlobStorage doesn't write azure.blobname properly
[
https://issues.apache.org/jira/browse/NIFI-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zenfenan updated NIFI-4826:
---
Description:
ListAzureBlobStorage as of now takes the substring from the blob's primary URI
i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname. For ex,
if the blob is in the path
"mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
It will write azure.blobname as "blob". So if we have the blob located under a
multiple hierarchy directory structure such as the above one, it will be
troublesome in the downstream processors like FetchAzureBlobStorage which
expects the full blob name to be given i.e. "path/to/the/blob". Giving just
"blob" here will fail.
A workaround that can be followed right now, is to use "ExecuteScript" and get
the substring from primary URI i.e. everything after the
"https://"+storageAccountName+"/"+containerName+"/;. A better approach would be
to make use of the CloudBlob.getName() API provided in Azure SDK. It should be
a minor change since we are already using this SDK and the said class in our
processor.
was:
ListAzureBlobStorage as of now takes the substring from the blob's primary URI
i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname. For ex,
if the blob is in the path
"mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
It will write azure.blobname as "blob". So if we have the blob located under a
multiple hierarchy directory structure such as the above one, it will be
troublesome in the downstream processors like FetchAzureBlobStorage which
expects the full blob name to be given i.e. "path/to/the/blob". Giving just
"blob" here will fail.
A workaround that can be followed right now, is to use "ExecuteScript" and get
the substring from primary URI i.e. everything after the
"https://"+storageAccountName+"/"+containerName+"/;. A better approach would be
to make use of the CloudBlob.getName() API provided in Azure SDK. It should be
a minor change since we are already using this SDK and the class in our
processor.
> ListAzureBlobStorage doesn't write azure.blobname properly
> --
>
> Key: NIFI-4826
> URL: https://issues.apache.org/jira/browse/NIFI-4826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>Reporter: zenfenan
>Priority: Minor
>
> ListAzureBlobStorage as of now takes the substring from the blob's primary
> URI i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname.
> For ex, if the blob is in the path
> "mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
> It will write azure.blobname as "blob". So if we have the blob located under
> a multiple hierarchy directory structure such as the above one, it will be
> troublesome in the downstream processors like FetchAzureBlobStorage which
> expects the full blob name to be given i.e. "path/to/the/blob". Giving just
> "blob" here will fail.
> A workaround that can be followed right now, is to use "ExecuteScript" and
> get the substring from primary URI i.e. everything after the
> "https://"+storageAccountName+"/"+containerName+"/;. A better approach would
> be to make use of the CloudBlob.getName() API provided in Azure SDK. It
> should be a minor change since we are already using this SDK and the said
> class in our processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-4826) ListAzureBlobStorage doesn't write azure.blobname properly
zenfenan created NIFI-4826:
--
Summary: ListAzureBlobStorage doesn't write azure.blobname properly
Key: NIFI-4826
URL: https://issues.apache.org/jira/browse/NIFI-4826
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 1.5.0, 1.4.0, 1.3.0, 1.2.0
Reporter: zenfenan
ListAzureBlobStorage as of now takes the substring from the blob's primary URI
i.e. primaryUri.lastIndexOf('/') + 1 and writes that as azure.blobname. For ex,
if the blob is in the path
"mystorageaccountname.blob.core.windows.net/container-name/path/to/the/blob".
It will write azure.blobname as "blob". So if we have the blob located under a
multiple hierarchy directory structure such as the above one, it will be
troublesome in the downstream processors like FetchAzureBlobStorage which
expects the full blob name to be given i.e. "path/to/the/blob". Giving just
"blob" here will fail.
A workaround that can be followed right now, is to use "ExecuteScript" and get
the substring from primary URI i.e. everything after the
"https://"+storageAccountName+"/"+containerName+"/;. A better approach would be
to make use of the CloudBlob.getName() API provided in Azure SDK. It should be
a minor change since we are already using this SDK and the class in our
processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-3711) Change default size of provenance details dialog to show Replay button by default
[
https://issues.apache.org/jira/browse/NIFI-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342428#comment-16342428
]
zenfenan commented on NIFI-3711:
I think it is a fair point to move the replay button.
> Change default size of provenance details dialog to show Replay button by
> default
> -
>
> Key: NIFI-3711
> URL: https://issues.apache.org/jira/browse/NIFI-3711
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Priority: Minor
> Labels: beginner, dialog, provenance, replay, ui
> Attachments: Screen Shot 2017-04-18 at 11.19.12 AM.png, Screen Shot
> 2017-04-18 at 11.19.46 AM.png, Screenshot from 2017-09-10 20-56-41.png
>
>
> The provenance details dialog is sized such that the "Replay" button on the
> "Content" tab is not visible by default and there is no visible scroll bar
> indicating that the panel has additional content. Some users have reported
> being unaware of the replay feature because of this. I propose changing the
> default dialog size to ensure the panel is fully displayed ({{min-height}}?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
