[
https://issues.apache.org/jira/browse/NIFI-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seokwon Yang updated NIFI-7473:
-------------------------------
Description:
Some stuff to think through would be if there was one failure in a batch of
many and it all ran again is the result the same and does the SDK throw any
exceptions?
For a process that's just modifying some content or adding attributes it's as
simple as adding the annotation.
But when it modifies some external resource you sometimes have to take care to
make sure it'd do the right thing if there's a failure.
I think the Fetch won't require any code changes. Not sure about Delete or Put.
{{[https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html]}}
{{SupportsBatching}}: This annotation indicates that it is okay for the
framework to batch together multiple ProcessSession commits into a single
commit. If this annotation is present, the user will be able to choose whether
they prefer high throughput or lower latency in the Processor’s Scheduling tab.
This annotation should be applied to most Processors, but it comes with a
caveat: if the Processor calls {{ProcessSession.commit}}, there is no guarantee
that the data has been safely stored in NiFi’s Content, FlowFile, and
Provenance Repositories. As a result, it is not appropriate for those
Processors that receive data from an external source, commit the session, and
then delete the remote data or confirm a transaction with a remote resource.
was:
{{}}
Some stuff to think through would be if there was one failure in a batch of
many and it all ran again is the result the same and does the SDK throw any
exceptions?
{{}}
For a process that's just modifying some content or adding attributes it's as
simple as adding the annotation.
But when it modifies some external resource you sometimes have to take care to
make sure it'd do the right thing if there's a failure.
I think the Fetch won't require any code changes. Not sure about Delete or Put.
{{}}
{{}}
{{}}
{{[https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html]}}
{{SupportsBatching}}: This annotation indicates that it is okay for the
framework to batch together multiple ProcessSession commits into a single
commit. If this annotation is present, the user will be able to choose whether
they prefer high throughput or lower latency in the Processor’s Scheduling tab.
This annotation should be applied to most Processors, but it comes with a
caveat: if the Processor calls {{ProcessSession.commit}}, there is no guarantee
that the data has been safely stored in NiFi’s Content, FlowFile, and
Provenance Repositories. As a result, it is not appropriate for those
Processors that receive data from an external source, commit the session, and
then delete the remote data or confirm a transaction with a remote resource.
> Add @SupportsBatching to Azure storage processors
> -------------------------------------------------
>
> Key: NIFI-7473
> URL: https://issues.apache.org/jira/browse/NIFI-7473
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Seokwon Yang
> Assignee: Seokwon Yang
> Priority: Major
> Labels: azureblob
>
> Some stuff to think through would be if there was one failure in a batch of
> many and it all ran again is the result the same and does the SDK throw any
> exceptions?
> For a process that's just modifying some content or adding attributes it's as
> simple as adding the annotation.
> But when it modifies some external resource you sometimes have to take care
> to make sure it'd do the right thing if there's a failure.
> I think the Fetch won't require any code changes. Not sure about Delete or
> Put.
>
> {{[https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html]}}
> {{SupportsBatching}}: This annotation indicates that it is okay for the
> framework to batch together multiple ProcessSession commits into a single
> commit. If this annotation is present, the user will be able to choose
> whether they prefer high throughput or lower latency in the Processor’s
> Scheduling tab. This annotation should be applied to most Processors, but it
> comes with a caveat: if the Processor calls {{ProcessSession.commit}}, there
> is no guarantee that the data has been safely stored in NiFi’s Content,
> FlowFile, and Provenance Repositories. As a result, it is not appropriate for
> those Processors that receive data from an external source, commit the
> session, and then delete the remote data or confirm a transaction with a
> remote resource.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
