Andrew Doumaux created NIFI-6728:
------------------------------------
Summary: InvokeHttp / PostHttp should yield on HTTP-503 (Service
Unavailable)
Key: NIFI-6728
URL: https://issues.apache.org/jira/browse/NIFI-6728
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 1.7.1
Reporter: Andrew Doumaux
h3. Problem:
InvokeHttp/PostHttp only penalize a flowfile when a the remote host returns an
error code. It has no way to yield the processor when the remote host
indicates it's unavailable.
h3. Desired Behavior:
A configurable property to have the processor yield on server error codes
(5xx). Specifically HTTP-503.
Maybe user exceptions as well (4xx), specifically HTTP-401 & HTTP-403 .
h3. Use Case:
I'm using NIFI to load balance data to multiple horizontally scalable ETL
processes (Standalone java spring boot apps). The ETL application returns a
HTTP-503 when it is saturated with work and have no available threads to
process a new piece of data. I have a separate PostHttp processor for each ETL
application instance/host. It was done this way since I had the misconception
that NIFI would yield that specific processor and thus pause sending data to
that ETL application as it worked off it's current load.
This behavior of penalizing the file and not yielding the processor is
especially problematic when attempting to process an excessive backlog of data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
