I have an Elastic Search cluster that is setup with SSL. It uses a self-signed
cert for this. I am working with Apache Nifi 1.9.2. I have a flow that has the
PutElasticSearchHttp component. I have setup a SSLContextService for that
component where I have specified a trust store that has the
Hi Joe,
In high data volume circumstance, seems it is impossible to keep order accross
the whole cluster.
Even if EnforceOrder is used, it can be only scheduled on primary node and
becomes a performance bettleneck.
I tried to set the concurrency to 1 from the begining and not load
Hi Bryan,
lamentably I didn't keep a copy of the template, didn't think could be useful.
But Im pretty sure to remember two escenarios.
First one an input port in a Process Group which was straight connected to 2
flows (no condition in between), first connection to an active processor and
Hi Luis,
Can you describe the part of the flow that turned out to be a problem
a little more?
Was it a port on the root canvas used for s2s that was then connected
into a process group where everything inside was disabled?
And what did you do to solve the problem, did you stop the port?
Thank you for that tip, Andy!
This is actually a bug I've wanted to track down and fix but it's in parts
of the NiFi codebase I'm really not familiar with and wasn't sure how to
start ... if you can connect me with someone who knows that area
(scheduling and clusters, mainly) I would be happy to
Ok, figured out a way to do it. I noticed that QueryRecord uses Apache
Calcite so I tried some of Calcite functions until I got into this
statement that seems to work:
SELECT index FROM FLOWFILE WHERE CAST( SUBSTRING(index FROM
CHAR_LENGTH(index)-9) AS DATE) <= CURRENT_DATE
Anyhow, I would still
Hi Folks,
I'd like to use a SQL statement in QueryRecords like this uses the extract
the last 10 chars of field and compare it to today, similar to the line
below:
SELECT substring( '/index', 10, -1) as expired FROM FLOWFILE WHERE expired
<= '${now():format("-MM-dd")}'
This statement is
Lei
The order won't necessarily be the same. You'd want EnforceOrder to follow
the load balanced connection most likely.
It is important to keep in mind the queues are basically insertion order
and the system is inherently multi-threaded so the data can be shuffled in
such cases. You can
Since data will be balanced to different node, the order after balanced will be
the same as before?
Thanks,
Lei
wangl...@geekplus.com.cn
Thanks very much. I got it.
wangl...@geekplus.com.cn
From: Joe Witt
Date: 2019-10-16 17:24
To: users@nifi.apache.org
Subject: Re: Re: NiFi backpressure not work
Lei
Please do not post to both users and dev list. Dropping dev.
Yes these scenarios are based on limitations of a given
Lei
Please do not post to both users and dev list. Dropping dev.
Yes these scenarios are based on limitations of a given processor
implementation or the nature of a given protocol/mechanism. NiFi enforces
back pressure by slowing/stopping scheduling a component and making fact of
back pressure
11 matches
Mail list logo
