Awesome, thanks for the detailed steps!
Em qua, 18 de set de 2019 às 11:41, Jerry Vinokurov
escreveu:
> This certainly works. You can create a schema registry and define an Avro
> schema listing your fields. Then make sure that when you set up the reader,
> it's configured to read the header so
I managed to find the culprit.. it was the init script that I was using that
was doing something weird.
I added MAX_FD=5 to my nifi-env.sh file and everything seems to be fine now
thanks
From: Abdou B
Sent: Wednesday, September 18, 2019 10:42 AM
To: users@ni
Hello,
It seems to me that for some distribution, you should modify those values
in the Cluster management tool.
For example in HDF, with Ambari, you should change the parameter
: nifi_user_nofile_limit. for the change to take effect.
Best regards
Abdou
Le mer. 18 sept. 2019 à 16:34, Jean-Sebast
This certainly works. You can create a schema registry and define an Avro
schema listing your fields. Then make sure that when you set up the reader,
it's configured to read the header so that it knows which fields go where
in the record, set up the mode of the schema access to read from the
regist
Does not seem to help... processes are still limited to 4096 fds
From: Jean-Sebastien Vachon
Sent: Wednesday, September 18, 2019 10:31 AM
To: users@nifi.apache.org
Subject: Re: Too many open files
Oups.. just saw the following:
Your distribution may require an e
Oups.. just saw the following:
Your distribution may require an edit to /etc/security/limits.d/90-nproc.conf
by adding:
* soft nproc 1
I will try this
From: Jean-Sebastien Vachon
Sent: Wednesday, September 18, 2019 10:30 AM
To: users@nifi.apache.org
Subject
Hi all,
I've started to see "Too many open files" error messages in Nifi. I checked
https://nifi.apache.org/quickstart.html to see the recommended values to fix
this
and made the required changes to /etc/security/limits.conf, exited my shell and
restarted Nifi. When I check the limits of the Ja
Hi folks,
Is it possible to validate fields/columns in Record or CSV by its name
instead of it's position? For example I have a record with two mandatory
fields and some optional fields but they may be on different position on
each ingested file. Should I use a script or there is already a process
Thanks very much Bryan, chaning the overall timer-driven thread pool value
works.
wangl...@geekplus.com.cn
From: Bryan Bende
Date: 2019-09-18 20:53
To: users
Subject: Re: NiFi active thread count is no more than 10 ?
The overall timer-driven thread pool defaults to 10 (configured from
the
The overall timer-driven thread pool defaults to 10 (configured from
the controller settings in top right menu).
So even if a processor has 100 concurrent tasks, there are still only
10 threads available.
On Wed, Sep 18, 2019 at 8:20 AM Joe Witt wrote:
>
> Hello
>
> The 100 threads for the contr
Hello
The 100 threads for the controller overall is the maximum number of threads
that could run concurrently. On a 16 core system and a flow which is very
I/O bound this is definitely achievable. Generally you want to look at
some multiple of the number of physical cores such as 2,4,8, etc.. but
My NiFi server is 16 cores. I also configed some processor cocurrent tasks to
100.
But why the active thread count shown on the NiFi web ui is no more than 10?
wangl...@geekplus.com.cn
12 matches
Mail list logo