Re: Nifi python usage
Hi, As James mentioned in your example the "system" memory would be used by the process/script in this case. I wanted to add that if you use the "ExecuteScript" processor that the NiFi memory would be used. This is because NiFi uses Jython for processing Python files. So where a native Cython process would typically create a heap for a Python objects in a native process, Jython will instead use the Java heap Jeremy Dyer On Wed, Sep 14, 2022 at 8:36 AM James Halfpenny wrote: > Hi, > The memory you allocate to NiFi will typically be the maximum heap size > for the Java process. If you spawn a separate Python process then it will > not be bound by the JVM heap size, so in your example it would come from > the 8GB allocated for the “system”. > > I would recommend if you see high resource usage on the server again you > take note of which processes are consuming the resources. > > Kind regards, > Jim > > > On 14 Sep 2022, at 14:18, never more wrote: > > > > Hello! > > Can you help me with one question - can't find any information :( > > > > We have a nifi on my server (16 CPU, 32 RAM), in config our system adm. > > set ram usage for nifi = 24 Gb, rest 8 Gb - for the system. > > In Nifi we execute many Python scripts by using ExecuteStreamCommand > (just > > set path of script on server in properties). > > So the question is - which resources python script uses - Nifi (24 gb) > or > > system (8 gb)? > > > > Maybe its a little stupid question😅, but I am afraid that improper use > of > > resources may lead to a service failure, > > because we already have several cases of service failure when we > noticed > > high utilization of resources (RAM 90-95% and CPU 80-90%) >
Re: Nifi python usage
Hi, The memory you allocate to NiFi will typically be the maximum heap size for the Java process. If you spawn a separate Python process then it will not be bound by the JVM heap size, so in your example it would come from the 8GB allocated for the “system”. I would recommend if you see high resource usage on the server again you take note of which processes are consuming the resources. Kind regards, Jim > On 14 Sep 2022, at 14:18, never more wrote: > > Hello! > Can you help me with one question - can't find any information :( > > We have a nifi on my server (16 CPU, 32 RAM), in config our system adm. > set ram usage for nifi = 24 Gb, rest 8 Gb - for the system. > In Nifi we execute many Python scripts by using ExecuteStreamCommand (just > set path of script on server in properties). > So the question is - which resources python script uses - Nifi (24 gb) or > system (8 gb)? > > Maybe its a little stupid question😅, but I am afraid that improper use of > resources may lead to a service failure, > because we already have several cases of service failure when we noticed > high utilization of resources (RAM 90-95% and CPU 80-90%)
Nifi python usage
Hello! Can you help me with one question - can't find any information :( We have a nifi on my server (16 CPU, 32 RAM), in config our system adm. set ram usage for nifi = 24 Gb, rest 8 Gb - for the system. In Nifi we execute many Python scripts by using ExecuteStreamCommand (just set path of script on server in properties). So the question is - which resources python script uses - Nifi (24 gb) or system (8 gb)? Maybe its a little stupid question😅, but I am afraid that improper use of resources may lead to a service failure, because we already have several cases of service failure when we noticed high utilization of resources (RAM 90-95% and CPU 80-90%)
Re: Jira contributor access needed
Hi, Granted you contributor access to the NIFI project. Looking forward to your contribution. Thanks, Marton On Wed, 14 Sept 2022 at 08:54, Saumya Gurtu wrote: > > Hello, > > Please grant "Jira contributor access" to sgurtu > > > Thanks, > Saumya
Jira contributor access needed
Hello, Please grant "Jira contributor access" to sgurtu Thanks, Saumya
Re: "Jira contributor access" needed
Hi Akshara, Granted Jira contributor access. Looking forward to your contributions. Regards, Peter Turcsanyi On Wed, Sep 14, 2022 at 6:38 AM Akshara Uke wrote: > > Hello, > > Please grant "Jira contributor access" to aksharau > > > Thanks much, > > akshara
