[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940112#comment-16940112 ] Seokwon Yang commented on NIFI-6488: We are seeing the same UI unresponsiveness or UI not available (with SocketTimeoutException). In our flow, we heavily use the PutAzureStorage processor to upload flow files to azure clod blog storage, and we are seeing the throttling on the azure side when UI is not available. We increase the timeout parameters and reduce the thread count as suggested in multiple blogs, but when we load data heavily on our cluster, we starts to see this issue again and again. It would be nice if NIFI UI does not get affected by the data flow and and run in a separate JVM like nifi bootstrap process. Any plan to do separate out UI from data flow execution engine? > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png, > image-2019-08-30-08-03-39-877.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919480#comment-16919480 ] David Sargrad commented on NIFI-6488: - !image-2019-08-30-08-03-39-877.png! > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png, > image-2019-08-30-08-03-39-877.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917817#comment-16917817 ] David Sargrad commented on NIFI-6488: - The flows I am using are relatively small... perhaps a dozen processors or so. I am doing a lot of JMS, Kafka, and Mongo stuff... > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917814#comment-16917814 ] David Sargrad commented on NIFI-6488: - I am using chrome. The UI seems to become unresponsive over time. Typically it has no problem handling the initial page load. However when it becomes unresponsive, it seems to be that initial load.. Once it loads (when it loads) it seems ok. I sometimes just restart NiFi, since I wait for the page to load and it doesnt. I have no problems of this nature for my "production NiFi" instances. I've only experienced this, as I mentioned, in what I would call "developer instances".. However even in these instances it strikes me that there is an issue that is worth looking at. > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917808#comment-16917808 ] Matt Gilman commented on NIFI-6488: --- And does the UI become unresponsive over time? Or can it happen on the initial page load? > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917807#comment-16917807 ] Matt Gilman commented on NIFI-6488: --- [~dsargrad] I don't think what your seeing is being caused by the running dataflow. What browser are you running in? How big is the flow (the number of components in your root group)? When this happens, can you please open the browsers developer tools watch the network tab. I would be interested to see how long the various XHR requests are taking. Thanks > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917726#comment-16917726 ] David Sargrad commented on NIFI-6488: - [~joewitt] I do understand the need to ensure that the data flow is treated as the number one concern. However I do believe that there are HMI architectures that can ensure that this is the case. The current issue that I see seems to be that the HMI is so tightly coupled to the core data processing application. I do know there is work afoot to refactor the HMI. I wonder if this includes an eye towards a design element that further decouples the HMI, from the core. > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > Attachments: image-2019-08-28-08-15-51-702.png > > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > > I've updated the following picture. Since it did not seem to attach before. > I've also added a view of the memory and CPU consumption. In this particular > case I waited for about 30 seconds and eventually the NiFi user interface > came to life. > !image-2019-08-28-08-15-51-702.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893144#comment-16893144 ] Joseph Witt commented on NIFI-6488: --- on the operational state preservation comment one of the things we tend to favor/preserve is the actual running dataflow btw. The UI being unresponsive can reflect this bias. That is just a general statement - we should figure out what you're hitting. Also, where /what environment is this NiFi instance running in? Is it a cluster? > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > !image-2019-07-25-08-23-46-031.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (NIFI-6488) NiFi web interface often non-responsive
[ https://issues.apache.org/jira/browse/NIFI-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893120#comment-16893120 ] Matt Gilman commented on NIFI-6488: --- It appears you've tried to attach an image but it didn't work. Is the browser itself unresponsive or is the browser issuing requests that are not returning? Try opening the browsers Developer Tools and looking at the Network tab to see if there are any Pending requests. If so, try taking a thread dump [1] and looking at any threads name NiFi Web Server. The thread dump should go to the nifi-bootstrap.log by default. [1] ./bin/nifi.sh dump > NiFi web interface often non-responsive > > > Key: NIFI-6488 > URL: https://issues.apache.org/jira/browse/NIFI-6488 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.2 > Environment: centos 7 >Reporter: David Sargrad >Priority: Major > > I have been using NiFi for close to a year. Currently most of my experience > has been with building custom processors and prototyping a variety of flows. > > Often the NiFi web interface becomes non-responsive. Its typically during a > flow integration phase, where I leave flowfiles sitting in queues, sometimes > 3 or 4 queues may be maxed at 10,000 flow files. These are relatively small > flowfiles (perhaps 200 or 300 bytes). Yet, of course, the sum total of the > filled queues is in the tens of megabytes. > > I am guessing that the JVM is nearing, or perhaps has exceeded, its maximum > allocated heap size. I'd think that NiFi protects against non-responsive UI's > by shedding load, presenting a warning, or doing something to preserve its > operational state. > > What is the best way for me to isolate this issue? I'd like to understand > exactly what drives NiFi to be non-responsive. If there is anything that I > can do to isolate this, and to help the NiFi team make this more robust, I'm > glad to dig. > > Its been impossible for me to exactly characterize the conditions that lead > to a non-responsive NiFi UI. However I see this fairly frequently. > !image-2019-07-25-08-23-46-031.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016)
