I have a simple dataflow running on 1.0.0-SNAPSHOT that looks like this ConsumeMQTT -> MergeContent (batch them up by 1000000 items) -> PutHDFS
The ConsumeMQTT is receiving about 1000000 msgs per 5 minutes. >From time to time I need to restart Nifi because it seems to hang / starts processing very slowly. I believe this is due to memory related issues (sometimes I see OutOfMemory errors, but I already have it running on a 6GB heap). The problem is that when restarting the flow, it sometimes takes a very very long time before the UI comes up. I had to restart the flow when there were 1000000 msgs in the queue (50MB) between ConsumeMQTT / MergeContent. (For some reason ConsumeMQTT was grinding to a halt). After restarting it took over 7 minutes for the UI to come up. Sometimes the UI doesn't come up at all, even after waiting for 20 minutes. At that point you see similar messages in the logs. Sometimes if fails with an out of memory. At that point I never get the opportunity to stop the flow as I cannot access the UI / API. Any tips on optimizing memory usage and are there any plans to make the UI / API available immediately upon startup so that corrective actions can take place before nifi starts grinding to a halt ? -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
