-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brown, Robin wrote: | This is not working for us, it seems to spawn hundreds of nfsend and | nfprofile processes and logs these to syslog: | | Sep 7 13:35:41 munchies nfsen[21090]: 348 channels/alerts to profile | Sep 7 13:35:45 munchies nfsen[21090]: nfprofile failed: Too many open | files | Sep 7 13:35:45 munchies nfsen[23128]: Use of uninitialized value in | string ne at /var/local/nfsen/bin/nfsend line 303. | Sep 7 13:35:45 munchies nfsen[23128]: Use of uninitialized value in | print at /var/local/nfsen/bin/nfsend line 304. | Sep 7 13:35:45 munchies nfsen[23128]: Use of uninitialized value in | concatenation (.) or string at /var/local/nfsen/bin | /nfsend line 305. | | Using nfsen-1.3b-20070720 and we're exceeding the 5 minute run time, | this would help a lot if we could get it to work. | | -Robin
Hi Robin, The reason for the amount of processes is most likely because your system still doesn't have enough resources to run the periodic updates in time. What you should start with is looking at how long it actually takes for the periodic updates to complete. In your debug log, you can see when the profiling stages starts and ends, and when the plugin stage starts and ends. Another important thing to look at if the bottleneck is either your cpu capacity or harddisk throughput. I am not an expert in assessing the performance of a system, but the amount of processing time going to user and to the system could give some pointers. Possible ways of speeding things are: - - Do not set the $PROFILERS variable higher than the number of available processing cores. All instances of nfprofile run independently, so they all read the complete live profile separately. So it only makes sense to run multiple instances of nfprofile if they don't have to fight each other for processing time. - - Use compression. Peter has told me that there are reports of increased performance when using compression. This is likely because compression moves the strain from your storage hardware to your cpu. - - Use a ramdisk (tmpfs in linux or freebsd 7). If you are not actively using your live profile, you could consider moving it to a tmpfs partition, this decreases the load on your storage hardware during the profiling stage and a tmpfs partition has a considerably higher throughput. But at the cost of having a very short and non-persistent live profile. - - Reconsider the used plugins. If the bottleneck is the plugin phase, you could consider disabling some of them or look into ways of increasing their performance. - - Upgrade your harddisks. If the bottleneck is your harddisk troughput, this can help solving your problem. - - Upgrade your cpu hardware. see above. - - Consider moving part of your profiles to another machine. If everything fails, just buy more hardware :) I hope this helps. Regards, Werner | | | | -----Original Message----- | From: [EMAIL PROTECTED] | [mailto:[EMAIL PROTECTED] On Behalf Of Werner | Schram | Sent: Wednesday, August 13, 2008 8:27 AM | To: [EMAIL PROTECTED] | Cc: [email protected] | Subject: Re: [Nfsen-discuss] nfprofile takes more than 5 min to complete | | Hi Dirk-Jan, | | Dirk-Jan van Helmond wrote: | | The nfprofile process takes more than 5 minutes to complete. | | I have 20 profiles, and about 90G/sec of data to be graphed. | | | | The machine used is a quad-core Xeon 2.4G, with 2GB of memory. | | | | | | Is there any possible way to increase the performance of nfprofile? | | We also had this problem. I solved it by patching nfsen to run multiple | instances of nfprofile parallel. It is a simple patch as nfprofile can | actually run in parallel very well by design. I actually promised to | submit this patch on several occasions, but I never got to it. I posted | it in the patches section of the nfsen project on sourceforge: | | https://sourceforge.net/tracker/index.php?func=detail&aid=2049518&group_ | id=134525&atid=730182 | | If you apply this patch, you can add a $PROFILERS=x variable in your | nfsen.conf, where you can set x to the amount of nfprofile processes you | want to use. We keep this number at least one below the number of | available cores, because this allows the OS to take control of the | remaining one to process the IO. We haven't tested if this is actually | necessary, as this gives us enough processing time either way. | | regards, | Werner Schram - ------------------------------------------------------------------------ - - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkjE7XkACgkQ3ULkMS4OADnlGgCfclPweV2uBKc+Ypk/n3pMf1zo fy4AoKExd/hMMghVelF6/ZHcfrfLGEdq =0XrQ -----END PGP SIGNATURE----- ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
