> -----Original Message----- > From: Werner Schram [mailto:[EMAIL PROTECTED] > Sent: Monday, September 08, 2008 5:17 AM > To: Brown, Robin > Cc: [EMAIL PROTECTED]; [email protected] > Subject: Re: [Nfsen-discuss] nfprofile takes more than 5 min > to complete > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Brown, Robin wrote: > | This is not working for us, it seems to spawn hundreds of > nfsend and > | nfprofile processes and logs these to syslog: > | > | Sep 7 13:35:41 munchies nfsen[21090]: 348 channels/alerts > to profile > | Sep 7 13:35:45 munchies nfsen[21090]: nfprofile failed: > Too many open > | files Sep 7 13:35:45 munchies nfsen[23128]: Use of uninitialized > | value in string ne at /var/local/nfsen/bin/nfsend line 303. > | Sep 7 13:35:45 munchies nfsen[23128]: Use of uninitialized > value in > | print at /var/local/nfsen/bin/nfsend line 304. > | Sep 7 13:35:45 munchies nfsen[23128]: Use of uninitialized > value in > | concatenation (.) or string at /var/local/nfsen/bin /nfsend > line 305. > | > | Using nfsen-1.3b-20070720 and we're exceeding the 5 minute > run time, > | this would help a lot if we could get it to work. > | > | -Robin > > Hi Robin, > > The reason for the amount of processes is most likely because > your system still doesn't have enough resources to run the > periodic updates in time. What you should start with is > looking at how long it actually takes for the periodic > updates to complete. In your debug log, you can see when the > profiling stages starts and ends, and when the plugin stage > starts and ends. Another important thing to look at if the > bottleneck is either your cpu capacity or harddisk > throughput. I am not an expert in assessing the performance > of a system, but the amount of processing time going to user > and to the system could give some pointers. Possible ways of > speeding things are: > > - - Do not set the $PROFILERS variable higher than the number > of available processing cores. All instances of nfprofile run > independently, so they all read the complete live profile > separately. So it only makes sense to run multiple instances > of nfprofile if they don't have to fight each other for > processing time. > - - Use compression. Peter has told me that there are reports > of increased performance when using compression. This is > likely because compression moves the strain from your storage > hardware to your cpu. > - - Use a ramdisk (tmpfs in linux or freebsd 7). If you are > not actively using your live profile, you could consider > moving it to a tmpfs partition, this decreases the load on > your storage hardware during the profiling stage and a tmpfs > partition has a considerably higher throughput. But at the > cost of having a very short and non-persistent live profile. > - - Reconsider the used plugins. If the bottleneck is the > plugin phase, you could consider disabling some of them or > look into ways of increasing their performance. > - - Upgrade your harddisks. If the bottleneck is your > harddisk troughput, this can help solving your problem. > - - Upgrade your cpu hardware. see above. > - - Consider moving part of your profiles to another machine. > If everything fails, just buy more hardware :) > > I hope this helps. > > Regards, > Werner >
Thanks Werner, compression did not help. I'll try deleting some profiles next. Then if I can get this in under 5 minutes I'll try the patch once more to see if it spawns over 1000 processes again. But should I be getting the 'Use of uninitialized value' errors in syslog when the patch is applied? -Robin > > | > | > | > | -----Original Message----- > | From: [EMAIL PROTECTED] > | [mailto:[EMAIL PROTECTED] On Behalf Of > | Werner Schram > | Sent: Wednesday, August 13, 2008 8:27 AM > | To: [EMAIL PROTECTED] > | Cc: [email protected] > | Subject: Re: [Nfsen-discuss] nfprofile takes more than 5 min to > | complete > | > | Hi Dirk-Jan, > | > | Dirk-Jan van Helmond wrote: > | | The nfprofile process takes more than 5 minutes to complete. > | | I have 20 profiles, and about 90G/sec of data to be graphed. > | | > | | The machine used is a quad-core Xeon 2.4G, with 2GB of memory. > | | > | | > | | Is there any possible way to increase the performance of > nfprofile? > | > | We also had this problem. I solved it by patching nfsen to run > | multiple instances of nfprofile parallel. It is a simple patch as > | nfprofile can actually run in parallel very well by design. > I actually > | promised to submit this patch on several occasions, but I > never got to > | it. I posted it in the patches section of the nfsen project > on sourceforge: > | > | > https://sourceforge.net/tracker/index.php?func=detail&aid=2049518&grou > | p_ > | id=134525&atid=730182 > | > | If you apply this patch, you can add a $PROFILERS=x > variable in your > | nfsen.conf, where you can set x to the amount of nfprofile > processes > | you want to use. We keep this number at least one below the > number of > | available cores, because this allows the OS to take control of the > | remaining one to process the IO. We haven't tested if this > is actually > | necessary, as this gives us enough processing time either way. > | > | regards, > | Werner Schram > > - > -------------------------------------------------------------- > ---------- > - - > This SF.Net email is sponsored by the Moblin Your Move > Developer's challenge Build the coolest Linux based > applications with Moblin SDK & win great prizes Grand prize > is a trip for two to an Open Source event anywhere in the > world http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Nfsen-discuss mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.9 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkjE7XkACgkQ3ULkMS4OADnlGgCfclPweV2uBKc+Ypk/n3pMf1zo > fy4AoKExd/hMMghVelF6/ZHcfrfLGEdq > =0XrQ > -----END PGP SIGNATURE----- > ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
