So basically, what you are saying is that my volume is really too low to take advantage of the persistent sniffer (and such may actually decrease my performance), and I should stick with the non-service version. Is that right? That is about what I thought (without the details of how sniffer works, I just wanted to be sure).
Thanks, Pete. Dan Horne > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil > Sent: Tuesday, August 02, 2005 4:09 PM > To: Dan Horne > Subject: Re[2]: [sniffer] Sniffer taking a long time? > > After following through all of this and looking at the .stat > file, I think I see what's going on. > > Now that it is running and producing a .stat file, the flow > rate is very low. According to the stat data, about 6 msgs / minute. > > Note the poll and loop times are in the 450 - 550 ms range. > > SNF with the persistent engine is built for high throughput, > but it's also built to play nice. > > The maximum poll time gets up to 2 seconds or so (sound familiar?) > > If there are no messages for a while, then everything slows > down until the first message goes through. For that first > message, the SNF client will probably wait about 2 seconds > before looking for it's result because that's what the stat > file will tell it to do. > > Since the next message probably won't come around for a few > seconds, that next message will probably wait about 2 seconds also. > > If you were doing 6 messages a second then all of the times > would be much lower and so would the individual delays. > > When you turn off the persistent instance, each new message > causes a client to look and see if there are any other peers > acting a servers... Since the messages are far and few > between, the client will elect to be a server (momentarily), > will find no work but it's own, will process it's own message > and leave. -- This is the automatic peer-server mode. It will > always work like this unless more than one message is being > processed at the same moment. > > In peer-server mode, since there is nothing else going on and > no persistent instance to coordinate the operations, each > message will get processed as fast as the rulebase can be > loaded and then the program will drop. > > When the persistent instance is introduced, it sets the pace > - and sicne there are no other messages, each client will > wait about 2 seconds (or half a second or so with the .stat > file contents you show) before it begins looking for it's results. > > The server instance will also wait a bit before looking for > new jobs so that the file system isn't constantly being scanned. > > Of course, if a burst of messages come through then the > pacing will speed up as much as necessary to keep up with the volume. > > Hope this helps, > > _M > > On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote: > > DH> No, I followed your instructions exactly (and not for the first > DH> time). I didn't add those extra values until today. Prior to > DH> adding the AppDirectory value, the service was taking a minute to > DH> scan emails; after adding it the scan time went to around 2 > DH> seconds. I can't get it any lower than that. Initially > mine was > DH> set up exactly as you said, with only "Application" > containing the > DH> path, authcode and persistent. Today after hearing no > suggestions > DH> from the list, and based on recent list messages > mentioning the home > DH> directory for the service, I looked at the srvany.exe > doco to find > DH> out how to give it a home directory. > DH> That's when I added AppDirectory. I also saw and added > DH> AppParameters at the same time and added those as well, > though they > DH> seem not to be needed. > DH> > DH> Prior to adding the AppDirectory value, I never got any > .stat file > DH> or any .SVR file in my sniffer dir. After adding that value and > DH> starting the service those files appeared. > DH> > DH> > > > DH> From: [EMAIL PROTECTED] > DH> [mailto:[EMAIL PROTECTED] On Behalf Of Matt > DH> Sent: Tuesday, August 02, 2005 3:24 PM > DH> To: sniffer@SortMonster.com > DH> Subject: Re: [sniffer] Sniffer taking a long time? > > > > > DH> Dan, > > DH> There is no AppDirectory value on my server either. The > DH> Parameters key has only one value under it besides Default > DH> which is "Application", and it contains exactly what I provided > DH> below. Could it be that you tried to hard to get everything > DH> right by tweaking these additional keys? > > DH> Something else. Did you make sure that the Sniffer > DH> service that you created was started? No doubt it will work if > DH> you follow those directions to a T, and there aren't any issues > DH> with your server apart from this. > > DH> Matt > > > > DH> Dan Horne wrote: > > > > DH> I removed the AppParameters value and put the authcode > DH> and persistent back in the Application value where it was before. > DH> It didn't make any difference at all in the processing time, > DH> still right around 2 seconds. I don't know how your setup is > DH> working without at least the AppDirectory value, because mine > DH> didn't start working until I put that in, but if it is, I > DH> can't argue. My server load isn't anywhere near yours, so I > DH> don't see what the problem could be with mine. Oh well, unless > DH> Pete responds with a suggestion, I guess I'll just keep using the > DH> non-service version. > DH> > DH> Thanks anyway. > > > > > DH> From:[EMAIL PROTECTED] > DH> [mailto:[EMAIL PROTECTED] On Behalf Of Matt > DH> Sent: Tuesday, August 02, 2005 2:37 PM > DH> To:sniffer@SortMonster.com > DH> Subject: Re: [sniffer] Sniffer taking a long time? > > > DH> Dan, > > DH> I seem to recall trying to use the AppParameters key > DH> and having difficulty with it. I think that you might want > DH> to try removing that key and putting everything in the > DH> Parameters key, or at least that works for me. If you change > DH> > HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters > DH> in RegEdit to the following it might fix the issue that you > DH> are having: > > DH> C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe ***AUTH-CODE*** > DH> persistent > > DH> You should of course adjust the path and service > name as well. > > DH> The directions that I provided are working perfectly > DH> on my server so far as I can tell. I'm running dual 3.2 > DH> Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. > DH> The following three debug log entries shows between 300 ms and 550 > DH> ms per message: > > DH> 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] > DH> SNIFFER-IP: External program started: > DH> C:\IMail\Declude\Sniffer\executable.exe auth-code > DH> F:\\DB93D976201222A43.SMD > DH> 08/02/2005 14:19:47.676 QB93D976201222A43 [2616] > DH> SNIFFER-IP: External program reports exit code of 61 > DH> ----------------- > DH> 08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] > DH> SNIFFER-IP: External program started: > DH> C:\IMail\Declude\Sniffer\executable.exe auth-code > DH> F:\\DB9418A4800EC2A49.SMD > DH> 08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] > DH> SNIFFER-IP: External program reports exit code of 51 > DH> ----------------- > DH> 08/02/2005 14:19:49.879 QB943711501382A4D [6388] > DH> SNIFFER-IP: External program started: > DH> C:\IMail\Declude\Sniffer\executable.exe auth-code > DH> F:\\DB943711501382A4D.SMD > DH> 08/02/2005 14:19:50.176 QB943711501382A4D [6388] > DH> SNIFFER-IP: External program reports exit code of 59 > > DH> My stat file shows the following: > > DH> TicToc: 1122992104 > DH> Loop: 154 > DH> Poll: 0 > DH> Jobs: 118392 > DH> Secs: 155137 > DH> Msg/Min: 45.7887 > DH> Current-Load: 24.4275 > DH> Average-Load: 23.8719 > > DH> I'm not sure why people use FireDaemon for this. My > DH> experience with SRVANY.exe has been absolutely flawless > DH> since I integrated this, and it has worked on both Win2k and > DH> Windows 2003. > > DH> Matt > > > > > > DH> Dan Horne wrote: > DH> OK, I have managed to get SOMETHING working, but it still > seems too > DH> slow and something is still not right. I originally set up the > DH> persistent sniffer using the instructions from this > DH> > post:http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.ht > DH> mlThis uses SRVANY.exe. I conjectured that possibly the service > DH> needed a home directory, so I added an AppDirectory value to the > DH> sniffer service's "Parameters" key in the registry. This > value is > DH> set to the directory sniffer resides in. I also (based on my > DH> reading of the srvany.exe documentation) added another > value to the > DH> same key called AppParameters. This is set to my auth > code followed > DH> by a space, followed by the word persistent. > > DH> Now when I start the service, the time spent processing a single > DH> message goes down to something around 2 seconds, but is still far > DH> longer than the non-service version. I also still had no > .stat file > DH> in my sniffer directory. I did get a *.SVR file, which I > never got before. > > DH> So then I'm thinking, let's just make sure that I have the latest > DH> version of sniffer. I downloaded that, did the necessary > renaming > DH> of the files and then started the service. NOW there is a > DH> *.persistent.stat file. However, the scan time is still > at around 2 > DH> seconds. > > DH> Average Scan times (based on average scan times of 5 emails each): > DH> Without sniffer service running: .033 seconds With > sniffer service > DH> running: 2.244 seconds > > DH> The *.persistent.stat file has the following contents: > > DH> TicToc: 1122990610 > DH> Loop: 512 > DH> Poll: 445 > DH> Jobs: 34 > DH> Secs: 303 > DH> Msg/Min: 6.73267 > DH> Current-Load: 8.69565 > DH> Average-Load: 10.6371 > > DH> Any suggestions? > > DH> Thanks, > DH> Dan Horne > > DH> This E-Mail came from the Message Sniffer mailing list. For > DH> information and (un)subscription instructions go to > DH> http://www.sortmonster.com/MessageSniffer/Help/Help.html > > > DH> -- > DH> ===================================================== > DH> MailPure custom filters for Declude JunkMail > DH> > Pro.http://www.mailpure.com/software/=============================== > DH> ====================== > > > > > This E-Mail came from the Message Sniffer mailing list. For > information and (un)subscription instructions go to > http://www.sortmonster.com/MessageSniffer/Help/Help.html > This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
