Yep. Despite the fact that one could design a process to work properly
with the -N option (also leaving the old file for comparison), since
this is generally scheduled by users on the hour, it would still
produce a run on the bandwidth at the top of some or even every hour.
Enforcing a time bracket is not realistic.
Using the program alias is the best way all around for now, and I
believe that this should be promoted as the only option for IMail users
at least. It appears that Pete times his notifications so that it
doesn't produce backups, and I assume that notifications are sent
immediately upon publishing the new customized rulebases, so it is also
the fastest method to achieving an update.
The code is there, but I just think that it can be better variablized
to adjust for different directories and codes, so it fits appropriately
in everyone's config. Packing this together with gzip and including
that in the default setup would also be seemingly preferable. Throwing
together a how-to that was written for the lowest common denominator
would enhance the ease of use for many (pictures are nice where
appropriate), and would help with reducing support.
Matt
Woody G Fussell wrote:
Why do you not use a "program alias" and only download when you receive
notification that a new rule base is available? If everyone used gzip and
only downloaded when notified the bandwidth could be controlled by
staggering the notifications.
Woody Fussell
Wilbur Smith Associates
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Jim Matuska
Sent: Tuesday, December 28, 2004 12:49 PM
To: [email protected]
Subject: Re: [sniffer] Downloads are slow...
I agree that something needs to be done about the update scripts that are
inadvertently downloading the full rulebase all the time. I didn't even
know it but we were doing this until I went through our update script again
this morning and found it didn't have the -N option in Wget, so we were
downloading the entire rulebase whether we needed it or not. The gzip
compression is cool, and I will likely implement it soon, but I think the
major problem is everyone that is using scripts that keep downloading the
same file over and over again tying up the bandwidth.
I would recommend 2 things to help alleviate this problem:
1. Monitor connections to rulebase downloads to see who is downloading the
rulebase everytime they connect on a schedule to determine who has their
scripts setup wrong, and contact them to correct it. It took me under a
minute to add the -N option to wget, it should be a no brainer.
2. Correct the scripts posted on the Sniffer website to include date
checking, and possibly gzip compression, I used one of those scripts for our
system and assumed it would be setup correctly, but it was not.
Jim Matuska Jr.
Computer Tech2, CCNA
Nez Perce Tribe
Information Systems
[EMAIL PROTECTED]
----- Original Message -----
From: "Matt" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Monday, December 27, 2004 10:03 PM
Subject: Re: [sniffer] Downloads are slow...
I agree entirely. If bandwidth has become an issue, it would be resolved
with a focus on producing very tight and easily customizable scripts (a
variables section in the top of the scripts). I believe that going the
VBScript route might be the best way to go, or at least I believe that more
of us can hack a more involved VBScript than a batch or CMD file.
Enforcing compressed downloads and checking for timestamps prior to
downloading should be done in these scripts as well.
Right now the script examples assume a familiarity with scripting, and
while local participants can mostly handle that stuff, the non-vocal ones
are most likely to not even be aware of the issues or how to fix them, and
might have scripted timed downloads because it is definitely the easiest
way to go. This is probably the majority of the customer base. There is
an impression for instance with Declude's user base that +80% use
primarily the default config which most of us know is severely lacking in
comparison to the potential that exists by tweaking the settings.
With better script examples and a careful step-by-step readme promoted in
a mailing to your customers, I believe that this issue could go away, or
at least theoretically it should.
Personally, I have mine tied to the E-mails, I download the zipped
versions, I don't bother checking on the status, and have never noticed
any issues as a result. It would be a small shame if I was missing
downloads due to timeouts, but not that big of a deal if this has never
caused a noticeable problem.
Matt
Andy Schmidt wrote:
Pete,
With all due respect - I think the download problem is "self-inflicted",
because your web site is providing unsuitable examples to your customers!
Even with moderate bandwidth, your server would be able to handle tens of
thousands of hits a day. Checking if an updated file exists should barely
be noticeable - as long as it doesn't result in an unnecessary download.
You probably suffer TWO problems:
A) Most of your customers are downloading rules based on a schedule, even
if
no rules exists. Potential savings: 100% per download attempt.
B) Your customers are not downloading "compressed" rule files. Potential
savings: about 66%, but that's not bad either.
One likely explanation is that at least THREE of your sample scripts do an
unconditional and uncompressed download! Here the 3 URLs you list on your
web site and WGET command they are using:
http://www.sortmonster.com/MessageSniffer/Help/UserScripts/david_snifferUp
da
teMethod.zip
wget http://www.sortmonster.net/Sniffer/Updates/xxxxxxxx.snf -O
xxxxxxxx.new
--http-user=username --http-passwd=password
http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Hank_SnifferScr
ip
ts.zip
wget http://www.sortmonster.net/Sniffer/Updates/xxxxxxxx.snf -O
xxxxxxxx.new
--http-user=sniffer --http-passwd=ki11sp8m
http://www.sortmonster.com/MessageSniffer/Help/UserScripts/Michiel_AutoUpd
at
e.zip
wget
http://sniffer:[EMAIL PROTECTED]/Sniffer/Updates/12345678.snf
-O
<serial>.tst
My recommendation: Replace these with examples that implement conditional,
compressed downloading.
Best Regards
Andy Schmidt
H&M Systems Software, Inc.
600 East Crescent Avenue, Suite 203
Upper Saddle River, NJ 07458-1846
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
http://www.HM-Software.com/
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Pete McNeil
Sent: Monday, December 27, 2004 08:10 AM
To: Chuck Schick
Subject: Re: [sniffer] Downloads are slow...
On Monday, December 27, 2004, 1:17:21 AM, Chuck wrote:
CS> Pete:
CS> It appears on weekends the sniffer downloads are really slow. I am CS>
downloading at 14 minutes past the hour and I am about 1/20 th of CS> the
normal speed.
That is an unusual observation - I don't think weekends have anything to
do
with making things slower. I will look at the logs to see if I can figure
out what heppened.
You're not manually downloading I hope?
_M
This E-Mail came from the Message Sniffer mailing list. For information
and
(un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html
This E-Mail came from the Message Sniffer mailing list. For information
and (un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
This E-Mail came from the Message Sniffer mailing list. For information
and (un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html
This E-Mail came from the Message Sniffer mailing list. For information and
(un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html
This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
|
