No virus scanner on this machine.
I dont see any large files being corrupted only small files like you see too.
The files we are having problems with is all small jpg files.

H.


Andy Schmidt wrote:
Hi,

Yes it started around this weekend - and, in our case too, those are small
JPEG/GIF thumbnail images of up to 4K (so probably exactly one allocation
unit).

I've asked my client to intentionally change the compression factor to
create the files slightly larger than 4096 bytes - to see if this theory
holds true.

It's pretty unlikely that no larger files would be effected by a hardware
error, considering that they should have a higher chance to be effected (due
to their larger size).

Are you using an on-access virus scanner? We use McAfee - just trying to
cover every base.

Best Regards
Andy Schmidt

Phone:  +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206


-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Heimir
Eidskrem
Sent: Wednesday, August 23, 2006 06:46 PM
To: declude.junkmail@declude.com
Subject: Re: [Declude.JunkMail] OT: Disk pattern 0xDF in files


we are having the exact problem on one of our servers.
We create small thumbnail pictures about 4k in size.
They work fine at first but later they are corrupted.

Windows 2000 server.

I have no clue what it could be at this time.
It started around this weekend I think.

Please keep me posted if you find something.

H.


Andy Schmidt wrote:
Hi,

I have two older servers (but not same models or same purchase years) running Windows 2000 with mirrored disks (software Raid-1).

Two days ago a customer noticed that they uploaded files to their FTP space, and initially they see the files on the browser - but a while later the data is corrupted.

I investigated - and oddly enough the problem so far always seems to appear with small thumbnail graphics files that occupy less than 4095 bytes. When I inspect the files I may see the "correct" data through a share, but if I access the files through some other method, I always see the byte pattern of 0xDF.

I ran a standalone checkdisk a day ago against the first server, sure enough, it reported and fixed several problems "Windows replaced bad clusters in file xxxx". But, the problem recurred the next day.

Now, my first instinct was that ONE of the two mirrored disks was truly on its way out and depending on which drive was being used to read the data it would either get good or bad data.

However, a day later a second customer had the same complaint but on an entirely different machine. In this case, the error occurs with a set of relatively new SCSI drives (not even a year old).

So now that I'm looking at two totally different server models, from entirely different years, one with fairly new disks - what are the chances that the SAME problem and symptom would show at the same time. Both on software mirrored disks, in both cases files that are less than 4 MB large.

Now I'm wondering if this is some "software" issue.

Best Regards
Andy Schmidt

Phone:  +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206

-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Barker
Sent: Wednesday, July 12, 2006 03:53 PM
To: declude.junkmail@declude.com
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20 anew

When the decludeproc services start under your windows services and the first email is processed. A file call diags.txt is created in your \Declude directory. This should contain the version and diagnostics. The valid options on decludeproc from the cmd prompt are:

Decludeproc -v   displays the version and build

Decludeproc -i   installs the decludeproc service

Decludeproc -u   uninstalls the decludeproc service

David B
www.declude.com


From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Schmidt
Sent: Wednesday, July 12, 2006 3:43 PM
To: declude.junkmail@declude.com
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20 anew


Dave -
That's what I call catch 22: D:\IMail>decludeproc -diag
Invalid command line parameter:
-install     Install Declude
-diag        Print diagnostics

Hm - so let's see, after "-install", I used "-diag" to figure out what's wrong. But, "-diag" is invalid. The ony valid parameters are... "-install" and "-diag"?


Best Regards
Andy Schmidt

Phone:  +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
________________________________

From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Schmidt
Sent: Wednesday, July 12, 2006 03:09 PM
To: declude.junkmail@declude.com
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20 anew


Hi Dave,
thanks. Next question: I noticed that your Virus.CFG is missing two options from Version 2: AUTOFORGE ON BANEZIPEXTS ON If I recall correctly, the idea was that:
BANZIPEXTS OFF
# BANEXT  EZIP
BANEZIPEXTS ON
would PERMIT banned extensions inside zipped files (where they could be scanned), but DENY banned extensions if they were contained inside encrypted zipped files. Where those options forgotten in your config file - or are they no longer available in Version 3?


Best Regards
Andy Schmidt

Phone:  +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
________________________________

From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Barker
Sent: Wednesday, July 12, 2006 02:43 PM
To: declude.junkmail@declude.com
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20 anew


The Program Files\Declude is a temp directory that can be deleted after the install. The original purpose of this directory was to make available the latest configs as we do not overwrite your configs. This has since been removed in version 4.x where you will find a \Declude\Resources directory which has the same purpose.

David B
www.declude.com

________________________________

From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Schmidt
Sent: Wednesday, July 12, 2006 2:36 PM
To: Declude.JunkMail@declude.com
Subject: [Declude.JunkMail] Trying to install Declude 3.1.20 anew


Hi,
I'm trying to set up a server from scratch and thus downloaded and ran: Declude_IM_N310.exe and chose the option to let it do its install (rather than the option for "experienced" admins). PS - that screen has a typo! The setup created a C:\Program Files\Declude folder that contains just the 5 config files it also created the SAME files
in:
D:\Imail\Declude together with binaries and the various other Declude files. I'm at loss! Which location is the "right" one for the config files (I'm assuming the D:\Imail\Declude)? What's the point of creating a "dummy" Folder in the C:\Program Files\ that contains no programs and that contains files that are not being used at all (assuming that being the case)? Should I be deleting this Program Files folder to avoid confusion when someone else maintains this server? Come on, the cold war has been over since Reagan - are we still trying to confuse the Russians?


Best Regards
Andy Schmidt

Phone:  +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
________________________________

From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Matt
Sent: Tuesday, May 23, 2006 03:25 PM
To: Declude.JunkMail@declude.com
Subject: Re: [Declude.JunkMail] Experience with 4.x


Andrew,

Thanks for your notes and their history.

I'm using the following settings right now:


        THREADS        30
        WAITFORMAIL    500
        WAITFORTHREADS        200
        WAITBETWEENTHREADS    100
        WINSOCKCLEANUP        OFF
        INVITEFIX    ON
        AUTOREVIEW        ON
        

There are a few reasons for trying these values.


THREADS 30 - I'm pretty confident that dual 3.2 Ghz Xeons and RAID can only handle 30 threads with average messages. In reality, one single message can spike the system to 100%, but these are uncommon. I figure that if I open this up too wide and I am dealing with a backup or something, launching more threads when at 100% CPU utilization will actually slow the system down. This was the same with 2.x and before. There is added overhead to managing threads and you don't want that to happen on top of 100% CPU utilization. I am going to back up my server later tonight to see if I can't find what the magic number is since I don't want to be below that magic number, and it would probably be best to be a little above it.
        
WAITFORMAIL 500 - On my server, this never kicks in, but if it did, it wouldn't make sense to delay for too long because I could build up messages. A half second seems good.
        
WAITFORTHREADS 200 - This apparently kicks in only when I reach my thread limit; sort of like a throttle. I don't want it to be too long because this should only happen when I am hammered, but it is wise not to keep hammering when you are at 100%. Sort of a mixed bag choice here.
        
WAITBETWEENTHREADS 100 - I see this setting as being the biggest issue with sizing a server. Setting it at 100 ms means that I can only handle 10 messages per second, and this establishes an upper limit
for what
the server can do.   I currently average about 5 messages per second
coming
from my gateways at peak hours, so I figured that to be safe, I should double that value.
        
INVITEFIX ON - I have it on because it comes on by default and I don't know any better. I know nothing about the cause for needing this outside of brief comments. It seems strange that my Declude setup could ruin an invitation unless I was using footers. If this is only triggered by footer use, I would like to know so that I could turn it off. I would imagine that this causes extra load to do the check.
        
AUTOREVIEW ON - I have this on for the same reason that Andrew pointed out. When I restart Decludeproc, messages land in my review folder, and I don't wish to keep manually fishing things out. If there is an issue with looping, it would be wise for Declude to make this only trigger say every 15 minutes instead of more regularly.
        

Feel free to add to this if you want.

Matt











Colbeck, Andrew wrote:

        I'd second that... on both the observed behaviour and the request
for
documentation.
I'm attaching my highly commented declude.cfg as a reasonable
sample.
Andrew 8)

________________________________

                From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Matt
                Sent: Tuesday, May 23, 2006 10:36 AM
                To: Declude.JunkMail@declude.com
                Subject: Re: [Declude.JunkMail] Experience with 4.x
                
                
                David,
                
                That did the trick.  I can't even see any messages in my
proc folder
any more. I might suggest adding your explanation to the comments in the file just in case others feel the need to turn this on like I did. I recalled the issues from the list and I turned it on because I didn't want the possibility of DNS crapping out and the leakage that this would cause.
                
                Here's a screen cap of what my processor graph looks like
now:
                
                
                
                

                Thanks,
                
                Matt
                
                
                
                David Barker wrote:

                        The purpose of WINSOCKCLEANUP        ON is to reset
the winsock, what
                        happens when using this setting is that when the
\proc directory hit 0
                        decludeproc will finish processing all the messages
in the \work before
                        checking the \proc again. As WINSOCKCLEANUP is to be
used only by those who
                        experience DNS issues I would suggest running your
tests again with
                        WINSOCKCLEANUP commented out and see how the
behavior differs. Also having
                        the WAITFORMAIL to low can cause the CPU to process
very high as it is
                        constantly checking the \proc I would suggest a
minimum of 500-1000
                        
                        David B
                        www.declude.com
                        
                        -----Original Message-----
                        From: [EMAIL PROTECTED]
                        [mailto:[EMAIL PROTECTED] On
Behalf Of Matt
                        Sent: Monday, May 22, 2006 8:12 PM
                        To: Declude.JunkMail@declude.com
                        Subject: Re: [Declude.JunkMail] Experience with 4.x
                        
                        Darrell,
                        
                        I put up two Windows Explorer windows side-by-side
under normal volume and the pattern was consistent where the proc folder grows while the work folder shrinks until the work folder hits zero at which point the proc folder empties out and everything lands in work and then the pattern repeats with proc growing while work
shrinks.
                        
                        My settings are as follows:
                        
                        THREADS        50
                        WAITFORMAIL    100
                        WAITFORTHREADS        10
                        WAITBETWEENTHREADS    50
                        WINSOCKCLEANUP        ON
                        AUTOREVIEW        ON
                        INVITEFIX    ON
                        
                        Matt
                        
                        
                        
                        
                        Darrell ([EMAIL PROTECTED]) wrote:
                        
                                        It's a faulty design that leaves
more than half a server's CPU capacity unused due to the mere fact that they wait for all threads to complete before moving in a new
batch.
                                I can't speak to what you see on your
server, but that is not how it is running on my server. I just double checked again to make sure I am not crazy, but as I watch the thread count on my server (decludeproc) the threads fluctuate between 7 - 30 ( threads currently set to 50). It is not uncommon to see the threads move as follow: 11,8,10,7,15,.... While I was watching it I never seen a case where it went down low enough for the WAITFORMAIL setting to kick in. Watching the proc/work directory you can see files moving in and out, but never really emptying out. Its possible what I am seeing is an anomaly or maybe I am interpreting it wrong.
                                
                                Maybe David can comment on this.
                                
                                Darrell
        
------------------------------------------------------------------------
                                invURIBL - Intelligent URI filtering plug-in
for Declude, mxGuard, and ORF. Stop spam at the source the spamvertised domain. More effective than traditional RBL's. Try it today - http://www.invariantsystems.com
                                ---
                                This E-mail came from the Declude.JunkMail
mailing list.  To
                                unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
                                type "unsubscribe Declude.JunkMail".  The
archives can be found
                                at http://www.mail-archive.com.
                                
                                
                        ---
                        This E-mail came from the Declude.JunkMail mailing
list.  To
                        unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
                        type "unsubscribe Declude.JunkMail".  The archives
can be found
                        at http://www.mail-archive.com.
                        
                        ---
                        This E-mail came from the Declude.JunkMail mailing
list.  To
                        unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
                        type "unsubscribe Declude.JunkMail".  The archives
can be found
                        at http://www.mail-archive.com.
                        
                        

---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.

---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.

---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.

---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.




---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.




---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.






---
This E-mail came from the Declude.JunkMail mailing list.  To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail".  The archives can be found at
http://www.mail-archive.com.




---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail".  The archives can be found
at http://www.mail-archive.com.






---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail".  The archives can be found
at http://www.mail-archive.com.

Reply via email to