Good day Thomas. Thank you for the reply. Sorry for the super long delay replying back on this. I have a few more questions just to clear some things up.
Looking at the spamdb file in the ASSP folder I several common words with some numbers after it that I would assume are a score of sorts. Upon further inspection I see a bunch of entries for users we host and domain names listed in here. My concerns here are that this is what is causing the issue with so many legitimate emails getting trapped. I have seen a infux of emails requesting to have an email address safe listed sent to a special account I have setup that is unfiltered. In the past I maybe got a dozen a month, at the present time I could easily get that in a few days. So my questions are this, Is this the proper way to go to get this under control? 1. Clean out the spam, notspam, errors folders and get a good collection going again 2. Once I have enough collected (a week or so would be suffice with the amount of users we have now) perform a rebuild How can I go about cleaning out the MySQL database? Just delete it and recreate it? I just want to make sure I go about this the smoothest way I can so I don't impact my customers too much with spam issues. I am just looking for some guidance and advice. Thank you in advance. On 7/24/2015 2:01 AM, Thomas Eckardt wrote: >> So if I grab a copy maybe from a >> previous backup and start the database over again I would think I would >> be good right? > If 'ReplaceOldSpamdb' is set, the rebuildspamdb process will overwrite the > complete spamdb database. > The only way to get better detection results by the HMM and Bayesian is to > maintain the corpus. Advice your users to report SPAM. Select > 'DelResendSpam' to keep assp reacting on blockreport resends. > >> I would think if the database was way out of wack I would not getting a > confidence rating like I am. > > The corpus norm and confidence values shown in the rebuild report are > mathematical values based on the COUNT of the processed HMM sequences and > Bayesian pairs, > They do NOT show the logical sense of the corpus from a human point of > view. If your corpus is not maintained in a proper way, but contains > enough files (words), the norm and confidence will be fine, but the > detection rate will be bad. > >> I am just wondering if some >> of this might be poisoned information from when we had an issue with a >> user's email account sending spam. > Yes, if outgoing mails are stored in the corpus. > > > > Thomas > > > > > Von: Jay <[email protected]> > An: [email protected] > Datum: 24.07.2015 00:09 > Betreff: Re: [Assp-user] ASSP spam database question - MYSQL > > > > We have been running ASSP for quite a few years now, but only started > using MYSQL as the database in the past year. So why all of the sudden > now are we seeing this issue. I do a rebuild once a week and the corpus > confidence comes out at 1.00000000. > > Here's a snippet from last rebuild 2 days ago: > > Jul-21-15 11:42:31 Spam Weight: 2,333,146 > Jul-21-15 11:42:31 Not-Spam Weight: 2,333,802 > > Jul-21-15 11:42:31 Corpus norm: 0.9997 - (very good - balanced) > Jul-21-15 11:42:31 Corpus confidence: 1.00000000 > > I would think if the database was way out of wack I would not getting a > confidence rating like I am. Looking at the row count from PHPMyAdmin > the spamdb 1,237,310 rows in the database. I am just wondering if some > of this might be poisoned information from when we had an issue with a > user's email account sending spam. I know our whitelist database had to > be recovered from a backup because it grew 3 times the size from what it > was when the spam issue was going on. Hence my question about taking > care of the MySQL database now. So if I grab a copy maybe from a > previous backup and start the database over again I would think I would > be good right? > > On 7/23/2015 4:11 PM, Data Packet Networks wrote: >> You may wish to set Bayesian filtering to "monitor" until you get it >> under control. >> >> Go through your spam and not spam folders and be sure each has >> appropriate messages for Bayesian training. If a user sent spam be sure >> you remove those spam messages from the not spam folder and rebuild the >> db. White listing would not hurt in your case as those messages that >> were incorrectly identified would be placed in your not spam folder for >> future Bayesian training. >> >> You could clear the Bayesian table of your MySQL database, but i would >> just white-list to gather more non spam messages and then rebuild. >> >> On 7/23/2015 3:09 PM, Jay wrote: >>> Hey all. I have a question. Recently I have noticed that the spam > filter >>> has been blocking emails that otherwise would seem legitimate. So for >>> instance I see a lot of weird behavior from the blocked report: >>> >>> spam reason: (Bayesian) [I need a quote for laminate flooring install > in >>> Lakeland FL] >>> spam reason: (Bayesian) [Inv 5895] >>> spam reason: (Bayesian) [Invoice 10735] >>> spam reason: (Bayesian) [Aflac Policyholder Services Email Address >>> Verification Required] >>> spam reason: (Bayesian) [Payment Confirmation for Support] >>> spam reason: (Bayesian) [Your Progressive auto quote confirmation] >>> >>> These are just a few examples I found. There are others from banks, >>> insurance companies, customers, payroll services etc. Looking at some > of >>> the emails that got blocked it doesn't seem to me that they should > have. >>> If I use the built in analyzer ASSP thinks they are spam. I know I can >>> just whitelist these, but if the users don't inform me or they don't >>> check the blocked report on a daily basis they have no idea. We have > the >>> spam filter set up that if a email gets rejected it sends a response >>> email back to the user with instructions on how to get safe listed. >>> Basically they contact an email address at my office that is unfiltered >>> and request to be whitelisted. The biggest issue we deal with is that >>> either the originating email address is a unmonitored email address or >>> the user just ignores it. I know I will run into this from time to time >>> still, but what concerns me is that our spam database is poisoned and >>> flagging emails that would have otherwise gone through previously. It >>> seems within the last 2 months this issue has cropped up. We did have > an >>> issue where one of our users email got compromised and sent out spam. > So >>> I am thinking this is where the issue started and poisoned the spam >>> database. >>> >>> We currently use MYSQL as the database for the spam database. Is there > a >>> easy way to just wipe the database clean and start over from a previous >>> spamdb file? Is this as easy as just grabbing the spamdb from one of > the >>> ASSP installations and dropping it into the import folder for MYSQL? Or >>> is this the case where I have to purge the database in MYSQL and >>> recreate it fresh. Any help/advice is greatly appreciated. Thanks. >>> >>> >>> > ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Assp-user mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/assp-user >> > ------------------------------------------------------------------------------ >> _______________________________________________ >> Assp-user mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/assp-user >> >> >> ----- >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2015.0.6081 / Virus Database: 4392/10291 - Release Date: > 07/23/15 >> > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-user mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/assp-user > > > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-user mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/assp-user > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2015.0.6081 / Virus Database: 4392/10291 - Release Date: 07/23/15 > > ------------------------------------------------------------------------------ _______________________________________________ Assp-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/assp-user
