Re: Need help with setting up MySQL storage for SA
Thanks for pointing me into the right direction. That helped me solve the issue. In fact there was no issue. I just had to run sa-learn -u user --spam miss-classified-message.txt. This command added some content into the Bayes tables and the warning in the debug output was gone. So that means, that actually I do not have to do any action on newly created users. Once they retrain their first message, the Bayes entries are getting created. Before that, Bayes is not used for that user. Is that correct? What about the autolearn functionality? Where are these infos getting stored? Is it also stored in the Bayes tables? What happens, if they are not yet initialised? Thanks, Michael Quoting Matteo Dessalvi mte...@yahoo.it: Hi. I did test a similar configuration a while ago and had the same problem. If you take a look at this thread on the mailing list: http://spamassassin.1065346.n5.nabble.com/Bayes-vars-records-on-MySQL-not-created-automatically-td104615.html you'll see it was a problem of running 'sa-learn --sync' as the user who is running the test. Best regards, Matteo On 20.08.2014 16:07, Michael wrote: Hi, I'm using Spamassassin in a virtual user environment. To store preferences like settings, Bayes and AWL for each user I'm trying to set up a MySQL storage. I created the MySQL tables according the instructions from the files awl_mysql.sql, bayes_mysql.sql, README.awl, README.bayes, README and userpref_mysql that came with my Spamassassin 3.4 installation on Ubuntu 14.04. The connection to the database seem to be working. For me the debug output looks like if Spamassassin would expect to be already some data in the tables. Where shall I get this data from? Do I have to manually create entries for each user? What am I missing? When calling spamc -u t...@michi.su testmail.txt I'm getting the following debug output (shortened): Aug 20 08:14:46.563 [16682] dbg: config: Conf::SQL: executing SQL: select preference, value from userpref where username = 't...@michi.su' or username = '@GLOBAL' order by username asc Aug 20 08:14:46.563 [16682] dbg: config: retrieving prefs for t...@michi.su from SQL server Aug 20 08:14:46.564 [16682] dbg: info: user has changed Aug 20 08:14:46.564 [16682] dbg: bayes: learner_new self=Mail::SpamAssassin::Plugin::Bayes=HASH(0x30fdce0), bayes_store_module=Mail::SpamAssassin::BayesStore::MySQL Aug 20 08:14:46.564 [16682] dbg: bayes: using username: t...@michi.su Aug 20 08:14:46.564 [16682] dbg: bayes: learner_new: got store=Mail::SpamAssassin::BayesStore::MySQL=HASH(0x3d1a768) Aug 20 08:14:46.565 [16682] dbg: bayes: database connection established Aug 20 08:14:46.566 [16682] dbg: bayes: found bayes db version 3 Aug 20 08:14:46.566 [16682] dbg: bayes: unable to initialize database for t...@michi.su user, aborting! The MySQL relevant options that I added are: user_scores_dsn DBI:mysql:spamassassin:localhost user_scores_sql_usernamespamassassin user_scores_sql_passwordpass bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username spamassassin bayes_sql_password pass auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList user_awl_dsnDBI:mysql:spamassassin:localhost user_awl_sql_username spamassassin user_awl_sql_password pass
Re: Help determining what is causing mails being marked as Spam
On Wed, 20 Aug 2014, Nicolás wrote: Ok, already done that, waited a few hours and now the 'correct' DNS host appears in the header, but is still marked as spam. On 20.08.14 20:14, Nicolás wrote: This would be the latest test: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on vps81276.ovh.net X-Spam-Level: X-Spam-Status: No, score=0.5 required=3.0 tests=ALL_TRUSTED,MISSING_DATE, MISSING_MID,T_DKIM_INVALID autolearn=no autolearn_force=no version=3.4.0 this does not say it's spam. it says it is NOT spam. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe.
Re: Help determining what is causing mails being marked as Spam
El 21/08/2014 11:07, Matus UHLAR - fantomas escribió: On Wed, 20 Aug 2014, Nicolás wrote: Ok, already done that, waited a few hours and now the 'correct' DNS host appears in the header, but is still marked as spam. On 20.08.14 20:14, Nicolás wrote: This would be the latest test: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on vps81276.ovh.net X-Spam-Level: X-Spam-Status: No, score=0.5 required=3.0 tests=ALL_TRUSTED,MISSING_DATE, MISSING_MID,T_DKIM_INVALID autolearn=no autolearn_force=no version=3.4.0 this does not say it's spam. it says it is NOT spam. I guess this is the header added by my own SpamAssassin for outgoing mails (vps81276.ovh.net is my server). However, the problem is still the same: Although this mail doesn't seem to be spam, it is always marked as it would be by Google. I just don't understand what makes Google think it is!
Re: Help determining what is causing mails being marked as Spam
Although this mail doesn't seem to be spam, it is always marked as it would be by Google. I just don't understand what makes Google think it is! If it is marked by Google, it has nothing to do with your SA. IMHO, Google is not making such a good job as marking spam, I get way to many FP (many mails about this list, because they talk about spam; system periodic mail, once a week, a couple of the nightly system security messages are classified as spam, while I have like 30 of same messages/day). Best regards, Olivier --
Re: Help determining what is causing mails being marked as Spam
On 08/21/2014 12:11 PM, Nicolás wrote: El 21/08/2014 11:07, Matus UHLAR - fantomas escribió: On Wed, 20 Aug 2014, Nicolás wrote: Ok, already done that, waited a few hours and now the 'correct' DNS host appears in the header, but is still marked as spam. On 20.08.14 20:14, Nicolás wrote: This would be the latest test: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on vps81276.ovh.net X-Spam-Level: X-Spam-Status: No, score=0.5 required=3.0 tests=ALL_TRUSTED,MISSING_DATE, MISSING_MID,T_DKIM_INVALID autolearn=no autolearn_force=no version=3.4.0 this does not say it's spam. it says it is NOT spam. I guess this is the header added by my own SpamAssassin for outgoing mails (vps81276.ovh.net is my server). However, the problem is still the same: Although this mail doesn't seem to be spam, it is always marked as it would be by Google. I just don't understand what makes Google think it is! Suggest you move this thread to the mailops list and/or contact Google.
Re: Help determining what is causing mails being marked as Spam
El 21/08/2014 11:16, Olivier Nicole escribió: Although this mail doesn't seem to be spam, it is always marked as it would be by Google. I just don't understand what makes Google think it is! If it is marked by Google, it has nothing to do with your SA. I know, I was just answering the previous mail. IMHO, Google is not making such a good job as marking spam, I get way to many FP (many mails about this list, because they talk about spam; system periodic mail, once a week, a couple of the nightly system security messages are classified as spam, while I have like 30 of same messages/day). Based on the amount of mail traffic that handles Google, I understand they tend to be as restrictive as they can, otherwise they would lose users. However, the scary thing is there is not (or at least I haven't found) any mechanism to make them know that some domain is used to send legitimate mail, so those mails are not marked directly as spam and at least are given a chance to gain some reputation. As I said, this domain has never had a compromised account nor has sent any junk mail, but still it is marked as spam. This is really frustrating, but at least now I know that the headers seem to be ok, I was wondering if I was missing something. Thanks. Best regards, Olivier
Re: Help determining what is causing mails being marked as Spam
On 08/21/2014 12:16 PM, Olivier Nicole wrote: Although this mail doesn't seem to be spam, it is always marked as it would be by Google. I just don't understand what makes Google think it is! If it is marked by Google, it has nothing to do with your SA. IMHO, Google is not making such a good job as marking spam, I get way to many FP (many mails about this list, because they talk about spam; system periodic mail, once a week, a couple of the nightly system security messages are classified as spam, while I have like 30 of same messages/day). Not really hard to make a filter to avoid that kidn of problems. In the case of this list we talk about crap, so it can be expected that a fair amount is treated like crap .-) And obviosuly, you get what you pay for...
Re: Help determining what is causing mails being marked as Spam
El 21/08/2014 11:26, Axb escribió: On 08/21/2014 12:16 PM, Olivier Nicole wrote: Although this mail doesn't seem to be spam, it is always marked as it would be by Google. I just don't understand what makes Google think it is! If it is marked by Google, it has nothing to do with your SA. IMHO, Google is not making such a good job as marking spam, I get way to many FP (many mails about this list, because they talk about spam; system periodic mail, once a week, a couple of the nightly system security messages are classified as spam, while I have like 30 of same messages/day). Not really hard to make a filter to avoid that kidn of problems. The problem is when you want to send e-mails to users that have not received yet a mail from your domain and don't have a rule for it. I want to send an e-mail to every user that just registered on my website, so this complicates the whole thing a bit more.
Re: sa-learn site-wide bayes on Redis
W dniu 20.08.2014 o 14:42, Axb pisze: On 08/20/2014 02:25 PM, Matteo Dessalvi wrote: Hi all. I am managing a bunch of Linux MTAs which are placed in front of some Exchange servers. In such a configuration the Bayes filter is deployed site-wide. For a new deployment of these servers I am planning to use Redis as a centralized backend (previously the bayes db were just files saved on the disk). My question is: do I have to use a specific option to tell sa-learn that the bayes db is now hosted on Redis? Or sa-learn will use the info from the bayes_sql_dsn directive in my local.cf? Looking into the wiki: http://wiki.apache.org/spamassassin/SiteWideBayesSetup or into the sa-learn docs: http://spamassassin.apache.org/full/3.4.x/doc/sa-learn.html did not give me any clues. see http://svn.apache.org/repos/asf/spamassassin/trunk/contrib/HOWTO.Bayes-Redis/ hope that helps. This is not an official doc, so if you see anything that needs to be added/changed, pls let me know. Hi! I'm reading bayes_redis.cf and I can see: #NOTE: We're not using authentication assuming the Redis server/port should not be reachable form the outside # You can add authentication once you've seen it work. Does it means that this example config doesn't include authentication options or it means that SA doesn't support auth for redis? Marcin
Re: sa-learn site-wide bayes on Redis
I am pretty sure SA support the Redis authentication mechanism. For my tests I have used the following line: bayes_sql_dsn server=127.0.0.1:6379;password=MySecretPWD;database=2 Matteo On 21.08.2014 12:56, Marcin Mirosław wrote: Hi! I'm reading bayes_redis.cf and I can see: #NOTE: We're not using authentication assuming the Redis server/port should not be reachable form the outside # You can add authentication once you've seen it work. Does it means that this example config doesn't include authentication options or it means that SA doesn't support auth for redis? Marcin
Re: Need help with setting up MySQL storage for SA
On 21.08.2014 09:20, Michael wrote: So that means, that actually I do not have to do any action on newly created users. Once they retrain their first message, the Bayes entries are getting created. Before that, Bayes is not used for that user. Is that correct? Yes, I would say it correct. To start using the bayesian filter SA needs at least to classify 200 spam and ham email otherwise the BAYES_* rules will not trigger What about the autolearn functionality? Where are these infos getting stored? Is it also stored in the Bayes tables? What happens, if they are not yet initialised? About the autolearning feature you can read about that here: http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_AutoLearnThreshold.html And about the info stored into the DB by SA you can take a look here: http://svn.apache.org/repos/asf/spamassassin/tags/spamassassin_current_release_3.4.x/sql/README.bayes Regards, Matteo Quoting Matteo Dessalvi mte...@yahoo.it: Hi. I did test a similar configuration a while ago and had the same problem. If you take a look at this thread on the mailing list: http://spamassassin.1065346.n5.nabble.com/Bayes-vars-records-on-MySQL-not-created-automatically-td104615.html you'll see it was a problem of running 'sa-learn --sync' as the user who is running the test. Best regards,
Re: sa-learn site-wide bayes on Redis
W dniu 21.08.2014 o 13:45, Matteo Dessalvi pisze: I am pretty sure SA support the Redis authentication mechanism. For my tests I have used the following line: bayes_sql_dsn server=127.0.0.1:6379;password=MySecretPWD;database=2 Thanks Matteo, firstly I should try then write to ML:) So now I did own check. It looks that SA doesn't authenticate when connects to redis. It didn't work for me with your example not when I used bayes_sql_password password When redis needs passowrd then SA throws bayes: Redis failed: Redis error: ERR operation not permitted, tcpdump also confirms that SA doesn't do AUTH. It's strange because in Redis.pm I can see that authentication is supported. Now I'm thinking where I could made mistake in configuration... Thanks, Marcin
Re: sa-learn site-wide bayes on Redis
Which version of Redis are you using? I did have some problems with the 2.4 version packaged by Debian and I did solve a similar problem using a more recent version, like the 2.7 or 2.8. Matteo On 21.08.2014 14:45, Marcin Mirosław wrote: W dniu 21.08.2014 o 13:45, Matteo Dessalvi pisze: I am pretty sure SA support the Redis authentication mechanism. For my tests I have used the following line: bayes_sql_dsn server=127.0.0.1:6379;password=MySecretPWD;database=2 Thanks Matteo, firstly I should try then write to ML:) So now I did own check. It looks that SA doesn't authenticate when connects to redis. It didn't work for me with your example not when I used bayes_sql_password password When redis needs passowrd then SA throws bayes: Redis failed: Redis error: ERR operation not permitted, tcpdump also confirms that SA doesn't do AUTH. It's strange because in Redis.pm I can see that authentication is supported. Now I'm thinking where I could made mistake in configuration... Thanks, Marcin
Fwd: [NOTICE] - Planned outage for VMWare Host. Affects Multiple VMs
FYI that the SA VM machine was added to the list and will be down shortly today. regards, KAM Forwarded Message Subject:[NOTICE] - Planned outage for VMWare Host. Affects Multiple VMs Date: Thu, 21 Aug 2014 10:51:35 +0100 From: Gavin McDonald ga...@16degrees.com.au To: operati...@apache.org operati...@apache.org CC: Infrastructure Apache infrastruct...@apache.org Hi All, This is 6 hours notice that RAM upgrades on one VMWare host machine is due to take place today (Thursday) at 1600 hours UTC. Estimated downtime would be around 1 hour to allow for graceful power down of each VM , RAM addition + testing, graceful and staggered power up of each VM. Apologies for the much shorter notice on this one, OSUOSL only just recently received the RAM and our hands-on guy there wanted to make sure it was physically in his hands before confirming a time to do this. Estimate then that this should take about 1 hour from start to finish. Non essential VMs may be turned off ahead of time to get ahead (thus causing less downtime for more essential services.) Notices nearer the time via @infrabot on twitter and #asfnfra irc and #infrabot on hip chat. status.apache.org http://status.apache.org/ will also be your friend during this time. Project mailing lists will NOT be notified. (Feel free to pass on this info.) Machines/VMs Affected:- === Increase RAM to Eirene Host from 262GB to 326GB (+64GB) airavata-vm bb-centos-vm bb-fbsd2 bb-solaris bb-vm3 bb-win7 bloodhound-vm2 circonus-broker continuum-vm cordova-vm couchdb-vm cwiki-vm devops devopstest id je-win2012 mail-relay ooo-wiki2 openmeetings-vm pkgrepo roller-vm spamassassin-vm struts-vm svnqa-vm2 sysconfig tac-vm2 tika-vm uima-ducc-vm vmgump Gav… (ASF Infra)
Re: BayesStore::Redis can't do AUTH when Redis is =2.6 (was: sa-learn site-wide bayes on Redis)
W dniu 21.08.2014 o 15:20, Matteo Dessalvi pisze: Which version of Redis are you using? I did have some problems with the 2.4 version packaged by Debian and I did solve a similar problem using a more recent version, like the 2.7 or 2.8. And you fixed my problem! Indeed, upgrading from redis-2.6.15 to 2.8.13 fixed problem with not working AUTH. Thanks Matteo!
RE: sa-update
How's this coming? I'm still showing July 28 as the last update. (Not complaining - I appreciate all you guys do!) ...Kevin -- Kevin Miller Network/email Administrator, CBJ MIS Dept. 155 South Seward Street Juneau, Alaska 99801 Phone: (907) 586-0242, Fax: (907) 586-4500 Registered Linux User No: 307357 -Original Message- From: John Hardin [mailto:jhar...@impsec.org] Sent: Tuesday, August 19, 2014 12:06 PM To: users@spamassassin.apache.org Subject: Re: sa-update On Tue, 19 Aug 2014, Scott Lewis wrote: Our sa-update's have not found fresh updates for a couple of weeks now. Does anyone know anything about this or are experiencing the same thing? It does not look like they are erring out. Masscheck corpus starvation, possibly due to timing issues. We're *almost* up to the lower limits needed to publish a rules update. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- You know things are bad when Pravda says we [the USA] have gone too far to the left. -- Joe Huffman --- 5 days until the 1935th anniversary of the destruction of Pompeii
Re: sa-update
On 08/21/2014 09:25 PM, Kevin Miller wrote: How's this coming? I'm still showing July 28 as the last update. (Not complaining - I appreciate all you guys do!) unless there's a new wave of starved masscheckers update should be published this weekend. Of couse, the project could always use more masscheck contributers Details about masschecks are in the SA Wiki
Re: Delays with Check_Bayes
On Wed, 20 Aug 2014, redtailjason wrote: We are seeing about 4000-7000 delayed messages per day. We do utilize a dedicated MySQL Server for the Bayes and all 8 scanners share it. Are you open to the possibility of upgrading to 3.4.0 and using the Redis backend for Bayes? (Just offering an alternative.) We have been developing and upgrade plan to 3.4. Based on this, we are prioritize this upgrade and will be expediting it. Thanks. Jason -- View this message in context: http://spamassassin.1065346.n5.nabble.com/Delays-with-Check-Bayes-tp111067p18.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Delays with Check_Bayes
On 21/08/14 09:00, John Hardin wrote: Are you open to the possibility of upgrading to 3.4.0 and using the Redis backend for Bayes? (Just offering an alternative.) We just last week moved over to 3.4.0 with a central Redis backend with 6 spamd servers spread over USA and Europe. Bit of a stretch in terms of WAN latency but it seems to be working really well. I love doing a spamc -L spam against one SA server and then immediately re-scanning the same message by a different one and seeing the BAYES_99 light up :-) So far, so good! Thanks for sharing this. We will take a look at that to see if it works for us. Regards, Jason -- View this message in context: http://spamassassin.1065346.n5.nabble.com/Delays-with-Check-Bayes-tp111067p19.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
RE: sa-update
On Thu, 21 Aug 2014, Kevin Miller wrote: How's this coming? I'm still showing July 28 as the last update. (Not complaining - I appreciate all you guys do!) From the latest run: On 8/20/2014 10:25 PM, UpdatesD Cron wrote: Exit Status 9 is not zero for do-nightly-rescore-example. HAM: 208428 (15 required) SPAM: 127817 (15 required) S close! From: John Hardin [mailto:jhar...@impsec.org] On Tue, 19 Aug 2014, Scott Lewis wrote: Our sa-update's have not found fresh updates for a couple of weeks now. Masscheck corpus starvation, possibly due to timing issues. We're *almost* up to the lower limits needed to publish a rules update. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Judicial Activism (n): interpreting the Constitution to grant the government powers that are popularly felt to be needed but that are not explicitly provided for therein (common definition); interpreting the Constitution as it is written (Brady definition) --- 3 days until the 1935th anniversary of the destruction of Pompeii
Re: Delays with Check_Bayes
On Thu, 2014-08-21 at 13:13 -0700, redtailjason wrote:
Are you open to the possibility of upgrading to 3.4.0 and using the Redis
backend for Bayes? (Just offering an alternative.)
We have been developing and upgrade plan to 3.4. Based on this, we are
prioritize this upgrade and will be expediting it. Thanks.
Thanks for including the part you're directly referring to, as I
requested. However, please do distinguish the quoted part from your
comments. The first paragraph actually was written by John, but your
post lacks any hint of the author, and even worse displays the quote and
your text visually identical.
See the difference between your latest two posts and any other post in
this thread?
I blame Nabble for even making this possible. In a reply, the quoted
text must be visually distinctive. More reason to avoid Nabble.
View this message in context:
http://spamassassin.1065346.n5.nabble.com/Delays-with-Check-Bayes-tp111067p18.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Sic. This is a mailing list. And Nabble a third-party list archive
service and poor forum-style web frontend to the mailing list.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
