Re: SA bayes file db permission issue
On Sat, 11 Jun 2016, RW wrote:
On Fri, 10 Jun 2016 15:38:44 -0400
Joseph Brennan wrote:
This is a nice test I found:
echo -n I | od -to2 | awk '{ print substr($2,6,1); exit}'
1 little-endian
0 big-endian
I don't see how this can output anything other than 1.
Endianness is about the addressing of bytes within integer words. This
is looking at the ordering of human-readable octal digits displaying
the contents of a single byte.
On big-endian system:
$ echo -n I | od -to2
000044400
001
On little-endian system:
# echo -n I | od -to2
000000111
001
So it works.
It's a single data byte but since the display field is a two byte
object, where within that two byte object does that single byte show up?
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: Slightly offtopic: postfix/spamass-milter or CentOS 7
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jari Fredriksson kirjoitti 12.6.2016 2:00: > Hellos. > > I tried to ask @ #freenet #postfix but somehow that does not allow me > send, no matter that I'm registered and identified myself... > > I have small but blocking issue on this new box: the case is described > at http://pastebin.ca/3623877 > > Any ideas? There are similar stuff obviously in the net, I did google, > but those do not help. This is solved, closed it is. Got it working. The #irc I mean ;) And got help to postfix from them. - -- Jari Fredriksson Bitwell Oy +358 400 779 440 ja...@bitwell.biz https://www.bitwell.biz - cost effective hosting and security for ecommerce -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAldc3LIACgkQKL4IzOyjSrZd8gCg72mPkgDTAszUxVpX3l/XW5O2 u8IAn2TqMw+Vi8P1TI3DxkPJyJ6+fdbl =fHYO -END PGP SIGNATURE-
Slightly offtopic: postfix/spamass-milter or CentOS 7
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hellos. I tried to ask @ #freenet #postfix but somehow that does not allow me send, no matter that I'm registered and identified myself... I have small but blocking issue on this new box: the case is described at http://pastebin.ca/3623877 Any ideas? There are similar stuff obviously in the net, I did google, but those do not help. - -- Jari Fredriksson Bitwell Oy +358 400 779 440 ja...@bitwell.biz https://www.bitwell.biz - cost effective hosting and security for ecommerce -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAldcmCYACgkQKL4IzOyjSraztgCgttezsXMhM1Sntk0iTNFdvVXT I7YAoK4p5DgFq39QxVWBN4M/1DLTsX0T =BanX -END PGP SIGNATURE-
Re: why does that mail not get any bayes-classification
Am 12.06.2016 um 00:27 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 9:31 AM: headers don't help when you have a "spamd: result" log-line with a ton Ah, finally I understand what you are trying to do! You analyze the spamd result log lines, and they currently have two deficiencies: 1) They do not distinguish between Bayes failing and Bayes simply not finding significant tokens in the message; 2) They don't provide a way of matching the result log line to the actual message when the message does not contain an mid. Your proposed solution of a BAYES_NOTOK rule would solve the first one. The second is a bit trickier. Really there ought to be a way to configure custom output in the spamd result log line, or to have a rule that can include some information in addition to its name and score in its report both would require small changes in SA itself the second is not really trickier, when the code is able to put the MID in the log line than it has also the informations about sender/rcpt and just don't put it into the logs signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 9:31 AM: > > headers don't help when you have a "spamd: result" log-line with a ton Ah, finally I understand what you are trying to do! You analyze the spamd result log lines, and they currently have two deficiencies: 1) They do not distinguish between Bayes failing and Bayes simply not finding significant tokens in the message; 2) They don't provide a way of matching the result log line to the actual message when the message does not contain an mid. Your proposed solution of a BAYES_NOTOK rule would solve the first one. The second is a bit trickier. Really there ought to be a way to configure custom output in the spamd result log line, or to have a rule that can include some information in addition to its name and score in its report. Sidney
Re: why does that mail not get any bayes-classification
Am 11.06.2016 um 23:26 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 9:08 AM: and it's not worth to discuss since the *real* solution would be a "BAYES_NOTOKS" which would appear *everywhere* and clearly explain why no other BAYES_XX is present I can't argue with that. Without the ability to make it a rule or a meta-rule that would only show if there are no BAYES_NN rules triggered, adding the tags to the report unconditionally would not be the same as a BAYES_NOTOK rule. As far as I can tell implementing BAYES_NOTOK would require a (small) change in the Bayes plugin. It could not be done in the configuration file or by writing a new rule. So you are right about that. that's exactly the point However, what you said about _SENDERDOMAIN_ and _AUTHORDOMAIN_ could be handled by adding a report line that contains those tags just before or just after the report _SUMMARY_ line in the configuration headers don't help when you have a "spamd: result" log-line with a ton of rules or a new rule you are trying out appears when the message has no message-id since your only anchor is the mid=<> part of the logline from which you can grep the other relevant MTA lines and find out who was the sender, who was the rcpt and from where did that message arrive at all keep in mind: you get all that headers only in your own mails, they are not helping you much as sysadmin for a lot of users where you try to find out if rules needs to be rescored in whatever direction signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 9:08 AM: > > and it's not worth to discuss since the *real* solution would be a > "BAYES_NOTOKS" which would appear *everywhere* and clearly explain why > no other BAYES_XX is present I can't argue with that. Without the ability to make it a rule or a meta-rule that would only show if there are no BAYES_NN rules triggered, adding the tags to the report unconditionally would not be the same as a BAYES_NOTOK rule. As far as I can tell implementing BAYES_NOTOK would require a (small) change in the Bayes plugin. It could not be done in the configuration file or by writing a new rule. So you are right about that. However, what you said about _SENDERDOMAIN_ and _AUTHORDOMAIN_ could be handled by adding a report line that contains those tags just before or just after the report _SUMMARY_ line in the configuration. Sidney
Re: why does that mail not get any bayes-classification
Am 11.06.2016 um 23:00 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 8:37 AM: it is not part of the report itself while tags, scores and descriptions are - a report is something like this: what you showed is defined in the configuration file using "report". Those just happen to be the last lines of it in the default configuration. That default uses the template tags _SCORE_, _REQD_, and _SUMMARY_. What I'm saying is that you can include "report" lines in the configuration that use the _SENDERDOMAIN_, _AUTHORDOMAIN_, and the various Bayes related tags and they will show up in the report. If you can use the report that you showed, then you can make use of those tags. They won't be in the table of points, rules, and description. That table is what _SUMMARY_ expands into. But you can insert them into the report that you see using spamc -R. They can even come after _SUMMARY_ if you want to you can do a lot the whole purpose here is to a) upload a eml file on a webserver b) spamc -R -l -s 2000 --socket /socket-path < upload.eml c) display the part startign with "Content analysis details" d) combine it with clamd results e) display the raw-eml on the bottom of the website all the header tricks are *not* part of it and it's not worth to discuss since the *real* solution would be a "BAYES_NOTOKS" which would appear *everywhere* and clearly explain why no other BAYES_XX is present signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 8:37 AM: > > it is not part of the report itself while tags, scores and descriptions > are - a report is something like this: what you showed is defined in the configuration file using "report". Those just happen to be the last lines of it in the default configuration. That default uses the template tags _SCORE_, _REQD_, and _SUMMARY_. What I'm saying is that you can include "report" lines in the configuration that use the _SENDERDOMAIN_, _AUTHORDOMAIN_, and the various Bayes related tags and they will show up in the report. If you can use the report that you showed, then you can make use of those tags. They won't be in the table of points, rules, and description. That table is what _SUMMARY_ expands into. But you can insert them into the report that you see using spamc -R. They can even come after _SUMMARY_ if you want to. Sidney
Re: why does that mail not get any bayes-classification
Am 11.06.2016 um 19:06 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 4:44 AM: look above "sadly that works only for mail-headers - but don't appear in the logs" Oh, you mentioned spamc -R before and it does appear in that output. I got that confused with the spamd logs - You're right, I don't see them there. it is not part of the report itself while tags, scores and descriptions are - a report is something like this: Content analysis details: (28.1 points, 5.5 required) pts rule name description -- -- -0.1 CUST_DNSWL_2_SENDERSC_LOW RBL: score.senderscore.com (Low Trust) [145.253.224.163 listed in score.senderscore.com] 1.0 NIXSPAM_IXHASH DIGEST: ix.dnsbl.manitu.net 7.5 BAYES_99 BODY: Bayes spam probability is 99 to 100% [score: 0.9995] 1.5 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 4.0 DKIM_ADSP_DISCARD No valid author signature, domain signs all mail and suggests discarding the rest 2.0 DATE_IN_FUTURE_06_12 Date: is 6 to 12 hours after Received: date 2.5 CUST_BODY_CONTAINS_M BODY: Contains Medium 0.5 CUST_BODY_CONTAINS_VL BODY: Contains Very Low 0.0 HTML_MESSAGE BODY: HTML included in message 0.4 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% [score: 0.9995] 2.0 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level above 50% [cf: 100] 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% [cf: 100] 1.0 FSL_BULK_SIG Bulk signature with no Unsubscribe 0.7 LOTS_OF_MONEY Huge... sums of money 0.1 MSGID_FROM_MTA_HEADER Message-Id was added by a relay 1.5 IXHASH_CHECK Message hits one ore more IXHASH digest-sources 2.5 DIGEST_MULTIPLE_LOCAL Message hits more than one network digest check (razor, pyzor, ixhash) 0.0 T_REMOTE_IMAGE Message contains an external image signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 4:44 AM: > look above "sadly that works only for mail-headers - but don't appear in > the logs" > Oh, you mentioned spamc -R before and it does appear in that output. I got that confused with the spamd logs - You're right, I don't see them there. Sidney
Re: why does that mail not get any bayes-classification
Am 11.06.2016 um 18:43 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 2:13 AM: sadly that works only for mail-headers - but don't appear in the logs I just tried adding this to my local configuration, not pretty, just to see what it would do report tag values sender _SENDERDOMAIN_ author _AUTHORDOMAIN_ bayesh _BAYESTCHAMMY_ bayses _BAYESTCSPAMMY_ and I got this line in the report tag values sender vantoll.nl author vantoll.nl bayesh 3 bayses 6 So even though the documentation is not clear about it, you can use those template tags in the report option too look above "sadly that works only for mail-headers - but don't appear in the logs" signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 2:13 AM: > sadly that works only for mail-headers - but don't appear in the logs I just tried adding this to my local configuration, not pretty, just to see what it would do report tag values sender _SENDERDOMAIN_ author _AUTHORDOMAIN_ bayesh _BAYESTCHAMMY_ bayses _BAYESTCSPAMMY_ and I got this line in the report tag values sender vantoll.nl author vantoll.nl bayesh 3 bayses 6 So even though the documentation is not clear about it, you can use those template tags in the report option too. Sidney
Re: why does that mail not get any bayes-classification
Am 11.06.2016 um 17:03 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 2:13 AM: sadly that works only for mail-headers - but don't appear in the logs nor in a report generated with "/usr/bin/spamc -R" over a webinterface which proceeds uploads of eml-files :-( Yes I see how that could be useful. It might work if you define a meta rule that checks for there not being any of the BAYES_NN rules but that wouldn't say anything useful because you can't distinct between "bayes not working at all", "to few training-messages" and "not enugh useful tokens" That doesn't give you access to _SENDERDOMAIN_ and _AUTHORDOMAIN_ tags, though jep signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 2:13 AM: > sadly that works only for mail-headers - but don't appear in the logs > nor in a report generated with "/usr/bin/spamc -R" over a webinterface > which proceeds uploads of eml-files :-( Yes I see how that could be useful. It might work if you define a meta rule that checks for there not being any of the BAYES_NN rules. That doesn't give you access to _SENDERDOMAIN_ and _AUTHORDOMAIN_ tags, though. Sidney
Re: why does that mail not get any bayes-classification
Am 11.06.2016 um 15:55 schrieb Sidney Markowitz: Reindl Harald wrote on 12/06/16 1:04 AM: output of "spamassassin -D < ignored_by_bayes_stripped.eml" attached See this line in that output: Jun 11 14:47:00.510 [5188] dbg: bayes: cannot use bayes on this message; not enough usable tokens found i would expect a bayes result in any case and even if it's just a informational BAYES_NOTOKS Not by default, but see the tags in the next lines in your debug output, starting with Jun 11 14:47:00.510 [5188] dbg: check: tagrun - tag BAYESTCHAMMY is now ready, value: 0 Jun 11 14:47:00.510 [5188] dbg: check: tagrun - tag BAYESTCSPAMMY is now ready, value: 0 And see how you can add custom headers to your output that includes these tags as documented here: https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html#template_tags sadly that works only for mail-headers - but don't appear in the logs nor in a report generated with "/usr/bin/spamc -R" over a webinterface which proceeds uploads of eml-files :-( a test-tag BAYES_NOTOKS or however called would appear on all places while for the spamd-log _SENDERDOMAIN_ and _AUTHORDOMAIN_ would give the benefit that you could also find out something useful about a result when there is no message-id signature.asc Description: OpenPGP digital signature
Re: why does that mail not get any bayes-classification
Reindl Harald wrote on 12/06/16 1:04 AM: > output of "spamassassin -D < ignored_by_bayes_stripped.eml" attached See this line in that output: Jun 11 14:47:00.510 [5188] dbg: bayes: cannot use bayes on this message; not enough usable tokens found > i would expect a bayes result in any case and even if it's just a > informational BAYES_NOTOKS Not by default, but see the tags in the next lines in your debug output, starting with Jun 11 14:47:00.510 [5188] dbg: check: tagrun - tag BAYESTCHAMMY is now ready, value: 0 Jun 11 14:47:00.510 [5188] dbg: check: tagrun - tag BAYESTCSPAMMY is now ready, value: 0 And see how you can add custom headers to your output that includes these tags as documented here: https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html#template_tags Sidney
Re: SA bayes file db permission issue
On Fri, 10 Jun 2016 15:38:44 -0400
Joseph Brennan wrote:
> This is a nice test I found:
> echo -n I | od -to2 | awk '{ print substr($2,6,1); exit}'
>
> 1 little-endian
> 0 big-endian
I don't see how this can output anything other than 1.
Endianness is about the addressing of bytes within integer words. This
is looking at the ordering of human-readable octal digits displaying
the contents of a single byte.
Re: Where to find DETAIL for spamassassin default RULES
On 11/06/2016 05:09, Bill Cole wrote: So, you thought validating email addresses was a problem demanding a solution? And you "solved" it with a regular expression? Congratulations on now having 2 problems. They should be very happy together. The regex I quoted was out of context to the problem and completely unrelated (sorry if you feel so confused with that). It is actually for another software project (a mail server) that, being a mail server, must ensure email addresses are valid. I quoted the regexp in context of showing my point about how 'squiggly' they can be and that I am able to read them.to a point. (I was proud because 'googling' around for a regex email address validator string shows some VERY suspicious and extortionately,seemingly unnecessarily, long offerings. So I had a go myself).
Re: why does that mail not get any bayes-classification
On Sat, 11 Jun 2016 04:52:48 +0200 Reindl Harald wrote: > Am 10.06.2016 um 23:52 schrieb RW: > > On Fri, 10 Jun 2016 16:57:45 +0200 > > Reindl Harald wrote: > > > >> see attachemnt, no bayes tag at all looks like a major bug > >> somewhere > > > > In the absence of any debug it's hard to say. > > hence i attached the sample An email is not debug. I can't run it on *your* system. > > It is possible for no tokens to make it through the selection, in > > which case there is no result. That's more likely than normal in > > your case since you don't train on headers. > > if you would have looked at the message you would have seen that > there is content and not only headers and it looks like the message > has just incorrect mime-definitions (missing end headers) Of course I looked at it. And I ran it through spamassassin. Aside from header tokens, what made it past the token selection on my database was only: 'marcus','Marcus','enclosed','invoice','business' and 'thank' It's quite possible that all the body tokens in that email were in the neutral range on your system, which would cause Bayes to exit without producing a classification. In the absence of any debug against your database, there is nothing particularly suspicious here.
