Re: Suse OpenExchange forwarding to Microsoft Exchange
I'm having a strange problem with SpamAssassin running on my Linux box which then forwards (with .forward) the mail to some users on Exchange. When the mail gets scanned and makes it through and then forwards to Exchange the email comes up completely blank with 'From: Spam Filter [EMAIL PROTECTED] The rest of the email is lost in the ether. Are you sure that the SA caller is not breaking the MIME, and that some form of the original body is not present (yet not displayed within OL/OWA)? Is this completely predictable for certain messages or certain users? Have you gotten your hands on a raw message, preferably by copying off your Linux box in addition to forwarding it to Exchange? Tell you this much: if you're sending Exchange a truncated message, there's clearly nothing you can do on the Exchange side, natch. If the MIME is still RFC, but OL/OWA won't render it, that's an OL/OWA/Exchange problem. But if the MIME is indeed non-RFC after mangling by your MTA/SA caller/SA, kudos to OL/OWA for not fixing it for once --Sandy
Re: SA and MTA message filtering
1) if message is marked as SPAM and mail address doesn't exist - delete it or move to a local folder on SA_MACHINE, 3) if message is not marked as SPAM and mail address exist - pass it to the Exchange mail server, 4) if message is not marked as SPAM and mail address doesn't exist - pass it to the Exchange mail server, Three out of your four objectives are markedly off-topic: there's no reason for SA to ever see mail for unknown local recipients. Those messages should be rejected by the MTA, using either your text file or direct LDAP lookup: you should Google or post elsewhere for the specifics. There's a large archive of envelope-rejection methods for every popular MTA. 2) if message is marked as SPAM and mail address exist - pass it to the Exchange mail server or move to a local folder on SA_MACHINE, As for this objective, you've listed two options. - If you want to quarantine the messages on the MTA, it'd be up to the MTA docs to tell you how to deal with the returned info from SA, either by interpreting the returned SA weight directly or by parsing headers. - If you want to pseudo-quarantine them on the Exchange box, you can use individual header rules or a store-wide event sink to dump each user's marked spam into into a mailbox subfolder with a fixed name. --Sandy
Re: Using SA to prevent bouncing spam?
Hi, in order to avoid bouncing spam back to the (almost certainly) faked sender-addresses, I thought I could use SA directly: What's your MTA and/or SA-invoking app? Surely it is easier to have that agent parse SA's feedback (headers, subject mod or score) in deciding the final disposition of the msg than to try to trick the MTA into dumping the mail. Please elaborate on the use case in which you can't use MTA processing rules to prevent backscatter, given that you trust SA markup completely here, right? --Sandy
Re[2]: What changes would you make to stop spam? - United Nations Paper
Does anyone use [XTND XMIT]? These days, not really. But when Eudora was king and the feature was usually enabled when supported on the MTA side, I would guess maybe 1% of Eudora users knew of and used the feature. The point is more that the extension's already been built, but never got a foothold. --Sandy
Re[2]: What changes would you make to stop spam? - United Nations Paper
So you think that viruses are going to know how to find and decrypt the passwords of all email programs? Any data that must be decrypted without user intervention can be accessed in its unencrypted form without user intervention. If user intervention is required for decryption, well, you pretty much just have to be there when it happens. These are fundamental rules. A virus needs no decryption feature per se. A sniffer can readily isolate plain-text passwords as they go over the wire. Alternately, yep, specific memory inspection routines could be built for all email programs that are likely to be found on compromised machines -- all, what, 3 or 4 of them -- regardless of what happens on the wire. This part is child's play for a hacker, relative to the harder part of finding new attack vectors for those boxes that are lucky enough to get disinfected and patched. Marc, I have some respect for your optimism, a rare trait in a place where others have (themselves well-earned) chips on their shoulders from pushing back a surging, inarguably criminal element from their networks all day. I also think that the accusations that you're an agent of some government, enterprise, NGO, etc., are ludicrous based on the fundamental naïveté of your proposal (like the fact that you suggested an enhancement which was already BTDT 8 years ago -- not going to get you a lot of followers on such a technical list). Yet: I concur that you don't have anywhere near sufficient knowledge of current, let alone historical, technologies for mail sending and retrieval to be suggesting... well, to be suggesting any enhancements or improvements at all. Look, it's okay to admit that you have to go back to school on those subjects. From your bio, you have grounding in other technical areas that many people here do not. I didn't know much about mail until 1999 or so, and that was after supporting mail systems (along with other systems I actually understood) for, like, 6 years! But I also kept my mouth shut until 1999. Because of that experience, I find myself agreeing with the overall reaction of, in essence: Kill me now, if his proposal is going to be disseminated by any entity who doesn't have enough techies on staff to shoot it down. Please, for the good of the world, take a couple of months to study before your next proposal. Warmly-- --Sandy
Re[2]: What changes would you make to stop spam? - United Nations Paper
Please don't pollute the IMAP and POP protocols this way. POP3 XTND XMIT submission extensions already polluted POP3 many years ago, supported by many thousands of servers (tho' not necessarily enabled). --Sandy
Re[2]: What changes would you make to stop spam? - United Nations Paper
MAPI. [is]..implemented over DCE/RPC (i.e. LAN-only). Maybe a nit... but technically not LAN-only using ncacn_http. --Sandy
Re[3]: Various plugins for Windows version?
win can lock the file on occasion and cause SA to fail. By which I think you mean that *your Windows-based MTA* can lock the file and cause SA to fail. A properly-written MTA-SA hook, or MTA-name_your_external_hook, has no such problems on Windows. We process millions of messages through Windows MTAs with standalone SA.pl shells, SPAMC shells, and integrated SPAMD calls and don't have any such locking issues. Issues like yours likely depend on whether your calling MTA attempts to maintain existing file handles while shelling to an external application, and also may be linked to how the MTA waits on the external app and obtains its results (WFSO + exit code lookup/exit code lookup after fixed timeout/notify on sentinel file creation/reopen the message file after a fixed delay/other IPC). There are numerous Win32 API approaches to perform these same tasks; some are safe, some are unsafe. In addition to stress-testing your choice of MTA for poor locking code, ensure that real-time AV and other external hooks are also taken out of the picture (including, as you have already mentioned, the Indexing Service). --Sandy
Re[2]: The Future of Email is SQL
If we are talking about making a SQL application that is usable for a multitude of people then why lock them into something. That's the easiest way to drive them away from supporting it. Word. Perl can play nice with plenty of RDBMSs. If this discussion belongs here at all, I can't see how RDBMS partisanship is going to take it anywhere good. FTR, there are several (commercial) spam quarantine applications, and at least three very big compliance/archival services, that take a SQL-based back-end as a given. Their traffic and access patterns have clearly been taken into account here, but nonetheless these are proof that the concept already has real-world purchase, depending on budget and application. --Sandy
Re: Tricky DNS Question - Advanced
Server B is a regular DNS server set up for caching and running BIND. It's the one that will be the public face for the blacklist providing caching for Server A so as not to load down Server A. Make B -- and, believe me if you are operating a public blacklist, C and D and E as well :) -- a secondary to primary A, with A thus your unpublished stealth primary for the zone. B, C, D, E are the published authoritative NSs for the zone, while A is secretly, as you put it, the truly authoritative source of the zone data served to the world by B, C, D, E. This is a good way to take advantage of the flexibility/updatability of RDBMS backends without worrying about the RDBMS-backed server seeing any direct traffic. --Sandy
Re[3]: Hiring for Spam Assassin Troubleshooting
Wayne,
Pro bono, here's the way to get SmarterMail to bypass SpamAssassin for
authenticated users.
BACKGROUND after playing around with SM for a few minutes
--
Like many/most MTAs, SM writes two spool files for every e-mail. One
is the message header/body, the other is the message envelope. In
naming the former, SM uses a message number + the extension .EML. THe
latter uses the same message number + the extension .HDR. So for every
message, there will be two files in the spool like so:
1234545.EML
1234345.HDR
In the HDR file for SMTP AUTH-enticated e-mails -- and only in those
e-mails -- there's a line that begins with the string 'auth'. What you
need is thus for your SpamAssassin caller to quick-check the contents
of the HDR file before deciding whether or not it should call
SpamAssassin.
HOWTO
-
(1) Go to SM Admin and choose Settings-General Settings-Spool tab.
(2) Check the value in 'Command-line file to run on new mail.' You
will see a value like
c:\progra~1\spamassassin\spamassassin.bat %filepath
or maybe
c:\smfilter\smfilter.exe %filepath
or similar.
(3) Copy the string to the left of '%filepath' to the clipboard. That
is, just the 'c:\smfilter\smfilter.exe' part in the second example.
(4) Open a new batch file. Here are the first two lines of the batch
file:
for /f %%I in ('echo %1') do find /i /c auth %%~dI%%~pI%%~nI.hdr
if %ERRORLEVEL%==0 exit
You must copy-and-paste or retype these lines exactly as shown.
(5) The third and final line of the batch file will be the value you
copied out in step (3) followed by the string '%1' (no quotes). If you
copied out 'c:\smfilter\smfilter.exe' in (3) then the third line of
the batch will be
c:\smfilter\smfilter.exe %1
(6) Save the three-line batch file as WLAUTH.CMD in a path you choose.
You can put it wherever SM was installed. For example, say you save it
to c:\progra~1\spamassassin\wlauth.cmd.
(7) Go back into the SM Admin page. Now replace the contents of the
'Command-line file...' box with your batch file name, followed by the
string '%filepath' (no quotes). If you saved the batch file as
c:\progra~1\spamassassin\wlauth.cmd, the value will be:
c:\progra~1\spamassassin\wlauth.cmd %filepath
(8) Save changes in SM Admin.
You now have a system that prechecks for auth'd users and doesn't run
SpamAssassin against their mail. QED.
That'll be $4500. LOL. Contact me off-list if you have probs.
--Sandy
Re: Hiring for Spam Assassin Troubleshooting
We already have SA setup and working with Smartermail. Well, not really. I'd say part and parcel of any SA-MTA integration is finding a way to whitelist messages _before SA is launched_, thus (a) saving you SA's CPU and disk time (esp. since you're using process-based spamassassin and not client/server spamc/spamd), and (b) eliminating problems like this caused when DNS and/or local content issues force SA to judge your messages as spammy. SA itself is not the place for true whitelisting; you really want to do this pre-SA. Anyway, SmarterMail shells to the external command-line you specify -- you may be calling SMFilter (a tiny wrapper for Clam and SpamAssassin), orcallingspamassassin.batorcompiled spamassassin.exe directly -- passing the body file name as last argument. You can thus retrieve the body file name from within a batch file as %1, if you've ever worked with batch file syntax. Within the first couple of headers added by SmarterMail, you'll see whether the mail was submitted by an authenticated user or not. I don't know their exact syntax offhand, but I know that SM does pass the authentication info to external apps. Ask on the SM forum what the exact string'll be. Then I'd suggest writing a batch file that, at the very top before calling spamassassin, uses a text search/replace utility to search only the first, say, 10 lines for a known string and skips calling SA if the string is found. Pseudo-batch: head -10 %1 | find /i /c string if %ERRORLEVEL%==0 (exit from batch file and return to smartermail, mail was auth'd) else (scan with SA, since authentication string was not found) This example uses the unxtools version of head.exe, which you can get from http://gnuwin.epfl.ch/apps/unxutils/en/index.html. While you may not be comfortable with this course of action either, it does tap into a different skill set that may be more familiar to you than DNS. You should have a way going forward of excluding mail from being even touched by SA. BTW, I consider this a SmarterMail issue as much as an SA one, and as such I hope you have been thorough enough to ask on their forum, or at least you will now. --Sandy
Re[2]: Hiring for Spam Assassin Troubleshooting
I have talked to the SM tech support and have searched through their forum but they believe this is SA issue. It is and it isn't. The fact that they don't have a simple way of skipping their external test (and they have had many, many bugs over the past few years relating to incomplete bypassing of auth'd or whitelist'd mails) is an essential design flaw. That they are pushing back at you like that, instead of at least saying they'll add the feature in the future, is typical of their developers... and is the reason I never adopted SM. That makes a lot of sense. Yes, I am familiar with writing batch files. I am not a god at writing them but I can read and write them. Okay, you should be off to the races. Yes we do use SMFilter but we call the spamassassin.bat from it. (I didn't know we could compile an exe file) No matter. How you're calling SA (bat or exe), when you call it, is pretty much immaterial now. What you need to do is insert a layer above everything else, so that you have SM run _your batch file_, which in turn _optionally_ runs SMFilter or whatever you want. Now, here is the issue I see with the bat file. There are millions of key phrases. Really? Isn't the key phrase just the authenticated... line? Did you ask SM about that? _That's_ the only thing you need to search on. Is there a way in the bat file to point it to some type of a list such as the one spam assassin already has or some type of a database. In a sense, yes, but you don't need to do that. If I utilize: head -10 %1 | find /I /c string then this is going to require me to enter each string on a separate line. I think you're confused about what you're searching for. The idea is to catch the header flag that means this mail was auth'd. Then that brings me to a point of asking, If I can utilize the SA Database using that method, then what good is SA? The answer to that is way off this topic. Using a find command against a filter file is a tiny subset of what SA does. --Sandy
Re[2]: Hiring for Spam Assassin Troubleshooting
If SMFilter adds its own headers, couldn't you just write a custom rule for SA that gives any message with those headers a negative score? SMFilter doesn't add anything; SM does add an authenticated header. But rather than plunge Wayne into SA rules -- which are still the wrong place to bypass SA, and where he is clearly less comfortable -- I am encouraging a solution at his wrapper/caller layer that will both save considerable resources and possibly be more in his comfort zone. --Sandy
Re[2]: Hiring for Spam Assassin Troubleshooting
I have talked to the SM tech support and have searched through their forum but they believe this is SA issue. P.S. You didn't start *a new thread* on their forum, which is as much community-supported as vendor-supported. This is not being thorough, for what seems like an urgent issue. --Sandy
Re[2]: lint failure with 3.1.2
internal_networks 192.168/16 Isn't this the internal_networks entry? Yes, and the warning is telling you that the value '192.168/16' (not thethename 'internal_networks') is not duplicated in trusted_networks, as would normally be expected. 3.0.x docs say: MXes for your domain(s) and internal relays should also be specified using the internal_networks setting. When there are 'trusted' hosts that are not MXes or internal relays for your domain(s) they should only be specified in trusted_networks. As for why this linted OK before, couldn't tell ya. Haven't looked at 3.1.2 yet. --Sandy
Re[2]: checksumming image spam
And to me that sounds like me running a Small Business Server I should be alrighht? Yes, absolutely. --Sandy
Re: Naming conventions for tests
The main problem with this approach is that it requires monitoring of the SPAM assassin tests being applied as the software is updated... Well, I'd say this is a problem chiefly because whoever _is_ administering the server -- not spamassassin.apache.org -- is clearly not encouraging the use of granular client-side filtering. If filtering on more than the Spam Score were an expectation from end-to-end, you would have a consistently updated list provided to you by your mail admin, through an intranet portal or whatever. It's a virtual certainty that your mail admin is using rules and metas that don't ship with SA. What would you do about those? --Sandy
Re: 20_bodytests
Dan-80 wrote:
2) What do 'tflags' do?:
describe MIME_CHARSET_FARAWAY MIME character set indicates foreign
language
tflags MIME_CHARSET_FARAWAY userconf
tflags SYMBOLIC_TEST_NAME [ {net|nice|learn|userconf|noautolearn} ]
Used to set flags on a test. These flags are used in the score-determination
back end system for details of the test's behaviour. Please see
bayes_auto_learn and use_auto_whitelist for more information about tflag
interaction with those systems. The following flags can be set:
net
The test is a network test, and will not be run in the mass checking system
or if -L is used, therefore its score should not be modified.
nice
The test is intended to compensate for common false positives, and should be
assigned a negative score.
userconf
The test requires user configuration before it can be used (like language-
specific tests).
learn
The test requires training before it can be used.
noautolearn
The test will explicitly be ignored when calculating the score for learning
systems.
Dan-80 wrote:
3) What are 'test' lines?:
test SYMBOLIC_TEST_NAME (ok|fail) Some string to test against
Define a regression testing string. You can have more than one regression
test string per symbolic test name. Simply specify a string that you wish
the test to match.
These tests are only run as part of the test suite - they should not affect
the general running of SpamAssassin.
--Sandy
--
View this message in context:
http://www.nabble.com/20_bodytests-t1553240.html#a4312791
Sent from the SpamAssassin - Users forum at Nabble.com.
