Re: getmail headers

[Martin Gregorie]

OK, Understood. Thanks for the explanation.

Martin


On Thu, 2016-12-15 at 23:30 +, RW wrote:
> On Thu, 15 Dec 2016 22:35:10 +
> Martin Gregorie wrote:
> 
> > 
> > On Thu, 2016-12-15 at 22:19 +, RW wrote:
> > 
> > > 
> > > If you are using getmail/fetchmail it commonly just works. SA has
> > > explicit support for fetchmail, and getmail headers are
> > > unparseable.
> > > Either way there is typically a chain of private and localhost IP
> > > addresses up to the MX server.
> > >   
> > Yes, that's been my experience with both fetchmail and getmail. I
> > used
> > to use fetchmail until its habit of 'accidentally' leaving mail its
> > read but forgotten to delete in the source mailbox. Forever.
> > Eventually this annoyed me enough make the switch to getmail, which
> > hasn't caused me any problems at all.
> > 
> > I haven't noticed any problems arising from getmail's 'unparseable
> > headers': in fact I hadn't noticed that they weren't being
> > processed.
> > Which brings me to the question I wanted to raised in this
> > (renamed)
> > thread: 
> > 
> > What would we gain if getmail generated headers were parsed? 
> 
> The unparseable header breaks ALL_TRUSTED, but that doesn't really
> matter for a retriever. I don't recall any other practical
> difference.
> 
> Making SA parse the header normally is a really bad idea because the
> public IP address of the POP/IMAP server will break the chain of
> trust, requiring trusted_network to be set explicitly.
> 
> The fetchmail support causes the received header parsing to restart
> on the next header down.  
> 
> > 
> > IOW, should I raise a bug or would I just be wasting my and other
> > peoples, time? If it would not be a waste of time, should the bug
> > be
> > raised against getmail or SA?
> 
> There is a bug report with a patch to make the fetchmail support
> generic to all POP/IMAP, but it's not been committed.
> 
> It doesn't make much difference for getmail. There might possibly be
> other retriever software that could benefit, although they're likely
> to be unparseble like getmail.

Re: getmail headers

[RW]

On Thu, 15 Dec 2016 22:35:10 +
Martin Gregorie wrote:

> On Thu, 2016-12-15 at 22:19 +, RW wrote:
> 
> > If you are using getmail/fetchmail it commonly just works. SA has
> > explicit support for fetchmail, and getmail headers are unparseable.
> > Either way there is typically a chain of private and localhost IP
> > addresses up to the MX server.
> >   
> Yes, that's been my experience with both fetchmail and getmail. I used
> to use fetchmail until its habit of 'accidentally' leaving mail its
> read but forgotten to delete in the source mailbox. Forever.
> Eventually this annoyed me enough make the switch to getmail, which
> hasn't caused me any problems at all.
> 
> I haven't noticed any problems arising from getmail's 'unparseable
> headers': in fact I hadn't noticed that they weren't being processed.
> Which brings me to the question I wanted to raised in this (renamed)
> thread: 
> 
> What would we gain if getmail generated headers were parsed? 

The unparseable header breaks ALL_TRUSTED, but that doesn't really
matter for a retriever. I don't recall any other practical difference.

Making SA parse the header normally is a really bad idea because the
public IP address of the POP/IMAP server will break the chain of
trust, requiring trusted_network to be set explicitly.

The fetchmail support causes the received header parsing to restart
on the next header down.  

> IOW, should I raise a bug or would I just be wasting my and other
> peoples, time? If it would not be a waste of time, should the bug be
> raised against getmail or SA?

There is a bug report with a patch to make the fetchmail support
generic to all POP/IMAP, but it's not been committed.

It doesn't make much difference for getmail. There might possibly be
other retriever software that could benefit, although they're likely
to be unparseble like getmail.

Re: getmail?

[Nix]

On Thu, 09 Feb 2006, Gene Heskett spake:
>>From re-reading a 'man fetchmail' I don't see the fileing ability.  It 
> only presents it to localhost:25 and apparently sendmail takes it from 
> there.  The comm thru port 25 is apparently bilateral as it can be told 

?! Definitely not.

> to summarily delete unwanted mail from the server, while sendmail at 

That's the *POP3* server.

> the some time is deleting its copy.

sendmail is extremely careful to never `delete a copy' of mail (I'd call
that `losing mail'). It simply takes it and relays or delivers it.

>>- run an MDA. so you could run procmail or maildrop or a (correct)
>>script. In short, fetchmail runs a command (it pipes the message).
> 
> eg sendmail?, which is running here.

It only does that in extremis if a local SMTP server isn't running.

> But, here is the headache:  At no place in the various files sitting 
> in /etc/mail that serve to configure sendmail, is there an example of 
> how to configure sendmail to make use of these feature facilities.

That's why there's a vast book covering it in agonizing detail, and
docs in the source tree --- and probably in /usr/share/doc or wherever
your distro keeps them --- explaining how to use the .mc M4 macro-expander
to generate a configuration file.

You probably need *none* of this, though: on virtually all Linux
distros, procmail is invoked by sendmail to do delivery, so you can just
put everything in .procmailrc.

> Spamassassin 3.10 contains only very scant references to using it with 
> sendmail, apparently sanctioning only the procmail interface, which in 
> turn then is set to call spamc or spamassassin, adding needless time 
> wasting cpu cycles to what should be a pretty simple job.  I fail to 

procmail is so lightweight that I was running it on a 386/25 a decade
ago and not noticing the CPU hit. Just ignore it.

Worrying about the CPU hit of the local delivery agent on a box that
must run a vast CPU and memory hog like spamd is... quixotic, to say
the least.

> understand why (although it will take smarter people than me what with 
> sendmails configuration complexity) there is no readily published 
> recipe for incorporating spamc into the sendmail processing chain, 
> either by pipeing, or when the libmilter feature is there?

Well, there is spamass-milter, which is fairly frequently mentioned on
this list.

-- 
`... follow the bouncing internment camps.' --- Peter da Silva

Re: getmail?

[Craig White]

On Sat, 2006-02-11 at 23:23 +, Craig McLean wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Gene Heskett wrote:
> [snip sendmail discussion]
> 
> > I've about come to that conclusion myself, so I'm now investigating the 
> > fetchmail->procmail_>dovecot solution right now.  But the dovecot 
> > mailing list might be a problem, I've subbed about an hour ago but have 
> > rx'd no please confirm message yet.
> > 
> > Joanne has me about straight on the fetchmail and procmail stuffs, and I 
> > may even see if I can turn that part on just for grins, but 
> > dovecot's .conf looks like it'll need a philly lawyer to decode it 
> > correctly so it works.
> 
> Heh, yeah. The dovecot config can be pretty daunting, I'll try and
> summarise how I've got it set up here, but many things may not be needed
> where you are.
> The only uncommented lines in my config are:
> 
> - -quote-
> protocols = imap imaps # We don't use POP
> ssl_cert_file = /etc/mail/certs/fukka.co.uk.cert # SSL stuff
> ssl_key_file = /etc/mail/certs/fukka.co.uk.key   # SSL stuff
> disable_plaintext_auth = no# Nasty Squirrelmail
># hack
> login_user = dovecot   # Discrete user for
># processes
> login_processes_count = 1  # Tuning
> login_max_processes_count = 12 # Tuning
> login_max_logging_users = 12   # Tuning
> first_valid_uid = 1000 # Security
> first_valid_gid = 0# Hack for my GID
> mail_extra_groups = mail   # Permissions tweak
> default_mail_env = mbox:/var/mail/%u   # YMMV - check the docs
> lock_method = flock# Multiple things lock
># mail here
> maildir_copy_with_hardlinks = yes  # Dunno. Check docs
> mbox_read_locks = flock# Locking
> mbox_write_locks = flock   # Locking
> mbox_lazy_writes = no  # Tweak
> protocol imap {# IMAP settings in {}
>   login_greeting_capability = no
>   imap_client_workarounds = delay-newmail outlook-idle netscape-eoh
> tb-extra-mailbox-sep
> }
> auth_verbose = yes # Just because
> auth default { # User auth setings in
># {}
>   mechanisms = plain
>   passdb pam {
>   }
>   userdb passwd {
>   }
>   user = root
> }
> - -quote-
> 
> I've found the docs for dovecot to be fairly good, if a little tech-heavy.
> 
> On the other hand, FC also includes both UW-IMAP and Cyrus, more about
> UW at http://www.washington.edu/imap/ and Cyrus at
> http://asg.web.cmu.edu/cyrus/imapd/
> 
> Either of these is likely to be easier to configure that dovecot.
> 
> >> In a configuration where you don't readily run sendmail to accept
> >> mail, I would suggest staying the hell away from it and:
> > 
> > Sendmail does run to collect local mail here, like from amanda and 
> > cron/logwatch, that sort of stuff.  And I'd like to figure out a way to 
> > collect mail from the firewall box so I didn't have to log in via ssh 
> > 2-3 times a week and read the chkrootkit reports and such.  Its 
> > normally a mounted samba share from here, so maybe I could get kmail to 
> > do that now that I think about it.  Humm, off to try it by golly.
> > 
> 
> You'll be a whizz at installing IMAP servers soon, you could install one
> on the firewall box and use fetchmail to pull it onto the main server.
> Assuming you felt suitably insane.

he can also ask dovecot questions on fedora list too rather than adding
noise here.

Craig

Re: getmail?

[Craig McLean]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Gene Heskett wrote:
[snip sendmail discussion]

> I've about come to that conclusion myself, so I'm now investigating the 
> fetchmail->procmail_>dovecot solution right now.  But the dovecot 
> mailing list might be a problem, I've subbed about an hour ago but have 
> rx'd no please confirm message yet.
> 
> Joanne has me about straight on the fetchmail and procmail stuffs, and I 
> may even see if I can turn that part on just for grins, but 
> dovecot's .conf looks like it'll need a philly lawyer to decode it 
> correctly so it works.

Heh, yeah. The dovecot config can be pretty daunting, I'll try and
summarise how I've got it set up here, but many things may not be needed
where you are.
The only uncommented lines in my config are:

- -quote-
protocols = imap imaps   # We don't use POP
ssl_cert_file = /etc/mail/certs/fukka.co.uk.cert # SSL stuff
ssl_key_file = /etc/mail/certs/fukka.co.uk.key   # SSL stuff
disable_plaintext_auth = no  # Nasty Squirrelmail
 # hack
login_user = dovecot # Discrete user for
 # processes
login_processes_count = 1# Tuning
login_max_processes_count = 12   # Tuning
login_max_logging_users = 12 # Tuning
first_valid_uid = 1000   # Security
first_valid_gid = 0  # Hack for my GID
mail_extra_groups = mail # Permissions tweak
default_mail_env = mbox:/var/mail/%u # YMMV - check the docs
lock_method = flock  # Multiple things lock
 # mail here
maildir_copy_with_hardlinks = yes# Dunno. Check docs
mbox_read_locks = flock  # Locking
mbox_write_locks = flock # Locking
mbox_lazy_writes = no# Tweak
protocol imap {  # IMAP settings in {}
  login_greeting_capability = no
  imap_client_workarounds = delay-newmail outlook-idle netscape-eoh
tb-extra-mailbox-sep
}
auth_verbose = yes   # Just because
auth default {   # User auth setings in
 # {}
  mechanisms = plain
  passdb pam {
  }
  userdb passwd {
  }
  user = root
}
- -quote-

I've found the docs for dovecot to be fairly good, if a little tech-heavy.

On the other hand, FC also includes both UW-IMAP and Cyrus, more about
UW at http://www.washington.edu/imap/ and Cyrus at
http://asg.web.cmu.edu/cyrus/imapd/

Either of these is likely to be easier to configure that dovecot.

>> In a configuration where you don't readily run sendmail to accept
>> mail, I would suggest staying the hell away from it and:
> 
> Sendmail does run to collect local mail here, like from amanda and 
> cron/logwatch, that sort of stuff.  And I'd like to figure out a way to 
> collect mail from the firewall box so I didn't have to log in via ssh 
> 2-3 times a week and read the chkrootkit reports and such.  Its 
> normally a mounted samba share from here, so maybe I could get kmail to 
> do that now that I think about it.  Humm, off to try it by golly.
> 

You'll be a whizz at installing IMAP servers soon, you could install one
on the firewall box and use fetchmail to pull it onto the main server.
Assuming you felt suitably insane.

Regards,
C.
- --
Craig McLeanhttp://fukka.co.uk
[EMAIL PROTECTED]   Where the fun never starts
Powered by FreeBSD, and GIN!
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFD7nHaMDDagS2VwJ4RAklCAJ4yFrD5DTEtx6kY6fM/wdr9ocsESwCfZQfB
wnAhZlgEFECvt98TsXiL5GA=
=8ACT
-END PGP SIGNATURE-

Re: getmail?

[Gene Heskett]

On Saturday 11 February 2006 08:25, Craig McLean wrote:
>-BEGIN PGP SIGNED MESSAGE-
>Hash: SHA1
>
>Gene Heskett wrote:
>[snip fetchmail discussion]
>
>> In further reading tonight, sendmail grew the libmilter freature at
>> 8.12, which is the base version running here, and yum won't update
>> it, says its current.
>
>What version of the OS are you running, Gene? FC4 has 8.13.4-2 as the
>latest, not that it necessarily makes any odds.
>
FC2 with lots of tarball installed stuff to replace the originally drain 
bamaged FC2 stuff, like cups, gutenprint (pick a random proggy, its 
possibly a tarball install, or maybe a checkinstalled version.

>> Right now, I'm looking at the
>>  site, trying to see how
>> this is done.
>>
>> But, here is the headache:  At no place in the various files sitting
>> in /etc/mail that serve to configure sendmail, is there an example
>> of how to configure sendmail to make use of these feature
>> facilities.
>
>Basic milter information can be found at:
>http://www.sendmail.org/~ca/email/doc8.12/cf/m4/adding_mailfilters.htm
>l and more in-depth here:
>http://www.milter.org/
>
>An example of how to get sendmail to use spamass-milter (and
>clamav-milter, I use both) looks like this, from sendmail.mc:
>
>- -quote-
>dnl ** Milter Configurations **
>define(`confMILTER_MACROS_CONNECT',`b, j, _, {daemon_name}, {if_name},
>{if_addr}')
>INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clmilter.sock,
> F=, T=S:4m;R:4m')dnl
>INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=,
>T=C:15m;S:4m;R:4m;E:10m')
>dnl define(`confINPUT_MAIL_FILTERS', `spamassassin,clmilter')
>- -quote-
>
>Over here I use spamass-milter to pass mail to spamc as it passes
>through the MTA, because I *am* my domain's MX.
>This means sendmail needs to be configured to accept mail via SMTP
> which is fine for me, but might be far more overhead than you need.
>
>> Spamassassin 3.10 contains only very scant references to using it
>> with sendmail, apparently sanctioning only the procmail interface,
>> which in turn then is set to call spamc or spamassassin, adding
>> needless time wasting cpu cycles to what should be a pretty simple
>> job.  I fail to understand why (although it will take smarter people
>> than me what with sendmails configuration complexity) there is no
>> readily published recipe for incorporating spamc into the sendmail
>> processing chain, either by pipeing, or when the libmilter feature
>> is there?
>
>libmilter just provides a mechanism for sendmail to pass the email,
> via a socket, to a small C program, thence to spamc. Talk about
> "needless time wasting CPU cycles"?

I've about come to that conclusion myself, so I'm now investigating the 
fetchmail->procmail_>dovecot solution right now.  But the dovecot 
mailing list might be a problem, I've subbed about an hour ago but have 
rx'd no please confirm message yet.

Joanne has me about straight on the fetchmail and procmail stuffs, and I 
may even see if I can turn that part on just for grins, but 
dovecot's .conf looks like it'll need a philly lawyer to decode it 
correctly so it works.

>In a configuration where you don't readily run sendmail to accept
> mail, I would suggest staying the hell away from it and:

Sendmail does run to collect local mail here, like from amanda and 
cron/logwatch, that sort of stuff.  And I'd like to figure out a way to 
collect mail from the firewall box so I didn't have to log in via ssh 
2-3 times a week and read the chkrootkit reports and such.  Its 
normally a mounted samba share from here, so maybe I could get kmail to 
do that now that I think about it.  Humm, off to try it by golly.

>a) configuring fetchmail to simply use procmail as the MDA. ("--mda
>/usr/bin/procmail" or similar, IIRC)
>b) having procmail run everything handed to it through spamc, and
> filter accordingly.
>
>Peice of cake (relatively speaking) to set up, no sendmail black magic
>and fairly quick to run.
>
>> Or am I simply on the wrong mailing list?  I've sent 3 subscribe
>> messages to the getmail-user list over the last 3 days with no
>> response which is discouraging.  OTOH, now that I know it can't do
>> what I want, who cares.  It might be that if there was a manpage for
>> getmail, it might be possible.  A pox on software that doesn't come
>> with readable manuals.
>
>Or *any* manuals

Yup.  I wonder if the author is reading the traffic.  Obviously not, 
else I'd think the background noise would prompt an attempt at it at 
least. :)

Thanks Craig.

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Re: getmail?

[Craig McLean]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Gene Heskett wrote:
[snip fetchmail discussion]

> 
> In further reading tonight, sendmail grew the libmilter freature at 
> 8.12, which is the base version running here, and yum won't update it, 
> says its current.

What version of the OS are you running, Gene? FC4 has 8.13.4-2 as the
latest, not that it necessarily makes any odds.

> Right now, I'm looking at the  
> site, trying to see how this is done.
> 
> But, here is the headache:  At no place in the various files sitting 
> in /etc/mail that serve to configure sendmail, is there an example of 
> how to configure sendmail to make use of these feature facilities.

Basic milter information can be found at:
http://www.sendmail.org/~ca/email/doc8.12/cf/m4/adding_mailfilters.html
and more in-depth here:
http://www.milter.org/

An example of how to get sendmail to use spamass-milter (and
clamav-milter, I use both) looks like this, from sendmail.mc:

- -quote-
dnl ** Milter Configurations **
define(`confMILTER_MACROS_CONNECT',`b, j, _, {daemon_name}, {if_name},
{if_addr}')
INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clmilter.sock, F=,
T=S:4m;R:4m')dnl
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=,
T=C:15m;S:4m;R:4m;E:10m')
dnl define(`confINPUT_MAIL_FILTERS', `spamassassin,clmilter')
- -quote-

Over here I use spamass-milter to pass mail to spamc as it passes
through the MTA, because I *am* my domain's MX.
This means sendmail needs to be configured to accept mail via SMTP which
is fine for me, but might be far more overhead than you need.

> Spamassassin 3.10 contains only very scant references to using it with 
> sendmail, apparently sanctioning only the procmail interface, which in 
> turn then is set to call spamc or spamassassin, adding needless time 
> wasting cpu cycles to what should be a pretty simple job.  I fail to 
> understand why (although it will take smarter people than me what with 
> sendmails configuration complexity) there is no readily published 
> recipe for incorporating spamc into the sendmail processing chain, 
> either by pipeing, or when the libmilter feature is there?

libmilter just provides a mechanism for sendmail to pass the email, via
a socket, to a small C program, thence to spamc. Talk about "needless
time wasting CPU cycles"?

In a configuration where you don't readily run sendmail to accept mail,
I would suggest staying the hell away from it and:

a) configuring fetchmail to simply use procmail as the MDA. ("--mda
/usr/bin/procmail" or similar, IIRC)
b) having procmail run everything handed to it through spamc, and filter
accordingly.

Peice of cake (relatively speaking) to set up, no sendmail black magic
and fairly quick to run.

> Or am I simply on the wrong mailing list?  I've sent 3 subscribe 
> messages to the getmail-user list over the last 3 days with no response 
> which is discouraging.  OTOH, now that I know it can't do what I want, 
> who cares.  It might be that if there was a manpage for getmail, it 
> might be possible.  A pox on software that doesn't come with readable 
> manuals.

Or *any* manuals

All the best!
C.

- --
Craig McLeanhttp://fukka.co.uk
[EMAIL PROTECTED]   Where the fun never starts
Powered by FreeBSD, and GIN!
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFD7eXCMDDagS2VwJ4RAklQAKDDkbeOOGGfp7I5RuubaSmAAJCjiwCgjwbM
bVGx27+TfZgUG9QwfK6VJU8=
=QDMd
-END PGP SIGNATURE-

Re: getmail?

[Gene Heskett]

On Thursday 09 February 2006 22:45, jdow wrote:
>From: "Gene Heskett" <[EMAIL PROTECTED]>
>
>On Thursday 09 February 2006 03:47, jdow wrote:
>>From: "Gene Heskett" <[EMAIL PROTECTED]>
>>
Procmail calls SpamAssasin and feeds the return off to the spool
 file.
>>>
>>> Ok, sub getmail for both fetchmail and procmail, since getmail can
>>> handle the SA pipeing you are doing with procmail.  Then run
>>> dovecot on that box to serve kmail on this box?  I have the kmail
>>> fetching turned off on that box, so I'd assume I can give getmail a
>>> trial run and see if what it fetches it shows up in kmail on that
>>> firewall box as a new mail, if that works, then setup dovecot as a
>>> pop3 server to serve the kmail requests from this box.  Have I got
>>> it right?  All running as the user gene I'd assume?
>>
>>Only if getmail combines fetchmail and procmail including procmail's
>>ability to write rules for redirecting mail or applying filtering to
>> it.
>>
>>Kmail would simply read from the imap port you create. It's still
>> write to your ISP's mail server.
>
>First, getmail is out as far as putting it on the FW box, the python
>install there is several releases too old.
>
>Further reading on fetchmail tells me that it hands the incoming mail
>off to sendmail via stuffing it into port 110.  Datapoint as I try to
>get my head around the mechanics of this.
>
><>==
> Read further. The interesting line is:
>defaults mda "/usr/bin/procmail -d jdow"
>
>That sends it to procmail and procmail sends it to
> /var/spool/mail/.
> <>
>=
>
>It is sendmail then that listens on port 110 and writes to
>the /var/spool/mail/username file. Datapoint again.
>
>So there is a potential place to put a |spamc| is there not?, between
>fetchmail and port 110?  A hack to fetchmail maybe?  Datapoint.
>
>
><>==
> You use procmail for doing this.
><>=
>
>
>Experimental results..  I grabbed a copy of /var/spool/mail/gene to
>another file while it had some content, then did a 'cat filename|spamc
>
>>filename2'
>
><>==
> Award issued for today's "egregious use of cat award". {^_-}
><>=
>
>
Awww, gee Joanne.

>This did properly scan & add the headers that it had done so to the
>first of the 3 messages that were merged into filename, but did not
>re-trigger itself on the next 2 messages also in that file, therefore
>they were not scanned and marked up by spamc.
>
><>==
> Gene, I have sent you working instructions for this setup. Have
> you lost them already? It was only a month ago. {o.o}
><>=
>
I'll look them up in the morning, didn't sleep at all well last night, 
listening to my missus & her emphasema cough bother the hell outta me.

>So it appears that wherever spamc is inserted into the path, it must
> be presented with a single message complete with an EOF indicator of
> come kind.  Looking at the src file, it doesn't appear there is a
> quick, dirty, and 100% dependable way to filter the output of the cat
> command and break it up into one stream per message. But I haven't
> ran a tcpdump to see how its formatted on the network traffic yet. 
> The only thing I can see is linefeed,linefeed,From and since theres
> no way to stop me from doing it in a message I send, it doesn't look
> that reliable to me.
>
>From
>
>Is that treated as a new message?  I think not.  I'm going to go look
> at the fetchmail code, maybe I can make a patch for it to do this.
>
>>{^_^}
>
>1) Procmail is what calls spamassassin. This is not the whole thing.
> But... ===8<---
>##
>### # Necessary generic definitions
>##
>### DROPPRIVS=yes
>#VERBOSE=yes
>LOGNAME=procmail
>
>## rawmbox is no longer needed at this time.
>#:0c: clone.lock
>##* ^List-Id: .*(spamassassin\.apache.\org)
>#$HOME/mail/rawmbox
>#===---Remove one level of "#" to save a clone of the raw mail.
>
>##
>### # Then we install some deaths and diversions
>##
>### # Example - this mailer daemon got out of hand long ago.
>
>:0:
>
>* ^From: [EMAIL PROTECTED]
>/dev/null
>
># This one STILL is out of sane control.
>
>:0:
>
>* ^From: AntiSpam UOL <[EMAIL PROTECTED]>
>#/dev/null
>/$HOME/mail/uol_crap
>
>##
>### # Then we install some potential f

Re: getmail?

[jdow]

From: "Gene Heskett" <[EMAIL PROTECTED]>

On Thursday 09 February 2006 03:47, jdow wrote:

From: "Gene Heskett" <[EMAIL PROTECTED]>


Procmail calls SpamAssasin and feeds the return off to the spool
file.


Ok, sub getmail for both fetchmail and procmail, since getmail can
handle the SA pipeing you are doing with procmail.  Then run dovecot
on that box to serve kmail on this box?  I have the kmail fetching
turned off on that box, so I'd assume I can give getmail a trial run
and see if what it fetches it shows up in kmail on that firewall box
as a new mail, if that works, then setup dovecot as a pop3 server to
serve the kmail requests from this box.  Have I got it right?  All
running as the user gene I'd assume?


Only if getmail combines fetchmail and procmail including procmail's
ability to write rules for redirecting mail or applying filtering to
it.

Kmail would simply read from the imap port you create. It's still
write to your ISP's mail server.


First, getmail is out as far as putting it on the FW box, the python
install there is several releases too old.

Further reading on fetchmail tells me that it hands the incoming mail
off to sendmail via stuffing it into port 110.  Datapoint as I try to
get my head around the mechanics of this.

<>==
Read further. The interesting line is:
defaults mda "/usr/bin/procmail -d jdow"

That sends it to procmail and procmail sends it to /var/spool/mail/.
<>=

It is sendmail then that listens on port 110 and writes to
the /var/spool/mail/username file. Datapoint again.

So there is a potential place to put a |spamc| is there not?, between
fetchmail and port 110?  A hack to fetchmail maybe?  Datapoint.


<>==
You use procmail for doing this.
<>=

Experimental results..  I grabbed a copy of /var/spool/mail/gene to
another file while it had some content, then did a 'cat filename|spamc

filename2'


<>==
Award issued for today's "egregious use of cat award". {^_-}
<>=

This did properly scan & add the headers that it had done so to the
first of the 3 messages that were merged into filename, but did not
re-trigger itself on the next 2 messages also in that file, therefore
they were not scanned and marked up by spamc.

<>==
Gene, I have sent you working instructions for this setup. Have you lost
them already? It was only a month ago. {o.o}
<>=

So it appears that wherever spamc is inserted into the path, it must be
presented with a single message complete with an EOF indicator of come
kind.  Looking at the src file, it doesn't appear there is a quick,
dirty, and 100% dependable way to filter the output of the cat command
and break it up into one stream per message. But I haven't ran a
tcpdump to see how its formatted on the network traffic yet.  The only
thing I can see is linefeed,linefeed,From and since theres no way to
stop me from doing it in a message I send, it doesn't look that
reliable to me.

From

Is that treated as a new message?  I think not.  I'm going to go look at
the fetchmail code, maybe I can make a patch for it to do this.


{^_^}


1) Procmail is what calls spamassassin. This is not the whole thing. But...
===8<---
#
# Necessary generic definitions
#
DROPPRIVS=yes
#VERBOSE=yes
LOGNAME=procmail

## rawmbox is no longer needed at this time.
#:0c: clone.lock
##* ^List-Id: .*(spamassassin\.apache.\org)
#$HOME/mail/rawmbox
#===---Remove one level of "#" to save a clone of the raw mail.

#
# Then we install some deaths and diversions
#
# Example - this mailer daemon got out of hand long ago.

:0:
* ^From: [EMAIL PROTECTED]
/dev/null

# This one STILL is out of sane control.
:0:
* ^From: AntiSpam UOL <[EMAIL PROTECTED]>
#/dev/null
/$HOME/mail/uol_crap

#
# Then we install some potential forged markups
#

:0
* ^X-Spam-Status:
{
   :0 fw
   | formail -R "X-Spam-Status:" "X-False-Spam-Status:"

   :0 fw
   | formail -A "X-Nasty: Aren't we?"
}

:0
* ^X-Spam-Level
{
   :0 fw
   | formail -R "X-Spam-Level" "X-False-Spam-Level"
}

:0
* ^X-Spam-Checker-Version:
{
   :0 fw
   | formail -R "X-Spam-Checker-Version:" 

Re: getmail?

[Gene Heskett]

On Thursday 09 February 2006 16:30, mouss wrote:
>Gene Heskett a écrit :
>> On Thursday 09 February 2006 03:47, jdow wrote:
>>>From: "Gene Heskett" <[EMAIL PROTECTED]>
>>>
>Procmail calls SpamAssasin and feeds the return off to the spool
>file.

Ok, sub getmail for both fetchmail and procmail, since getmail can
handle the SA pipeing you are doing with procmail.  Then run
 dovecot on that box to serve kmail on this box?  I have the kmail
 fetching turned off on that box, so I'd assume I can give getmail
 a trial run and see if what it fetches it shows up in kmail on
 that firewall box as a new mail, if that works, then setup dovecot
 as a pop3 server to serve the kmail requests from this box.  Have
 I got it right?  All running as the user gene I'd assume?
>>>
>>>Only if getmail combines fetchmail and procmail including procmail's
>>>ability to write rules for redirecting mail or applying filtering to
>>>it.
>>>
>>>Kmail would simply read from the imap port you create. It's still
>>>write to your ISP's mail server.
>>
>> First, getmail is out as far as putting it on the FW box, the python
>> install there is several releases too old.
>>
>> Further reading on fetchmail tells me that it hands the incoming
>> mail off to sendmail via stuffing it into port 110.  Datapoint as I
>> try to get my head around the mechanics of this.
>>
>> It is sendmail then that listens on port 110 and writes to
>> the /var/spool/mail/username file. Datapoint again.
>>
>> So there is a potential place to put a |spamc| is there not?,
>> between fetchmail and port 110?  A hack to fetchmail maybe? 
>> Datapoint.
>>
>> Experimental results..  I grabbed a copy of /var/spool/mail/gene to
>> another file while it had some content, then did a 'cat
>> filename|spamc
>>
>>>filename2'
>>
>> This did properly scan & add the headers that it had done so to the
>> first of the 3 messages that were merged into filename, but did not
>> re-trigger itself on the next 2 messages also in that file,
>> therefore they were not scanned and marked up by spamc.
>>
>> So it appears that wherever spamc is inserted into the path, it must
>> be presented with a single message complete with an EOF indicator of
>> come kind.  Looking at the src file, it doesn't appear there is a
>> quick, dirty, and 100% dependable way to filter the output of the
>> cat command and break it up into one stream per message. But I
>> haven't ran a tcpdump to see how its formatted on the network
>> traffic yet.  The only thing I can see is linefeed,linefeed,From and
>> since theres no way to stop me from doing it in a message I send, it
>> doesn't look that reliable to me.
>>
>> From
>>
>> Is that treated as a new message?  I think not.  I'm going to go
>> look at the fetchmail code, maybe I can make a patch for it to do
>> this.
>
>once fetchmail has read the message, it can:
>
>- put it in files. this is what you do.

From re-reading a 'man fetchmail' I don't see the fileing ability.  It 
only presents it to localhost:25 and apparently sendmail takes it from 
there.  The comm thru port 25 is apparently bilateral as it can be told 
to summarily delete unwanted mail from the server, while sendmail at 
the some time is deleting its copy.  Or at least thats how I'm reading 
it.

>- run an MDA. so you could run procmail or maildrop or a (correct)
>script. In short, fetchmail runs a command (it pipes the message).

eg sendmail?, which is running here.

>- forward to an smtp server. This is the simplest to configure if you
>can afford to run an MTA.
>
>you'll need to choose which method is appropriate for your situation.
> if you don't feel yourself installing an MTA (this is not difficult,
> but requires some efforts to do it correctly), then go for the MDA
> method. reread fetchmail docs in both cases.

In further reading tonight, sendmail grew the libmilter freature at 
8.12, which is the base version running here, and yum won't update it, 
says its current.

Right now, I'm looking at the  
site, trying to see how this is done.

But, here is the headache:  At no place in the various files sitting 
in /etc/mail that serve to configure sendmail, is there an example of 
how to configure sendmail to make use of these feature facilities.

Spamassassin 3.10 contains only very scant references to using it with 
sendmail, apparently sanctioning only the procmail interface, which in 
turn then is set to call spamc or spamassassin, adding needless time 
wasting cpu cycles to what should be a pretty simple job.  I fail to 
understand why (although it will take smarter people than me what with 
sendmails configuration complexity) there is no readily published 
recipe for incorporating spamc into the sendmail processing chain, 
either by pipeing, or when the libmilter feature is there?

Or am I simply on the wrong mailing list?  I've sent 3 subscribe 
messages to the getmail-user list over the last 3 days with no re

Re: getmail?

[Craig White]

On Thu, 2006-02-09 at 22:36 +0100, mouss wrote:
> Craig White a écrit :
> > 
> > oops...meant to send to list...
> > 
> > and by all means...
> > 
> > setup dovecot for maildir
> > 
> > use IMAP not POP3 for you local email. IMAP frees you from tying your
> > email experience to one program on one computer and makes all programs
> > on all computer on your network accessible to email. POP3 is for mass
> > mail delivery to end users. IMAP is for intelligent mail usage.
> 
> imap has benefits, but it also brings problems. That really depends on
> his situation and mostly on his MUAs. imap is an "unversioned protocol".
> The specs changed but the versions did not. so different MUAs implement
> different things to talk with different servers when you need to
> restart your MUA to reread your mail, then things aren't as good as they
> should...
> 
> that said, I use imap...

Stop using Outlook - Microsoft has deliberately crippled Outlook's IMAP
functionality - you can tell that they know how to do IMAP when you use
Outlook express...but again, this has nothing whatsoever to do with
spamassassin so I really don't want to prolong the thread.

Craig

Re: getmail?

[mouss]

Craig White a écrit :
> 
> oops...meant to send to list...
> 
> and by all means...
> 
> setup dovecot for maildir
> 
> use IMAP not POP3 for you local email. IMAP frees you from tying your
> email experience to one program on one computer and makes all programs
> on all computer on your network accessible to email. POP3 is for mass
> mail delivery to end users. IMAP is for intelligent mail usage.

imap has benefits, but it also brings problems. That really depends on
his situation and mostly on his MUAs. imap is an "unversioned protocol".
The specs changed but the versions did not. so different MUAs implement
different things to talk with different servers when you need to
restart your MUA to reread your mail, then things aren't as good as they
should...

that said, I use imap...

Re: getmail?

[mouss]

Gene Heskett a écrit :
> On Thursday 09 February 2006 03:47, jdow wrote:
> 
>>From: "Gene Heskett" <[EMAIL PROTECTED]>
>>
Procmail calls SpamAssasin and feeds the return off to the spool
file.
>>>
>>>Ok, sub getmail for both fetchmail and procmail, since getmail can
>>>handle the SA pipeing you are doing with procmail.  Then run dovecot
>>>on that box to serve kmail on this box?  I have the kmail fetching
>>>turned off on that box, so I'd assume I can give getmail a trial run
>>>and see if what it fetches it shows up in kmail on that firewall box
>>>as a new mail, if that works, then setup dovecot as a pop3 server to
>>>serve the kmail requests from this box.  Have I got it right?  All
>>>running as the user gene I'd assume?
>>
>>Only if getmail combines fetchmail and procmail including procmail's
>>ability to write rules for redirecting mail or applying filtering to
>>it.
>>
>>Kmail would simply read from the imap port you create. It's still
>>write to your ISP's mail server.
>>
> 
> First, getmail is out as far as putting it on the FW box, the python 
> install there is several releases too old.
> 
> Further reading on fetchmail tells me that it hands the incoming mail 
> off to sendmail via stuffing it into port 110.  Datapoint as I try to 
> get my head around the mechanics of this.
> 
> It is sendmail then that listens on port 110 and writes to 
> the /var/spool/mail/username file. Datapoint again.  
> 
> So there is a potential place to put a |spamc| is there not?, between 
> fetchmail and port 110?  A hack to fetchmail maybe?  Datapoint.
> 
> Experimental results..  I grabbed a copy of /var/spool/mail/gene to 
> another file while it had some content, then did a 'cat filename|spamc 
> 
>>filename2'
> 
> 
> This did properly scan & add the headers that it had done so to the 
> first of the 3 messages that were merged into filename, but did not 
> re-trigger itself on the next 2 messages also in that file, therefore 
> they were not scanned and marked up by spamc.
> 
> So it appears that wherever spamc is inserted into the path, it must be 
> presented with a single message complete with an EOF indicator of come 
> kind.  Looking at the src file, it doesn't appear there is a quick, 
> dirty, and 100% dependable way to filter the output of the cat command 
> and break it up into one stream per message. But I haven't ran a 
> tcpdump to see how its formatted on the network traffic yet.  The only 
> thing I can see is linefeed,linefeed,From and since theres no way to 
> stop me from doing it in a message I send, it doesn't look that 
> reliable to me.
> 
> From 
> 
> Is that treated as a new message?  I think not.  I'm going to go look at 
> the fetchmail code, maybe I can make a patch for it to do this.
> 

once fetchmail has read the message, it can:

- put it in files. this is what you do.

- run an MDA. so you could run procmail or maildrop or a (correct)
script. In short, fetchmail runs a command (it pipes the message).

- forward to an smtp server. This is the simplest to configure if you
can afford to run an MTA.

you'll need to choose which method is appropriate for your situation. if
you don't feel yourself installing an MTA (this is not difficult, but
requires some efforts to do it correctly), then go for the MDA method.
reread fetchmail docs in both cases.

Re: getmail?

[Kris Deugau]

Gene Heskett wrote:
Further reading on fetchmail tells me that it hands the incoming mail 
off to sendmail via stuffing it into port 110.  Datapoint as I try to 
get my head around the mechanics of this.


Er, I think you mean "..stuffing it into port 25.".  Port 110 is where 
fetchmail connects to on the *other* side to download your mail via 
POP3.  IIRC fetchmail also has a mode of operation where it feeds 
directly to procmail rather than feeding a local MTA;  but I can't 
comment further as I've never used (or had need of ) fetchmail.


It is sendmail then that listens on port 110 and writes to 
the /var/spool/mail/username file. Datapoint again.  

> So there is a potential place to put a |spamc| is there not?, between
> fetchmail and port 110?  A hack to fetchmail maybe?  Datapoint.

sendmail typically uses procmail to actually write to the mailbox, and 
it's very well-documented how to call SpamAssassin from a procmail 
recipe.  Aside from a great deal of other stuff I do to my mail before 
stuffing it into a server-side mail folder, this is what I've used since 
I started using SA, and I've never had trouble unless I was making 
changes somewhere that affected how I might call SA.  (Switching from 
TCP socket to Unix socket, for instance...)


The easiest next step for you is probably to look for procmail in the SA 
docs, and put the appropriate bits in your ~/.procmailrc file.  To start 
with, you might want to just have SA tag mail, and leave out the extra 
procmail bits that actually file the tagged mail in a spam folder.


So it appears that wherever spamc is inserted into the path, it must be 
presented with a single message complete with an EOF indicator of come 
kind.


Yep.  Doing this through the program that actually writes the message to 
the mailbox is normally the simplest way to do so.


 Looking at the src file, it doesn't appear there is a quick, 
dirty, and 100% dependable way to filter the output of the cat command 
and break it up into one stream per message. But I haven't ran a 
tcpdump to see how its formatted on the network traffic yet.


Formatting on the wire (POP3 or SMTP) will be different (slightly) than 
storage in an mbox mail folder.  


 The only 
thing I can see is linefeed,linefeed,From and since theres no way to 
stop me from doing it in a message I send, it doesn't look that 
reliable to me.


Most delivery agents actually watch for that, and prefix any appropriate 
lines with something - usually a >.  procmail does for sure.  (In 
theory, this should not be necessary as your mail has already been run 
through a POP3 delivery and download... but if the server uses maildir 
instead of mbox folders, it may not.)


Is that treated as a new message?  I think not.  I'm going to go look at 
the fetchmail code, maybe I can make a patch for it to do this.


The cannonical separator for mbox mail folders is a line beginning with 
"From " (note the space!).  If you look for "\n\nFrom ", you'll miss the 
first message because it won't have any preceding newlines.Many 
POP3 daemons actually read more of the line to make sure it looks 
something like:


From [EMAIL PROTECTED]  Sun Jan  1 00:00:02 2006

But some don't.  :/  I got bitten by that briefly with a custom delivery 
agent I wrote for the domain hosting system I'm administering.


-kgd

Re: getmail?

[Gene Heskett]

On Thursday 09 February 2006 03:47, jdow wrote:
>From: "Gene Heskett" <[EMAIL PROTECTED]>
>
>>>Procmail calls SpamAssasin and feeds the return off to the spool
>>> file.
>>
>> Ok, sub getmail for both fetchmail and procmail, since getmail can
>> handle the SA pipeing you are doing with procmail.  Then run dovecot
>> on that box to serve kmail on this box?  I have the kmail fetching
>> turned off on that box, so I'd assume I can give getmail a trial run
>> and see if what it fetches it shows up in kmail on that firewall box
>> as a new mail, if that works, then setup dovecot as a pop3 server to
>> serve the kmail requests from this box.  Have I got it right?  All
>> running as the user gene I'd assume?
>
>Only if getmail combines fetchmail and procmail including procmail's
>ability to write rules for redirecting mail or applying filtering to
> it.
>
>Kmail would simply read from the imap port you create. It's still
> write to your ISP's mail server.
>
First, getmail is out as far as putting it on the FW box, the python 
install there is several releases too old.

Further reading on fetchmail tells me that it hands the incoming mail 
off to sendmail via stuffing it into port 110.  Datapoint as I try to 
get my head around the mechanics of this.

It is sendmail then that listens on port 110 and writes to 
the /var/spool/mail/username file. Datapoint again.  

So there is a potential place to put a |spamc| is there not?, between 
fetchmail and port 110?  A hack to fetchmail maybe?  Datapoint.

Experimental results..  I grabbed a copy of /var/spool/mail/gene to 
another file while it had some content, then did a 'cat filename|spamc 
>filename2'

This did properly scan & add the headers that it had done so to the 
first of the 3 messages that were merged into filename, but did not 
re-trigger itself on the next 2 messages also in that file, therefore 
they were not scanned and marked up by spamc.

So it appears that wherever spamc is inserted into the path, it must be 
presented with a single message complete with an EOF indicator of come 
kind.  Looking at the src file, it doesn't appear there is a quick, 
dirty, and 100% dependable way to filter the output of the cat command 
and break it up into one stream per message. But I haven't ran a 
tcpdump to see how its formatted on the network traffic yet.  The only 
thing I can see is linefeed,linefeed,From and since theres no way to 
stop me from doing it in a message I send, it doesn't look that 
reliable to me.

From 

Is that treated as a new message?  I think not.  I'm going to go look at 
the fetchmail code, maybe I can make a patch for it to do this.

>{^_^}

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Re: getmail?

[Kris Deugau]

Gene Heskett wrote:
See above, half a gig, interleaved access.  pc1600 I think, haven't 
looked since last year when I blew the dust bunnies out in its annual 
blow job.


  The P3 boxes I've been running have PC133 memory, the domain 
hosting box is ECC EDO (yes, both) PC100 IIRC.  (It's a Compaq 
Proliant.)  If any of these systems had faster memory, that would 
probably increase their peak handling capacity.



(As for firewall services  For quite some time, my DSL connection
at home was through a P133/48M box.  It was quite happy to pass
full-bandwidth downloads through to my desktop while I was torturing
it with hosting a local Quake2 server for my LAN.  )


Good grief, no rest for the weary I take it?  Or is it so righteous it 
doesn't need a cooldown occasionally...  Not a Q, just an 
observation. :)


>:)  It's been retired for a while;  first replaced by a quad 
PII-Xeon/450/256M (also ECC EDO PC100), which was in turn replaced by an 
AMD Sempron 2400+ (cheapest CPU I could find new at the time) with 
DDR400 memory (IIRC - can't find the mobo [A7NX8] on Asus' website any 
more).  I would have used the dual P2 motherboard I had spare, but the 
LSI RAID card wouldn't work on that board.  :(


On the bright side, that old P133 will take an install of RH5.2 (which I 
found I need to compile a particular Q2 mod so it doesn't segfault).


-kgd

Re: getmail?

[jdow]

From: "Gene Heskett" <[EMAIL PROTECTED]>


Procmail calls SpamAssasin and feeds the return off to the spool file.


Ok, sub getmail for both fetchmail and procmail, since getmail can 
handle the SA pipeing you are doing with procmail.  Then run dovecot on 
that box to serve kmail on this box?  I have the kmail fetching turned 
off on that box, so I'd assume I can give getmail a trial run and see 
if what it fetches it shows up in kmail on that firewall box as a new 
mail, if that works, then setup dovecot as a pop3 server to serve the 
kmail requests from this box.  Have I got it right?  All running as the 
user gene I'd assume?


Only if getmail combines fetchmail and procmail including procmail's
ability to write rules for redirecting mail or applying filtering to it.

Kmail would simply read from the imap port you create. It's still write
to your ISP's mail server.

{^_^}

Re: getmail?

[Craig White]

On Thu, 2006-02-09 at 03:27 +0100, mouss wrote:
> Gene Heskett a écrit :
> > 
> > 
> > Except pipe it through spamc, although I'd assume a shell pipe might be 
> > able to do that, just haven't given it any great amount of thought yet.
> 
> it's better to run an MTA, so that fetchmail gets back to sleep fast.
> the MTA then will hand mail to SA "peacefully". you can configure how
> many messages get scanned simultaneousely. This way, even your old box
> won't cry.
> 
> if you don't like putting the MTA on the FW, then put it on any other
> box (provided the OS is supported by the MTA).

oops...meant to send to list...

and by all means...

setup dovecot for maildir

use IMAP not POP3 for you local email. IMAP frees you from tying your
email experience to one program on one computer and makes all programs
on all computer on your network accessible to email. POP3 is for mass
mail delivery to end users. IMAP is for intelligent mail usage.

Craig

Re: getmail?

[mouss]

Gene Heskett a écrit :
> 
> 
> Except pipe it through spamc, although I'd assume a shell pipe might be 
> able to do that, just haven't given it any great amount of thought yet.

it's better to run an MTA, so that fetchmail gets back to sleep fast.
the MTA then will hand mail to SA "peacefully". you can configure how
many messages get scanned simultaneousely. This way, even your old box
won't cry.

if you don't like putting the MTA on the FW, then put it on any other
box (provided the OS is supported by the MTA).

Re: getmail?

[Gene Heskett]

On Wednesday 08 February 2006 03:48, jdow wrote:
>From: "Gene Heskett" <[EMAIL PROTECTED]>
>
>> Greetings all;
>>
>> I just stumbled over an announcement on freshmeat about getmail as a
>> substitute for fetchmail, but from looking at the web page & FAQ,
>> its not clear if getmail can both filter by passing the incoming
>> mail thru SA, and put it in the /var/spool/user mailfile format that
>> kmail expects to retrieve incoming messages from.  I'd expect it
>> needs a scratchfile someplace in order to do the pipeing thru spamc,
>> but its not at all clear from the web page.
>>
>> Its my intention of moveing the spamc function from being used as a
>> filter via a pipe from one of kmails filter settings, to a function
>> of getmail which would be asynch from kmail, and possibly making
>> kmail many times more responsive is as its UI is locked for many
>> seconds at a time while this filter is waiting for spamc to finish. 
>> If getmail can run in the background like fetchmail is now I
>> hopefully wouldn't have the lags slap me in the face.  As is kmail
>> can hang long enough for me to type a line and a half, and while
>> thats disconcerting, the slow cursor movements when attempting to go
>> back and fix a typo are 7000% exasperating.
>>
>> If this transfer of jobs can be done, then kmails filters would be
>> reduced to just sorting the mail to the right folder, something I
>> think it can do many times faster than it is now since its always
>> waiting on spamc.
>>
>> Does anyone here have any experience with previous versions of this
>> utility?  And if so, any hints to toss my way?
>
>What does getmail supposedly bring to the table that fetchmail does
> not? Fetchmail in daemon mode does everything you want.
>{o.o}

Except pipe it through spamc, although I'd assume a shell pipe might be 
able to do that, just haven't given it any great amount of thought yet.


-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Re: getmail?

[Gene Heskett]

On Wednesday 08 February 2006 11:37, Kris Deugau wrote:
>Gene Heskett wrote:
>> A good idea, if the firewall box had the cojones to do it.  Its a
>> 500mhz k6-III, and already running the firewall duties fairly fast,
>> but adding that load to it does concern me somewhat.
>
>I don't recall you posting any details of what sort of mail volume
>you're running, but I have some experience running SA on small hosts
> for anywhere from low to high (for the hardware) mail volumes.
>
Well, I'd guess somewhere between 1 and 2 thousand messages a day on a 
busy day, 300 or more on a light day.  So that has the possibilioty of 
maybe 40k a month?

>One system I had an account calling SA on for a time was a P133 with
>~48M memory, IIRC.  (This was ~SA2.2 or so, though.)  That system
>handled maybe five active accounts, to the tune of ~20K messages per
>month.  Along with webhosting, FTP, SSH, VNC, some bits and pieces of
> X, etc, etc, etc.  (I think that box may still be in service.)
>
>Somewhere along the line I moved my own domain off of that machine
> (due to @#^#$%^$^% Bell Canada doing something funky to its DSL
> connection for a month or so) and onto a dual PII/266/128M box. 
> Between changes in my mail flow and SA versions, that system held up
> fine under
>SA2.6something, with my ~6K spams per month, plus Asian mailing lists
>that I still haven't figured out where I got on, plus webhosting,
>anonFTP for my domain, etc, etc, etc...  I replaced (with a dual
> P3/866) it because I wanted more CPU power for something, not because
> it couldn't handle the day-to-day mail load.  
>
>In a more formal environment, under higher load, I've been
> administering both a domain hosting box, and "regular" ISP customer
> spamfilter box for quite a few years now.  The hosting box has seen a
> peak of ~40 domains, mostly spamfiltered, and hosts EVERYTHING on one
> box.  Currently, it's a PII/400/192M, and the overall load is
> near-zero most of the time.  It's been handling between 8K and 10K
> messages per week for at least a year and a half.

Sounds good from here, so my hardware should be able to do this I'd 
think.  500mhz k6-III, half a gig of ram on an elderly tyan S-1590 
mobo.

>The spamfilter box has gone through a number of incarnations.  It
>started out as a P2/450/256M, and due to the SA load was quickly
>upgraded to 512M.  Due to customer demand, it was later replaced with
>the current hardware (P3/866/512M).  At peak usage, it was handling
> over 100K messages per week, to the tune of ~1G of data volume - for
> more than 6 months.  In December 2004, the systems that feed it had
> front-line RBLs added and the message volume dropped to ~40K per
> week, where it's pretty much stayed since then.  (It's at the tail
> end of ~3 relays - ah, legacy setups.)
>
>It *did* need a little tweaking and beating to keep mail moving, in
> the form of tuning sendmail and MIMEDefang process/load limits, and a
> custom program called from /etc/procmailrc to delay SA processing and
> mail delivery while system load was high.  That box was last rebooted
> in April 2004.  
>
>Your idling K6 should NOT have trouble with your mail flow.  About the
>only upgrade you MIGHT want to make is add more memory (you didn't say
>how much is in it), but I don't think that will really be an issue.

See above, half a gig, interleaved access.  pc1600 I think, haven't 
looked since last year when I blew the dust bunnies out in its annual 
blow job.

>(As for firewall services  For quite some time, my DSL connection
> at home was through a P133/48M box.  It was quite happy to pass
>full-bandwidth downloads through to my desktop while I was torturing
> it with hosting a local Quake2 server for my LAN.  )

Good grief, no rest for the weary I take it?  Or is it so righteous it 
doesn't need a cooldown occasionally...  Not a Q, just an 
observation. :)

>-kgd

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Re: getmail?

[Gene Heskett]

On Wednesday 08 February 2006 03:57, jdow wrote:
>From: "Gene Heskett" <[EMAIL PROTECTED]>
>
>>>Ultimately, using fetchmail or getmail or whatever mail retrieval
>>> tool you use isn't likely to make much of a difference...segregate
>>> your services and don't force your desktop computer to do
>>> everything.
>>>
>>>This really has little to do with spamassassin so I hesitate to go
>>> on.
>>
>> Well, SA is the main cause of the lag, so its almost germain. :)
>
>This is the topology used here, one for me and one for Loren.
>
> SpamAssassin
> /\  /->pop3 email
> reading fetchmail->procmail->/var/spool/mail/->Dovecot<
> \->imap spam/ham
> samples
>
>Procmail calls SpamAssasin and feeds the return off to the spool file.

Ok, sub getmail for both fetchmail and procmail, since getmail can 
handle the SA pipeing you are doing with procmail.  Then run dovecot on 
that box to serve kmail on this box?  I have the kmail fetching turned 
off on that box, so I'd assume I can give getmail a trial run and see 
if what it fetches it shows up in kmail on that firewall box as a new 
mail, if that works, then setup dovecot as a pop3 server to serve the 
kmail requests from this box.  Have I got it right?  All running as the 
user gene I'd assume?

>In your case you'd skip the pop3 part and use imap mailboxes if you do
>not use a remote X-Window to your user machine and its KMail account.
>Fetchmail runs once every one or two minutes getting email from three
>or four accounts for each of us and funneling it all into one account.
>That seems to suit our current uses. And reading delays are zilch. The
>mail readers are on other machines and as it happens we both use
>Outlook. It polls the appropriate Dovecot pop3
>mailbox once every few minutes and stores it on our main workstations.
>(On that one I gotta give OE some credit. It is more flexible for
> storage than imap can manage.)
>
>{^_^}

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Re: getmail?

[Kris Deugau]

Gene Heskett wrote:
A good idea, if the firewall box had the cojones to do it.  Its a 500mhz 
k6-III, and already running the firewall duties fairly fast, but adding 
that load to it does concern me somewhat.


I don't recall you posting any details of what sort of mail volume 
you're running, but I have some experience running SA on small hosts for 
anywhere from low to high (for the hardware) mail volumes.


One system I had an account calling SA on for a time was a P133 with 
~48M memory, IIRC.  (This was ~SA2.2 or so, though.)  That system 
handled maybe five active accounts, to the tune of ~20K messages per 
month.  Along with webhosting, FTP, SSH, VNC, some bits and pieces of X, 
etc, etc, etc.  (I think that box may still be in service.)


Somewhere along the line I moved my own domain off of that machine (due 
to @#^#$%^$^% Bell Canada doing something funky to its DSL connection 
for a month or so) and onto a dual PII/266/128M box.  Between changes in 
my mail flow and SA versions, that system held up fine under 
SA2.6something, with my ~6K spams per month, plus Asian mailing lists 
that I still haven't figured out where I got on, plus webhosting, 
anonFTP for my domain, etc, etc, etc...  I replaced (with a dual P3/866) 
it because I wanted more CPU power for something, not because it 
couldn't handle the day-to-day mail load.  


In a more formal environment, under higher load, I've been administering 
both a domain hosting box, and "regular" ISP customer spamfilter box for 
quite a few years now.  The hosting box has seen a peak of ~40 domains, 
mostly spamfiltered, and hosts EVERYTHING on one box.  Currently, it's a 
PII/400/192M, and the overall load is near-zero most of the time.  It's 
been handling between 8K and 10K messages per week for at least a year 
and a half.


The spamfilter box has gone through a number of incarnations.  It 
started out as a P2/450/256M, and due to the SA load was quickly 
upgraded to 512M.  Due to customer demand, it was later replaced with 
the current hardware (P3/866/512M).  At peak usage, it was handling over 
100K messages per week, to the tune of ~1G of data volume - for more 
than 6 months.  In December 2004, the systems that feed it had 
front-line RBLs added and the message volume dropped to ~40K per week, 
where it's pretty much stayed since then.  (It's at the tail end of ~3 
relays - ah, legacy setups.)


It *did* need a little tweaking and beating to keep mail moving, in the 
form of tuning sendmail and MIMEDefang process/load limits, and a custom 
program called from /etc/procmailrc to delay SA processing and mail 
delivery while system load was high.  That box was last rebooted in 
April 2004.  


Your idling K6 should NOT have trouble with your mail flow.  About the 
only upgrade you MIGHT want to make is add more memory (you didn't say 
how much is in it), but I don't think that will really be an issue.


(As for firewall services  For quite some time, my DSL connection at 
home was through a P133/48M box.  It was quite happy to pass 
full-bandwidth downloads through to my desktop while I was torturing it 
with hosting a local Quake2 server for my LAN.  )


-kgd

Re: getmail?

[jdow]

From: "Gene Heskett" <[EMAIL PROTECTED]>


Ultimately, using fetchmail or getmail or whatever mail retrieval tool
you use isn't likely to make much of a difference...segregate your
services and don't force your desktop computer to do everything.

This really has little to do with spamassassin so I hesitate to go on.


Well, SA is the main cause of the lag, so its almost germain. :)


This is the topology used here, one for me and one for Loren.

SpamAssassin
/\  /->pop3 email reading
fetchmail->procmail->/var/spool/mail/->Dovecot<
\->imap spam/ham samples

Procmail calls SpamAssasin and feeds the return off to the spool file.

In your case you'd skip the pop3 part and use imap mailboxes if you do
not use a remote X-Window to your user machine and its KMail account.
Fetchmail runs once every one or two minutes getting email from three
or four accounts for each of us and funneling it all into one account.
That seems to suit our current uses. And reading delays are zilch. The
mail readers are on other machines and as it happens we both use
Outlook. It polls the appropriate Dovecot pop3
mailbox once every few minutes and stores it on our main workstations.
(On that one I gotta give OE some credit. It is more flexible for storage
than imap can manage.)

{^_^}

Re: getmail?

[jdow]

From: "Gene Heskett" <[EMAIL PROTECTED]>


Greetings all;

I just stumbled over an announcement on freshmeat about getmail as a 
substitute for fetchmail, but from looking at the web page & FAQ, its 
not clear if getmail can both filter by passing the incoming mail thru 
SA, and put it in the /var/spool/user mailfile format that kmail 
expects to retrieve incoming messages from.  I'd expect it needs a 
scratchfile someplace in order to do the pipeing thru spamc, but its 
not at all clear from the web page.


Its my intention of moveing the spamc function from being used as a 
filter via a pipe from one of kmails filter settings, to a function of 
getmail which would be asynch from kmail, and possibly making kmail 
many times more responsive is as its UI is locked for many seconds at a 
time while this filter is waiting for spamc to finish.  If getmail can 
run in the background like fetchmail is now I hopefully wouldn't have 
the lags slap me in the face.  As is kmail can hang long enough for me 
to type a line and a half, and while thats disconcerting, the slow 
cursor movements when attempting to go back and fix a typo are 7000% 
exasperating.


If this transfer of jobs can be done, then kmails filters would be 
reduced to just sorting the mail to the right folder, something I think 
it can do many times faster than it is now since its always waiting on 
spamc.


Does anyone here have any experience with previous versions of this 
utility?  And if so, any hints to toss my way?


What does getmail supposedly bring to the table that fetchmail does not?
Fetchmail in daemon mode does everything you want.
{o.o}

Re: getmail?

[Gene Heskett]

On Wednesday 08 February 2006 01:33, Craig White wrote:
>On Wed, 2006-02-08 at 01:10 -0500, Gene Heskett wrote:
>> Greetings all;

>> Does anyone here have any experience with previous versions of this
>> utility?  And if so, any hints to toss my way?
>
>
>personally, I think you should handle your email entirely differently.
> I have gathered that you use a number of computers around your house
> and thus, the logical method that I see is to set one computer up -
> not your primary desktop...as a mail server and have an MTA (sendmail
> or postfix) and a IMAP server (dovecot or cyrus) and run fetchmail
> and spamassassin on that system too. Mail would get retrieved by this
> system on very frequent intervals like you are doing now, it would be
> analyzed by SA and delivered to your IMAP spool or maildir or cyrus
> mailstore.

A good idea, if the firewall box had the cojones to do it.  Its a 500mhz 
k6-III, and already running the firewall duties fairly fast, but adding 
that load to it does concern me somewhat.  As it is, its job is fairly 
well defined and easy to maintain.  My contention is that things would 
be improved immensely if the SA functions were removed from the 
threadless execution kmail does.  Stuff that SA does are totally 
charged aganst kmails time at the cpu trough, and moving it to a 
background task seems like a desirable thing.

Thats not saying your idea is not a good one, it is, but that now 8 year 
old box would need a heart tansplant to do it all I fear.

>This would allow a lot more flexibility...
>
>You could then use any computer to read/respond to mail and with IMAP,
>if you have read it/deleted it on one system, all mail clients on any
>computer would likewise see the same.

That would be an advantage in that I could then do email from the shop 
box, probably while emc is carving parts on my milling machine.  As it 
is, I spend entirely too many hours in this chair when I could be up 
doing other things.

I'll think about it, maybe even do it.  If the box isn't up to it, thats 
a good excuse to upgrade it, right? :)

>Computer sluggishness from things like SA would not be apparent as
> they aren't occurring on your desktop system.
>
>Ultimately, using fetchmail or getmail or whatever mail retrieval tool
>you use isn't likely to make much of a difference...segregate your
>services and don't force your desktop computer to do everything.
>
>This really has little to do with spamassassin so I hesitate to go on.

Well, SA is the main cause of the lag, so its almost germain. :)

>Craig

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Re: getmail?

[Craig White]

On Wed, 2006-02-08 at 01:10 -0500, Gene Heskett wrote:
> Greetings all;
> 
> I just stumbled over an announcement on freshmeat about getmail as a 
> substitute for fetchmail, but from looking at the web page & FAQ, its 
> not clear if getmail can both filter by passing the incoming mail thru 
> SA, and put it in the /var/spool/user mailfile format that kmail 
> expects to retrieve incoming messages from.  I'd expect it needs a 
> scratchfile someplace in order to do the pipeing thru spamc, but its 
> not at all clear from the web page.
> 
> Its my intention of moveing the spamc function from being used as a 
> filter via a pipe from one of kmails filter settings, to a function of 
> getmail which would be asynch from kmail, and possibly making kmail 
> many times more responsive is as its UI is locked for many seconds at a 
> time while this filter is waiting for spamc to finish.  If getmail can 
> run in the background like fetchmail is now I hopefully wouldn't have 
> the lags slap me in the face.  As is kmail can hang long enough for me 
> to type a line and a half, and while thats disconcerting, the slow 
> cursor movements when attempting to go back and fix a typo are 7000% 
> exasperating.
> 
> If this transfer of jobs can be done, then kmails filters would be 
> reduced to just sorting the mail to the right folder, something I think 
> it can do many times faster than it is now since its always waiting on 
> spamc.
> 
> Does anyone here have any experience with previous versions of this 
> utility?  And if so, any hints to toss my way?

personally, I think you should handle your email entirely differently. I
have gathered that you use a number of computers around your house and
thus, the logical method that I see is to set one computer up - not your
primary desktop...as a mail server and have an MTA (sendmail or postfix)
and a IMAP server (dovecot or cyrus) and run fetchmail and spamassassin
on that system too. Mail would get retrieved by this system on very
frequent intervals like you are doing now, it would be analyzed by SA
and delivered to your IMAP spool or maildir or cyrus mailstore.

This would allow a lot more flexibility...

You could then use any computer to read/respond to mail and with IMAP,
if you have read it/deleted it on one system, all mail clients on any
computer would likewise see the same.

Computer sluggishness from things like SA would not be apparent as they
aren't occurring on your desktop system.

Ultimately, using fetchmail or getmail or whatever mail retrieval tool
you use isn't likely to make much of a difference...segregate your
services and don't force your desktop computer to do everything.

This really has little to do with spamassassin so I hesitate to go on.

Craig