tl;dr: mail will be processed more reliably and with less latency.
----
Not yet connected to SVN, and only accessible by the secretarial
assistant team[1]:
https://whimsy-test.apache.org/secmail/
Highlights: auto-deployment, and secretary mail subscription. Not being
connected to svn means "file" doesn't work. There probably are other
bugs (e.g., I note that signature verification blows up). These will be
fixed over time.
Regarding auto-deployment. Commit a change to
https://github.com/apache/whimsy/tree/master/www/secmail and it will be
deployed the next time Puppet runs. Puppet is scheduled to run every 30
minutes.
---
Now for the bigger news, regarding secretary mail subscription:
Old process:
1) minotaur is subscribed to secretary@ (more precisely: listed in the
alias for secretary@).
2) minotaur files emails received to the appropriate mbox file
3) every 30 minutes (used to be 10, but there were problems), a script
would look for updates to the mbox file, parse the new emails, and store
the results in svn in the documents/received folder.
4) every time the secretary loads the main workbench page svn is updated
(there also is a cron job to do this every 60 minutes).
5) documents placed into svn would then be deleted or moved as they are
processed by the secretary.
New process:
1) whimsy-vm2 is subscribed to secretary@ (i.e., listed in the alias)
2) whimsy-vm2 parses emails as they are received.
3) selected documents will be imported into svn as they are processed by
the secretary
Advantages: less latency and less moving parts. If there was a hiccup
with LDAP or SVN or a particularly malformed spam email, the flow could
get interrupted and require manual intervention. Also, minotaur and
access to mail archives are scheduled to be replaced, this new flow will
not be affected by those changes.
Mail servers automatically retry when the target site is down - I sent
test emails before this was set up and they were processed the next day
when I got it working.
Should there be a problem with LDAP or SVN, it will only affect the
filing step; the secretary can simply retry that same step later.
Mails are processed individually as they are received. While my new
parsing logic can handle every email received by the secretary since
October of 2007, should something new come in that breaks the parser,
the most it can affect is that one email.
Worst case recovery, if an email needs to be reprocessed: resend or
forward that email to secret...@whimsy-vm2.apache.org (or to
secret...@apache.org).
- Sam Ruby
[1] https://whimsy.apache.org/roster/group/asf-secretary
