Re: Is this a valid From: address?
Steve Brewin wrote: Secondly, assuming MailAddress and I are correct, is it a good idea for James to strictly enforce the RFC for originator fields? It does look invalid, but it's been a while since I've examined the spec exactly. There's a crazy regex that checks for email address format you could try as well to get confirmation another way... http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html I could see this enforcement as an optional mailet (or matcher). Most likely use would be to help catch spam, but they just forge the addresses nowadays anyway. -- Serge Knystautas President Lokitech >> software . strategy . design >> http://www.lokitech.com p. 301.656.5501 e. [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Is this a valid From: address?
On (2003/05/30 14:42), Steve Brewin wrote: > org.apache.mailet.MailAddress throws a parse exception proceessing the > following From: address... > > <"nlt: Mail Administrator"> > > My reading of RFC 822 suggests that MailAdress is correct, this is an > invalid From: address. > > Firstly, am I correct? I don't think so. It looks valid, because: 1) Unqualified localparts are legal, and 2) Quoted strings in localparts are legal, and whitespace is legal within them. > Secondly, assuming MailAddress and I are correct, is it a good idea for > James to strictly enforce the RFC for originator fields? I've just given up on implementing a strictly compliant syntactic validator because it's so complex. I just leave it up to $other_mta [1], now. I've looked at the C code used in $other_mta, and it's hair-raising. :-) Assuming you're smartrouting outbound mail through a more mature system, I'd recommend leaving the decision up to that host for now. After all, you're using James for extensibility and application integration, rather than high performance delivery and mature spool management and auditing features, right? Ciao, Sheldon. [1] Some established Unix MTA. No need to get into specifics and risk a flamewar. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> org.apache.mailet.MailAddress throws a parse exception proceessing the
> following From: address...
> <"nlt: Mail Administrator">
What exactly is the exception? Text and stacktrace context would be
helpful.
> My reading of RFC 822 suggests that MailAdress is correct, this is an
> invalid From: address.
RFC 2822 says (abbreviated):
address = mailbox / group
mailbox = name-addr / addr-spec
name-addr = [display-name] angle-addr
angle-addr = [CFWS] "<" addr-spec ">" [CFWS] / obs-angle-addr
group = display-name ":" [mailbox-list / CFWS] ";"
[CFWS]
display-name= phrase
mailbox-list= (mailbox *("," mailbox)) / obs-mbox-list
address-list= (address *("," address)) / obs-addr-list
addr-spec = local-part "@" domain
local-part = dot-atom / quoted-string / obs-local-part
domain = dot-atom / domain-literal / obs-domain
Normally, a mailbox is comprised of two parts: (1)
an optional display name that indicates the name of the recipient
(which could be a person or a system) that could be displayed to the
user of a mail application, and (2) an addr-spec address enclosed in
angle brackets ("<" and ">"). There is also an alternate simple form
of a mailbox where the addr-spec address appears alone, without the
recipient's name or the angle brackets. The Internet addr-spec
address is described in section 3.4.1.
When it is desirable to treat several mailboxes as a single unit
(i.e., in a distribution list), the group construct can be used. The
group construct allows the sender to indicate a named group of
recipients. This is done by giving a display name for the group,
followed by a colon, followed by a comma separated list of any number
of mailboxes (including zero and one), and ending with a semicolon.
Because the list of mailboxes can be empty, using the group construct
is also a simple way to communicate to recipients that the message
was sent to one or more named sets of recipients, without actually
providing the individual mailbox address for each of those
recipients.
Reviewing the above, and Appendix A of RFC 2822, it would appear that
<"nlt: Mail Administrator">
should be properly interpreted as a quoted local-part lacking a domain. I
do not see where the pseudo-BNF permits a missing domain. RFC 2821 section
4.1.2 says nothing different, other than adding that must be
accepted without a domain, as the exception.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
Noel,
> > org.apache.mailet.MailAddress throws a parse exception
> > proceessing the
> > following From: address...
> > <"nlt: Mail Administrator">
>
> What exactly is the exception? Text and stacktrace context would be
> helpful.
OK. Changed the code to print a stacktrace and here it is...
javax.mail.internet.ParseException: Out of data at position 26
at org.apache.mailet.MailAddress.(MailAddress.java:173)
at org.apache.mailet.MailAddress.(MailAddress.java:199)
at
org.apache.james.pop3mail.FetchPOP3Message.createMail(FetchPOP3Message.java:
149)
at
org.apache.james.pop3mail.FetchPOP3Message.fetch(FetchPOP3Message.java:71)
at
org.apache.james.pop3mail.FetchPOP3Folder.fetch(FetchPOP3Folder.java:68)
at
org.apache.james.pop3mail.FetchPOP3Store.fetch(FetchPOP3Store.java:64)
at
org.apache.james.pop3mail.FetchPOP3Mail.targetTriggered(FetchPOP3Mail.java:9
7)
at
org.apache.avalon.cornerstone.blocks.scheduler.DefaultTimeScheduler.doRunEnt
ry(DefaultTimeScheduler.java:242)
at
org.apache.avalon.cornerstone.blocks.scheduler.DefaultTimeScheduler.access$0
00(DefaultTimeScheduler.java:35)
at
org.apache.avalon.cornerstone.blocks.scheduler.DefaultTimeScheduler$1.run(De
faultTimeScheduler.java:217)
at
org.apache.excalibur.thread.impl.ExecutableRunnable.execute(ExecutableRunnab
le.java:90)
at
org.apache.excalibur.thread.impl.WorkerThread.run(WorkerThread.java:135)
The code that triggers this is...
protected Mail createMail(MimeMessage message, MailAddress recipient)
throws MessagingException
{
Collection recipients = new ArrayList(1);
recipients.add(recipient);
>>> MailAddress sender = new MailAddress(new
InternetAddress(message.getFrom()[0].toString()));
return new MailImpl(
getServer().getId(),
sender,
recipients,
message);
}
where >>> indicates the point the exception is thrown.
Incidentally, the package org.apache.james.pop3mail is a heavily refactored
version of org.apache.james.fetchmail with a bunch of enhancements and
corrections for obscure cases (such as this). I'll fold it back into
fetchmail once I have IMAP in and tested.
> > My reading of RFC 822 suggests that MailAdress is correct,
> > this is an invalid From: address.
>
> Reviewing the above, and Appendix A of RFC 2822, it would appear that
>
> <"nlt: Mail Administrator">
>
> should be properly interpreted as a quoted local-part lacking
> a domain. I
> do not see where the pseudo-BNF permits a missing domain.
> RFC 2821 section
> 4.1.2 says nothing different, other than adding that
> must be
> accepted without a domain, as the exception.
That's the way I interpreted it too. In short, a local-part without a domain
part is illegal.
I'ld like your thoughts on my second point regarding how strict James should
be on this. The email in question is the standard welcome email sent by a
major ISP in the UK. To be fair, I would guess that in 99.99% of cases this
email is read succesfully by a mail client. Rarely would an intervening MTA
enter the equation, though it is obviously sent by one. But such an email
>may< be introduced into James, as I have just discovered, in this case via
a fetchMail like route. What should James do with such a message? Propogate
it or reject it?
-- Steve
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
Sheldon wrote: > I don't think so. It looks valid, because: > > 1) Unqualified localparts are legal, and > 2) Quoted strings in localparts are legal, and whitespace is legal >within them. I agree James should be handling this as a quoted local part with no host or domain part. i.e. "nlt: Mail Administrator"@localhost What you say below I disagree with.. > I've just given up on implementing a strictly compliant syntactic > validator because it's so complex. I just leave it up to $other_mta > [1], now. I've looked at the C code used in $other_mta, and it's > hair-raising. :-) > Assuming you're smartrouting outbound mail through a more mature system, > I'd recommend leaving the decision up to that host for now. After all, > you're using James for extensibility and application integration, rather > than high performance delivery and mature spool management and auditing > features, right? It is not IMO our intention to create a dependant piece of software, nor should it be. James aims to be a fully functional MTA (and then some), we do strive to comply with all relevant rfcs, but as you say 822 & 2822 are the mother of all bitches, and seem to be comprised mainly of escapes allowing the perverse exceptions required to maintain historic compatibilities with mediaeval address formats. James doesn't rely on routing outbound mail though a more mature system, unless you consider JavaMail more mature (IMO *not*, just better funded) I'm quite sure that "power users" of James _will_ be gateing James with sendmail or some other worthy MTA, but James is equally at home as the primary MTA of SME systems. Finally, (rant nearly over!) in this particular case an unqualified local part shouldn't really have been used as the MAIL FROM address by an MTA with an ounce of brain, as James (or $other_mta) should have rejected it based upon it not being a valid local user, never mind any other considerations. d.
RE: Is this a valid From: address?
> javax.mail.internet.ParseException: Out of data at position 26 no @ presumably.. can james handle a quoted local part without an @ is the question. d.
RE: Is this a valid From: address?
> > javax.mail.internet.ParseException: Out of data at position 26 > no @ presumably.. can james handle a quoted local part without an > @ is the question. > I agree James should be handling this as a quoted local part with > no host or domain part. i.e. "nlt: Mail Administrator"@localhost Please point out where in either RFCs 821, 822, 2821, 2822 it indicates that such an address is legal. I looked earlier, and I do not see anything that permits a localpart with no domain, except for . A bug in the code, which certainly exists, is that it should check the position before trying to access the @, so that it can report the error that it wants to report. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
Danny, > > javax.mail.internet.ParseException: Out of data at position 26 > > no @ presumably.. can james handle a quoted local part > without an @ is the question. > > d. Well MailAddress expects an @, hence the exception. My feeling is that James should be agnostic to originator addresses and propogate them as is. I haven't had time to examine the code, let alone test, to understand the impact of this. >> I agree James should be handling this as a quoted local part with no host or domain part. >> i.e. "nlt: Mail Administrator"@localhost Why should @localhost be a valid default? We simply do not know what the domain is, so no default is safe. This is only one example of an infinite number of possible malformed originator addresses. James needs a rule that says how to deal with them. Once the rule is agreed, any code modifications should be relatively straight forward. -- Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> Well MailAddress expects an @, hence the exception. My feeling is that James > should be agnostic to originator addresses and propogate them as is. That would be propagating invalid content that cannot be used to reply: The originator fields also provide the information required when replying to a message. When the "Reply-To:" field is present, it indicates the mailbox(es) to which the author of the message suggests that replies be sent. In the absence of the "Reply-To:" field, replies SHOULD by default be sent to the mailbox(es) specified in the "From:" field unless otherwise specified by the person composing the reply. The SMTP handler would, and should, reject those addresses when passed via SMTP. You are getting these messages via IMAP with FetchMail? I am curious to know what Eric's Fetchmail (http://catb.org/~esr/fetchmail/) does when retrieving messages with invalid fields. > Why should @localhost be a valid default? We simply do not know what the > domain is, so no default is safe. Precisely. Which is why the RFCs mandate routable addresses. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
After a fit of similar frustration I wrote a short article on how to validate e-mail addresses programmatically. It's half serious and half entertainment, but it may be worth looking at. Someone in one of the comments below the article mentioned that O'Reilly's 'Mastering Regular Expressions' has a RegEx that can be used that is 'pretty good'. According to the comment: "The most comprehensive version of this expression that I have seen is printed in Mastering Regular Expressions from O'Reilly. It is on page 316 and weighs in at 6,598 bytes long." See "How to validate an e-mail address" - http://www.oreillynet.com/pub/wlg/2379 |-+-> | | | | | "Noel J. Bergman" | | | <[EMAIL PROTECTED]> | | | 05/30/2003 07:30 PM| | | Please respond to "James | | | Users List"| | | | |-+-> >-| | | | To: "James Users List" <[EMAIL PROTECTED]> | | cc: (bcc: Kevin Bedell/Systems/USHO/SunLife) | | Subject: RE: Is this a valid From: address? | >-| > > javax.mail.internet.ParseException: Out of data at position 26 > no @ presumably.. can james handle a quoted local part without an > @ is the question. > I agree James should be handling this as a quoted local part with > no host or domain part. i.e. "nlt: Mail Administrator"@localhost Please point out where in either RFCs 821, 822, 2821, 2822 it indicates that such an address is legal. I looked earlier, and I do not see anything that permits a localpart with no domain, except for . A bug in the code, which certainly exists, is that it should check the position before trying to access the @, so that it can report the error that it wants to report. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --- This e-mail message (including attachments, if any) is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, proprietary , confidential and exempt from disclosure. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender and erase this e-mail message immediately. --- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> > My feeling is that James > > should be agnostic to originator addresses and propogate them as is. > > That would be propagating invalid content that cannot be used > to reply: > > > > The SMTP handler would, and should, reject those addresses > when passed via SMTP. The way I see it is there are two distinct cases... 1) An MTA should reject locally originated messages that are invalid in anyway. As Danny says, no MTA should ever introduce invalid messages into the system. 2) An MTA should be configurable as to if it should reject remotely originated messages that are invalid but still deliverable. A relaying MTA may not wish to act as an Internet Message format cop. If a remote message can be delivered without modification, the MTA should be configurable to do so. > That would be propagating invalid content that cannot be used to reply Yes it would, but mail clients are free to choose to reject messages for any reason, including those with invalid formats, such as invalid originator addresses. Individual organisations choose to configure their MTAs to filter spam, viruses and inappropriate content. This is a similar case. They should be free to configure James to reject invalid, but deliverable messages. Maybe using the matcher/mailet approach suggested by Serge? -- Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> 2) An MTA should be configurable as to if it should reject remotely > originated messages that are invalid but still deliverable. > A relaying MTA may not wish to act as an Internet Message format cop. Please review RFC 2821 section 3.8.4. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> > 2) An MTA should be configurable as to if it should reject remotely
> > originated messages that are invalid but still deliverable.
> > A relaying MTA may not wish to act as an Internet Message format cop.
>
> Please review RFC 2821 section 3.8.4.
This section is about gateways, systems that connect different mail
transports, like UUCP and X400.
I don't think it applies to mails recieved using SMTP. The initial problem
at hand here (FetchPOP etc.) is lack of SMTP data.
I just did a test on my ISP's mailservers (chain of two Exim servers and two
unknown) and it accepted and delivered messages without any header and with
improperly formatted headers. When I specified no headers, the MTA inserted
a From: line with the SMTP sender (MAIL FROM:), but wheb I inserted a bogus
From: header ("From: Vincent de Lau") the header wasn't modified. In the
latter example, a mail client wouldn't have the faintest idea where the mail
came from (without looking at the Recieved: headers).
Vincent de Lau
[EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> > > 2) An MTA should be configurable as to if it should reject remotely > > > originated messages that are invalid but still deliverable. > > > A relaying MTA may not wish to act as an Internet Message format cop. > > Please review RFC 2821 section 3.8.4. > This section is about gateways, systems that connect different mail > transports, like UUCP and X400. See section 2.3.8, which says in part: For the purposes of this specification, firewalls that rewrite addresses should be considered as gateways, even if SMTP is used on both sides of them (see [11]). In this case, we're talking about POP3/IMAP fetching on one side, SMTP on the other. > I don't think it applies to mails recieved using SMTP. The initial problem > at hand here (FetchPOP etc.) is lack of SMTP data. Correct. I don't dispute whether or not Fetchmail should try to do something corrective. But I don't believe that MailAddress should be changed to support invalid addresses. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> > <"nlt: Mail Administrator">
> javax.mail.internet.ParseException: Out of data at position 26
Would you please try the following code:
if (pos >= address.length() || address.charAt(pos) != '@') {
That extra check should cause the correct exception to be thrown.
>>> MailAddress sender = new MailAddress(
>>> new
InternetAddress(message.getFrom()[0].toString()));
Check for a Sender: field (RFC 2822, section 3.6.2). If it exists, use it
in preference for the envelope.
> org.apache.james.pop3mail is a heavily refactored version
> of org.apache.james.fetchmail with a bunch of enhancements
> and corrections for obscure cases (such as this). I'll fold
> it back into fetchmail once I have IMAP in and tested.
Cool.
> I'ld like your thoughts on my second point regarding how strict James
should
> be on this.
James should adhere to the RFC. Being strict in what you send, lenient in
what you accept. If there is some action that Fetchmail can take in a
corrective fashion to properly bridge between POP3 and SMTP, that's
different from modifying MailAddress to propogate invalid addresses where
the RFC prohibits that behavior.
The James SMTP handler has the following:
if (sender.indexOf("@") < 0) {
sender = sender + "@localhost";
}
and similar for doRCPT. I won't get into whether or not that is appropriate
(Danny said that it is; I still can't find where the RFC allows it), but it
does ensure a correct address.
> The email in question is the standard welcome email sent by a
> major ISP in the UK.
Interesting. Ask them about it. I'm curious to know what their postmaster
has to say.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
Noel, RFC 2821 section 3.8.4 applies to gateways. My point (2) refers to MTAs acting as a relay. >From RFC 2821 section 3.7, Relaying... "As discussed in section 2.4.1, a relay SMTP has no need to inspect or act upon the headers or body of the message data and MUST NOT do so except to add its own "Received:" header (section 4.4) and, optionally, to attempt to detect looping in the mail system (see section 6.2)." I was wrong in as much as there is no optional element to this. According to the RFC, a relaying MTA should never act as an Internet Message format cop. It shouldn't even be inspecting the headers, let alone rejecting a message due to their content. The case of fetchmail et al. is interesting. Mail fetched from a POP3 or IMAP server is NOT Internet mail as defined by RFC2821 as POP3 and IMAP are a foreign (non-SMTP) systems. Nor is an MTA configured to fetch mail from a POP3 or IMAP server an SMTP gateway >>> as long as the fetched mail is delivered locally <<<. According to the RFC, the server is acting as a gateway only if it is injecting mail into the internet, that is, delivering remotely over the 'net. One of the modifications I have made to fetchmail is to have the default configuration reject mail not intended for a recipient in a local domain. The original reason for doing this was that mail served by POP3/IMAP servers has arrived on them because they are the local delivery points for the domain(s) being served. Fetchmail is simply acting as a POP3/IMAP client that happens to have some very powerful mail tools at its disposal. They should not be used to deliver the fetched mails anywhere but locally. We now have a second reason for this restriction. Doing otherwise would create an SMTP gateway. There are circumstances when it is impossible to fulfil the rules for a gateway as specified by RFC2821, such as we are unable to correct invalid originator fields. If we can't guarantee to lay by the rules, we shouldn't enter the game. Hopefully this clarifies the responsibilities on James when acting as an SMTP relay and why a fetchmail configuration is not, and should never act as, an SMTP gateway. -- Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> RFC 2821 section 3.8.4 applies to gateways. My point (2) refers to MTAs > acting as a relay. > According to the RFC, a relaying MTA should never act as > an Internet Message format cop. It shouldn't even be > inspecting the headers, let alone rejecting a message > due to their content. Since POP3 isn't SMTP, it does seem to me that the gateway concept does apply. When acting as an SMTP relay, James does not check those headers. James tries not to touch the content of a message, except where a matcher or mailet might chose to do so. > Mail fetched from a POP3 or IMAP server is NOT Internet mail > as defined by RFC2821 as POP3 and IMAP are a foreign (non-SMTP) > systems. This is why I would view it as a gateway situation. It certainly isn't a relay. Fetchmail is attempting to fabricate an SMTP envelope from SMTP content, so that it can insert it into the SMTP spool for processing along with other mail. If the message had arrived via SMTP, James would have had a valid SMTP envelope. > Fetchmail is simply acting as a POP3/IMAP client that happens to > have some very powerful mail tools at its disposal. They should > not be used to deliver the fetched mails anywhere but locally. > There are circumstances when it is impossible to fulfil > the rules for a gateway as specified by RFC2821, such as > we are unable to correct invalid originator fields. If > we can't guarantee to lay by the rules, we shouldn't enter > the game. It seems to me that James is capable of ensuring that a Fetchmail gateway does not insert mail with invalid routing information. It caught the one that you received. A question is whether or not Fetchmail has the tools needed to properly construct a valid envelope. This has been a subject of discussion in the past, and I believe that the there were changes made to faciliate administrator control over the fabrication process. Certainly if Fetchmail cannot provide a valid envelope, James should not accept it. If Fetchmail were optionally able to take fetched messages for local addresses and deposit them directly into the correct message stores, I don't believe that the POP3Handler would care if the content were bogus. The loss of the mailet pipeline is another issue. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> Since POP3 isn't SMTP, it does seem to me that the gateway > concept does > apply. RFC2821 is clear enough. If James/fetchMail is only delivering locally, it is not an SMTP gateway as defined therein. Semantically, it is a gateway between a POP3/IMAP server and James' SMTP spool, but that isn't the same thing. We are free to do what we consider to be 'the right thing', which is what I am trying to get a consensus on. > If the message had arrived via SMTP, James > would have had > a valid SMTP envelope. Not necessarily. Forget about fetchmail for a moment. If an SMTP MTA that allows invalid envelopes injects a such a message into the 'net , James would receive a message with an invalid envelope. James would have to deal with it according to RFC2821. Does this happen? Oh yes. I tried inject mail with an invalid From: header (missing domain part) via a number of non-James MTAs and didn't get a single rejection or failed delivery. One rewrote the From: header adding its hostname as the domain part. This SMTP relay made the From: header legal, but according to the RFC acted illegally as the headers should not be modified by a relay! So, in real world deployments, the statement "If the message had arrived via SMTP, James would have had a valid SMTP envelope" is incorrect. The real world does not enforce the RFCs and does not play by the rules! > If Fetchmail were optionally able to take fetched messages for local > addresses and deposit them directly into the correct message > stores, I don't > believe that the POP3Handler would care if the content were > bogus. The loss > of the mailet pipeline is another issue. I do not think this is a viable approach as... 1) If fetchmail bypasses the SMTP spool we lose all the benefits of James's delivery chain. 2) Without going through mailet processing, we cannot determine "the correct message stores". It seems to me that to progress we need to agree that... 1) fetchmail will only deliver locally 2) Further consideration is required regarding James' policy for dealing with invalid SMTP envelopes however they are received. This should be at a different time in a different thread. 3) As an >interim< measure, in my fetchmail enhancements, I will modify the code so that a From: header with a missing domain part is made valid by appending the local host name (didn't Danny suggest that a while back)? Adios! -- Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Is this a valid From: address?
> > If the message had arrived via SMTP, James > > would have had > > a valid SMTP envelope. > > Not necessarily. Forget about fetchmail for a moment. If an SMTP MTA that > allows invalid envelopes injects a such a message into the 'net , James > would receive a message with an invalid envelope. James would have to deal > with it according to RFC2821. Yes it would James assumes unqualified names used in the SMTP conversation to be local users and appends @localhost. Admttedly this is v. simplistic, but it works. > So, in real world deployments, the statement "If the message had > arrived via > SMTP, James would have had a valid SMTP envelope" is incorrect. No it is correct. James would either have rejected an invalid address or appended @localhost to an unqualified one. > The real > world does not enforce the RFCs and does not play by the rules! Thats not true, the real world does play by the rules, thats what the rules are for. If people didn't play by the rules the whole shaky edifice would come tumbling down, and you would get no mail at all. The issue here is that the mail headers are (perversely) not used by SMTP to route mail, the addresses in the RCPT TO: and MAIL FROM: commands are. It is up to the mail clients at each end to worry about the content of the FROM, TO, CC, BCC headers. > It seems to me that to progress we need to agree that... > > 1) fetchmail will only deliver locally No I disagree, fetch needs only to apply some sensible rules. > 2) Further consideration is required regarding James' policy for dealing > with invalid SMTP envelopes however they are received. This should be at a > different time in a different thread. James is quite happy rejecting invalid adresses, and appending @localhost to valid usernames. Perhaps James ought to append @fqldn instead, but its a small point. Otherwise it is pretty much a case of rejecting mail with illegal headers, something it is our *right* to do, if you don't like that make up your own rules, James is a good platform for implementing them. d.
RE: Is this a valid From: address?
Danny beat me to it, and I agree with most of his points. However, since I had already written this before Danny's reply came in, I might as well post it ... > RFC2821 is clear enough. If James/fetchMail is only delivering locally, it > is not an SMTP gateway as defined therein. Fetchmail doesn't know that. :-) > > If the message had arrived via SMTP, James would have had > > a valid SMTP envelope. > Not necessarily. Forget about fetchmail for a moment. If an SMTP MTA that > allows invalid envelopes injects a such a message [...] I was referring to the SMTP commands MAIL and RCPT. And the way that James handles those right now is by adding @localhost to any address that lacks a domain, matching the proposal from you and Danny. James would not correct (or see) a bad header. Please note that a side effect of using @localhost instead of @domain is that if the message needs to be relayed, the next MTA may reject it. I've seen MTAs that detect and reject MAIL commands with an explicit localhost domain. This is not necessarily a bad thing. > I do not think [that direct storage] is a viable approach as... I agree, which was why I pointed out the pipeline issue. So now we've established that the messages must go through the mailet pipeline. > 1) fetchmail will only deliver locally Fetchmail seems primarily intended to retrieve mail for local domains, but that does not mean local storage. Fetchmail is just a tool, which may need to implement additional rules. Prior work was done to address recipient side issues. You are addressing the case where the origin information is defective. > 2) Further consideration is required regarding James' policy for dealing > with invalid SMTP envelopes however they are received. James' policy should be based upon the RFC documents, IMO. > 3) I will modify the code so that a From: header with a missing domain > part is made valid by appending the local host name All From headers, or just [0]? What about other address headers, e.g., Sender, To, Bcc, etc.? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Fetchmail (was Re: Is this a valid From: address?)
Steve Brewin wrote: I do not think this is a viable approach as... 1) If fetchmail bypasses the SMTP spool we lose all the benefits of James's delivery chain. 2) Without going through mailet processing, we cannot determine "the correct message stores". I agree with this, because the mailet capabilities are probably one of the reasons for ever wanting to use fetchmail anyway. My main reason for it was to get some spam "weeding". However that reason has mostly disappeared now because of Mozillas excellent Junk mail handling (IMO, the client is the only place for content-based spam-handling, and that is the only type that seems to be successful now) It seems to me that to progress we need to agree that... 1) fetchmail will only deliver locally 2) Further consideration is required regarding James' policy for dealing with invalid SMTP envelopes however they are received. This should be at a different time in a different thread. 3) As an >interim< measure, in my fetchmail enhancements, I will modify the code so that a From: header with a missing domain part is made valid by appending the local host name (didn't Danny suggest that a while back)? I am a bit concerned about "will only deliver locally". How do you ensure that? The first problem is that the account you fetch from is usually never a local account, and should not be set up as a handled domain else you will get all sorts of other problems. I found that mail with multiple recipients caused problems when used with fetchpop, either generating duplicates if I forwarded all to my local account or risked being resent outwards if I tried to be smarter. So there seems to be a lot of reasons for not injecting fetched mail into James's delivery chain. And yet the main reason for using fetchmail is to be able to use mailets on it. Let me try to invent some "user stories" for fetchmail: 1 - As an email user I want to collect the mail from all my separate accounts and have them put in my local account. Note: Personally, it works better for me to keep my accounts separate, like putting on different caps for "public internet mail", "personal friends", "business". My mail client handles that well for me. 2 - As an email user I want James to help clean out the spam. Note: My mail client handles this better. 3 - As a user of a joint e-mail account I would like to use a mailet that that delivers mail to the separate owners, e.g. mail for "John <[EMAIL PROTECTED]>" to [EMAIL PROTECTED], mail for "Mary <[EMAIL PROTECTED]>" to [EMAIL PROTECTED] and other mail for "[EMAIL PROTECTED]" to both. 4 - variations of 3 for business, e.g. with content-based routing 5 - As an owner of a B2B e-business setup I fetch mail from the incoming account and use mailets to distribute the content of e-mails to appropriate applications, via message queues, EJBs, files, SMTP, whatever. I would also handle logging of messages and protocols like ebXML Reliable Messaging in custom mailets. For ebXML there might be a corresponding mailet for outgoing messages if SMTP is used internally, but most probably I have another process that gets its input from the message queue, etc. Therefore I would usually in this scenario set up a James process to only handle fetched mail. Note: In this scenario one could relay the incoming messages to my James server and handle them directly in the SMTP spool, but there might be reasons for avoiding it. I am sure there are other scenarios I have not listed here, please add to the list. My conclusion so far is that fetched mail should go into a separate spool. The handling is completely different from what I want to do with the SMTP-spool and there are inherent dangers from injecting fetched mail into the SMTP spool. There are probably scenarios where I actually do want to forward or reply to a fetched mail (via SMTP), but this is, IMO, the exception rather than the rule and could probably be solved by a special mailet. Regards, tobe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Fetchmail (was Re: Is this a valid From: address?)
> -Original Message- > From: tobe [mailto:[EMAIL PROTECTED] > Sent: 15 June 2003 12:27 > To: James Users List > Subject: Fetchmail (was Re: Is this a valid From: address?) > > Steve Brewin wrote: > > Tobe, > I am a bit concerned about "will only deliver locally". How do you > ensure that? My musings have moved on to a new version of fetchmail that has been submitted, but not as yet accepted. In the submitted code, for each of the mail filter conditions, such as rejecting remote mail, rejection is now optional and the status of the rejected mail on the POP3 server can be deleted, marked as seen or left asis. The main reasons for rejecting mail at the fetchmail stage are to avoid the overhead of fetching mail that is simply going to be junked and the extra work of defining matcher/mailets to weed out such fetched mail. If it is decided to accept mail for which a filter condition applies, this is indicated by an X-Header added to the mail which describes the filter match. This allows the injected message to be detected by a matcher and processed in a mailet as required. Tere are many possible fetch 'stories'. The intent is to provide the flexibility to accomodate as many as possible. > The first problem is that the account you fetch from is > usually never a > local account, and should not be set up as a handled domain else you > will get all sorts of other problems. Depends on the story. There are a several where *not* defining the domain locally causes all sorts of other problems. As described above, the rejection of remote mail is now optional and can be done at the fetch stage or during matcher/mailet processing. > I found that mail with multiple recipients caused problems when used > with fetchpop, either generating duplicates if I forwarded all to my > local account or risked being resent outwards if I tried to > be smarter. A key difference between fetchpop and fetchmail is that the latter injects mail for a sole intended recipient. This avoids the problems you had with fetchpop. There are circumstances when it is not possible to determine the sole intended recipient, in this case the released version of fetchmail uses a default recipient. The submitted version adds the option of leaving it on the POP3 server. The latter accomodates scenarios in which the default recipient can be determined by refetching with a different POP3 account. This is explained in the comments within the submitted config.xml. > So there seems to be a lot of reasons for not injecting fetched mail > into James's delivery chain. And yet the main reason for > using fetchmail > is to be able to use mailets on it. > > Let me try to invent some "user stories" for fetchmail: The beauty of James is that it is extremely flexible and can accomodate these and a huge number of 'stories'. I can't really see the motivation for delivering fetched mail to another spool in any of your 'stories'. If you want to do something special with all fetched mail, use a matcher that checks for the X-fetched-from header, such as "fetchedfrom". All mail injected into the spool by fetchpop and fetchmail has this header appended for just this purpose. -- Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Fetchmail (was Re: Is this a valid From: address?)
Steve, you seem to have thought a lot about this. As long as I can handle the fetched mail fairly easily without risking something being sent out again by mistake, I don't get lots of duplicates, and I can still send mail to other people in the fetched domain, great! Thanks for your work on this. /tobe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Fetchmail (was Re: Is this a valid From: address?)
> -Original Message- > From: tobe [mailto:[EMAIL PROTECTED] > Sent: 17 June 2003 00:15 > To: James Users List > Subject: Re: Fetchmail (was Re: Is this a valid From: address?) > > > Steve, > > you seem to have thought a lot about this. Light relief from the day job. > As long as I can handle the fetched mail fairly easily > without risking > something being sent out again by mistake, I don't get lots of > duplicates, The version of fetchmail in CVS HEAD will do this. I didn't write this version, I think Danny did(?), so thank him. > and I can still send mail to other people in the fetched > domain, great! That is a simple mod. to config.xml. After the check for a local account, check if the recipient host is in the fetched domain... remoteTransport error Host is local but recipient not found Hope this helps. -- Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
