Re: Why do we need fast-fail?
On Tuesday 10 May 2005 18:59, [EMAIL PROTECTED] wrote: On Tue, 2005-05-10 at 16:37 +0200, [EMAIL PROTECTED] wrote: I think that this check should be done on the MAIL FROM or the RCPT TO and so not directly related to the STARTTLS and AUTH. I would add to my list: B2. mail from allowed only after AUTH/STARTTLS C2. rcpt to you can write to this recipient only when using AUTH/STARTTLS. The mechanisms for requiring STARTTLS are described in the STARTTLS RFC, http://www.faqs.org/rfcs/rfc2487.html see Section 5. Thank you Mike, This seems to confirm that we could check wether STARTTLS has been sent when we receive a MAIL FROM or RCPT TO and reply with 530 Must issue a STARTTLS command first when the recipient is not local (for example). Does anyone think that it would be useful to select wether to allow the STARTTLS or not depending on some business rule and not only via a smtp server configuration parameter? This would be a further use-case to add to the list, but I think this is not so usefull: I can't find why we should disable STARTTLS to specific IPs Consider that some servers relay though James from an internal trusted network, they do not need to issue STARTTLS, others however are relaying through a public network an are thus required to issue STARTTLS (perhaps even with client-certificate authentication). So we do not disable STARTTLS for the internal servers, but on the other hand do not require it either. --Søren - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. RD manager Phone: +45 72 30 64 00 TietoEnator IT+ A/S Fax:+45 72 30 64 02 Ved Lunden 12 Direct: +45 72 30 64 57 DK-8230 Åbyhøj Email: soren.hilmer at tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Why do we need fast-fail?
Consider that some servers relay though James from an internal trusted network, they do not need to issue STARTTLS, others however are relaying through a public network an are thus required to issue STARTTLS (perhaps even with client-certificate authentication). So we do not disable STARTTLS for the internal servers, but on the other hand do not require it either. Sure, but this seems the normal standard behaviour. We only need a configuration for StartTLSSupport = disabled | enabled | required Isn't this enough to support the STARTTLS reply? The check to see wether the relay is supported or not because of AUTH or STARTTLS will be done after the first RCPT so the extension point is the RCPT and not the STARTTLS. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Why do we need fast-fail?
Consider that some servers relay though James from an internal trusted network, they do not need to issue STARTTLS, others however are relaying through a public network an are thus required to issue STARTTLS (perhaps even with client-certificate authentication). So we do not disable STARTTLS for the internal servers, but on the other hand do not require it either. Sure, but this seems the normal standard behaviour. We only need a configuration for StartTLSSupport = disabled | enabled | required Isn't this enough to support the STARTTLS reply? The check to see wether the relay is supported or not because of AUTH or STARTTLS will be done after the first RCPT so the extension point is the RCPT and not the STARTTLS. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Serious bandwidth begin consumed by James
(cross posted as code change proposal at bottom) Having had a quick look at the RFCs it looks like the 552 error should not be sent until the final crlf.crlf has been received. Looking at james code (SMTPHandler.java): James recieves data command. Responds accordingly. James starts receiving data. Once hit's maximum limit, an exception causes james to send a 552 error (immediately). Now, this is where i might be missing somthing, but James then seems to return to reading commands, but as the state MESG_FAILED is set, parseCommand skips logging, and doesnt return 500 error messages! In terms of DOS, this seems ok, as once the mail hits the size limit, data is no longer built into a mail. (so cant do OutOfMemory DOS) Sure, you can use bandwidth, but you can do that anyway with lots of smaller messages that will get accepted and use more CPU! Assuming my understanding of the code and rfcs is correct, it means that james enters a blackhole phase, where james basically does nothing until it receives a proper smtp command, and then it doesnt get it's state reset, which all seems wrong in light of the rfcs. If i am right, it should: Notice to size limit has been reached, and stop buffering the message. Continue reading the message until crlf.crlf is reached Send 552 error! Reset state, and continue to accept other messages. I think it should do this by: Upon catching the size exception, gobble away at the CharTerminatedInputStream (need to keep a reference to this) until eof. then continue as before, but: Also, Get rid of MESG_FAILED state, and call resetState so we can allow the parsing/logging/responses-to further commands/messages to work properly! Does this make sense? Daniel. -Original Message- From: Chris Hane [mailto:[EMAIL PROTECTED] Sent: 10 May 2005 18:53 To: James Users List Subject: Re: Serious bandwidth begin consumed by James [EMAIL PROTECTED] wrote: I tested by: - I had two test servers both setup with james (Sender, Receiver) - I set the Receiver to a max message size of 7MB - I set the Sender to a max message size of 0MB (unlimited) - When I sent a 20MB file from Sender to Receiver I saw the following (very rough times from memory but I can repeat and get more accuruate timings if anyone cares. (I don't really at this point since I really wanted to solve the bounce issue I was having originally). 1. Receiver.smtp.log time 0 - connection accept and message started to be transmited 2. Receiver.smtp.log time 1 minute - message to large (552) 3. Sender.mailet.log time 3 minute - error message 552 message too large I made an assumption that message #2 on the Receiver generated the error message back to the Receiver; but based on what you said I assume this is informational that an error message will be sent once the message has been completely sent. However, wouldn't this behavior allow for a denial of service attack? Someone could repeatedly send large files and clog up my bandwidth.. This behaviour simply double the bandwith used: once to receive the message, once to send the bounce. You can configure james to avoid creating the bounce or to create the bounce with no message attached. hmm... Maybe I'm not fully understanding; but that is not what I am seeing. First, the Receiver email server should not bounce the email. Is that correct? Instead it sends back a 552 error code (I'm only dealing with the use case where the email exceeds the max message size ). Second, I have configured the Sender RemoteDelivery with a bounceProcessor and set the attachmentnone/attachment option. What I am seeing is a log message on the Receiving Email server indicating that a message has exceeded max size and will send a 552 error message. I'm assuming it will only send the error code back via the connection and *not* the entire message. Once this error message (552) is written to the Receiving.smtp.log file, the Sending email server continues to send the email. The send portion is not aborted. When I look at ntop which shows me network traffic broken by IP address / direction (in/out) / and protocol, I confirm that the entire 20MB message is sent (Sender - Receiver) and very little traffic (measured in KBs) is received (Receiver - Sender). Also, the Receiver contains the mbox of the actual user sending the email so I would not expect double the bandwidth until the user actual downloads the email (but it still wouldn't be double since the bounceProcessor strips the attachment - which I have confirm it is doing). I still believe that if james is not stopping an incoming message (receiver side) when a message has exceeded it's max message size, that a denial of service attack it possible. This type of attack would more consume bandwidth than cpu, but bandwidth is the expensive resource in this equation. Maybe I'll take a look at the smtp receiving code (pointer on which class to look in). It
Re: Why do we need fast-fail?
No, a) if you specify required the internal servers will also need to execute STARTTLS b) if you specify enabled the external servers can relay without doing STARTTLS No, If you specify enabled then a server is able to run STARTTLS. Relay check can only be done after the RCPT TO and not after the STARTTLS so you only need to know where STARTTLS has been sent or not but you need to do that in the RCPT TO call. This is not what I want. Internal servers can always relay and external servers must use STARTTLS. Of course if the authorizedAddresses, has priority over this setting, we are home free. This is possible if you know the current STARTTLS/AUTH status when you receive the RCPT TO. The check to see wether the relay is supported or not because of AUTH or STARTTLS will be done after the first RCPT so the extension point is the RCPT and not the STARTTLS. Agreed, we just need to capture the state information (preferably in a stateobject as opposed to the current Hashmap) --Søren So, by now, we don't need STARTTLS as an extension point. We should support STARTTLS and the enabled|required|disabled values, then we need to have the SMTP status object updated when we handle the RCPT TO / MAIL FROM commands. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Why do we need fast-fail?
No, a) if you specify required the internal servers will also need to execute STARTTLS b) if you specify enabled the external servers can relay without doing STARTTLS No, If you specify enabled then a server is able to run STARTTLS. Relay check can only be done after the RCPT TO and not after the STARTTLS so you only need to know where STARTTLS has been sent or not but you need to do that in the RCPT TO call. This is not what I want. Internal servers can always relay and external servers must use STARTTLS. Of course if the authorizedAddresses, has priority over this setting, we are home free. This is possible if you know the current STARTTLS/AUTH status when you receive the RCPT TO. The check to see wether the relay is supported or not because of AUTH or STARTTLS will be done after the first RCPT so the extension point is the RCPT and not the STARTTLS. Agreed, we just need to capture the state information (preferably in a stateobject as opposed to the current Hashmap) --Søren So, by now, we don't need STARTTLS as an extension point. We should support STARTTLS and the enabled|required|disabled values, then we need to have the SMTP status object updated when we handle the RCPT TO / MAIL FROM commands. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Serious bandwidth begin consumed by James
If i am right, it should: Notice to size limit has been reached, and stop buffering the message. Continue reading the message until crlf.crlf is reached Send 552 error! Reset state, and continue to accept other messages. IMHO when the limit is reached, we should either (soft fail) listen to everything and ignore the data as you describe above, or (hard fail) kill the connection. Right now what we does not make a lot of sense, since we haven't reduced bandwidth. I think that the correct behaviour is to ignore the rest of the message. If we kill the connection then the server will retry to send the message later (for most MDA probably at least 10 times in 5 days) and so they will use much more bandwith. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Serious bandwidth begin consumed by James
On 5/11/05, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: IMHO when the limit is reached, we should either (soft fail) listen to everything and ignore the data as you describe above, or (hard fail) kill the connection. Right now what we does not make a lot of sense, since we haven't reduced bandwidth. I think that the correct behaviour is to ignore the rest of the message. If we kill the connection then the server will retry to send the message later (for most MDA probably at least 10 times in 5 days) and so they will use much more bandwith. Sorry, forgot to mention one point... that the size limit should be annouced in EHLO (if it's not there now, it should and I think a higher priority that the behavior here). Since this is dealing with a remote client that is ignoring or violating what they're told in EHLO, I could see a aysadmin wanting to go either way. -- Serge Knystautas Lokitech software . strategy . design http://www.lokitech.com p. 301.656.5501 e. [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Serious bandwidth begin consumed by James
On 5/11/05, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: IMHO when the limit is reached, we should either (soft fail) listen to everything and ignore the data as you describe above, or (hard fail) kill the connection. Right now what we does not make a lot of sense, since we haven't reduced bandwidth. I think the RFC mandates a soft-fail - ie. error response after data has finished. I think that the correct behaviour is to ignore the rest of the message. If we kill the connection then the server will retry to send the message later (for most MDA probably at least 10 times in 5 days) and so they will use much more bandwith. Yes, that will happen! Sorry, forgot to mention one point... that the size limit should be annouced in EHLO (if it's not there now, it should and I think a higher priority that the behavior here). Since this is dealing with a remote client that is ignoring or violating what they're told in EHLO, I could see a aysadmin wanting to go either way. It already announces this. However, there is no rule that clients must use the size command, or that the size they send must be correct, or that they even use EHLO! I think that is it important for james to be standards compliant. I'm not sure if the current code would produce any serious problems, but i suspect the lack of a resetState() call may effect the next email in the same session - ie the next MAIL command will get a response to say already specified sender, and the recipients of the first failed message will get a copy of the second message. Daniel. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [EMAIL PROTECTED]: Project james-server (in module james-server) failed
Buildfile: build.xml does not exist! Build failed Huh? According to http://brutus.apache.org/gump/public/james-server/james-server/gump_file/list_source_james-server.html, GUMP has the file. Anyone know GUMP well enough to know what needs fixing? I can do a clean checkout from SVN, add the necessary Sun JAR files, and all works fine. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 15755] - javax.mail.internet.ParseException
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
http://issues.apache.org/bugzilla/show_bug.cgi?id=15755.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=15755
[EMAIL PROTECTED] changed:
What|Removed |Added
Status|RESOLVED|REOPENED
Resolution|INVALID |
--- Additional Comments From [EMAIL PROTECTED] 2005-05-11 21:48 ---
I was able to replicate this bug in version 2.2.0a8
javax.mail.MessagingException: Exception spooling message: Exception caught
while storing Message Container: javax.mail.internet.ParseException;
nested exception is:
java.lang.RuntimeException: Exception caught while storing Message
Container: javax.mail.internet.ParseException
at org.apache.james.James.sendMail(James.java:507)
at org.apache.james.James.sendMail(James.java:464)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at
org.apache.avalon.phoenix.components.application.BlockInvocationHandler.invoke(BlockInvocationHandler.java:92)
at $Proxy9.sendMail(Unknown Source)
at
com.privacyinc.logic.runtime.mppmodule.james.MailAllForward.forwardEmail(MailAllForward.java:755)
at
com.privacyinc.logic.runtime.mppmodule.james.MailAllForward.service(MailAllForward.java:496)
at
org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:439)
at
org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:441)
at
org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:370)
at
org.apache.james.util.thread.ExecutableRunnable.execute(ExecutableRunnable.java:89)
at org.apache.james.util.thread.WorkerThread.run(WorkerThread.java:125)
And this time we have the mail that created this exception. I had written a Perl
script to send out a test emails to James. This script had an error and I assume
that this error is responsible for the bug.
Here is the Perl script
sub mailer{
my $sendmail = /usr/sbin/sendmail -t;
my $reply_to = Reply-to: [EMAIL PROTECTED];
my $from = From: [EMAIL PROTECTED];
my $subject = Subject: Hi\n;
my $content = Thanks for your efforts\n;
my $to = To: [EMAIL PROTECTED];
open(SENDMAIL, |$sendmail) or die Cannot open $sendmail: $!;
print SENDMAIL $from;
print SENDMAIL $to;
print SENDMAIL $reply_to;
print SENDMAIL $subject;
print SENDMAIL 'Content-type: text/plain\n\n';
print SENDMAIL $content;
close(SENDMAIL);
}
As we can see in line print SENDMAIL 'Content-type: text/plain\n\n';
I had mistakenly added single quotes instead of double quotes. Once I placed
double quotes, the mail went through James without any issues.
Thanks,
Sudip
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the assignee for the bug, or are watching the assignee.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Fast-Fail first proposal
-- Here is the updated list of useful fast-fail scenarios. In the bottom you will find my first proposal. -- A. SMTP connection: james currently always reply 220 server ready, we could fast-fail based on: A1. remote address A2. current james resources/status. B. SMTP MAIL FROM:: james currently only checks address syntax, we could fast-fail based on: B1. banned sender addresses B2. previous smtp status (AUTH/STARTTLS). C. SMTP RCPT TO:: james currently checks local/remote domains to decide wether to relay or not, but we could add: C1. provide user unknown for unavailable local addresses C2. provide mailbox full/disabled and similar task C2. previous smtp status (AUTH/STARTTLS). D. SMTP DATA: james currently reply 250 Message received for each message under the maximum size handled: D1. add antivirus/antispam and other content filters and provide dedicated failure feedback. -- Here is the specific discussion for each scenario: --- A. This could be a common handling for all TCP/IP protocol services: we should add a way to attach plugins to the servers so that they can dinamically switch their availability. out-of-resources, anti-flood/dos, rbl filters, and other IP/connection based filters. A simple boolean accept(InetAddress remote-ip) or better a method that is able to return a string message. We could also add an identical callback for accepting the socket connection so we are able to refuse via socket or via SMTP protocol. The second one allow us to tell something to the client, and a more advanced usage would be to add a wait() in the accept() implementation when system resources are low (many servers accept the connection immediatly but only reply 220 server ready where they are really ready). This filters will be applicable to POP3, IMAP, NNTP, SMTP and any other similar protocol and are indipendent from the specific protocol. B+C. Here perhaps we can find a common solution for the macroscenario: I'll name it route filters as it will filter based on source email address or/and destination email address/addresses. What we need here is: - the specified address: from or to email address (probably not RFC822 address to support sourceroute - the command parameters: probably as a single String (mail from could have an ENVID parameter, rcpt to could have NOTIFY and ORCPT parameters) - the smtp conversation status: remote IP, remote PORT, time elapsed, previous AUTH, TLS. Probably the best thing is to put everything as a Properties as smtp session information - the newly created Mail: I think that the MAIL FROM should create the empty email and start filling-in data, so that at every RCPT TO call we will know not only the current recipient but also all the previosly used recipients (we can limit the number of recipient per mail doing so) Another possibility could be to have the MailImpl be created from the call to this handler like a factory instead of providing the empty MailImpl from the smtphandler. probably the code that we will run in this handler will know wether a recipient will be local or remote and will know the final name of a recipient (after the virtusertable) before being able to reply, so I think we should find a way to reuse this information when we will be in the spooler: probably we can directly fill the MailImpl with the expanded/aliased recipients and remove the corresponding mailets from the spooling (I see problems here, but I'll tell them later ;-) ). D. I think that SMTP DATA fastfail should be (IMO) handled in a fully different way: we need something more integrated with the spoolmanager and the DSNhandling. I think we should add this support after a good refactoring of the delivery status tracking (I'm working on this, I will post my full proposal soon). -- Here is my current goal: -- It would be great if we could agree on this general scenarios approach and agree on the proposed solutions for A and D so we can start an in depth analysis of the B and C handling. If we agree on a partial solution I will take care to add a better formatted thing to the wiki as a reference. I think I will not commit to the fast-fail stuff soon because I have higher priorities: delivery status tracking, spooler optimization, remote delivery optimization. So I think I only will write code for the D point and only after my coming-soon delivery status tracking proposal will be accepted and committed. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
