Re: [jira] Commented: (JAMES-282) Mail not correctly deliverd after DNS time-out on From: address

[Richard O. Hammer]

[EMAIL PROTECTED] wrote:
OK, this partial message problem is reproducible and comes from the fact that when the socket prematurely closes (as opposed to times out, which does generate an exception), the associated stream just returns -1 when asked for more data.  This gets passed back through the chain until it reaches James, which couldn't tell the difference between the pre-mature end of stream and the legitimate end of the "Character Terminated" stream.
Thanks, Noel, for your clarifying explanation of this part of the problem.
Rich
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: "New Standard Could Reduce Spam"

[Richard O. Hammer]

It seems to me that spam is a big problem that many of us (or perhaps 
most of us) have ideas about.  I guess that most of the work now being 
done on mail-handling programs (including James) is motivated by a 
desire to limit spam.

In this vein, I have spent most of the last four years working toward 
a kind of solution.  But after four years the best I can say is that 
I've learned a lot.  Also I have a preliminary offering at 
Mailscreen.net, but I grant this is so blunt and unrefined that it 
will please almost no one.

My work on spam mixes in a heavy dose of my philosophy, which is 
libertarian.  I mistrust government and hope to see solutions 
developed by private free market entrepreneurs.

On my to-do list, after I get caught up in the J2EE course which I'm 
slogging through , I want 
to put up a blog focused on a search for market-oriented network 
protocols.  I posit that basic protocols at the IP level would contain 
better provisions for assuring the mutual gain of participants if 
those protocols had evolved in a marketplace between business traders, 
rather than having been developed on contract for a monolithic power 
(the US Department of Defense).

Rich Hammer
my present project: 
resume: 
a previous project now in hibernation: 
Vincenzo Gianferrari Pini wrote:
I found this article today:
http://www.wired.com/news/print/0,1294,63513,00.html
This is something that I already developed last fall at the sender MTA side 
("AddServerSignature" mailet), but stopped because I was counting on having a MUA 
check at the recipient side, and Outlook Express was not behaving correctly.
The threads were "From email address validation" and "[PROPOSAL] Release Plan" in this 
list.
This thing could become important in my country (Italy) also because the Italian 
Government is setting new rules regarding MTAs sending back signed receipt 
confirmations, in order to have email messages become legally valid etc. I'm going to 
dig a little around this.
Perhaps other countries are doing something similar now ...
Any thoughts?

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re:(JAMES-282) Mail not correctly deliverd after DNS time-out on From: address

[Richard O. Hammer]

[EMAIL PROTECTED] wrote:
The following comment has been added to this issue:
 Author: Hes Siemelink
Created: Wed, 19 May 2004 6:46 AM
   Body:
I'm afraiud the problem is not potential but quite real and reproducible. 

We have nailed down the problem to the following:
 - Open a conenection to James
 - Give the HELO, MAIL FROM, RCPT TO and DATA commands
 - Close the connection after the DATA command. (On Windows, the most effective way to 
do this is by closing the telnet window)
James will deliver a stub message. 
Hes,
After reviewing RFC 2821 sections 3.3 and 4.1.1.4, I am not convinced 
this behavior is a bug.  Those RFC sections do not seem to say what 
the SMTP receiver should do in this case.  So I would think we are 
free to choose the present behavior, unless the present behavior 
violates a standard.

Rich Hammer
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: keeping a JDBC connection active

[Richard O. Hammer]

Thank you for the information, Danny.

Rich

Danny Angus wrote:
If you don't use transactions and transaction locking you'll most likely
never see a data issue.
If you do you may find ... 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: keeping a JDBC connection active

[Richard O. Hammer]

Serge Knystautas wrote:
... permanently holding a single 
database connection open is a Bad Idea(tm).  Some downsides:
- your code is inherently single-threaded.
I have not noticed any single-thread limitation.  I have a singleton 
(only one instance) database class.  It has the one Connection as an 
instance variable which is set when the class is constructed at server 
startup.  Multiple threads call the methods in this database class.

Each call to one of the database methods prepares its own 
PreparedStatement from the one Connection, and closes the 
PreparedStatement before returning.  (I would have avoided that 
pattern of preparing a PreparedStatement on each method call, since it 
seems wasteful in my limited knowledge.  But if I recall correctly I 
got the pattern from James, where I gather it works well enough, and I 
have not had reason to change it in my code.)

Anyhow, I have the impression that the Connection keeps track of its 
many individual PreparedStatements, like a mama knows her children 
apart, and never mixes up their interactions.  Do you think it might? 
 I've never noticed any jumbling of data.

I think I've got the necessary synchronization, pertaining to the 
reading and setting of database data, handled higher up in my code. 
But maybe there is some database programming issue here which I need 
to learn.

- if you restart your db server, you get a failure in your app.
Yes!  This one has hit me over the head.  You've helped me see which 
direction I need to go to fix it.

...  Database connection pools are pretty 
prevalent, even most JDBC3 drivers have something bundled.  I would 
really encourage you to look into using one.
Thank you, Serge.  I've got work to do.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

keeping a JDBC connection active

[Richard O. Hammer]

In my mailserver (a cousin of James) I use one java.sql.Connection 
throughout the life of the server.  I call 
java.sql.DriverManager.getConnection() once at server startup.  I've 
never suspected there was anything wrong with this, but I guess there 
might be.

If this is of interest to anybody, several times I have gotten 
"java.net.SocketException Connection reset by peer", indicating a dead 
connection to the database and requiring me to restart the server. 
This happens several hours after server startup, when I am running the 
mailserver in a Windows 2000 desktop and the database (PostgreSQL) in 
a RH7.3 Linux box on the LAN.  I have found that I can work around 
this by running both mailserver and database on the same Linux box; 
then the Connection seems to last forever.

Rich Hammer

Noel J. Bergman wrote:
Do you have any reason to believe that there are legitimate
reasons for a JDBC connection to be active for several
minutes, much less an hour?
I ran grep against the logs in my production environment.  There have been
32 timeouts since August 2002.  Getting them tends to indicate a need to
repair tables, at least in our environment.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [jira] Commented: (JAMES-248) memory management - finalize/nullify after successful delivery of retrieved MailImpl

[Richard O. Hammer]

Noel J. Bergman wrote:
Serge posted an article today that discusses this issue in some detail.  See
"Explicit Nulling" in
http://www-106.ibm.com/developerworks/java/library/j-jtp01274.html.


Great article!  Thank you, Noel.



Java performance has few absolutes.  For example, the author of that article
is not a fan of user-level object pooling.  Neither, for that matter, is the
J2SE tech lead.  However, when we added object pooling of certain objects in
James, we found that performance improved significantly under load.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [jira] Commented: (JAMES-248) memory management - finalize/nullify after successful delivery of retrieved MailImpl

[Richard O. Hammer]

The original issue  says:

"Isn't this kind of a memory leak? After all, attachments to such a 
MIMEMessage can be 13MB large.
Shouldn't those objects be null-ed latest after the POP3Handler.doQUIT 
of a session?"

Commenting upon this, Noel wisely asks "Do you have any reason to 
believe that there is currently a problem in James?"

I guess that this IS NOT a kind of memory leak.  The people who 
specified the JVM seem to believe that they have the JVM's memory 
managment under control, and they seem to want us Java developers to 
leave that problem to them.  Generally, I trust the JVM to do garbage 
collection using parameters which are known to the JVM.

Generally, all you need to do with a variable name when you are done 
with it is to forget it.  Just let it fall out of scope, while making 
sure of course that your scope is not larger than needed.

When I see variables set to null after their use, I think that this 
programmer does not know Java.

Rich Hammer

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: MxSorter and changes to DNSServer.java

[Richard O. Hammer]

Noel J. Bergman wrote:
The more important change in DNSServer was reverting to the previous
technique of using InetAddress.getAllByName to get the IP addresses for the
SMTP hosts, rather than the Type.A lookup.
I haven't looked at this code recently, but I guess this change might 
end the reliance upon the org.xbill jar.  Do you know offhand if it does?

> ... The code was failing to resolve
hosts that use CNAME on the right hand side of an MX record.  That may be an
incorrect DNS configuration, but it is all too common as I have noticed over
the past couple weeks of testing the new code.
It is good that you caught this, Noel.  When I was working with this 
functionality back in December, I felt unsure of the proper ways to 
use DNS since I don't know of a good reference which tells how 
mailservers should use DNS.  Do you know of such a reference? or do we 
just push ahead learning from our experience?

Rich Hammer

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [jira] Commented: (JAMES-248) memory management - finalize/nullify after successful delivery of retrieved MailImpl

[Richard O. Hammer]

[EMAIL PROTECTED] wrote:
Java does not guarantee that finalizers run,
I believe this is correct.  There is no guarantee that garbage 
collection will run, or will run on a given object.  But I believe the 
following is mistaken

and it certainly does not guarantee that they will run before Java decides to garbage collect that object.
I believe there is a guarantee that the finalize() method will be 
called on an object before it is garbage collected (IF it is garbage 
collected).

Rich Hammer



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: mail list loop, reverse path <-> Return-Path header

[Richard O. Hammer]

I had some difficulty with the way James handles Return-Path headers, 
while developing my mail service which borrows much code from James 
but which also has significant architectural changes.

As Hontvari reports, the SMTPHandler in the version of James with 
which I started (about 18-24 months ago) added a return-path header to 
each message which came in.  At first I copied that.  Then I was 
incorrectly forwarding SMTP messages with the Return-path header still 
in place.  Now my code adds a Return-path header only when it places a 
message in a POP3 mailbox.

The James MailImpl class seems to keep the return path, since it has 
the method getSender().  With the return path thus accessible in 
James, addition of a Return-Path header might be moved to those places 
where the message is leaving the SMTP environment.

Hope this helps,

Rich Hammer
mailscreen.net
Hontvari Jozsef wrote:
(I am not sure what is the status of the merge, so I don't send a patch now,
and anyway, you may not agree on my evaluation)
org.apache.james.James class has a bounce method, which is intended to be
used for bouncing invalid emails to the reverse path (i.e. the email address
supplied in the SMTP MAIL FROM command). RemoteDelivery mailet does use this
method.
This method tries to determine the reverse path from the Return-Path header
(which is stored _within_ the email) instead of using the reverse path
stored along the email in the Mail class. According to the mail RFCs, the
Return-Path header is not indeded to be used by an smtp service, it should
only be used when final delivery occurs, e.g. when storing the mail in a
mailstore used by a pop service. I consider this as a hack, and really don't
like it in James, but because the James smtp handler always inserts the
Return-Path header it seems to be working.
However, the GenericListserv mailet removes the Return-Path header. I think
that is quote correct, on the other hand it breaks the above, not so correct
assumption about the Reverse-Path header. The bounce method in these cases
defaults to the Reply-To, i.e the mail list itself, so you get a mail loop
involving as many people as you have on the list... It happened to me
several times.
In my oppininon the correct solution would be to enirely remove the
dependency on the Return-Path header, but I don't have enough time and
understanding of James. So I only implemented a fallback specifically for
the case when the Return-Path header is not present, which eliminates this
type of loop in my case:
I replaced the
getLogger().warn("Mail to be bounced does not contain a
Return-Path header.");
line with this code:

MailAddress reversePath = mail.getSender();
if (reversePath == null) {
getLogger().warn("Mail to be bounced does not contain a
Return-Path header and reverse path is null too, sending to postmaster.");
reply.setRecipient(MimeMessage.RecipientType.TO,
getPostmaster().toInternetAddress());
} else {
getLogger().warn("Mail to be bounced does not contain a " +
"Return-Path header although expected (wrongly!), "
+
"reverse path used (it should be used always): " +
reversePath.toString());
reply.setRecipient(MimeMessage.RecipientType.TO,
reversePath.toInternetAddress());
}
(I copied this code directly from my source, maybe the log message isn't
suitable for a release)
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

JavaMail InternetHeaders could be more friendly to server developers

[Richard O. Hammer]

Three months ago in November we exchanged a few ideas about how the 
JavaMail API might be made more useful for development of mail servers 
-- as distinct from development of mail clients.  Now I have 
discovered one more point that I would like to add.  I might be 
mistaken again because of my incomplete understanding of the API, but 
I hope this is right.  My comment concerns 
javax.mail.internet.InternetHeaders.

When a SMTP server receives a message it needs to add a Received: 
header at the top of the headers.  But InternetHeaders does not seem 
to offer any way to add a header specifically at the top of the list. 
 This class seems to offer no way for its users to learn or set the 
order of the headers. (I am referencing the JavaDocs for JavaMail 1.3).

As a consequence the code in James* goes through an awkward 
workaround: constructing a new InternetHeaders object, adding the new 
Received: header, and then adding the remaining header lines by 
looping through an Enumeration of header lines retrieved from the 
original incoming InternetHeaders object.  This workaround relies upon 
behavior which I have not seen documented, namely that the 
addHeaderLine() method always adds to the bottom of the list of headers.

So, I would suggest that InternetHeaders could be improved, for the 
use of server developers, by addition of a method 
addHeaderLineAtTop(), and by promising and documenting the present 
behavior of addHeaderLine(), that it adds to the bottom.

Rich Hammer
Hillsborough, N.C.
*The code in James to which I refer is in:
org.apache.james.smtpserver.SMTPHandler.processMailHeaders(InternetHeaders)
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: when to dot-stuff, dot-strip

[Richard O. Hammer]

Danny Angus wrote:
-- BUT I
NEVER USE IT IN THE DOT-STRIPPED CONDITION.
Huh? internally do you mean?
I never use the bodies of the messages which have come in except to 
send them out again.  They sit in a file until they are shipped out 
again, and later the file is deleted.

The choice is to develop some kind of logic to determine and mark the
stuffing status of messages, or strip and stuff everything.
I think the latter is safer 
I agree, given that a system is already established and working.  But 
for future development of MTAs there might be some simpler MTAs which 
could entirely skip the coding of dot-stuffing and dot-stripping.  For 
instance, if all you are running is an open relay :-?, then you can 
entirely skip this dot business.

Thank you for your response, Danny.  I'll mention this on the 
<[EMAIL PROTECTED]> list, since it seems like an issue which may 
relate to the specification.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

when to dot-stuff, dot-strip

[Richard O. Hammer]

The code of James, as I have come to know it, follows the directions 
about dot stuffing and dot stripping, in RFC 2821 section 4.5.2 and 
RFC 1939 section 3 paragraph 4.  I believe it is correct.

But, perhaps because my mind froze up decades ago when computer cycles 
mattered, I notice that everything going through my server (an 
offshoot of James) gets dot-stripped when it comes in through SMTP, 
and dot-stuffed again when it goes out through SMTP or POP3 -- BUT I 
NEVER USE IT IN THE DOT-STRIPPED CONDITION.  Why don't I just skip the 
stuffing and stripping entirely?

Now I will need to keep dot-stuffing functionality around, at least 
for use upon my own locally generated messages.  But it looks to me 
like simplification and optimization are possible here, by confining 
dot-stuffing and dot-stripping to those cases where it is ultimately 
necessary.

My questions: Do you agree with this idea?  Have you seen it discussed 
before?

Thanks,
Rich Hammer
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Determine Message Size in Mailet

[Richard O. Hammer]

Oh, and let me add that documentation (or perhaps I should say hints) 
about that memory-increasing command line option can be found in 
places like this 
.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Determine Message Size in Mailet

[Richard O. Hammer]

Kraig Schario wrote:
I am having a problem with a mailet when copying the Mail mail object to a
MimeMessage.
MimeMessage message = mail.getMessage();

It only occurs when the message contains large attachments. Usually larger
than 2 MB.  I receive the following error in the Mailet.log
java.lang.OutOfMemoryError


I think one way to deal with this is to give the JVM more memory to 
work with than it gets by default.  You can do this with a command 
line option to the "java" command, an option like "-Xmx96m".

I had this working once, after experimenting some, and was thereby 
able to process bigger messages through JavaMail.  Unfortunately, now 
I can't find specifically what option I used.  You may succeed with 
trial and error.

As I recall, the JVM gets 64 megs by default in my situation.

Rich Hammer

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: escaping of SQL strings

[Richard O. Hammer]

Thank you, Serge.  PreparedStatement.setString() does the escaping 
trick, all right.  I've confirmed this with testing.  The API 
documentation didn't give me a clue.

Rich

Serge Knystautas wrote:
Richard O. Hammer wrote:

I stumbled into this question when, using my James-offshoot server, I 
sent a test message with a possessive (single quote) in the subject:
Subject: Friday's test
and it failed with
java.sql.SQLException: ERROR:  parser: parse error at or near "s"


You shouldn't do escaping yourself in Java.  Do 
PreparedStatement.setString(paramNum, stringValue)





-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

escaping of SQL strings

[Richard O. Hammer]

When James stores message headers or other text in a database, I 
assume that it escapes the single-quote characters in those strings 
(and perhaps a few more characters as needed).  But my look at the 
code this morning does not find that functionality, so I wonder if it 
is happening somewhere I am not looking.

I am looking at these calls:
mailrepository.JDBCMailRepository.store(Mail), which
calls MimeMessageWrapper.writeTo(various)
and I don't see any escaping going on there.

Unfortunately I do not have a running copy of James with which to test 
this myself.

I stumbled into this question when, using my James-offshoot server, I 
sent a test message with a possessive (single quote) in the subject:
Subject: Friday's test
and it failed with
java.sql.SQLException: ERROR:  parser: parse error at or near "s"

So I have to be escaping my headers before I feed them into SQL, and I 
went looking in James for a good idea of a way to do it.  But I have 
not found it yet.

Assuming that James does this escaping, what method does it use?

Thank you,
Rich
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Regarding BUG 24885: RemoteDelivery only tries one of multiple A record

[Richard O. Hammer]

Below you may find code which I have developed and tested to deal with 
this issue.

This solution has a new little type to wrap the data needed in 
RemoteDelivery.  It carries a String hostName and an array of InetAddress.

import java.net.InetAddress;
import org.xbill.DNS.ARecord;
import org.xbill.DNS.Record;
/** A holder of data generated in the DNSServer and needed in 
RemoteDelivery.  Especially needed for multihomed hosts.
 */
public class SMTPHostAddresses {
  String hostName;
  InetAddress[] ipAddresses;

  SMTPHostAddresses(Record[] aRecords, String hostName){
this.hostName = hostName;
if (aRecords == null){
  ipAddresses = new InetAddress[0];
}else{
  ipAddresses = new InetAddress[aRecords.length];
  for (int i = 0; i < ipAddresses.length; i++){
ipAddresses[i] = ((ARecord)aRecords[i]).getAddress();
  }
}
  }
  public String getHostname(){
return hostName;
  }
  public InetAddress[] getAddresses(){
return ipAddresses;
  }
}
The following new method, added to DNSServer, provides the interface 
to the underlying DNS functionality.  This will be used instead of 
DNSServer.findMXRecords(hostname).

  /** Performs DNS lookups as needed to find servers which should or 
might
   * support SMTP.  Returns one SMTPHostAddresses for each such host 
discovered
   * by DNS.  If no host is found for domainName, the Enumeration
   * returned will be empty and the first call to hasMoreElements() 
will return
   * false.
   * @param domainName the String domain for which SMTP host 
addresses are
   * sought.
   * @return an Enumeration in which the Objects returned by 
nextElement()
   * are instances of SMTPHostAddresses.
   */
  public Enumeration getSMTPHostAddresses(final String domainName){
return new Enumeration(){
  private Enumeration mxHosts = new MxSorter(domainName);
  public boolean hasMoreElements(){
return mxHosts.hasMoreElements();
  }
  public Object nextElement(){
String nextHostname = (String)mxHosts.nextElement();
Record[] aRecords = lookup(nextHostname, Type.A);
return new SMTPHostAddresses(aRecords, nextHostname);
  }
};
  }

You will notice that this method simply wraps another Enumeration, 
which I will supply below.

But first let me touch upon how this would be used in RemoteDelivery, 
if that is not obvious.  The code immediately below is an untested 
sketch of the structure which I suppose could work in 
RemoteDelivery.deliver().  It contains an outer loop going through the 
Enumeration of SMTPHostAddresses, and an inner loop going through the 
array of InetAddresses in each SMTPHostAddresses.

Enumeration smtpHosts = /* something like */
getMailetContext().getSMTPHostAddresses(host);
if(!smtpHosts.hasMoreElements()){
  //error condition as in RemoteDelivery
  log("No mail server found for: " + host);
  StringBuffer exceptionBuffer =
  new StringBuffer(128)
  .append("There are no DNS entries for the hostname ")
  .append(host)
  .append(".  I cannot determine where to send this message.");
  return failMessage(
mail, new MessagingException(exceptionBuffer.toString()), 
false);
}
while(smtpHosts.hasMoreElements()){
  SMTPHostAddresses thisHost = 
(SMTPHostAddresses)smtpHosts.nextElement();
  String thisHostName = thisHost.getHostname();//if needed for 
logging
  InetAddress[] addresses = thisHost.getAddresses();
  for (int addressIndex = 0; addressIndex < addresses.length;
 addressIndex++){
InetAddress thisAddress = addresses[addressIndex];
//attempt SMTP connection at thiAddress
//do other work
  }
}

I agree with Noel that in some ways a single Enumeration, 
necessitating only a single loop, would be simpler.  But I've already 
got this written, and there are also some ways in which this structure 
seems better.

Now for the biggest block of my code, the inner class MxSorter below 
also goes into DNSServer.  This is the Enumeration that was wrapped 
above in the method DNSServer.getSMTPHostAddresses().

  /** A way to get mail hosts to try.  If any MX hosts are found for the
   * domain name with which this is constructed, then these MX hostnames
   * are returned in priority sorted order, lowest priority numbers 
coming
   * first.  And, whenever multiple hosts have the same priority then 
these
   * are returned in a randomized order within that priority group, as
   * specified in RFC 2821, Section 5.
   *
   * If no MX hosts are found for the domain name, then a DNS search is
   * performed for an A record.  If an A record is found then 
domainName itself
   * will be returned by the Enumeration, and it will be the only 
object in
   * the Enumeration.  If however no A record is found (in addition 
to no MX
   * record) then the Enumeration constructed will be empty; the 
first call to
   * its hasMoreElements() will r

Re: Regarding BUG 24885: RemoteDelivery only tries one of multiple A record

[Richard O. Hammer]

Søren Hilmer wrote:
> ... I am adding a
> DNSServer.findARecords method to do it (basically a copy/paste of
> findMXRecords).
I find this very interesting, and I am working on code to deal with 
it.  But I don't want to step on Søren's toes.

Within a day or two, I hope to post some code here which works -- as 
far as I have been smart enough to test it.  Then I hope to benefit 
from the list's critical review of that code.  I don't know enough 
about DNS to feel sure of myself here.

I want to confirm my understanding of "preferences".  As Noel cited, 
RFC 2821, section 5, says:

   The destination host (perhaps taken from the preferred MX record) may
   be multihomed, in which case the domain name resolver will return a
   list of alternative IP addresses.  It is the responsibility of the
   domain name resolver interface to have ordered this list by
   decreasing preference if necessary, and SMTP MUST try them in the
   order presented.
I believe that these preferences pertain to the host targets of the MX 
records.  The hosts are ranked.

But, for any given host, if that host is multihomed, then the IP 
addresses for that host are not ranked.  They can be returned in any 
order within the list of IP addresses for that host.  Does that sound 
right?

Rich Hammer

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Regarding BUG 24885: RemoteDelivery only tries one of multiple A record

[Richard O. Hammer]

Noel J. Bergman wrote:
Søren Hilmer wrote:
...
So it is handling a multihomed destination that needs to be addressed.
I want to make sure I understand this issue.  I've put some debugging 
statements into org.apache.james.dnsserver.DNSServer, which print out 
Record.toString() in places of interest.  Today, when I call 
findMXRecords("yahoo.com"), the subsequent lookup("yahoo.com", 
Type.MX); returns the following three records, shown here before they 
are sorted for priority order.

   yahoo.com.  2320 IN  MX  1 mx2.mail.yahoo.com.
   yahoo.com.  2320 IN  MX  5 mx4.mail.yahoo.com.
   yahoo.com.  2320 IN  MX  1 mx1.mail.yahoo.com.
The problem being discussed here is that host mx2.mail.yahoo.com. may 
exist at several IP addresses (i.e., that a subsequent lookup of 
mx2.mail.yahoo.com. may produce several A records.)(?)  So that if, 
after sorting for priorities, we decide to connect to 
mx2.mail.yahoo.com. , then we need to try each of the IP addresses for 
mx2.mail.yahoo.com. before we go on to try the next host-name in our 
priority ordering (which would be mx1.mail.yahoo.com.)  Is that the 
problem?

Do we know for sure that the JavaMail code underlying the call:
  transport = session.getTransport(urlname);
(in method RemoteDelivery.deliver(Mail, Session)) does not do this for us?
Do you have an example, Søren, of a MX host which you know has 
multiple A records, so that I can test with it?

The return is a Collection of String objects.  Each one is currently of the
form "host", but if we were to handle multi-homed hosts by using "host/IP",
it seems to me that we could either parse it directly  ...
I guess this may require an additional call to the underlying 
DNSServer.rawDNSLookup(), because I don't think 
DNSServer.findMXRecords() ever has the redundant A records.  This is 
interesting -- but with my ignorance of DNS I am just guessing.

Rich



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Sieve Proposal (Request For Comments)

[Richard O. Hammer]

Noel J. Bergman wrote:
... please be aware that sometime next year, we will be
switching from CVS to Subversion, and probably all ASF code will reside in a
single SVN repository.
Gak!  Just when I bought a $25 book on CVS.



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Let's just tell Sun to rewrite the JavaMail API. Was: What about IMAP support?

[Richard O. Hammer]

> Richard O. Hammer wrote:
Another class which does more than I want is Transport. 
... it sets message headers
Bill Shannon wrote:
I'm not sure what you're referring to.
You are right Bill.  I wrote based upon my poor memory of experiments 
I did a year ago or more.  I had noticed that headers were being set 
in a way which seemed out of my control, and I guessed incorrectly it 
was being done in Transport.

But the frustrations which mail-server developers face in using 
JavaMail don't worry me quite so much anymore.  In my project I have 
gotten around some of those frustrations by writing some simple 
classes for my basic needs.  Note that my project is not James, but an 
offshoot of James.

Thank you,
Rich Hammer
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Let's just tell Sun to rewrite the JavaMail API. Was: What about IMAP support?

[Richard O. Hammer]

I want to add a few observations to what Danny has already said in 
this thread.  I mention two classes, Message and Transport, which I 
think could be improved by being super-classed, by giving each of them 
a simpler parent.  But this is based upon my limited experience with 
using JavaMail's API for developing a mail server.  So my list is 
probably not complete.  This may be just a sample.

First, obviously, in a mail server I need an object to represent a 
message.  Since it might require a lot of work for me to develop the 
code myself, I look hopefully in JavaMail.  There I find Message and 
its descendant MimeMessage.  These have many great methods which I can 
use, already written.  But Message has more than I need in some ways 
too.  It has a Session and a Folder, objects which I don't think I 
need in a server, and it is difficult for me to construct a Message 
without supplying a Session or Folder.

Now I guess it is true, as was suggested earlier, that I could get the 
functionality I want by subclassing Message (or MimeMessage) and 
supplying empty stubs for any objects (such as Session or Folder) 
which it wants but that I don't need.  But this seems backwards given 
my understanding of object-oriented design.  It seems to me that 
Message ought to have an ancestor, perhaps named GenericMessage, 
useful in both clients and servers, which would contain the wonderful 
methods needed in both clients and servers, but would not contain a 
Session or a Folder.  The Session and Folder should be added in a 
subclass for clients.

Another class which does more than I want is Transport. 
Transport.sendMessage() sure is a handy method.  It takes many 
problems off the shoulders of developers.  It is used in James, even 
though I understand it was intended only for clients.  But it gives a 
problem to server developers as it sets message headers in ways 
appropriate for clients -- but not appropriate for servers.  Here 
again it seems to me that Transport could descend from a class which 
did less.  If there were a GenericTransport class which did not set 
message headers, that would be a boon to server developers.

Rich Hammer
Hillsborough, N.C.
mailscreen.net
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: LocalDelivery and Delivered-To

[Richard O. Hammer]

Is the addition of a Delivered-To header called for in an RFC?  If so, 
where?  I'm having trouble finding it.

From the a comment in the source of the LocalDelivery mailet, I 
surmise that this practice originated with Qmail.  Is that correct?

By the way, I want to thank the James team for assembling the folder 
of relevant RFCs and including them with the distribution.  I've got a 
link to that on my desktop.  Very useful.

Rich Hammer

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problems with outgoing email

[Richard O. Hammer]

Two tools which I find useful for troubleshooting such difficulties are:
1. telnet to port 25 of the server in question (one of the Yahoo 
servers which DNS is finding for you) and talk SMTP to it.  That is, 
type in SMTP commands and observe the responses.  You can send test 
messages to yourself in this low-level way.  For an example of a SMTP 
exchange, see RFC 2821, Appendix D.

2. Ethereal to capture the packets of a given exchange.  It is 
possible that some error message is being generated, in the exchange 
between javax.mail.Transport and the yahoo mail server, which is not 
being reported up to you.  By scrutinizing the SMTP exchange, which 
you can capture with Ethereal, you might find an error message.

Rich Hammer

Serge Knystautas wrote:
Chandru wrote:
Hi,
Took me a bit of time to figure out that I need to point James to the
apropriate dns server (default is localhost).  The nameservers are set
correctly (I am using yahoo's namesservers currently).
The emails seem to be going out from James:



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Let's just tell Sun to rewrite the JavaMail API. Was: What about IMAP support?

[Richard O. Hammer]

Thank you, Bill.

Even though I raised this point, I doubt that I'm the best person from 
the James Developers list to respond to your saying, "If you think 
some great advantage could be had by splitting the APIs in this way, 
tell me more."

I guess that Danny, Noel, Serge, or others, might be able to quickly 
throw together a list of ways that life could be made easier for us if 
more of the basic functionality inside JavaMail were made accessible 
without the requirements for clients.  Assuming that I have actually 
represented the needs of James developers, I hope that others from 
this list might respond.

I don't know if a list of our needs might rise to the level of a 
"great advantage" from Sun's viewpoint.  You are probably right that 
there are many fewer server-side projects than client-side projects.

Rich Hammer

Bill Shannon wrote:
Richard O. Hammer wrote:

Danny Angus wrote:
 > ... JavaMail is explicitly and
 > in its detail a mail client API, it lacks support in may areas for 
the kind
 > of server functionality James provides and it is not always easy, 
and often
 > impossible, to force James to use JavaMail interfaces and classes 
for many
 > of the tasks of a server. ...

Reflecting upon what Danny wrote, I wonder if Sun ever considered the 
possibility of dividing the existing functionality in the JavaMail 
API.  It would be great for us if this existing functionality were 
divided into a basic set, tools useful to both servers and clients, 
and an extended set, just for clients.  Then we developers of servers 
could employ the basic set without having to disentangle ourselves 
from the features intended for clients.

JavaMail includes a lot of important, basic work, promising to ease 
the burdens of other developers.  Because of this, we developers of 
servers find ourselves sucked in, trying to use the good stuff that is 
inside JavaMail for our purposes.  But JavaMail would be much more 
valuable to us if the basic tools were exposed free of the 
stipulations which have been added for clients.


We did some of this.  We separated the pieces of JavaMail into the
core API piece and the protocol provider pieces.  That allows you
to pick just the pieces you need.
But we haven't considered carving the core API piece into multiple
pieces.  And we wouldn't consider doing anything of that sort that
would break compatibility with the existing API.  Requiring servers
to carry around the additional "client" parts of the core API doesn't
seem like a great burden.  I'd be more concerned about adding significant
new server support that clients were required to carry around.
If you think some great advantage could be had by splitting the APIs in
this way, tell me more.
I think the more interesting question is how much can we morph the APIs
into something that's more useful for servers without compromising ease
of use for clients?  I expect there to be many thousands of applications
using JavaMail for client mail access.  I only expect tens (at most) of
applications using JavaMail to implement mail server functions.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Let's just tell Sun to rewrite the JavaMail API. Was: What about IMAP support?

[Richard O. Hammer]

Danny Angus wrote:
> ... JavaMail is explicitly and
> in its detail a mail client API, it lacks support in may areas for 
the kind
> of server functionality James provides and it is not always easy, 
and often
> impossible, to force James to use JavaMail interfaces and classes 
for many
> of the tasks of a server. ...

Reflecting upon what Danny wrote, I wonder if Sun ever considered the 
possibility of dividing the existing functionality in the JavaMail 
API.  It would be great for us if this existing functionality were 
divided into a basic set, tools useful to both servers and clients, 
and an extended set, just for clients.  Then we developers of servers 
could employ the basic set without having to disentangle ourselves 
from the features intended for clients.

JavaMail includes a lot of important, basic work, promising to ease 
the burdens of other developers.  Because of this, we developers of 
servers find ourselves sucked in, trying to use the good stuff that is 
inside JavaMail for our purposes.  But JavaMail would be much more 
valuable to us if the basic tools were exposed free of the 
stipulations which have been added for clients.

Rich Hammer
Hillsborough, N.C.
mailscreen.net


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Bug in RemoteDelivery mailet?

[Richard O. Hammer]

Mike Morris wrote:
It seems that the RemoteDelivery mailet is trying to connect to the IP 
address that the DNS returns for the domain, and NOT (one of) the 
servers listed as the MX host.  I am pretty sure of this, based on 
watching netstat while James is making the connection.
Mike,

This reminds me (again -- I wrote the same thing on October 11, in 
response to a question) of something I've seen in the finally block in 
org.apache.james.dnsserver.DNSServer.findMXRecords(hostname).  You can 
see the code below.  If the DNS lookup fails to find MX records, then 
findMXRecords() may return simply the hostname from the email address.

So possibly your MX lookup is not working.  I would check first to 
confirm that you have the nameservers configured correctly.

Rich Hammer



   } finally {
//If we found no results, we'll add the original domain 
name if
//it's a valid DNS entry
if (servers.size () == 0) {
StringBuffer logBuffer =
new StringBuffer(128)
.append("Couldn't resolve MX records for 
domain ")
.append(hostname)
.append(".");
getLogger().info(logBuffer.toString());
try {
InetAddress.getByName(hostname);
servers.add(hostname);
} catch (UnknownHostException uhe) {
// The original domain name is not a valid host,
// so we can't add it to the server list.  In this
// case we return an empty list of servers
logBuffer = new StringBuffer(128)
.append("Couldn't resolve IP address 
for host ")
.append(hostname)
.append(".");
getLogger().error(logBuffer.toString());
}
}
}



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Bug 23906 - Large emails throw OutOfMemoryError

[Richard O. Hammer]

I wrote:
In my mailserver development I get around this by not constructing a
MimeMessage (can't see that I need one), writing the message body
right into a file.
Noel J. Bergman wrote:
... if you or anyone else has some patches,
please feel free to contribute them.
Noel,

Below you can see the little part of my present code that copies in a 
message.

But this is not a "patch", in that I have not considered how it might 
fit into James.  Many of my ideas come from James.  But there are many 
differences too.  For instance, I've decided that I can do what I need 
without ever constructing a MimeMessage out of a message which comes 
into my server.

Rich Hammer
Hillsborough, N.C.
mailscreen.net  -- a hope, a dream
public class Message {
  InternetHeaders headers;
  MailAddress envelopeFrom;
  ArrayList envelopeTo;
  File dataFile; //will hold message data but not headers
  String sentHereFromHost, sentHereFromIP, uniqueID;
  long receivedTime;
  long dataLength = -8,
   totalSize = -22;
  /* A constructor to be used by the SMTP receiver, after envelope 
information
   * has been received, headers have been parsed and updated, and the 
message
   * input stream is positioned to start reading
   * the message body.
   */
  public Message(String remoteHost, String remoteIP, MailAddress 
envelopeFrom,
  ArrayList envelopeTo, InternetHeaders headers, InputStream dataIn)
  throws IOException{
sentHereFromHost = remoteHost;
sentHereFromIP = remoteIP;
this.envelopeFrom = envelopeFrom;
sortByDomain(envelopeTo);
this.envelopeTo = envelopeTo;
this.headers = headers;
setText(dataIn);
  }

  protected void setText(InputStream dataIn) throws IOException{
uniqueID = getNewUniqueId();
dataFile = new File(messagesDirectory, uniqueID);
OutputStream out = new BufferedOutputStream(new 
FileOutputStream(dataFile));
byte[] block = new byte[fileTransferBlockSize];
int readCount;
while ((readCount = dataIn.read(block)) > -1) {
out.write(block, 0, readCount);
}
out.close();
dataLength = dataFile.length();
receivedTime = System.currentTimeMillis();
  }

...

}//class Message

The above code is copyrighted, by which I mean to copy it is right!



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problems with outgoing email

[Richard O. Hammer]

Chandru wrote:
I have a similar issue where the connection times out trying to contact
yahoo.com (exception below).  Shoudn't james be trying to contact the
mail server (mx?.mail.yahoo.com) instead?
This reminds me of something I've seen in the finally block in 
org.apache.james.dnsserver.DNSServer.findMXRecords(hostname).  You can 
see the code below.  If the DNS lookup fails to find MX records, then 
findMXRecords may return simply the hostname from the email address.

So possibly your MX lookup is not working.  I would check first to 
confirm that you have the nameservers configured correctly.

Rich Hammer



   } finally {
//If we found no results, we'll add the original domain 
name if
//it's a valid DNS entry
if (servers.size () == 0) {
StringBuffer logBuffer =
new StringBuffer(128)
.append("Couldn't resolve MX records for 
domain ")
.append(hostname)
.append(".");
getLogger().info(logBuffer.toString());
try {
InetAddress.getByName(hostname);
servers.add(hostname);
} catch (UnknownHostException uhe) {
// The original domain name is not a valid host,
// so we can't add it to the server list.  In this
// case we return an empty list of servers
logBuffer = new StringBuffer(128)
.append("Couldn't resolve IP address 
for host ")
.append(hostname)
.append(".");
getLogger().error(logBuffer.toString());
}
}
}



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: RemoteDelivery progressively increasing delay-time

[Richard O. Hammer]

Regarding the timing of subsequent tries to overcome network failures, 
I have seen the phrase "exponential backoff strategy" used.  I thought 
I had seen that in RFC 2821.  But if it were there I expect Noel would 
have reported it.

As I understand it, the delays in an exponential backoff strategy 
would be computed as in the following example.

If the goal is to have 5 retries which end after 5 days, then my 
calculator says the first delay should be about 13.4 seconds (5 
days)**(1/5).  (Where ** means "raised to the power of".)

13.4**1 =  13 ~= 13 seconds
13.4**2 = 180 ~= 3 minutes
13.4**3 =   2,406 ~= 40 minutes
13.4**4 =  32,242 ~= 9 hours
13.4**5 = 432,040 ~= 5 days
Rich Hammer
Hillsborough, N.C.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: building from james head

[Richard O. Hammer]

This is valid source as far as the Java language is concerned. 
InetNetwork is not an inner class, but rather is another class in the 
same package.  While it is customary to put each class in its own file 
this is not required by Java standards.

But, from scanning the Exception message which you attach, this 
practice seems to violate a design assumption of Avalon.

Although I have not tried this, I suspect you could make InetNetwork 
into an inner class simply by moving the closing brace of class 
NetMatcher down below the closing brace of InetNetwork.  That might 
fly, and might even have been intended, if no class but NetMatcher 
uses reference to InetNetwork.

Rich Hammer
Hillsborough, N.C.
Stephen McConnell wrote:
This one is interesting:

The class org.apache.james.util.NetMatcher includes references to the 
inner class InetNetwork.  However, if we look at the source we see that 
InetNetwork is not defined as an inner class (at least not in a way I am 
familiar with).

What we have is :



   public class NetMatcher
   {
// stuff
   }
   class InetNetwork
   {
// stuff
   }
   // more stuff



Ok - it compiles but is this really a valid source?

Stephen.

Internal error during kernel instantiation.
---
Exception: org.apache.avalon.merlin.kernel.KernelException
Message: Deployment failure.
Cause: org.apache.avalon.activation.appliance.DeploymentException
Message: Composite deployment failure in block: [block:/james]
Cause: org.apache.avalon.activation.lifecycle.LifecycleException
Message: Unable to create a new component instance in appliance 
[/james/spool] due to a component deployment failure.

Cause: org.apache.avalon.activation.lifecycle.LifecycleException
Message: Component initiated initialization failure.
Cause: java.lang.NoClassDefFoundError
Message: org/apache/james/util/InetNetwork
 stack trace --
java.lang.NoClassDefFoundError: org/apache/james/util/InetNetwork
   at 
org.apache.james.transport.matchers.AbstractNetworkMatcher.init(AbstractNetworkMatcher.java:95) 

   at org.apache.mailet.GenericMatcher.init(GenericMatcher.java:154)
   at 
org.apache.james.transport.MatchLoader.getMatcher(MatchLoader.java:108)
   at 
org.apache.james.transport.JamesSpoolManager.initialize(JamesSpoolManager.java:212) 

   at 
org.apache.avalon.activation.appliance.impl.DefaultAppliance.applyInitialization(DefaultAppliance.java:952) 

   at 
org.apache.avalon.activation.appliance.impl.DefaultAppliance.newInstance(DefaultAppliance.java:637) 

   at 
org.apache.avalon.activation.lifestyle.impl.SingletonLifestyleHandler.resolve(SingletonLifestyleHandler.java: 

94)
   at 
org.apache.avalon.activation.appliance.impl.DefaultAppliance.resolve(DefaultAppliance.java:585) 

   at 
org.apache.avalon.activation.appliance.impl.DefaultAppliance.resolve(DefaultAppliance.java:571) 

   at 
org.apache.avalon.activation.appliance.impl.DefaultAppliance.deploy(DefaultAppliance.java:520) 

   at 
org.apache.avalon.activation.appliance.impl.DefaultBlock.deploy(DefaultBlock.java:625) 

   at 
org.apache.avalon.activation.appliance.impl.BlockThread.run(BlockThread.java:111) 

---



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

dynamic configuration, was: Message path tracing

[Richard O. Hammer]

To suggest a simple approach, I suppose it would be possible to add 
lines such as the following to RemoteManagerHandler.parseCommand():

} else if (command.equals(START_PER_MESSAGE_LOGGING)) {
return doSTARTPERMESSAGELOGGING(argument);
} else if (command.equals(STOP_PER_MESSAGE_LOGGING)) {
return doSTARTPERMESSAGELOGGING(argument);
These commands would set a publicly accessible boolean variable. 
Logging routines would test that variable to govern their behavior. 
Access to the variable does not need to be synchronized, and no server 
restart is needed as far as I can see.

But would such an implementation go against one of the design goals of 
James?

Rich

Serge Knystautas wrote:
There's nothing in James right now that supports any dynamic 
reconfiguration, so it would require building this layer to support it. 
 In theory with JMX you can manage your James configuration, and maybe 
trigger a server restart.  Not sure though as I haven't kept tabs on 
that development.



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Message path tracing

[Richard O. Hammer]

Serge Knystautas wrote:
The only issue perhaps is what happens when some bloak leaves this on in 
a production environment... do we stop logging or do roll-overs after 
some point, or what?


This strikes me as a sort of thing that could be configured 
dynamically.  Some centrally accessible class, perhaps James, could 
have a variable:
public boolean per_message_logging_enabled;

This value, which would be tested before any per-message logging was 
done, could be turned on and off from an administrative interface 
while everything was running.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Synchronous processing of inbound emails

[Richard O. Hammer]

Obviously, I guess, synchronous processing will be achieved through a 
series of method calls.  The project on which I am working (inspired 
by James but not part of James) responds synchronously to SMTP clients 
based upon some method calls from SMTPHandler doRCPT() and doDATA().

In the first draft of my server, most of the processing (including 
forwarding or sending error email messages) is done after doDATA() has 
received the end of mail data indicator but before doDATA() has 
responded "250 Message received" -- so that a more revealing response 
might be "Message processed".  But that functionality needs to get 
fancier because the indicated processing can not always be achieved in 
reasonable real time.

I will not have a sendMail(Mail) method in the mama class of the 
server because the only reason I have seen for having a such a method 
(so named and so situated) is in the historical precedent set by Sendmail.

Rich Hammer
Hillsborough, N.C.
Guðlaugur Stefán Egilsson wrote:
I've been looking at James in order to use its SMTP service capability.
The problem is that I need synchronous processing, e.g. I need to be
able to process an email while the SMTP client is still connected, and
give an SMTP error response if I could not process the mail. In the
current implementation of James this is not provided for, since all
email messages go through the spool first, and are processed
asynchronously in a separate thread.
My question to the developers is what is the most sensible way of doing
this? It seems to me that the simplest way of achieving synchronous
processing is to modify James.sendMail(Mail mail) to bypass the spool
and call JamesSpoolManager.process(MailImpl mail) directly in the case
you want synchronous processing. 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

Noel J. Bergman wrote:
It all depends upon the RFC compliance.
To summarize this question in my view, RFC 2821 clearly offers two 
interpretations (that the end of data indicator is a period alone in a 
line; that the end of data indicator is CRLF.CRLF) which lead to two 
different sets of behavior in a few cases which probably are not very 
important.

This confusion originates in the wording of the RFC.  I think the 
RFC's writers did not understand the difference between "a period 
alone in a line" and "CRLF.CRLF", a difference which may be noticed by 
only a few writers of parsers.

Because the RFC offers these two interpretations, I expect that each 
interpretation has been expressed in some presently working code. 
Each interpretation probably has people who would fight for it.  As 
such, if we assume that the writers of a future revision to the RFC 
become conscious of the confusion possible on this issue, I expect 
they will deliberately adopt wording which continues to allow both 
interpretations.

As such, I suppose James is safely RFC compliant on this issue as it 
is now.

But, between the two interpretations, both of which I believe must 
ultimately be acceptable, I sort of like the "a period alone in a 
line" interpretation better, because I suppose that was first 
historically.  The first idea was probably to put "a period alone in a 
line" as a way to signal the end of data.

A subsequent, later idea, probably thought up by programmers who 
needed to implement "a period alone in a line"  was to scan for 
"CRLF.CRLF" (as I am guessing the history).  In fact "CRLF.CRLF" was 
probably favored by many programmers as the way to indicate "a period 
alone in a line" because it is easier to search for than "a period 
alone in a line".

But, as I continue to insist, the two interpretations lead to 
different behavior in a few minor ways in programs which attempt to 
comply with RFC 2821.

The SMTPDataInputStream class which I have written expresses the "a 
period alone in a line" interpretation, and thus behaves differently 
from the present James classes in two small ways, as described in its 
Javadoc.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

I attach a revision of the class SMTPDataInputStream which I submit 
for your consideration.  This replaces the version which I mailed to 
this list on July 2.

The external behavior of the code has not changed.  It still passes 
the JUnit tests.

This code should run faster.  I had arranged the code for logical 
clarity as much as for executing speed.  But Noel's comments made me 
realize just how rare would be the case in which this class enters its 
BUFFERING_STATE.  It may never get into that state.  Yet the code was 
testing if(receivedState == BUFFERING_STATE) for every byte read.  Now 
that contingency is handled as a case in a switch, so the most common 
reading of ordinary sequences goes faster.

In addition to the file containing SMTPDataInputStream, I attach once 
again TestSIS.java (a file of JUnit tests).  I also attach two little 
stub classes, for WatchDog and MessageSizeException, which you may 
find useful if you want to compile and test SMTPDataInputStream 
separate from a James installation.

Rich
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;

/** 
 * An InputStream for SMTP message body data. Performs four of the functions
 *  needed while receiving the message body in the SMTP DATA command:
 * 
 *   watches for the end of mail data indicator, and signals this by
 *   returning EOF, that is -1.
 *   
 *   removes dot stuffing as described in RFC 2821 section 4.5.2.
 *   
 *   works with james.util.watchdog.Watchdog to police the minimum rate of 
 *   data transfer.  Calls Watchdog.reset() every time a certain number of
 *   bytes have been read, thus forestalling intervention of the watchdog for
 *   another time increment.
 *   
 *   optionally polices the total size of the data.  Throws 
 *   MessageSizeException if this exceeds a limit.
 *  
 * 
 * 
 * The end of mail data indicator which this class recognizes is a period
 * alone in a line.  This indicator is often described as "CRLF.CRLF", but that
 * description leads to errors in possibly minor ways.  The better 
 * description which this class recognizes, "a period alone in a line", leads
 * to better behavior in two ways:
 * 
 *   When the end of mail data indicator is recognized in the input stream,
 *the CRLF which immediately preceded the period in the indicator is 
 *returned as part of the mail data as the CRLF which concludes the final
 *line of mail data, rather than being discarded as part of the end of mail
 *data indicator.
 *   
 *   The end of mail data indicator can occur in the very first line of 
 *   mail data, with the period being the first character read.
 *   
 * 
 * RFC 2821 discusses this in sections 2.3.7, 3.3, 4.1.1.4, 4.5.2.
 * 
 *  This class resets the WatchDog each time it has read a quota of bytes
 * as specified in the constructor.  But it does not reset or stop the 
 * WatchDog when it recognizes the end of mail data indicator and returns EOF.
 * 
 * This class returns EOF in two circumstances: when it recognizes the end 
 * of mail data indicator in the stream (a normal occurrence); when the 
 * underlying stream signals EOF (probably an error of some sort).  This 
 * behavior may be okay, in that it mimics the behavior of the earlier James 
 * class CharTerminatedInputStream, but it may need further examination at some
 * point.
 * 
 * An instance of this class can not be reset.  A new instance must 
 * constructed for each message's data.
 * 
 */
public class SMTPDataInputStream extends InputStream{
BufferedInputStream in;

/* For a discussion of some decisions made in designing this class,
 * see the comment at the end.
 */

// The kinds of bytes we care about
static final int
EOF= -1,
CR = 13,
LF = 10,
PERIOD = 46;

//the states in which this SMTPDataInputStream may be
static final int
LINE_STARTING_STATE  = 0, //at the start of a line
MID_LINE_STATE   = 1, //the most common state
CR_STATE = 2, //a CR has been received
INIT_PERIOD_STATE= 3, //a period at start of line
INIT_PERIOD_CR_STATE = 4, //initial period then CR
BUFFERING_STATE  = 5, //see comments further down
EOF_STATE= 6; //either EOF of end of data

//the variable in which we keep the present state
private int receivedState = LINE_STARTING_STATE;

/* This comment describes the strategy for monitoring message size and
 * data transfer rate.  The five variables below serve these purposes.
 * 
 * The use of maxMessageSize should be obvious, but note that if it is
 * set to zero then it signals that there is no limit on message size.
 * 
 * Both of the limits (message size and data rate) are checked with one
 * operation in the most frequently used code by using a quota, kept in
 * currentQuota.  When the quota is reached (by decrementing 
 * bytes

Re: new InputStream class for mail data

[Richard O. Hammer]

Noel J. Bergman wrote:
FilteredInputStream is the right thing to extend.
Thank you again, Noel, for calling my attention to FilterInputStream. 
 I had not previously understood what purpose FilterInputStream 
serves in the API.  But your suggestion, coming after this programming 
exercise in which I've had to understand the consequences of calling 
read(byte[]) when my class does not override that method, has helped 
me understand a need for FilterInputStream.

> ... SMTPInputStream
implements read() based upon the real stream, which is the one
non-constructor method that is correct in the code.  The rest are wrong.
For example:
public void close() throws IOException{
super.close();
}
explicitly illustrates a problem that is implicit with all of the inherited
methods.  The code invokes the inherited implementation, often a NO-OP.  It
ought to be delegating to the real stream.
In the particular case with SMTPDataInputStream, the class which I 
submit for your consideration, the underlying stream (the "real 
stream" as I understand your usage) is the BufferedInputStream which 
will later be read again by the envelope-command-line Reader.  So, 
unless I am mistaken, we do not want to be closing the underlying 
stream in this case.

> ... FilteredInputStream provides the
core wrapper for delegation, allowing you to override just those methods
that implement your unique behavior.
As I describe in the comment at the end of class SMTPDataInputStream, 
this implementation relies upon the behavior of the two InputStream 
methods read(byte[]) and read(byte[], int, int) which work by making 
repeated calls to the read() method in this descendant 
SMTPDataInputStream.  In this case that behavior is needed, in order 
to use the functionality which I have added only in the read() method.

The read(byte[]) and read(byte[], int, int) methods of 
BufferedInputStream, which evidently would be employed if I extended 
FilterInputStream, do not behave this way, according to my tests. 
They appear to read directly from somewhere deeper, and thus would not 
express the functionality added in the one read() method.

But FilterInputStream does look like a class I will want to use 
another time, in a similar task.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

[Fwd: Your Mail has been Quarantined: Re: new InputStream class formail data]

[Richard O. Hammer]

I am getting a message such as the following each time I post email to 
this list of James developers.

Is this something I should deal with individually?  Or is this an 
issue for the list administrator?

Rich

 Original Message 
Subject: Your Mail has been Quarantined: Re: new InputStream class for
mail data
Date: Wed, 16 Jul 2003 22:50:01 +0800
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]




Antejunk has quarantined your mail due to suspected spam content.

Mail sent to: [EMAIL PROTECTED]
Subject: Re: new InputStream class for mail data
Date sent: 16/07/2003 22:14
To release the mail to the recipient, please verify yourself as a valid
sender by clicking here



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

In case you are considering using that class SMTPDataInputStream, you 
may want to know that I am now rewriting part of it, making what I 
consider to be an improvement, an improvement stimulated by the 
discussion of the last few days.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

Noel J. Bergman wrote:
I disagree.  An empty data set can be a valid message.  I find support
in RFC 2821 Section 4.1.1.4.
Not RFC 2821.  RFC 2822, section 3.6:

   The only required header fields are the origination date field and
   the originator address field(s).  All other header fields are
   syntactically optional.
Outside RFC 2821 I don't know so well.  But I have the impression that 
SMTP message body can -- optionally -- contain a RFC 2822 message. 
But SMTP does not demand a RFC 2822 message in the body.

I wonder, do most MTA's demand that the SMTP message body be in the 
form specified by RFC 822 (or 2822)?

In any event, if you feel that your interpretation of the RFC is correct,
and that everyone else is wrong, please contact the IETF to explain where
they went wrong, and ask them to issue a correction.
I thought I was doing such homework when I took the question to the 
[EMAIL PROTECTED] email list.  But is that not the place to go with 
such issues?

As I have aged I think I have reduced the scope in which I take it 
upon myself to correct what I judge to be other peoples' errors.  I am 
inclined to let this drop here.  But I am not hiding.  If they need 
the truth they can find me.

Obviously, we want to be RFC compliant.
On that score you probably need not worry.  The RFC offers two 
distinct interpretations, and offers passages for each side to cite.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

Noel J. Bergman wrote:
I agree with you that [RFC 2821, 2.3.7] seems to clearly prohibit
SMTP clients from sending a lone CR character in message body data.


Since I am working on server-side SMTP code, I suppose I should allow
the possibility that a lone CR might come in.
And what would you do with it then?  Throw the exception, as in
CRLFTerminatedReader?
Based upon my present understanding, I would not throw an exception 
(and would not throw an exception in CRLFTerminatedReader either).  I 
would leave the lone CR in the stream, to be dealt with by whatever 
code handles it next.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

Noel J. Bergman wrote:
... as both Valdis and Keith pointed out,
[ in the thread starting at 
http://www.imc.org/ietf-smtp/mail-archive/msg00703.html ]
 an empty data set isn't a valid message.
I disagree.  An empty data set can be a valid message.  I find support 
in RFC 2821 Section 4.1.1.4.  The second paragraph starts as follows:

"The mail data is terminated by a line containing only a period, that 
is, the character sequence "." (see section 4.5.2).  This 
is the end of mail data indication.  Note that the first  of 
this terminating sequence is also the  that ends the final line 
of the data (message text) or, if there was no data, ends the DATA 
command itself."

The last sentence, as I understand it, says there can be no data.  It 
says that the first CRLF in the CRLF.CRLF sequence can be the CRLF at 
the end of the DATA command.

... As Keith illustrated, code can
ensure the proper data terminator when delivering the message via SMTP or
POP3.  What you do internally is up to the program.
The illustration given by Keith in 
 (in which he 
adds CRLF.CRLF to each outgoing message without checking to see if the 
last "line" of message data already concluded with CRLF)  violates the 
RFC as I understand it.  The paragraph from 4.1.1.4, started above, 
concludes:

"... An extra  MUST NOT be added, as that would cause an empty 
line to be added to the message.  The only exception to this rule 
would arise if the message body were passed to the originating 
SMTP-sender with a final "line" that did not end in ; in that 
case, the originating SMTP system MUST either reject the message as 
invalid or add  in order to have the receiving SMTP server 
recognize the "end of data" condition."

Note the clause, "The only exception to this rule would arise if the 
message body were passed to the originating SMTP-sender with a final 
"line" that did not end in ".  In order to pass this point, to 
go on to the clause which allows the addition of a CRLF, a program 
would have to test whether there was already a concluding CRLF present.

Please note that JavaMail sends an explicit "\r\n.\r\n" to terminate the
data stream.
In this case I believe JavaMail also violates the intent of that 
paragraph in 4.1.1.4.

I guess that this whole confusion originates in writing "CRLF.CRLF" as 
a way to indicate "a period alone in a line".
It is a darn good way to indicate a period alone in a line, because 
you need to indicate that the period in the line which you are talking 
about is the first and only character in that line.  First, before you 
start looking to see if you have a period alone, you need assurance 
that you are at the start of a line.  But once you have that assurance 
you need to set it aside and not confuse it with the thing you are 
looking for, which in this case is a period alone before CRLF.

Much of the time, for most of our purposes, we can substitute the 
indication of a thing for the thing itself and swear there is no 
difference.  The writers of RFC 2821 didn't notice the difference, as 
in this authoritative tomfoolery, quoted again from 4.1.1.4:
"The mail data is terminated by a line containing only a period, that 
is, the character sequence "." "

This blithely equates a thing (a line containing only a period) with 
the indication of the thing (CRLF.CRLF).  But CRLF.CRLF is not a 
period alone in a line.  It is (assurance of the end of a preceding 
line) + (a period alone in a line).  A + B does not equal B, not so 
long as A amounts to anything, and in this case A does amount to 
something, to CRLF.

Sometimes, when you get down into the code, you need to know that 
there is a difference between, for instance, a pointer to a variable 
and the variable itself.  But much of the time, at a certain level, 
you can forget that too.  That's the sort of confusion under this 
debate, I believe.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

I have described how I believe the recognition of "a period alone in a 
line" differs from the recognition of "CRLF.CRLF" in a few places, 
including this list:



And I have described the different behavior which results from 
recognizing one or the other as the "end of data indicator" in the 
Javadoc at the head of the file SMTPDataInputStream.java which I 
submitted to this list in email on July 2.

I think I have said it clearly in those places, and I doubt that I do 
any good by saying it again now.

Noel J. Bergman wrote:
Now, as for the code, itself.

As I said to Serge, I hadn't had time to test your code.  Also, I'm not
quite sure what goal you are trying to achieve with the change.  Would you
please elaborate?
I am responding in large part to my desire to understand my own code 
and to believe in it.  As I told before I am developing an email 
server which draws from James but which will differ in a number of 
respects.  I appreciate the lessons I learn from James and would like 
if possible to give back in some way.

In developing SMTPDataInputStream I was imagining that it might find 
use in both James and in my project.  I was trying mainly to write for 
my own project, taking the best of what I could learn from James while 
adding what I consider to be my own improvements.   But I thought that 
it might also be acceptable to the James project, so I shaped it to 
fit into James with as few changes as possible.

I see that this submission makes work for you if you undertake to 
implement and test it.  And it is in a part of James which, so far as 
I know, already seems to be working fine.  As such it may be best for 
you to set it aside and consider it no further, until such time as 
your priorities might bring you back to this area to do some refactoring.

You wrote that "The code we are using now employs buffer after buffer, and I
suspect that this redundant buffering may be unnecessary", but the only
buffers that I am finding present in the SMTP handler at the moment (I could
have missed something) are the BufferedInputStream assigned to "in", and the
line buffer in CRLFTerminatedReader.
Thank you, I stand corrected in large part.  I guess I was assuming 
that one of CharTerminatedInputStream, BytesReadResetInputStream, 
SizeLimitedInputStream, or DotStuffingInputStream, employed a 
BufferedInputStream in addition to the BufferedInputStream already 
created in SMTPHandler.handleConnection(), and I see that is not the 
case.  As you point out, there is redundant buffering in that 
CRLFTerminatedReader extends BufferedReader (in code which I suggested 
last month); I now believe that redundant buffering should be removed. 
 I notice that DotStuffingInputStream keeps a two-byte buffer with 
every byte that passes through, and CharTerminatedInputStream keeps a 
little buffer whenever a CRLF passes through.

> ...  The rest of the streams are
> unbuffered, and just add behavior.  One is a FilteredInputStream 
subclass,
> and the rest probably should be, including yours.

Thank you.  I had not considered extending extending 
FilteredInputStream.  From the Javadoc it does like that might be an 
improvement.  Why would you say it might be better?

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

Noel J. Bergman wrote:
It is not possible, by definition, because it is not permitted.  The RFC is
crystal clear on this point:
RFC 2821, 2.3.7 Lines

   SMTP commands and, unless altered by a service extension, message
   data, are transmitted in "lines".  Lines consist of zero or more data
   characters terminated by the sequence ASCII character "CR" (hex value
   0D) followed immediately by ASCII character "LF" (hex value 0A).
   This termination sequence is denoted as  in this document.
   Conforming implementations MUST NOT recognize or generate any other
   character or character sequence as a line terminator.  Limits MAY be
   imposed on line lengths by servers (see section 4.5.3).
   In addition, the appearance of "bare" "CR" or "LF" characters in text
   (i.e., either without the other) has a long history of causing
   problems in mail implementations and applications that use the mail
   system as a tool.  SMTP client implementations MUST NOT transmit
   these characters except when they are intended as line terminators
   and then MUST, as indicated above, transmit them only as a 
   sequence.

the fact that it will probably be munged in transport is a srong
disincentive to sending it, but doesn;t prohibit it


I would say that the above paragraphs constitute prohibition.
Thank you, Noel, for calling my attention back to that section, 2.3.7. 
 I had been thinking that section dealt principally with envelope 
command lines, but now I see that it does also deal with message body 
data.  And yes, I agree with you that this seems to clearly prohibit 
SMTP clients from sending a lone CR character in message body data.

Since I am working on server-side SMTP code, I suppose I should allow 
the possibility that a lone CR might come in.

Rich

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: new InputStream class for mail data

[Richard O. Hammer]

Serge Knystautas wrote:
... if you can 
submit this to bugzilla, it will help us track that it's reviewed.
It did not occur to me to use bugzilla since the improvements, which I 
believe SMTPDataInputStream offers, are not necessarily fixes to bugs. 
  The changes in behavior are arguable.  I argue that the right end 
of data indicator to recognize is "a period alone in a line" rather 
than "CRLF.CRLF", but it seems that many people see it differently.

Should I use bugzilla to express my opinions about improvements needed?

> ... JUnit tests.

I did some JUnit tests, not comparing old vs. new, but only testing 
the new.  I attach a file containing that class.  Earlier I had tested 
the old (existing) behavior, as I reported briefly to this list in 
this message on June 10, 
.

Rich

import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;


import junit.framework.TestCase;

public class TestSIS extends TestCase {
SMTPDataInputStream sis;

public TestSIS(String whooo){
super(whooo);
}

static String food = 
  ".dogs\r. sp\n"
+ ".\r\n"
+ ".\r8\r\n"
+ ".\r\r\no*"
+ " \r\n.\r\n"
+ "I guess we shouldn't get here.\r\n";

//test response to a challenging sequence of characters
public void testFood0() throws Exception {
sis=constructSIS(food, 25, 7);
assertEquals((byte)sis.read(),(byte)'d');
assertEquals((byte)sis.read(),(byte)'o');
assertEquals((byte)sis.read(),(byte)'g');
assertEquals((byte)sis.read(),(byte)'s');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)'.');
assertEquals((byte)sis.read(),(byte)' ');
assertEquals((byte)sis.read(),(byte)'s');
assertEquals((byte)sis.read(),(byte)'p');
assertEquals((byte)sis.read(),(byte)012);
assertEquals((byte)sis.read(),(byte)'.');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)10);
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)'8');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)10);
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)10);
assertEquals((byte)sis.read(),(byte)'o');
assertEquals((byte)sis.read(),(byte)'*');
assertEquals((byte)sis.read(),(byte)' ');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)10);
assertEquals((byte)sis.read(),(byte)-1);
assertEquals((byte)sis.read(),(byte)-1);
assertEquals((byte)sis.read(),(byte)-1);
sis.close();
}

//test attempt to read beyond maxMessageSize
public void testFood1() throws Exception {
sis=constructSIS(food, 6, 7);
assertEquals((byte)sis.read(),(byte)'d');
assertEquals((byte)sis.read(),(byte)'o');
assertEquals((byte)sis.read(),(byte)'g');
assertEquals((byte)sis.read(),(byte)'s');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)'.');
Exception ex = null;
try{
sis.read();
}catch (Exception e){
ex = e;
}
assertTrue(ex instanceof MessageSizeException);
sis.close();
}

//test same with different reset interval
public void testFood2() throws Exception {
sis=constructSIS(food, 6, 1);
assertEquals((byte)sis.read(),(byte)'d');
assertEquals((byte)sis.read(),(byte)'o');
assertEquals((byte)sis.read(),(byte)'g');
assertEquals((byte)sis.read(),(byte)'s');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)'.');
Exception ex = null;
try{
sis.read();
}catch (Exception e){
ex = e;
}
assertTrue(ex instanceof MessageSizeException);
sis.close();
}

//test same with different reset interval
public void testFood3() throws Exception {
sis=constructSIS(food, 6, 6);
assertEquals((byte)sis.read(),(byte)'d');
assertEquals((byte)sis.read(),(byte)'o');
assertEquals((byte)sis.read(),(byte)'g');
assertEquals((byte)sis.read(),(byte)'s');
assertEquals((byte)sis.read(),(byte)13);
assertEquals((byte)sis.read(),(byte)'.');
Exception ex = null;
try{
sis.read();
}catch (Exception e){
ex = e;
}
assertTrue(ex instanceof MessageSizeException);
sis.close();
}

//test same with different reset interval
public void testFood4() throws Exception {
sis=constructSIS(food, 6, 3);
assertEquals((byte)sis

new InputStream class for mail data

[Richard O. Hammer]

Attached you will find a class that I offer for your consideration. It 
is called SMTPDataInputStream and it would be used in the doDATA() 
method of SMTPHandler.  There it would replace 
CharTerminatedInputStream, BytesReadResetInputStream, 
SizeLimitedInputStream, and DotStuffingInputStream, since I believe it 
does the work of all those.

There are minor ways in which this class responds differently, and
more correctly I believe, than the current CharTerminatedInputStream. 
 The end of mail data indicator which this recognizes is a period
alone in a line, rather than CRLF.CRLF as recognized in James at
present.  As such this class returns the CRLF which terminates the
last line of message body data (the CRLF before the period), and it
allows the possibility of empty mail data (with a period being the
first character sent in the data).

I raised the question of whether these changes are indeed a better
interpretation of RFC 2821 on the mailing list [EMAIL PROTECTED]  If
you like you can see that thread starting at
http://www.imc.org/ietf-smtp/mail-archive/msg00703.html .  If you read
that thread and think the question remains unresolved you might adopt
the bias which I use in this case: multiply the value of each word
from Dan Bernsetin by 1000.
Improved performance would be another advantage which I would expect
from adopting this class in James.  The present stack of InputStreams
makes more method calls for each byte read than this replacement. But, 
I have to admit, it could be that performance in this code is not a 
major concern.

It is possible that this class could also serve in NNTPHandler to
replace CharTerminatedInputStream there, but I am not familiar with
the code in NNTPHandler.
I also attach a file showing old and new blocks of code in 
SMTPHandler, showing changes needed to employ this new 
SMTPDataInputStream.  In addition to these changes, the package name 
for SMTPDataInputStream will need to be corrected, as well as the 
importing of WatchDog and MessageSizeException.

I have tested this SMTPDataInputStream as a unit, and it performs 
correctly in every test I have thought to give it.  But I have not 
tested it installed in SMTPHandler.

Rich Hammer
Hillsborough, N.C.

import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;

/** 
 * An InputStream for SMTP message body data. Performs four of the functions
 *  needed while receiving the message body in the SMTP DATA command:
 * 
 *   watches for the end of mail data indicator, and signals this by
 *   returning EOF, that is -1.
 *   
 *   removes dot stuffing as described in RFC 2821 section 4.5.2.
 *   
 *   works with james.util.watchdog.Watchdog to police the minimum rate of 
 *   data transfer.  Calls Watchdog.reset() every time a certain number of
 *   bytes have been read, thus forestalling intervention of the watchdog for
 *   another time increment.
 *   
 *   optionally polices the total size of the data.  Throws 
 *   MessageSizeException if this exceeds a limit.
 *  
 * 
 * 
 * The end of mail data indicator which this class recognizes is a period
 * alone in a line.  This indicator is often described as "CRLF.CRLF", but that
 * description leads to errors in possibly minor ways.  The better 
 * description which this class recognizes, "a period alone in a line", leads
 * to better behavior in two ways:
 * 
 *   When the end of mail data indicator is recognized in the input stream,
 *the CRLF which immediately preceded the period in the indicator is 
 *returned as part of the mail data as the CRLF which concludes the final
 *line of mail data, rather than being discarded as part of the end of mail
 *data indicator.
 *   
 *   The end of mail data indicator can occur in the very first line of 
 *   mail data, with the period being the first character read.
 *   
 * 
 * RFC 2821 discusses this in sections 2.3.7, 3.3, 4.1.1.4, 4.5.2.
 * 
 *  This class resets the WatchDog each time it has read a quota of bytes
 * as specified in the constructor.  But it does not reset or stop the 
 * WatchDog when it recognizes the end of mail data indicator and returns EOF.
 * 
 * This class returns EOF in two circumstances: when it recognizes the end 
 * of mail data indicator in the stream (a normal occurrence); when the 
 * underlying stream signals EOF (probably an error of some sort).  This 
 * behavior may be okay, in that it mimics the behavior of the earlier James 
 * class CharTerminatedInputStream, but it may need further examination at some
 * point.
 * 
 * An instance of this class can not be reset.  A new instance must 
 * constructed for each message's data.
 * 
 */
public class SMTPDataInputStream extends InputStream{
BufferedInputStream in;

/* For a discussion of some decisions made in designing this class,
 * see the comment at the end.
 */

// The kinds of bytes we care about
static final int
EOF= -1,
CR = 13,
LF = 

new list address?

[Richard O. Hammer]

Should I update my address book to this new email address for the 
James developer list?

Two messages which I have sent this morning to 
[EMAIL PROTECTED] seem to have fallen into a black hole.

Sorry if I missed the announcement.

Rich Hammer
Hillsborough, N.C.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]