Deprecate Mail.getSender/setSender
I propose that we deprecate Mail.getSender/setSender in favor of unambiguous names that relate to the SMTP RFC: Mail.getReversePath/setReversePath. As it stands, we have: Mail.get/setSender === mail.smtp.from === reverse-path Mail.get/setSender !== Message.get/setSender Danny has often had to remind people of that last point. If we don't fix the terminology now, it is only going to become worse when we start making use of VERP. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: sender, from, returnPath
These are different things. The confusion is that the term "sender" is
overloaded in common parlance and some APIs, but the RFCs use unambiguous
terms. The Return-Path header (aka "envelope-from", Mail.set|getSender,
mail.smtp.from) comes from what the RFC calls the reverse-path.
Consider the headers for the message I am replying to:
Return-Path: <[EMAIL PROTECTED]>
Received: from apache.org (daedalus.apache.org [208.185.179.12])
by ziggy.conterra.com (8.12.8/8.12.8) with SMTP id h5UD5RWq046011
for <[EMAIL PROTECTED]>; Mon, 30 Jun 2003 09:05:27 -0400 (EDT)
(envelope-from
[EMAIL PROTECTED])
Reply-To: "James Developers List" <[EMAIL PROTECTED]>
From: "Vincenzo Gianferrari Pini" <[EMAIL PROTECTED]>
To: "James Developers List" <[EMAIL PROTECTED]>
These are described in:
Return-Path: header, but records MAIL FROM , RFC 2821
Reply-To: header, RFC 2822 3.6.2
From: header, RFC 2822 3.6.2
See also:
Sender: header, RFC 2822 3.6.2]
The confusion about "sender" often comes from using it to refer to something
other than the "Sender:" header, which is different from the parameter given
to "MAIL FROM" *and* different from the "From:" header. The Return-Path
header records the "reverse-path", which you can see in RFC 2821 3.3:
MAIL FROM: [SP ]
The portion of the first or only argument
contains the source mailbox (between "<" and ">" brackets),
which can be used to report errors (see section 4.2 for a
discussion of error reporting).
RFC 2821 4.1.1.2 elaborates:
"MAIL FROM:" ("<>" / Reverse-Path) [SP Mail-parameters] CRLF
Finally, RFC 2821 4.4 introduces the "Return-Path" header:
The return-path line preserves the information in the from the MAIL command.
The primary purpose of the Return-path is to designate the address to
which messages indicating non-delivery or other mail system failures
are to be sent.
Also, see http://cr.yp.to/proto/verp.txt, which discusses an important use
of the reverse-path.
This confusion appears to still be present in the new code. Look at
setSender, which sets both the From header and the reverse path to the
passed value. Those are separate issues.
I would suggest that we clean up this area in AbstractRedirect, and document
how each element relates to the RFCs.
--- Noel
-Original Message-
From: Vincenzo Gianferrari Pini [mailto:[EMAIL PROTECTED]
Sent: Monday, June 30, 2003 9:06
To: James Developers List
Subject: RE: sender, from, returnPath
Joszef,
> Another idea for AbstractRedirect:
>
> Currently the envelope sender and the from address come from a
> single config
> parameter. Two examples which show that a separate config
> parameter would be
> better:
> 1. bounce letter: envelope sender is <>, from address is
> [EMAIL PROTECTED]
> 2. checked bounce letter: envelope sender is [EMAIL PROTECTED],
> from address is <[EMAIL PROTECTED]>
>
> I don't know much about returnPath, but it may be related.
>
It is related, although is a different thing.
In RFC 2821:
The primary purpose of the Return-path is to designate the address to
which messages indicating non-delivery or other mail system failures
are to be sent. For this to be unambiguous, exactly one return path
SHOULD be present when the message is delivered. Systems using RFC
822 syntax with non-SMTP transports SHOULD designate an unambiguous
address, associated with the transport envelope, to which error
reports (e.g., non-delivery messages) should be sent.
In RFC 2822:
3.6.2. Originator fields
The originator fields of a message consist of the from field, the
sender field (when applicable), and optionally the reply-to field.
The from field consists of the field name "From" and a
comma-separated list of one or more mailbox specifications. If the
from field contains more than one mailbox specification in the
mailbox-list, then the sender field, containing the field name
"Sender" and a single mailbox specification, MUST appear in the
message. In either case, an optional reply-to field MAY also be
included, which contains the field name "Reply-To" and a
comma-separated list of one or more addresses.
from= "From:" mailbox-list CRLF
sender = "Sender:" mailbox CRLF
reply-to= "Reply-To:" address-list CRLF
The originator fields indicate the mailbox(es) of the source of the
message. The "From:" field specifies the author(s) of the message,
that is, the mailbox(es) of the person(s) or system(s) responsible
for the writing of the message. The "Sender:" field specifies the
mailbox of the agent responsible for the actual transmission of the
message. For example, if a secretary were to send a message for
another person, the mailbox of the secretary would appear in the
"Sender:" field and the mailbox of the actual author would appear in
the "From:" field. If the originator of the message can be indica
RE: WORA Considered Evil ;-)
>> I'll tell you one thing. SHIT happens, whether you use Qmail, Notes >> or James. And unfortunately YOU are not the ones who get paged at >> 11:30 pm out on a dinner date because the friggin's server has >> gone down... No ... but I have had my cell phone called from the data center when a system failed, and have to decide about abandoning my vacation and driving back 300 miles to deal with it. So far the only break-in I've ever had on a system came courtesy of a PHP 3 vulnerability caused by the one site I allowed to use it. Fortunately, it was limited by other security precautions, and the exploit has since been patched by both belts and suspenders. > > IMO Don't use James in critical heavy traffic scenarios. Do use james where > > having a java mail API in the server gives your business a real advantage. > > In part I agree this discussion addresses James' competitor in one field, but > > don't forget the whole other half of James' uphill struggle is our other > > competitor, M$ Exchange. > James, then, is addressing concerns different from mine: you're playing the > Notes/iPlanet/Exchange game, a sector of the market I don't care about. I would not put Notes and James in the same category. James and Exchange maybe, in the sense of James wanting to offer an IMAP service, which would support IMAP-based groupware. In many respects, James and qmail seem more similar: they are built from modular services, a pipeline and message stores. Other than feature differences (James lacks ezmlm's functionality right now), the significant difference is that currently James has its analogues to qmail, ezmlm and qmail-pop3d all in one JVM. That is the only similarity to Notes that I see, and it is superficial, in my view. >> Now, I _know_ that Qmail gives me a very good chance to finish my dinner, >> get back home and have a good night of sleep. Will James do the >> same thing? It does for me. Don't use it until it you feel that it would for you, too. > Turns out that our bandwidth is saturated by Sobig/E messages, > our front-exchanger sendmails and qmails have no whatsoever > problem in handling that ... What allows them to keep up? Are they fast enough to rejecting messages in real-time, or is it simply a much better pipeline, with rejection happening on the backend(s) independent of the frontend? Notes wasn't designed to be a high throughput MTA. A lot of newer Notes features (we were using Notes 4.x and 5, if I recall correctly), always felt like misplaced hacks bolted onto the core. I'm amazed that Notes still holds up as well as it does. It ought to have been ditched in the original Domino timeframe, re-architected from the ground up, and built using a real database, real MTA, real web server, etc., in a modular fashion. Which is why I would not want to add Notes features to James. In my view, it would be better to facilitate integration between James, web app servers, and other applications; and let James do what it does best. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: WORA Considered Evil ;-)
> > I admit that doing that is not my highest priority right now. We've > > got a lot of nice new contributions that need to be merged. > Noel, I'm not suggesting that you do do it, certainly not that > you do it soon either, chill out man! LOL Don't worry. It hadn't even occured to me to take it that way, Danny. :-) I was just setting expectations in case someone wanted to look for it soon. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Regarding attachment
> How do i go about [attaching a signature from a file to an e-mail]? > Can you please help me out. Start with AddFooter and modify it to read the footer from a file or other location. > Can you please tell how do i remove the horizontal rule that > comes in between attachments. What horizontal rule? I don't see anything in AddFooter that would add one by default. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: invalid email problem
James v2.2 (currently available as a test build) enables partial sends. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Relay Host
> Could any one tell me how to specify relay host in James. See the the configuration element for RemoteDelivery. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: I18N of mailet parameters?
> > One thing I've "noticed" (and this is common to all of the
> > mailets) is parameters like "notice" or "prefix" are just
> > plain text that is added to the mail message. Works great
> > if we all speak the same language but not so good if the
> > server is handling users with different languages.
> it would be useful, but what language header do you think about? Is this
in
> a new RFC (new for me, at least :-) ) or is it an internal convention?
I believe that his primary point was something like:
I don't understand ${command}
No comprendo ${command}
How you would know to use a different language? That's a separate issue.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: AstractRedirect and non-ascii characters
> Currenlty Forward eats those non-ascii characters in the subject which are > not in the default charset. > The root of the problem is the same as was in GenericListserv. (a) Would you like to submit a patch based upon your previous work? > But this is more then a few lines and maybe it will be used > in a third mailet, so it shouldn't be duplicated. Where > should I put the common code? IMO, you can submit it for AbstractRedirect. Eventually, I would expect either to see more mailets that resend mail to inherit from AbstractRedirect, or we'd refactor some of those field handling out of AbstractRedirect into a common class. (b) "The easiest workaround is to not change the subject, if not necessary." AbstractRedirect is in the process of being changed, as you know. It has been setting values where there is no request to change a value, which is redundant since we've already started with a clone. Vincenzo has already changed some of that, and is still looking at the code to complete that change. Your example, that if both and are null, then there is no need to touch the subject header, is a good one. > I noticed that AbstractRedirect does use the empty string to indicate > that no prefix was supplied. Shouldn't we change that to null, it > seems to me that usually null is used in James for this? I haven't looked at the original code, but the "" was probably for convenience since "" + subject would always be a valid operation. I agree with you that null for no change is probably the better choice, and consistent with the newly recognized convention. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [james-dev] Prefix?
> > Would it be possible to add the list prefix in the title > > for all the James lists? > I don't think EZMLM allows subject rewriting... Sorry. Personally, I like not having those prefixes. I filter by envelope, instead. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: WORA Considered Evil ;-)
> I'm sure that everyone is in favour of hardening James as much as possible. > Its just that we should approach it from a Java perspective, not a C on > Unix one. The issues are different. Not so much. We seem to all be in agreement except on the specific case of how many JVMs per machine, and why. We can discuss that in more detail on james-dev@, and take Community@ off the distribution. But we should keep Pier on the list (and add Stefano?) for this discussion. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: WORA Considered Evil ;-)
> > The internals of James models the desired architecture. The issue > > concerning you is that the current implementation does it in a single > > process. That is at least partially addressable with no change to the > > architecture, minimal (if any) change to the implementation, and some > > changes to configuration files. > I don't see why you couldn't run multiple instances of James each only > configured to provide one service (SMTP, SPOOL, POP etc) as it stands > using shared repositories. Exactly. > I think it is probably just a configuration excercise (nightmare?). Pretty much. :-) I think might best be a tweak to the spooler, since it would no longer be getting a notify() from the SMTP handler. And both James (the class) and dnsserver would be shared, since they provide services to all. > The problem is probably as simple as working out how to define more than > one $PHOENIX_HOME for start/stop scripts I admit that doing that is not my highest priority right now. We've got a lot of nice new contributions that need to be merged. But Stefano and Pier have raised excellent points, which should be addressed. If anything, they validate James' architecture. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: WORA Considered Evil ;-)
Steve and Kenny, There is a balance. Not all of Pier's issues may apply, but many of the important ones do. Frankly, I don't want to run anything at root that can avoid it. That is just good practice. Consider Vincenzo's anti-spam matcher. Would you want that to run as root? I am not convinced that an exploit cannot be written that would cause the commercial, native code, virus scanner to have a buffer overflow causing it to execute code embedded in a carefully crafted message. Can you prove otherwise? How big a bet would you be willing to place? Because that is the core of decision theory: what are you willing to risk on your choice? > There is no need to distribute James' components over multiple JVMs to > enhance security or robustness. I disagree as a technical matter. As a practical matter, it depends upon one's configuration and comfort level. Clearly most users experience levels of both security and robustness with James that meet their requirements. > The paranoia comes from what's called a buffer overflow > exploit. > Java however doesn't suffer from this kind of attack As a matter of fact, there was one involving the native compression code. I agree that Java is safer than C/C++, but that does not diminish the point. I can give you plenty of good reasons for why you want the ability to run the mailet pipeline at privileges other than root: - the chance of a JVM exploit. - potential exploits via native code in a JDBC driver. - the use of native code in matchers/mailets, e.g., the anti-virus matcher. --- - the use of third party matchers/mailets. - the use of user-defined scripting matchers/mailets - support for SOAP - one pipeline being extra busy or big performing lots of processing and/or handling large messages, should not deny service to other users. None of the items below the bar are related to native code exploits. A single JVM is not secure when you start taking into account such things as SOAP, user-defined scripts, etc. Would you run Tomcat as root, and allow users to install whatever servlets they want at root privilege? For most users, the risk involved may be effectively non-existent, but the more flexible a system becomes, the more the system must be architected to prevent exploits due to an oversight failure. For Tomcat, I make sure that each virtual host's JVM(s) run(s) with the access right's of the host's owner. If I have mail applications for multiple hosts, I also want to ensure separation, security, and privacy. Next, consider the dynamic reconfiguration issue, which thread I have no had time to respond to over the past couple of days. The key issue there is that to reconfigure the pipeline, I should not have to impact the other services. What you might do for a graceful reconfig is to put the pipeline into a mode where it will process all messages EXCEPT for those in the root spool. That lets you flush out the current pipeline, while allowing the SMTP server to continue spooling messages. Then I can gracefully restart the pipeline processor without any impact on SMTP, POP3, NNTP or other service. We've already had the plan for a distributed pipeline cluster, so I don't see any reason to be arguing the point. If you look at Pier's message, you should see GOOD things, not bad. The architecture is basically exactly what he wants. > Using multiple VMs forces the use of some kind of remote interface. We already have one: the spooler and message stores. Separation would occur where natural divisions exist. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Dynamic Reconfiguration
> Avalon frameworks manage the core tasks and should ideally manage the > reconfiguration of those tasks. Achieving this means that all Avalon > components benefit from these advances. Ideally, yes. This is something that we should bring up with Avalon, not just here. They should provide the core facilities, and we should know how to use them. > The configuration file, config.xml, is essentially a persistent store for > the configuration parameters. As it is XML based we may as well go with it. > We may well want to expose the parsed parameters as Java objects through > some kind of interface. You mean, other than the existing ones? > Probably James would use the Java objects as its configuration source > and perist any changes to the Java objects by updating config.xml. Avalon provides the configuration interfaces, and should be responsible for the core support. However, I would not want to see normal components able to effect configuratin changes. The JMX support should be able to do so. Basically, I agree with your thoughts. I am simply emphasizing that the core integration of (re-)configuration and JMX should be part of Avalon. If we do it here, for example if that is something you want to undertake, it really should be done by contributing to Avalon. --- Noel cc: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: WORA Considered Evil ;-)
> I was raising concerns that *real* sysadms have. I don't know any sysadm > who, free to choose his server operating system, would choose windosh. I personally don't consider Windows to be robust and secure enough as a server operating system. But that is far from a universal view, and I don't feel compelled to enforce my personal opinion by abandoning over 40% of my community. Is httpd deciding to drop support for Windows (rhetorical)? > If you want James to be good-enough for the ASF (even if it doesn't get > into production), forget your windows user base and target the real > heavy duty servers. I don't believe it necessary to architect or implement something that excludes Windows in order to deliver a suitable product. The internals of James models the desired architecture. The issue concerning you is that the current implementation does it in a single process. That is at least partially addressable with no change to the architecture, minimal (if any) change to the implementation, and some changes to configuration files. Providing that configuration as an alternative to a single JVM is certainly something we can take a close look at offering. There are good reasons for allowing the mail administrator to choose (easy of use and reduced memory consumption vs robustness and flexibility). --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: I18N of mailet parameters?
Jim, > One thing I've "noticed" (and this is common to all of the mailets) > is parameters like "notice" or "prefix" are just plain text that is > added to the mail message. > One thought was to add a resource bundle name and key names > as parameters to the mailets. We could use either resource bundles or XML resources. See the new XMLResources.java, which is a generalization of SQLResources.java, which came from some other source whose history is lost (Darrell may remember). Mark Imel's new list server is the first code to use it (other than my XMLTest mailet). I will be posting that up later today; this weekend at the latest. There is code in XMLResources for doing symbolic parameter substitution, which might be good to pull out into a general utility class, since it can be useful other than in the context of an XML resource file. Is this something that you would like to contribute to? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Regarding attachment
> I have to attach a signature file to all outgoing > mails with the username, designation and mobile > number. Take a look at AddFooter. That is the code you want to use for a start, but you'll need to have a convention where you store footer files in some location keyed by the user. In James v3, that could be an attribute on the user record, but for now you'll need to do something else (like look it up from a database record). > also if there is a signature that is present then that has to be > removed or replaced with the signature that is created by me. How are you going to recognize that a message has an existing signature? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [PATCH] Scripted Matcher & Mailet Classes - Take 2
Steve, I'll try to get to this over the weekend, and clear up a backlog of patches. For future reference: - please just submit new files intact. - please do a build just prior to sending the file. The build system will remove tabs and ensure UNIX line termination. - we don't use @author tags, so we'll remove those when posting, and we will use @version $Id$. Nothing for you to do now, but if you do rememeber in the future, it will just help ease the workload, especially when we get a bunch of patches, as we have from you, Soren, and Mark. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Regarding attachment
> So please can anyone let me know if there is any possible > way to attach a file to the mails outgoing from james server. There isn't a standard mailet for doing so, but it would be fairly straightforward to do. Are you saying that you need something like: path-to-file [mime-type] [subject] Or is there something more to it? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Piping to external applications
[From community@: RE: WORA Considered Evil ;-)]
> Java is the glue, let's glue it to what we have and trust... I believe
that
> 3 years ago, I asked him to do a mailet that could be an interface to
> "/usr/(lib|sbin)/sendmail", using Runtime.createProcess... After 2 years,
< that mailet didn't come up, and well, I decided to use EZMLM.
That is "before my time", but I will point out that we do have users who
have implemented anti-virus scanning in a similar manner. The message is
written out to disk, and Runtime.exec() is used to launch the native
application.
ref: http://portale.praxis.it/pub/james/
I don't see why the same basic code couldn't be used for:
// [replaceable parameter list]
[false]
e.g.,
/usr/lib/sendmail -B7bit -f${sender} -- ${recipients}
This would not be a difficult, or even non-portable, construction. One
question is what, if anything, to do with the output from the command. What
did you have in mind?
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: WORA Considered Evil ;-)
[Reply in multiple pieces based on sub-topic] > A few months ago, I had a very interesting conversation with Pier on JAMES. Thanks for the background. I'd heard some of it from Serge over time. And the servlet topic gets brought up from time to time by people who see the obvious similarities, but not the key difference. > Pier had pretty rock solid arguments *not* to use JAMES as a MTA > and all came from the sysadm paranoia One example would be that you run James as root in order to access the privileged ports. And if James runs as root for the ports, all of the code runs as root. There are OS specific workarounds, but it is a valid view. > Unfortunately, I don't recall exactly what his arguments were, Pier, do > you have a minute to chime in? I think the JAMES people would love to > hear your criticism. Absolutely! :-) Although the discussion would probably be more appropriate on the James Developer mailing list than here. > [qmail's] separation-of-concerns-driven design is incredibly solid > and very scalable, something that can be matched (or even improved, > I believe) in a multi-threaded environment, but this gives a single > point of failure to the system that is not acceptable from a paranoid > sysadm point of view. James already has a pipeline architecture. The issue you are raising is that the pipeline is in a single process. One of the things we've talked about is allowing multiple distributed processes coupled through stores. There is a plan; it just takes time and resources to get there. Actually ... hmmm ... now that you've got me thinking about it, it occurs to me that it should be possible even now to separate James into separate JVMs for the SMTP service, mailet pipeline, and POP3 service. Once the SMTP service puts a message in the pipeline, its done with it, so that's OK. Once the mailet pipeline puts a message in the pop3 store, its done with it, so that's OK. This approach would not have a multi-process mailet pipeline (which requires the new spooler), but it would allow the majority of the code to run in normal user mode, and would allow different parts to be restarted if/when desired. You know ... I think that would work just fine. Maybe a bit of a tweak in some spooler code. > Java doesn't have concepts like native process forking capabilities Java has threads, instead. Not all operating systems support fork(). You can launch new processes, but they don't start as quickly as a fork. It would be nice if JVM startup were light(er)weight in the presence of an already running a JVM. > no notion of OS security The Java 2 security model and JAAS are different, but effective. In terms of file system items, it seems to me that a JNDI service provider could provide full OS functionality by mapping file system specific properties to JNDI attributes. > As you, I would love to see JAMES be used as the ASF email > infrastructure because it would allow fancy interoperability, > expecially between the email realm and the web realm, which are, > historically, hard to mix in an elegant and coherent way. > At the same time, I strongly believe that without a healthy amount of > native code, it is impossible to match the sysadm needs and fears. As I've said, one of the reasons for picking the ASF mail server as a goal is that provides a concrete set of important, real-world, requirements that should be met. So when you say: > I'm aware that Noel wants the apache mail infrastructure to run on james, > which is a goal I would *love* to see happening, but I fear that he's not > taking into account the sysadm paranoia that runs in the apache > infrastructure team. please understand that simply being the ASF mail server is only the destination. The value comes from what will have had to happen to get there. In other words, being the ASF mail server isn't the point. Being good enough to be the ASF mail server is the point. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: subject prefix mailet or unaltered recipients in Redirect
> > Always was. I propose that for undocumented defaults, that we change them > > to be consistent (as discussed in this thread), and document the changes. > We can't do that. It would break most config.xml files around. Yes, I know ... <> I said as much in an earlier message. Wishful thinking in a moment of weakness. I was kind of hoping that no one would object, but you are correct to do so. > I suggest to complete what I'm doing getting a totally back compatible > and documented Redirect, and later on we can have a new one [that] > behaves as Noel says, and Redirect extends it just changing the > defaults to keep the compatibility. Probably the best thing to do. Redirect used to call itself Resend. We could adopt that name for the new mailet if we don't come up with a better one. Would it be reasonable to have AbstractRedirect have the consistent defaults, which are then picked up by AbstractNotify, Forward, and other new subclasses of AbstractRedirect; and have Redirect introduce its compatibility values? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: subject prefix mailet or unaltered recipients in Redirect
> > As you already said, Redirect suffers from the need of backward compatibility, > > that we must guarantee. We could in the future create a new mailet, with a > > different name (which name BTW?) following more consistently the conventions. > This mailet has become a veritable swiss-army knife of confusion. :) Always was. I propose that for undocumented defaults, that we change them to be consistent (as discussed in this thread), and document the changes. Doing: should effectively do nothing because you haven't told it what to change. > It might benefit by having some mailets that extend this with options > hardset and a more understandable name. Forward, for example. :-) By default, Forward requires nothing more than a recipient list, and should normally do nothing but change it. The Notify and Bounce mailets are also pretty clean. The one parameter whose default isn't related to "change/don't change" is . AbstractRedirect defaults it to false. AbstractNotify changes that so that Notify/Bounce mailets leave the original message in the spool by default. That behavior seems fine, so long as it is documented. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: subject prefix mailet or unaltered recipients in Redirect
> > Redirect would insert a new, Redirected, message into the spool. > I wasn't sure that it _always_ does this. Technically, it would depend upon a variety of parameters. For example, if passthrough is false, then it may be possible to reuse the message. > I already added a wiki comment about the endless loops related to > this (otherwise reasonable) behaviour. Looping, again, depends upon the matching criteria. > > The default for Redirect is that the inline type is BODY. Try using: > This is another case where the Redirect use a different rule > then the "if not specified it is not changed". BTW this > should also be documented. I can submit a patch. Please do. Personally, I wish that the Redirect defaults were different from what they are, but changing them would potentially break existing configurations. In my opinion, should be effectively a no-op, since nothing would change. On the other hand, for defaults that are not documented, perhaps we can make some changes to establish some consistency and sanity. Obviously, we'd have to document the change(s). Opinions? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Jabber IM server #1
Alexis Agahi wrote: > On Tuesday 24 June 2003 20:11, Noel J. Bergman wrote: > > > I've restarted coding from scratch a clean Jabber IM protocol server. > > > Future plan include server2server jabber protocol implementation, > > > James integration, EOB usage. > > Might I suggest that you join us over at the James project? If you look at > > > > http://nagoya.apache.org/wiki/apachewiki.cgi?James/Development > > > > you will see that Jabber is on the list of protocols to support. > I know ;) (you contacted me few weeks/months ago) And it is still a good idea. :-) > Any suggestion where I should look at / how I could start? Take a look at the other services, NNTP, POP3, SMTP, etc. Odds are that you can start by cloning one of them off and stripping out the protocol handling from the handler class. Also take a look at JavaMail, particularly Message. I think that there are things in JavaMail that didn't keep Message as content-neutral as it could have been, but I think that they are resolvable with an XMPPMessage subclass, and perhaps using the XMPP/MIME mapping (http://www.jabber.org/jeps/jep-0081.html). I would expect some change within James to accommodate this work, but the overall effect shouldn't be too great, and long term beneficial. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Jabber IM server #1
> > > I've restarted coding from scratch a clean Jabber IM protocol server. > > > Future plan include server2server jabber protocol implementation, > > > James integration, EOB usage. > > Might I suggest that you join us over at the James project? If you look > > at http://nagoya.apache.org/wiki/apachewiki.cgi?James/Development > > you will see that Jabber is on the list of protocols to support. > If you're supporting Jabber, has the James team thought about BEEP? I've > been looking into it since it'll be used in the iCal spec. We are not supporting Jabber (XMPP) at this time. It is on our list of enhancements. We are always looking for developers who want to help enhance James, and this is a perfect example. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Gump
Great. :-) When the new modules are available, don't forget that we have some offers of help for the site. :-) --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Jabber IM server #1
> I've restarted coding from scratch a clean Jabber IM protocol server. > Future plan include server2server jabber protocol implementation, > James integration, EOB usage. Might I suggest that you join us over at the James project? If you look at http://nagoya.apache.org/wiki/apachewiki.cgi?James/Development you will see that Jabber is on the list of protocols to support. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: subject prefix mailet or unaltered recipients in Redirect
> However there is another problem, now I have an endless loop. The symptom is > obviously that the Redirects reinserts the message to the spool. I am not > sure, if this is the intended behaviour in this case, or only an accident. Redirect would insert a new, Redirected, message into the spool. true [SPAM blackholes-easynet-nl] true Did you mean to have two copies? The Redirected one, and the original? Or do you want to change passThrough to false? > If you look at the attached message the body seems to be recursively > included, this is surprising. Oh? Soren was reporting something similar, and reported the cause as: The problem was that the original mimemessage was "attached" to the new mimemessage, this behaviour in some circumstances (here I am a little on thin ice, cause it did not always happen) led to a self-referencing MimeMessage, and storing those leads to the observed behaviour. The default for Redirect is that the inline type is BODY. Try using: UNALTERED false which should use the original message. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Imap in James
> > The issue with IMAP is that we need to get the new repository structures > > going for James v3, and we need people whose interest is IMAP rather than > > high performance mailing list delivery, anti-SPAM, administration, or > > what-have-you. > If that's the case, as I do want to use James, and won't use it until it > has IMAP, I shall have a look into the internals of it's IMAP, and see > what I can do. Have you tips as where is best to start, or should I just > immerse myself, and see what I think needs improving most? Check out http://nagoya.apache.org/wiki/apachewiki.cgi?JamesIMAP. AFAIK, that is the latest information. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Mails expanding (like crazy)
Glad to hear that you have it sorted out. > Now back to testing attributes on mail. Please look for a message sent yesterday from Vincenzo. He'd like to work with you on finishing up the mail attributes. :-) --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: NOTICE: *IMPORTANT* CVS MODULES ARE MOVING
Stephen, We've been talking about it for quite some time. This is just an opportune point to do it. > Personally I would really love to see a breakout of the different > components in James as seperate subprojects (i.e. going beyond > james/mailet separation and into the seperation of the basic building > blocks - smtp, dns, remote, pop3, nntp, etc. I don't believe that is going to happen. We do not need a CVS module per service, IMO. > Is this a good time to be thinking about migration to Maven as well? One step at a time, but yes. Didn't you see my comment in [EMAIL PROTECTED] asking if we could borrow Leo? :-) Also, Dion Gillard had done some work on it, but he felt a bit stymied, IIRC, by the lack of a proper Avalon release. As you know, that's changing. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Removing Paul Hammant's temporary commit access
> Paul Hammant graciously offered to help James keep in step with > Avalon's changes. And did a great job, too. :-) > Since then, Paul's code updates are complete (thanks!). Also, Steve > McConnell has largely taken over as the Avalon comitter with temporary > James access. > I propose we remove Paul's commit rights to James. +1. However, if Paul wants to remain on and contribute to James, I would welcome his presence. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
NOTICE: *IMPORTANT* CVS MODULES ARE MOVING
Folks, This is a notice to let you know that the James CVS modules will be changing. The current jakarta-james module will become james-server. Initially, everything will be in there, but we're going to separate out the web-site into a new module (probably james-site), and eventually other modules, e.g., james-mailet. Once the change has been made, you can edit cvs/Repository, and change it to read james-server instead of jakarta-james. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: strange exception
> While testing my custom mailet under high load > (approx. 5 messages per second) I get this > Exception from James [v2.1.2]: > java.lang.RuntimeException: Exception caught while retrieving a stream: > java.io.FileNotFoundException: .../var/mail/spool/...Repository.FileStreamStore > at filepair.File_Persistent_Stream_Repository.get(line 71) This is James trying to READ an existing stream, and being told that it no longer exists. > at MimeMessageAvalonSource.getInputStream(line 59) > at MimeMessageWrapper.writeTo(line 198) > at MailImpl.writeMessageTo(line 376) > at AvalonMailRepository.store(line 274) > at James.sendMail(line 444) This is James putting a message into the spool. An InputStream for the original message is requested so that James can stream from one file to the other. > at MailService.forwardMail(line 317) > at ForwardMailet.service(line 146) This is the code that created a new message to be copied from the other, specifically: > myMailetContext.sendMail(mail.getSender(), > recipientsToAdd, > mail.getMessage()); > If I limit the load (eg one mail / second) the exception does not occur. > Any ideas ? Please try v2.2.0a7, if you don't mind testing, to see if this has already been addressed. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Emeritus status
+1 to give emeritus status to duncan, jon, bburns --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: subject prefix mailet or unaltered recipients in Redirect
> Sure it can be done, although this could be done everywhere, > simply changing in AbstractRedirect.getSpecialAddress > specialAddress = SpecialAddress.UNALTERED; > to > specialAddress = null; Actually, I had in mind handling UNALTERED explicitly in the code, just as we do the other special addresses. > it would break in Redirect.getTo(Mail) the fact that if is not > coded, means set it as the value That would be an exception to the rule, for compatibility with the prior behavior. > I think that we should stress even more in the docs the adopted > convention that not coding anything means null that means *no change*. Sure. That is intuitive for the Administrator. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: subject prefix mailet or unaltered recipients in Redirect
> In the 2.2.0a7 Redirect, not coding any "recipient" parameter > means keeping it unaltered. I was just going to point that out, having noticed it in the comments. But for ease of use, and symmetry, perhaps we should allow an explicit "unaltered", too? We already have it as a recognized special address. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Mails expanding (like crazy)
Søren, > A mail (a small one a few kbytes and text only) is sent and it is > then repeatedly written into the same file in spool, making this > file explode to several gigabytes in no time. You are saying that it is not many copies being spooled, but a single spool entry having the same data concatenated over and over and over again in a single message? What is causing the loop? > some of [our custom mailets ] are using: > MimeMessage msg = new MimeMessage(...); > ... > mail.setMessage (msg); AbstractRedirect does the same thing, except on a new, duplicated, instance. Looking over the code for the file-based message store, the code path for actually storing a message is shared regardless of whether it is a new or old message. Have you made sure that your mailets aren't accidentally concatenating the content? In the AddFooter mailet, you'll find a section of commented out code that you can use to dump the raw message contents to the log for testing. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Imap in James
Max, IMAP support at the protocol level is present in one of the proposals. Missing is repository support. More importantly, missing are interested parties eager to work on IMAP support. Development on James has been anything but stalled. There has been constant and active James development at a pretty good clip. There are quite a few things actively in the pipeline as we speak, with more contributors than before. The issue with IMAP is that we need to get the new repository structures going for James v3, and we need people whose interest is IMAP rather than high performance mailing list delivery, anti-SPAM, administration, or what-have-you. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: zero length array in AbstractRedirect
Vincenzo, We're talking about the same thing. :-) My reaction to seeing: apparentlyTo = new InternetAddress[0]; was that it was a bit of an odd construction, since 99 times out of a 100, someone would just assign a null. > I wrote [because] later on the call > >setTo(newMail, getTo(originalMail), originalMail); > > would behave differently if getTo(originalMail) returned null: in > this case the TO: value would be the original one (unchanged); > with an empty array it would, as you said, be converted to null And my point was that we should explain that so it doesn't get broken in the future when someone new goes to make changes, and says "Oh, that's odd. We don't need to create an object. I'll just set this to null." --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: XML Entities in config.xml
Steve, ]> &testScript; Although I like the approach that you appear to be taking, you might want to consider that Avalon keeps the XML parser and all generated objects in-memory for the lifetime of the application. I haven't looked in their code to see why; I am basing this upon what I see when using the heap profiler. If we are going to expand the size of config.xml, which your approach would potentially explode, then we should try to get more memory efficient handling from Avalon. Another thing to consider is reconfiguration. This approach would appear to require reloading the entire configuration to change a script, at least with a simplistic implementation. By the way, is that file url being interpreted relative to the phoenix bin, as implied by your example? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
zero length array in AbstractRedirect
Vincenzo,
Did you mean to do this?
if (mailAddress == null) {
// set to <>
apparentlyTo = new InternetAddress[0];
}
I suspect that you did, but please add a comment explaining why in more
detail. Which code was throwing the NPE if you used null instead? From
what I can see, JavaMail should accept null as a parameter to
MimeMessage.setRecipients(), and internally converts a zero length array to
null.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Wish to contribute towards development of James
> My name is Jayashree Ravi and am a student at Wayne State Univeristy. I am > very much interested in becoming and user and a developer for James project. Welcome. :-) > Please include me in the mailing list Just follow the general instructions to subscribe: http://jakarta.apache.org/site/mail.html > I would highly appreciate if I am given some guidelines for contributing > towards development of James See: http://james.apache.org/contribute.html --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Catching Throwables thrown by matchers/mailets (was [PATCH]Endless loop in JamesSpoolManager)
> RuntimeException is a subclass of Exception > It's java.lang.Error that is a Throwable and not an Exception. True enough. I had 3 hours of sleep. :-) In any event, reviewing exception handling is always a good thing. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Catching Throwables thrown by matchers/mailets (was [PATCH]Endless loop in JamesSpoolManager)
Vincenzo, Welcome to one of my favorite James nightmares. I've made a number changes from Exception to Throwable becaue of RuntimeException. No, do not rename to onThrowable. Users don't make the distinction. Also please see: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=18124 Are you sure about your assessment that the message is lost? I looked over that code before, and it seems to me that JamesSpoolManager.process() would leak the exception back to run(), where it would be caught. The message would remain in the spool, the key would stay locked, and the message would be unavailable for processing. That might or might not be the desired result in all cases, but the message would not be lost. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: cvs commit: jakarta-james/src/java/org/apache/james/dnsserver DNSServer.java
As stated on the bottom of every message: To unsubscribe, e-mail: [EMAIL PROTECTED] --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Improving temporary failure handling in RemoteDelivery
Serge, The particular error I received: 451 4.7.1 Please try again later follows RFC 2034 (http://www.ietf.org/rfc/rfc2034), The 451 code is from RFC 2821, and the 4.7.1 is from RFC 3463 (http://www.ietf.org/rfc/rfc3463.txt), which tries to clean up the SMTP status code mess. Mind you, the use of 4.7.1 seems wrong, although common to both qmail and sendmail. The RFC says (in two parts): 4.XXX.XXX Persistent Transient Failure A persistent transient failure is one in which the message as sent is valid, but persistence of some temporary condition has caused abandonment or delay of attempts to send the message. If this code accompanies a delivery failure report, sending in the future may be successful. X.7.1 Delivery not authorized, message refused The sender is not authorized to send to the destination. This can be the result of per-host or per-recipient filtering. This memo does not discuss the merits of any such filtering, but provides a mechanism to report such. This is useful only as a permanent error. But getting back to RFC 2821, here are its descriptions for the 4xx codes: 421 Service not available, closing transmission channel (This may be a reply to any command if the service knows it must shut down) 450 Requested mail action not taken: mailbox unavailable (e.g., mailbox busy) 451 Requested action aborted: local error in processing 452 Requested action not taken: insufficient system storage Based upon that, 421 and 451 imply trying another server, in my view. And speaking to the operations center at Road Runner, they were quite clear that they expect 451 exceptions to cause mail servers to go to the next one. Those are errors that are likely to be local to the server. 450 and 452 are more debatable, however, I will point out that RFC 2821 #5 says: When the lookup succeeds, the mapping can result in a list of alternative delivery addresses rather than a single address, because of multiple MX records, multihoming, or both. To provide reliable mail transmission, the SMTP client MUST be able to try (and retry) each of the relevant addresses in this list in order, until a delivery attempt succeeds. However, there MAY also be a configurable limit on the number of alternate addresses that can be tried. In any case, the SMTP client SHOULD try at least two addresses. > To fabricate a case that could justify current behavior, say I have a > quota on Joe Smith's mailbox at 10 MB. His mailbox is at quota, so I > don't want to accept more messages for Joe. That particular example is a mess. The RFC is ambiguous, and regardless of what some people might say is correct, the unfortunate fact is that some servers reject with 450, some with 451, some with 452, and still others with 552. Dreadful. > Meanwhile, I have a remote backup server that accepts messages when the > primary is down. Because it is remote, I can't check whether a mailbox > is full, so I don't reject based on the quota. > Right now our server would see that Joe is overquota, and try again > later, until either Joe clears some messages or our retry fails. If > you change it, you'll deliver to my remote server And when the backup server is able to deliver to the primary, the primary can do the quota checking. > But I would have preferred Joe's sister never send those 10 > additional 5 MB messages in the first place. That is why some servers reject a full mailbox with a 552, so that the mail is never resent. > Sorry the lengthiness Don't be! These are good issues. If I didn't want your view, I would't ask (and I wouldn't be here). > it depends on whether 451 is a reply for the domain > or for this server. I agree. It seems to be ambiguous. > I tried reading through the RFCs, and am only the worse for it. :) I know. They give me headaches, too. One thing that we could instead do is change our default retry behavior. RFC 2821 recommends "favoring a policy of two connection attempts in the first hour", and every few hours for at least the next 4-5 days. We default to waiting 6 hours, and abandon with two days. If we retry sooner, and we make sure that the MX records are randomized, it should help. I've just committed a change to do randomization, although the DNS also does some of that, too. By the way, RFC 2821 #4.5.4.1 suggests that we change our queuing to follow suggestions that we've made earlier. It says that if we fail in sending a message to a domain, we should not send other messages to that domain until a timeout has passed (or we get a message from that domain). A client SHOULD keep a list of hosts it cannot reach and corresponding connection timeouts, rather than just retrying queued mail items. Something for the future queuing model. --- Noel -
RE: Null sender
> > But I'm asking myself too on what James does if it tries
> > to send a message to an empty list of recipients
> Ideally the NotifySender should see the sender is null and not try to
> bounce to it.
Agreed.
> Separate from that, the LinearProcessor should see that there are
> no recipients anymore on a message, and end processing.
Going back as far as the CVS module records, this code handles that case:
if (recipients.size() == 0) {
//Everything was not a match... store it in the next spot in the array
unprocessed[i + 1].add(mail);
continue;
}
As far as I can see, we will continue to call the Matchers, but never a
Mailet. Please check me on that analysis.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Server statistics (was: Personal IP blacklists)
Jason, I suggest to you that the JMX MBeans are orthogonal from authenticated user. Let's get JMX into James, and let me go talk to Avalon about helping us lock out unauthorized users. In fact, they are voting today to give James Committers access to the components area of their CVS module. We won't release James with JMX until we've secured it, but lets get the instrumentation part of the project going. Fair enough? :-) --- Noel -Original Message- From: Jason Webb [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 18, 2003 4:01 To: 'James Developers List' Subject: RE: Server statistics (was: Personal IP blacklists) I'd still like to do this (give James real JMX usability), but I have one hang-up with the current Avalon implementation in that I need Avalon to provide authentication which it currently doesn't do. Once this is done then I can start Mbean'ing up James properly for control. Without this I won't start any work as it would James a serious security hole. My eventual goal is to instrument James as completely as possible :) I have the JMX book and I am ready... -- Jason > -----Original Message- > From: Noel J. Bergman [mailto:[EMAIL PROTECTED] > Sent: 18 June 2003 08:05 > To: James Developers List > Subject: RE: Server statistics (was: Personal IP blacklists) > > > JMX is very conceptually compatible with the ad hoc approach > Jason and I had discussed last summer. Remember what I said > about using a statistics object and a registry? From the JMX > specification: "The MBean server is a registry for MBeans [in > the application.]" Very similar concepts. Not that I would > claim uniqueness. SNMP, /proc, and other schemes are all in > the same pattern. > > JMX is the right way to do this for James, since it is a > standard, provides control, is embedded in Phoenix, is > growing in use, and provides a replacement for the aging > remote manager interface. > > > I imagine that the class doing the work would have a variable which > > holds the statistic we care about. > > Classes. Plural. Lots of beans. :-) > > > This would cause SMTPHandler to refuse or defer some incoming > > connections. > > Minor nit, but this is wrong. You need to understand the > archtectural relationships between some of the components. > The protocol handler is used after a new connection is > established. Rejecting a new connection based upon busy > state would happen elsewhere. > > As I've said before, please feel free to contribute. If > you're interested in working on JMX for James, got for it. :-) > > Avalon has been using http://mx4j.sourceforge.net/. > > --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Null sender
> Should I change it to something like the following: use the sender,
> if null use the return path, if null leave it null (or throw an
> exception)?
RFC 2821 says:
If there is a delivery failure after acceptance of a message, the
receiver-SMTP MUST formulate and mail a notification message. This
notification MUST be sent using a null ("<>") reverse path in the
envelope. The recipient of this notification MUST be the address
from the envelope return path (or the Return-Path: line). However,
if this address is null ("<>"), the receiver-SMTP MUST NOT send a
notification.
For a Bounce, it seems to me that the null (<>) means don't send a
notification message.
> I'm fixing a bug in AbstractRedirect that was causing a NPE trying
> to build a new TO using the sender, that it turned out null in some
> cases (spam or virus). I'm going to do the following in such case:
> use the sender, if null use the return path, if null set it to <>.
Bounce is clearly defined by RFC 2821. My thought is that NotifySender
should be the same as Bounce, but with a non-null return-path. That is what
we agreed upon previously (I checked the archives), is closest to the
current use, and consistent with the somewhat ambiguous use of "Sender."
Another mailet (or magic address) could follow the reply policy from RFC
2822:
The originator fields also provide the information required when
replying to a message. When the "Reply-To:" field is present, it
indicates the mailbox(es) to which the author of the message suggests
that replies be sent. In the absence of the "Reply-To:" field,
replies SHOULD by default be sent to the mailbox(es) specified in the
"From:" field unless otherwise specified by the person composing the
reply.
For reasons suggested in RFC 2822, the "Sender:" header would not be used to
reply.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Improving temporary failure handling in RemoteDelivery
In RemoteDelivery, we have the following:
// This was not a connection or I/O error particular to one
// SMTP server of an MX set. Instead, it is almost certainly
// a protocol level error. In this case we assume that this
// is an error we'd encounter with any of the SMTP servers
// associated with this MX record, and we pass the exception
// to the code in the outer block that determines its severity.
throw me;
This is when handling a MessagingException that is not SendFailedException
during an SMTP transaction. I just encountered one of these with:
RemoteDelivery: Exception delivering message (Mailxxx) - 451 4.7.1 Please
try again later
RemoteDelivery: Temporary exception delivering mail (Mailxxx:
javax.mail.MessagingException: 451 4.7.1 Please try again later
This was a temporary outage for 1 of 3 primary, 2 secondary, 1 tertiary, and
1 quaternary mail servers, i.e., 1 out of 7 possible servers. It seems to
me that James should try the others in the case of a temporary exception,
e.g.,
if (ex.getMessage() != null && '4' == ex.getMessage().trim().charAt(0)) {
// best guess at a temporary failure
lastError = me;
continue;
} else {
throw me;
}
Comments?
--- Ne
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Javadoc @version, @since and CVS $Id$
> > One more reason for Subversion, which I believe supports sharing. > Really? Hmmm ... I think so. Support for symlinks is on the TODO list, but as I understand the SVN repository model, it already supports lazy branching. See: http://svnbook.red-bean.com/html-chunk/ch04s02.html. If you don't want to read the whole thing, skip to the highlighted text labeled "Cheap copies." --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: External Jars for Scripting
> What, if any, is the policy for packaging [additional] jars? bsf.jar > should probably be added to /lib, but what about the optional jars? > Unless I'm missing something, they have to end up in the .sar file > to be found at runtime. This doesn't make them very optional! You are missing something. :-) This is for a Mailet, therefore as of version 2.2, you ought to be able to put them into SAR-INF/lib, courtesy of the new loader. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Welcome, new Committer Vincenzo
> I finally got my CVS Apache account Welcome :-) One of your first tasks should be to edit src/xdocs/weare.xml and move yourself to Committer status. You can also build the weare.html and commit that file. Please note: normally, a build will recreate all javadoc files. We generally don't update that portion of the site each time. [All CVS users please take note: we are very shortly going to change the CVS configuration considerably. The initial change will rename the CVS module. Subsequently, we'll be moving files into new modules. The web site, for example, will have its own CVS module.] --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Exception caught in RemoteDelivery.run(): java.lang.NullPointerException
Came through just fine. Good morning (and good night :-)). --- Noel -Original Message- From: Vincenzo Gianferrari Pini [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 18, 2003 2:51 To: James Developers List Subject: RE: Exception caught in RemoteDelivery.run(): java.lang.NullPointerException I did a cvs update on 2.2.0a6 that contains your update. If THIS reply gets through *as is* it means that the problem was fixed, as it was occurring to me when I was sending replies to this list (strange...). Vincenzo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Server statistics (was: Personal IP blacklists)
JMX is very conceptually compatible with the ad hoc approach Jason and I had discussed last summer. Remember what I said about using a statistics object and a registry? From the JMX specification: "The MBean server is a registry for MBeans [in the application.]" Very similar concepts. Not that I would claim uniqueness. SNMP, /proc, and other schemes are all in the same pattern. JMX is the right way to do this for James, since it is a standard, provides control, is embedded in Phoenix, is growing in use, and provides a replacement for the aging remote manager interface. > I imagine that the class doing the work would have a variable which > holds the statistic we care about. Classes. Plural. Lots of beans. :-) > This would cause SMTPHandler to refuse or defer some incoming > connections. Minor nit, but this is wrong. You need to understand the archtectural relationships between some of the components. The protocol handler is used after a new connection is established. Rejecting a new connection based upon busy state would happen elsewhere. As I've said before, please feel free to contribute. If you're interested in working on JMX for James, got for it. :-) Avalon has been using http://mx4j.sourceforge.net/. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Javadoc @version, @since and CVS $Id$
> as an old user of SourceSafe I'm used to share files among projects, > and I'm surprised to infer from your words that it is not a common > practice (or even not possible) to do it in the CVS world. Is it true? CVS cannot share files as SourceSafe does. Each branch has its own copy. I really liked SourceSafe when it was a One Tree Software product, and multi-platform, but I can't fault Brian & Company for selling out, considering what they got. One more reason for Subversion, which I believe supports sharing. > > Sun's convention is to use the SCCS version > > So the header could be: > > @version [CVS] $Id$ > > @since > Better then: > CVS $Id$ > @version CVS $Revision$ > @since $Revision$ is part of $Id$. As far as I'm concerned, it is a sufficient part, but that's not my personal call. I'm not sure if $Name$ will give you what you want. Danny or Serge might know. > two different .java files patched in the same moment for the same reason may have two > totally unrelated $Revision$ numbers. And I would like to see a reference to a shared > milestone of the project That's not the purpose of the @version tag. The purpose of the @version tag is to identify the code. You might want to get a copy of cvs2cl, which can produce nice change logs, and provide a good overview of related changes. > I see a relationship between @version and @since > http://java.sun.com/j2se/1.4.1/docs/tooldocs/windows/[EMAIL PROTECTED] > it contradicts the link above from this point of view Actually, not. If you go to the bottom of the section you referenced, they say "For more details, see writing @version tags", where they go on to elaborate that "The Java Software convention for the argument to the @version tag is the SCCS string." Using @version as: CVS $Id$ @version 3.0 @since 2.2.3 doesn't use the javadoc @version tag to hold the critical version information. @since is important to let outside developers know which version of the released product introduced the interface. @version is important to cross-reference a problem back into the source code control system. > And also notice that @since (and not @version) can be coded for a method Correct. @since tells you when something in the interface was introduced. @version tells you what version of that file you are looking at, so that you can match it up against the source control system. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [PATCH]Endless loop in JamesSpoolManager
Vincenzo,
//We got an error... send it to the requested processor
+if (!(e instanceof MessagingException)) {
+ //We got an error... send it to the error processor
+ mail.setState(Mail.ERROR);
+}
mail.setErrorMessage(e.getMessage());
Are you sure about this change? It is correct in terms of the previous one,
but it points out a lack in the prior patch: not all Exceptions will be
MessagingExceptions. Do we only want onException to catch
MessagingException instances, or do we want it to catch others?
--- Noel
-Original Message-
From: Vincenzo Gianferrari Pini
[mailto:[EMAIL PROTECTED]
Sent: Tuesday, June 17, 2003 8:08
To: List James-Dev
Subject: [PATCH]Endless loop in JamesSpoolManager
The new "on{Match|Mailet}Exception" support introduced in 2.2.0a5 was
causing an endless "config.xml" loop caused by JamesSpoolManager not setting
state to Mail.ERROR in case of an Exception not instance of
MessagingException being thrown by a Mailet or Matcher. In such situation
the message was resent to the current processor instead of going to
Mail.ERROR.
Vincenzo
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Exception caught in RemoteDelivery.run(): java.lang.NullPointerException
Try adding && cached.isSuccessful() to
cached = cache.addMessage(response);
if (cached != null) {
return processSetResponse(cached);
}
Let me know if that fixes it.
It will be nice to use Brian's code directly soon.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Exception caught in RemoteDelivery.run(): java.lang.NullPointerException
> if (!destroyed) log("Exception caught in RemoteDelivery.run(): " +
e);
> to:
> if (!destroyed) log("Exception caught in RemoteDelivery.run()", e);
Yes, please. Let's find out what is causing that NPE.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Exception caught in RemoteDelivery.run(): java.lang.NullPointerException
> Exception caught in RemoteDelivery.run(): java.lang.NullPointerException Can you provide a stack trace? I'm not spotting anything obvious in recent changes that would cause an NPE in run(), but I'll look more closely. > Strangely, It occurred to me three times while sending messages > *to this list* (I had to resend it), and only to another message > sent by someone else to a long list of recipients in different > domains I wonder if there is an error in the DNSServer change. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Server statistics (was: Personal IP blacklists)
Most of what you want is in the logs, just not bucketed for real-time querying. > total incoming messages/(unit time) INFO smtpserver: Successfully spooled mail from for [] > total incoming connections/(unit time) INFO smtpserver: Connection from () > total incoming bytes/(unit time) This we don't do at the moment. > total outgoing ... (the same three as above) Two of the three are already in RemoteDelivery. Only size is missing. > total volume of mail held in local mailboxes See the AbstractStorageQuotaMatcher code for how to do that. One could add a command to the remote manager to also return the figure for a mailbox. > total volume of mail held in local spools Same as above, since spools are currently mailboxes. That will change with v3. > ip addresses generating most of the traffic/(recent unit of time) Could be inferred from the logs if we logged size. > ip addresses generating most of the connections/(recent unit of time) Already in the logs. > local addresses receiving large quantities of mail Could be inferred from the logs if we logged size. > local addresses sending large quantities of mail Same as above. Again, most of this is already in the logs, except for size. What isn't present is a real-time statistics system, such as I laid out earlier, which you could view similarly to the /proc interface. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Personal IP blacklists
> I've already got the database open, so I'd rather keep all the relative > info there. You also have the DNS available, and I suspect that it might be quicker (actually, if I correctly recall my benchmarks from before, forget the "might" part). > And actually, in the back of my mind, I've been thinking about how to > make these personal blacklists sharable... kinda a napster-blacklist. DNS RBLs are sharable (obviously). > But anyway, what it did lead me to was the idea of having the blacklist > matcher point at the *message store*. As spam fills my spam folder, > when it was something I wanted to blacklist, I would copy it from my > spam store to my blacklist store. Then the matcher would look for any > messages from the sending IP address in this blacklist store Major performance issue. However, a service could periodically poll a message store, and populate a block list. That sort of service could resolve some other issues, e.g., another derivation of the same abstract service could periodically poll a "reprocess" folder, and insert those messages back into the queue. Take a look at FetchScheduler, and perhaps abstract from there. > This also preserves the evidence for why I blacklist them. Maintaining the spam/evidence database is certainly worth doing. But for publishing the block list, itself, I would suggest the DNS RBL approach. FWIW, we can fairly easily integrate block listing into the fast-fail code. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Server statistics (was: Personal IP blacklists)
This topic just came up, again, in james-user last week (http://archives.apache.org/eyebrowse/ReadMsg?listId=21&msgNo=6059), and is related to a discussion from last summer (http://archives.apache.org/eyebrowse/[EMAIL PROTECTED] apache.org&by=thread&from=225096). If you want to skip other discussion, you could go directly to http://archives.apache.org/eyebrowse/[EMAIL PROTECTED] he.org&msgNo=4076. Perhaps you, Chris, Steve, Jason and others would like to work on it? --- Noel -Original Message- From: Richard O. Hammer [mailto:[EMAIL PROTECTED] Sent: Monday, June 16, 2003 14:57 To: James Developers List Subject: Re: Personal IP blacklists My dream SMTP server has a GUI reminiscent of the control room at a power plant, with gauges which show every parameter which might be important to the operator, and with valves to throttle or completely shut down any segment of the operation which seems to be getting out of hand. Parameters such as the following would have both gauges (and/or warning lights and whistles) and values (operator-settable limits): total incoming messages/(unit time) total incoming connections/(unit time) total incoming bytes/(unit time) total outgoing ... (the same three as above) total volume of mail held in local mailboxes total volume of mail held in local spools ip addresses generating most of the traffic/(recent unit of time) ip addresses generating most of the connections/(recent unit of time) local addresses receiving large quantities of mail local addresses sending large quantities of mail Also, another approach, I would prefer to have the server return an error upon RCPT TO: , rather than accept the message as James does at present. I think Noel said there were plans to change this in an upcoming James. Rich - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: EL for matching
FWIW: http://jakarta.apache.org/commons/el.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: DNS problem?
> It would be pretty simple to define a basic logging object that provided > error/debug/warn/info/whatever If you define that as an interface, that would probably be sufficient (anyone who disagrees, PLEASE speak up). We could write an adapter to wrap your interface over our implementation. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Personal IP blacklists
Serge, It would be very simple to code a JDBC based block list matcher, but why don't you simply create a local DNS zone, and populate the zone file? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Javadoc @version, @since and CVS $Id$
> Then now I would put @version 3.0.0, unless the source is (again now) > totally shared with @version 2.2.0; in the latter case it should be > 2.2.0. There is a lot of shared source. If we were using something like SourceSafe instead of CVS, I suggest that quite a few files would be 100% common. In most cases, any differences are caused by one of a few factors: - changes to the Mailet API - package reshuffling (e.g., moving from james.utils to o.a.mailet) - interface updates for Avalon This is likely to change as we actually start work on James v3. > Current stable is v2, currently v2.2, first alpha would be 2.2a1 > first release candidate 2.2rc1 > Small fixes, eg bugfix, might be released as 2.2.1, larger > changes would result in 2.3 There is no magic, but I agree that the basic idea has been major.minor.patch, where a major change involves interface changes or substantial structural change. The minor version is incremented when the update is more functional than fix. The patch is incremented with each update to a minor release. We've not been using the "rc" designation so far, but that would be a good idea, especially when bumping anything other than the patch level. > "@version 2.2.0" matching the release version means in javadoc > "*last changed* in version 2.2.0", so we do *not* need to change > it for no reason when the version changes. That is not how the @version tag has been conventionally used, or was intended. Sun, and the tools vendors, have referred to it as the code version. Sun's convention is to use the SCCS version (http://java.sun.com/j2se/javadoc/writingdoccomments/[EMAIL PROTECTED]). Looking at the use of @version in the JDK, that is what you will see, and that is how I would suggest using it, e.g., @version CVS $ID$. It doesn't match against a release version, but it does identify the code version, which is the real point. The @since tag, on the other hand, refers to a product version, not a code version. So the header could be: @version [CVS] $Id$ @since The use of "CVS" is optional, but would clarify the intention for anyone who though that it referred to a release version. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: cvs commit: jakarta-james/src/java/org/apache/james/transport/mailets AbstractNotify.java AbstractRedirect.java Bounce.java
Vincenzo, Please check your setup. The patches for v3 changed the imports to the v2 packages. I have committed a fix to CVS. --- Noel Log: Fixed package names from latest patch. Index: AbstractNotify.java -import org.apache.james.util.RFC2822Headers; +import org.apache.mailet.RFC2822Headers; Index: AbstractRedirect.java -import org.apache.james.util.RFC2822Headers; -import org.apache.james.util.RFC822DateFormat; +import org.apache.mailet.RFC2822Headers; +import org.apache.mailet.dates.RFC822DateFormat; Index: Bounce.java -import org.apache.james.util.RFC2822Headers; +import org.apache.mailet.RFC2822Headers; - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: DNS problem?
Brian, Thanks for your response. I didn't spot code to eliminate TTL 0 records when I did my first pass looking at the code. If I get some time, I'll look again. We could modify the code to use the result from cache.addMessage. When the code was updated from dnsjava 1.2.3 to dnsjava 1.3.2, I don't believe that Serge noticed the change in cache handling. If we could use your code directly, that would be even better. That way we would not have to maintain the DNS lookup code. The problem is supporting logging. Our DNSServer.rawDNSLookup method, is pretty closely derived from the Lookup.lookup() method, and dns.lookup() before that, adapted for our logging needs. http://cvs.apache.org/viewcvs/jakarta-james/src/java/org/apache/james/dnsser ver/DNSServer.java Do you have a suggestion as to how we can address the logging situation? We can't use JDK 1.4 logging: there is no implementable interface, just a heavyweight logging object. However, we could easily use the Jakarta Commons Logging log interface: http://jakarta.apache.org/commons/logging/api/org/apache/commons/logging/Log .html --- Noel -Original Message- From: Brian Wellington [mailto:[EMAIL PROTECTED] Sent: Thursday, June 12, 2003 12:34 To: Noel J. Bergman Cc: James Users List Subject: RE: DNS problem? On Thu, 12 Jun 2003, Noel J. Bergman wrote: > Brian (on the BCC list), > > Would you care to comment on this? The code in James appears to be > essentially the same as your Lookup method, except for adaptations. > > The claim is that the algorithm checks the cache, finds nothing, does a > lookup, puts the record into the cache, and then queries the cache. But > since TTL is 0, the cache wouldn't have the record. > > >From what I see in the Cache.Element code, the cache should be valid for the > second within which the value is cached. Of course, if that happens at the > trailing edge, then the value will be missing, but then the process should > repeat, and should be valid. If the element was added to the cache, it would be valid for the remainder of the second. However, there's code to avoid putting 0 TTL elements in the cache, for the reason mentioned by Jeff - records with 0 TTL should not be cached at all. > Jeff, you should turn on DEBUG in environment.xml to see what gets logged. > Jeff, also to answer your question, the reason why the cache is used is that > the result of a lookup is a Message. The dnsjava Cache class has the code > for handling the set of records in that Message. The lookup code doesn't > replicate that code. It leverages the cache to do the work. Is there some reason you're not using the Lookup class? If you're not able to (and it's not easily fixable in a future version of dnsjava), I'd recommend using the same logic, where it uses the result of cache.addMessage instead of looking up the cache entry again; it was explicitly added to work with the 0 TTL case (as well as being a lot faster, since it avoided a cache lookup). Brian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: how hard is it to write a SMTP client?
> > Why would you need to write a replacement for JavaMail for
> > sending mail?
> I don't "need" to as much as I hanker to.
Have fun.
> But [JavaMail] seems a little less than user friendly, to the
> extent that I have used it.
I've coded simple SMTP transports by hand. JavaMail seems perfectly
reasonable. Assuming that you have a MimeMessage, the simplest code is:
try { Transport.send(msg); } catch (MessagingException _) {}
The basic msgsendsample.java example is small, and handles SMTP errors. But
it seems to me that if you try to accommodate all of the possibilities in
SMTP and MIME, you are going to end up with something quite close to
JavaMail.
> As I recall it requires a Session (which seems silly in the context of a
> MTA)
A Session carries context information. You need to keep it somewhere. It
has more of a role with other types of transports than SMTP.
> and it sets a message Header or two in ways which you just have
> to discover
This can be more of an issue, although not really with the SMTP transport.
The irony of your complexity argument is that JavaMail tries to keep things
simple, and sometimes does too much by default.
But getting back to your first point, if you just want to write an SMTP
transport for fun, have fun. :-)
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: [no virus] RE: [PATCH] New redirect
Applied to CVS. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Newsgroup admin/filtering (was: Bug 20746)
> It maybe be better to not have newsgroups in config.xml. Chages to > config.xml require restart and repository should know the set of > news groups in it. > - Remove newsgroups from config.xml > - Have commands in admin telnet session to add/remove newsgroups That's one way. Possibly the best choice. If you've time, please do submit a patch to the remote manager. This gets back to the question I had raised in January: ''If you delete newsgroups from config.xml, they are still present unless you delete the repositories as well. NNTPRepositoryImpl.getGroups() doesn't know to check the configuration for valid groups. We have the information to check against, but looking at the code for configure(), the variable known as "addGroups" ... someone knew and didn't document this feature. Is this behavior intentional, or should I fix it by filtering out newsgroups that are not listed in config.xml?'' ref: http://archives.apache.org/eyebrowse/[EMAIL PROTECTED] he.org&msgId=601424 I don't know what our users think the list of newsgroups is intended to do. Certainly, it is not scalable to list all newsgroups in config.xml, not to mention the issue that you raised. It has acted as an "add newsgroups", but I don't know if anyone wanted to use it as a filter. Considering the bug report, the author actually prefered the option to use it as a filter. I've completed the filtering mechanism that Peter started, and made it conditional. The default is false, since the previous semantic was just a non-intuitive "add newsgroup" operation. So far, from my telnet tests, it seems to be working. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: EL for matching
> > > As a starting point, is there any reason to expose more or less > > > than is currently exposed via the current Mailet/Matcher Java > > > interfaces? > > Not for doing normal Matcher/Mailet type of work, no. > My thinking too. A scripted Matcher/Mailet should be able to do > exactly what a Java coded Matcher/Mailet can do. Submit it. :-) Please note that until there are changes to the matcher configuration (see the v3 plans), the matcher is a bit of a problem (no configuration parameters). You could hack it with something like match="BSFMatcher=bsf://?". --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: EL for matching
> I've used BSF and Rhino extensively in my "real" work > I'm a big fan of it and would love to see someone write > a scriptable Mailet and Matcher, but IMO it would have > to have it's scriptable Objects carefully thought out, > and it's context closely defined to prevent abuse. Danny, do you have an outline for what you consider appropriate? Would it be a reasonable thing to start with BSFMatcher and BSFMailet classes that expose the underlying Mailet API? --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: EL for matching
> As a starting point, is there any reason to expose more or less > than is currently exposed via the current Mailet/Matcher Java > interfaces? Not for doing normal Matcher/Mailet type of work, no. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: EL for matching
> More prosiac I admit, but it would be a simple task to add a BSF (Bean > Scripting Framework)interface that supports the creation of matchers > and mailets written in any language which runs on a JVM and understands > BSF, see http://jakarta.apache.org/bsf/index.html. > BSF is out there and widely used. Worth considering? See: http://nagoya.apache.org/wiki/apachewiki.cgi?JamesV3/Plans Third item from the top. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: EL for matching
> I think BeanShell (or whatever the Java scripting language is) would be a > good first choice for a scripting language BSF is on the list for James v3. Feel free to work on it! :-) We could do a matcher, but for scripting languages like those supported by BSF, as well as Sieve and EL, we might want a custom processor. Sieve, for example, mixes both Matcher and Mailet characteristics. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [no virus] [PATCH]On matcher/mailet exception handler
> Thanks for all the patches! Absolutely! I hope to have time to wade through them soon. Right now they are all in my mailbox waiting for me to finish work on a server. Vincenzo, to answer your question, yes I had the HD on my laptop fail earlier this year. This time it was a SCSI drive on a public server. All told, we've have four drive failures in the past 18 months. From what the drive recovery folks told me when I was talking to them about my laptop drive, their business is way up. Drives are not nearly as reliable as they had been. Density is up, speed is up, but so is the failure rate. Fortunately, the new server has a six drive hot swappable RAID, and we've already purchased a spare drive for the cage. Its been an interesting week. Yesterday we had a T-storm frell our cable modem. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: On match/mailet exception handler (was in Separating internal errors from addressing errors in config.xml)
> I have in the past felt the need of having a richer way > to interact with matchers than just through the > condition string, using instead child elements as with > mailets. This has been a frequent area of discussion. For James v3, there will be a change. See http://nagoya.apache.org/wiki/apachewiki.cgi?JamesV3/MailetConfiguration for some proposals. You should be aware that there is little support for Proposal #2. Regardless, we would simply add an "onException" attribute to the matcher element. Also, a JNDI context could be used to provide information to matchers and mailets. FWIW, the example in proposal #1 shows multiple matchers. I'm actually not sure how Danny has that in mind to work. I don't know if they are parallel or serial, nor if the set operation (matchers don't return boolean, they return recipient sets) is a union or an intersection. > I agree on switching to attributes. :-) > I don't know how to access attributes. I can dig around the code > but if you could give me a hint ... ;-) Look in the initialize code for the spool manager, which is responsible for loading the configuration. There would need to be some internal changes between the spool manager and the processor. I probably can't help you much right now. I had a SCSI drive fail on a server, and dealing with that will probably consume me for a little while. I've got the server running again, but I need to get a new server into service. > I think that the feature we are discussing is totally transparent > and backwards compatible, as defaults to the current behaviour. I agree. But when we expose a new configuration option, we want to make sure that we do it in a way that we want to support for a while. :-) > Regarding what to do about per-recipient exceptions, instead of per-message > exceptions, I agree that we may handle it better in the future. Something to put on the TODO list. :-) > To avoid future backward compatibility in configuration files, we can [use] > matchException="[matchall|nomatch|error|aProcessorName]", > in the future > matchException="[match|matchall|nomatch|error|aProcessorName]" I don't believe that we need to make that change. The exception, itself, would provide that information. Depending upon the type of exception, it will either require matchall or it would provide information on individual exceptions. Please note that ERROR *is* a processor name. It is a required processor by that name. > please do commit for me now the patch to AbstractRedirect > as I am doing small things here and there, and I would > prefer not having a too big patch accumulating different > things. Under normal circustances, I'd be happy to do it. If I can, I will. But right now I have a server that I must attend to with some urgency. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: On match/mailet exception handler (was in Separating internal errors from addressing errors in config.xml)
Vincenzo, Please see my reply from a few minutes ago suggesting different syntax. With moving the configuration to an attribute on mailet, I don't have a problem with it. So far no comments from Danny, Serge or anyone else. We can operate on the idea of a "lazy consensus", which means it is OK unless there is an objection. However, there are two things I would take into account: - Changes to branch_2_1_fcs have to be carefully considered. That is our stable branch. We should make every effort to keep it entirely stable. - Changes to the public interfaces should also be carefully considered, since they are the hardest to change after people start using them. I have not committed your patches. You'll have your account shortly, and should be able to do so directly. :-) --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Separating internal errors from addressing errors in config.xml
> I'm not willing to control at the config.xml > entry level, but at the level. I'll write an > example. I would like to be able to write: > >heads >none >false >[REJECTED] >... >nomatch > I don't believe that I like that use of XML. The body of the mailet tag is for the mailet, not the matcher or the processor. As you noted, this would be "done with a few lines of code in LinearProcessor." There is also a proposed change to the DTD for James v3, which impacts this area. An alternative would be: making it an attribute of the mailet tag, instead of a child element. > if I had this possibility, I would like to have all matchers and > mailets catching and handling only the exceptions that they can > locally manage, and throw up (or not catch) things that they > cannot deal with. I agree. Still unresolved is what to do about per-recipient exceptions, instead of per-message exceptions. But I believe that we can extend that handling later by throwing a specific exception conveying the information, either by chaining or collection. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
James as a Mailing List Delivery Agent
I am moving this discussion to the James Developer List, since none of it is considered sensitive. Messages ordered from bottom (oldest) to top. Mark Imel, Jason Webb, Craig Mattson, and others interested in mailing list management with James please take note. Feedback, corrections, etc. are all actively solicited. Oh, but guys ... please do NOT reply with this message embedded. It is long enough as it is. :-) Just trim quotes, if any are needed at all. -- > Any realistic performance test has to account for the fact that > SMTP deliveries take anywhere from seconds to minutes depending > on network connections; the average for a fresh message seems to > be about 5 seconds, based on the fact that on daedalus qmail's > max concurrency is 509 and when delivering to a fresh list you > see 80-100 deliveries per second. That actually matters more > than any internal performance in terms of adding and removing > messages from the queue. I agree, which is why I wanted to clarify the situation. I am sorry for the initial confusion. FWIW, my normal load test generates about 1/2 the normal ASF load in individual outgoing messages, one per connection, although that is on a LAN. My current tests don't simulate the mailing list connection situation right now, but I'll put together a suitable test as James gets closer to being able to deliver on your needs. > Also, is the queue database transactional? Are we sure if the system > goes down that we won't lose mail? We may not be able to guarantee that the message won't be sent twice, if the system happens to go down at the wrong time, but the message is not removed from the queue until after it is confirmed to have been sent. > delivery can't be sequential to a list; when you're given a list of 100 > recipients, the mail server should be sending to those recipients in > parallel, not waiting for the first recipient delivery to finish before > beginning the next. If this is already addressed, great. We can tell James how many delivery threads to use. Yes, multiple threads would be used except where a message has multiple recipients at the same domain. The current code separates the recipients by domain, and spools a separate outgoing message per domain. In the case of VERP, each message delivery would be unique because of the MAIL FROM. However, that does not mean that we could not optimize message delivery. Doing VERP with the current code would mean a unique Mail spooled per receipient, each with its own unique sender. I believe that we can optimize that process. The link I usually use for VERP is http://cr.yp.to/proto/verp.txt. Unless I'm missing something, it could be done similarly to how we take a message in LocalDelivery, and then for each mailbox add a unique Delivered-To header to a clone of the message being streamed into the mailbox. For VERP, we would use a unique sender (an envelope attribute, not a Sender: header). > With VERP, the MAIL FROM line is different for every recipient, and > thus every recipient requires a separate SMTP connection. 100K AOL > users on a list? 100K connections. Yes, for each recipient we need a unique MAIL FROM. But both RFC 821 & 2821 permit more than one MAIL FROM per connection (for that matter, my mail client expects it). We wouldn't save anything on the data transfer, but why not reuse the connection, and save the connection establishment delay, when the remote MTA permits it and there are multiple mail messages to transfer? What am I missing? I've no ego investment in any particular idea; just mentally exploring what more we could do inside the delivery engine that would reduce transfer time and cycles. If I've missed something, e.g., if I had overlooked that VERP means that we cannot send one copy to AOL with list of RCPT TO commands, then I'm wrong. It has been known to happen. Occassionally. ;-) > Is there some web-based means for admins to cruise through the > pending delivery queue? Not a one. Something to do. What operations would you like to see supported? The probable solution will use JMX. --- Noel -Original Message- From: Noel J. Bergman [mailto:[EMAIL PROTECTED] Sent: Monday, June 09, 2003 2:22 To: Brian Behlendorf Cc: James-PMC Mailing List Subject: RE: James delivery volume Brian, Don't get too carried away by the performance. :-) A key point that you might not have caught was that I was using an SMTP sink, which means that instead of opening a connection per recipient, there was one connection per message. There would be a performance degradation opening connections per recepient. But from these few measurements, it does appear that the key issue is going to be optimizing those outgoing connections. Everything else appears to be well within the performance that James is capable of providing. For
RE: Separating internal errors from addressing errors in config.xml
> That could be per-message or per-recipient depending, again, > on the meaning of the exception. Actually, in the context of malformed messages, it would just be per-message. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Separating internal errors from addressing errors in config.xml
> better would be to give more control to the > administrator as said before. > Perhaps an additional useful thing would be to have a > kind of "MessageIsMalformed" matcher We could do that, but it would have an impact on performance. And not every matcher cares; only those that do process body content. Depending upon the meaning of the problem, one might want to: - decide that malformed messages do/don't match - send them to a separate processor - detect and correct common exceptions as necessary to perform the match That could be per-message or per-recipient depending, again, on the meaning of the exception. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Separating internal errors from addressing errors in config.xml
Basically, matchers and mailets ought to do the best that they can. If they can handle an exception, they should. If the exception can't be reasonably handled, the current expectation is for the mailet to set the state to ERROR, and to indicate why. Right now that latter isn't happening except when LinearProcessor catches an MessagingException thrown from a matcher or mailet. You seem to be proposing that the matcher or mailet could throw it back to the processor. I don't have a problem with the idea of adding an attribute to the processor level to control what to do if a matcher or mailet throws an exception. But we should think it through first. I've also thought to add a class attribute, so that we can have custom processor types. You've identified several options: - matchException=match - matchException=nomatch - [mailet|match]Exception= The default would be the "error" processor. On the surface, it seems reasonable, but there are issues to consider. For example, if I were to have: error what do you propose should be done? The case I provided is an explicit case, but there are also implicit cases. Another issue to consider: what about where there is an exception effecting a single recipient, or individual sets of recipients? Do we require Matchers and Mailets to build and throw a chain of MessagingException objects? Do we need an exception class similar to SendFailedException, with collections of which ones failed and which ones succeeded? Is there another way? For example, Serge pointed out that although it isn't supported, the intent of the Matcher was to be able to return a collection of collections. Perhaps Serge could discuss that again, and we could consider how that might be used to address exceptions. Please note that for James v2, we don't want to do anything that breaks any third party code. For James v3, we would like not to, but could if necessary, and by using the class attribute of the processor element, we could have a processor compatible with James v2 matchers/mailets if the interface conventions changed. I'm not saying that we should preserve the status quo, but am pointing out some of the issues that need to be addressed. Above all else, we need to keep in mind that the paramount obligation of a mail server is reliable delivery. It can try to filter, etc., but it should always operate in such fashion as to ensure that mail is not lost. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [no virus] [PATCH] AttachmentFileNameIs catches all exceptions
Based upon the current discussions, perhaps we should defer this patch? I'll also leave it for you to apply when you get your account next week. --- Noel -Original Message- From: Vincenzo Gianferrari Pini [mailto:[EMAIL PROTECTED] Sent: Saturday, June 07, 2003 16:54 To: List James-Dev Subject: [no virus] [PATCH] AttachmentFileNameIs catches all exceptions AttachmentFileNameIs was throwing a "java.io.IOException: Unknown encoding: xxx" and other exceptions when processing malformed messages. Now catches any Throwable, logs it and returns null (no match). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Separating internal errors from addressing errors in config.xml
> when I started using James I changed the default > configuration in this way: > -error processor is used exclusively for errors (i.e. exceptions catched) > and nothing else. It forwards these emails to the postmaster, and not to > the sender. It also stores the mail in the error storage, just to be safe Same here. This is my error processor: file://var/mail/error/ There are no references to it in config.xml. It is used only for internal errors. I have separate processors for spam, relay-denied, and address-error: [EMAIL PROTECTED] file://var/mail/spam/ file://var/mail/relay-denied/ none file://var/mail/address-error/ which are invoked as appropriate: local-address-error 550 - Requested action not taken: no such user here relay-denied 550 - Requested action not taken: relaying denied > I think the default James configuration should be changed in a similar way. > Postmaster must be nitified if unexpected things happen, sender must be > notified if his mail was not accepted for a specific, known reason. I agree. What changes (other than commenting) would you propose to the above? I am not proposing it as the right way, just as a stake in the ground. With respect to exception handling, I don't believe that we need to complicate the lives of administrators by giving them control over the outcome of each thrown exception. Exceptions should be considered in terms of their semantic meaning. In many cases, there is appropriate behavior, and no need to expose it to the administrator. In many cases, it may not even make sense to log it, other than at debug level. Yes, there will be exceptions that should result in sending a message to the ERROR spool, but that will probably not be the norm, IMO. As for Runtime Exceptions, they are a constant source of errors, and should rarely be used. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Exception in AbstractRedirect
I see the problem. The orginal code preserved the MessageID any time the inline type is UNALTERED. The revision did it only when passthrough was also true. Thanks for catching that, Vincenzo. :-) --- Noel -Original Message- From: Vincenzo Gianferrari Pini [mailto:[EMAIL PROTECTED] Sent: Sunday, June 08, 2003 15:18 To: James Developers List Subject: RE: Exception in AbstractRedirect Noel, After testing a bit, I think that your fix "Fix in AbstractRedirect to avoid re-rendering a message that we aren't changing" to AbstractRedirect on 2003/06/06 16:55:05 and related to this posting, is no longer preserving the message id. Vincenzo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBCSpoolRepository & lock/unlock
> is the lock and unlock method invocation in JDBCSpoolRepository really needed? > If getNextPendingMessage() in eg accept() is called, the Message > is removed from the LinkedList [=pendingMessages] with > .removeFirst(), so another Thread couldn't get this removed Message. What prevents the message from being put back on the list by loadPendingMessages, if the message is still in the spool? :-) --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Separating internal errors from addressing errors in config.xml
James.Mailet: Error while storing mail. log-java.lang.RuntimeException: Exception caught while storing mail Container: java.sql.SQLException: Communication link failure: java.net.SocketException -at mailrepository.JDBCMailRepository.store(JDBCMailRepository.java:650) -at James.storeMail(James.java:856) ... Very rare (the above is the only occurance of it this year), but when this happens, the message is sent to the error processor. That is also where address errors are sent. My suggestion is that we separate server errors and address errors in the default configuration. Address errors are something that you might want to bounce, but you might not want to bounce the above, since it is something that can be fixed locally. We'd want a Bounce processor to be able to report one as a 5xx, the other a 4xx, assuming a bounce was configured for either. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: how hard is it to write a SMTP client?
Chris, You could store the message in a repository, and send a notification message with a special URL (subclass of NotifySender). The URL could be used to initiate payment. Once payment is received, another message would cause the message to move from the repository back to the spool for delivery. Your idea of pre-payment, and a quota matcher that deducts payment until there is no credit left is an interesting idea. Mind you, I doubt that there is really any spammer who wants to pay, so I'm curious to see if that would *ever* work socially. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
CVS Changes
Basically committing changes people had posted that I've been running this week. I'm going to post a test build, and then I'm hoping for a nice quite weekend, unless there are any reported problems in the test build. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
New AbstractNetworkMatcher
I am about to check in a new AbstractNetworkMatcher that uses the same
NetMatcher class used by the SMTP Handler for fast fail.
RemoteAddrNotInNetwork and RemoteAddrInNetwork now extend
AbstractNetworkMatcher. Other types of network matchers are faciliated.
The RemoteAddrNotInNetwork code demonstrates how trivial it is to code a
network matcher:
public class RemoteAddrNotInNetwork extends AbstractNetworkMatcher {
public Collection match(Mail mail) {
return matchNetwork(mail.getRemoteAddr()) ? null :
mail.getRecipients();
}
}
The default behavior of AbstractNetworkMatcher is to build the network
matcher from the matcher condition. It could be subclassed, e.g., to load
the list from a file.
I've no current plans for any other subclasses, but the code is there to be
reused.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Understanding the SMTPHandler Data method
> I have totally different objectives from JAMES. What are your objectives? :-) > So I'm trying to understand how SMTPHandler: doData works. > I've tried to do a little refactoring FWIW, see a message I posted this morning for more information on how the handlers may change in the future. > how does the doDATA and its collaborating streams > work? The different streams add behavior to the pipe, as detailed below: > in = new BufferedInputStream(socket.getInputStream(), 1024); This is the raw input stream. > InputStream msgIn = new CharTerminatedInputStream(in,SMTPTerminator); The CharTerminatedInputStream is an InputStream that reads until it recognizes the particular pattern of data specified. In this case, it is looking for CRLF.CRLF. > msgIn = new SizeLimitedInputStream(msgIn, >smtpProtocol.getPropertyLong(MAX_MSG_SIZE)); This imposes a limit on the amount of data that will be allowed through the stream. > msgIn = new DotStuffingInputStream(msgIn); This decodes dot stuffing. > Mail mail = Mail.create(msgIn); This is a change to the Mailet API, not the James server implementation. We do not want to see the Mailet API balkanized. Do what you want with the org.apache.james.* packages in your fork. In the above case, you could do: Mail mail = MailImpl.create(msgIn); and no one would care. If you do feel that it is necessary to change the Mailet Specification, please bring up your proposals for discussion. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: fix for bug 20370
Yes, it is obvious. :-) But I doubt that any major (or perhaps even minor) clients will have an issue. That sort of thing is usually violated only by someone's hacked script or using telnet. By the way, take a look at http://james.apache.org/weare.html. :-) --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Logging in the new DNSServer service
> If there is no MX record, A record represents mail host.
> It is not an error to have no MX Records and neither is
> it necessary to recieve mail.
Correct. The code handles that. The only issue was the log level.
> if ( collection.size() == 0 ) {
>try {
>InetAddress.getByName(hostname);
>} catch (UnknownHostException ex) {
>// there is no A record.
>collection.add(hostname);
>}
> }
That is almost what is in the code, except you have the add in the wrong
place. You are adding the host name if there is no A RR. You want to add it
if there IS an A record. :-)
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Bug 18471 missing attributes on mail
> Just started working on the changes for the JdbcMailRepository
> (I hope late is better than never)
Absolutely. :-)
> public String getSqlString(String name, boolean required)
> String sql = getSqlString(name);
> if ( sql == null ) {
Fixed.
> I have proposed a set of matchers/mailets which operates
> on mail attributes.
Probably one patch for existing matchers/mailets, and one for new ones.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: james docs wrong?
Docs are clearly wrong: maxMessageSize = handlerConfiguration.getChild( "maxmessagesize" ).getValueAsLong( maxMessageSize ) * 1024; I have had the max message size set to 3072 on my server. --- Noel > -Original Message- > From: Adam Neilson [mailto:[EMAIL PROTECTED] > Sent: 05 June 2003 15:17 > To: Danny Angus > Subject: james docs wrong? > > > james smtp documentation states: > > "maxmessagesize - This is an optional tag with a non-negative integer > body. It specifies the maximum size, in bytes, of any message that > will be transmitted by this SMTP server. It is a service-wide, as > opposed to a per user, limit. If the value is zero then there is no > limit. If the tag isn't specified, the service will default to an > unlimited message size" > > so i set the max size to 12582900 (for 12 meg) > > but when starting james, the smtp log says: > > 05/06/03 15:10:02 INFO smtpserver: The maximum allowed message size is > 12884889600 bytes which is 12 gig! > > should the doc say KB instead? > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
