RE: Stream prob in SMTP handler

[Peter M. Goldstein]

Noel,

> > It's fairly simple to beef up the suggested change to make it compatible
> > with the pipelining stuff.  As the pipelining RFC discusses, only a few
> > commands (for our purposes RSET, MAIL, and RCPT) can actually be placed
> in
> > the middle of a pipeline (section 4.1).   This RFC doesn't allow the
> DATA
> > command (or EHLO, NOOP, or QUIT) to be in the middle of a pipeline,
> > basically for the reasons we've already gone over here.
> 
> I made my objection clear.  I made a specific objection based upon an RFC.
> What I wanted was a revised solution that would support the RFC *and*
> address the issue.

You did indeed.  That's why it was easy to answer it.  :)  I think the
suggestion I provided should both support the RFC and address the issue.  Do
you see a problem with it?

> From what I can see, this is how a technical objection and conflict
> resolution *should* work.

Yep.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Stream prob in SMTP handler

[Peter M. Goldstein]

Noel,

> Please review the RFC's I mentioned earlier that deal with command
> pipelining.  I believe, from a cursory review, that your solution would
> break them.  I'm interested in SUPPORTING those RFCs when feasible.
> 
> I will object to any change (from anyone) that *prevents* an RFC from
> being
> implemented, even an elective one.

It's fairly simple to beef up the suggested change to make it compatible
with the pipelining stuff.  As the pipelining RFC discusses, only a few
commands (for our purposes RSET, MAIL, and RCPT) can actually be placed in
the middle of a pipeline (section 4.1).   This RFC doesn't allow the DATA
command (or EHLO, NOOP, or QUIT) to be in the middle of a pipeline,
basically for the reasons we've already gone over here.

So you put the isReady() check on all commands except RSET, MAIL, and RCPT
and you can support RFC 2197 while garnering all the benefits of "fail-fast"
command processing.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Stream prob in SMTP handler

[Peter M. Goldstein]

Serge,

> Yes, that is a nice way to do it... you can call inReader().ready(), and
> if true, tell them to go away with a 5xx.

Ok, I find this far more acceptable than the alternative of processing the
message.

> > I'd also argue that there are a number of additional, command level
> parsing
> > mechanisms that would be far more effective at dealing with this problem
> > than the suggested alternative of accepting the message.  For example,
> > imposing line length limits.  Or imposing limits on the number of
> malformed
> > commands that are allowed before the connection is closed.  Both of
> these
> > would be "fail-fast" techniques that would save the James server
> bandwidth
> > and minimize the number of open connections.  They are also compatible
> with
> > the RFC and don't encourage standard-violating clients.
> 
> Yes, the fast-fail might be nice.  I would prefer though if we could
> leave this decision in the hands of the admin rather than us.

I've got no problem making this configurable.  Not too hard to do.  Two
additional parameters (max command line length, max # of bad commands) for
the SMTP server.

> > As far as reactive action, I'm not sure what you've got in mind.  Let us
> > imagine that, in fact, this data is all read in to the appropriate
> buffers
> > and the commands are processed.  It's going to be stuck on the spool and
> > processed.  Now you've seen it - it's in your spam repository or
> something.
> > What're you going to do?  I mean I guess you could globally block
> > connections from the server that sent it, but if this is really a
> problem
> > you've got the IP address of the server in either case.  What does
> > processing and storing the message get you?
> 
> If you can receive and process the message, then you can determine if
> it's spam or not (based on relay rules, Bayesian filter, whatever).  If
> it is spam, there are a wide range of options for you...
> - You can log who was spamming you in case you want to brag to people
> about how stupid spammers are, or more seriously report on the # of
> attempts spam.
> - You can generate an email to the ISP saying someone from this IP
> address at these times sent 300 pieces of spam to you (which gives them
> enough information to close that account)
> - You can get the IP address blacklisted on one of the various lists.
> - Assuming you determined it was spam from a relaying rule, you can add
> it to your Bayesian filter to learn more patterns of spam.
> - If you're in certain states or countries, you can send the spammer an
> invoice for sending you spam.
> - You can charge certain customers more money for extra spam blocking
> tools.
> 
> Anyway, just some thoughts... so I plan to implement, some I don't.

I guess.  I mean you've already got most of this logged (IP, etc.).  The
Bayesian point is reasonable, as is the invoice issue.

But there seems to be a piece of circular logic here.  We still don't have a
single solitary example of a valid client manifesting this behavior, much
less a widely used one.  Certainly no general incompatibilities have been
reported.  The only assertion is that this is behavior manifested by some
spammers.  So why do we need to determine whether the message is spam?  It
seems that 

In this case just fail it immediately.  Certainly seems like a more
desirable solution that wasting bandwidth, processing power, disk space,
etc. on reading in a message that we're assuming a priori to be spam.

> We could also add a header to the message stating whether this message
> violated SMTP by having data in the stream already when we received DATA.

Uh...why?  Again, there isn't a single documented client that demonstrates
this behavior.  What's the motivation for adding this?  Simply amounts to
one more piece of state that we'd have to manage internally.
 
> This is why I think it's a DoS opportunity.
> a) the connection is kept open for a long time... it gives the attacker
> a chance to load up multiple connections.
> b) the buffer can't be disposed while the connection is open... if you
> fast-failed or accepted the message, you could take those bytes, stick
> them where you want (maybe /dev/null), and then recycle that memory.

Again, I still don't get it.  Keeping the connection open for a long time is
no big trick (see trivial example below, comments in previous email).

The buffer to which you're referring is presumably the underlying TCP/IP
socket buffer, since there is no buffer growth in the Java code.  The only
buffer in the Java code is the one in BufferedInputStream, which is of a
constant size as long as the connection is open.  And as I said above,
keeping the connection open is no big trick.  But even the TCP/IP buffer is
size limited.  So what buffer is necessarily going to be cleaned up in this
case?
 
> Well, sure, I'll grant you can overload someone's bandwidth with
> anything, and SMTP makes it pretty easy... heck you can PING someone to
> death.
> 
> But right now you can write

RE: Stream prob in SMTP handler

[Peter M. Goldstein]

Aaron,

> *Interoperability Rule Number One:* Be strict in what you send and
> lenient in what you accept.

Ok.  I don't agree.  :)

It's partially this sort of thinking that had led to huge chunks of modern
browsers being devoted to parsing invalid HTML.  

My proposed counter rule is this:

Be strict in what you send and knowledgeable about what you accept.

That is, it's ok to deviate from specification when you know why you're
deviating.  If you're just deviating to accept random input, then all you do
is weaken the specification.  Clients (or servers) that don't follow the
specification have no real reason to improve, and bugs in those clients and
servers are harder to catch when they're deployed.

Moreover, design and architectural decisions for the server become far more
wishy-washy, because now you've got to accept random deviations that may or
may not be important.  There's no documentation of exactly which problems
are real, and which were dreamed up by developers making up test cases.  The
software becomes brittle.  Badly so, as there are constraints on the
software that are ill-defined and probably poorly understood by the group.

> Peter M. Goldstein wrote:
> 
> > It makes it possible to write applets that are in
> > violation of spec.  Which is a good reason to not
> > support the "functionality".
> 
> 
> IMHO, this view is incorrect.  It is not the business of the JAMES
> development community to attempt to be the RFC police of the SMTP world.
>If people want to write cheap hacks to meet a goal, who are we to say
> (without any knowledge of the circumstances) that their particular
> situation does not warrant the tradeoff?

Because that's what standards are.  They are specifically designed to draw
this line.  That's also why most RFCs explicitly discuss the leniency vs.
strictness issues.  They detail the areas where commonly used clients are in
violation.  And they discuss some of the tradeoffs involved in supporting
those clients.

As far as being the RFC police, that's just hyperbole.  If someone has a
client that has some incompatibility with James, they can bring up the
specific client/server that is having the issue.  Again, there's no problem
making a specific deviation for a specific client.  What I object to is
arbitrary deviations without cause.

> The goal of any development team should be to make their app as useful
> and robust as possible.  Robustness is about dealing with errors in such
> a way that service is still provided in the most adverse conditions
> possible.  This includes resource shortages/outages and also spec
> violations.

Sorry, I disagree with this definition.  The goal of a robust app is to
provide the service that has been contracted.  I certainly agree that
dealing with errors is extremely important - I simply don't agree that
dealing with errors means that you try and figure out under arbitrary
circumstances what the client wants and go do it.

Taken to its logical extreme, it implies chaos.  For example, let's say a
client sends out of order MAIL, RCPT, and DATA commands.  Shouldn't we be
handling that by sending the message?  I mean all the necessary information
has been provided.  So why not?  Doesn't my unhappy client have a right to
complain?

 
> There are a large number of widely used mail clients and servers out
> there that are in violation of spec (though maybe not this particular
> issue).  If we carry this attitude when dealing with them, then JAMES
> will have problems talking to them.  This will make JAMES less useful.

Ok.  Find me one widely-used application that has this problem.  Adding the
ability to deal with specific violations of the RFC on a case by case basis
is valuable.  You can make rational judgments and tradeoffs, potentially
communicate with the development group behind the spec violating client, 

> I think that Serge's issue should be fixed, assuming that it can be done
> without putting us in violation of the spec ourselves.  (Post 2.1
> release, of course.)

See my other emails about my opinion as to the correct way to resolve this
issue.

--Peter



--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

RE: Stream prob in SMTP handler

[Peter M. Goldstein]

Serge,

Now to respond to the technical points:

> Spammers will send email this way, and I'd rather not have their already
> wasteful emails leave me with a bunch of open connections.  I'd prefer
> to close the connection and be done with the transaction.  As is, my
> mail server will receive all the TCP data, it sits in some odd buffer,
> sit with an open connection until it times out, and then the data gets
> dumped and I don't even get a chance to log and do whatever else to make
> sure I can catch more spam in the future.

I'm not sure I buy the first assertion, especially when I've never seen a
spammer send mail that way.  Spammers have to pay for bandwidth just like
everybody else, and in general it's in their best interest to confirm that a
server is an open relay before wasting their time and bandwidth sending
arbitrary mail to be relayed through the server.

But let us grant that some spammers do send email this way.  IMO the logical
way to handle this situation is to require more strict compliance from the
clients, not less.  This means that when a command is read off the wire, it
would be useful to confirm that no additional data is pending.  If it is, a
5xx is immediately returned informing the client that they've sent a
malformed command.  I've got no real objection to that, provided it doesn't
break any of the standard clients.

I'd also argue that there are a number of additional, command level parsing
mechanisms that would be far more effective at dealing with this problem
than the suggested alternative of accepting the message.  For example,
imposing line length limits.  Or imposing limits on the number of malformed
commands that are allowed before the connection is closed.  Both of these
would be "fail-fast" techniques that would save the James server bandwidth
and minimize the number of open connections.  They are also compatible with
the RFC and don't encourage standard-violating clients. 

As far as reactive action, I'm not sure what you've got in mind.  Let us
imagine that, in fact, this data is all read in to the appropriate buffers
and the commands are processed.  It's going to be stuck on the spool and
processed.  Now you've seen it - it's in your spam repository or something.
What're you going to do?  I mean I guess you could globally block
connections from the server that sent it, but if this is really a problem
you've got the IP address of the server in either case.  What does
processing and storing the message get you?

> Anyway, this also smells like a great DOS opportunity.

Honestly, I don't see how this is any kind of DoS opportunity.  At least not
one above and beyond those provided by the protocol itself.

Fundamentally, SMTP is not a particularly DoS resistant protocol.  That's
just how it is.  If I want to kill an SMTP server, all I have to do is open
up enough connections and send NOOPs, RSETs, or EHLOs over each of them,
over and over and over again.  Since endless loops of these commands are
permitted by the SMTP protocol grammar, there's basically nothing to be done
about that.

The problem is even worse, since James doesn't have any of the data
filtering mechanisms described above.  So I can attach a base64 encoded
version of my favorite MP3 and send it along as an argument to RSET.  I'll
get a 5xx, but it will tie up bandwidth and resources while James is
processing.

So James is vulnerable to DoS attacks, but that's basically because it's a
mail server.  We could harden it a bit (using some or all of the techniques
I describe above) but I don't see how DoS is relevant to the issue under
discussion.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Stream prob in SMTP handler

[Peter M. Goldstein]

Serge,

> And , I'm sorry I snapped at you Peter.
>  I feel I have a 
> lot of experience with SMTP having built the
> original and watched James 
> run in production for 4 years, so I was upset that
> my bug report was so 
> quickly categorized as invalid without a discussion.
>  As you accurately 
> pointed out, I am upset because we butted heads on
> the pop3-file issue. 
>   I felt you categorizing this report as invalid was
> linked to that 
> discussion, so I escalated rather than dealing with
> it like an adult.

It wasn't.  If you honestly feel there are other
issues aside from making invalid test cases pass, then
those should be put in the bug report.  I may not
agree with you, but on the basis of the bug report as
written (my invalid test cases fail) I still feel that
closing it was appropriate.  As Noel points out, if
you feel there are other issues outstanding
(vulnerability to DOS attacks, etc.) you can simply
reopen the bug and add the relevant details.  I may
still disagree with you, but I'm happy to discuss.

In any case, apology accepted.

--Peter


__
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com

--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Stream prob in SMTP handler

[Peter M. Goldstein]

Hontvari,

> This feature also makes possible to write simple and
> small email sender
> function, which is useful in applets.

Actually, it doesn't.

It makes it possible to write applets that are in
violation of spec.  Which is a good reason to not
support the "functionality".

Consider the following situation.  The James code is
modified so that, rather than a 354, a 5xx code is
returned some fraction (say 1/2) of the time.  This
would be weird, but totally legal behavior for a
server.  This simulates a heavy loaded server where
resources are often unavailable.  Consider one of
several servers in use by an ISP.

Now run your mail applet/test.  Does it behave in
accordance with spec?  No, of course not.  And thus
the state of the interaction is indeterminate (an
extremely strict server would simply cut off the
client, while a lax server might process the data). 
Very bad.  Your applet is tightly coupled to spec
violating behavior in the server.

--Peter






__
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com

--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Stream prob in SMTP handler

[Peter M. Goldstein]

David,

> > I know it's unkosher to be filling the buffer before you've
> > received DATA, but I know sendmail, qmail, exchange, and probably others
> > handle this ok.  I'll file this in bugzilla.
> 
> It's not only unkosher, it's illegal.  See RFC 2821 section 4.3.1
> (http://www.ietf.org/rfc/rfc2821.txt) and RFC 2920
> (http://www.ietf.org/rfc/rfc2920.txt).  Those rules are important to me as
> someone who intends to implement SMTP with nonblocking IO.  There's
> nothing
> wrong with servers that happen to accept the illegal input, but clients
> "MUST NOT" send it.

:)  See my comment on the bug.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: There is no active development on James

[Peter M. Goldstein]

Serge,

> "The Apache Way" is about trust and respect... Greg is someone who's
> trusted
> a lot of people, and I think has earned a huge amount of respect.  This
> isn't quaking in fear... it's a response of... my gosh, I'm only working
> on
> my one project and have no idea how much it's costing those people in
> California to support my CVS account and emails and downloads... he asks
> that we start considering this, and so we listen.

I couldn't agree more.  It is about respect.  Respect for disparate points
of view.  Something that seems to be lacking here.

As my last email made clear, I did consider the issue.  I find this
assumption that any view which dissents from the current group-think shows
no consideration to be disturbing.  It certainly isn't the way to do good
software development.  Or good community building.

I considered the issue not simply in response to Greg's email, but in
response to Nicola's original post.  I responded to that email with my
belief that the tradeoff wasn't in the best interest of the project.
Mirroring was not objected to on principle, but rather as being yet another
thing that would push back the release.  When it became clear that Noel
wanted to pick this up and that he'd still get the marketing stuff done, I
let it drop.  If Noel manages to get his questions answered by the Apache
infrastructure people, then mirroring will be part of the 2.1 release cycle.
If he can't (which would indicate to me that it's not exactly a priority
with them), then we'll consider the issue.  In that case I'd push for a
release w/o mirroring.  What, exactly, is the problem here?

> Also the open dialogue quickly ends with a -1 vote.  Because of the Apache
> voting rules, when someone votes -1, the conversation is just about over.
> We can lobby the vetoer, but more than likely whatever brought that person
> to veto will not likely change.  And even if the person was open to
> change,
> the lobbying is draining to the community, the lobby'ers, and the vetoer.
> Perhaps you don't appreciate how negative your voting is appearing to the
> community.

Perhaps it is.  It is certainly not my intention to damage the community.
But as I've said before, I only have my professional judgment to rely on.
In my view it is equally (if not more so) damaging to make bad decisions and
have to deal with the consequences.  So I have (and continue to) express my
opinion as to the best options at assorted decision points.

On some occasions I stick to my guns, because I believe that it's a crucial
issue.  On other occasions I have gone along with opinions that differ
substantially from my own because I believe that they won't be horribly
detrimental.  On still others I've tried to emphasize the issues that I find
most relevant, and find a third path that addresses both those and the
issues put forward by the other developers.  That's how it works.

As far as the open dialogue ending with a '-1' vote, I'll have to disagree.
I certainly haven't felt restrained from debating with '-1' votes with which
I disagree.  And I wouldn't expect others to fail to lobby me when they
disagree with my vote.  My observation (on this and other lists) has been
that the vote is rarely the end of the story.

As far as this system being "draining", I don't see what your alternative
is.  It's a voting system, designed to allow disparate points of view to be
debated.  Sometimes it leads to gridlock.  Most of the time it doesn't.  I
don't see any other way to do it.  To me the choice seems to be unproductive
lassitude or some degree of conflict.  I'm all for minimizing the latter,
provided we don't accept too much of the former.
 
> Anyway, I'm done lobbying for you to change your veta of the pop3-
> filestore
> bug.  You have the right to block any bug fixes and push the release out.
> So be it.

Ok.  As I said in my recent email, your demeanor of "hurt victim" here
baffles me.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: There is no active development on James

[Peter M. Goldstein]

Serge,

> We have a release process... we voted on a release plan, and the code is
> now
> frozen.  If we see a bug, we raise it, discuss it, and vote whether we
> should fix it.  You don't like this process and want to use your own, and
> I
> can't get around your veto.

Excuse me, but this is exactly the process we discussed and approved.
Pre-existing bugs were evaluated and judged.  We had an explicit discussion
about this bug.

New bugs are raised, discussed and submitted for evaluation by their
discoverers/committers.  Some have been approved (the Oracle resource fix)
some were held off on (the deluser problem).  Opportunities for discussion
were always there.

There is no process of "my own".  Your implication is erroneous.

We, not me, decided to put it off.  Now you're asking, at the 11th hour, to
revisit the decision to put it off.  Ok, let's agree that you have the right
to make that request.  In fact, you've already made that request.  What
you've really got a problem with is that I disagree with you.

You're annoyed that I don't agree with you that it should be reversed.  In
fact, it's even worse that that, because I'm advocating a third way that
would allow us to support the user base without further delaying the
release.  So I'm willing to grant that the bug should be fixed, and your
problem is that I feel that getting 2.1 out the door is a value above and
beyond fixing this bug right now.  And why exactly does this make me a bad,
bad man?

I'm not going to try and vilify you here in the interest of winning points.
I believe that you believe that your proposal is the best path for James.  I
just don't agree.  Why you find that so inconceivable is beyond me. 
 
> You're just not making very many friends by changing rules and -1
> everything
> you don't want to work on.  I voted to support the 2.1 release plan and
> code
> freeze without new rules being introduced.

No rules were changed, nothing was altered.  See the above.

And I'm certainly not "-1 everything I don't want to work on".  What I am
doing is tightly gating the 2.1 release to try and move it out the door.  

> Anyway, I was very supportive of someone bringing rigor to our testing,
> documentation, and release process.  I'll admit I'm upset this moment, but
> I'm going to find it difficult to +1 to any more of your release plans
> with
> the way the past month has gone.

Ok.  Quite honestly, given the failure of anyone to actually contribute to
the release plan, I doubt I'll be proferring any new ones up.  This has, as
may be apparent to you at this point, a tremendously disheartening
experience.  I find it sad that the developers on this list (for the most
part) cannot muster up a few hours over the course of months to advance the
pieces of the project that, while they might not be the center of their
interest, nonetheless make the product more appealing to the end users and
hence benefit the whole.  That, while many may talk a good game, few seem
willing to put in the actual effort.  Certainly it's helped undermine my
belief in the effectiveness of open source development.

> And finally, this is a simple bug and I'm not going to sit here and talk
> for
> days about how to go about this change... I said this is a quick fix,
> maybe
> a bit more to support backwards compatibility.  I think you're being
> clearly
> beligerent about the QA issue... pop3 file system DOES NOT WORK!!!  To
> your
> comment of it being usable as a simple test configuration... who downloads
> a
> mail server for a simple test configuration?  By comparison I suppose you
> would say if the SMTP handler couldn't support remote connections, that's
> acceptable?

I've already explained my view of this bug.  You don't like it, that's fine.
You disagree with it, that's fine.  That's your prerogative.  Feel free to
try and convince me otherwise.  Or don't.  But this trivialization does you
little credit.

There is certainly no need for you to spend days talking about the change.
Go off and fix it if you want.  As I'm trying to make crystal clear, there
is absolutely nothing stopping you from doing so.

Speaking as probably the only committer who does run the file repository on
any system (I use it for one of my low traffic domains, and also for testing
file repository issues in James) I can confirm that the bug behaves exactly
as I stated.  Provided the server doesn't restart (inducing renumbering),
things work ok.  My server has been running for weeks without a problem.
It's a robustness issue and, as I granted, a nasty one.  But there is a work
around, which to me means it isn't critical.  Sorry you disagree.

As far as the above analogy, it is false.  Your comparison is false
precisely because if the SMTP handler didn't accept remote connections,
there would be no workarounds.  False analogies rarely advance arguments.
I'm not sure why you don't see the difference.

I certainly don't feel I'm being belligerent about the QA issue - what I'm
doing is illustrat

RE: There is no active development on James

[Peter M. Goldstein]

Danny,

> Well lets get it released then, IMO the code freeze has gone on for too
> long
> already.

Well, hey, do you think James 2.1 might be ready to go right now (or a month
ago) if any of the folks here aside from Noel and me had contributed to the
tasks proposed to and agreed upon in the release plan?  Just curious...

There seems to be a sense from some of the folks on this list that things
happen by "magic".  They don't.  As with all things of value, they happen
because people put in effort to make them happen.  You want something to
happen, put in a little effort.  If you don't put in the effort, don't
complain.  See Serge's very correct comment on the matter.

> Please also note that your resistance to implementing mirroring have
> attracted the intervention of Greg Stein in his role as ASF chairman,
> something which I'm sure none of the longer serving commiters are
> particularly pleased about, and again if it were not for the drawn out
> release this could have been implemented prior or post the freeze.

As far as Greg's intervention, it hardly seems in line with "the Apache Way"
to quake in fear of the presence/intervention of any individual, ASF
chairman or other.  I appreciate Greg's POV, I'm happy to hear what he has
to say, and am more than happy to discuss the issue with him.  I'm sorry
this sort of open dialogue doesn't please you, but it hardly seems
catastrophic or even to be a major issue.

My position remains basically unchanged - I don't see any reason why 2.1
should be gated by getting the mirroring right.  Despite Nicola's somewhat
emotional response (I was half expecting a follow-up lecture on Hardin's
"Tragedy of the Commons") it is neither ingratitude nor laziness that
motivated my vote in response to the mirroring proposal.  I'm very
appreciative for the resources provided by the Apache foundation, and very
cognizant of the fact that these resources cost money.  But I also have some
experience pricing bandwidth, and I found the tradeoff (one more thing to do
before release vs. a change of doubtful financial value) to be undesirable.

While I'm not going to go into a lengthy explanation of my economic
reasoning here, a simple Fermi estimate of the overall percentage of Jakarta
bandwidth consumed by James distribution downloads (remember, mirroring
doesn't affect anything but that) should make it clear why I feel this is
not a priority.  

It's pretty clear to me that James downloads are far on the small side of
the 80/20 (or for that matter 90/10 or 95/5) rule.  Considering that
bandwidth is not a fungible commodity, and more importantly is not generally
priced that way for larger sites, this is a rather substantial argument as
to the insignificant financial impact of mirroring for James downloads.
Add to that the fact that it is something that can be addressed post-release
with a minimum of bother, and I still don't see why it needs to be done as
part of this release process.  

All that said, Noel has been trying to set it up pre-release.  He wants to
confirm some things with the Apache infrastructure folks.  If he can get a
response, great (he'd been unsuccessful before last Friday).  If not, I'd
still like to move for a release if all the marketing channels are lined up.
We can discuss that in the unlikely event Noel doesn't get a response from
the infrastructure folk.

In any case, I'm sorry you're not pleased, but it doesn't please me that no
one save Noel/me has put in any serious effort on James 2.1 since October.
Life is full of unpleasant things.

I agree with you, the drawn out release is unfortunate.  I am doing what I
can to get it out the door.  Next time around a more general effort would
probably help shrink the release cycle substantially.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: There is no active development on James

[Peter M. Goldstein]

Serge,

> I'm very much sick of 2.1.  I don't know why it's taking so long, but
> I'm not helping enough to complain.

You're not alone.  It's taking so long because no one is helping.  Just look
at the last two months of CVS messages.  
 
> You dodged the issue of whether this was an important bug by saying it
> was "discussed over and over and over again".  Rather than doing so
> indirectly, I'd prefer if it you directly answered whether you would
> prefer to ship 2.1 with an unusable configuration (POP3 on, and using
> file mailstore) or fix this bug.

No I didn't.  But to be absolutely clear, I would rather see 2.1 released
now with this problem in place than wait to fix this bug.  I believe that
the appropriate way to resolve this would be with release note/documentation
and a subsequent bug fix release (2.1.1 or 2.2).

First off, I regard the statement that it makes file repository unusable as
not quite accurate.  The truth of the matter is that POP3 file repositories
are not robust upon restart.  Agreed, very bad.  I certainly wouldn't want
to use a file repository in a production, multi-user system.  But not
unusable as a simple test configuration.  Or for proof of concept work.  As
I said, we can document the problem to make this distinction clear.

Second, my view of release management is that you make a decision at some
point in the process, and you stick with it.  We made this decision.  It was
discussed.  I don't think that fact is irrelevant.  I think it's important.
Otherwise there is the tendency to revisit every decision in the release
process.  Everything is game, nothing is settled.  I think that's one of the
worst ways to do a release.

Third, much as some on the list won't want to hear it, there is a
workaround.  Use a database for your production systems.  MySQL is free for
personal use, reasonably easy to set up, and consumes minimal resources.
Even the commercial license is very inexpensive.  This is a realistic option
that maintains the functionality.

Compare this situation with the bug in 2.0a3 (and I believe in 2.0a2) where
use of SMTP auth automatically opened up the server as an open relay.  In
that case there was no workaround whatsoever - SMTP auth is the only way for
James to securely process relayed mail from arbitrary hosts - yet the
release still went out the door. 

Finally, we don't even have a patch for this yet.  While you've now
volunteered to do this work, we don't know how much time/effort it will take
to fix this.  Will it make the upgrade more difficult by breaking file
repository backwards compatibility?  Will we have to document the upgrade
procedure?  How much testing effort will you be putting in to avoid new
issues?  Will it be equivalent testing to the amount done in the Q/A portion
of this release (processing of ~1 million messages, for a total of ~2GB of
disk space consumed)?  The time/effort and the potential repercussions are
totally unknown.  Given the lateness in the process, I find that highly
undesirable.

I still believe that the correct solution is to document the issue and
commit to a post-2.1 bug fix release (I said 2.2 in my previous email, I
meant 2.1.1).  That will allow us to resume development work on 3.0 and get
a little momentum back in the game.  You can continue with your work on
patching the problem, and it won't be holding up other features.

> I understand your frustration with a possible delay.  But the feature is
> glaringly broken, and in my mind it is worth fixing.  I think holding
> people accountable for knowing 100% of what needs to be done in an
> onsite project is unrealistic at best, and doing so for a globally
> diverse group of volunteers is even more dangerous.

I guess.  I don't know how you intend to do a release process if people
aren't willing to take some responsibility.  Certainly it's going to lead to
a tepid feature set, continual alpha/beta releases because no one want to
make the commitment to a released product, and a general lack of
stability/performance.  Having the project maintainers take some level of
responsibility is the only way I know how to avoid these problems.  And
other projects with a "globally diverse group of volunteers" (i.e. the
Apache web server) seem to do ok with this.

All projects, open-source or closed-source, require a willingness to say at
some point, "Good enough, push it out the door".  That's how imperfect
products both closed source and open source (just look at Linux, Apache,
etc.) get distributed to a user base.  Otherwise you might as well call the
product Taligent and accept obsolescence/irrelevance a priori.   

At this point we're essentially done.  Docs are imperfect, but have enough
coverage so that they answer most of the questions we see on james-user.
Some tweaking can occur post-release, if people feel like it.

Noel has the marketing stuff lined up, and was supposed to return home
yesterday night (I haven't seen him pop up yet, so I don't know if he's
back).  He was

RE: There is no active development on James

[Peter M. Goldstein]

All,

IMHO, this is ridiculous.

If the bug is glaring now, it was glaring when we
discussed the list of bugs for release.  And when it's
come up on the list before.  Again and again and
again.  There was a conscious decision (with
discussion) to defer this bug because we're intending
to redo the repositories.

Consider this a -1 vote on further delaying the
release for something we've discussed over and over
and over again.

If you want to update the 2.1 POP3 docs to clarify
this issue, that would be fine.

If you want to put out a 2.2 with this fix, great.
Please do it after the seemingly endless 2.1 is out
the door.

But personally I'm just plain sick of 2.1 and want to
be done with it.

--Peter

--- Danny Angus <[EMAIL PROTECTED]> wrote:
> 
> On the basis that it should be on the list of things
> to fix anyway, having
> hung around for, I say go fix it.
>
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=8839
> 
> d.
> 
> > -Original Message-
> > From: Serge Knystautas
> [mailto:[EMAIL PROTECTED]]
> > Sent: 19 December 2002 18:38
> > To: James Developers List
> > Subject: Re: There is no active development on
> James
> >
> >
> > Danny,
> >
> > This is such a glaring bug, to not patch it is to
> say we DO NOT support
> > POP3 file storage support (and would need to
> announce that very loudly
> > since this is the configuration out of the box). 
> I think it's great the
> > code is more scalable, cleaner, better documented,
> but this has been
> > such a glaring bug for so long.  Heck, it's just
> one of those poorly
> > thought out "features" of Avalon, and it wouldn't
> take much work to just
> > copy the code and undo the change that broke this.
>  Ok, it'd be a little
> > more work to be backwards compatible, but still,
> really not that much.
> >
> > Can we vote to let me patch this? (or show me the
> previous vote so I can
> > appeal to however blocked it... I've searched the
> archives for the bug #
> > and can't find it).
> >
> > --
> > Serge Knystautas
> > Loki Technologies - Unstoppable Websites
> > http://www.lokitech.com
> >
> > Danny Angus wrote:
> > > James in in a code freeze pending release of
> version 2.1
> > >
> > > I can't say when the release is likely to
> happen, but it is
> > unlikely that
> > > this issue is going to be addressed before then.
> > >
> > > The reason for it not being fixed before now is
> because we
> > inherit it from
> > > Avalon.
> > > We are intending to replace that with our own
> filesystem code
> > in the next
> > > cycle.
> > >
> > > d.
> > >
> > >
> > >
> > >>-Original Message-
> > >>From: Dejan Nenov
> [mailto:[EMAIL PROTECTED]]
> > >>Sent: 19 December 2002 07:44
> > >>To: [EMAIL PROTECTED]
> > >>Subject: There is no active development on James
> > >>
> > >>
> > >>To any committers / active contributors to James
> development out there -
> > >>can you please review and verify bug 15460 -
> unless no one else can
> > >>reproduce the problem - it seems to be making
> James totally unusable!
> > >>
> > >>
> > >>
> > >>Thank you,
> > >>
> > >>
> > >>
> > >>Dejan
> >
> >
> > --
> > To unsubscribe, e-mail:
> 
> For additional commands, e-mail:
> 
> 
> 
> --
> To unsubscribe, e-mail:  
> 
> For additional commands, e-mail:
> 
> 


__
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com

--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Bugzilla Maintenance

[Peter M. Goldstein]

Charles,

> Done

Thank you.
 
> I think we can use the "Milestones" field to distinguish where in the
> 2.1 cycle a bug is. This has only recently been activated, so there are
> no entries for James.
> 
> How about:
> pre-alpha, alpha1, alpha2..., beta#, rc#,  etc?

Add "release" or 0 for the actual release and a set of minor version numbers
(1, 2, 3, 4, 5) to deal with subsequent bugfix releases and I think that'd
work just fine. 

> >A full revisit of the component list is probably appropriate early in the
> >3.0 cycle.  I'd rather we did this all at once, rather than piecemeal.
> >
> >
> Fine by me

Cool.  Probably should be done at the same time as the version discussion.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Bugzilla Maintenance

[Peter M. Goldstein]

Serge,

> Would you reconsider your vote?
> 
> If we're going to publish a build, why would we NOT want to track a bug
> in that build?  Why else are we publishing an alpha/beta/RC EXCEPT to
> collect bugs?

Nope.  See my reply to Danny for the answers to the above.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Bugzilla Maintenance

[Peter M. Goldstein]

Danny,

> > > I have just added 2.1a1 as a version.
> >
> > -1.  Please revert this.
> >
> > This is something I didn't request.
> 
> Perhaps not, but everything is not up to you alone.

Of course not, but as I'm the one who made the original Bugzilla request, I
think the content of that request is significant.  There's a tacit
acceptance of the situation in request.  I wanted to express that I did not
request this situation.  Others are free to disagree.
 
> >  Alphas, betas, and release candidates
> > should not be specified as part of the major version number in a
> > bug system.
> 
> Says whom? and why? if rc's are released in order for testing it makes
> perfect sense to include them in the bug system.

Absolutely, include the alpha, beta, and release candidate.  Just do it
separately from the major version.

It's precisely this confusion on version numbers that led to the recent
discussion where it was seriously proposed that our final gold release be
labeled 2.1.1.  We can't even seem to figure out whether third digits in
version #s should be used for rcs or not.  Indicates a bit of confusion
IMHO.

And I say.  This is my opinion, both as an active committer on this project
and as a software professional.  You are welcome to dissent.  You are even
welcome to veto.  But I'm certainly not going to feel constrained in
expressing my opinion.

As to why, that's explained both in the previous emails and in this one.

> I don't know where you've worked but ever release I've participated in, on
> whichever side of the fence, has had alpha beta and rc's in the bug system
> because the versions change quickly and so we can know which version the
> tester was using.

Well, among other jobs, I managed the Continuing Engineering department for
an enterprise security product with three APIs and approximately thirty
different product components (three servers, three web server agents, two
app server agents, five platforms with mixed coverage).  In my time in that
role I and my team handled several hundred bug reports (some valid, some
not), often from clients requiring fixes in hours/days.  We issued 40+
patches on a variety of versions/components and produced four or five
roll-up bug fix releases.  I used Bugzilla constantly, both   I worked
closely with release management throughout my time in that role,
coordinating version numbers on those patches.  I also worked with core
engineering to help them integrate pre-release bug tracking and testing with
the bug system.  So I feel I've got some relevant experience.

As I made clear in my previous email, I've got no objection to recording
alpha, beta, and rc data in the bug system.  I think it's valuable.  But
preserving the ability to simply query and organize bugs by major version
dictates that the two be kept apart.  It also simplifies milestone, alpha,
and beta release since by using a product of several fields ( major version
(2.1, 3.0) * minor version (a1, a2, a3, a4, a5, b1, b2, b3, b4, b5, rc1,
rc2, rc3, rc4, rc5, 0, 1, 2, 3, 4, 5)) it no longer becomes necessary to
alter the bug system everytime an alpha, beta, or milestone is released.

 
> > Aside from simply increasing the number of values for an important field
> > (hence making searches/categorization more onerous), it doesn't serve
> any
> > useful purpose.
> 
> It serves to allow develoeprs to identify the codebase in which the bug
> has
> been identified, surely.

Uh, no it doesn't.  Does 2.1a1 refer to the code in July (pre-FetchPOP) or
in late August (post-FetchPOP)?  Who knows?  No one does.  Why?  Because
there has been no effective labeling/tracking of said versions.  We released
two milestone builds and i) didn't label them correctly and ii) didn't even
update the web site to refer correctly to the release candidate.  I'd say
the release management needs a little work.  And part of that work is
separating major versions from minor versions.
 
> > > Do we want to add 2.1 and 3.0 before they are released? It could make
> > > life confusing for bug reporters and for us.
> > > I suggest we add new versions to Bugzilla as part of the release
> cycle.
> 
> Yes 2.1 and 3.0
> 
> and 3.0 as a milestone. it is also IMO reasonable to have point versions
> specified in milestones too.

What milestone?  You have no idea at this point what milestones, alphas, and
betas are going to be released for 3.0.  All you know is that we are going
to be working towards a 3.0 version.  How do you handle bugs in features
that are introduced after one version and fixed before the next?  Again, not
in 2.1a1, not in 2.1a2.  But it is in 2.1, because that's a global container
for the process.   

One of the big problems with 2.1 bug tracking was the introduction of a
2.1a1 build.xml label essentially instantaneously after 2.0a3.  This is an
error.  It meant that 2.1a1-cvs was a catch-all for anything that's been
found in the last several months.  It gives the illusion of specificity,
while giving nothing of the kind.

> > R

RE: Bugzilla Maintenance

[Peter M. Goldstein]

Charles,

> I have just added 2.1a1 as a version.

-1.  Please revert this.

This is something I didn't request.  Alphas, betas, and release candidates
should not be specified as part of the major version number in a bug system.
Aside from simply increasing the number of values for an important field
(hence making searches/categorization more onerous), it doesn't serve any
useful purpose.  It also makes it more difficult to i) evaluate the bugs
that need to be evaluated before final release and ii) list the bugs that
actually are present in the release, post official release.

> Do we want to add 2.1 and 3.0 before they are released? It could make
> life confusing for bug reporters and for us.
> I suggest we add new versions to Bugzilla as part of the release cycle.

Absolutely.  The versions that should be added to Bugzilla should be 2.1 and
3.0.  They should be added now.  2.1 will allow us to record user and
developer discovered issues in 2.1.  3.0 will allow us to record bugs
discovered in the course of developing 3.0.  The actual Release is the
middle of the game, not the beginning. 

Right now the bug system is effectively useless for accurately recording
James bugs.  That's one of the reasons that nobody uses it (I think I've
recorded over half the new bugs in Bugzilla since August, and there have
been a lot more bugs than that).  Fixing this would go a small way towards
making it more usable.

Properly designed bug systems always include version numbers pre-release.
That allows developers working on a particular release to record bugs found
in new features, bugs in old features being improved, or bugs just exposed
via the results of new test cases.  If it is necessary to include a notation
that the bug was discovered pre-release, either a "stage"
(pre-release/post-release) or a minor version (alpha#, beta#, rc#) field is
appropriate.

Ask yourself the following question.  How would I have recorded a bug found
in FetchPOP in September?  Version unspecified?  Not very useful.  Version
2.0a3?  This feature wasn't even present in that version.  The correct
answer was (and should've been) 2.1, as that indicates both the time of
discovery (sometime in the 2.1 cycle) and allows committers to easily
evaluate the relevant bugs when considering release.  It also allows users
to discover which bugs remain in their version post-release.

There's a lot of room for additional improvement in the bug system (i.e
found-in vs. fixed-in versions, etc.) but this alone would make accurate bug
reporting easier.
 
> Also, would it be useful to add Mailet API versions e.g. API_1.0 etc.?

A full revisit of the component list is probably appropriate early in the
3.0 cycle.  I'd rather we did this all at once, rather than piecemeal.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Bugzilla Maintenance

[Peter M. Goldstein]

All,

Does anyone know who manages the Bugzilla installation linked on the Jakarta
site?  I want to ask the administrator to update the version fields for
James - adding both a 2.1 and a 3.0 field.  Is an email to
[EMAIL PROTECTED] necessary, or is there another contact email?

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Version numbers

[Peter M. Goldstein]

Danny,

> I understood that jakarta version numbers are x.x.x where x is a
number,
> therefore 2.1 should be 2.1.?

Ok, I guess.  Although as I recall, Phoenix 4.0 was labeled 4.0 and not
4.0.0.  But 2.1 or 2.1.0 is fine with me.
 
> And since version numbers should always increment for every published
> build the released version should be 2.1.2 since we have used 2.1.1
for
> betas and release candidates.

This I don't agree with.  Everywhere I've worked, betas and release
candidates did not advance the version number.  There's a really good
reason for this - In general, you don't know a priori how many betas
you're going to have.  Thus, you wouldn't be able to settle on a final
version number ahead of time.

Instead, betas or release candidates are usually marked with the
intended final version number followed by a 'b' or 'rc' and the
beta/release candidate number.

It seems especially undesirable to try and increment the third version
digit based on the release candidates considering that, until yesterday,
everything that went out was labeled alpha (2.1a1-cvs).  We've never
actually referred to the release candidates as 2.1.0, 2.1.1, or 2.1.x.
In the code they were labeled as 2.1a1-cvs (all versions) and on the web
page they were referred to as 2.1a1 followed by a date.

Finally, from a marketing perspective, 2.1.2 sounds like a far less
important milestone to decimal-centric minds than 2.1.0.  As part of the
point of this release is to gain some exposure and market James to a
wider community, I'd argue that this is a good reason to stick with 2.1
or 2.1.0.  Otherwise we risk making our announcement look like a bugfix
release.

--Peter


   



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: v3 mailrepository maildir

[Peter M. Goldstein]

Hi Alexander,

> i'm an author of javamaildir (http://javamaildir.sourceforge.net/)
> project. few days ago noel bergman ([EMAIL PROTECTED]) contacted me and
> asked if i wanted to contribute to james.
> as i understood from his words (and from googling this page:
>
http://www.mail-archive.com/avalon-dev@jakarta.apache.org/msg18473.html)
>   maildir is going to be a major mailrepository for the james v3.
> 
> the last time i saw james' code was half a year ago, by that time
james
> had no (well unusable) imap support, weak ldap user store and no
> maildir/maildir++ support. these are basically the areas of my
interest.
> as i see on james' page imap support is removed(was
alpha/experimental)
> and ldap is still experimetal.

Yep, we're basically at the same point.  IMAP is receiving some new
attention, and is a high priority for the next revision.  But LDAP
hasn't received much attention, and there isn't a maildir implementation
of a mail repository.

> i'd like to bring imap back, but that would require a major redesign
of
> mailrepository interface. it leads me to conclusion that i should
focus
> my work on james v3.

I would agree.  Such a redesign has been discussed for version 3.0.
Desire has been expressed by a number of parties (myself included) that
we revamp the mail, spool, and user repositories.  Input on that revamp
would be extremely valuable.

> noel said that no v3 code currently exists only ideas.
> 
> a question: are there any web-pages/papers that i could look through
to
> get a quick understanding of has been done about v3 and where should i
> start, or possibly anyone could describe the situation?

Here's where I'd start if I were you.

First, the James 2.1 documentation.  Currently this is in beta at
http://jakarta.apache.org/~pgoldstein/documentation_2_1.html .  It will
be moved to the James home page upon finalization/release.  This should
give you a good idea of where James stands today.

Second, the updated "todo" -
http://jakarta.apache.org/~pgoldstein/todo.html .  Again, this should be
moved to the home page shortly.  This is a list of most of the features
that have been discussed either on james-dev or on james-user as being
desirable for v3.  It is not complete, and I expect others to contribute
more to it shortly, but it's a start.

As far as v3 discussions go, they're imminent.  We've had an agreement
in place to focus on getting version 2.1 out the door before moving on
the version 3.0.  Even with this agreement, movement has been slow.  But
the documentation is basically good to go, and Noel expects to have the
marketing elements in place by Wednesday, so v3.0 discussions could
begin in earnest as early as the end of this week.

--Peter 




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Error Compiling AvalonMailRepository James 2.1a1-cvs

[Peter M. Goldstein]

All,

I revamped the build.xml so that all targets will work with the more
limited source distribution.  The new source distributions have been
posted.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Error Compiling AvalonMailRepository James 2.1a1-cvs

[Peter M. Goldstein]

All,

Ok.  The build.xml is updated.  New source distributions have been
placed in the "latest" distribution directory.  The java source for
these is unchanged from the associated binaries (specifically, Noel's
patch isn't applied).  Please let me know if this doesn't resolve your
problems.  Thanks.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Error Compiling AvalonMailRepository James 2.1a1-cvs

[Peter M. Goldstein]

Aaron and Kwirirai,

> Kwirirai is dead right.  Neither of the "latest" James code drops
> (james-with-phoenix or without) have the necessary phoenix libs in
them.
>   I downloaded them and dropped in the entire phoenix-bin directory
from
> cvs and it now builds fine for me.  (Both of them work fine after this
> treatment.)

He most certainly is.  Ouch.  That bug in the dist has been sitting in
CVS since Paul redid Phoenix over the summer.
 
> I don't know how much of that is strictly necessary, but you certainly
> need more than is distributed in either of the src drops.

For the source without Phoenix only the phoenix-bin/lib directory is
necessary.  With Phoenix obviously you need the whole shebang.

Thanks to both of you for exposing this rather embarrassing bug - all of
us use CVS and I guess it never occurred to any of us that the dist
would have been corrupted.  Anyway, I'm fixing now and should have a new
milestone source distribution posted shortly.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Erroneous handling of SendFailedException (was: Urban tapestries)

[Peter M. Goldstein]

Noel and Danny,

> You are correct.  A 4xx should be retried.  Looks like the handling
for a
> SendFailedException is a problem.

Agreed.
 
> I don't know if there was a regression when the exception handling was
> reorganized in the most recent patch.  Prior to that, the code said
that
> any
> SendFailedException was always permanent.

Don't think there was a regression.  In fact, there's an explicit
comment in there that predates the reorganization that seems to regard
any SendFailedException with a non-empty success list as permanent.
And, as I mentioned when I did the reorganization, the post-connection
logic remains unchanged.

> However, since a SendFailedException is a subclass of
MessagingException,
> differing only in the partial delivery nature of the exception, so
would
> you
> please try replacing the return call in that catch clause with:
> 
>  return failMessage(mail, sfe, (('5' == sfe.getMessage().charAt(0)) ?
true
> :
> false));

I agree.  That should resolve the problem.  The in situ comments should
also be updated.

> which is what we use for MessagingException, and see if that fixes it
for
> you.  I am putting that into my build, too.
> 
> If so, I think it is a serious enough bug to patch.

+1

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Linefeeds in www (was RE: cvs commit: jakarta-james/www/)

[Peter M. Goldstein]

Danny,

> >  I'd argue that the RFC
> > stuff should be put in xdoc format and incorporated into the
standard
> > web site build process.
> 
> I don't think so, unless you are likely to have enough free time to
> transcribe them what would the point be?

Uniform look and feel.

By putting them in xdoc format (and the transcription isn't necessarily
particularly hard) you get the title bar, side bar, etc.  The RFCs look
like part of the site, and navigation to the rest of the James site is
easy.  It's a far more professional look.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Linefeeds in www (was RE: cvs commit: jakarta-james/www/)

[Peter M. Goldstein]

Noel,

> > As far as ASF/CVS requirements, I don't believe that there is any
such
> > animal.
> 
> > I've never read any documentation that indicated that it is either
> > necessary or desirable in the eyes of the ASF that the website
> > HTML/images be stored in CVS in an immediately deployable form.  If
> > anyone knows of such docs, please send a link to the list so we can
> > discuss.
> 
> Here you go: http://www.apache.org/dev/committers.html#web
> 
> I suspect that the correct place to discuss the issue with the ASF
would
> be
> in infrastructure@.  From a recent discussion there, it appears that
there
> are certain topics, e.g., site security, that they do not want
discussed
> in
> a publicly archived list.

Ok.  Will do.
 
--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Linefeeds in www (was RE: cvs commit: jakarta-james/www/)

[Peter M. Goldstein]

Noel,

> My point was simply that such content is there, and needs to be
accounted.
> And we should address our www building process when we look at
> Maven/Forrest.  But we also have to deal with whatever requirements we
> have
> from the ASF in terms of having a CVS entry for their controlled
> management
> of the web site.

Ok.  I don't quite get what your point is here.  Of course we need to
account for it.  But it's not like that's terribly difficult.  There
isn't much of it, as I detailed in my previous email.

As far as ASF/CVS requirements, I don't believe that there is any such
animal.  Right now our website update process is completely manual and
totally instigated by James committers.  That is, unless Danny, myself,
or someone else goes into daedalus and manually updates the subdirectory
via CVS update, the website remains unchanged.  There is no automatic
ASF process that updates it for us (and hence might require www in CVS).

I've never read any documentation that indicated that it is either
necessary or desirable in the eyes of the ASF that the website
HTML/images be stored in CVS in an immediately deployable form.  If
anyone knows of such docs, please send a link to the list so we can
discuss.  If there is such a requirement, I'll be happy to argue against
it with the ASF.
 
> Perhaps it would be better to have a separate james-site module, and
> publish
> to it, so that the normal James module does not have any generated
> content.
> I don't know.

Ugh.  Same problem with updates, just deferred to a different module.
We'd have to manually update changes to the Javadoc.  That sort of thing
has continuously led to discrepancies that lasted for months in terms of
the Javadoc available via the website and the Javadoc produced from the
source code.

Let me reiterate.  Dynamic build products being stored in source control
is a terrible idea.  I don't care what source control system, I don't
care what module, this remains a bad idea.  We just shouldn't do it. 

--Peter
  



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Linefeeds in www (was RE: cvs commit: jakarta-james/www/)

[Peter M. Goldstein]

Noel,

> > running the build (with no source code changes)
> > should not introduce any diffs with the source
> > control tree.
> 
> I appreciate your distinction between authored and generated material,
but
> I
> should point out that the www/ directory does have more in it than
just
> the
> generated files.  We'd have to move the other material out of there,
too.

Of course.  It shouldn't be in there anyway.  I'd argue that the RFC
stuff should be put in xdoc format and incorporated into the standard
web site build process.  That would leave a single image file as the
only non-generated file in the www directory.  And yes, we'd have to
move that.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Linefeeds in www (was RE: cvs commit: jakarta-james/www/)

[Peter M. Goldstein]

All,

> I personally don't think that generated html in /www/, which is there
to
> allow controlled website updates not to version the docs themselves
which
> are versioned in /src/, nedds to have this as strictly enforced as
> elsewhere, IMO it takes effort for no appreciable benefit.
> 
> Thoughts?

I have to disagree.

First off, I'm going to comment that I very much hope the www directory
goes the way of the dodo early in the 3.0 cycle.  The presence of this
directory violates one of the basic tenets of good build design.
Specifically, that running the build (with no source code changes)
should not introduce any diffs with the source control tree.  In other
words, the build operation itself should be idempotent.  To see that
this isn't true, do the following:

1) Check out a new Jakarta-james tree
2) Run 'build website'
3) Diff the tree - you'll see every javadoc file is changed.  

Add to this that specific changes in the src/java directory (additions,
deletions) have to be manually carried over to the www directory and it
is obvious that the current process is fragile and should be changed.
(Note, for example, that the "fetchpop" classes weren't added until
yesterday).  We've discussed all this on the list before, so I'll let it
go at that.

That all said, if the www stuff (and most importantly the javadocs and
mailet API docs) are going to be in source control for the moment, it's
important to minimize the diffs so that one can discern actual issues
among all the noise.  After I did my largest recent check-in on the
javadocs directory I was able to find several issues in the Javadocs
precisely because not every line contributed to the diff.  So
standardization on LF served a very useful purpose, as well as being a
really easy change to make.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Outstanding tasks for release.

[Peter M. Goldstein]

David,

Nope.  2.1 is considered a stable release.

--Peter

> -Original Message-
> From: David Jenkins [mailto:[EMAIL PROTECTED]]
> Sent: Saturday, December 07, 2002 1:00 PM
> To: James Developers List
> Subject: Re: Outstanding tasks for release.
> 
> Peter M. Goldstein wrote:
> > 2) Completion of a press release - We need a press release to go
with
> > the actual version release.  I've been toying with this, and have a
> > partial draft together.  Noel is going to take a look at it, and
then
> > we'll post the whole thing for general review and comment.
> 
> Speaking of versions.  Was a decision reached as to follow the Odd and
> zero minor versions are development and even 2 and up are stable
> releases?  I saw a thread started on several lists, but wasn't sure if
> JAMES was going to follow it as well
> 
> Thanks,
> David J
> 
> 
> --
> To unsubscribe, e-mail:   <mailto:james-dev-
> [EMAIL PROTECTED]>
> For additional commands, e-mail: <mailto:james-dev-
> [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

RE: Outstanding tasks for release.

[Peter M. Goldstein]

Darrell et al,

Here are the remaining tasks for release:

1) Completion of the user documentation - As my recent notice to
james-user stated, there are a couple of omission/inaccuracies in the
documentation.  They are repository configuration information, database
configuration, TLS/SSL configuration.  In addition to
completing/correcting these sections, the whole shebang should be vetted
by a couple of people who are familiar with James and are not me (as I
wrote most of it).

2) Completion of a press release - We need a press release to go with
the actual version release.  I've been toying with this, and have a
partial draft together.  Noel is going to take a look at it, and then
we'll post the whole thing for general review and comment.

3) Update of the version string in CVS - I've got this one.
4) Build of the actual distribution - I've got this one.
5) Tagging of the source tree at this point

6) Update of the distribution directories' README.html
7) Deployment of the release to the release distribution directories

8) Distribution of the press release through channels - Noel has
collected the list of channels and the relevant distribution info.

IMO #1 is the big blocker.  The rest will probably go fairly quickly
once the documentation is up to snuff.  So if people can kick in on #1,
that would be good.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Release mirroring - let's do the release right

[Peter M. Goldstein]

Noel,

> When we actually have to copy the files over to do the release, I
think
> that
> we can do this.  I really don't see it as a problem.  I'm looking at
the
> XML
> document now, and will try to work with it.
> 
> Our downloads don't appear to be a significant resource drain by
> themselves,
> but the language on http://www.apache.org/dev/mirrors.html implies
that
> this
> is a priority for the ASF: "Any project willing to use the ASF
> infrastructure to distribute source, binary and documentation builds
> archive
> must follow the following guidelines."
> 
> If folks want, I'll take the lead responsibility for getting this page
> done.
> If I run into problems (daedelus access), I'll let you know.

And this is exactly what I don't want.

You have already signed up to manage another aspect of the release
process (most notably compiling a list of outlets and notifying them of
James' existence).  How will you taking on this responsibility affect
that one?  It will delay it of course.  And as I think the former is
much more important than the latter (as you mention above, we're not a
substantial drain on resources), I think this is a bad call.

The legalese argument simply doesn't apply (there is exactly one project
that currently follows these guidelines, and notification didn't come
from the ASF but rather from a concerned contributor).

So, while I may be in danger of beating a dead horse, I veto this with a
-1.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Release mirroring - let's do the release right

[Peter M. Goldstein]

Danny,

> > > I suggest the release guys to consider doing the next release
> > following
> > > the above guidelines, and thank the infrastructure team and Stefan
for
> > > the effort taken in being the first and in making the guide :-)
> 
> I didn't see this message, FWIW I'm +1 and if you're going to veto
this
> I'd very much like to know why, beyond "additional required effort".

I admit my last message was a tad terse, but I think the basic argument
holds.

To put it mildly, there is a lot that goes into a release.  Some of this
(i.e. documentation, marketing) has largely been skipped in previous
releases.  That's not happening this time, which in the long run will be
a good thing for James.  These things are essential to building a
user/developer community.

But these pieces of the puzzle take work.  And it tends to be work that
developers don't like to do.

So that leaves us where we are today.  Several requests for others to
pick up some of this work have gone largely unheeded.  Noel has done
some - organizing the list of outlets, writing up mailing list
instructions.  You volunteered to do some documentation (FetchPOP) but
never delivered.  I've done the rest.

This is all fine, as we all have different priorities in our lives.  I
am not trying to scold anyone, I'm just laying it out the way it is.
But this does mean that we have to make (or at least try to make)
realistic assessments of what's going to get done, how it will get done,
and how getting that done will affect the release.  Adding additional
work at this point in the release cycle is, in my view, not acceptable. 

So, my -1 veto stands.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Release mirroring - let's do the release right

[Peter M. Goldstein]

Nicola,

> I suggest the release guys to consider doing the next release
following
> the above guidelines, and thank the infrastructure team and Stefan for
> the effort taken in being the first and in making the guide :-)

-1.  Considered and rejected.

We're having enough trouble pushing through the final stages without
additional required effort.  For 3.0 we will certainly examine the guide
in some detail and see how James can work within the recommended
context.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [VOTE] Drop @author tags from source files

[Peter M. Goldstein]

All,

> +1 to removing named authors, but -0 putting to @author james dev
team...
> I
> don't think an author tag is worth having at all.

Either way is fine with me.
 
> I think it's important to credit in the changelogs of who did what, as
the
> changelog is the real meassure of the vitality and velocity of the
> project,
> and the people doing that deserve recognition (IMHO).  This continues
the
> shift away from @author tags, which credit who *did* what, to
changelogs,
> which credit who *is doing* what.

Speaking of which, please check out the updated changelog.  I think I
got most everything in 2.1, but it is possible that I left out a couple
of items.  I was working primarily off the commit messages.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [VOTE] Drop @author tags from source files

[Peter M. Goldstein]

> > 
> > 1) We replace all @author tags in the source with a single line
> > @author mailto:[EMAIL PROTECTED]";>James
Development
> > Team
> >
> > 2) We make sure that any author that has had their name removed gets
a
> > mention in the contributors page.
> > 

Provided the change occurs after 2.1 goes out the door, +1.  -1 to
breaking code freeze for this.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Easy protocol logging

[Peter M. Goldstein]

Darrell,

> This is unfortunate. I was unaware that anyone was working on the
existing
> IMAP proposal. (Although looking back on an email of yours, you did
> mention
> that you were working on some patches). I never felt that proposals
were
> included in the James2.1 code freeze, since they aren't a part of the
> release.

Proposals aren't included in the code freeze.  But in the interest of
getting some momentum behind 2.1, the 2.1 committers agreed

 
> As you are probably aware, I have written a new IMAP proposal (in
> proposals/imap2) which shares very little code with the existing
proposal.
> I
> have a bunch of time on my hands at the moment, and I needed to
immerse
> myself in a design/coding problem. IMAP was it - I hope I haven't
caused
> waves by continuing to develop while other committers felt the need to
> hang
> fire.

No waves.  I've always contended that it's open season in proposals.
Feel free to check in whatever you want in the proposals directory.  Not
going to bug me.  :)

> Earlier this year I did a significant amount of work on the first IMAP
> proposal. I added in the commands package, and got
> SingleThreadedConnectiionHandler down to 1/4 it's original size. But
it
> was
> still complicated, convoluted, and hard to modifiy, debug etc. So when
I
> came
> back to it this year, I decided it would be easier, better, and more
fun
> to
> start from scratch.

I understand.  I don't really agree, but I understand.

> Being a bit of an XP fan, I spent a day jotting down some design
ideas,
> grabbed the existing tests, and started trying to get them to pass.
I've
> continually refactored as I've gone, and I'm pretty happy with the
result.
> Where it made sense, I pulled classes from the original proposal, but
not
> many, as it turned out. We now have an imap proposal which is IMHO
> cleaner,
> easier to develop, and more modular that the earlier proposal. It also
> doesn't carry the cut-and-paste-hell baggage of the original.

Well, that's more debatable.  See my comments below.

> The benefits of the new proposal over the old are:
> 1) Just as functional as the earlier, but easier to prove (since I
have
> more
> tests)
> 2) Much closer to the spec on protocol issues like permissible
arguments
> (handles both synchonised and non-synchronised literals whereever they
are
> allowed).
> 3) More modular and extensible design, so new features and bugs are
easier
> to
> implement/fix (you implement the features and fix the bugs, hopefully)
> 4) Doesn't include another MailRepository format, but simply defines
the
> requirements it needs for storage. This should make it easier to
arrive at
> a
> unified MailRepository in the upcoming James3 discussions.

1) The tests, if correct should work for any IMAP server, right?  That's
the whole point - they're protocol tests.

2) It is better on the literals, although I don't particularly care for
the form of solution as implemented. 

3) This is arguable.  Honestly, from my examination, it looks like the
design has most of the modularity flaws and bad extensibility tradeoffs
that were inherent in the other design.  It's been my approach to evolve
these out.  Still in process.

4) True.  And that's a good thing for API development, a bad thing for
actually testing functionality.  My approach was just to lock away the
mailbox behind a simple interface.  This is especially easy, as much of
the implementation inside the mailbox is the previous implementation
simply shouldn't be there.  IMAP mailbox requirements are 95% defined by
the UID rules and the requirements of the SEARCH command.

> Let's get this sorted out straight away. The last thing we need is 2
> competing
> IMAP proposals. Please commit the changes you've made - there's no
point
> developing solo at home. And please have a look at what I've done.
It's
> not
> perfect by any means, but I think it provides a much more solid
foundation
> for an IMAP module for James.

Uh, as I said before, I've got a commitment to focus on 2.1.  Your
schedule doesn't change that one iota.  That commitment precludes me
devoting any serious time to 3.0 before 2.1 is out the door.  Sorry.

After 2.1 is out the door, and I've got time to neaten up the details on
what I currently have, I'll commit it.

Also, I don't really consider the "2 competing IMAP proposals" to be a
manifest evil.  Why is this a bad thing?  

I have actually taken a glance at what you've done.  No offense, but I
don't really see it as an improvement.  Looks to me like a step
backwards, honestly.  It has most of the problems inherent in the old
implementation.  You're free to work on it, and I'll take a more
detailed look at it once 2.1 is out the door, but my first impressions
haven't been favorable.  I suspect I'll want to stick with the older
version, with the modifications I've completed.
 
> Then, let's get these James3 discussions started. I can't see any
reason
> to
> wait.

Because we've got a release to get out the door, and a commitment t

RE: Easy protocol logging

[Peter M. Goldstein]

Darrell,

> It won't be quite the same. The other protocols work by reading one
line
> at a
> time and passing this to a command-specific method, where it is
parsed,
> optionally logged, etc. IMAP uses a parser-style approach, where the
line
> is
> never read in it's entirety. This is important, because IMAP commands
> often
> can span multiple lines (eg when "literal" strings are used). Rather
than
> log
> the raw command, I think I'll construct a log message consisting of
the
> command name and parsed parameter values.

It's actually almost exactly the same.  I can say this with some
authority because I've done most of it.  :)  It's in the revision of
IMAP that I've got sitting on my box.  I will, as I mentioned before, be
checking it in shortly after 2.1 is out the door.  I, like all the other
committers working on 2.1, deferred further work on 3.0 until after 2.1
is out the door.

I'm well aware of the literal issue - you may notice that I mentioned it
you in my earlier message to you regarding IMAP.  It's one of the big
problems with the previous revision.  IMO, it needs to be fixed using a
builder pattern, not a parser.  A parser is essentially object-level
state unaware, while a builder is state-smart.

As far as logging only after parsing, I'd actually reconsider.  Part of
the reason you want to get the raw input as your starting point to your
process.  Logging only after parsing/building will of course hide errors
in your parser/builder.  It is common to log something approaching the
raw data (with appropriate filters for secure/overlong data) and then
have abbreviated log entries to describe the post-parse/build.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Easy protocol logging

[Peter M. Goldstein]

Darrell,

> I thought I read something on this list awhile ago where someone had
made
> it
> dead-easy to log all of the wire comms on a socket. This would be very
> helpful to see what was going on, now that I've started connecting
real
> IMAP
> clients to James.
> 
> I'm thinking of a SocketManager/ConnectionManager implementation that
> returns
> a logging socket, so that the logging code doesn't need to be in the
> actual
> protocol server at all.
> 
> Has anyone tried something like this? Or does someone feel like coding
one
> up
> while I sleep?  :)

Uh, this is not a particularly good idea.

It would introduce regressions - most notably reintroducing the security
holes where passwords were recorded in log files.  Currently the logging
infrastructures for each protocol server are designed to record in the
log all command/replies with the exception of password data and message
content.

Implementing the same logging method as the other protocol services on
IMAP is relatively trivial.

--Peter  



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: regarding versions of james that partly/works with NW6/JVM1.4

[Peter M. Goldstein]

Michael,

> james-binary-2.1a1-cvs
> - I can get it to startup fine, and i can post to the news groups
>though OE behaves pretty wierd, as if the postings did expiere.
>my version of IE/OE is (5.50.4807.1700)
> 
> - With NS7.0 and Morzilla 1.x it is a little like the same as with OE,
> it behaves very strange.
> 
> - I'am able to use the Groupwise 6.02 NNTP client and i can then
> read/post e.t.c. fine.
>   but i believe it has something to do with GW caching it to the GW
> databases.(not sure about that though :)

Can you expand on this behavior a bit?  For example, what do you mean
when you say NS7.0 and Mozilla behave strangely?  I would also encourage
you to turn on debug logging for the NNTP server and send the logs to
the list.  It would give us the ability to look at the dialog from the
server side.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Aaron,

> I agree that having both the mailet container and Avalon trying to
> manage lifecycle would be bad.  But that makes it no less desirable to
> allow mailets to have access to Avalon services.  On reflection, this
> could as easily be achieved by having a Mailet that delegates its mail
> handling to another Avalon component.  I suspect that trying to wrench
> JAMES around to do this for me would not be good for me or JAMES.  (I
> wonder how easy it would be to write a generic handoff mailet?)

I don't have any idea what you mean by a "generic handoff mailet".  It
is trivial to write a mailet that uses Avalon components.  I described
(in some detail) how exactly one would do that in some of the earlier
emails in this thread.
 
> Unless I am mistaken, embedding a mailet container in order to run a
> single mailet would still involve configuring a root processor with an
> all matcher for my mailet.

You are mistaken.  Write your own container and no root processor is
necessary.  The processor tree is an implementation-dependent construct,
not fundamentally part of the Mailet API.  All the API dictates is that
the mail message being processed has a "state".  The associate of that
state with a spoolmanager processor is a product of the implementation.

> The "administrative overhead" I am refering to is the overhead of
> writing (and maintaining) a configuration file for the processor.  If
I
> don't need matchers and mailet chains, why configure them?

Then don't put them in your container.

> An I simply mistaken in my expectation that processors, matchers and
> configuration files are necessary to an embedded mailet container?

Bingo.  You are mistaken.

> > I'm not going to comment on the design issues that you and your team
may
> > face, except to say that backwards compatibility with your apps
should
> > not and cannot be a factor in our API design.
> 
> Of course not - that is why I have not made any attempt to justify or
> even describe my design.  It is simply not relevant.  What I am
talking
> about is widening the applicability of JAMES to include another
possible
> deployment scenario.  My personal view is that this is a generally
> useful thing.  (You obviously do not agree.)

I don't think the deployment scenario you describe is left uncovered by
the current API or any proposed API we're likely to come up with.  I
think I've been pretty clear on this one.  And I think the solution you
propose is extremely dangerous and underestimates the difficulty of
writing a correct, robust SMTP dialog handler.

To put this in some perspective, prior to this version James has never
matched the basic SMTP specification.  Even with version 2.1 there are
some known issues (minor bug in dot stuffing) and deliberate violations
(lack of support for certain SMTP commands) in place.

Opening up an API for any Tom, Dick, or Harry to plug in an SMTP
interceptor is roughly the equivalent of giving a six year old a loaded
pistol.  You know they're going to hurt someone, you're just not sure
who.  There is no way random users are going to be able to write correct
implementations.  It's just not going to happen.

The mailet API allows us to maintain a very strong and clear abstraction
barrier between the SMTP service implementation and the post-SMTP mail
processing.  That ensures that James administrators do not have to
concern themselves with the details of the SMTP transaction, while
retaining all of the mail-processing power.

> This is not entirely accurate.  As mentioned above, it is not really
> even relevant.  To me, this discussion is about the best way of
> embedding support for mail receipt into an application.  Your
contention
> is that I should embed an entire mailet container.  My contention is
> that I just want the minimum code possible to turn an SMTP
conversation
> into a Mail object.

And what exactly do you think is the gap between these two?
 
> Speculation about whether or not *I* need lifecycle support is
pointless
> unless you want to get specific to my own application.  (I can't see
why
> you would.)  Our point of contention is whether or not full lifecycle
> support is so useful that it precludes the usefulness of embedding
mail
> support without lifecycle.

The lifecycle is deliberately lightweight.  That's one big reason not to
incorporate the entirety of the Avalon lifecycle.  There is a complete
discussion of the Avalon lifecycle here:

http://jakarta.apache.org/avalon/framework/reference-the-lifecycle.html

To contrast the Mailet API lifecycle would include equivalents of phases
1,2,3,4,6, and 14.  These phases need not necessarily be separated in
the same way in the Mailet version, but I think there is general
concurrence that we need the same functionality.

So what would you toss away?  What phase is unnecessary?  Moreover, what
phase is expensive?  If your mailets are so trivial that they don't need
configurations, initialization, etc. then why begrudge a couple of
method calls?  If 

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Aaron,
 
> *Avalon dependencies*
> 
> I understand and agree with the objections to Avalon dependencies in
the
> mailet container.  I think that JAMES should continue to use Avalon,
but
> care should be taken to ensure that the JAMES design is not not
infected
> with dependencies on Avalon.  Rather, the JAMES _implementation_ of
the
> mailet container should _use_ whatever avalon services it finds
useful.
> Similarly, in my mailets, I should not be _required_ to make my mailet
> into
> an Avalon block, but I would like to be able to.  (I guess this is
> probably
> harder than it sounds, but it would still be nice.)

This is actually a typical server anti-pattern.  Essentially, it ignores
the inversion of control idea.  Inversion of control inherently implies
a chain of command (and hence a top of the chain of command).   If you
have two managing containers (and hence two responsible parties) you
have two competing tops of the chain of command.  Ordering problems,
initialization and disposal issues, etc. result.  Very contrary to good
server design.

See this link for more info:

http://jakarta.apache.org/avalon/framework/guide-patterns-ioc.html  

> 
> *Mailet Container vs MailListener*
> 
> My objection to having to embed the entire Mailet container is not a
> particularly strong one - if I have to embed the whole thing, that
will
> not
> be the end of the world - but here are a few reasons:
>  *Using a mailet container implies configuration of
processors,
> mailets, etc.  This is an administrative overhead that is simply
> unnecessary if you are not using that functionality.

Uh, we still haven't identified this supposed "administrative overhead"
that you'd be able to dispose of.  I think we're on a snark hunt.  Take
a look at the spoolmanager code.  Ask yourself exactly what you'd cut
out.  And what exactly you'd gain.

>  *Writing to spool implies that you must provide a place on
disk
> to
> write to info.  Not the end of the world, but again - unnecessary.  An
> in-memory spool manager implementation would help here.

No, it doesn't imply that at all.  As your 2nd sentence points out,
there is nothing to prevent you from implementing an in-memory spool.
Or a spool that provides storage in any other way you want (networked
file system, punch cards, audio-encoded, what-have-you).  Honestly, if
you really, really wanted to, you could write a spool repository
implementation that directly accepted messages from the SMTP service and
synchronously passed them to your favorite mail processing (be it Mailet
or otherwise) tool.  It wouldn't necessarily be a particularly good
idea, but you could do it.

The current implementation requires the above, but that's because the
James user and developer base hasn't had a compelling reason to develop
alternate spools.  If you want to, please feel free to write some local
filestore independent spool implementations.  That solves this problem.

> As far as life cycle issues go - yes lifecycle support for mailets is
> important.  There are many cases, however, where an application will
have
> its own way of addressing these issues.  When a requirement then crops
up
> to start receiving mail, I do not want to be forced to by in to a
whole
> new
> design approach, (however good it may be,) just to do this.

Uh...you're not.

I think that much of your confusion on this topic is due to a general
misunderstanding of features available through the Avalon framework and
through Phoenix.  Please take a careful look at the Avalon and Phoenix
docs.  I've already given an overview in my earlier responses about the
proper way to do in-process communication.  Major APIs are already
supported (i.e. EJBs via OpenEJB) and Tomcat support is underway.
Running custom servers where you don't want to expose an component
interface inside Phoenix is literally a piece of cake - it's just a
wrapper for a well written server.  I've done it a couple of times
already.

I'm not going to comment on the design issues that you and your team may
face, except to say that backwards compatibility with your apps should
not and cannot be a factor in our API design.

>From what I can tell, you've got a handoff from the James process to
your EJB engine that you don't like.  There are two realistic options to
address this - either deploy James inside your J2EE container or deploy
your J2EE container inside Phoenix.  Adding interceptor APIs at the
level you propose is not only unnecessary, but it doesn't actually get
you anywhere in solving this problem.

> To address Peter's comments about listeners vs managed components:
The
> JMS
> API *does* give you the opportunity to receive JMS messages without
> implementing a MessageDrivenBean.  This is an excellent analogy.  I am
not
> forced to do the whole J2EE appserver thing in order to use JMS.  I
can if
> a want the whole lifecycle thing, but I don't have to.  That is the
point.
> JAMES restricts developers to the appserver approach (JAMES being the

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Aaron,

> Standardised access to mail repositories.  Whatever form that may
take,
> I would like to be able to read and write (and even query?) the mail
> repository.  I would also like to be able to define arbitrary
> repositories - although doing this in via config files is probably
more
> useful than doing it in code.  I would also like to be able to define
> named repositories and reference them by name from my code.

Standardized interfaces for all James repositories is on tap for the 3.0
discussions.  The other questions are somewhat more complex, but I'm
sure we'd be willing to discuss feasibility.

> Mailets should be (able to be) first class Avalon citizens, with
access
> to the same services, lifecycle support and deployment semantics as
any
> other avalon component.  This should be done without requiring that
they
> be made into standalone components.  (The current "put your mailet on
> the classpath and reference it in the config file" method should still
> work.)  This could be supported by exposing the JAMES config as a
(dare
> I say it?) API that other components can use to register themselves as
> mailets.  (This is realy just an idea for discussion - I'm not even
sure
> it has merit, myself!)

I'm not nuts about this myself.

Basically I find this idea problematic because i) IMO, James should
control the mailet lifecycle and ii) I personally have no desire to
rebuild Phoenix inside Phoenix.

This has actually gotten some discussion recently, both here and on the
Avalon-dev list.  Please take a look at those messages.

> An embeddable way of handling incoming mail is desirable.  I still
don't
> think an embeddable mailet container is necessarily the best approach.
> Something that lets me accept a SMTP connection and spits out a Mail
> object for me to do as I please with is all that I require.  This
means,
> essentially, a mailet.  However, a Mailet no more necessitates the
> present whole mailet container shebang than does a JMS
MessageListener!

This just isn't true.

It's precisely because simple listener mechanisms were so woefully
inadequate for server applications (no lifecycle, management, etc.) that
the concept of a "Message Bean" was introduced in J2EE.  In fact,
mailets are fairly analogous to MessageBeans.  They are container
managed objects that respond to asynchronous events.  They maintain no
request state.  Can you think of any reason why a J2EE container would
be required to add a new API that would allow interception of the
incoming JMS message before it is passed to the container?

Again, I still just don't get your objection to a mailet container.  The
"whole mailet container" is actually pretty lightweight.  This concept
that somehow it is critical to get ahold of the mail immediately as it
comes in as opposed to a few milliseconds later when it is passed to the
spool manager is beyond me.  I just don't get what you think this
achieves?  Can you clarify?

>   I believe that the Mailet container could easily be a Mailet!
> (Composite pattern.)  Ooops - there I go again.  Never mind I am
wearing
> my asbestos suit.  :-)

Nope.  Again, you're not taking into account lifecycle issues,
management, configuration, etc.  All the hard parts of a server app.
Everything looks easy until you try and do it.  Building a mailet
version of a mailet container would basically force you to move all the
spoolmanager code into the mailet, as well as figuring out how to force
the relevant lifecycle info through the more restrictive Mailet
interface.  And what exactly is this supposed to get you...?
 
> I don't like the idea of reproducing (even some of) the Avalon
Framework
> features in a Mailet-specific clone.  This smacks of re-inventing the
> wheel.  What is it about the Avalon Framework that makes it unsuitable
> for adaptation to this use?  (Or even use without adaptation?)

Again, read my emails.  I've already addressed this there in quite some
detail.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Automating site and build updates

[Peter M. Goldstein]

> > Also, if Danny and I are too slow in making the updates to daedalus
and
> > Peter is going to be the release manager for the foreseeable future,
I
> > would think we could get him an account there.
> 
> +1
> 
> I should have the time to be release manager for the next one, or let
> Peter do it again.

If it counts for anything, +1 to giving me an account on daedalus.  :)

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java

[Peter M. Goldstein]

Danny,

Ok.  Well, let us know when you get access again.  Thanks.

--Peter

> -Original Message-
> From: Danny Angus [mailto:[EMAIL PROTECTED]]
> Sent: Monday, November 25, 2002 11:44 AM
> To: James Developers List; [EMAIL PROTECTED]
> Subject: RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java
> 
> not until I can get acces to the servers :-( some kind of network
problem
> at my end.
> 
> > -Original Message-
> > From: Peter M. Goldstein [mailto:[EMAIL PROTECTED]]
> > Sent: 25 November 2002 17:57
> > To: 'James Developers List'
> > Subject: RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java
> >
> >
> >
> > Danny,
> >
> > > A mere instant of attention is required.
> >
> > Can we get that instant of attention.  The files were uploaded
> > yesterday.
> >
> > --Peter
> >
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <mailto:[EMAIL PROTECTED]>
> > For additional commands, e-mail:
> > <mailto:[EMAIL PROTECTED]>
> >
> 
> 
> --
> To unsubscribe, e-mail:   <mailto:james-dev-
> [EMAIL PROTECTED]>
> For additional commands, e-mail: <mailto:james-dev-
> [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java

[Peter M. Goldstein]

Danny,

> A mere instant of attention is required.

Can we get that instant of attention.  The files were uploaded
yesterday.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java

[Peter M. Goldstein]

Noel,

> > In the future I hope to abandon this rather silly process.
> 
> What do you have in mind?

An automated build process (perhaps Gump/Alexandria, although that seems
to serve a different purpose) that would generate nightly builds off a
pure, unadulterated CVS of the source.  A milestone build would just be
a particular nightly build that was moved into a "milestone" directory.
This would make it easy to i) generate milestones and release and ii)
keep track of the exact binaries we've distributed.  All committers
would have necessary write access to the relevant directories, so any
committer could put out a milestone.  Releases would basically be
handled identically to milestones.  Having set this up in a commercial
environment (although we used Perforce and not CVS) I know that it's not
particularly difficult.  Especially with Ant in the picture.  But we can
hash this out later, after the current release is out the door.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java

[Peter M. Goldstein]

> In other words, Danny or Serge (or other with Daedalus access) just
needs
> to
> copy from ~pgoldstein/downloads/downloads/ to the nightly build
directory?
> I should hope that doesn't take but a matter of a moment, but perhaps
you
> might indicate specifically which file(s)?

All of them.
 
> In the future, perhaps you might address immediate needs of short
duration
> by placing the download under a public_html/ in your home directory,
so
> that
> cvs.apache.org/~pgoldstein/ is available.

In the future I hope to abandon this rather silly process.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH][VOTE] AbstractJdbcUsersRepository.java

[Peter M. Goldstein]

Noel,

> We have to support the Release.  I would prefer not to have to point
> Oracle
> users at a patch or immediately issue a point release.
> 
> +1 to commit the change pending successful testing by the Oracle user
who
> is
> working with us on james-user, as well as normal testing.

I'd prefer that too, but there are other considerations.  I'll be
willing to vote +1 on this provided that we can get the new milestone
build up and posted by 24 hours from the date of this email.  I don't
have appropriate access to daedalus, so I can't guarantee that.  If we
can get someone with that access to guarantee that, I've got no problem
with this fix.  Otherwise I've got concerns that this will further push
out an already delayed release.

To speed the process, I've applied the patch, build a dist, and am
uploading to a public directory in my home directory on cvs.apache.org.
This will enable Danny or someone else just to copy it into the nightly
build directory.

If we can't make this timeframe I'll have to vote -1.  Getting 2.1 out
the door is a very high priority, even if it means we leave some issues
unaddressed.  This issue affects only a small segment of our user base.
If necessary, we'll issue that point release.

--Peter 




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Adam,

> > To achieve this end, I'd like a Mailet API that either uses or
copies in
> > another package the equivalents of the Contextualizable, Composable,
> > Configurable, Initializable, and Disposable interfaces.  I'll leave
the
> > logging question alone for the moment.  I don't think we need any of
the
> > other phases of the Avalon Framework lifecycle (Startable,
> > Reconfigurable, etc.).  So I don't think the Mailet API should be a
full
> > Avalon Framework container.
> 
> Avoiding using Avalon Framework in the Mailet API is a good plan, I
feel.
> The
> API already has equivalents to most of the lifecycle interfaces you
list
> above:

Well, not really.  That's one of the reasons everyone want to revamp the
API.  First off, there is the logging issue.  Just search the archives
for the relevant discussions.
 
> * Mailet.init() and destroy() give you Initializable and Disposable.

True.  Though the contracts for these aren't quite as strict as their
Avalon equivalents.  Especially since init() isn't actually in the
Mailet API.  But these details can be addressed when the API is being
hammered out.

> * MailContext.getAttribute() gives you Contextualizable (as do the
other
> get
> methods on MailContext).

Here's where we'd start to disagree.  See some of the discussion on
Avalon-dev about what precisely is involved in defining a context.  Just
having a context that's a glorified hash map doesn't buy us all that
much.  This is one area I'd like to see addressed by any API/framework
we throw together.

> * Composable can be done with a simple lookup method on MailetContext:
> Object getService( String roleName );
>   or, perhaps better:
> Object getService( Class serviceInterface );

This I don't like.  The component lookup should be a part of the Mailet
interface, and separate from the context.  For basically the same
reasons the two are separate in the Avalon interfaces.  They are
conceptually different.
 
> * Configurable could be done using MailetConfig's init parameter
methods.
> Some way of getting at structured config would be better.  It doesn't
> necessarily have to be done via MailetConfig.  A good approach might
be to
> configure the mailets (and maybe the matchers, too) using a
> reflection/javabean based scheme similar to the way Ant configures its
> tasks.

I don't think MailetConfig is anywhere near good enough.  We need
something that is basically identical to the Avalon Configuration
interface - something that allows us to pass in structured configuration
data and supports defaulting and type validation.  Whether the
underlying impl (or even the interface class) is the same really isn't
the relevant point.  The functionality just has to be there.

> Just out of interest, how much scope is there in v3 for changing the
> Mailet
> API, and the way james is partitioned into components?

For the API there is tremendous scope for revamp.  One of the stated
goals for v3 is a new version of the Mailet API.  Input is welcomed.  :)

As far as repartitioning James component-wise, I haven't heard much
discussion of this aside from my comments about RemoteDelivery.  I'm
sure we'd be open to discuss, although I don't think anyone wants to
rewrite everything from scratch.  So the reaction to a repartitioning
request would likely depend on the stated goals as well as the amount of
effort involved.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Noel et al,

> > Unfortunately [embedding] is a limitation of the Avalon framework
> 
> Actually, Merlin and Fortress are embeddable or going that way, so I
don't
> view that as an issue.  Remember, Cocoon embeds inside of Tomcat.

Agreed.  Embedded containers are one of the big themes on the Avalon-dev
list, third on the list after discord and sarcasm.  :)

Moreover, there is an EJB container that is embeddable inside Phoenix,
there is an active effort to embed Tomcat inside Phoenix, etc.  So not
only is the idea to embed Avalon framework containers inside other
containers, but also to embed other types of containers (EJBs, servlets,
etc.) inside Avalon framework containers.  It's almost Escher-esque.
 
> I don't think that the current version of James gets down to quite as
trim
> a
> package as we can build a mailet container, but that doesn't mean that
we
> need a separate codebase.  Let's just accept as a request for v3 that
> being
> able to embed a mailet container is a good thing, and see where that
takes
> us.

I agree.  But my approach would be to delegate the embeddedness to our
Avalon framework container.  There should be nothing in the Mailet API
that prohibits such embeddeding, but I don't think we need to re-invent
the wheel.  Merlin/Fortress/Avalon Container++ can handle this.

> Personally, I have some ideas but we'll see how this falls out.  I
will
> point out that you appear to be leaning towards making Avalon a part
of
> even
> embedded Mailet Containers.  I'm curious to see how Danny and Peter
> respond.
> And, of course, if I am wrong please clarify.

I actually (as I've mentioned before) really like the Avalon framework
lifecycle.  That said, I don't see any reason that we should build rings
within rings.  With the theme of not reinventing the wheel, I don't
think we need to re-implement Phoenix inside Phoenix.  The mailet
container doesn't need to support components of arbitrary complexity,
provided it has an effective API for getting other components in the
larger container (Avalon or another type of container).

For example, the RemoteDelivery mailet could be rewritten as a more
lightweight, single threaded class that passes the mail to a
RemoteDelivery component that is managed directly by Phoenix.  This
would allow the mailet to be kept nice and simple, the RemoteDelivery
component would be made easily pluggable, and the component could take
advantage of features of the container (i.e. JMX) that we might not want
to expose directly through the Mailet API.  This approach seems very COP
to me.

To achieve this end, I'd like a Mailet API that either uses or copies in
another package the equivalents of the Contextualizable, Composable,
Configurable, Initializable, and Disposable interfaces.  I'll leave the
logging question alone for the moment.  I don't think we need any of the
other phases of the Avalon Framework lifecycle (Startable,
Reconfigurable, etc.).  So I don't think the Mailet API should be a full
Avalon Framework container.

But after 2.1 goes out we'll see what Danny's got written up and start
from there.  If it provides the same basic functionality as the Avalon
lifecycle phases above, that would probably be ok with me. 

--Peter
 



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: James and LDAP

[Peter M. Goldstein]

Kris,

> I'm interested in getting James tied into my company's LDAP service. I
> know
> that the documentation update is in progress, so I perused the source
to
> get
> a feel for how to do this. That's when I saw this:
> 
>  * This clas is a dummy for the proposal!
> 
> Having looked through the mail archives as well, and found little
chatter
> about the LDAP support, I am curious to know if it was deferred or
> abandoned
> for a future with JAAS.

I don't think there has been any active development lately, and I don't
think it's been because of a conscious deferral for a future with JAAS.
I think it's basically not been a high priority because of a lack of
demand.
 
> More importantly though, this one is a need for me, so I'm going to
> implement it. I would like to get in line with whatever direction the
> project may already be headed, so I don't create something useless to
> everyone else. Feedback?

This is a great time for you to get involved.  Part of the v3 design
will be a discussion of user repository interfaces.  We're about to
start those discussions.  An LDAP perspective would be valuable.

If your time frame is short, I'd encourage you to implement the current
UserRepository interface and check the behavior.  We can then morph it
along with the other repositories when we do the redesign.

> BTW, great piece of software, it's been very helpful. That's why I'm
here.

Glad to hear it.

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Release Plan v2.1 (update)

[Peter M. Goldstein]

All,

> Documentation and testing comprise the bulk of the remaining work.  We
> still need to carry out the following documentation tasks:
> 
> i) Document the configuration files in toto.  Format the documentation
> consistently, as well as vet it for correctness.

In process.  Assigned to me.  I will be checking in some stuff tonight.

> ii) Document the top 5 items on the TODO list

In process.  Assigned to me.

> iii) Add FetchPOP documentation

I'd really like someone to contribute on this one.  Volunteers?
 
> iv) Produce a consistent documentation package for the 2.1 version
that
> details the product services, as well as the basic steps of
installation
> and configuration.
> 
> Build tasks:
> 
> i) Build a milestone build for testing.

Done.  And posted.
 
> ii) Adjust the CVS to allow future maintenance of the 2.1 branch while
> permitting radical change for the 3.0 revision.

I'd like a volunteer for this as well.  I'm not CVS saavy, but I'll do
it if I have to.  Basically I'd suggest that we form a release/2.1
directory off the base directory that contains all the necessary files
to build a 2.1 release.  This will allow bugfixing on that release.

> i) Run general tests on a variety of platforms.  We've already done a
> great deal of testing on both Windows and Linux platforms, but it
would
> be good to get a great deal more testing done before the official
> release.  We should look to our user base for potential testers.

The user base has started doing some testing.  A request for feedback
has been posted on the web site.  Not sure if anyone else wants to sign
up for more testing.

> ii) Test TLS support for POP3 and update the status from experimental
to
> stable.

Any volunteers?

> Marketing tasks:
> 
> i) Produce a press release for distribution through channels
announcing
> both the release of 2.1 and inviting developers to participate in
> version 3.0.

In process.  Assigned to me.

> ii) Distribute this release through channels (i.e. Slashdot)

In process.  Assigned to me.

> iii) Ensure that we are listed on common sites that provide lists of
> available mail/news servers.

Any volunteers?

> I'm working on items i, ii, and iv on the documentation task list, as
> well as items i and iii on the marketing task list.  Anyone who wants
to
> step up for any of the other items, please do.  I'd like to ask Danny
> specifically to address the milestone build issue, since he has write
> access to the web site.  If he can get me such write access, I'd be
> happy to build and post the milestone build.

I haven't really gotten any response from any of the
committers/contributors to this request for contributions.  If we're
going to get this out the door, it's going to require some effort from a
number of interested parties.  I know this stuff doesn't interest most
of us (it certainly doesn't interest me) but it's part and parcel of
putting out an enterprise class product.  

> Here's my desired schedule:
> 
> November 4th, 2002 - Creation and posting of milestone build

Completed November 13th, 2002.

> November 4th-17th, 2002 - Testing period.  Aggressive testing of James
> under a variety of conditions.

Testing can be considered to be started November 13th, which would lead
to a close of this period on November 26th.  There has been some testing
to date, 

> November 8th, 2002 - Documentation freeze, which requires completion
of
> all documentation tasks.

We're way off on this one.  :)

I'm still not done with the segments of documentation I signed up for,
and no one else has picked up the remainder.  I'd like to set a nominal
freeze date of November 26th, but unless we get some contributions,
that's not going to happen.

> November 15th, 2002 - Completion of press release, list of sites to
> which the release needs to be distributed.  Mail server info sites
> should be updated with links at this point.

I'd say this should be November 30th.  But it depends on all the earlier
steps.  And hence depends on people contributing.

> November 18th, 2002 - Release.  Update of web site to reflect newly
> released version.  Distribution of press release to sites should be
> completed immediately after release.

I'd say December 2nd for this.  But again, this depends on everyone
stepping up and contributing.

Thoughts?  Replies?  Anyone?  :)

--Peter

P.S.: I understand that Thanksgiving falls in the middle of this, but
there just isn't that much left to do.  One last push should get us over
the edge.




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Aaron,

> Hmmm.  I don't mean to be argumentative, or to divert JAMES off on my
own
> private development direction, but I do not believe that my goals in
any
> way conflict with the current JAMES project goals.

I don't mind some argument, and personally I don't find your goals to be
the problem.  It's the approach I find problematic and ill-advised.
 
> Basically, my goal is to receive and process mail in-process with my
own
> app, as simply as possible and with as little overhead as possible.
JAMES
> appears to provide all of the building blocks required to do this -
they
> just need to be extracted (which doesn't seem all that hard).

This goal can be achieved with the current design quite simply.  No
additional APIs necessary or desirable.  To wit, run your app as an
Avalon component within Phoenix.  This takes advantage of a pre-existing
API that is both widely used and well tested - the Avalon Framework.
Have your mailet access your app directly, and communicate with your
custom API that's part of your app.  Piece of cake.  And it's something
that, in one form or another, we've committed to support in the next rev
of the mailet API.

I'd strongly suggest that you read the Avalon/Phoenix docs to get a
clear picture of how something like this would work.

> I will comment on both Danny's and Noel's replies, here.

Not sure why my reply doesn't merit a response, but I'll reply to this
nonetheless.

> 
> the goal of the James project to provide James as a complete solution,
not
> a set of components.
> 
> 
> Yes, ok.  The JAMES project, as a whole would still produce a complete
> mail
> server, assembled from a set of components, which would include those
> provided by Avalon and those provided by JAMES sub-projects.

Fine, although a sub-project would need to be justified first.  The
additional complexity of a sub-project would need to be justified by one
or another meaningful goal that are currently unachievable or difficult
to achieve with the current product.  So far I haven't heard any.

> 
> modify the handler for incoming SMTP such that it could be extended by
> "verb classes" to support new SMTP verbs.
> 
> 
> Unless I misunderstand you, this would allow for custom extensions to
the
> SMTP protocol.  This is not the goal that I am aiming at.  I want to
avoid
> having to have a separate process, with the implied inter-process
> communication, for receiving mail.

You are correct.  Danny's suggestion really doesn't help for your stated
goal - mail processing in-process - unless you want to hijack the entire
SMTP dialog.  But allowing that capability seems to be part of the API
you suggest (see my earlier email) which I find extremely problematic.
There are right and wrong ways to do such things, and I'm pretty sure
publishing a public API where the consumer of the API is required to
assure RFC compliance is the wrong way.

> 
> I believe that the approach desired by James would be an embedded
Mailet
> Container
> 
> 
> If I understand the distinction you are making, then this would
involve
> embedding a much more heavy-weight component than necessary for a lot
of
> cases.  Correct me if I am wrong, but this would mean both an SMTP
> listener
> (for receiving the mail) and a mailet container for processing it
would be
> embedded.  That would give you all of the mailet chaining stuff, etc.
but
> this is not necessary in many cases (although I do not deny that it
would
> be useful in many others).

This is not a correct interpretation.  The SMTP server component and the
mailet container are distinct components.  That's not an accident.

The SMTP server component is responsible for SMTP communication.  The
SpoolManager is responsible for mail processing.

> 
> I think that your original goal, to facilitate projects that
> need to receive and process in-board e-mail, is something to support.
I
> just disagree with your initial proposal to solve that problem.
> 
> 
> I am open to suggestions.  I lean away from your embedded mailet
container
> in the form that I have described it above, due to it's being too
> heavy-weight.  Perhaps your tuple-spaces idea could be used to aid
with
> this?  Have James receive the SMTP connections and write it to a
> persistent
> spool, and have other apps able to receive that mail and process it by
> reading mail from the spool - a kind of publish/subscribe scenario?  I
> haven't thought about this one very hard, but it seems to dovetail
nicely
> with your massively scalable distributed JAMES idea from last week.  I
> still think that an embeddable SMTP interpreter would be useful in
> addition
> to this, however.

See my comments above.  Everything you want to do is already doable.  No
need for additional sub-projects, API, or what-not.  The SMTP server
publishes messages to a spool.  The SpoolManager picks those messages
off the spool (publish/subscribe, in process) and sends them to the
first processor.  Your custom mailet, which does whatever you want it to
do, sits 

RE: JAMES SMTP API sub-project proposal

[Peter M. Goldstein]

Aaron,

> I have come across several projects with requirements to receive and
> process inbound mail.  Currently, the options for implementing this
(in
> java) are a JAMES mailet, or some kind of POP3/IMAP client.  However,
I
> would like to have a third option of embedding a SMTP listener into my
> application.
> 
> I propose that the JAMES SMTP handling code be split out from the
JAMES
> server and packaged as a separate API, which JAMES itself uses and
which
> can used separately from the rest of JAMES.
> 
> Requirements for this API would include:
> 
> *The API must have no reliance on the rest of JAMES.
> *The API must be entirely suitable for use by JAMES, without the
need
> for any 'special case' concessions being made by either side.
> *The API must not manage it's own threads.  (The embedding
application
> should be free to control its own thread creation/destruction.)
> *The API must not manage it's own sockets. (Again, opening of
> ServerSockets should be left to the parent application.)
> *The API is responsible for conducting an SMTP converstion over a
pair
> of streams (one in, one out) and creating mail objects, which are
passed
> to
> a handler (implemented by the parent app).
> *My own requirement is for inbound communication only, by does not
> preclude outbound communication if anyone else thinks that this would
be
> good.
> *The API is responsible for compliance with the relevant RFC's
with
> respect to communications.  Where the RFC's specify mail server
behaviour,
> enough information should be passed to the parent app to allow it to
> implement the required behaviour.
> 
> I am willing to do the work required to extract this API from the
current
> JAMES codebase, with a view to its being made a JAMES sub-project.
(This
> would be on my own time, so results may not be instantaneous.)  Before
> making any start on such an endeavour, I require two things from the
JAMES
> developer community:
> 
> 1)   Acceptance that this is a desirable direction for JAMES to take.
> 2)   Some further discussion/ratification of the requirements and the
form
> that this API should take.  (e.g. public interface to the API.)
> 
> Any help would also be appreciated (of course).
> 
> If this kind of thing is aided by the promotion of JAMES to a top
level
> project, then that proposal get a +1 from me (else +0).  The separate
test
> project proposed by Harmeet also gets a +1 from me (subject to the
> availability of developers willing to do the work).  These last two
points
> count regardless of whether this particular proposal is accepted, of
> course.
> 
> 
> What say you all?

IMHO, this is a terrible idea.  The mailet API was designed to do
exactly what this proposal specifies as the primary purpose of the new
SMTP listener API (receive and process inbound mail).  There is no
compelling other reason presented for developing and exposing a brand
new API.  Developing, exposing, and supporting a new API has all sorts
of implications that I don't believe have been properly thought out.
Those include backwards compatibility, performance, robustness, etc.

Most of the behavior described here is not appropriate to an API - i.e.

> *The API is responsible for conducting an SMTP converstion over a
pair
> of streams (one in, one out) and creating mail objects, which are
passed
> to
> a handler (implemented by the parent app). 
> *The API is responsible for compliance with the relevant RFC's
with
> respect to communications.  Where the RFC's specify mail server
behaviour,
> enough information should be passed to the parent app to allow it to
> implement the required behaviour.

which is arguably appropriate to a provider architecture (ala JDBC,
JSSE, or JCE) but is clearly not something we should expect writers to a
an API to be responsible for. 

Moreover, there are a number of enhancements to the SMTP service that
are desired by the user community (i.e. 8BITMIME support, configurable
additional extensions a la sendmail/qmail, additional auth options) and
would be impeded if we stuck a brand new API in the middle of the SMTP
logic.

I view this as a highly undesirable direction for James to take, and
hence would -1 any changes as described here.

--Peter






--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [VOTE] James as an official Apache project

[Peter M. Goldstein]

All,

> I propose Serge to have the honor of being the initial PMC Chairman,
and
> all
> Committers as the initial PMC, unless they wish to demure.

+1
 
--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: cvs commit: jakarta-james/www index.html

[Peter M. Goldstein]

Yep, please do.  Thanks.

--Peter

> -Original Message-
> From: Danny Angus [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 20, 2002 2:17 PM
> To: James Developers List
> Subject: RE: cvs commit: jakarta-james/www index.html
> 
> peter, also commit the updated xdoc, do you want me to "cvs up" on
> daedalus?
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Introduction (was Re: cvs commit: jakarta-james/proposals/imap/java/org/apache/james/imapserver/commands CommandTemplate.java)

[Peter M. Goldstein]

Darrell,

> > 1) Conceptual misunderstanding of the role of the message sequence
> > numbers.  MSNs cannot be associated with a mailbox object, because
they
> > are a per connection, not per mailbox, concept.  It's more than
possible
> > for two different connections to have different MSN=>UID maps for a
> > given mailbox.  Mailboxes should only know about UIDs.
> 
> You may be right, but can you give an example of where 2 different
> connections
> would have different MSN's for the same underlying mailbox. From my
> understanding, the rule for mapping MSN=>UID is strictly a contiguous
> sequence in ascending order of UIDs. This doesn't allow allowing
different
> mappings. The MSN=>UID mapping may change between sessions when the
> underlying mailbox has changed, which may be what you meant by
> "connections".

It's a little more complicated than this.  Consider the following
situation.

Two separate connections (sockets) are opened by clients.  Over these
connections they log in, etc., and select the same mailbox.

One user proceeds to run a series of SEARCH commands.  According to
section 7.4.1, no EXPUNGE response can be sent during this series of
commands.  This is necessary so the msns can be kept in synch between
the client and the server.  So this client maintains the original set of
msns as seen upon selection of the mailbox.

Meanwhile, the other user stores a \Deleted flag for some messages,
using a STORE.  The user then executes an EXPUNGE command, which results
in these \Deleted messages being cleared from the mailbox and a series
of EXPUNGE responses being sent.  This changes the set of msns.

Now we have two connections, with different MSN=>UID mappings.

What will happen, for a well behaved client, is that the client in the
first case will received an EXISTS response when the size of the mailbox
changes.  After completing the last sent SEARCH command and before
sending the next one, it will issue a NOOP or other command that allows
the client to reset the msn sequence.  But even this behavior isn't
required by spec.  See section 5.5 for more discussion of this.


> > 11) Information is duplicated between the ImapRequest and
ImapSession
> > interfaces.
> 
> 11a. ImapSession interface is a travesty. Originally, there was no
> interface
> between SingleThreadedConnectionHandler and the commands - at least
this
> makes it clear what the commands are using (and not).

I've actually fleshed out ImapSession some.  I definitely don't think
the ImapCommands should be accessing SingleThreadedConnectionHandler
directly.  Basically the ImapSession now allows the commands to access
session specific data (session state, current user, currently selected
mailbox, etc.) as well as provides some basic utilities used by multiple
commands (generate an untagged response, generate an expunge response,
get a mailbox, etc.)

> a) Every command without an rfc protocol test doesn't work.
> b) Some of the commands with rfc protocol tests may not work.
> - tests are incomplete
> - tests fail
> c) Many browsers may not work, even when the rfc protocol tests do.
> d) Licence headers need to be full Apache licence (I think).
> e) Test suite should be updated to use regular expressions rather than
> dinky
> custom codes. (All my fault, I'm afraid).
> 
> Plenty to keep us busy, anyhow.

Definitely.  :)  I'm looking forward to it.  But for now, 2.1 awaits
completion...

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: MailRepository

[Peter M. Goldstein]

Noel and Aaron,

> As I just said in James-User (but let's move the discussion over to
here):
> 
> Seems to me that you don't need what you've done.  A mailet and the
> internal
> spool would provide sufficient decoupling to generate SMS to an EJB
> backend.
> 
> Just have the mailet do this when the service method is called.  If
you
> need
> something else, just clone what RemoteDelivery does, which is
basically
> the
> same as your mailet.  It uses an internal spool when it can't
immediately
> deliver.  We could probably refactor RemoteDelivery into a base class
for
> a
> certain category of mailet, and then provide methods for performing
the
> remote delivery, e.g., to another SMTP server, to your EJB server, to
> another server.

I agree with Noel - I don't understand why the special spool is
necessary.  Use of a custom mailet to generate your EJB calls seems more
than sufficient.

> > I think the JAMES mail repository needs to be exposed to mailet
authors.
> 
> I agree that repository access needs to be provided, but I do NOT that
the
> Repository interface currently present is what ought to be used,
either by
> James or by Mailets.  I also agree that whatever the interface to
> repositories, James and Mailets ought to use the same one.

I agree with all of this.  I'll take it one step further.  There needs
to be a way for mailets to access all components provided by the server.
This should be supported by the mailet API.  If a required component is
not available the mailet needs to be able to throw an exception on
startup to notify the mailet container.

As far as the repository interfaces go, they all need a lot of work.  In
James 3.0 I expect we will be making substantial revisions to these
APIs.  I know Noel and Serge have a lot to say about the mail
repositories (properties/attributes) and I have more than a few thoughts
on the user repositories.  I have no idea what anyone else has to say,
but I'd be pretty naïve if I didn't believe that Danny and others will
champion certain modifications.  I also think that support for IMAP and
NNTP in the standard repositories will force certain restrictions on
these interfaces.  So calling these interfaces in flux would be an
understatement. :)

 > > We would like to be able to define an arbitrary mail repository,
store
> > incoming mail into it and process that mail asynchronously.
> 
> See above.  It sounds as if you team is going about it all wrong.
Please
> read through the RemoteDelivery mailet.

You should be able to do this, but I concur with Noel.  Why would you
want to do this for your app?

> We're very close.  I'm hoping that Peter (v2.1 Release Manager) can
post a
> Release Plan Status, so we can talk about who is doing what to help
push
> this to completion and get into some really fun new stuff.

Will do.  We are close.  Watch for it soon.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH/VOTE] - POP3 Performance/Functionality Fix.

[Peter M. Goldstein]

Serge,

> +1 as long as you've got flush's in all the right places.

Yep.  We use a BufferedReader everywhere else, so the flushes were
already in place (see some of the late stage bug fixes on POP3 for this
revision).  It's only in doTOP and doRETR that we directly access the
underlying OutputStream, and thus only there that additional flush()
calls were required.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

[PATCH/VOTE] - POP3 Performance/Functionality Fix.

[Peter M. Goldstein]

All,

This is a fix for the observed performance problem with POP3 on some
platform JVM combinations.  Basically it amounts to adding a
BufferedOutputStream around the OutputStream produced by the socket.
This also seems to resolve this issue encountered by JRC (Randy),
although the reason for this is less clear.

As we are past code freeze, this requires a vote of the committers to be
accepted.  I think the problem is severe enough (as it is impacting our
user community) and the fix is localized enough to merit a patch, so I
vote +1.  Thoughts?

--Peter

Index: jakarta-james/src/java/org/apache/james/pop3server/POP3Handler.java
===
RCS file: 
/home/cvs/jakarta-james/src/java/org/apache/james/pop3server/POP3Handler.java,v
retrieving revision 1.17
diff -u -r1.17 POP3Handler.java
--- jakarta-james/src/java/org/apache/james/pop3server/POP3Handler.java 2 Nov 2002 
09:03:52 -   1.17
+++ jakarta-james/src/java/org/apache/james/pop3server/POP3Handler.java 12 Nov 2002 
+07:24:45 -
@@ -238,7 +238,7 @@
 }
 
 try {
-outs = socket.getOutputStream();
+outs = new BufferedOutputStream(socket.getOutputStream(), 1024);
 out = new InternetPrintWriter(outs, true);
 state = AUTHENTICATION_READY;
 user = "unknown";
@@ -846,8 +846,6 @@
 writeLoggedFlushedResponse(responseString);
 return;
 }
-//?May be written as
-//return parseCommand("TOP " + num + " " + Integer.MAX_VALUE);?
 try {
 MailImpl mc = (MailImpl) userMailbox.elementAt(num);
 if (mc != DELETED) {
@@ -859,6 +857,8 @@
   theWatchdog,
   
theConfigData.getResetLength());
 mc.writeMessageTo(nouts);
+nouts.flush();
+// TODO: Is this an extra CRLF?
 out.println();
 out.println(".");
 out.flush();
@@ -888,7 +888,6 @@
 responseString = responseBuffer.toString();
 writeLoggedFlushedResponse(responseString);
 }
-// ---?
 } else {
 responseString = ERR_RESPONSE;
 writeLoggedFlushedResponse(responseString);
@@ -935,6 +934,7 @@
   theWatchdog,
   
theConfigData.getResetLength());
 mc.writeContentTo(nouts, lines);
+nouts.flush();
 out.println(".");
 out.flush();
 } else {


--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH/VOTE] - MX Chaining fix.

[Peter M. Goldstein]

Noel,

> I've now read the bug report and the code.  Seems reasonable.
However,
> there are useful comments in the bug report that probably ought to be
in
> the
> code (or at least in the CVS).
> 
> Specifically: "there should be a separate try/catch block around the
> transport.connect() statement, to handle connect errors separately
from
> errors that arise during message transmission.  Connect errors almost
> always
> indicates that further SMTP servers associated with the MX record
should
> be
> tried, while errors in message transmission are generally
protocol-level
> errors that would occur with any SMTP server associated with the MX
record
> (the exception being IOExceptions that indicate a failure in the
> underlying
> transport)" provides more elaboration than the current comments.
> 
> I guess I'm not a big fan of having the bug database as the only (or
best)
> source for why the code is as it is.

Ok.  I thought the comment I dropped in the code was sufficient, but we
can certainly cut and paste this into the code itself.

Personally I think we under use Bugzilla as a repository for useful
information.  This is exactly the sort of information a bug tracking
system is supposed to track.  :)

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Introduction (was Re: cvs commit: jakarta-james/proposals/imap/java/org/apache/james/imapserver/commands CommandTemplate.java)

[Peter M. Goldstein]

Nicola,

> May I humbly siggest that you and other developers add some personal
> info about themselves on their homepage?
> 
> It's because I will probably start linking them from a page that lists
> all Apache committers.
> 
> Yours should be http://cvs.apache.org/~darrell/ ;
> 
> For example mine is:
> http://cvs.apache.org/~nicolaken/
> 
> Log into cvs.apache.org and look in my home dir as a "template" if you
> wish.
> 
> Anyway, just a suggestion.

Probably a reasonable idea.  I'll throw one together for myself after
2.1.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Introduction (was Re: cvs commit: jakarta-james/proposals/imap/java/org/apache/james/imapserver/commands CommandTemplate.java)

[Peter M. Goldstein]

Darrell,

> I thought I'd better (re)introduce myself, as I've been inactive in
James
> since February or so, and there seem to be a few new names around
(which
> is
> fantastic). I'm currently looking for work in Brisbane, Australia, and
> given
> the state of the job market I might have a fair bit of time for open-
> source
> coding coming up.

Nice to meet you.  :)  I'm Peter.

 
> I plan to do a bit more work on the IMAP proposal in the next few
months.
> How
> much work, I can't be sure, and please feel free to help out, with
> contributions to code, constructive criticism, or flame mail ;).

I'd love to coordinate with you and others on this one.  I've been
looking at the IMAP code for a month or so now and have been working on
patches for a number of issues.  Among the issues I've found are:

1) Conceptual misunderstanding of the role of the message sequence
numbers.  MSNs cannot be associated with a mailbox object, because they
are a per connection, not per mailbox, concept.  It's more than possible
for two different connections to have different MSN=>UID maps for a
given mailbox.  Mailboxes should only know about UIDs.

2) Lack of handling for the "literal" string argument type.  Basically,
the spec allows (and requires in the case of
internationalization/localization support, since quoted strings are
restricted to US-ASCII) the use of {} as an argument to
specify that the server should send a continuation response and take the
next  as the string data.

3) Mailbox name encoding (as per spec) is not supported

4) The Search command is not implemented

5) Support for the special "INBOX" mailbox alias is not present

6) Three of the commands, for no reason I can discern, are not in the
commands package.  These commands also don't seem to take advantage of
any of the encapsulated methods in SingleThreadedConnectionHandler.
This led to at least one set of malformed NO responses, and a lot of
duplicate code.  I'm assuming these classes predate the others, and
simply need to be modified/moved.

7) The SingleThreadedConnectionHandler class has bad code at the point
where commands are read off the wire.  The whole
"setCanParse/getCanParse" concept violates basic threading techniques
(if the connection handling is single threaded it's superfluous, if
multi-threaded it's a race condition) and there is a busy loop at this
point.

8) EXPUNGE responses are being sent after command completion responses,
in violation of spec.

9) MSN/UID set handling was incorrect - using a '-1' as an end delimiter
in some cases as opposed to producing a correct range.  No real reason
to do this - the comment about Microsoft Outlook's behavior is not
correct, as Outlook was behaving according to spec.

10) Comments are almost wholly absent.

11) Information is duplicated between the ImapRequest and ImapSession
interfaces.

12) Despite not being ConnectionHandlers, many of the command objects
extended BaseConnectionHandler (all functionality was unused).  The
command hierarchy in general is a little confused, with both BaseCommand
and CommandTemplate appearing to assert central roles.  Usage and
argument validation mechanisms are not unified and, in at least a few
cases, will need to be redesigned to account for point 2 above.

13) Lots and lots of "magic strings".

14) The partial fetch (allowing one to specify octets to be fetched) is
not implemented.

15) The highest UID stuff is not currently implemented for mailboxes -
highest UID is null on start.

This is the "off the top of my head" list - I think there were a few
other issues I ran into.  I'll have to look at the code again to be
sure.  Any other issues you're aware of?

2.1 has been my first priority, so I haven't done as much here as I
would've liked.  But I do have partial or complete fixes for much of the
above - as well as proper integration with the AbstractJamesService a la
the other services.  Once 2.1 is actually out the door, I'll post my
changes to date for comment, etc.  I'd really like to see some activity
on IMAP.  Originally I was hoping to see it in 2.1, but with the number
of serious open issues and the lack of concerted effort it became clear
to me that this wouldn't be the case.  At this point I'm hopeful that it
will be part of the next release.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

[PATCH/VOTE] - MX Chaining fix.

[Peter M. Goldstein]

All,

This is a fix for bug #14381.  MX Chaining is broken for the reasons
explained in that bug report.  This patch separates the connect and the
send stages of the delivery and simplifies a large block of exception
handling code.  It's deliberately a fairly minimal change.  Were this
earlier in the cycle I'd suggest a more radical code change.  But this
one solves the problem and presents minimal risk.

As we are past code freeze, this requires a vote of the committers to be
accepted.  I think the problem is severe enough and the fix is localized
enough to merit a patch, so I vote +1.  Thoughts?

--Peter

Index: jakarta-james/src/java/org/apache/james/transport/mailets/RemoteDelivery.java
===
RCS file: 
/home/cvs/jakarta-james/src/java/org/apache/james/transport/mailets/RemoteDelivery.java,v
retrieving revision 1.31
diff -u -r1.31 RemoteDelivery.java
--- jakarta-james/src/java/org/apache/james/transport/mailets/RemoteDelivery.java  
 26 Oct 2002 04:01:12 -  1.31
+++ jakarta-james/src/java/org/apache/james/transport/mailets/RemoteDelivery.java  
+ 8 Nov 2002 23:38:52 -
@@ -195,17 +195,17 @@
 Iterator i = targetServers.iterator();
 while ( i.hasNext()) {
 try {
-String outgoingmailserver = i.next().toString ();
+String outgoingMailServer = i.next().toString ();
 StringBuffer logMessageBuffer =
 new StringBuffer(256)
 .append("Attempting delivery of ")
 .append(mail.getName())
 .append(" to host ")
-.append(outgoingmailserver)
-.append(" to ")
+.append(outgoingMailServer)
+.append(" to addresses ")
 .append(Arrays.asList(addr));
 log(logMessageBuffer.toString());
-URLName urlname = new URLName("smtp://" + outgoingmailserver);
+URLName urlname = new URLName("smtp://" + outgoingMailServer);
 
 Properties props = session.getProperties();
 //This was an older version of JavaMail
@@ -227,7 +227,18 @@
 Transport transport = null;
 try {
 transport = session.getTransport(urlname);
-transport.connect();
+try {
+transport.connect();
+} catch (MessagingException me) {
+// Any error on connect should cause the mailet to 
+attempt
+// to connect to the next SMTP server associated with 
+this MX record,
+// assuming the number of retries hasn't been 
+exceeded.
+if (failMessage(mail, me, false)) {
+return true;
+} else {
+continue;
+}
+}
 transport.sendMessage(message, addr);
 } finally {
 if (transport != null) {
@@ -240,7 +251,7 @@
 .append("Mail (")
 .append(mail.getName())
 .append(") sent successfully to ")
-.append(outgoingmailserver);
+.append(outgoingMailServer);
 log(logMessageBuffer.toString());
 return true;
 } catch (MessagingException me) {
@@ -254,34 +265,22 @@
 log(exceptionBuffer.toString());
 //Assume it is a permanent exception, or prove ourselves 
otherwise
 boolean permanent = true;
-if (me.getNextException() != null && me.getNextException() 
instanceof java.io.IOException) {
+if ((me.getNextException() != null) && 
+(me.getNextException() instanceof java.io.IOException)) {
 //This is more than likely a temporary failure
 
-//If it's an IO exception with no nested exception, it's 
probably
-//  some socket or weird IO related problem.
-permanent = false;
-}
-if (me instanceof SendFailedException) {
-SendFailedException sfe = (SendFailedExcept

RE: DO NOT REPLY [Bug 14396] New: - ExtraDotOutputStream doesn't properly implement dot stuffing

[Peter M. Goldstein]

Danny,

> Peter, I may be wrong but I thought it was only CRLF. that *should*
get
> stuffed.

Exactly.  That's the bug.  LFLF. and CRCR. would both get
inappropriately stuffed.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Release Plan v2.1

[Peter M. Goldstein]

Nicola,

While I appreciate the offer, I don't think now is the time.  The idea
of being drawn into the web site creation tool argument that plagues the
Avalon-dev list from time to time is a personal nightmare.  It would
certainly kill any chance of meeting the proposed release schedule.

This rev will use the currently employed tool in the James build.  I
don't even know exactly what we're using, but whatever it is is what
we'll be using through this release.  Output documentation will be in
the form of a few simple linked HTML pages.

--Peter

> -Original Message-
> From: Nicola Ken Barozzi [mailto:nicolaken@;apache.org]
> Sent: Monday, November 04, 2002 11:34 PM
> To: James Developers List
> Subject: Re: Release Plan v2.1
> 
> 
> Alan Gerhard wrote:
> >
> > -Original Message-
> >
> >>iv) Produce a consistent documentation package for the 2.1 version
that
> >>details the product services, as well as the basic steps of
installation
> >>and configuration.
> >
> >
> > what is the current state of this document package, I mean are we
> looking at
> > generating a whole new set and if so, based on a particular format
??
> 
> I'm willing to help you setup the site using Forrest
> http://xml.apache.org/forrest/  .
> 
> It gives you automatic PDFs for every page, it's skinnable (we're
almost
> ready to have a skin like the one used on the avalon site, thanks to
> Peter Donald) and you can now edit the pages and see the result in
> realtime in the browser since we include the Jetty server in the
distro.
> 
> Examples:
> 
> xml.apache.org/forrest
> incubator.apache.org
> xml.apache.org
> 
> Coming soon:
> 
> XIndice
> Cocoon
> FOP
> Apache Commons
> POI
> other xml.apache projects
> etc...
> 
> --
> Nicola Ken Barozzi   [EMAIL PROTECTED]
>  - verba volant, scripta manent -
> (discussions get forgotten, just code remains)
> -
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: RFC 2554

[Peter M. Goldstein]

Uwe,

This has been discussed at great length on both the james-user list and
the james-dev list.  Please consult the archives to view these messages.

James 2.1 is in full accord with the RFC, supporting both clients that
use the optional initial response argument and those that don't.  I'm
not sure what you're talking about as far as the colon goes, but I'd
strongly suggest that you more carefully examine the relevant RFCs and
the source code to SMTPHandler.java to resolve your misunderstanding.

--Peter

> -Original Message-
> From: Uwe Rosebrock [mailto:uwe@;rosebrock.org]
> Sent: Sunday, November 03, 2002 7:10 PM
> To: [EMAIL PROTECTED]
> Subject: RFC 2554
> 
> Hi guys,
> 
> parameter separation after AUTH PLAIN
> 
> I have recently encountered a discrepancy in a number of e-mail
clients
> and servers  using the
> AUTH PLAIN command - JAMES expects a parameter after this command is
> issued to be separated by colon (for adding the Base64
username/password)
> MacOsX Mail, Calypso,Outlook separate by space after AUTH PLAIN
 username\0password> - an attempted of sending mail with authentication
> fails therefore. The same happens using other e-mail servers.
> Can anybody enlighten me where I am going wrong here?!
> Uwe
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Release Plan v2.1

[Peter M. Goldstein]

Noel,

> Two config file items:
> 
>  1) Add Danny's patch to sqlResources.xml (and document it for
existing
> databases)
>  2) Increase number of max db connections
> 
> I would certainly do #1.  #2 already has a comment telling people to
> increase it, but I'd still consider increasing the default from 10 to
20,
> since 20 was able to support stress testing with 20 clients and 10
spool
> workers.

Agreed.  Will roll in by EOD.

 
> I would immediately put out a milestone build for all parties willing
to
> act
> as test sites.  Consider it a release candidate.

I'd love to.  As I said in my earlier email, I don't have write access
to the web site so I can't post a release candidate.  Hoping to get
Danny to do it...

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Release Plan v2.1

[Peter M. Goldstein]

All,

One more item that I left off the list -

Build Tasks:

iii) Updating the changelog to reflect the changes in version 2.1

I'm taking care of this one as well.

--Peter

> -Original Message-
> From: Peter M. Goldstein [mailto:peter_m_goldstein@;yahoo.com]
> Sent: Saturday, November 02, 2002 3:58 PM
> To: 'James Developers List'
> Subject: RE: Release Plan v2.1
> 
> 
> All,
> 
> Well, here's my suggested scheduler for release 2.1, given the
> developments to this point.  Let me first summarize what we've gotten
> done, then what remains to be done before release, and then lay out
the
> schedule I think is realistic.
> 
> We have, at this point, achieved all the code specific goals discussed
> in the release plan of September 28th.  We have replaced the
> ConnectionHandler infrastructure, addressed the TimeScheduler issue,
and
> closed all the code specific bugs in item #4 of the earlier release
> plan.  We have also made a number of substantial performance and
memory
> usage improvements over the last month, as well as bringing the NNTP
> server into better agreement with the specification.  We've upgraded
our
> distribution to Phoenix 4.0.1, resolving the logging bug.  The
javadocs
> have also been brought up to speed.  As far as I can tell, there are
no
> outstanding code items required for 2.1.  So I'd like to declare code
> freeze for this version.
> 
> Documentation and testing comprise the bulk of the remaining work.  We
> still need to carry out the following documentation tasks:
> 
> i) Document the configuration files in toto.  Format the documentation
> consistently, as well as vet it for correctness.
> 
> ii) Document the top 5 items on the TODO list
> 
> iii) Add FetchPOP documentation
> 
> iv) Produce a consistent documentation package for the 2.1 version
that
> details the product services, as well as the basic steps of
installation
> and configuration.
> 
> Build tasks:
> 
> i) Build a milestone build for testing.
> 
> ii) Adjust the CVS to allow future maintenance of the 2.1 branch while
> permitting radical change for the 3.0 revision.
> 
> Testing tasks:
> 
> i) Run general tests on a variety of platforms.  We've already done a
> great deal of testing on both Windows and Linux platforms, but it
would
> be good to get a great deal more testing done before the official
> release.  We should look to our user base for potential testers.
> 
> ii) Test TLS support for POP3 and update the status from experimental
to
> stable.
> 
> Marketing tasks:
> 
> i) Produce a press release for distribution through channels
announcing
> both the release of 2.1 and inviting developers to participate in
> version 3.0.
> 
> ii) Distribute this release through channels (i.e. Slashdot)
> 
> iii) Ensure that we are listed on common sites that provide lists of
> available mail/news servers.
> 
> I'm working on items i, ii, and iv on the documentation task list, as
> well as items i and iii on the marketing task list.  Anyone who wants
to
> step up for any of the other items, please do.  I'd like to ask Danny
> specifically to address the milestone build issue, since he has write
> access to the web site.  If he can get me such write access, I'd be
> happy to build and post the milestone build.
> 
> Here's my desired schedule:
> 
> November 4th, 2002 - Creation and posting of milestone build
> 
> November 4th-17th, 2002 - Testing period.  Aggressive testing of James
> under a variety of conditions.
> 
> November 8th, 2002 - Documentation freeze, which requires completion
of
> all documentation tasks.
> 
> November 15th, 2002 - Completion of press release, list of sites to
> which the release needs to be distributed.  Mail server info sites
> should be updated with links at this point.
> 
> November 18th, 2002 - Release.  Update of web site to reflect newly
> released version.  Distribution of press release to sites should be
> completed immediately after release.
> 
> How does this sound to everyone?  Danny, what are your thoughts on
> making a milestone build?
> 
> --Peter
> 
> 
> 
> --
> To unsubscribe, e-mail:   <mailto:james-dev-
> [EMAIL PROTECTED]>
> For additional commands, e-mail: <mailto:james-dev-
> [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   <mailto:james-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@;jakarta.apache.org>

RE: Release Plan v2.1

[Peter M. Goldstein]

All,

Well, here's my suggested scheduler for release 2.1, given the
developments to this point.  Let me first summarize what we've gotten
done, then what remains to be done before release, and then lay out the
schedule I think is realistic.

We have, at this point, achieved all the code specific goals discussed
in the release plan of September 28th.  We have replaced the
ConnectionHandler infrastructure, addressed the TimeScheduler issue, and
closed all the code specific bugs in item #4 of the earlier release
plan.  We have also made a number of substantial performance and memory
usage improvements over the last month, as well as bringing the NNTP
server into better agreement with the specification.  We've upgraded our
distribution to Phoenix 4.0.1, resolving the logging bug.  The javadocs
have also been brought up to speed.  As far as I can tell, there are no
outstanding code items required for 2.1.  So I'd like to declare code
freeze for this version.

Documentation and testing comprise the bulk of the remaining work.  We
still need to carry out the following documentation tasks:

i) Document the configuration files in toto.  Format the documentation
consistently, as well as vet it for correctness.

ii) Document the top 5 items on the TODO list

iii) Add FetchPOP documentation

iv) Produce a consistent documentation package for the 2.1 version that
details the product services, as well as the basic steps of installation
and configuration.

Build tasks:

i) Build a milestone build for testing.

ii) Adjust the CVS to allow future maintenance of the 2.1 branch while
permitting radical change for the 3.0 revision.

Testing tasks:

i) Run general tests on a variety of platforms.  We've already done a
great deal of testing on both Windows and Linux platforms, but it would
be good to get a great deal more testing done before the official
release.  We should look to our user base for potential testers.

ii) Test TLS support for POP3 and update the status from experimental to
stable.

Marketing tasks:

i) Produce a press release for distribution through channels announcing
both the release of 2.1 and inviting developers to participate in
version 3.0.

ii) Distribute this release through channels (i.e. Slashdot)

iii) Ensure that we are listed on common sites that provide lists of
available mail/news servers.

I'm working on items i, ii, and iv on the documentation task list, as
well as items i and iii on the marketing task list.  Anyone who wants to
step up for any of the other items, please do.  I'd like to ask Danny
specifically to address the milestone build issue, since he has write
access to the web site.  If he can get me such write access, I'd be
happy to build and post the milestone build.

Here's my desired schedule:

November 4th, 2002 - Creation and posting of milestone build

November 4th-17th, 2002 - Testing period.  Aggressive testing of James
under a variety of conditions.

November 8th, 2002 - Documentation freeze, which requires completion of
all documentation tasks.

November 15th, 2002 - Completion of press release, list of sites to
which the release needs to be distributed.  Mail server info sites
should be updated with links at this point.

November 18th, 2002 - Release.  Update of web site to reflect newly
released version.  Distribution of press release to sites should be
completed immediately after release.

How does this sound to everyone?  Danny, what are your thoughts on
making a milestone build?

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: JavaMail mime header parsing

[Peter M. Goldstein]

Serge,

I'm not sure that's the relevant RFC.  The type name/parameter pair is
not addressed in RFC 2045, since it's a parameter specific to the
multipart/related Content-Type.

That said, RFC 2387 which defines the Multipart/Related type, seems to
indicate that it's actually the unquoted version which is correct.
Check out section 3.4 of this RFC.  Although it also seems to say that,
in general, parameters will be quoted.

--Peter

> -Original Message-
> From: Serge Sozonoff [mailto:serge@;globalbeach.com]
> Sent: Thursday, October 31, 2002 1:14 PM
> To: James Developers List
> Subject: JavaMail mime header parsing
> 
> Hi All,
> 
> Sorry for this slightly off topic subject, hoping someone has a simple
> answer.
> 
> My understanding of RFC2045 section 3.  MIME Header Fields leaves me
to
> beleive that both of the following representations are correct:
> 
> Content-Type: multipart/related;
> boundary=mmsc-mgw-unique-boundary-
> 1;type="application/smil";start="<89168876
> 9>"
> Content-Type: multipart/related;
> boundary=mmsc-mgw-unique-boundary-
> 1;type=application/smil;start=<891688769>
> 
> Note that one has some double quotes and one does not.
> 
> However when I run the non-quoted version through JavaMail 1.2 and 1.3
I
> get
> a ParseException or MessagingException depending on the operation I am
> doing.
> Is there a but in JavaMail or is my understanding of the RFC wrong?
> 
> Any thoughts on upgrading the JavaMail extension in James, I beleive
we
> are
> still using 1.2 (I have found no specific reason for suggesting this,
but
> there were some bug fixes and it might benefit those using JavaMail
for
> their mailets.
> 
> Thanks, Serge
> 
> Example code to see this in action
> 
> import javax.mail.internet.*;
> 
> class test {
> public static void main(String[] args) {
> try {
> String header = "multipart/related;
> boundary=mmsc-mgw-unique-boundary-
> 1;type=application/smil;start=<891688769>"
> ;
> ContentType cType = new ContentType(header);
> System.out.println(cType.toString());
> } catch(ParseException pe) {
> System.out.println("Parse error");
> pe.getMessage();
> }
> }
> }
> 
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: App to Update User Data?

[Peter M. Goldstein]

Kenny,

> I'm trying to build a web app so that my users can login and change
their
> user passwords, so I'm trying to figure out the best way to go about
> updating the table.
> 
> I figured I could either a) use the DigestUtil to encode the password
and
> then do my own SQL to update the user table, or I could b) use the
> existing
> James objects to interface with the database.
> 
> Option B sounds best to me, so I went digging. I'll explain more
below,
> but
> the roadblock that I'm stopped at is... I can't find the
> RemoteManagerHandlerConfigurationDataImpl class.
> 
> This is how I got there (tell me if I'm approaching this poorly):

I don't think this is quite the right approach.  Might I suggest that
you read a bit more of the Avalon docs before you try and write an app
using the Avalon components.  It will probably save you quite a lot of
grief.

A discussion of the Avalon lifecycle can be found here:

http://jakarta.apache.org/avalon/framework/reference-the-lifecycle.html
 
> I looked in the RemoteManagerHandler to see how it added users, and
found
> that it interacts with a UsersRepository (an instance of
> JamesUsersJdbcRepository in my case). I looked up the UsersRespository
> interface and found an updateUser(User) method, which appears to be
> exactly
> what I'm looking for.
> 
> So, I need to get an instance of the UsersRepository, and found that
it is
> fetched from getUsersRepository() in an instance of a
> RemoteManagerHandlerConfigurationData which is actually an instance of
> RemoteManagerHandlerConfigurationDataImpl.
> 
> RemoteManager gets one by simply calling the constructor, I'm hoping
that
> all I have to do is call the constructor on it too, but I can't find
the
> source file anywhere to peek inside and see what it's doing.

You're missing a key step here.  RemoteManagerHandlerConfigurationImpl
is an inner class of RemoteManager.  You definitely shouldn't be trying
to instantiate it - that would be a very bad idea.  In fact, all
RemoteManagerHandlerConfigurationImpl's getUsersRepository() method does
is return the UsersRepository that was obtained in the compose lifecycle
method of RemoteManager.  So the logic expressed in the compose() method
of RemoteManager is what you should be looking at.

There is some discussion of Composable on the page referenced above,
even though it has been deprecated in favor of Serviceable.  The basic
idea behind both Composable and Serviceable is that classes that
implemented these interfaces can lookup other components that are
managed by the container.

In the RemoteManager class, RemoteManager looks up the UsersStore
implementation, which manages the list of repositories.  The relevant
code fragment is:

public void compose( final ComponentManager componentManager )
throws ComponentException {

...
usersStore = (UsersStore)componentManager.
lookup( "org.apache.james.services.UsersStore" );
users = usersStore.getRepository("LocalUsers");

...
}
}

This is how one gets ahold of components and children of components -
not by direct instantiation.  Instantiation is left to the container.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Build Error (newbie question)

[Peter M. Goldstein]

Kenny,

Can you please step through the build.sh script to determine the line
that is generating the problem?

--Peter

> -Original Message-
> From: Kenny Smith [mailto:kenny@;journalscape.com]
> Sent: Wednesday, October 30, 2002 3:03 PM
> To: James Dev
> Subject: Build Error (newbie question)
> 
> Hi all,
> 
> I built the CVS version of James on my local server and it worked
great.
> I'm
> trying to build it on the production server and I'm getting this
error:
> 
> --- snip --
> bash-2.03# ./build.sh
> 
> James Build System
> ---
> ./build.sh: ANT_HOME=: is not an identifier
> --- snip ---
> 
> I don't have ANT_HOME defined in either of my environments.
> 
> OS Local: Mandrake Linux 2.4
> OS Production: Solaris 8/Intel
> 
> JDK Both: Sun 1.4.0_01
> 
> Checked both out from cvs fresh after Peter Goldstein's message about
> commiting Phoenix 4.0.1
> 
> Thanks for any help,
> Kenny Smith
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Phoenix 4.0.1

[Peter M. Goldstein]

All,

I've just checked in the Phoenix 4.0.1 version.  It resolves the logging
bug (all log entries go to a single file) that was introduced in the
Phoenix 4.0 release version.

I've been building and running with this version successfully on a
Windows 2k box for over a week now.  Noel has tested it on Linux, and
found that it runs successfully there.

I have also changed the build.xml so that deprecation warnings are
turned off.  With this upgrade the number of Component related
deprecation warnings goes up to 200+.  IMHO that's just really annoying.
So I flipped them off until we do the upgrade to
Serviceable/ServiceManager in the next rev.

There is an observed issue with Red Hat Linux 6.2, Sun's 1.3.0 JVM, and
this revision of Phoenix.  This has been confirmed to be a JVM issue,
and can be resolved by switching to either IBM's 1.3.1 JVM or Sun's 1.4
JVM.  Other than this issue there are no known platform/JVM
incompatibilities.

Please let me know immediately if you encounter any issues, as I'd like
to make this upgrade as smooth as possible.  Please make sure that you
upgrade both your phoenix-bin directory and your build.xml.  Thanks.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Build From CVS has Warnings?

[Peter M. Goldstein]

Kenny,

It's ok and expected.  The warnings will not go away unless we

i) Temporarily disable deprecation warnings, as we're likely to do with
the checkin of Phoenix 4.0.1 (the # of warnings grows to 200+)

or

ii) Upgrade to use ServiceManager and Serviceable, which is scheduled
for the next release.

--Peter

> -Original Message-
> From: Kenny Smith [mailto:kenny@;journalscape.com]
> Sent: Wednesday, October 30, 2002 11:09 AM
> To: James Dev
> Subject: Build From CVS has Warnings?
> 
> Hey all,
> 
> Since I'm still new getting and building releases out of CVS I wanted
to
> check to see if it's ok that I got this and if you experienced it when
you
> built:
> 
> warning: ComponentException(java.lang.String,java.lang.Throwable) in
> org.apache.avalon.framework.component.ComponentException has been
> deprecated
>   throw new ComponentException( message, e );
> 
> I got it in 11 different places.
> 
> Kenny Smith
> JournalScape.com
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [VOTE] James as an official Apache project

[Peter M. Goldstein]

+1. 

> -Original Message-
> From: Danny Angus [mailto:danny@;apache.org]
> Sent: Wednesday, October 30, 2002 9:47 AM
> To: James Developers List
> Subject: [VOTE] James as an official Apache project
> 
> All,
> 
> Now that we've had a chance to digest whats meant by a top level
project,
> I'll subject you to my opinion, which I've held back so far.
> 
> Basically I'm in favour of proposing james as an apache top level
project
> (tlp), I'm prepared to take the lead on our proposal, and here's why:
> 
>  James has a small yet mature community, we seldom seek recourse to
the
> jakarta PMC, and equally seldom are we scrutinised by them. We are not
the
> most active project, and I feel that this sometimes causes James to be
> disregarded. Likewise, apart from Avalon, we have few direct ties with
> other jakarta projects, and little in common apart from the
> language/platform and culture (but the culture is Apache)
> 
> The tlp issue is more about management than web-site and mail
addresses, I
> don't believe that james.apache.org will bring many benefits, but I do
> think that normalising our managerial relationship with Jakarta by
> becoming a sibling rather than a child, and taking official control of
all
> the issues we currently inherit and "interpret" from Jakarta would
benefit
> James. The James PMC would be responsible to The Board.
> 
> I do believe that Jakarta is becoming too big to function as a single
> project, that community and culture become diluted as you descend the
> heirarchy and that one solution is for mature projects leave the nest.
Of
> course other projects are free to make their own choices but as James
> consists primarily of the server which is an end-user product I feel
that
> top level project status, emphasisng its purpose rather then its
> technology, would suit it well.
> 
> The proposals being discussed on reorg & community include the notion
of
> federated projects, James, with the approval of the Jakarta PMC, could
> continue to be associated with Jakarta, I would like to think that we
> wouldn't be leaving Jakarta, just growing up. I also know that James
would
> continue to rely on Jakarta for code, insight and cool thinking, but
we
> don't need to be a Jakarta sub-project for that.
> 
> It would also give us the opportunity, as Serge mentioned, to better
> promote our sub-projects, theres Mailet, and mailets, and there are
the
> beginings of full blown mailet applications.
> 
> So.. we've had time to examine the issues, lets vote..
> 
> Should we prepare a proposal for submission to the board, that James
be
> elevated to a top level project?
> 
>[ ] +1  I think it's a good idea
>[ ] +0  I'll accept the majority decision, stop bothering me
>[ ] -0  I have reservations
>[ ] -1, I don't think its a good idea
> 
> 
> d.
> 
> 
> --
> To unsubscribe, e-mail:    [EMAIL PROTECTED]>
> For additional commands, e-mail:  [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

[PATCH] Adding Interrupts to idleClose()

[Peter M. Goldstein]

All,

During load testing with James and JDBC, Noel was able to produce an
interesting problem where there was a problem (not due to James) down at
the database.  Closing the socket was not enough to get the handler's
attention down at lower levels.  Threads were not being returned to the
pools. Eventually, James refused to allow any more connections rather
than spawn new threads in an unbounded fashion.

This patch supplements the socket.close() approach that has been used in
the James handler timeout with an additional Thread.interrupt().  Noel
tested it by replacing the sendMail() call in smtphandler with:

  while (!Thread.currentThread().isInterrupted()) {
  try {
  Thread.sleep(1000);
  } catch(InterruptedException ex) {
  if ((System.currentTimeMillis() & 1L) == 0)
  throw new MessagingException("Watchdog Test");
  else
  Thread.currentThread().interrupt();
  }
  }

which tested both the scenario were a method would be interrupted that
threw an exception, and the scenario where the interrupted flag is
simply set. Noel also tested the patch with a variant of that code
embedded inside of store().

Please note that this is designed to address a pre-existing bug.

Unless something else arises in our testing, this should be the last
change of any significance in the Java source code before we release
2.1.  After this, we can focus on the remaining documentation and
testing required for release.

--Peter


Index: src/java/org/apache/james/nntpserver/NNTPHandler.java
===
RCS file: 
/home/cvspublic/jakarta-james/src/java/org/apache/james/nntpserver/NNTPHandler.java,v
retrieving revision 1.24
diff -u -r1.24 NNTPHandler.java
--- src/java/org/apache/james/nntpserver/NNTPHandler.java   26 Oct 2002 20:16:30 
-  1.24
+++ src/java/org/apache/james/nntpserver/NNTPHandler.java   30 Oct 2002 08:12:23 
+-
@@ -203,6 +203,11 @@
 private final static String AUTHINFO_PARAM_PASS = "PASS";
 
 /**
+ * The thread executing this handler 
+ */
+private Thread handlerThread;
+
+/**
  * The TCP/IP socket over which the POP3 interaction
  * is occurring
  */
@@ -301,6 +306,16 @@
 }
 } catch (Exception e) {
 // ignored
+} finally {
+socket = null;
+}
+
+synchronized (this) {
+// Interrupt the thread to recover from internal hangs
+if (handlerThread != null) {
+handlerThread.interrupt();
+handlerThread = null;
+}
 }
 }
 
@@ -311,6 +326,9 @@
 
 try {
 this.socket = connection;
+synchronized (this) {
+handlerThread = Thread.currentThread();
+}
 reader = new BufferedReader(new 
InputStreamReader(socket.getInputStream(), "ASCII"), 1024);
 writer = new InternetPrintWriter(new BufferedWriter(new 
OutputStreamWriter(socket.getOutputStream()), 1024), true);
 } catch (Exception e) {
@@ -387,6 +405,10 @@
 getLogger().warn("NNTPHandler: Unexpected exception occurred while 
closing socket: " + ioe);
 } finally {
 socket = null;
+}
+
+synchronized (this) {
+handlerThread = null;
 }
 
 // Clear the selected group, article info
Index: src/java/org/apache/james/pop3server/POP3Handler.java
===
RCS file: 
/home/cvspublic/jakarta-james/src/java/org/apache/james/pop3server/POP3Handler.java,v
retrieving revision 1.16
diff -u -r1.16 POP3Handler.java
--- src/java/org/apache/james/pop3server/POP3Handler.java   26 Oct 2002 19:57:37 
-  1.16
+++ src/java/org/apache/james/pop3server/POP3Handler.java   30 Oct 2002 08:12:24 
+-
@@ -84,6 +84,11 @@
 private MailRepository userInbox;
 
 /**
+ * The thread executing this handler 
+ */
+private Thread handlerThread;
+
+/**
  * The TCP/IP socket over which the POP3 interaction
  * is occurring
  */
@@ -176,7 +181,18 @@
 }
 } catch (Exception e) {
 // ignored
+} finally {
+socket = null;
 }
+
+synchronized (this) {
+// Interrupt the thread to recover from internal hangs
+if (handlerThread != null) {
+handlerThread.interrupt();
+handlerThread = null;
+}
+}
+
 }
 
 /**
@@ -190,6 +206,9 @@
 
 try {
 this.socket = connection;
+synchronized (this) {
+handlerThread = Thread.currentThread();
+}
 in = new BufferedReader(new InputStreamReader(socket.getInputStream(), 
"ASCII"), 512);
 remoteIP = socket.getInetAddress().getHostAddress ();
 remoteHost = socket.getInetAddress().get

RE: performance figures w/ spool enabled

[Peter M. Goldstein]

Danny,

My feelings aren't particularly strong on this one, so I'm ok with
deferring.

One minor question - why do you seem to want us to put off that minor
change to the NNTPSpooler?  Releasing the reference before the thread
sleeps seems like a good idea.  It's a trivial modification and very
safe.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: performance figures w/ spool enabled

[Peter M. Goldstein]

> Noel J. Bergman wrote:
> > Serge wrote:
> >   (2) Use the same approach found in JCBCSpoolRepository,
> >   where we maintain a small cache of keys.  When the
> >   list is exhausted, it is reloaded.  I believe that I
> >   can pretty much clone the code from one to the other,
> >   with the obvious change in populating the list.
> >
> > Are either of those approaches something that we want to undertake
for
> v2.1?
> > Do you want to consider this a bug fix, or a performance enhancement
to
> > defer?  I don't have a problem implementing the change, but I don't
want
> to
> > undertake it today if we don't want it until after the release.
> 
> If you can get the code from JDBCSpoolRepository and pretty quickly
get
> this alternate approach running, then see how your performance tests
> do... if it's an improvement, I would go ahead and put it in.  In my
> mind it's known logic, so it's not that big of a deal.

I'm of the same mind as Serge on this one.  If the logic is already
written and has been shown to work, I say put it in.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: latest checkin...

[Peter M. Goldstein]

 

Harmeet,

 

> Alright Peter here is the attached service. I basically decided to let
you

> do your thing and that was not due to laziness, but just did not want
the

> conflict.

 

Again, it's like you are illiterate.

 

This AuthService does the following:

 

i) Publicly exposes an interface to other Phoenix components

 

ii) That interface fails to use long standardized (since Java 1.1) APIs
for authentication and authorization.  You've commented about how you
"just can't understand how the Avalon folks wouldn't use Doug Lea's
code, since it's best of breed" (paraphrase).  Yet you fails to use the
long established (longer than Doug Lea's code and more widely used) Java
Authentication and Authorization Service.  Sure seems hypocritical to
me.

 

iii) This moves the authentication requirement outside of the server
configuration.  That is wrong.  A service needs to be able to control
whether or not it requires authentication/authorization

 

I went through all of this and more in my email here:

 

http://www.mail-archive.com/james-dev@;jakarta.apache.org/msg03837.html

 

You haven't used standard interfaces, and you haven't addressed my other
concerns.  Instead, you've served up a warmed over version of the
previous AuthService.

 

> No it wasn't. I had read the code before but did not remember. What is

> your

> point ?. I added a class, realized it was redundant and then removed
it.

> So

> is that an offence in Apache ?

 

No, but its a further point that you haven't contributed anything
useful.

 

 

> > > > iii) An incorrect scheduler that exhibited almost exactly the
same

> > > > behavior as the current Scheduler

> > > >

> > > > iv) Another Scheduler implementation that no one but you seems
to

> > > > think we need, and that you haven't put the time in to build and

> > > > test with Noel.

> > >

> > > Sceduler implementation has already been verified. You made a
number

> > > of fixes too in watchdog.

> >

> > Noel seems to disagree with this statement.  He states that there is
at

> > least one open bug in your scheduler implementation.

> 

> 

> 

> The only thing I know of is optimization issues not correctness ones.
Did

> not want to optimize as I was not sure it would be production code.

> 

 

See Noel's email.  You're a liar.

 

> > So we have no idea, save for your word (which you also gave for the

> > Timer implementation of the scheduler, as I recall), that the thing

> > works properly under load.  It may.  It may not.

> 

> 

> Noel's test confirmed that it stays up under load.

 

Again, Noel seems to disagree.  He's still waiting for you to fix your
scheduler.  So this is another lie.

 

 

> You are fixing things that I wrote 9 months back and after I have sent
the

> exact problem and place where it is broken.

> 

 

This is unbelievable.  I'm not sure whether to be more aghast at the
sheer gall of this statement or at its presumption of group stupidity.
Let's consult the archives, shall we:

 

The first email that has a fix for AuthService.  It is mine.  And as of
this date, you had made no comments whatsoever about why AuthService was
broken:

 

http://www.mail-archive.com/james-dev@;jakarta.apache.org/msg03791.html

 

Your initial reply, where you defend the AuthService interface, and you
clearly don't believe there is a problem.  You continue to assert that
it might be an implementation problem, but the interface is ok.

 

http://www.mail-archive.com/james-dev@;jakarta.apache.org/msg03798.html

 

My reply:

 

http://www.mail-archive.com/james-dev@;jakarta.apache.org/msg03803.html

 

How on earth you can claimed that you've "identified the exact problem"
after this interchange is beyond me.  But you've already demonstrated
that you're both shameless and deceitful.

 

 

> Rest of the email doesn't merit a response, but I do wish you could
cool

> down.

 

Then you should start acting like an ethical human being.

 

--Peter

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

> Don't know what to say to that... Peter is Peter.
> There is no reason to burn oneself over volunteer work.
> We are all trying to build the right thing, and it is better to work
> together than get upset.

And this is exactly why you shouldn't be a committer.

As I've said before, this is not a game.  Those of us who devote our
time and energy to making James an enterprise class piece of software do
not consider it a game.  If you do (as you clearly do), get out.

I take my duties seriously - be they volunteer or otherwise.  My
professionalism is a fundamental part of me, and it doesn't matter
whether I'm getting paid or not.  I take my responsibilities seriously
when I'm volunteering for charity, playing on a team, or working at a
job.  I simply don't understand those who don't...

How you can claim to be working together when you lie and commit slander
I will never know...

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

> > It was resolved, as it always is, in a matter of hours
> > of being reported.  Not days.  Or months.
> 
> Agreed, but you broke it, I reported it and you fixed it. I could have

> fixed it too.

Just like you could've fixed the broken AuthService anytime in the last
nine months.  But you didn't.  And thus it doesn't count.  You see,
saying you're going to do something doesn't cut any mustard when you've
got no follow through.

This is in fact just like AuthService.  You claimed to be able to write
a service that would replace it.  You never submitted it for
consideration.  It doesn't exist.  You never bothered to make it exist.
It's much easier to not do the work and play the victim.

You simply ignored all the posed technical points and rather than trying
to address them, you went off and sulked.  And then you took out your
passive-aggressive tendencies on Noel's committer vote.  Lovely.

> 
> > Perhaps this is the core of the issue.  You seem to think that 
> > you're work contributed prior to this release allows you dictatorial

> > control over the code.
> 
> I have reported the issue and exact code where things were broken. If 
> I felt what you are saying, I woundn't have been sending code 
> problematic snippets
> to the list.
> 
> 
> > i) Test code with an unmarked protocol script in violation of the 
> > RFC
> 
> Some test code is better than none.
> If there is a test script that voilates RFC why don't you suggest a 
> patch or add tests.

Test code for test code's sake is useless.  It's only when it's part of
a comprehensive notion of testing - and when those test cases are
clearly labeled - that the testing is at all worthwhile.  See any of
Noel's emails regarding test code.


> > ii) A superfluous class (CRLFWriter.java) that didn't even address a

> > documented issue in the NNTP code.  This issue means that the 
> > CRLFWriter code was wrong.
> 
> This is no longer there. As soon as I saw something eqivalent I 
> removed it. This was yesterday or a couple of days back.

Exactly.  So it wasn't a useful contribution.  Of course you didn't
bother to actually read the code base to investigate this...

 
> >
> > iii) An incorrect scheduler that exhibited almost exactly the same 
> > behavior as the current Scheduler
> >
> > iv) Another Scheduler implementation that no one but you seems to 
> > think we need, and that you haven't put the time in to build and 
> > test with Noel.
> 
> Sceduler implementation has already been verified. You made a number 
> of fixes too in watchdog.

Noel seems to disagree with this statement.  He states that there is at
least one open bug in your scheduler implementation.  Moreover, as far
as I can tell from the posts, you've never sent him a .sar for testing.
So we have no idea, save for your word (which you also gave for the
Timer implementation of the scheduler, as I recall), that the thing
works properly under load.  It may.  It may not.  But you haven't put in
the effort to test and possibly fix.  As far as the Watchdog goes, I
have.  It works and has been demonstrated to work by other people.

> > A regression indeed.  And fixed.  In an hour.  Not nine months.
> 
> After I posted the exact snippet that exhibitted the problem. Would 
> have been nicer if you would have once tested it yourself or 
> discovered the snippet rather than adding code and breaking the 
> server.


And it would've been nice if you hadn't implemented an architecturally
unsound AuthService.  We all make mistakes.  The difference is that I
fix my own.  You do not.

So to sum up, in the entirety of 2002 you haven't contributed a single
useful line of code or documentation.  Yet you wish to block those
who've been working diligently on the project for months.  It all comes
back to ethics...
 
> 
> As the NNTP code stood today morning at least, it was much worse than 
> anytime in last 9 months. Auth was broken for 9 months but in the last

> few days everything was broken. I am trying to test and fix.

In a word, this is a lie.

There are zero (that's none) outstanding bugs against the current code
base.  All the issues you reported have been addressed - quickly and
efficiently.  What's your issue?  

Name your issue and I'll look at it.  But the server works - and was
tested a great deal more than it ever was before.

Just the sheer unmitigated gall of this response is amazing.  We have
here a committer who abandoned the product for nine months, with a slew
of open critical bugs against his nominally owned component. 

For god's sake, one of them was a typo in a string.  It took a grand
total of 20 seconds for me to find it.

And yet it's like every nugget of code he wrote came down from the
mountain with Moses.  The attitude just leaves me speechless.

And we once again come back to the same point - if you don't like the
changes in James Harmeet, use an older version.  You haven't contributed
one whit to this or to the last version (2.0a3), so I don't see what
you're compla

[VOTE] - Move NNTP to Proposals until it meets the basics of the NNTP spec

[Peter M. Goldstein]

All,

I'm quite frankly fed up.  The NNTP code clearly doesn't work, and my
attempts to resolve its issues have been met with hostility.

I won't put out a release with my name on it that has such poorly
functioning code in its bowels.  It violates spec all over the place
(see open bugs) and is just unpleasant to work with.  It is not in
active use by anyone on the james-user list.  So I'm going to take up a
suggestion Noel made a couple of months ago and officially propose it.
And apologize to him for not taking it up in the first place.

I propose that the NNTP server be moved to the proposals directory until
and unless it meets spec and provides the basic functionality expected
of an NNTP server.  That way it won't interfere with the current
release.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Noel,

> Is there any additional language that refers to a prefered solution,
or to
> any additional semantics for re-authentication?  Without the RFC
> specifying
> a preference, or common practice if the RFC is neutral, I don't
> particularly
> care which optional behavior is manifested in the code, so long as it
> works.
> Do anyone else?  Harmeet seems to have a preference.

Nothing documented in the RFC.  There might be something somewhere else.

It's a basically three line change to change the current code to allow
re-authentication (rather than returning a 482, the user, pass, and
isAlreadyAuthenticated values get wiped).  I don't really care - so long
as the other behavior (handling double USER submission, out of order,
double PASSWORD) is there.
 
> > This is a MAY, not a MUST, so it is indeed optional.
> 
> I generally find that MAY is used to specify minimally required
behavior,
> when alternatives are more complex and not universally necessary.

Yep.
 
--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Noel et al,

> As far as I'm concerned, no one --- not me, not you, not Peter, no one
---
> should be submitting changes to the wire-level-protocol syntax and
> semantics
> without documenting the relevant RFC issues.  If there is disagreement
on
> how to interpret the RFC, we can check with other sources.  But this
is
> not
> a subjective matter.  There may be options that render multiple
choices
> correct, but that also needs to be demonstrated from the RFC.

The only statement on reauthentication in the RFC is the following:

"If a client attempts to reauthenticate, the server may return 482
response 
indicating that the new authentication data is rejected by the server."

This is a MAY, not a MUST, so it is indeed optional.  It would certainly
be possible to make this configurable.  I chose to implement the
specified behavior, because that's what is discussed in the RFC.

Note that this is not a regression.  Since the AuthService never worked
- try having two different authenticated users on James 2.0a3 and see
what happens when you do a re-authentication.  Bad, bad things happen.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

> > As discussed earlier, there are a number of RFC violating bugs.
This
> > was central to one of them.
> 
> Please point out where you have discussed removing article writer

>From your reply to my message:

> If these bugs can be resolved without restructuring code, you should
check
> them in.
> 
> If you need to rearchitect or refactor for these fixes, you should
post
> your
> proposal and your fixes.
>

As I've already mentioned, you seem to put removing a class in some sort
of special "design change" category.   I have no idea why.  I don't even
know what you mean by not "restructuring code", since every change is
restructuring.  In accord with our discussions, I committed my changes.
If you don't like it, -1 it.  Personally, I'm sick of the buggy NNTP
server and will be posted a vote to remove it from James shortly.

> > There was an attempt in the handler to collapse the STAT, HEAD,
BODY,
> > and ARTICLE implementations into a single implementation.  This led
> > directly to item #5 in Bug #13564.
> >
> 
> ArticleWriter did not impede this.
> One could have passed the response code to
> private void doARTICLE(String param,ArticleWriter articleWriter)
> rather than collapse and refactor

This is ridiculous.  Why on earth would you break abstraction like this?

Aside from violating the semantic intent of the protocol - STAT, BODY,
ARTICLE, and HEAD are all separate commands, there's no actual benefit.
The code is further obfuscated (which is why there were a couple of
hidden bugs in it) and the individual messages can't be customized
without adding yet another parameter to the doARTICLE.

But more critically, we're seeing the same "that's not the way it used
to work" argument.  It didn't work before.  It works now.

And it gets rid of an extra interface and factory class.  It reduces the
semantic weight of the code.  Decouples the individual commands (those
that are not aliases of one another).

> >
> > Moreover, the ArticleWriter was conceptually flawed.  Because it
> > included protocol specific items (terminating ".", what-have-you),
> > wasn't really a writer of articles.
> 
> It is nntp protocol specific so what are your concerns about being
> protocol
> spcific ?

Specific to the protocol itself.  Not specific to the server, where it
is exposed.

> > wasn't really a writer of articles.  If it were used in a repository
> > implementation, it would corrupt all the articles in a repository.
> 
> It could write out to the client the article information. If you write
to
> source, sure you can corrupt.

So it isn't an ArticleWriter.  It doesn't write articles.  If I pass it
a FileWriter wrapped in a PrintWriter, it won't work.
 
> > Moreover, it contained no logic other than whether these protocol
> > specific items should be written - the remainder of the article
writer
> > logic is internal to the NNTPArticle implementation.
> 
> That was the intent.

Well why?

You introduced a new class to no effect.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

> > It was run a great deal more than once.  Of course in my config, the
var
> > is deliberately kept at root.  So I wouldn't see this issue.
> 
> The last refactoring had caused a NullPointerException and Sever not
> starting.

A NPE that only occurred in a fresh server, not in a test server.
Largely because the previous NNTP code mixed up the directory lookup and
validation code.  It was resolved, as it always is, in a matter of hours
of being reported.  Not days.  Or months.
 
> MODE READER is the first command sent by an NNTP Client.
> As you can probabaly see it is broken...
> 
> Your intent is good and changes knowledgeable, but neither justifies
> removing work done by others(effective veto over other folks work) nor
> does
> it justify breaking things. At least have a test or two. I could have
> checked in the auth fix a few hours back but I am holding off till I
have
> a
> regression test...

What work?

If you want to keep your current NNTP service, you're welcome to use an
older version James.  I contributed absolutely nothing prior to James
2.0a3, so you're welcome to use that.  It's wrong, but you haven't
contributed any changes to fix it, so I don't see how you have any
problem with it.

Perhaps this is the core of the issue.  You seem to think that you're
work contributed prior to this release allows you dictatorial control
over the code.

These are the hard facts.  James has almost no one using NNTP because it
simply didn't work.  Check the traffic on james-user.  The only mention
of NNTP in the last three months was a user rejecting the implementation
as unsuitable for production.  No one was maintaining it.  There was a
typo bug in the SSL connection that had gone unattended for nine months.
There was the AuthService bug, an architectural problem, that lay
dormant for the same nine months.  Your return coincided almost exactly
with actual repairs to the NNTP server to bring it up to a production
quality server (or close to it).

Since your return you have contributed:

i) Test code with an unmarked protocol script in violation of the RFC

ii) A superfluous class (CRLFWriter.java) that didn't even address a
documented issue in the NNTP code.  This issue means that the CRLFWriter
code was wrong.

iii) An incorrect scheduler that exhibited almost exactly the same
behavior as the current Scheduler

iv) Another Scheduler implementation that no one but you seems to think
we need, and that you haven't put the time in to build and test with
Noel.

So whose work am I wasting?  Certainly not yours.
 
> Here is the MODE READER implementation. Argument as sent by outlook
> express
> is null.
> yielding
> S: 501 Syntax error - unexpected parameter
> S: 200 Posting Permitted
> 
> private void doMODEREADER(String argument) {
> // 7.2
> if ( argument != null ) {
> writer.println("501 Syntax error - unexpected parameter");
> }
> writer.println(theConfigData.getNNTPRepository().isReadOnly()
>? "201 Posting Not Permitted" : "200 Posting
> Permitted");
> }

A regression indeed.  And fixed.  In an hour.  Not nine months.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

Auth issue isn't a regression, as discussed in my previous email.

> - NNTP Stores should be under apps, now they are created under root
> directory.

This is correct, but the identification of cause is off.

The previous code was broken - it didn't handle absolute URLs correctly.
So it places file:/// URLs in the apps directory, contrary to every
other URL in the system.  

Correcting the default configuration so that the URLs are relative by
default.

> Possible cleanup regressions... Nothing to do with scheduler or
service
> patch.
 
> Fixing second issue, first one is a bit more problematic and would
have
> been
> caught if the server was started once post cleanup/redesign. Will look
> into
> it and fix soon.

Second issue is fixed, thank you.

It was run a great deal more than once.  Of course in my config, the var
is deliberately kept at root.  So I wouldn't see this issue.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

> Peter, why have you removed ArticleWriter and pulled code into handler
?
> 
> What was the intent and how does that relate to anything you have been
> sending around ?

As discussed earlier, there are a number of RFC violating bugs.  This
was central to one of them.

There was an attempt in the handler to collapse the STAT, HEAD, BODY,
and ARTICLE implementations into a single implementation.  This led
directly to item #5 in Bug #13564.

Moreover, the ArticleWriter was conceptually flawed.  Because it
included protocol specific items (terminating ".", what-have-you), it
wasn't really a writer of articles.  If it were used in a repository
implementation, it would corrupt all the articles in a repository.
Moreover, it contained no logic other than whether these protocol
specific items should be written - the remainder of the article writer
logic is internal to the NNTPArticle implementation.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: latest checkin..

[Peter M. Goldstein]

Harmeet,

> One flaw is that if the authentication credentials are passed in again
the
> alreadyAuthenticated state needs to be cleared.
> 
> 

Actually this description of the problem is not necessarily correct.
>From section 3.1.1 of RFC 2980, which discusses the AUTHINFO command,
specifies a set of as of yet unimplemented behavior (rejecting out of
order authentication state, possibly rejecting reauthentication).  I'm
putting in the requisite return codes.  Missed them because I was
focusing on the original NNTP RFC.

--Peter
 



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Current changes (was RE: Code change the Apache way)

[Peter M. Goldstein]

Noel,

> > In short, if something doesn't work it doesn't
> > constitute a design.
> 
> I don't agree with that as a general statement, and I don't believe
that
> you
> do, either.  Bugs do not invalidate design.  Architecture, design and
> implementation each has a role, and a defect in one does NOT imply a
> defect
> in another.  Bugs exist and can be fixed.  For that matter, proper
> operation
> does not imply that the design or architecture is correct, but as
Thomas
> Kuhn would likely argue, we often require failure before we are
motivated
> to
> undertake structural change.

I actually do believe in this as a general statement.  A design is not a
design until it works.  That doesn't mean that the implementation can't
have bugs, but it does mean that the basic functionality has to be
provided and work as expected.  Anything else is virtually an invitation
for developers to spend their time doing what most of them enjoy most
(designing components) and not doing what most of them enjoy least
(testing functionality).  BTW, I put myself into this group.

A great example of this is AuthService.  AuthService was a singleton
component that stored authentication credentials in a global fashion.
This was a critical bug from day one.  Even the simplest negative test
case would've exposed the problem.  But none was done.  Thus I don't
consider that design to have any merit - since it never worked, I can't
really worry about breaking it.  Fixing it in any sense would've
required changing the "design", as the auth credentials would've had to
have been moved from the global context.

I do agree that basic proper function is not a sufficient condition for
optimal design, but I will argue that it's a necessary condition.  An
architecture that precludes proper function is not an optimal design -
it's not even a valid one.
 
> Again, as for the rest, I think that we're moving forward, should
leave it
> at that, and do what we can to keep James established as an attractive
> community for developers interested in a high quality, flexible, mail
> server.
> 
> I took a look at the change log that Serge put out for each prior
release;
> boy, do we have a lot of work to do pulling that together for 2.1!

Yeah, I've starting coordinating a list by examining the commit messages
for the last three months.  Lots to do...

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Where do we go from here.

[Peter M. Goldstein]

Danny and Harmeet,

> Peter is the release manager, and has already posted a plan, and had
it
> approved by Vote.
> Just where we are now at is not claer to me, but AFAIK we're
progressing
> in the right direction.

With the exception of the current -1 of features discussed in the
release plan, yes we are.  We're delayed date wise because of the
current discussion.

> > Here are some ideas.
> > - Have a single, standard repository API. Noel, others have
> > mentioned JNDI.
> 
> If you wait on this 'till after the release I've got some repository
> suggestions to make related to the Mailet API.
> Basically we need to have a either:
> 
> a) standard repository descriptor, and the URL we use seems to fit
that
> bill well,
> or
> b) mailet uses named repos and repository definition is left to the
> application,
> 
> and repository access has to be via MailetContext using the URL,
> alternatively we have named repositories.
> I vote heavily for a) over b) having had a system running using it, as
it
> allows mailets to dynamically define their own repositories.
> 
> This removes James' mailets' "backdoor" avalon dependance.

Discussed and agreed to push back to after the release in email
discussions prior to the release plan.
 
> > - Move all protocols including NNTP to use that repository API.
> 
> Move NNTP to use a "newslet" structure and the mailet repository
system to
> define repositories, this also allows news to be processed, say for
news-
> >mail.
> 
> 
> bearing all of that in mind I'm not concerned.

Discussed and agreed to push discussion/changes back to after the
release in email discussions prior to the release plan.  My work on NNTP
prior to the release was just an attempt to get it into minimal
compliance with the spec.  As I've stated before, I want to attract more
contributors to the project and I believe that having a non-functioning
service as part of the release build was somewhat embarrassing.

 
> > - IMAP Server support.
> 
> Already expected to be part of next cycle.

Discussed and agreed to push discussion/implementation back to after the
release in email discussions prior to the release plan.

Having now examined the IMAP server code at length, IMHO it will take at
least a month of dedicated developer effort to bring it up to spec.
Certainly possible within the context of rev++.
 
> > - Test Bed to check RFC compliance, performance and stability.
> 
> Also already mooted for next cycle

Discussed and agreed to push discussion back to after the release in
email discussions prior to the release plan.

 
> > - Experimental MS Exchange Protocol Support.
> 
> +0 I don't know enough about this, but if its experimental, then fine.

No idea what this is, but you're free to check in a new proposal at any
time.  Just create an appropriate directory/build structure under
proposals.  You can announce it on the list, and attempt to attract
developers to work on it.  That's what proposals is for.
 
> > - Move away from Avalon/Phoenix. There has been some talk on this
> > topic. We
> > could keep the framework part and not use Phoenix if this is a
sensible
> > goal.
> 
> If we fork james for this, and allow a non Phoenix version to develop
in
> parallel then Ok, otherwise -1

This is a total -1.  If someone wishes to take James and fork, that's
their prerogative.  But I certainly don't want to spend time and effort
supporting them.  This would constitute a real design change, and one
without any actual apparent motivation.  The release version of James
takes advantage of the Avalon Framework in general and Phoenix in
particular.  Certainly (as was discussed extensively in July and August)

i) James doesn't need to be dependent on Phoenix - it should be able to
run in any container, although not all features will necessarily be
available.

ii) The Avalon Framework is a good thing for the server internals.  It
makes life much easier.
 
--Peter
 



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Commit First? (was RE: Code change the Apache way)

[Peter M. Goldstein]

Steve,

If there's general consensus on that, I've got no objection to skipping
the post.  I've been posting largely so people can evaluate before
commit, as committing tends to be viewed by some as a "final" act.  

--Peter

> -Original Message-
> From: Steve Short [mailto:sshort@;postx.com]
> Sent: Friday, October 25, 2002 11:11 AM
> To: James Developers List
> Subject: RE: Code change the Apache way
> 
> 
> Given that you are going to commit this code, I don't see any benefit
in
> posting new updates without committing.  Surely we want to be testing
> this as much as possible and this is much easier to do out of CVS
rather
> than CVS plus patches.
> 
> Regards
> Steve
> 
> > -Original Message-
> > From: Peter M. Goldstein [mailto:peter_m_goldstein@;yahoo.com]
> > Sent: Friday, October 25, 2002 10:58 AM
> > To: 'James Developers List'
> > Subject: RE: Code change the Apache way
> >
> >
> >
> > Danny,
> >
> > > Are we near to seeing stuff commited.. ?
> > > apart from the obvious fact that some contentious items can
> > be vetoed
> > > *after* commit, I'd say that as long as it builds we should be
> > commiting
> > > often.
> >
> > Noel and I are currently running more tests, and everything
> > is coming up green.  What I will do is post the latest,
> > greatest code for evaluation later today.  If the consensus
> > is that I should commit said code, I'll be more than happy to
> > do so.  I have simply been unwilling to do so over an outstanding
-1.
> >
> > --Peter
> >
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <mailto:james-dev-> [EMAIL PROTECTED]>
> > For
> > additional commands,
> > e-mail: <mailto:james-dev-help@;jakarta.apache.org>
> >
> >
> 
> --
> To unsubscribe, e-mail:   <mailto:james-dev-
> [EMAIL PROTECTED]>
> For additional commands, e-mail: <mailto:james-dev-
> [EMAIL PROTECTED]>



--
To unsubscribe, e-mail:   <mailto:james-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@;jakarta.apache.org>

Current changes (was RE: Code change the Apache way)

[Peter M. Goldstein]

> Some suggestions to facilitate changes:
> - Cleanup/redesign few things at a time not 5 things together

We would have.  Except you keep blocking submission with -1 votes.  And
Noel keeps trying to figure out creative ways to appease you while
keeping the momentum to improve the product going

> - Keep scope as close to what is being attempted. We have been making
more
> design changes than a bug fix/cleanup needs.

I'm baffled by your definition of "design changes".  Basically you seem
to think that any change that

i) Removes a file
ii) Reorganizes the methods of a file

constitutes a design change.  It doesn't.

Of the changes discussed to date, the only one that is arguably a
"design change" is removing the dependency on the scheduler.  And that
one was discussed ad nauseum over the summer.  Everyone agreed, the
scheduler had to go.  Everyone but you still does.  We have successful
demonstrated tests of a scheduler-free James that runs like a bat out of
hell.  You haven't put the time and energy in to bolster your position -
you haven't provided a .sar upon request.  But we're still stuck on your
-1.

This behavior is exactly the sort of plodding thinking that let this
trivial bug

http://issues.apache.org/bugzilla/show_bug.cgi?id=5824

sit around for nine months.  In short, if something doesn't work it
doesn't constitute a design.  It's only when

i) An architectural issue has been discussed and documented
ii) The basic functionality expected of that feature is provided (bugs
are allowed, critical ones are not)

that you have anything that can possibly constitute a design.  The above
criteria ensure that the proposed feature/arrangement/etc. works, and
that understanding of the ideas involved is passed to the community.
Otherwise, you've got nothing.

I have a real problem with this sort of paralysis.  It basically
guarantees that the product goes nowhere.

In the last three months the James community has, by group effort:

i) Closed James as an open SMTP relay
ii) Added a new service for fetching POP3 mail to a single mailbox
iii) Improved the performance of the spooler by 50%+, depending on the
configuration.
iv) Removed multiple critical memory leaks
v) Made the NNTP service come into line with spec
vi) Made NNTP SSL and auth functional
vii) Improved SMTP performance by 900%.  Other handlers should see a
substantial improvement as well.
viii) Reduced heap size under load dramatically.
ix) Fixed god knows how many other bugs

At no time save this one did we debate whether these were "design
changes".  We examined the code base and modified it as necessary to
make James work properly.  In some cases files were reorganized or
deleted, but this was almost never a real consideration as to whether
the fix would be accepted. There was discussion in almost all cases of
whether the proposed fixes were the right fixes, but the community came
to a consensus and the fixes were made in short order.  The current
discussion, forced entirely by you, is easily the longest discussion
we've had on implementing a bug fix during my time at James.  And easily
the least productive.

As software developers, we are bound by our own sense of professionalism
to maintain support for our published APIs until and unless we announce
that the next version will not support them.  This is a responsibility
to our user base, and it is one I have championed vigorously.

The same logic doesn't hold true for the details of our internal
classes.  Frankly, I don't care how the system currently works
internally - that's not a criterion that we use to evaluate change.
There is nothing sacrosanct about the current code.  In most of the
above cases the current "design" was flawed, in either a major
(AuthService) or a minor (SMTPHandler) way.  So they were fixed.  That
applies whether the original author of the code is Serge, you, me, or
anyone else.  That's the way successful software projects work.   

--Peter




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Code change the Apache way

[Peter M. Goldstein]

Danny,

> Are we near to seeing stuff commited.. ?
> apart from the obvious fact that some contentious items can be vetoed
> *after* commit, I'd say that as long as it builds we should be
commiting
> often.

Noel and I are currently running more tests, and everything is coming up
green.  What I will do is post the latest, greatest code for evaluation
later today.  If the consensus is that I should commit said code, I'll
be more than happy to do so.  I have simply been unwilling to do so over
an outstanding -1.

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Where do we go from here.

[Peter M. Goldstein]

All,

> > Just speaking philosophically (and not about any particular
decision),
> > what do people think of this approach:  when we hit a piece of the
> > Avalon codebase (framework, excalibur, what-have-you) that we're
unhappy
> > with, then we make a copy of it, possibly even repackage it, add it
to
> > James, and make the code work for our needs.  Often we will know a
small
> > piece is not working, but may not know the perfect solution until we
use
> > it in James for a while (a few test runs, maybe a few months).  I
think
> > we would benefit by not getting (feeling) stuck waiting for an
Avalon
> > release, and the Avalon projects would benefit with more people
building
> > components.  This can be done as healthy competition, and if we
> > ultimately figure a better way to build a component, we can
certainly
> > pass it back to Avalon's codebase.
> 
> +1
> 
> this isn't far different from the way we might use OS code if we were
a
> commercial company building a product.
> As long as we contribute improvements back to Avalon, and don't just
rip
> it off, then I'm cool with that.

+1

Yep.  And this is exactly what we've been doing.  If you check out the
SimpleConnectionManager you'll note comments in the Javadoc that it
should probably be submitted to Avalon Cornerstone at some point.  So
here's what we did.  We

i) Took a look at some Avalon code and found that it didn't meet our
needs.
ii) Wrote code that was in large part based on reading the Avalon code
and modifying to our needs
iii) Prepared to submit code to Avalon Cornerstone for consideration.

Step (iv), actual submission, will happen after we get this release out
the door.

--Peter





--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: Code change the Apache way

[Peter M. Goldstein]

Serge,

> > - Keep scope as close to what is being attempted. We have been
making
> more
> > design changes than a bug fix/cleanup needs.
> 
> Yes which are fixing show-stopper bugs and improving performace by 30%
+.
> 
> Whats an email server if you can't count on it to do its job? How many
> people keep asking "is it production ready?"

Agreed.  One minor correction.  Performance improvement isn't 30%, it's
900%.  Prior to these changes James crashed under a load of ~6
mails/second (360 mails/minute).  With the current code base (Watchdog,
object pooling, deleteonexit removed) we are getting a performance of
~60 mails/second (3600 mails/minute).

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH] Removing Scheduler dependency, refactoring service code

[Peter M. Goldstein]

Harmeet et al,

> - There is no loss in keeping BaseHandler except for cleanup value.
> - I think it is going to be useful as there will be common code in
> handlers
> that would be best handled in base class.  There have been
> patches/cleanups
> proposed but not committed that you pointed out. There may be more in
> future.
> 
> Why don't you leave BaseHandler where it is for now and see if there
are
> some patterns across handlers that can be refactored into base.
> 

I find this logic ridiculous.  And I just don't buy it.  It just doesn't
explain the almost religious devotion you've expressed to
BaseConnectionHandler in the face of all logic and reason.  Even here
you can't express an actual commonality that the BaseConnectionHandler
would take advantage of measured against a clear benefit (what you refer
to as cleanup).

We have repeatedly stated that if and when a need for
BaseConnectionHandler or its like arose, we would introduce such a
class.  But it's clear from the current code base and from the proposed
patch that almost zero common behavior exists.  So why won't you let
this one go?

Let me give you my theory, shall I?  You mentioned in your email
retracting the -1 on Noel that you had used parts of James in other
projects.  How much do we want to be that you are using the handlers
with another service mechanism?  One that requires a common subclass for
your handlers - like BaseConnectionHandler ?  Could this be the unstated
motivation behind your religious devotion to BaseConnectionHandler.
Somehow I think so...

Why wouldn't you be explicit about this issue?  Again, the reason is
obvious.  You know that the list would reject the needs of your personal
software as a valid reason.  And rightly so.  So you make repeated
statements, none of which have any actual backing or coherent logic.
Like the one quoted above.

Personally, I didn't sign up to work for kodemuse.org.  I find this
behavior an unethical abuse of committer privileges.  I'll let the other
members of this list read the back and forth and decide whether they
agree with my assessment.  But I'd bet $100 that if we ripped open the
Kodemuse code we'd find a non-James service class using
BaseConnectionHandler...  Am I wrong?


--Peter  



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

RE: [PATCH] Removing Scheduler dependency, refactoring service code

[Peter M. Goldstein]

All,

> > if we are adding value to Avalon's service,
> > why don't we push this into Avalon.
> 
> As I understand it from Peter and Andrei (I don't consider myself an
> Avalon
> guru), the question is what value does AbstractService provide?  It
isn't
> a
> generic interface, it is a particular subclassable implementation
provided
> by Cornerstone.  In the case of AbstractService and
AbstractJamesService,
> every method would need to be overridden, as well as the new ones
added.
> 
> If you want to argue that there should be an Avalon Frameworks
interface
> for
> these kinds of services, that'd be for the Avalon folks to take up,
and
> I'm
> sure that Peter and Peter could do something about the service model
if
> Peter thought it worthwhile (I'll leave interpretation of the
overloaded
> name as an exercise for the reader ;-)).  But in the meantime,
> AbstractJamesService models the behavior James needs, and would gain
> nothing
> from extending AbstractService.
> 
> Frankly, Peter and Andrei are far more Avalon knowledgable than I, and
> ought
> to be able to explain (or correct) this better to everyone.
> 

Noel is correct, on all points.

To put this in perspective, not a single application in the Avalon-apps
repository uses AbstractService as a base class for their components.
That includes applications such as an FTP server for which such an
implementation might make sense.  AbstractService is just a random
implementation that someone wrote - there's nothing special about it.

Andrei and I rewrote AbstractService because it didn't meet our needs -
it was tied to the old ConnectionManager implementation, didn't match
our configuration, etc.  

--Peter



--
To unsubscribe, e-mail:   
For additional commands, e-mail: