Re: Spoolmanager blues
> >>I have now tried to downgrade to 2.2.0, and I cannot reproduce the > >>effect there (only tried filerepositories on this version). > > > >This is strange because I experienced exactly the same > identical issue. > > > I never such issue with file repositories in my company production > system: neither in 2.2.0 nor in 2.3.0 (up to end of June this > year) nor > in any CVS/SVN intermediate versions. This only happens the first time and with a single message. With an empty repository run james. Send a single message via SMTP. It hangs in the spool. When you send more messages the hanging mail is processed, too. I can reproduce this in windows/jdk1.4.2_07. My last patch to AvalonMailRepository solve the problem in my box but I don't know what other bugs can introduce. I'm trying a different patch for JDBC spool repository delays. I'm running stress-test now. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
Stefano Bagnara wrote: I have now tried to downgrade to 2.2.0, and I cannot reproduce the effect there (only tried filerepositories on this version). This is strange because I experienced exactly the same identical issue. I never such issue with file repositories in my company production system: neither in 2.2.0 nor in 2.3.0 (up to end of June this year) nor in any CVS/SVN intermediate versions. Vincenzo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
I at least cannot reproduce the behaviour, which lead to this patch any longer. I will do some more tests, and report if any anomalies occur. --Søren On Wednesday 07 September 2005 12:53, Stefano Bagnara wrote: > I just committed my patch to AvalonMailRepository. > > IMHO this is a critical change and we should test it a lot before the > release. > > I'm now trying the same chages in the JDBCMailRepository to see if this > removes the "delay". > > Stefano > > > > I also tried using Derby/db on trunk, and saw "just" the delay. > > > > This is "known". > > > > > I will try out your patch. > > > > Don't try the one I pasted. I already had to change it due to > > unsynchronized notification. > > Just testing a further change, hope I'll commit the change > > soon (stay tuned). > > > > Stefano > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. R&D manager Phone: +45 72 30 64 00 TietoEnator IT+ A/S Fax:+45 72 30 64 02 Ved Lunden 12 Direct: +45 72 30 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Re: Spoolmanager blues
I just committed my patch to AvalonMailRepository. IMHO this is a critical change and we should test it a lot before the release. I'm now trying the same chages in the JDBCMailRepository to see if this removes the "delay". Stefano > > I also tried using Derby/db on trunk, and saw "just" the delay. > > This is "known". > > > I will try out your patch. > > Don't try the one I pasted. I already had to change it due to > unsynchronized notification. > Just testing a further change, hope I'll commit the change > soon (stay tuned). > > Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Re: Spoolmanager blues
> I have now tried to downgrade to 2.2.0, and I cannot > reproduce the effect there (only tried filerepositories on > this version). This is strange because I experienced exactly the same identical issue. > I also tried using Derby/db on trunk, and saw "just" the delay. This is "known". > I will try out your patch. Don't try the one I pasted. I already had to change it due to unsynchronized notification. Just testing a further change, hope I'll commit the change soon (stay tuned). Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
I have now tried to downgrade to 2.2.0, and I cannot reproduce the effect
there (only tried filerepositories on this version).
I also tried using Derby/db on trunk, and saw "just" the delay.
I will try out your patch.
--Søren
On Wednesday 07 September 2005 11:30, Stefano Bagnara wrote:
> > > Can you try downgrading to 2.2.0 and verify wether the
> >
> > issue is there
> >
> > > or not?
> >
> > Will do that today.
> >
> > > Can you try using db/derby to check wether the issue is
> >
> > there or not?
> >
> >
> > Sure.
>
> I'm testing a patch to both JDBC and Avalon repositories.
> It seems working better than before. I'll run my stress-tests and
> eventually commit the code so you can test it, too!
>
> Stefano
>
> --
>
>
> Index:
> james/src/java/org/apache/james/mailrepository/AvalonMailRepository.java
> ===
> ---
> james/src/java/org/apache/james/mailrepository/AvalonMailRepository.java
> (revision 233041)
> +++
> james/src/java/org/apache/james/mailrepository/AvalonMailRepository.java
> (working copy)
> @@ -205,6 +205,7 @@
> //synchronized (this) {
> //notifyAll();
> //}
> +notify();
> return true;
> } else {
> return false;
> Index:
> james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
> ===
> --- james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
> (revision 233041)
> +++ james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
> (working copy)
> @@ -494,7 +494,7 @@
> *
> * @return true if successfully released the lock, false otherwise
> */
> -public synchronized boolean unlock(String key) {
> +public boolean unlock(String key) {
> if (lock.unlock(key)) {
> if ((DEEP_DEBUG) && (getLogger().isDebugEnabled())) {
> StringBuffer debugBuffer =
> @@ -508,6 +508,7 @@
> getLogger().debug(debugBuffer.toString());
> }
> //notifyAll();
> +notify();
> return true;
> } else {
> return false;
> @@ -521,7 +522,7 @@
> *
> * @return true if successfully obtained the lock, false otherwise
> */
> -public synchronized boolean lock(String key) {
> +public boolean lock(String key) {
> if (lock.lock(key)) {
> if ((DEEP_DEBUG) && (getLogger().isDebugEnabled())) {
> StringBuffer debugBuffer =
> @@ -546,7 +547,17 @@
> */
> public void store(MailImpl mc) throws MessagingException {
> Connection conn = null;
> +boolean wasLocked = true;
> +String key = mc.getName();
> try {
> +synchronized(this) {
> + wasLocked = lock.isLocked(key);
> +
> + if (!wasLocked) {
> + //If it wasn't locked, we want a lock during the
> store
> + lock.lock(key);
> + }
> +}
> conn = datasource.getConnection();
>
> //Need to determine whether need to insert this record, or
> update it.
> @@ -775,6 +786,10 @@
> throw new MessagingException("Exception caught while storing
> mail Container: " + e);
> } finally {
> theJDBCUtil.closeJDBCConnection(conn);
> +if (!wasLocked) {
> +//If it wasn't locked, we need to now unlock
> +lock.unlock(key);
> +}
> }
> }
>
>
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 72 30 64 00
TietoEnator IT+ A/S Fax:+45 72 30 64 02
Ved Lunden 12 Direct: +45 72 30 64 57
DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
> > Can you try downgrading to 2.2.0 and verify wether the
> issue is there
> > or not?
>
> Will do that today.
>
> > Can you try using db/derby to check wether the issue is
> there or not?
> >
>
> Sure.
I'm testing a patch to both JDBC and Avalon repositories.
It seems working better than before. I'll run my stress-tests and eventually
commit the code so you can test it, too!
Stefano
--
Index:
james/src/java/org/apache/james/mailrepository/AvalonMailRepository.java
===
--- james/src/java/org/apache/james/mailrepository/AvalonMailRepository.java
(revision 233041)
+++ james/src/java/org/apache/james/mailrepository/AvalonMailRepository.java
(working copy)
@@ -205,6 +205,7 @@
//synchronized (this) {
//notifyAll();
//}
+notify();
return true;
} else {
return false;
Index:
james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
===
--- james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
(revision 233041)
+++ james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
(working copy)
@@ -494,7 +494,7 @@
*
* @return true if successfully released the lock, false otherwise
*/
-public synchronized boolean unlock(String key) {
+public boolean unlock(String key) {
if (lock.unlock(key)) {
if ((DEEP_DEBUG) && (getLogger().isDebugEnabled())) {
StringBuffer debugBuffer =
@@ -508,6 +508,7 @@
getLogger().debug(debugBuffer.toString());
}
//notifyAll();
+notify();
return true;
} else {
return false;
@@ -521,7 +522,7 @@
*
* @return true if successfully obtained the lock, false otherwise
*/
-public synchronized boolean lock(String key) {
+public boolean lock(String key) {
if (lock.lock(key)) {
if ((DEEP_DEBUG) && (getLogger().isDebugEnabled())) {
StringBuffer debugBuffer =
@@ -546,7 +547,17 @@
*/
public void store(MailImpl mc) throws MessagingException {
Connection conn = null;
+boolean wasLocked = true;
+String key = mc.getName();
try {
+synchronized(this) {
+ wasLocked = lock.isLocked(key);
+
+ if (!wasLocked) {
+ //If it wasn't locked, we want a lock during the
store
+ lock.lock(key);
+ }
+}
conn = datasource.getConnection();
//Need to determine whether need to insert this record, or
update it.
@@ -775,6 +786,10 @@
throw new MessagingException("Exception caught while storing
mail Container: " + e);
} finally {
theJDBCUtil.closeJDBCConnection(conn);
+if (!wasLocked) {
+//If it wasn't locked, we need to now unlock
+lock.unlock(key);
+}
}
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
On Tuesday 06 September 2005 15:25, Stefano Bagnara wrote: > > > I'm experiencing spooling issues with file repositories too. > > > > Well, I haven't until now :-( > > Can you try downgrading to 2.2.0 and verify wether the issue is there or > not? Will do that today. > Can you try using db/derby to check wether the issue is there or not? > Sure. > My tests results are that filerepositories have this problem in 2.2.0 and > in trunk. Db is working in both. > > > > Can you try using the db/derby repositories and see wether > > > > you see the > > > > > same problem? > > > > I could, but I need file repositories for my production > > deployments, so I will try to figure this one out instead. > > I would start looking for differences between JDBCMailRepository and > AvalonMailRepository (e.g: JDBCMailRepository.unlock is synchronized) > More: AvalonMailRepository.store does lock/unlock a message not already > locked while JDBCMailRepository does not lock/unlock such messages. > > I've experienced issues with DB repositories also, but they just delayed > the spooling by 60 seconds (probably, for db, the notify does not work > fine). > > I don't use FileRepositories in production because my stresstests provided > exceptions in locking/unlocking EVERY time: > http://issues.apache.org/jira/browse/JAMES-397 > > Stefano > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. R&D manager Phone: +45 72 30 64 00 TietoEnator IT+ A/S Fax:+45 72 30 64 02 Ved Lunden 12 Direct: +45 72 30 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
On Tuesday 06 September 2005 15:12, Stefano Bagnara wrote: > > I am having trouble with the JamesSpoolManager in the trunk. > > I experience mails hanging in the spool, it looks like the > > offending piece of code is the return statement in line 418. > > The reason I suspect that line is that I cannot reproduce the > > effect if I remove the line. > > If you remove the return you change the current behaviour. Yes, I realize that. I just remembered (like Noel) that I had seen a patch a while back which included this exact return, commented out. > > Currently each spool thread get a message from the spool, run it in the > processor associated with the current spool and returns. > > If you remove the return then a single thread will bring the mail to GHOST > in a single processing. At the end of each processor the LinearProcessor > will store each mail in the spool (currently there are no drowbacks in > storing the same message multiple time without accepting them again, but I > did not changed it because I'm not sure it is a good thing) > > IMHO the problem is in the spool.unlock implementation for the file > repository. Yes, you are probably right. > > Stefano > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. R&D manager Phone: +45 72 30 64 00 TietoEnator IT+ A/S Fax:+45 72 30 64 02 Ved Lunden 12 Direct: +45 72 30 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Spoolmanager blues
> > > I'm experiencing spooling issues with file repositories too. > > Actually, I have occassionally seen the same symptom with JDBC, too. > Simple delay of messages or full hangs of single messages? Just the simple delay. As an aside, we need to make sure that dbfile is working with Derby. It is a common production configuration to use a database with a single file location for all message content. > > The problem *appears* to be with notify(). This is why I > > have wanted us to simplify the logic by having one spool > > thread and plain worker threads that just process what > > they are given. > it's not an easy change mainly because the contract between > mail/spool repositories and james is not clear enough. We should define it. We own those contracts. > What is threadsafe in a repository? The exposed put/get type operations should be thread safe. > What is the correct use of locking/unlocking? Ideally, we should be able to eliminate much of it, and isolate any remaining notion of it. A single spool thread / multiple worker pool change should be helpful in this regard. > Can I store the same message in 2 different repositories? I'd like to be able to say no, but I suspect that we need for some version of the answer to be yes. > Is it safe to store the same message in the same repository > multiple times? We need to distinguish between calling store() and actually storing copies. > Currently we run the notify when the message is stored but > when the message is stored we still got the lock on the Mail I'll try to find time to look at the spooler code again. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
> > Soren wrote: > > > I am having trouble with the JamesSpoolManager in the trunk. > > > I experience mails hanging in the spool, it looks like > the offending > > > piece of code is the return statement in line 418. > > To be clear, that is the return statement in > process(MailImpl) that does a return after a processor has > successfully handled the message? Yes. > > > The reason I suspect that line is that I cannot reproduce > the effect > > > if I remove the line. > > For some reason, I had thought that Stefano was changing the > spooler to process the entire pipeline for an e-mail in one > shot, rather than queuing for each processor, but since he > just said that he hadn't made the change, and obviously the > code doesn't have it, perhaps it was just something he had > talked about. I seem to recall being concerned about that change. I just talked about it. I also changed it locally and attached a patch to a JIRA issue but it was not working so I commented this on JIRA and never applied changes that way. > > I'm experiencing spooling issues with file repositories too. > > Actually, I have occassionally seen the same symptom with JDBC, too. Simple delay of messages or full hangs of single messages? > > If you send more mails you will see that the one in the spool will > > start being elaborated. > > The problem *appears* to be with notify(). This is why I > have wanted us to simplify the logic by having one spool > thread and plain worker threads that just process what they are given. I now understand better the spooling mechanism and it's not an easy change mainly because the contract between mail/spool repositories and james is not clear enough. What is threadsafe in a repository? What is the correct use of locking/unlocking? Can I store the same message in 2 different repositories? Is it safe to store the same message in the same repository multiple times? Currently we run the notify when the message is stored but when the message is stored we still got the lock on the Mail (unless it was generated by a matcher split/duplicate or by a new sendmail). When we run the unlock there is no notify so the thread will wait for another mail and no other spoolthread will get the message we just stored. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Spoolmanager blues
Stefano Bagnara wrote: > Soren wrote: > > I am having trouble with the JamesSpoolManager in the trunk. > > I experience mails hanging in the spool, it looks like the > > offending piece of code is the return statement in line 418. To be clear, that is the return statement in process(MailImpl) that does a return after a processor has successfully handled the message? > > The reason I suspect that line is that I cannot reproduce > > the effect if I remove the line. For some reason, I had thought that Stefano was changing the spooler to process the entire pipeline for an e-mail in one shot, rather than queuing for each processor, but since he just said that he hadn't made the change, and obviously the code doesn't have it, perhaps it was just something he had talked about. I seem to recall being concerned about that change. > I'm experiencing spooling issues with file repositories too. Actually, I have occassionally seen the same symptom with JDBC, too. > If you send more mails you will see that the one in the spool > will start being elaborated. The problem *appears* to be with notify(). This is why I have wanted us to simplify the logic by having one spool thread and plain worker threads that just process what they are given. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
> > I'm experiencing spooling issues with file repositories too. > > Well, I haven't until now :-( Can you try downgrading to 2.2.0 and verify wether the issue is there or not? Can you try using db/derby to check wether the issue is there or not? My tests results are that filerepositories have this problem in 2.2.0 and in trunk. Db is working in both. > > Can you try using the db/derby repositories and see wether > you see the > > same problem? > > I could, but I need file repositories for my production > deployments, so I will try to figure this one out instead. I would start looking for differences between JDBCMailRepository and AvalonMailRepository (e.g: JDBCMailRepository.unlock is synchronized) More: AvalonMailRepository.store does lock/unlock a message not already locked while JDBCMailRepository does not lock/unlock such messages. I've experienced issues with DB repositories also, but they just delayed the spooling by 60 seconds (probably, for db, the notify does not work fine). I don't use FileRepositories in production because my stresstests provided exceptions in locking/unlocking EVERY time: http://issues.apache.org/jira/browse/JAMES-397 Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
> I am having trouble with the JamesSpoolManager in the trunk. > I experience mails hanging in the spool, it looks like the > offending piece of code is the return statement in line 418. > The reason I suspect that line is that I cannot reproduce the > effect if I remove the line. If you remove the return you change the current behaviour. Currently each spool thread get a message from the spool, run it in the processor associated with the current spool and returns. If you remove the return then a single thread will bring the mail to GHOST in a single processing. At the end of each processor the LinearProcessor will store each mail in the spool (currently there are no drowbacks in storing the same message multiple time without accepting them again, but I did not changed it because I'm not sure it is a good thing) IMHO the problem is in the spool.unlock implementation for the file repository. Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
On Tuesday 06 September 2005 14:59, Stefano Bagnara wrote: > > Hi, > > > > I am having trouble with the JamesSpoolManager in the trunk. > > I experience mails hanging in the spool, it looks like the > > offending piece of code is the return statement in line 418. > > The reason I suspect that line is that I cannot reproduce the > > effect if I remove the line. > > > > Have anyone seen something similar, or can someone (Stefano) > > provide the rationale behind this particular return statement. > > I will do some more debugging to better understand what happens. > > I'm experiencing spooling issues with file repositories too. Well, I haven't until now :-( > > The return statement is there since long before I committed my patches ;-) > (the annotation show that the return line has been written on may 2001). In > fact I only done minor changes to the spoolmanager (it now just gets > mailetloader/matcher loader as avalon services instead of running its own). > > I've also tested 2.2.0 and I've experienced the same spool problems using > file repositories. > > IMHO file repositories lock/unlock is not (has never been?) working fine. Probably true. > > If you send more mails you will see that the one in the spool will start > being elaborated. Yes, just tried that, maybe as you say it has allways been there, only I have just noticed it now. > > Can you try using the db/derby repositories and see wether you see the same > problem? I could, but I need file repositories for my production deployments, so I will try to figure this one out instead. --Søren > > Stefano > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. R&D manager Phone: +45 72 30 64 00 TietoEnator IT+ A/S Fax:+45 72 30 64 02 Ved Lunden 12 Direct: +45 72 30 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Spoolmanager blues
> Hi, > > I am having trouble with the JamesSpoolManager in the trunk. > I experience mails hanging in the spool, it looks like the > offending piece of code is the return statement in line 418. > The reason I suspect that line is that I cannot reproduce the > effect if I remove the line. > > Have anyone seen something similar, or can someone (Stefano) > provide the rationale behind this particular return statement. > I will do some more debugging to better understand what happens. I'm experiencing spooling issues with file repositories too. The return statement is there since long before I committed my patches ;-) (the annotation show that the return line has been written on may 2001). In fact I only done minor changes to the spoolmanager (it now just gets mailetloader/matcher loader as avalon services instead of running its own). I've also tested 2.2.0 and I've experienced the same spool problems using file repositories. IMHO file repositories lock/unlock is not (has never been?) working fine. If you send more mails you will see that the one in the spool will start being elaborated. Can you try using the db/derby repositories and see wether you see the same problem? Stefano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Spoolmanager blues
Hi, I am having trouble with the JamesSpoolManager in the trunk. I experience mails hanging in the spool, it looks like the offending piece of code is the return statement in line 418. The reason I suspect that line is that I cannot reproduce the effect if I remove the line. Have anyone seen something similar, or can someone (Stefano) provide the rationale behind this particular return statement. I will do some more debugging to better understand what happens. regards Søren -- Søren Hilmer, M.Sc. R&D manager Phone: +45 72 30 64 00 TietoEnator IT+ A/S Fax:+45 72 30 64 02 Ved Lunden 12 Direct: +45 72 30 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
