Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2 Sep 2003, Berin Loritsch wrote: > Giacomo Pati wrote: > > > On Tue, 2 Sep 2003, Bruno Dumon wrote: > > > > > >>On Tue, 2003-09-02 at 15:21, Carsten Ziegeler wrote: > >> > >>>Bruno Dumon wrote: > >>> > >Another idea: could you try my original changed jar available here: > >http://outerthought.net/~bruno/concurrent.jar > > Never mind, I can reproduce the problem now on marc's pc. > > >>> > >>>Great, so I'm not insane! (I tried your version anyway, but without success) > >> > >>Nope, in fact I'm very glad you tried it out because indeed there's > >>another problem in there. I don't have a quick solution for it though. > >>I've created (well, most of the credit goes to Marc) a testcase to > >>reproduce the problem and mailed it to Doug Lea. > > > > > > Hmm.. my Scheduler that uses Doug Leas kit dosn't have any problems at > > all. > > Are you using the PooledExecutor, or something else? PoolExecutor. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 12:48, Giacomo Pati wrote: > On Mon, 1 Sep 2003, Bruno Dumon wrote: > > > On Mon, 2003-09-01 at 16:29, Giacomo Pati wrote: > > > On Mon, 1 Sep 2003, Bruno Dumon wrote: > > > > > > > On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > > > > > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > > > > > > > > > > > > > > > > > On Behalf Of Giacomo Pati > > > > > > > > > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > Just in case you didn't know: > > > > > > > > Did you have a look at the Quartz Scheduler? > > > > > > > > http://www.part.net/quartz.html > > > > > > > > http://www.part.net/quartz_features.html > > > > > > > > Perhaps it already does what you need. > > > > > > > > It's licensed under a BSD and Apache-Style license. > > > > > > > > > > > > > > I've seen it some time ago but thought it is quite too much > > > > > > > for a simple cron mechanism we are discussing here. > > > > > > > > > > > > > > Any other thoughts? > > > > > > > > > > > > Do you think it is difficult to use/to integrate? From the features list > > > > > > it is exactly what we need, isn't it? > > > > > > > > > > IMHO, from the feature list it is way too much we need, isn't it? > > > > > > > > I think many of it are optional features, and those extra features might > > > > come in handy some day. If it does everything we need and if it's solid, > > > > why not use it? > > > > > > Hmm, the cocurrency util from Doug Lea does alot Quartz offers. > > > > Hmm, as far as I know, only the threadpool? (which is of course an > > important part). Quartz has either a very simple thread pool of its own > > or otherwise uses something from jakarta commons. I think it would be > > easy to put PooledExecutor behind it. > > Ok, I've rewritten the Scheduler using Quartz as scheduling engine (and > using its CronTrigger class) but put an PooledExecutor underneath as the > ThreadPool. > > > > So my > > > oppinion was not to add another dependency. > > > > Sometimes it sounds more compelling to add another dependency maintained > > by another community than to add more code to Cocoon. > > Ok, convinced :-) > > > > Quartz (if I've read the > > > feature list correctly) doesn't offer Cron spec parsing, > > > > Hope you look a bit further than the first web page to judge a piece of > > software (imagine if people would do that for cocoon...) > > Sorry, yes now I've seen it and it has quite more feature as my > implementation. > > > See this: > > http://quartz.sourceforge.net/javadoc/org/quartz/CronTrigger.html > > > > Furthermore, from having a quick look, quartz is quite modular, with > > different implementations available, in its simplest form without any > > third party jars requirements. > > Yup. I've seen. > > > > > It also seems to be able to calculate the time to perform to the next > > task to be performed. > > > > > so we have to > > > do it anyway. And because I've alread done this for another project some > > > times ago I thought to but it into a Scheduler by it own (mean in the > > > org.apache.cocoon package space). > > > > > > So, the scheduler I've proposed is already done. I just have to > > > repackage it and add some more javadocs etc. It's about to be ready to > > > be committed tomorrow. > > > > > > If there are still objection I hold on the commits until we reached > > > consensus. > > > > I have two more questions: > > > > * what do we need a scheduler for in Cocoon in the first place? > > > > * would your scheduler be able to handle the ContinuationsManager > > cleanup task? The specific requirement I'm thinking of is that it > > shouldn't be launched concurrently, so if the previous cleanup is still > > running it shouldn't be launched a second time. > > The interface for a Scheduler so far isn't complete but, yes, The > ContinuationsManager could launch it's own jobs as the Scheduler is > implemented as a regular Avalon component. > > I've also implemented configurable en-/disable parallel runs. ok. Sounds all nice and good. One open point now is what the scheduler is needed for. I mean, the ContinuationsManager cleanup task doesn't have any fancy cron scheduling needs, and could be run as well by the CommandManager (which I think is present anyway in fortress to manage component pool sizes assynchronously, so might as well be used). This would remove the requirement to put the scheduler into the core, so that it could be put in a block. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 21:09, Giacomo Pati wrote: > On Tue, 2 Sep 2003, Bruno Dumon wrote: > > > On Tue, 2003-09-02 at 15:21, Carsten Ziegeler wrote: > > > Bruno Dumon wrote: > > > > > > > > > > Another idea: could you try my original changed jar available here: > > > > > http://outerthought.net/~bruno/concurrent.jar > > > > > > > > Never mind, I can reproduce the problem now on marc's pc. > > > > > > > Great, so I'm not insane! (I tried your version anyway, but without success) > > > > Nope, in fact I'm very glad you tried it out because indeed there's > > another problem in there. I don't have a quick solution for it though. > > I've created (well, most of the credit goes to Marc) a testcase to > > reproduce the problem and mailed it to Doug Lea. > > Hmm.. my Scheduler that uses Doug Leas kit dosn't have any problems at > all. Nope, you won't notice it easily, the only reason that it's so easily reproducable in our current cocoon use is because on the one hand the thread expires after 100ms, and on the other hand it is reused every 100 ms, so we very easily get in the situation where a thread is just being freed when another one is being requested, and it's there where it goes wrong. Changing parameters a very little bit will make this situation much less likely to occur, but still this needs to be fixed. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: I don't want to spend very much time on this thread anymore, but here's the short summary: Understandable. Again, I do want to fix the issues. Also note that it is accessible to Cocoon committers. I don't have any problem with the concepts behind the CommandManager/ThreadManager or the event package in general, in fact I think they're very good. My last point about the continuous checking of events every 100 ms isn't really a big issue for me. I was catching up over the weekend. I only wanted to point out some problems in the current implementation as it is. Those problems can of course be fixed, and for Cocoon to depend on this system, they also need to be fixed. And maybe I'm also wrong on some of these problems. Some of them I think are misconceptions. One of these problems is the above mentioned fact that all events from one CommandManager are executed on one thread, no matter how many threads you configure in the ThreadManager. I got this only from looking at the code, here's the explanation I mentioned earlier in this same thread, I hope you can prove me wrong: This is one of the misconceptions. More info below. If you look at the code of the AbstractThreadManager: In the method "register( EventPipeline pipeline )" one PipelineRunner is created to handle events from the given EventPipeline (e.g. the one of the CommandManager). Then in the run() method the PipelineRunners are executed in parallel by using the PooledExecutor: m_executor.execute( nextRunner ); But in the PipelineRunner.run() method, and the CommandManager.handleEvents() which is called by it, no new threads are started. The PooledExecutor has a group of threads associated with it. In fact the executor is what creates/reuses/manages the threads. The CommandManager does not have to create the threads itself. If you set the THreadManager (which manages the threads in the PooledExecutor) to use a certain number of threads, it will use them all. You are never limited to only one thread. I usually keep two worker threads per processor working all the time. There is now also a new ThreadManager that has a thread pool (PooledExecutor) per PipelineRunner that is registered. I don't recommend its use just yet because it hasn't been tested fully. However, the CommandManager will use as many threads as the ThreadManager will allow. -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Bruno Dumon wrote: > On Mon, 2003-09-01 at 16:29, Giacomo Pati wrote: > > On Mon, 1 Sep 2003, Bruno Dumon wrote: > > > > > On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > > > > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > > > > > > > > > > > > > On Behalf Of Giacomo Pati > > > > > > > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > Just in case you didn't know: > > > > > > > Did you have a look at the Quartz Scheduler? > > > > > > > http://www.part.net/quartz.html > > > > > > > http://www.part.net/quartz_features.html > > > > > > > Perhaps it already does what you need. > > > > > > > It's licensed under a BSD and Apache-Style license. > > > > > > > > > > > > I've seen it some time ago but thought it is quite too much > > > > > > for a simple cron mechanism we are discussing here. > > > > > > > > > > > > Any other thoughts? > > > > > > > > > > Do you think it is difficult to use/to integrate? From the features list > > > > > it is exactly what we need, isn't it? > > > > > > > > IMHO, from the feature list it is way too much we need, isn't it? > > > > > > I think many of it are optional features, and those extra features might > > > come in handy some day. If it does everything we need and if it's solid, > > > why not use it? > > > > Hmm, the cocurrency util from Doug Lea does alot Quartz offers. > > Hmm, as far as I know, only the threadpool? (which is of course an > important part). Quartz has either a very simple thread pool of its own > or otherwise uses something from jakarta commons. I think it would be > easy to put PooledExecutor behind it. Ok, I've rewritten the Scheduler using Quartz as scheduling engine (and using its CronTrigger class) but put an PooledExecutor underneath as the ThreadPool. > > So my > > oppinion was not to add another dependency. > > Sometimes it sounds more compelling to add another dependency maintained > by another community than to add more code to Cocoon. Ok, convinced :-) > > Quartz (if I've read the > > feature list correctly) doesn't offer Cron spec parsing, > > Hope you look a bit further than the first web page to judge a piece of > software (imagine if people would do that for cocoon...) Sorry, yes now I've seen it and it has quite more feature as my implementation. > See this: > http://quartz.sourceforge.net/javadoc/org/quartz/CronTrigger.html > > Furthermore, from having a quick look, quartz is quite modular, with > different implementations available, in its simplest form without any > third party jars requirements. Yup. I've seen. > > It also seems to be able to calculate the time to perform to the next > task to be performed. > > > so we have to > > do it anyway. And because I've alread done this for another project some > > times ago I thought to but it into a Scheduler by it own (mean in the > > org.apache.cocoon package space). > > > > So, the scheduler I've proposed is already done. I just have to > > repackage it and add some more javadocs etc. It's about to be ready to > > be committed tomorrow. > > > > If there are still objection I hold on the commits until we reached > > consensus. > > I have two more questions: > > * what do we need a scheduler for in Cocoon in the first place? > > * would your scheduler be able to handle the ContinuationsManager > cleanup task? The specific requirement I'm thinking of is that it > shouldn't be launched concurrently, so if the previous cleanup is still > running it shouldn't be launched a second time. The interface for a Scheduler so far isn't complete but, yes, The ContinuationsManager could launch it's own jobs as the Scheduler is implemented as a regular Avalon component. I've also implemented configurable en-/disable parallel runs. > > Other than that, I'll leave it up to your good judgement on what path to > follow, I don't have any experience with any schedulers anyway... > > -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Giacomo Pati wrote: On Tue, 2 Sep 2003, Bruno Dumon wrote: On Tue, 2003-09-02 at 15:21, Carsten Ziegeler wrote: Bruno Dumon wrote: Another idea: could you try my original changed jar available here: http://outerthought.net/~bruno/concurrent.jar Never mind, I can reproduce the problem now on marc's pc. Great, so I'm not insane! (I tried your version anyway, but without success) Nope, in fact I'm very glad you tried it out because indeed there's another problem in there. I don't have a quick solution for it though. I've created (well, most of the credit goes to Marc) a testcase to reproduce the problem and mailed it to Doug Lea. Hmm.. my Scheduler that uses Doug Leas kit dosn't have any problems at all. Are you using the PooledExecutor, or something else? -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2 Sep 2003, Bruno Dumon wrote: > On Tue, 2003-09-02 at 15:21, Carsten Ziegeler wrote: > > Bruno Dumon wrote: > > > > > > > > Another idea: could you try my original changed jar available here: > > > > http://outerthought.net/~bruno/concurrent.jar > > > > > > Never mind, I can reproduce the problem now on marc's pc. > > > > > Great, so I'm not insane! (I tried your version anyway, but without success) > > Nope, in fact I'm very glad you tried it out because indeed there's > another problem in there. I don't have a quick solution for it though. > I've created (well, most of the credit goes to Marc) a testcase to > reproduce the problem and mailed it to Doug Lea. Hmm.. my Scheduler that uses Doug Leas kit dosn't have any problems at all. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 19:24, Berin Loritsch wrote: > Bruno Dumon wrote: > > > > > Looking at the code again, I still think that all events from one > > CommandManager are handled by one thread. > > > > If you look at the code of the AbstractThreadManager: > > > > In the method "register( EventPipeline pipeline )" one PipelineRunner is > > created to handle events from the given EventPipeline (e.g. the one of > > the CommandManager). > > > > Then in the run() method the PipelineRunners are executed in parallel by > > using the PooledExecutor: m_executor.execute( nextRunner ); > > The PooledExecutor takes care of initiating and managing the running threads. > It also takes care of running the tasks in those threads. D'oh! I looked over the fact that the same PipelineRunner can of course be run multiple times concurrently, and thus handle commands for the same CommandManager. Stupid of me :-) -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 15:52, Berin Loritsch wrote: > Bruno Dumon wrote: > > > > > > Or maybe not so great. I'm not sure the CommandManager is well suited > > for a general purpose scheduler. > > > > What follows now should be read with a 'AFAIU' disclaimer: > > > > All commands added to the CommandManager are executed sequentially and > > on one thread. So if there is one command that takes somewhat longer to > > execute, it will block the execution of other commands. This may be fine > > for management-tasks which should be executed repeatedly (but not at > > fixed times), but for a scheduler I would expect a command to be > > executed at the configured time. > > The number of threads is configurable. By default it is based on a function > of the number of processors. In Cocoon it is currently 1 thread per processor > (plus one to manage the queue). If you have two processors, you have two > threads. Alternatively you can implement two threads per processor (works > well in most situations). This approach makes sure that long running tasks > won't block all other items in the queue. > > The Scheduler/ThreadPool combo you have with the Cornerstone components does > not give you anything more in this area. The queue is moved to the Scheduler, > and the ThreadPool will block until a thread is made available. The main > difference is that most folks typically assign more threads to the ThreadPool > than you have assigned to the ThreadManager. > > > This could be solved by creating a different CommandManager for each > > scheduled task, since multiple CommandManagers can be executed in > > parallel. But then you'd still want to use a different ThreadManager > > than the TPCThreadManager, since that will block when all threads are in > > use. > > Huh? No. The ThreadManager does the actual execution. The CommandManager > merely organizes the commands and sets them up in the queue. > > > > The result will however be rather overkill IMHO: even commands that need > > to be executed only weekly, will be checked every 100 ms (= sleeptime > > parameter) to see if they need to be executed. > > > > Conclusion: while this event-based infrastructure is quite cool, I'm not > > convinced yet it is good for scheduling-purposes. > > No matter how you slice it, even the Scheduler has this tradeoff. You can > set the sleeptime longer if you never have anything checked less then 5 second > intervals or minute intervals, but in the end, the overhead of checking the > commands every 100 ms is so minimal that you would never see it. Esp. since > that checking is done asynchronously from your critical path. There is a > dedicated thread in each case that makes the checks for you--essentially > the only difference is in how many other components are needed to get the > job done. Berin, I don't want to spend very much time on this thread anymore, but here's the short summary: I don't have any problem with the concepts behind the CommandManager/ThreadManager or the event package in general, in fact I think they're very good. My last point about the continuous checking of events every 100 ms isn't really a big issue for me. I only wanted to point out some problems in the current implementation as it is. Those problems can of course be fixed, and for Cocoon to depend on this system, they also need to be fixed. And maybe I'm also wrong on some of these problems. One of these problems is the above mentioned fact that all events from one CommandManager are executed on one thread, no matter how many threads you configure in the ThreadManager. I got this only from looking at the code, here's the explanation I mentioned earlier in this same thread, I hope you can prove me wrong: If you look at the code of the AbstractThreadManager: In the method "register( EventPipeline pipeline )" one PipelineRunner is created to handle events from the given EventPipeline (e.g. the one of the CommandManager). Then in the run() method the PipelineRunners are executed in parallel by using the PooledExecutor: m_executor.execute( nextRunner ); But in the PipelineRunner.run() method, and the CommandManager.handleEvents() which is called by it, no new threads are started. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: Looking at the code again, I still think that all events from one CommandManager are handled by one thread. If you look at the code of the AbstractThreadManager: In the method "register( EventPipeline pipeline )" one PipelineRunner is created to handle events from the given EventPipeline (e.g. the one of the CommandManager). Then in the run() method the PipelineRunners are executed in parallel by using the PooledExecutor: m_executor.execute( nextRunner ); The PooledExecutor takes care of initiating and managing the running threads. It also takes care of running the tasks in those threads. But in the PipelineRunner.run() method, and the CommandManager.handleEvents() which is called by it, no new threads are started. There is no reason for the CommandManager to create new threads when that logic is handled by the PooledExecutor. -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Nicola Ken Barozzi wrote: I'll have to start to play golf just to find a use for my tonsils) Eeeck. Spare us the details! ;-) -- Steven Noelshttp://outerthought.org/ Outerthought - Open Source Java & XMLAn Orixo Member Read my weblog athttp://blogs.cocoondev.org/stevenn/ stevenn at outerthought.orgstevenn at apache.org
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 15:21, Carsten Ziegeler wrote: > Bruno Dumon wrote: > > > > > > Another idea: could you try my original changed jar available here: > > > http://outerthought.net/~bruno/concurrent.jar > > > > Never mind, I can reproduce the problem now on marc's pc. > > > Great, so I'm not insane! (I tried your version anyway, but without success) Nope, in fact I'm very glad you tried it out because indeed there's another problem in there. I don't have a quick solution for it though. I've created (well, most of the credit goes to Marc) a testcase to reproduce the problem and mailed it to Doug Lea. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Carsten Ziegeler wrote, On 02/09/2003 15.21: Bruno Dumon wrote: Another idea: could you try my original changed jar available here: http://outerthought.net/~bruno/concurrent.jar Never mind, I can reproduce the problem now on marc's pc. Great, so I'm not insane! (I tried your version anyway, but without success) OT: seems like me a few days ago, when I phoned my mom happily saying "I have fever! :-D " because I felt groggy but nobody really took it seriously before ;-) (I'm quite sane now BTW, apart from a sore throat that is again getting worse and the fact that I'll have to start to play golf just to find a use for my tonsils) -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) -
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: Or maybe not so great. I'm not sure the CommandManager is well suited for a general purpose scheduler. What follows now should be read with a 'AFAIU' disclaimer: All commands added to the CommandManager are executed sequentially and on one thread. So if there is one command that takes somewhat longer to execute, it will block the execution of other commands. This may be fine for management-tasks which should be executed repeatedly (but not at fixed times), but for a scheduler I would expect a command to be executed at the configured time. The number of threads is configurable. By default it is based on a function of the number of processors. In Cocoon it is currently 1 thread per processor (plus one to manage the queue). If you have two processors, you have two threads. Alternatively you can implement two threads per processor (works well in most situations). This approach makes sure that long running tasks won't block all other items in the queue. The Scheduler/ThreadPool combo you have with the Cornerstone components does not give you anything more in this area. The queue is moved to the Scheduler, and the ThreadPool will block until a thread is made available. The main difference is that most folks typically assign more threads to the ThreadPool than you have assigned to the ThreadManager. This could be solved by creating a different CommandManager for each scheduled task, since multiple CommandManagers can be executed in parallel. But then you'd still want to use a different ThreadManager than the TPCThreadManager, since that will block when all threads are in use. Huh? No. The ThreadManager does the actual execution. The CommandManager merely organizes the commands and sets them up in the queue. The result will however be rather overkill IMHO: even commands that need to be executed only weekly, will be checked every 100 ms (= sleeptime parameter) to see if they need to be executed. Conclusion: while this event-based infrastructure is quite cool, I'm not convinced yet it is good for scheduling-purposes. No matter how you slice it, even the Scheduler has this tradeoff. You can set the sleeptime longer if you never have anything checked less then 5 second intervals or minute intervals, but in the end, the overhead of checking the commands every 100 ms is so minimal that you would never see it. Esp. since that checking is done asynchronously from your critical path. There is a dedicated thread in each case that makes the checks for you--essentially the only difference is in how many other components are needed to get the job done. -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: > > > > Another idea: could you try my original changed jar available here: > > http://outerthought.net/~bruno/concurrent.jar > > Never mind, I can reproduce the problem now on marc's pc. > Great, so I'm not insane! (I tried your version anyway, but without success) Carsten
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 14:42, Bruno Dumon wrote: > On Tue, 2003-09-02 at 12:48, Carsten Ziegeler wrote: > > Bruno Dumon wrote: > > > > > > > > > Looks ok, I changed the parameters in the cocoon.xconf though, but > > > that's the same. > > > > > > Just tried it over here on a Windows box with java 1.4.2 and it also > > > works. With "working" I mean I added a System.out.println to the > > > ContinuationsInterrupt.execute() method and see it printed every 2 > > > seconds. > > Yepp, I did the same and I have w2k, jdk 1.4.2 as well. Hmm strange. > > > > > > > > > And then the execute() method is sometimes invoked once (before jetty is > > > > up completly) and sometimes not at all. > > > > > > Sounds like the "old" bug, but I don't know what's causing it now (and I > > > can't reproduce it over here). > > > > > Hmm, I did a clean checkout, changed the ContinuationManager as explained > > and then a simple build and it's not working. I also checked the classpath > > and it's a clean JDK 1.4.2 classpath. > > > > Just checking, but are you sure that your version is checked in? The jar is > > new, but is the contents? (Forgive me the question, but we must have a > > difference somewhere). > > No problem, but I'm rather sure. > > Another idea: could you try my original changed jar available here: > http://outerthought.net/~bruno/concurrent.jar Never mind, I can reproduce the problem now on marc's pc. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 12:48, Carsten Ziegeler wrote: > Bruno Dumon wrote: > > > > > > Looks ok, I changed the parameters in the cocoon.xconf though, but > > that's the same. > > > > Just tried it over here on a Windows box with java 1.4.2 and it also > > works. With "working" I mean I added a System.out.println to the > > ContinuationsInterrupt.execute() method and see it printed every 2 > > seconds. > Yepp, I did the same and I have w2k, jdk 1.4.2 as well. Hmm strange. > > > > > > And then the execute() method is sometimes invoked once (before jetty is > > > up completly) and sometimes not at all. > > > > Sounds like the "old" bug, but I don't know what's causing it now (and I > > can't reproduce it over here). > > > Hmm, I did a clean checkout, changed the ContinuationManager as explained > and then a simple build and it's not working. I also checked the classpath > and it's a clean JDK 1.4.2 classpath. > > Just checking, but are you sure that your version is checked in? The jar is > new, but is the contents? (Forgive me the question, but we must have a > difference somewhere). No problem, but I'm rather sure. Another idea: could you try my original changed jar available here: http://outerthought.net/~bruno/concurrent.jar -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: > > > Looks ok, I changed the parameters in the cocoon.xconf though, but > that's the same. > > Just tried it over here on a Windows box with java 1.4.2 and it also > works. With "working" I mean I added a System.out.println to the > ContinuationsInterrupt.execute() method and see it printed every 2 > seconds. Yepp, I did the same and I have w2k, jdk 1.4.2 as well. Hmm strange. > > > And then the execute() method is sometimes invoked once (before jetty is > > up completly) and sometimes not at all. > > Sounds like the "old" bug, but I don't know what's causing it now (and I > can't reproduce it over here). > Hmm, I did a clean checkout, changed the ContinuationManager as explained and then a simple build and it's not working. I also checked the classpath and it's a clean JDK 1.4.2 classpath. Just checking, but are you sure that your version is checked in? The jar is new, but is the contents? (Forgive me the question, but we must have a difference somewhere). What about others? Is it working for you? Or not? Carsten
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 10:10, Carsten Ziegeler wrote: > Bruno Dumon wrote: > > > > I just verified and it works here (both jdk 1.4 and 1.3), so I'm looking > > for alternative explanations: did you remove the old util.concurrent jar > > from your WEB-INF/lib ? > > > Yes. Hmm, strange - just to make sure that I did nothing wrong when testing, > I changed the following two methods in the interrupt: > > public long getRepeatInterval() { > return 2000; > } > > /** > * Get the number of millis to wait for the first invocation > */ > public long getDelayInterval() { > return 0; > } > Looks ok, I changed the parameters in the cocoon.xconf though, but that's the same. Just tried it over here on a Windows box with java 1.4.2 and it also works. With "working" I mean I added a System.out.println to the ContinuationsInterrupt.execute() method and see it printed every 2 seconds. > And then the execute() method is sometimes invoked once (before jetty is > up completly) and sometimes not at all. Sounds like the "old" bug, but I don't know what's causing it now (and I can't reproduce it over here). -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: > > I just verified and it works here (both jdk 1.4 and 1.3), so I'm looking > for alternative explanations: did you remove the old util.concurrent jar > from your WEB-INF/lib ? > Yes. Hmm, strange - just to make sure that I did nothing wrong when testing, I changed the following two methods in the interrupt: public long getRepeatInterval() { return 2000; } /** * Get the number of millis to wait for the first invocation */ public long getDelayInterval() { return 0; } And then the execute() method is sometimes invoked once (before jetty is up completly) and sometimes not at all. Carsten
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 09:16, Carsten Ziegeler wrote: > Bruno Dumon wrote: > > > > On Tue, 2003-09-02 at 09:01, Carsten Ziegeler wrote: > > > Hi, > > > > > > I just tested the last changes and it seems that the > > continuation manager > > > is still not working correctly. > > > > > > It seems that the ContinuationInterrupt is still not executed as soon > > > as jetty is listening. Can someone confirm this? > > > > Did you change the default offset in the cocoon.xconf? By default, it > > waits 18 ms before the first execution. > > > Yes, I changed it to 0, and the interval to 2 secs (2000 ms). The interrupt > is fired immediately until jetty is up and running then the execution stops. > I just verified and it works here (both jdk 1.4 and 1.3), so I'm looking for alternative explanations: did you remove the old util.concurrent jar from your WEB-INF/lib ? -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Bruno Dumon wrote: > > On Tue, 2003-09-02 at 09:01, Carsten Ziegeler wrote: > > Hi, > > > > I just tested the last changes and it seems that the > continuation manager > > is still not working correctly. > > > > It seems that the ContinuationInterrupt is still not executed as soon > > as jetty is listening. Can someone confirm this? > > Did you change the default offset in the cocoon.xconf? By default, it > waits 18 ms before the first execution. > Yes, I changed it to 0, and the interval to 2 secs (2000 ms). The interrupt is fired immediately until jetty is up and running then the execution stops. Carsten
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Tue, 2003-09-02 at 09:01, Carsten Ziegeler wrote: > Hi, > > I just tested the last changes and it seems that the continuation manager > is still not working correctly. > > It seems that the ContinuationInterrupt is still not executed as soon > as jetty is listening. Can someone confirm this? Did you change the default offset in the cocoon.xconf? By default, it waits 18 ms before the first execution. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Hi, I just tested the last changes and it seems that the continuation manager is still not working correctly. It seems that the ContinuationInterrupt is still not executed as soon as jetty is listening. Can someone confirm this? Carsten
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 2003-09-01 at 16:29, Giacomo Pati wrote: > On Mon, 1 Sep 2003, Bruno Dumon wrote: > > > On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > > > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > > > > > > > > > On Behalf Of Giacomo Pati > > > > > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > > > > > Hi! > > > > > > > > > > > > Just in case you didn't know: > > > > > > Did you have a look at the Quartz Scheduler? > > > > > > http://www.part.net/quartz.html > > > > > > http://www.part.net/quartz_features.html > > > > > > Perhaps it already does what you need. > > > > > > It's licensed under a BSD and Apache-Style license. > > > > > > > > > > I've seen it some time ago but thought it is quite too much > > > > > for a simple cron mechanism we are discussing here. > > > > > > > > > > Any other thoughts? > > > > > > > > Do you think it is difficult to use/to integrate? From the features list > > > > it is exactly what we need, isn't it? > > > > > > IMHO, from the feature list it is way too much we need, isn't it? > > > > I think many of it are optional features, and those extra features might > > come in handy some day. If it does everything we need and if it's solid, > > why not use it? > > Hmm, the cocurrency util from Doug Lea does alot Quartz offers. Hmm, as far as I know, only the threadpool? (which is of course an important part). Quartz has either a very simple thread pool of its own or otherwise uses something from jakarta commons. I think it would be easy to put PooledExecutor behind it. > So my > oppinion was not to add another dependency. Sometimes it sounds more compelling to add another dependency maintained by another community than to add more code to Cocoon. > Quartz (if I've read the > feature list correctly) doesn't offer Cron spec parsing, Hope you look a bit further than the first web page to judge a piece of software (imagine if people would do that for cocoon...) See this: http://quartz.sourceforge.net/javadoc/org/quartz/CronTrigger.html Furthermore, from having a quick look, quartz is quite modular, with different implementations available, in its simplest form without any third party jars requirements. It also seems to be able to calculate the time to perform to the next task to be performed. > so we have to > do it anyway. And because I've alread done this for another project some > times ago I thought to but it into a Scheduler by it own (mean in the > org.apache.cocoon package space). > > So, the scheduler I've proposed is already done. I just have to > repackage it and add some more javadocs etc. It's about to be ready to > be committed tomorrow. > > If there are still objection I hold on the commits until we reached > consensus. I have two more questions: * what do we need a scheduler for in Cocoon in the first place? * would your scheduler be able to handle the ContinuationsManager cleanup task? The specific requirement I'm thinking of is that it shouldn't be launched concurrently, so if the previous cleanup is still running it shouldn't be launched a second time. Other than that, I'll leave it up to your good judgement on what path to follow, I don't have any experience with any schedulers anyway... -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > From: Bruno Dumon > > > > > On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > > > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > > > > > > > > > On Behalf Of Giacomo Pati > > > > > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > > > > > Hi! > > > > > > > > > > > > Just in case you didn't know: > > > > > > Did you have a look at the Quartz Scheduler? > > > > > > http://www.part.net/quartz.html > > > > > > http://www.part.net/quartz_features.html > > > > > > Perhaps it already does what you need. > > > > > > It's licensed under a BSD and Apache-Style license. > > > > > > > > > > I've seen it some time ago but thought it is quite too > > much for a > > > > > simple cron mechanism we are discussing here. > > > > > > > > > > Any other thoughts? > > > > > > > > Do you think it is difficult to use/to integrate? From > > the features > > > > list it is exactly what we need, isn't it? > > > > > > IMHO, from the feature list it is way too much we need, isn't it? > > > > I think many of it are optional features, and those extra > > features might come in handy some day. If it does everything > > we need and if it's solid, why not use it? > > I think Giacomo is right that we won't need all those features within > the Cocoon project but I think our users will need the most in their > projects. No sure here. The users I know best are happy with full Cron like functionality. > If Quartz does everything we want I'm +1 for adding it to > Cocoon. So we would also avoid developing and maintainging our own > scheduler solution because what I understood the Avalon solutions don't > fit all our needs, do they? As I've written in response to Brunos mail, Quartz doesn't offer Cron spec parsing so we have to do that in the cocoon space anyway. The concurrency utils (we already have a dependency of) does all we need to trigger execution (as Quartz does) so why adding another dependency. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Bruno Dumon wrote: > On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > > > > > On Behalf Of Giacomo Pati > > > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > > > Hi! > > > > > > > > > > Just in case you didn't know: > > > > > Did you have a look at the Quartz Scheduler? > > > > > http://www.part.net/quartz.html > > > > > http://www.part.net/quartz_features.html > > > > > Perhaps it already does what you need. > > > > > It's licensed under a BSD and Apache-Style license. > > > > > > > > I've seen it some time ago but thought it is quite too much > > > > for a simple cron mechanism we are discussing here. > > > > > > > > Any other thoughts? > > > > > > Do you think it is difficult to use/to integrate? From the features list > > > it is exactly what we need, isn't it? > > > > IMHO, from the feature list it is way too much we need, isn't it? > > I think many of it are optional features, and those extra features might > come in handy some day. If it does everything we need and if it's solid, > why not use it? Hmm, the cocurrency util from Doug Lea does alot Quartz offers. So my oppinion was not to add another dependency. Quartz (if I've read the feature list correctly) doesn't offer Cron spec parsing, so we have to do it anyway. And because I've alread done this for another project some times ago I thought to but it into a Scheduler by it own (mean in the org.apache.cocoon package space). So, the scheduler I've proposed is already done. I just have to repackage it and add some more javadocs etc. It's about to be ready to be committed tomorrow. If there are still objection I hold on the commits until we reached consensus. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Hunsberger, Peter wrote: > Giacomo Pati <[EMAIL PROTECTED]> asks: > > > > I did this once upon a time, I think in C code. As I recall it's > > > basically the equivalent of building a Map of tasks and the > > calculated > > > delta time until they need to be executed next and then using the > > > minimum of these deltas. I don't think it's all that complex? This > > > could be a little problematic if the delta's are in the > > same order of > > > magnitude as the time to calculate them :-), but that certainly > > > doesn't seem to be the case here > > > > Where is that code, dude :-) > > Well, after looking around on my hard drive; I still have it! It is 10 > year old OS/2 text mode code for pop-up alarms... Looking at it, it has > a linked list of data structures that it scans after each alarm > occurrence looking for the min next delta time. That simple description > is probably more useful than the code, but I can send a Zip to you > privately if you really want it (careful what you ask for)... Ok, before you send me the zip, I have a question: Is that code also capable of determining the smallest delta from a set of Crontab like 'points-in-time' definitions? I do not have any linked list because it is too painfull to build such a list out of these definitions, isn't it? -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Giacomo Pati <[EMAIL PROTECTED]> asks: > > I did this once upon a time, I think in C code. As I recall it's > > basically the equivalent of building a Map of tasks and the > calculated > > delta time until they need to be executed next and then using the > > minimum of these deltas. I don't think it's all that complex? This > > could be a little problematic if the delta's are in the > same order of > > magnitude as the time to calculate them :-), but that certainly > > doesn't seem to be the case here > > Where is that code, dude :-) Well, after looking around on my hard drive; I still have it! It is 10 year old OS/2 text mode code for pop-up alarms... Looking at it, it has a linked list of data structures that it scans after each alarm occurrence looking for the min next delta time. That simple description is probably more useful than the code, but I can send a Zip to you privately if you really want it (careful what you ask for)...
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Hunsberger, Peter wrote: > Giacomo Pati <[EMAIL PROTECTED]> writes: > > > > > First, if you look at the Cornerstone Scheduler you'll see > > that the ability to define repeated tasks are somehow > > limited. Business apps can easily request administration > > tasks to be run twice a day on work days. To define this with > > the Cornerstone Scheduler you'll need to add 10 'cron' > > entries to it. This is why I'd like to extend it using the > > unix cron direction to define ranges and list of timing > > aspects. But then you're faced with the problem of defining > > sleep time for efficient use (which you need to be able to > > cooperate in the Cornerstone Scheduler machinery). > > > > The unix cron daemon runs every minute to check if there are > > any jobs to run. It checks initially creating bitsets for the > > minute, hour, day, month, and weekday definition for each > > task. This makes it easy to check whether a task has to be > > launched. I still haven't found an algorithm (and I think > > there isn't an easy one available) to determine the sleep > > time until the next task is due. > > I did this once upon a time, I think in C code. As I recall it's > basically the equivalent of building a Map of tasks and the calculated > delta time until they need to be executed next and then using the > minimum of these deltas. I don't think it's all that complex? This > could be a little problematic if the delta's are in the same order of > magnitude as the time to calculate them :-), but that certainly doesn't > seem to be the case here Where is that code, dude :-) -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
RE: CommandManager issues [was Re: Releasing 2.1.1?]
Giacomo Pati <[EMAIL PROTECTED]> writes: > First, if you look at the Cornerstone Scheduler you'll see > that the ability to define repeated tasks are somehow > limited. Business apps can easily request administration > tasks to be run twice a day on work days. To define this with > the Cornerstone Scheduler you'll need to add 10 'cron' > entries to it. This is why I'd like to extend it using the > unix cron direction to define ranges and list of timing > aspects. But then you're faced with the problem of defining > sleep time for efficient use (which you need to be able to > cooperate in the Cornerstone Scheduler machinery). > > The unix cron daemon runs every minute to check if there are > any jobs to run. It checks initially creating bitsets for the > minute, hour, day, month, and weekday definition for each > task. This makes it easy to check whether a task has to be > launched. I still haven't found an algorithm (and I think > there isn't an easy one available) to determine the sleep > time until the next task is due. I did this once upon a time, I think in C code. As I recall it's basically the equivalent of building a Map of tasks and the calculated delta time until they need to be executed next and then using the minimum of these deltas. I don't think it's all that complex? This could be a little problematic if the delta's are in the same order of magnitude as the time to calculate them :-), but that certainly doesn't seem to be the case here
RE: CommandManager issues [was Re: Releasing 2.1.1?]
From: Bruno Dumon > > On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > > > > > On Behalf Of Giacomo Pati > > > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > > > Hi! > > > > > > > > > > Just in case you didn't know: > > > > > Did you have a look at the Quartz Scheduler? > > > > > http://www.part.net/quartz.html > > > > > http://www.part.net/quartz_features.html > > > > > Perhaps it already does what you need. > > > > > It's licensed under a BSD and Apache-Style license. > > > > > > > > I've seen it some time ago but thought it is quite too > much for a > > > > simple cron mechanism we are discussing here. > > > > > > > > Any other thoughts? > > > > > > Do you think it is difficult to use/to integrate? From > the features > > > list it is exactly what we need, isn't it? > > > > IMHO, from the feature list it is way too much we need, isn't it? > > I think many of it are optional features, and those extra > features might come in handy some day. If it does everything > we need and if it's solid, why not use it? I think Giacomo is right that we won't need all those features within the Cocoon project but I think our users will need the most in their projects. If Quartz does everything we want I'm +1 for adding it to Cocoon. So we would also avoid developing and maintainging our own scheduler solution because what I understood the Avalon solutions don't fit all our needs, do they? Cheers, Reinhard
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 2003-09-01 at 11:37, Giacomo Pati wrote: > On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > > > > > On Behalf Of Giacomo Pati > > > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > > > Hi! > > > > > > > > Just in case you didn't know: > > > > Did you have a look at the Quartz Scheduler? > > > > http://www.part.net/quartz.html > > > > http://www.part.net/quartz_features.html > > > > Perhaps it already does what you need. > > > > It's licensed under a BSD and Apache-Style license. > > > > > > I've seen it some time ago but thought it is quite too much > > > for a simple cron mechanism we are discussing here. > > > > > > Any other thoughts? > > > > Do you think it is difficult to use/to integrate? From the features list > > it is exactly what we need, isn't it? > > IMHO, from the feature list it is way too much we need, isn't it? I think many of it are optional features, and those extra features might come in handy some day. If it does everything we need and if it's solid, why not use it? -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Reinhard Poetz wrote: > > > On Behalf Of Giacomo Pati > > > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > > > Hi! > > > > > > Just in case you didn't know: > > > Did you have a look at the Quartz Scheduler? > > > http://www.part.net/quartz.html > > > http://www.part.net/quartz_features.html > > > Perhaps it already does what you need. > > > It's licensed under a BSD and Apache-Style license. > > > > I've seen it some time ago but thought it is quite too much > > for a simple cron mechanism we are discussing here. > > > > Any other thoughts? > > Do you think it is difficult to use/to integrate? From the features list > it is exactly what we need, isn't it? IMHO, from the feature list it is way too much we need, isn't it? -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
RE: CommandManager issues [was Re: Releasing 2.1.1?]
On Behalf Of Giacomo Pati > On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > > > Hi! > > > > Just in case you didn't know: > > Did you have a look at the Quartz Scheduler? > > http://www.part.net/quartz.html > > http://www.part.net/quartz_features.html > > Perhaps it already does what you need. > > It's licensed under a BSD and Apache-Style license. > > I've seen it some time ago but thought it is quite too much > for a simple cron mechanism we are discussing here. > > Any other thoughts? Do you think it is difficult to use/to integrate? From the features list it is exactly what we need, isn't it? Cheers, Reinhard
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Mon, 1 Sep 2003, Andreas Hochsteger wrote: > Hi! > > Just in case you didn't know: > Did you have a look at the Quartz Scheduler? > http://www.part.net/quartz.html > http://www.part.net/quartz_features.html > Perhaps it already does what you need. > It's licensed under a BSD and Apache-Style license. I've seen it some time ago but thought it is quite too much for a simple cron mechanism we are discussing here. Any other thoughts? > > Here's another one: > http://jcrontab.sourceforge.net/ It's (L)GPL > > I found the links on this page: > http://wiki.cocoondev.org/Wiki.jsp?page=WyonaScheduler > > Bye, > Andreas > > Giacomo Pati wrote: > > On Sun, 31 Aug 2003, Bruno Dumon wrote: > > > > > >>On Sun, 2003-08-31 at 20:38, Giacomo Pati wrote: > >> > >>>First, if you look at the Cornerstone Scheduler you'll see that the > >>>ability to define repeated tasks are somehow limited. Business apps can > >>>easily request administration tasks to be run twice a day on work days. > >>>To define this with the Cornerstone Scheduler you'll need to add 10 > >>>'cron' entries to it. This is why I'd like to extend it using the unix > >>>cron direction to define ranges and list of timing aspects. But then > >>>you're faced with the problem of defining sleep time for efficient use > >>>(which you need to be able to cooperate in the Cornerstone Scheduler > >>>machinery). > >>> > >>>The unix cron daemon runs every minute to check if there are any jobs to > >>>run. It checks initially creating bitsets for the minute, hour, day, > >>>month, and weekday definition for each task. This makes it easy to > >>>check whether a task has to be launched. I still haven't found an > >>>algorithm (and I think there isn't an easy one available) to determine > >>>the sleep time until the next task is due. > >> > >>Ah, didn't think that far. > >> > >> > >>> Thus, a check every minute > >>>isn't much but still 1440 times a day. > >> > >>But a lot less then the 864000 times a day in case of a check every 100 > >>ms :-) > > > > > > Sure. > > > > > >>>The other thing to respect if you'd like to go as efficent as possible > >>>is the determination of Threads you'll need to run each cron task at the > >>>time specified. But this needs knowledge about how much elapsed time a > >>>task need to run to decide if you need one or two thread to complete to > >>>task defined neat in time to each other. If you have an algorithm for > >>>this you'll be able to define the maximum numbers of Thread you'll need. > >>>Otherwise you allocate a separate Thread per defined task. > >>> > >>>The consequences are that probably the easiest solution to a (more or > >>>less) complete Cron like Scheduler is an own heart beat of one minute to > >>>check for task to launch and a pool of Threads as big as you have task > >>>to process. As it seems Doug Leas concurrency tools could be configured > >>>to be as resource sparing as possible (have a look at > >>>http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/PooledExecutor.html) > >> > >>Yep, I think that will be fine too. > > > > > > Ok, so I'll gona build up a Scheduler based on its own heart beat (own > > Thread) with Crond expressions for timing specifications that uses a > > PooledExecutor for the Triggers in the size of the numbers of Task > > defined in the configuration but with lazy construction of Thread > > objects. This will not need any additional jars and the ones introduced > > by the Cornerstone Sccheduler can be eliminated again. Is everybody > > happy with that? > > > > -- > > Giacomo Pati > > Otego AG, Switzerland - http://www.otego.com > > Orixo, the XML business alliance - http://www.orixo.com > > > > > > > -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Hi! Just in case you didn't know: Did you have a look at the Quartz Scheduler? http://www.part.net/quartz.html http://www.part.net/quartz_features.html Perhaps it already does what you need. It's licensed under a BSD and Apache-Style license. Here's another one: http://jcrontab.sourceforge.net/ I found the links on this page: http://wiki.cocoondev.org/Wiki.jsp?page=WyonaScheduler Bye, Andreas Giacomo Pati wrote: On Sun, 31 Aug 2003, Bruno Dumon wrote: On Sun, 2003-08-31 at 20:38, Giacomo Pati wrote: First, if you look at the Cornerstone Scheduler you'll see that the ability to define repeated tasks are somehow limited. Business apps can easily request administration tasks to be run twice a day on work days. To define this with the Cornerstone Scheduler you'll need to add 10 'cron' entries to it. This is why I'd like to extend it using the unix cron direction to define ranges and list of timing aspects. But then you're faced with the problem of defining sleep time for efficient use (which you need to be able to cooperate in the Cornerstone Scheduler machinery). The unix cron daemon runs every minute to check if there are any jobs to run. It checks initially creating bitsets for the minute, hour, day, month, and weekday definition for each task. This makes it easy to check whether a task has to be launched. I still haven't found an algorithm (and I think there isn't an easy one available) to determine the sleep time until the next task is due. Ah, didn't think that far. Thus, a check every minute isn't much but still 1440 times a day. But a lot less then the 864000 times a day in case of a check every 100 ms :-) Sure. The other thing to respect if you'd like to go as efficent as possible is the determination of Threads you'll need to run each cron task at the time specified. But this needs knowledge about how much elapsed time a task need to run to decide if you need one or two thread to complete to task defined neat in time to each other. If you have an algorithm for this you'll be able to define the maximum numbers of Thread you'll need. Otherwise you allocate a separate Thread per defined task. The consequences are that probably the easiest solution to a (more or less) complete Cron like Scheduler is an own heart beat of one minute to check for task to launch and a pool of Threads as big as you have task to process. As it seems Doug Leas concurrency tools could be configured to be as resource sparing as possible (have a look at http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/PooledExecutor.html) Yep, I think that will be fine too. Ok, so I'll gona build up a Scheduler based on its own heart beat (own Thread) with Crond expressions for timing specifications that uses a PooledExecutor for the Triggers in the size of the numbers of Task defined in the configuration but with lazy construction of Thread objects. This will not need any additional jars and the ones introduced by the Cornerstone Sccheduler can be eliminated again. Is everybody happy with that? -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: CommandManager issues [was Re: Releasing 2.1.1?]
Giacomo Pati wrote: First, if you look at the Cornerstone Scheduler you'll see that the ability to define repeated tasks are somehow limited. Business apps can easily request administration tasks to be run twice a day on work days. To define this with the Cornerstone Scheduler you'll need to add 10 'cron' entries to it. This is why I'd like to extend it using the unix cron direction to define ranges and list of timing aspects. But then you're faced with the problem of defining sleep time for efficient use (which you need to be able to cooperate in the Cornerstone Scheduler machinery). I'll repeat this "issue". I'd like to see the "cron" type entries take "*" like normal crontabs do. We've used this with my GenericTaskManager and doing repeated entires is a pain. David
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Sun, 31 Aug 2003, Bruno Dumon wrote: > On Sun, 2003-08-31 at 20:38, Giacomo Pati wrote: > > > > First, if you look at the Cornerstone Scheduler you'll see that the > > ability to define repeated tasks are somehow limited. Business apps can > > easily request administration tasks to be run twice a day on work days. > > To define this with the Cornerstone Scheduler you'll need to add 10 > > 'cron' entries to it. This is why I'd like to extend it using the unix > > cron direction to define ranges and list of timing aspects. But then > > you're faced with the problem of defining sleep time for efficient use > > (which you need to be able to cooperate in the Cornerstone Scheduler > > machinery). > > > > The unix cron daemon runs every minute to check if there are any jobs to > > run. It checks initially creating bitsets for the minute, hour, day, > > month, and weekday definition for each task. This makes it easy to > > check whether a task has to be launched. I still haven't found an > > algorithm (and I think there isn't an easy one available) to determine > > the sleep time until the next task is due. > > Ah, didn't think that far. > > > Thus, a check every minute > > isn't much but still 1440 times a day. > > But a lot less then the 864000 times a day in case of a check every 100 > ms :-) Sure. > > > > > The other thing to respect if you'd like to go as efficent as possible > > is the determination of Threads you'll need to run each cron task at the > > time specified. But this needs knowledge about how much elapsed time a > > task need to run to decide if you need one or two thread to complete to > > task defined neat in time to each other. If you have an algorithm for > > this you'll be able to define the maximum numbers of Thread you'll need. > > Otherwise you allocate a separate Thread per defined task. > > > > The consequences are that probably the easiest solution to a (more or > > less) complete Cron like Scheduler is an own heart beat of one minute to > > check for task to launch and a pool of Threads as big as you have task > > to process. As it seems Doug Leas concurrency tools could be configured > > to be as resource sparing as possible (have a look at > > http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/PooledExecutor.html) > > Yep, I think that will be fine too. Ok, so I'll gona build up a Scheduler based on its own heart beat (own Thread) with Crond expressions for timing specifications that uses a PooledExecutor for the Triggers in the size of the numbers of Task defined in the configuration but with lazy construction of Thread objects. This will not need any additional jars and the ones introduced by the Cornerstone Sccheduler can be eliminated again. Is everybody happy with that? -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Sun, 2003-08-31 at 20:38, Giacomo Pati wrote: > On Sun, 31 Aug 2003, Bruno Dumon wrote: > > > On Sat, 2003-08-30 at 17:24, Bruno Dumon wrote: > > > On Sat, 2003-08-30 at 15:20, Giacomo Pati wrote: > > > > > > I like the way how the Cornerstone Scheduler recently integrated by > > > > Carsten more than the CommandManager way because of its > > > > componentisation. Still, I like to see a Scheduler as a single component > > > > definition in the xconf file. Today we need at least three component > > > > definitions (Scheduler, ThreadManager, TimeScheduler) as well as one for > > > > each Target. I think this can be made much easier, like: > > > > > > > > > > > > > > > >> > >class="my.comp.MyTarget" > > > >logger="mytarget"> > > > > > > > >-1 > > > >SUN > > > >* > > > >9-17/2 > > > >*/5 > > > > > > > > ... > > > > > > > > > > > > > > > > In the above, triggers are handled as Avalon components and the > > > > scheduler takes care of instatiation and the lifecycle stuff. > > > > > > > > Unfortunately, the Cornerstone Scheduler toolkit doensn't offer a full > > > > crontab like specification of defining trigger point in time like I've > > > > used above (and I'd be happy to have). > > > > > > > > My suggestion (which I can implement if accepted) is to have a > > > > Scheduler that is able to do the above by use of the CommandManager > > > > given someone can fix Doug Leas concurrent utilities with the patch > > > > from Bruno. > > > > > > great. > > > > Or maybe not so great. I'm not sure the CommandManager is well suited > > for a general purpose scheduler. > > > > What follows now should be read with a 'AFAIU' disclaimer: > > > > All commands added to the CommandManager are executed sequentially and > > on one thread. So if there is one command that takes somewhat longer to > > execute, it will block the execution of other commands. This may be fine > > for management-tasks which should be executed repeatedly (but not at > > fixed times), but for a scheduler I would expect a command to be > > executed at the configured time. > > This is a matter of configuring a ThreadManager (which admittedly > today the one created in the Cocoon class isn't configurable at all). > > > This could be solved by creating a different CommandManager for each > > scheduled task, since multiple CommandManagers can be executed in > > If you have a ThreadManager with as many threads as needed you're set. > You only need ONE CommandManager, AFAIU. Looking at the code again, I still think that all events from one CommandManager are handled by one thread. If you look at the code of the AbstractThreadManager: In the method "register( EventPipeline pipeline )" one PipelineRunner is created to handle events from the given EventPipeline (e.g. the one of the CommandManager). Then in the run() method the PipelineRunners are executed in parallel by using the PooledExecutor: m_executor.execute( nextRunner ); But in the PipelineRunner.run() method, and the CommandManager.handleEvents() which is called by it, no new threads are started. > Looking at the ThreadManager > code I quite don't understand what it is doing as it could use the > Executor from Doug Leas package more efficently. > > > parallel. But then you'd still want to use a different ThreadManager > > than the TPCThreadManager, since that will block when all threads are in > > use. > > Or at least a configurable one. > > > The result will however be rather overkill IMHO: even commands that > > need to be executed only weekly, will be checked every 100 ms (= > > sleeptime parameter) to see if they need to be executed. > > Well, IMHO 100ms is far too fast even for the ContinuationManager. For > the Scheduler a granularity of one minute is enough. > > > Conclusion: while this event-based infrastructure is quite cool, I'm not > > convinced yet it is good for scheduling-purposes. > > First, if you look at the Cornerstone Scheduler you'll see that the > ability to define repeated tasks are somehow limited. Business apps can > easily request administration tasks to be run twice a day on work days. > To define this with the Cornerstone Scheduler you'll need to add 10 > 'cron' entries to it. This is why I'd like to extend it using the unix > cron direction to define ranges and list of timing aspects. But then > you're faced with the problem of defining sleep time for efficient use > (which you need to be able to cooperate in the Cornerstone Scheduler > machinery). > > The unix cron daemon runs every minute to check if there are any jobs to > run. It checks initially creating bitsets for the minute, hour, day, > month, and weekday definition for each task. This makes it easy to > check whether a task has to be launched. I still haven't found an > algorithm (and I think there isn't an easy one available) to determine > the sleep time
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Sun, 31 Aug 2003, Bruno Dumon wrote: > On Sat, 2003-08-30 at 17:24, Bruno Dumon wrote: > > On Sat, 2003-08-30 at 15:20, Giacomo Pati wrote: > > > > I like the way how the Cornerstone Scheduler recently integrated by > > > Carsten more than the CommandManager way because of its > > > componentisation. Still, I like to see a Scheduler as a single component > > > definition in the xconf file. Today we need at least three component > > > definitions (Scheduler, ThreadManager, TimeScheduler) as well as one for > > > each Target. I think this can be made much easier, like: > > > > > > > > > > > >> >class="my.comp.MyTarget" > > >logger="mytarget"> > > > > > >-1 > > >SUN > > >* > > >9-17/2 > > >*/5 > > > > > > ... > > > > > > > > > > > > In the above, triggers are handled as Avalon components and the > > > scheduler takes care of instatiation and the lifecycle stuff. > > > > > > Unfortunately, the Cornerstone Scheduler toolkit doensn't offer a full > > > crontab like specification of defining trigger point in time like I've > > > used above (and I'd be happy to have). > > > > > > My suggestion (which I can implement if accepted) is to have a > > > Scheduler that is able to do the above by use of the CommandManager > > > given someone can fix Doug Leas concurrent utilities with the patch > > > from Bruno. > > > > great. > > Or maybe not so great. I'm not sure the CommandManager is well suited > for a general purpose scheduler. > > What follows now should be read with a 'AFAIU' disclaimer: > > All commands added to the CommandManager are executed sequentially and > on one thread. So if there is one command that takes somewhat longer to > execute, it will block the execution of other commands. This may be fine > for management-tasks which should be executed repeatedly (but not at > fixed times), but for a scheduler I would expect a command to be > executed at the configured time. This is a matter of configuring a ThreadManager (which admittedly today the one created in the Cocoon class isn't configurable at all). > This could be solved by creating a different CommandManager for each > scheduled task, since multiple CommandManagers can be executed in If you have a ThreadManager with as many threads as needed you're set. You only need ONE CommandManager, AFAIU. Looking at the ThreadManager code I quite don't understand what it is doing as it could use the Executor from Doug Leas package more efficently. > parallel. But then you'd still want to use a different ThreadManager > than the TPCThreadManager, since that will block when all threads are in > use. Or at least a configurable one. > The result will however be rather overkill IMHO: even commands that > need to be executed only weekly, will be checked every 100 ms (= > sleeptime parameter) to see if they need to be executed. Well, IMHO 100ms is far too fast even for the ContinuationManager. For the Scheduler a granularity of one minute is enough. > Conclusion: while this event-based infrastructure is quite cool, I'm not > convinced yet it is good for scheduling-purposes. First, if you look at the Cornerstone Scheduler you'll see that the ability to define repeated tasks are somehow limited. Business apps can easily request administration tasks to be run twice a day on work days. To define this with the Cornerstone Scheduler you'll need to add 10 'cron' entries to it. This is why I'd like to extend it using the unix cron direction to define ranges and list of timing aspects. But then you're faced with the problem of defining sleep time for efficient use (which you need to be able to cooperate in the Cornerstone Scheduler machinery). The unix cron daemon runs every minute to check if there are any jobs to run. It checks initially creating bitsets for the minute, hour, day, month, and weekday definition for each task. This makes it easy to check whether a task has to be launched. I still haven't found an algorithm (and I think there isn't an easy one available) to determine the sleep time until the next task is due. Thus, a check every minute isn't much but still 1440 times a day. The other thing to respect if you'd like to go as efficent as possible is the determination of Threads you'll need to run each cron task at the time specified. But this needs knowledge about how much elapsed time a task need to run to decide if you need one or two thread to complete to task defined neat in time to each other. If you have an algorithm for this you'll be able to define the maximum numbers of Thread you'll need. Otherwise you allocate a separate Thread per defined task. The consequences are that probably the easiest solution to a (more or less) complete Cron like Scheduler is an own heart beat of one minute to check for task to launch and a pool of Threads as big as you have task to process. As it seems Doug Leas concurr
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Sat, 2003-08-30 at 17:24, Bruno Dumon wrote: > On Sat, 2003-08-30 at 15:20, Giacomo Pati wrote: > > I like the way how the Cornerstone Scheduler recently integrated by > > Carsten more than the CommandManager way because of its > > componentisation. Still, I like to see a Scheduler as a single component > > definition in the xconf file. Today we need at least three component > > definitions (Scheduler, ThreadManager, TimeScheduler) as well as one for > > each Target. I think this can be made much easier, like: > > > > > > > >>class="my.comp.MyTarget" > >logger="mytarget"> > > > >-1 > >SUN > >* > >9-17/2 > >*/5 > > > > ... > > > > > > > > In the above, triggers are handled as Avalon components and the > > scheduler takes care of instatiation and the lifecycle stuff. > > > > Unfortunately, the Cornerstone Scheduler toolkit doensn't offer a full > > crontab like specification of defining trigger point in time like I've > > used above (and I'd be happy to have). > > > > My suggestion (which I can implement if accepted) is to have a > > Scheduler that is able to do the above by use of the CommandManager > > given someone can fix Doug Leas concurrent utilities with the patch > > from Bruno. > > great. Or maybe not so great. I'm not sure the CommandManager is well suited for a general purpose scheduler. What follows now should be read with a 'AFAIU' disclaimer: All commands added to the CommandManager are executed sequentially and on one thread. So if there is one command that takes somewhat longer to execute, it will block the execution of other commands. This may be fine for management-tasks which should be executed repeatedly (but not at fixed times), but for a scheduler I would expect a command to be executed at the configured time. This could be solved by creating a different CommandManager for each scheduled task, since multiple CommandManagers can be executed in parallel. But then you'd still want to use a different ThreadManager than the TPCThreadManager, since that will block when all threads are in use. The result will however be rather overkill IMHO: even commands that need to be executed only weekly, will be checked every 100 ms (= sleeptime parameter) to see if they need to be executed. Conclusion: while this event-based infrastructure is quite cool, I'm not convinced yet it is good for scheduling-purposes. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Sat, 2003-08-30 at 15:20, Giacomo Pati wrote: > Is someone on the Doug Lea list (if there is any) to report this? I got an answer from him, he provided me with a slightly edited change: protected synchronized void workerDone(Worker w) { threads_.remove(w); if (--poolSize_ == 0 && shutdown_) { maximumPoolSize_ = minimumPoolSize_ = 0; // disable new threads notifyAll(); // notify awaitTerminationAfterShutdown } if (!shutdown_) { // create a replacement now if needed if (poolSize_ < minimumPoolSize_) { try { Runnable r = (Runnable)(handOff_.poll(0)); if (r != null) addThread(r); } catch(InterruptedException ie) { return; } } } } I've patched the concurrent 1.3.2 package with this change and put it in CVS. This change will be part of the next maintenance release of concurrent (he had this problem fixed already in the JSR-166 version of that class). -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: CommandManager issues [was Re: Releasing 2.1.1?]
On Sat, 2003-08-30 at 15:20, Giacomo Pati wrote: > On Fri, 29 Aug 2003, Bruno Dumon wrote: > > > On Fri, 2003-08-29 at 19:55, Giacomo Pati wrote: > > > > if you add the following code before the closing bracket of the > > PooledExecutor.workerDone(...) method, then it --seems to-- work: > > > > if (!shutdown_) { > > if (poolSize_ < maximumPoolSize_) { > > Runnable r= null; > > try { > > r = getTask(); > > } catch (InterruptedException e) { > > e.printStackTrace(); > > } > > if (r != null) > > addThread(r); > > } > > } > > > > Could someone else also try this out? For your convenience, here's a > > compiled jar with this change: > > I've patched the 1.3.2 Doug Lea utils myself with your suggestion above > and can affirm it works now. ok, thanks (Jeff/Antonio too). > > Cocoon compiled and tested with sun jdk 1.4.2 as well as sun jdk 1.3.1 > on Linux. To stop the Jetty running under 1.3.1 I needed two [Ctrl][C] > hits whereas the first hit reports "Shutdown hook executing" and after > the second one Jetty finally shuts down. If've made the tests with > Tomcat 4.1.27 as well where it behaves correctly as expected. > > Now, there is still the double Ctrl-C on jdk 1.3. Can you confirm this, > Bruno (or someone else with a Win$ machine)? On Linux/JDK1.3.1 I have this problem too. It has nothing to do with the CommandManager though (I'm really sure since I disabled the code). Maybe a Java or Jetty problem. > > Is someone on the Doug Lea list (if there is any) to report this? I've just sent him a message, I'll keep the list up-to-date on any news from him of course. > > I like the way how the Cornerstone Scheduler recently integrated by > Carsten more than the CommandManager way because of its > componentisation. Still, I like to see a Scheduler as a single component > definition in the xconf file. Today we need at least three component > definitions (Scheduler, ThreadManager, TimeScheduler) as well as one for > each Target. I think this can be made much easier, like: > > > > class="my.comp.MyTarget" >logger="mytarget"> > >-1 >SUN >* >9-17/2 >*/5 > > ... > > > > In the above, triggers are handled as Avalon components and the > scheduler takes care of instatiation and the lifecycle stuff. > > Unfortunately, the Cornerstone Scheduler toolkit doensn't offer a full > crontab like specification of defining trigger point in time like I've > used above (and I'd be happy to have). > > My suggestion (which I can implement if accepted) is to have a > Scheduler that is able to do the above by use of the CommandManager > given someone can fix Doug Leas concurrent utilities with the patch > from Bruno. great. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
CommandManager issues [was Re: Releasing 2.1.1?]
On Fri, 29 Aug 2003, Bruno Dumon wrote: > On Fri, 2003-08-29 at 19:55, Giacomo Pati wrote: > > if you add the following code before the closing bracket of the > PooledExecutor.workerDone(...) method, then it --seems to-- work: > > if (!shutdown_) { > if (poolSize_ < maximumPoolSize_) { > Runnable r= null; > try { > r = getTask(); > } catch (InterruptedException e) { > e.printStackTrace(); > } > if (r != null) > addThread(r); > } > } > > Could someone else also try this out? For your convenience, here's a > compiled jar with this change: I've patched the 1.3.2 Doug Lea utils myself with your suggestion above and can affirm it works now. Cocoon compiled and tested with sun jdk 1.4.2 as well as sun jdk 1.3.1 on Linux. To stop the Jetty running under 1.3.1 I needed two [Ctrl][C] hits whereas the first hit reports "Shutdown hook executing" and after the second one Jetty finally shuts down. If've made the tests with Tomcat 4.1.27 as well where it behaves correctly as expected. Now, there is still the double Ctrl-C on jdk 1.3. Can you confirm this, Bruno (or someone else with a Win$ machine)? Is someone on the Doug Lea list (if there is any) to report this? I like the way how the Cornerstone Scheduler recently integrated by Carsten more than the CommandManager way because of its componentisation. Still, I like to see a Scheduler as a single component definition in the xconf file. Today we need at least three component definitions (Scheduler, ThreadManager, TimeScheduler) as well as one for each Target. I think this can be made much easier, like: -1 SUN * 9-17/2 */5 ... In the above, triggers are handled as Avalon components and the scheduler takes care of instatiation and the lifecycle stuff. Unfortunately, the Cornerstone Scheduler toolkit doensn't offer a full crontab like specification of defining trigger point in time like I've used above (and I'd be happy to have). My suggestion (which I can implement if accepted) is to have a Scheduler that is able to do the above by use of the CommandManager given someone can fix Doug Leas concurrent utilities with the patch from Bruno. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: Shutdown bug (was: Re: Releasing 2.1.1?)
Jeff Turner dijo: > On Fri, Aug 29, 2003 at 10:18:40PM +0200, Bruno Dumon wrote: > ... > [on bug http://nagoya.apache.org/bugzilla/show_bug.cgi?id=18131] > ... >> ok, 10 minutes later now, got an idea: >> >> if you add the following code before the closing bracket of the >> PooledExecutor.workerDone(...) method, then it --seems to-- work: >> >> if (!shutdown_) { >> if (poolSize_ < maximumPoolSize_) { >> Runnable r= null; >> try { >> r = getTask(); >> } catch (InterruptedException e) { >> e.printStackTrace(); >> } >> if (r != null) >> addThread(r); >> } >> } >> >> Could someone else also try this out? For your convenience, here's a >> compiled jar with this change: >> http://outerthought.net/~bruno/concurrent.jar >> >> just copy it over the util.concurrent-1.3.1.jar > > Great :) Tomcat shutdown works fine on my system with this jar. Thanks > muchly. Brilliant Bruno! Where you will commit this long awaited feature and close the bug. ;) Best Regards, Antonio Gallardo > > > --Jeff
Shutdown bug (was: Re: Releasing 2.1.1?)
On Fri, Aug 29, 2003 at 10:18:40PM +0200, Bruno Dumon wrote: ... [on bug http://nagoya.apache.org/bugzilla/show_bug.cgi?id=18131] ... > ok, 10 minutes later now, got an idea: > > if you add the following code before the closing bracket of the > PooledExecutor.workerDone(...) method, then it --seems to-- work: > > if (!shutdown_) { > if (poolSize_ < maximumPoolSize_) { > Runnable r= null; > try { > r = getTask(); > } catch (InterruptedException e) { > e.printStackTrace(); > } > if (r != null) > addThread(r); > } > } > > Could someone else also try this out? For your convenience, here's a > compiled jar with this change: > http://outerthought.net/~bruno/concurrent.jar > > just copy it over the util.concurrent-1.3.1.jar Great :) Tomcat shutdown works fine on my system with this jar. Thanks muchly. --Jeff
Re: Releasing 2.1.1?
On Fri, 2003-08-29 at 19:55, Giacomo Pati wrote: > On Fri, 29 Aug 2003, Bruno Dumon wrote: > > > For me it works with Linux/Sun jdk 1.4.2, it doesn't work with 1.3.1. > > For Carsten it doesn't work with Windows/jdk 1.4 either. I've done all > > my testing with the Jetty which ships with Cocoon. > > > > I've added some println's here and there and it appears that, with > > 1.3.1, the CommandManager starts working but stops as soon as Jetty > > prints out the following lines: > > Fri Aug 29 13:58:07 CEST 2003 Listening for connections ... > > 13:58:07.575 EVENT Started SocketListener on 0.0.0.0: > > I have the above also in with sun jdk 1.4.2 on Linux. It's probably a coincidence. If you supply a higher value for the "sleep-time" parameter of the CommandManager, for example 1000 ms, it will probably keep working for you too (with 1.3 also). I think the problem lies somewhere in the PooledExecutor and/or SynchronousChannel classes from the concurrent package. If you take a look at the PooledExecutor class, there's a getTask method that is called by worker threads (see the inner class Worker). Shortly before jetty binds to port it probably does something cpu intensive, causing the poll(sleep-time) call in the getTask method to timeout, and the getTask() method to return null, which in itself causes the worker thread to be released. More or less at the same time, the PooledExecutor is asked to execute another runnable but it thinks no threads are available anymore and "hands off" the runnable to the SynchronousChannel. How it then needs to get out of that situation is currently still a mystery for me... ok, 10 minutes later now, got an idea: if you add the following code before the closing bracket of the PooledExecutor.workerDone(...) method, then it --seems to-- work: if (!shutdown_) { if (poolSize_ < maximumPoolSize_) { Runnable r= null; try { r = getTask(); } catch (InterruptedException e) { e.printStackTrace(); } if (r != null) addThread(r); } } Could someone else also try this out? For your convenience, here's a compiled jar with this change: http://outerthought.net/~bruno/concurrent.jar just copy it over the util.concurrent-1.3.1.jar > > > It seems that the thread that was executing the commands is somehow not > > released (or whathever), so when the ThreadManager wants to execute the > > next command it keeps waiting for a thread to be released (this waiting > > happens in Doug Lea's Executor). If the "threads-per-processor" > > parameter is augmented to "2", then it keeps working. > > Have not tried using a second Thread. > > > > > According to Giacomo, the CommandManager works just fine in a > > non-servlet environment. > > Right. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Releasing 2.1.1?
On Fri, 29 Aug 2003, Bruno Dumon wrote: > For me it works with Linux/Sun jdk 1.4.2, it doesn't work with 1.3.1. > For Carsten it doesn't work with Windows/jdk 1.4 either. I've done all > my testing with the Jetty which ships with Cocoon. > > I've added some println's here and there and it appears that, with > 1.3.1, the CommandManager starts working but stops as soon as Jetty > prints out the following lines: > Fri Aug 29 13:58:07 CEST 2003 Listening for connections ... > 13:58:07.575 EVENT Started SocketListener on 0.0.0.0: I have the above also in with sun jdk 1.4.2 on Linux. > It seems that the thread that was executing the commands is somehow not > released (or whathever), so when the ThreadManager wants to execute the > next command it keeps waiting for a thread to be released (this waiting > happens in Doug Lea's Executor). If the "threads-per-processor" > parameter is augmented to "2", then it keeps working. Have not tried using a second Thread. > > According to Giacomo, the CommandManager works just fine in a > non-servlet environment. Right. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: Releasing 2.1.1?
On Fri, 29 Aug 2003, Berin Loritsch wrote: > Bruno Dumon wrote: > > On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > > > >>Hi, > >> > >>as several of you have already pointed out, a 2.1.1 release seems > >>to make sense. > >> > >>I can make a release next week, but I really would like to have the > >>scheduling (CommandManager) problem fixed in the release. Is anyone > >>working already on it? > > > > > > Not over here. If all else fails, we could disable the CommandManager > > and let the ContinuationsManager start a thread of its own (like e.g. > > the StoreJanitor does). > > > > What version of CommandManager are you using? I am very interested > in fixing it so that it works all the time. Also, under what circumstances > does it break? I'm using the latest (from CVS) put alot of gebugging into it to see wher it goes. Unfortunately at some time in the startup of the servlet container it stops working (I've put a test command right after constructing the CommandManager in Cocoon.java). -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: Releasing 2.1.1?
On Fri, 29 Aug 2003, Bruno Dumon wrote: > On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > > Hi, > > > > as several of you have already pointed out, a 2.1.1 release seems > > to make sense. > > > > I can make a release next week, but I really would like to have the > > scheduling (CommandManager) problem fixed in the release. Is anyone > > working already on it? > > Not over here. If all else fails, we could disable the CommandManager > and let the ContinuationsManager start a thread of its own (like e.g. > the StoreJanitor does). I'm still debugging that issues. So far I'm at the point where the Queue stops working at the point (more or less) at the time jetty opens the listening ports (). Stay tuned. -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com
Re: Releasing 2.1.1?
On Fri, 2003-08-29 at 15:35, Berin Loritsch wrote: > Bruno Dumon wrote: > > On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > > > >>Hi, > >> > >>as several of you have already pointed out, a 2.1.1 release seems > >>to make sense. > >> > >>I can make a release next week, but I really would like to have the > >>scheduling (CommandManager) problem fixed in the release. Is anyone > >>working already on it? > > > > > > Not over here. If all else fails, we could disable the CommandManager > > and let the ContinuationsManager start a thread of its own (like e.g. > > the StoreJanitor does). > > > > What version of CommandManager are you using? The one shipping with Cocoon. > I am very interested > in fixing it so that it works all the time. Also, under what circumstances > does it break? Don't know exactly. Here's what I've found out till now: For me it works with Linux/Sun jdk 1.4.2, it doesn't work with 1.3.1. For Carsten it doesn't work with Windows/jdk 1.4 either. I've done all my testing with the Jetty which ships with Cocoon. I've added some println's here and there and it appears that, with 1.3.1, the CommandManager starts working but stops as soon as Jetty prints out the following lines: Fri Aug 29 13:58:07 CEST 2003 Listening for connections ... 13:58:07.575 EVENT Started SocketListener on 0.0.0.0: It seems that the thread that was executing the commands is somehow not released (or whathever), so when the ThreadManager wants to execute the next command it keeps waiting for a thread to be released (this waiting happens in Doug Lea's Executor). If the "threads-per-processor" parameter is augmented to "2", then it keeps working. According to Giacomo, the CommandManager works just fine in a non-servlet environment. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: Releasing 2.1.1?
Have a look at this thread: http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=106189194527050&w=2 and this bug http://nagoya.apache.org/bugzilla/show_bug.cgi?id=18131 Now, it seems that the CommandManager is not working in all environments. In my it's not working but I don't have the shutdown problem and I guess it's vice versa. We are using the released version 1.0.3, but I tested the latest CVS and I guess Giacomo is also using the latest. Carsten Berin Loritsch [mailto:[EMAIL PROTECTED] wrote: > > What version of CommandManager are you using? I am very interested > in fixing it so that it works all the time. Also, under what > circumstances > does it break? > >
Re: Releasing 2.1.1?
Bruno Dumon wrote: On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: Hi, as several of you have already pointed out, a 2.1.1 release seems to make sense. I can make a release next week, but I really would like to have the scheduling (CommandManager) problem fixed in the release. Is anyone working already on it? Not over here. If all else fails, we could disable the CommandManager and let the ContinuationsManager start a thread of its own (like e.g. the StoreJanitor does). What version of CommandManager are you using? I am very interested in fixing it so that it works all the time. Also, under what circumstances does it break? -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin
RE: Releasing 2.1.1?
Bruno Dumon wrote: > > On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > > Hi, > > > > as several of you have already pointed out, a 2.1.1 release seems > > to make sense. > > > > I can make a release next week, but I really would like to have the > > scheduling (CommandManager) problem fixed in the release. Is anyone > > working already on it? > > Not over here. If all else fails, we could disable the CommandManager > and let the ContinuationsManager start a thread of its own (like e.g. > the StoreJanitor does). > We could simply switch back to the cornerstone scheduler which works. Carsten
RE: Releasing 2.1.1?
On Fri, 2003-08-29 at 14:20, Carsten Ziegeler wrote: > Bruno Dumon wrote: > > > > On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > > > Hi, > > > > > > as several of you have already pointed out, a 2.1.1 release seems > > > to make sense. > > > > > > I can make a release next week, but I really would like to have the > > > scheduling (CommandManager) problem fixed in the release. Is anyone > > > working already on it? > > > > Not over here. If all else fails, we could disable the CommandManager > > and let the ContinuationsManager start a thread of its own (like e.g. > > the StoreJanitor does). > > > We could simply switch back to the cornerstone scheduler which works. Fine with me too. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: Releasing 2.1.1?
Bruno Dumon wrote: > > On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > > Hi, > > > > as several of you have already pointed out, a 2.1.1 release seems > > to make sense. > > > > I can make a release next week, but I really would like to have the > > scheduling (CommandManager) problem fixed in the release. Is anyone > > working already on it? > > Not over here. If all else fails, we could disable the CommandManager > and let the ContinuationsManager start a thread of its own (like e.g. > the StoreJanitor does). > We could simply switch back to the cornerstone scheduler which works. Carsten
Re: Releasing 2.1.1?
On Thu, 2003-08-28 at 08:12, Carsten Ziegeler wrote: > Hi, > > as several of you have already pointed out, a 2.1.1 release seems > to make sense. > > I can make a release next week, but I really would like to have the > scheduling (CommandManager) problem fixed in the release. Is anyone > working already on it? Not over here. If all else fails, we could disable the CommandManager and let the ContinuationsManager start a thread of its own (like e.g. the StoreJanitor does). -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Releasing 2.1.1?
On Thu, 28 Aug 2003, Carsten Ziegeler wrote: > Hi, > > as several of you have already pointed out, a 2.1.1 release seems > to make sense. > > I can make a release next week, but I really would like to have the > scheduling (CommandManager) problem fixed in the release. Is anyone > working already on it? I've just made a standalone main app with the CommandManager and, confusing enought, it works correclty as expected. So, it must be the environment Cocoon runs in that causes troubles (Servlet?) -- Giacomo Pati Otego AG, Switzerland - http://www.otego.com Orixo, the XML business alliance - http://www.orixo.com