Re: [VOTE] Release on monday
Ugo Cei wrote: Upayavira wrote: I need to remove the test-suite and use samples/test, and confirm that Ugo's fixes have made the CocoonBeanTestCase work, and then re-enable it. A word of caution. My fixes add the blocks directory and block-provided jars to the classpath for tests and make the junit-tests target depend upon the blocks target. This is necessary because the CocoonBeanTestCase loads build/webapp/WEB-INF/cocoon.xconf, which contains references to components provided by blocks. This strikes me as a typical anti-pattern. What are we testing here? The CocoonBean or the components that it relies upon? If it's the latter, fine, but it's not a unit test anymore, it's an integration test and does not belong under the junit-tests target. If it's the former, it should be testable in isolation. In any case, it would be probably advisable to load the CocoonBean under test with a cocoon.xconf derived from a no-blocks-included configuration. This is all fair comment. Whilst I have a 'unit test' for the bean, in fact, the bean really depends upon the entirety of Cocoon, and is thus really more of a functional test. Given that some blocks have been known to break the bean, it is important that the test is run across the entire Cocoon. It is there somewhat more akin to an anteater test. Therefore, given these facts, I propose to leave the CocoonBeanTestCase disabled, and simply remove the test suite (as it isn't needed anyway.). I will continue to use the test case locally on my own testing, and will reflect on a better place for it (or some equivalent) within the build process, perhaps alongside the anteater tests. Upayavira
Re: [VOTE] Release on monday
Upayavira wrote: It is there somewhat more akin to an anteater test. Therefore, given these facts, I propose to leave the CocoonBeanTestCase disabled, and simply remove the test suite (as it isn't needed anyway.). I will continue to use the test case locally on my own testing, and will reflect on a better place for it (or some equivalent) within the build process, perhaps alongside the anteater tests. +1. I am also worried about the time it takes to run the CocoonBeanTestCase. We already run the unit tests all too seldom, and if they take a long time, nobody will ever want to run them more often. Ugo
RE: [VOTE] Release on monday
Antonio Gallardo wrote: I hope we soon will get rid of jisp. What about to deprecate current JISP support? If someone want to use JISP there is still a posibility using JCS. +1 Carsten
Re: [VOTE] Release on monday
Carsten Ziegeler wrote: Or is there anything serious that I did oversee? Nothing, apart a few failing anteater tests. I think those are due to faulty testcases more than to faulty code. Or maybe it's just wishful thinking. Anyway, +1 for a release on Monday. Hopefully I'll find some time to look after them in the next few days. Ugo
Re: [VOTE] Release on monday
Le 19 mai 04, à 10:49, Carsten Ziegeler a écrit : ...If noone objects I will release the current state on monday, 24th of May +1, but won't be able to help (once again), leaving for a holiday tomorrow till Sunday. -Bertrand
Re: [VOTE] Release on monday
On 19.05.2004 10:49, Carsten Ziegeler wrote: I think we fixed the most serious problems for 2.1.5 now. Can you sum up the status of the store issues? There were hundreds of mail on it especially on Monday. Before this week we had a non-working persistent store, now we have no longer a persistent store, but a two-stage caching system with both JCS and EHCache? What about the Serializable issues? Joerg
RE: [VOTE] Release on monday
Joerg Heinicke wrote: On 19.05.2004 10:49, Carsten Ziegeler wrote: I think we fixed the most serious problems for 2.1.5 now. Can you sum up the status of the store issues? No, I lost track of it... There were hundreds of mail on it especially on Monday. Before this week we had a non-working persistent store, now we have no longer a persistent store, but a two-stage caching system with both JCS and EHCache? What about the Serializable issues? Ok, let me try it: - Between 2.1.4 and 2.1.5 we had the refactoring of the stores. The basic idea is to have only two stores that should be used by clients: the store and the transient store. The transient store is - well - transient - it does not persist data. The store should also persist data. It should implement a two phase store (memory - disk). - The old (jisp) based implementation of the store used the persistent store to persist the data. - Due to the refactoring we had severe problems (don't know which) that prevented us from releasing - this came up on friday. (it was mentioned here and there earlier) - Now: we are using JCS for the store. JCS itself has a two-phase caching implemented and therefore we don't need a persistent store anymore. - We still use the memory store for the transient store. - If you put objects into the store, they have to be serializable now. Even if JCS holds the data just in memory, it assumes that the objects are serializable. Perhaps they will change this perhaps not. Although this is different from the jisp based version it doesn't affect us as we are using serializable objects anyway - which makes sense as they whole data should be persistent later on. - If you put objects into the transient store they don't have to be serializable. - Persisting cache entries inbetween application starts didn't work very well with Jisp as we had bugs in Cocoon. This works now with JCS. - There is only one bug: JCS is not always writing the data to disk. It seems that there has to be a critical mass of data before it gets written. - we reported this issue and I hope they will fix it. But this is not a serious issue as the persistence is not that important and didn't work before anyway. So, to sum it up: imho we have no a working system that is based on JCS. There are minor problems that we can fix in the next releases. I hope I didn't forget anything. Carsten
Re: [VOTE] Release on monday
Carsten Ziegeler wrote: I think we fixed the most serious problems for 2.1.5 now. If noone objects I will release the current state on monday, 24th of May. So the code freeze will continue until then. This gives us some more days to test and possibly fix bugs. Or is there anything serious that I did oversee? I need to remove the test-suite and use samples/test, and confirm that Ugo's fixes have made the CocoonBeanTestCase work, and then re-enable it. But, as that test is currently disabled, releasing with it in its current state is not a major problem. I'd just like to get it done before we release. Upayavira
RE: [VOTE] Release on monday
Upayavira wrote: I need to remove the test-suite and use samples/test, and confirm that Ugo's fixes have made the CocoonBeanTestCase work, and then re-enable it. But, as that test is currently disabled, releasing with it in its current state is not a major problem. I'd just like to get it done before we release. Sure, makes sense. You have some days time until monday :) Carsten
Re: [VOTE] Release on monday
On 19.05.2004 13:02, Carsten Ziegeler wrote: Before this week we had a non-working persistent store, now we have no longer a persistent store, but a two-stage caching system with both JCS and EHCache? What about the Serializable issues? Ok, let me try it: - Between 2.1.4 and 2.1.5 we had the refactoring of the stores. The basic idea is to have only two stores that should be used by clients: the store and the transient store. The transient store is - well - transient - it does not persist data. The store should also persist data. It should implement a two phase store (memory - disk). - The old (jisp) based implementation of the store used the persistent store to persist the data. - Due to the refactoring we had severe problems (don't know which) that prevented us from releasing - this came up on friday. (it was mentioned here and there earlier) That was Sylvain's refactoring IIRC. - Now: we are using JCS for the store. JCS itself has a two-phase caching implemented and therefore we don't need a persistent store anymore. But the second phase is not intended as persistent store, is it? - We still use the memory store for the transient store. How do JCS phase I and MRUMemoryStore play together here - or do I mix two different things? - If you put objects into the store, they have to be serializable now. Even if JCS holds the data just in memory, it assumes that the objects are serializable. Perhaps they will change this perhaps not. What about the non-Serializable objects? Logicsheets and Paginator stuff were mentioned. Are these objects not put into transient store? Although this is different from the jisp based version it doesn't affect us as we are using serializable objects anyway - which makes sense as they whole data should be persistent later on. - If you put objects into the transient store they don't have to be serializable. - Persisting cache entries inbetween application starts didn't work very well with Jisp as we had bugs in Cocoon. This works now with JCS. - There is only one bug: JCS is not always writing the data to disk. It seems that there has to be a critical mass of data before it gets written. - we reported this issue and I hope they will fix it. But this is not a serious issue as the persistence is not that important and didn't work before anyway. So, to sum it up: imho we have no a working system that is based on JCS. I get the feeling that JCS is not used for transient store (but MRUMemoryStore), so no JCS phase I. And JCS phase II (now replacement for old Jisp persistent store) has minor bugs as written above. There are minor problems that we can fix in the next releases. I don't want to block the release, you get my +1. I'm just curious and want to know how the stuff is working now - and if I can use the release in a project of course. Joerg
Re: [VOTE] Release on monday
Joerg Heinicke wrote:
On 19.05.2004 13:02, Carsten Ziegeler wrote:
Before this week we had a non-working persistent store, now we have
no longer a persistent store, but a two-stage caching system with
both JCS and EHCache? What about the Serializable issues?
Ok, let me try it:
- Between 2.1.4 and 2.1.5 we had the refactoring of the stores. The
basic
idea is to have only two stores that should be used by clients:
the store and the transient store. The transient store is - well -
transient - it does not persist data. The store should also persist
data. It should implement a two phase store (memory - disk).
- The old (jisp) based implementation of the store used the persistent
store to persist the data.
- Due to the refactoring we had severe problems (don't know which)
that prevented us from releasing - this came up on friday. (it was
mentioned here and there earlier)
That was Sylvain's refactoring IIRC.
Yep. And this refactoring made JISP bugs apparent since on-disk data was
never retrieved before (even if written)
- Now: we are using JCS for the store. JCS itself has a two-phase
caching implemented and therefore we don't need a persistent store
anymore.
But the second phase is not intended as persistent store, is it?
Since JCS does both memory front-end + disk back-end, it can be _the_
store, and we no more need the persistent store.
- We still use the memory store for the transient store.
How do JCS phase I and MRUMemoryStore play together here - or do I mix
two different things?
They live their lives on their own, orchestrated by the store janitor.
- If you put objects into the store, they have to be serializable now.
Even if JCS holds the data just in memory, it assumes that the
objects are serializable. Perhaps they will change this perhaps
not.
What about the non-Serializable objects? Logicsheets and Paginator
stuff were mentioned. Are these objects not put into transient store?
Yep. And the transient store does not (and should not) require objects
to be serializable (that's its main purpose compared to the regular
store that may persist objects).
Although this is different from the jisp based version it doesn't
affect us as we are using serializable objects anyway - which makes
sense as they whole data should be persistent later on.
- If you put objects into the transient store they don't have to
be serializable.
- Persisting cache entries inbetween application starts didn't work
very well with Jisp as we had bugs in Cocoon.
This works now with JCS.
- There is only one bug: JCS is not always writing the data to disk. It
seems that there has to be a critical mass of data before it gets
written. - we reported this issue and I hope they will fix it. But
this is not a serious issue as the persistence is not that important
and didn't work before anyway.
So, to sum it up: imho we have no a working system that is based on JCS.
I get the feeling that JCS is not used for transient store (but
MRUMemoryStore), so no JCS phase I. And JCS phase II (now replacement
for old Jisp persistent store) has minor bugs as written above.
IIUC the main store based on JCS uses the two phases (memory + disk),
and it seems we can live for now with the minor shutdown bug.
There are minor problems that we can fix in the next releases.
I don't want to block the release, you get my +1. I'm just curious and
want to know how the stuff is working now - and if I can use the
release in a project of course.
+1 from me also.
You guys did an amazing job to solve this issue. I lacked time to
participate, being very busy on a project. The good news however is that
this will lead to interesting new CForm features :-)
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: [VOTE] Release on monday
Upayavira wrote: I need to remove the test-suite and use samples/test, and confirm that Ugo's fixes have made the CocoonBeanTestCase work, and then re-enable it. A word of caution. My fixes add the blocks directory and block-provided jars to the classpath for tests and make the junit-tests target depend upon the blocks target. This is necessary because the CocoonBeanTestCase loads build/webapp/WEB-INF/cocoon.xconf, which contains references to components provided by blocks. This strikes me as a typical anti-pattern. What are we testing here? The CocoonBean or the components that it relies upon? If it's the latter, fine, but it's not a unit test anymore, it's an integration test and does not belong under the junit-tests target. If it's the former, it should be testable in isolation. In any case, it would be probably advisable to load the CocoonBean under test with a cocoon.xconf derived from a no-blocks-included configuration. WDYT? Ugo
RE: [VOTE] Release on monday
Vadim Gritsenko wrote: Let's clarify something here about all those stores: Transient store: MUST hold NON Serializable objects, MUST NOT persist objects. Persistent store: MUST reject NON Serializable objects, MUST persist objects. Store: MUST hold NON Serializable objects, CAN persist Serializable objects on overflow (or any other reason), MUST persist all Serializable objects on shutdown. For those who have not followed - this was the behavior of the stores in previous releases of Cocoon (before refactoring we had only two stores, IIRC, but same behavior). Given terminology above, we can have a working persistent store (JCS based), and working transient store (from Excalibur). General store is currently broken in two ways: * It does not stores non serializable objects, but should. * It does not persists cache on shutdown. I'm +1 on release, if and only if, we note these above bugs in the known issues list. Fair enough. Just add them. Thanks Carsten
Re: [VOTE] Release on monday
On 19.05.2004, at 14:52, Vadim Gritsenko wrote: I'm +1 on release, if and only if, we note these above bugs in the known issues list. Vadim Please give me that list!!! I updated a server hosting a bunch of XSPs to current CVS. Using the default cocoon.xconf settings, first everything is fine, but after about an hour under medium load it is getting slower and slower and is starving while heap is growing to ~240M (-Xmx260m). can anybody confirm that these settings do work? /leo transient-store logger=core.store.transient parameter name=maxobjects value=1000/ /transient-store store logger=core.store parameter name=maxobjects value=1000/ parameter name=use-cache-directory value=true/ /store store-janitor logger=core.store.janitor parameter name=freememory value=500/ parameter name=heapsize value=19600/ parameter name=cleanupthreadinterval value=10/ parameter name=adaptivethreadinterval value=true/ parameter name=threadpriority value=5/ parameter name=percent_to_free value=10/ parameter name=invokegc value=false/ /store-janitor
Re: [VOTE] Release on monday
On Wed, May 19, 2004 at 08:52:52AM -0400, Vadim Gritsenko wrote: snip/ For those who have not followed - this was the behavior of the stores in previous releases of Cocoon (before refactoring we had only two stores, IIRC, but same behavior). Given terminology above, we can have a working persistent store (JCS based), and working transient store (from Excalibur). General store is currently broken in two ways: * It does not stores non serializable objects, but should. * It does not persists cache on shutdown. I'm +1 on release, if and only if, we note these above bugs in the known issues list. Vadim +1 for release with the same condition as Vadim states above. --Tim Larson
Re: [VOTE] Release on monday
Carsten Ziegeler dijo: I think we fixed the most serious problems for 2.1.5 now. If noone objects I will release the current state on monday, 24th of May. So the code freeze will continue until then. This gives us some more days to test and possibly fix bugs. Or is there anything serious that I did oversee? +1 To release on Monday. And many thanks to you for doing the work! I hope we soon will get rid of jisp. What about to deprecate current JISP support? If someone want to use JISP there is still a posibility using JCS. Best Regards, Antonio Gallardo
