[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476624 ] Matej Knopp commented on WICKET-201: The tests mentioned in previous comment were executed on OSX with java 6 - server, 128MB heap. I executed same test, 40 sessions / 100 request on clean windows instalation, 128MB heap, java 6 -server SecondLevelCacheSessionStore: 67844 ms HttpSessionStore: 53531 ms Here the difference is 21% All tests are executed with WicketOutputStream. I think compared to Jonathan's first benchmark where 2nd level session store was almost 50% slower then httpsessionstore, these are very good numbers. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476638 ] Eelco Hillenius commented on WICKET-201: Cool. 10 request each is in fact 40 each as App1Test1 does 4 requests for one run, right? :) refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476108 ] Johan Compagner commented on WICKET-201: for 1.3 the SecondLevelCache and its FilePageStore is pretty much done. The performance lose we have for the serialization and saving (2 separate threads) are around 15% under very heavy load. (App1Test1) compared to HttpSessionStore. Thats in my eyes very acceptable. And under normal load you won't even notice it because the serializing thread can handle it fine. The only thing we could do now is move the soft cache we have now in the Store it self through the FilePageStore and soft cache the byte[] so that those don't have to be loaded from disk. But maybe we can do both. If a history entry in the browser is a new page in wicket then the current soft cache does it works. But if it is a version of the current page. the soft cache doesn't do anything anymore (because there is no undo information anymore in a page) Also maybe the FilePageStore should be abstracted then. So that the serialization is done by a common base class. And the actual saving can be done to anything you want. (and then port it all to 2.0!) refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476298 ] Eelco Hillenius commented on WICKET-201: What you implemented is slightly different though. The code I've seen (last week) stops accepting for a certain threshold per pagemap (or session?). What I'm talking about is to put the threshold on the number of sessions and/ or heap mem usage. And yes, I've tested with a couple of thousand sessions and was able to get OOME with this store. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476302 ] Johan Compagner commented on WICKET-201: i don't stop accepting anything the only thing that happens is that the request thread helps serializing if it wants to access a page that isnt serialized yet. what was your memory setting with you test it with a couple of thousand sessions? Did you test it also with the normal httpsessionstore with the same test?? and when did you test? Wit the latest code that i checked in today? Because one thread prio still wasn't correct. What i see now if i test it that the pages that are serialized are all saved. (see the output when jetty stops) you can there exactly see how many pages are serialized and how many where saved. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476371 ] Eelco Hillenius commented on WICKET-201: I didn't test with the code of today. However, the test is easy. Start up wicket.threadtest.tester.Tester directly (that's possible since a few days) so that the server keeps running in the back ground and you can execute the thread tests in a separate VM (works great on dual core btw). Run the tests a couple of times so that you'll have a bunch of sessions lingering around. My point is that every session still occupies some heap space, even if it isn't doing anything at the moment. What we should be looking for is a mechanism like I described above (so e.g. when we get over a certain number of active sessions, stop putting them in memory but blocking write them to disk right away to protect OOME). Additionally, a nice optimization would be to write out sessions after a bit, and removing any pages in memory there might be for that session. This should also be based on some threshold of sessions/ memory - no need to do this when there is plenty of memory - and if that would be combined with a time out (e.g. 30 seconds after the page was last requested) this would give the active sessions more priority and probably leads to a better load spread. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476383 ] Johan Compagner commented on WICKET-201: writing out (idle) sessions is something that should be done by the container. We shouldn't keep references to the session and alter the session when it is not in use in a request. (we don't know what the container does with the sessions (or terracotta)) So the only thing is that we can write pages directly to disk (don't sign them to the pagemap) for current request if the memory start to get low then the performance of that will go down of course, because those are most likely to return again (they are the active sessions). And when is memory low?? memory could for example be low at one time (max != current free X) but one GC can free up 30%. Thats a bit difficult to really see at a specific time.. That every session keeps lingering around is not a problem specific to the SecondLevelCache. I think that the SLC wil even improve how many live sessions we can handle compared to the normal http store. Because there we will have 7 page or page versions in the session. So the sessions that are still idle are potentially keeping 7 times more memory then with SLC Maybe if we know or calculate the average size of session. For the specific system. Then we can take that into account with the total number of sessions and then we know what those cost.. And maybe we can somehow use that to decide that we have to overflow. But i rather left that to the container (idle session to disk) refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476404 ] Eelco Hillenius commented on WICKET-201: I can see you point, and indeed in a clustered environment I guess we can't really know which sessions are an which server and actually consuming local heap. Re testing. Did you try testing with that new project I committed this weekend? That gives a pretty accurate size I think. I waiting for feedback on that anyway, so please test. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476405 ] Eelco Hillenius commented on WICKET-201: new project == wicket-objectsizeof-agent. See http://www.nabble.com/Objects.sizeOf-t3290492.html refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12470751 ] Johan Compagner commented on WICKET-201: fixed for 1.3 We don't keep any version info now in the page. only a versionnumber thats increased. the pages are just stored to disk. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Bug Reporter: Eelco Hillenius Assigned To: Johan Compagner Priority: Critical Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (WICKET-201) refactor storing pages and versions
[ https://issues.apache.org/jira/browse/WICKET-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12464764 ] Johan Compagner commented on WICKET-201: I thought about it and it will be pretty hard to really change the current behavior i only see one: Reduce the Page version of a page to max 1 (1extra then the current one) why? Store only changes This can't be done because those changes do have call backs to the page/parent anyway so this will be very, very hard to reliable implement. Store only pages when the fall out of the pagemap (not the current page anymore) This is also not possible. A page when created new or a version is created (is also 'new') then the page has to be saved to disc. There is no other way. Else we are to late the next time and we will loose versions of the page. (of course max page versions to 10+ will solve this but that is just what we don't want) So the only optimizations i can think if is: 1 If SecondLevelPageStore is used then the default max page versions is set to 1 (1 extra then the current) 2 save the page to disk when it is newly created or if there is a new version created in the current request. (this is really needed else we are to late the next time!) 3 make sure that the next version of a page is always ++ (and not decreased when a page version is roll backed) By the way, in wicket 2.0 there is code that can make versions of pages by only storing the changed real values (strings and numbers) I did develop that with discussions with igor how to not serialize the page but make a special snapshot of the complete page (hash mapping all the values) This code is now a bit untouched but maybe we could use it. The planning was to create a snapshot map when a request comes in for a page (just before anything happens to it) then compare it in the end and only store the really changed values. Then we have automatic page versioning.. But don't know if that could really work. refactor storing pages and versions --- Key: WICKET-201 URL: https://issues.apache.org/jira/browse/WICKET-201 Project: Wicket Issue Type: Improvement Reporter: Eelco Hillenius Assigned To: Johan Compagner Fix For: 1.3, 2.0 See http://www.nabble.com/refactor-storing-pages-and-versions-tf2943670.html for a discussion. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
