Re: AW: Cleanup NodeStore and MK implementations (was: Re: AW: Confusions about API's and Flavours in the documentation ...)
On 18.8.14 8:41 , KÖLL Claus wrote: The longer we wait now the harder it will be to create clean and easier to understand documentation I agree we should do the move, however doubt that right now is a good point in time. This will affect the ongoing work in theses areas, which is all merged back into the 1.0 branch. We should also consider Marcel and Alex, who will probably be most affected by the additional merge effort. Michael
Re: AW: Cleanup NodeStore and MK implementations (was: Re: AW: Confusions about API's and Flavours in the documentation ...)
On Mon, Aug 18, 2014 at 1:02 PM, Michael Dürig mdue...@apache.org wrote: This will affect the ongoing work in theses areas, which is all merged back into the 1.0 branch. We should also consider Marcel and Alex, who will probably be most affected by the additional merge effort. I also think that this refactoring should be delayed (if that is to be done) for some time till we get more stability in the two NodeStore implementations. Chetan Mehrotra
[Document Cache Size] Is it better to have cache size using number of entries
Hi, We were struggling past couple of weeks with severe performance issues on AEM6/Oak/MongoNS -- fortunately the issue was due to VM we were using. So, all seems well for now. BUT, during investigation, one of the things that we were worried about was document cache missing hits... we tried modifying its value to increase cache size to be put at 1GB. Although, that didn't quite help as the issue wasn't about cache size... but what worried us more was there was no good way for us to specify a cache size. While I agree that cache should be a memory hog... but entry size of a document in cache is quite variable in nature and as an admin I can make guesses about JCR nodes and their access patters. Document size, otoh, would vary even during the lifetime of repository. Moreover, the only downside (assuming RAM is cheap) of increasing cache to large number is making cache invalidation expensize. So, IMHO, document caches should be limited in terms of number of entries in cache. And if we still want to have byte size limitation, we can probably have both and cache respects whichever constraint hits first (sort of min(byte size, entry size)). We can log an issue for this and provide a patch too -- but it seemed better to have a conversation going before that. Thought? Thanks, Vikas
Re: [Document Cache Size] Is it better to have cache size using number of entries
Hello Vikas, On 18/08/2014 12:05, Vikas Saurabh wrote: Hi, ... specify a cache size. While I agree that cache should be a memory hog... but entry size of a document in cache is quite variable in nature and as an admin I can make guesses about JCR nodes and their access patters. Document size, otoh, would vary even during the lifetime of repository. Moreover, the only downside (assuming RAM is cheap) of increasing cache to large number is making cache invalidation expensize. So, IMHO, document caches should be limited in terms of number of entries in cache. And if we still want to have byte size limitation, we can probably have both and cache respects whichever constraint hits first (sort of min(byte size, entry size)). First of all I don't know MongoNS implementation details so I can be wrong. I'd rather keep the size in bytes as it gives me much more control over the memory I have and what I decide to provide to the application. If we say, to take an extreme example, 1 document only in cache and then this single document exceed the amount of available memory I fear an OOM. On the other hand having bytes ensure us the application keeps working and it will be task of a sysadmin to monitor the eventual hit/miss ratio to adjust the cache accordingly. About cache invalidation I'm not sure but it could be that the MongoNS implementation uses off-heap for caching. Cheers Davide
Re: [Document Cache Size] Is it better to have cache size using number of entries
we can probably have both and cache respects whichever constraint hits first (sort of min(byte size, entry size)). First of all I don't know MongoNS implementation details so I can be wrong. I'd rather keep the size in bytes as it gives me much more control over the memory I have and what I decide to provide to the application. If we say, to take an extreme example, 1 document only in cache and then this single document exceed the amount of available memory I fear an OOM. On the other hand having bytes ensure us the application keeps working and it will be task of a sysadmin to monitor the eventual hit/miss ratio to adjust the cache accordingly. Yes, sysadmin can modify cache size in bytes if miss ratio increases. But, in current scenario, I couldn't figure out a neat way (heuristic/guesswork) to figure out if it's application mis-behavior or lack of cache size (notice our issue didn't happen to be related to cache size... but still the question did bug us). On the other hand, an sysadmin can be provided with a rough idea about relation of (frequently used) repo nodes using which sysadmin can update cache size. Also, I do take the point of avoiding OOMs in case of pretty large documents which is why we can have both properties(byte size and entry count) with byte constraint being a fail safe. Thanks, Vikas
