Hi Keith, The reasons I chose secondary caching broadly were,
As I mentioned in my previous mail, MIFOS has HashtableCacheProvider configured as default cache mechanism in Hibernate. It was very important to have more control over the cache like invalidating and repopulating the caches through a batch process since that was a business as well as performance requirement. I sent out a mail to mifos developer list asking about any experience in using caching in MIFOS. But I didn't receive any relevant reply. As changing any of the caching related configuration in Hibernate would have application wide affect, I was not sure of how the application would behave to that. So I avoided that approach. Browsing through the source code I found many custom implementation of caching, (Cache, OfficeCache, BulkEntryCache) as those implementation were too specific in the context which they were being used, I thought it would not be a good idea to use them, also not to add another custom implementation, rather than that use a mature java caching library like EhCache, OSCache, JCS. Probably it is time that we look towards upgrading the caching mechanism in Hibernate so that developers have more control over caching of data coming from Database. As far as preferring second level of caching is concerned, I would take it on case by case approach. I would prefer second level caching if I don't want my domain objects to be cached by ORM layer, and having a consistent state for those particular cache objects is not a strict requirement. For example, I am using second level of caching for a service layer from where I fetch the branch offices applicable to a user, loan officers under that branch office, and active centers for those loan officers. Later on, I fetch meeting dates for generated collection sheets based on selection above. If I have to cache it on the ORM layer, I would be storing big object graphs on the cache, also forcing my domain layer to be cached. Rather than that probably having a separate service that is proxied and cached helps me avoiding above concerns. But if the above conditions do matter, probably using ORM cache will be a lot easier. As far as providing additional caching facilities like overflowing cache to disk, eviction strategy etc, these are provided by caching framework and will be available to be configured by spring cache module. Going forward, I think there may a be a conflict of having multiple caching mechanisms, so it would be a good idea to have a common caching strategy as per requirement. Hope this clarifies the questions you asked. Regards, Amiruddin Nagri, ThoughtWorks Technologies India Pvt. Ltd., 2nd Floor, Tower C, Diamond District, Airport road, Bangalore, 560008, KA India Y! IM : [EMAIL PROTECTED] GTalk : [EMAIL PROTECTED] "Keith Pierce" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 01/11/2008 08:31 AM Please respond to Developer <[email protected]> To Developer <[email protected]> cc Subject [Mifos-developer] Caching mechanism for reporting (was: Collection Sheet Report Patch for MIFOS) Thanks for your input, Amiruddin. I've taken a look at the Spring Caching module that you propose using for improving the performance of the Collection Sheet Report ( https://springmodules.dev.java.net/docs/reference/0.8/html_single/#cache ). It appears to be an effective extension of the Spring Framework in its use of AOP as well as its ability to be configured using Java 1.5-style annotations. Despite its benefits, though, the MifOS project will incur some cost with its adoption, costs that I elaborate on below. In that light, I'd like to explore more deeply the rationale for preferring it over Hibernate's mature caching architecture. So please consider these points. I may be missing some good reason for accepting the cost, so please don't hesitate to elaborate. Adding a second caching mechanism to the application adds coding, developing, and process complexity. We would have to add additional configuration resources and manage them. Developers would have to decide which of the mechanisms to use in specific scenarios, and I'm not sure what the criteria should be for making that choice. I'd like to be assured that the two mechanisms won't interfere with each other. It seems risky and possibly suboptimal to have two caching mechanisms. Spring and Hibernate differ fairly significantly in how they approach caching strategies. There would be a cost to understanding both approaches, and knowing which to use in specific scenarios: Hibernate configures strategies for each class, but Spring customizes on a method-by-method (or method pattern such as "get*") basis. In light of the above, I'm not sure how well Spring's cache module integrates with Hibernate -- I haven't done enough research to know whether this is a problem. Hibernate implements a two-level caching strategy: transaction-level cache and process-level cache. Transaction-level caching is always on so you don't have to think about it. Spring appears to implement just one level (although it might be very customizable). It appears that methods that "do not have a return value (return value is void) are ignored" by the AOP mechanism of Spring (see above reference, section 3.5.1). Might this force MifOS developers into workarounds of the way they normally code? I'm not sure. AOP is a very powerful programming style, but it is complex. We should carefully consider the decision to start moving in that direction, especially just prior to a major release. You mention a couple of reasons for preferring Spring. See my comments below: You state that Spring is capable of having multiple caching options. By that, I assume you mean that you can easily configure Spring to use any of a number of cache providers (EHCache, JBoss Cache, OSCache, etc). Hibernate can also be so configured by a simple entry in its global configuration file. You mention the requirement that the cache be refreshed every midnight. I could not find mention of such a feature in the above documentation. It appears that this a feature that is provided not by the Spring cache module, but by the cache provider (OSCache provides the capability of expiring cache entries with a cron expression). If this is true, then an alternative approach would be to replace the provider currently used by Hibernate with OSCache. This change would be transparent to the code base. In the limited time I've had to research the Spring module, I haven't been able to determine whether Spring supports some essential features of Hibernate's. I list some of them below. I'd appreciate help in tracking this down. Hibernate provides several levels of cache concurrency strategies for transaction isolation for particular items being cached (see "Hibernate in Action", p 183). How does Spring provide this flexibility? Hibernate provides several configuration properties that transparently manage such settings as: whether or not cached items can be evicted from the cache; eviction timeout time; maximum cache size which, when reached causes the oldest-lived items to be evicted; amount of idle time before an item is evicted; whether cached items can overflow to disk; and possibly others. Does Spring include these features, or are they provided by the cache provider? Caching will become an increasingly critical facet of MifOS's architecture, and we want to be sure that we're moving in the right direction that will enable us to maximize the application's performance and scalability. As I have said, my time for researching this has been limited, so I may have missed some important factor -- your input will be appreciated. I look forward to working with your team to choose the best direction for us. Keith Pierce On Jan 9, 2008 5:46 AM, Amiruddin Nagri < [EMAIL PROTECTED]> wrote: Hi Kieth, In MIFOS, currently HashtableCache Provider is configured to provide caching for Hibernate. I was reusing some the domain objects in MIFOS for report parameters and was not sure how changing the caching strategy in hibernate for those objects would affect the rest of the application. Also Hibernate manages the lifecycle of the cache, that is invalidating and repopulating it. As there was a requirement related to invalidating the cache at mid night and repopulating it, this required to have more control over caching life cycle. Therefore I had to go with an external caching mechanism, or at least with the one that provided me that control. I evaluated some of the caching libraries that are available in Java. Out of those, Ehcache had a good documentation and an active community around it. Spring also addressed the issue of having multiple caching options in java and have come up with an abstraction that allows to cache without being tightly coupled to a particular caching framework(Atleast that is the idea behind the project, I am still figuring how that will be possible). So ideally, we can change the caching library without much affect on the rest of the codebase. This was my understanding when I started on implementing caching in MIFOS. I am still working on caching, and as I get more clarity on the subject, I will glad to share it with MIFOS community. Regards, Amiruddin Nagri, ThoughtWorks Technologies India Pvt. Ltd., 2nd Floor, Tower C, Diamond District, Airport road, Bangalore, 560008, KA India Y! IM : [EMAIL PROTECTED] GTalk : [EMAIL PROTECTED] "Keith Pierce" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 01/08/2008 11:04 PM Please respond to Developer <[email protected]> To Developer <[email protected]> cc Subject Re: [Mifos-developer] Collection Sheet Report Patch for MIFOS Amiruddin, thanks for the additional information. For our benefit can you elaborate on your choice of the Spring caching module to implement caching? For example, Hibernate itself provides query caching capabilities, and since your reports access data through Hibernate, that would be a logical candidate. Did your team have a reason to prefer Spring's caching module over Hibernate's, or any of the many other implementations? Keith On Jan 8, 2008 11:05 AM, Amiruddin Nagri <[EMAIL PROTECTED]> wrote: Hi, Apologies, I forgot to mention some guidelines for applying the previous patch. I have tested the given patch using Eclipse by applying on the latest copy of MIFOS from svn. So, it should be good if you use Eclipse to apply the patch. Also, for caching of some of the parameters in application, I have used spring-modules-cache.jar. I am attaching the same along with this mail. Please copy it to your mifos/lib folder for testing the patch. If you face any other issue regarding the patch, kindly send a mail. I will try to get back to you as soon as possible. Regards, Amiruddin Nagri, ThoughtWorks Technologies India Pvt. Ltd., 2nd Floor, Tower C, Diamond District, Airport road, Bangalore, 560008, KA India Y! IM : [EMAIL PROTECTED] GTalk : [EMAIL PROTECTED] Amiruddin Nagri <[EMAIL PROTECTED] > Sent by: [EMAIL PROTECTED] 01/08/2008 09:54 PM Please respond to Developer <[email protected] > To [email protected] cc Pramod Biligiri <[EMAIL PROTECTED] >, Nandini Yadalam < [EMAIL PROTECTED]>, Apoorv G Gawde <[EMAIL PROTECTED]> Subject [Mifos-developer] Collection Sheet Report Patch for MIFOS Hi, We are working on adding Collection Sheet to the report module for MIFOS. Find the patch attached for the same. Regards, Amiruddin Nagri, ThoughtWorks Technologies India Pvt. Ltd., 2nd Floor, Tower C, Diamond District, Airport road, Bangalore, 560008, KA India Y! IM : [EMAIL PROTECTED] GTalk : [EMAIL PROTECTED] ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
