RE: [castor-dev] PerformanceI have asked to improve the Cache implementation. I think that there are several of us who would be willing to discuss how we can improve the cache. (ie. Changing LRU.java to a more generic Cache.java, allowing the user to create their own cache types....)
Adam ----- Original Message ----- From: Thomas Yip To: [EMAIL PROTECTED] Sent: Friday, September 28, 2001 5:41 PM Subject: Re: [castor-dev] Performance I am not convinced that Castor JDO is ready for distributed cache yet. Having a distributed cache may not solve the problem you want to solve. Well, we're moving to that direction. In fact, Castor's LockEngine have two levels. The lock level keeps all ROWs in action. The access to a ROW in the lock level is guarded by read/write lock, the value should not be changed, even in case of cache invalidation of that particular ROW. Because, the value is used for modification detection in later time. So, eviction should not be done on any locked object. But, it should be flagged instead, so in the commit of rollback time, the LockEngine can decide what to do. Not until the transaction is completed and no other transaction interested in that particular ROW, it is moved to the cache level. On the other hand, the make sure a particular ROW would only exist zero or one time in the lock level and the cache level, the adding/removing/moving of ROW is synchronized between the cache level and lock level. Because of that, the eviction should not be implemented outside the LockEngine. Consider the current architecture, the active cache in JCACHE or distributed cache might not be the best fit. Also, when a ROW is invalidated, in some case, the related ROW should be done too. And, it is not easy in the current implementation. If Castor JDO is a house, we are taking off walls in order to extend the house. At this moment, I rather wait until some extensions to finish first, before starts discussing and planning the next walls to be taken off. However, I am open to discuss building furniture that does not involve taking wall off. Examples of furniture are other passive cache types: weakReference cache, time-count limited cache or speedier implementation etc. Thomas -----Original Message----- From: Adam Esterline [mailto:[EMAIL PROTECTED]] Sent: Friday, September 28, 2001 12:12 PM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance I would. I have offered to clean up the current cache implementation. Adam -----Original Message----- From: Matthew Baird [mailto:[EMAIL PROTECTED]] Sent: Friday, September 28, 2001 2:09 PM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance the solution, imho, is a distributed cache. THere is a JCP in process with oracle for this kinda of technology. A stopgap would be to have an evict function and keep caches in sync with a pubsub jms implementation, but this doesn't solve the issue of updates in a highly distributed transactional system. Setting the caching to NONE is not an issue for those of us using castor in a j2ee environemnt (probably more and more of us every day). And setting the cache countlimited isn't a real solution for obvious reasons. If a distributed cache for castor project was started, who would be interested in helping? -----Original Message----- From: Sudhir Bhojwani [mailto:[EMAIL PROTECTED]] Sent: Friday, September 28, 2001 11:44 AM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance If you don't want Caching you can always select cache-type-mapping as "none"....no more caching overheads at Transaction commit time. Also selecting the default caching option "CountLimited" might not be that bad. I find the implementation quite decent. And the overall performance to be quite linear. Performed the test with quite a few client threads, stress tested the cache for inserts, updates and deletes in various locking modes. May be I will publish my detailed repot on the mailing list in a few days. Thanks ! Sudhir Bhojwani Altoweb, http://www.altoweb.com -----Original Message----- From: Liebster, Dan [mailto:[EMAIL PROTECTED]] Sent: Friday, September 28, 2001 6:19 AM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance I would be happy just to be able to force everything in the Castor cache to expire immediately with a method call on a single application server. Does anyone know if this is possible? Dan -----Original Message----- From: James Higginbotham [mailto:[EMAIL PROTECTED]] Sent: Friday, September 28, 2001 12:40 AM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance I have seen a this problem solved in TopLink by using a multicast approach, where each instance of Toplink sends an event to all other instances with the PK and type (or similiar event data) of the data modified. Whenever a retrieve of the data is requested, the particular record is known to be dirty and a refresh is made. Maybe this is a feature best suited to a "commercial grade" version of castor, or by someone with a lot of time to write and debug this logic. :) Its a nice feature, but one that is rare even in commercial packages outside of TopLink. James ----- Original Message ----- From: Matthew Baird To: [EMAIL PROTECTED] Sent: Thursday, September 27, 2001 3:17 PM Subject: Re: [castor-dev] Performance I imagine this is the same problem as running multiple app servers in a distributed environment with all of them using different VM's with castor banging on the same DB. For adding new objects, there is no problem, but the cache will become inconsistent if there are updates, and castor will not like that. Is there any way to accomplish updates for other sources not messing castor up? -----Original Message----- From: James Higginbotham [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 27, 2001 1:03 PM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance Dan, In my specific instances, the bulk loading was aggregated by a scheduled, background process once and never changed again. The aggregated data is read from castor, so no cache problems occurred. Anyone from the Castor team that could answer this question, I'm interested as well. I have seen TopLink, a very expensive but industrial strength mapper, have problems clearing the cache without a hack or VM restart. How does Castor deal with this, or does it? James ----- Original Message ----- From: Liebster, Dan To: [EMAIL PROTECTED] Sent: Thursday, September 27, 2001 11:57 AM Subject: Re: [castor-dev] Performance When using this approach, I would be interested what techniques you use to deal with reading cached objects in Castor that have had the underlying data changed by a batch update. Is there a clean way to "kick the cache" after running and external process? Dan -----Original Message----- From: James Higginbotham [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 27, 2001 12:13 PM To: [EMAIL PROTECTED] Subject: Re: [castor-dev] Performance Chris, I have used Castor for basic OR/Mapping and straight JDBC when performance is a factor. Speaking from a general perspective rather than from Castor benchmarks, one generally chooses OR Mapping tools such as Castor JDO when you want to save time and redundant JDBC code. If you need to achieve large performance gains when doing complex joins or bulk inserts, you may wish to use JDBC batch updates or optimized queries. I have done this on numerous J2EE projects using a combination of the Data Access pattern, Castor JDO, and JDBC 2.0. In short, use Castor to save time on the common CRUD (create, read, update, delete) and use straight JDBC for performance gains on bulk inserts using batch updates and manually tuned SQL. Hope this helps, James ----- Original Message ----- From: "Chris Cook" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, September 27, 2001 10:42 AM Subject: [castor-dev] Performance > > > I have recently downloaded and started to use Castor for a project. We had > in place a set of code to simulate a piece of our system, and measure the > inserts per second into the database. Using straight JDBC we usually > measure about 175 ins/s into the database. With Castor doing the > persistence, we measure about 40 ins/s. Does this figure seem right to you > all? Does Castor average about 4 or 5 times slower than straight JDBC? > > ----------------------------------------------------------- > If you wish to unsubscribe from this mailing, send mail to > [EMAIL PROTECTED] with a subject of: > unsubscribe castor-dev > > ----------------------------------------------------------- If you wish to unsubscribe from this mailing, send mail to [EMAIL PROTECTED] with a subject of: unsubscribe castor-dev ----------------------------------------------------------- If you wish to unsubscribe from this mailing, send mail to [EMAIL PROTECTED] with a subject of: unsubscribe castor-dev
