Bugs item #512826, was opened at 2002-02-04 08:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=376685&aid=512826&group_id=22866
Category: JBossCMP Group: v3.0 Rabbit Hole Status: Open Resolution: Accepted Priority: 5 Submitted By: Georg Schmid (giorgio42) Assigned to: Dain Sundstrom (dsundstrom) Summary: RH shows quadratic behaviour in inserts Initial Comment: RH 20020105 from CVS, W2K, Oracle 8.1.7. see thread http://main.jboss.org/thread.jsp?forum=46&thread=7700 on JBoss forums. 1) The JDBCStoreCommand.execute generates a disastrous amount of debug output by printing "...NOT updated." for each bean instance it looks at. Getting rid of that would be easy. 2) The root cause of this large amount of debug output is the really interesting issue: Whenever a new bean instance is about to be created the instance cache is "flushed". By this I mean the process where the storage manager walks through the list of all cached instances of the bean class, fetches the list of modified fields, and, if it detects a modification, writes the outstanding modifications to persistent store. The problem with this approach is, that it leads to an O(mn) behaviour, where m is the number of instances in the cache and n is the number of operations, that trigger a flush on the cache. Imagine this scenario: the instance cache is configured to hold 10000 instances. Now a bulk insert of 5000 new instances (as often occurs when importing data from an external table) is started. On the first insert JDBCStoreCommand.execute has to check 0 instances, on the second insert one instance, on the third insert two instances and so on. All in all the execute() method will be called (1 + 2 + 3 + ... + n-1) times where n is the number of inserts, resulting in (n-1)*(n-2)/2 calls to execute. In the case of 5000 new instances this means that there are 4999*4998/2 = 12492501, ie. more than 12 millions calls to execute(). The core problem is the strategy for detecting modified instances: instead of looking at each instance and deciding whether it has to be updated on each instance cache flush, the modified instances should be registered in a "modified instance" list, so that no extra effort is needed to detect "dirty" instances. Instead the set of modified instances would already be known. Commenting out the debug message reduces the time it takes to insert 3464 rows from more than 700 seconds to 225 seconds. ---------------------------------------------------------------------- >Comment By: David Jencks (d_jencks) Date: 2002-02-05 13:57 Message: Logged In: YES user_id=60525 Looks like a good list of things to do. If you will make sure my patch fixes your problem I will double check that it doesn't break any tests and commit it. A test case would be great but not essential. I think my patch prevents getting to the debug log statement, but it should be somewhat wrapped anyway... ---------------------------------------------------------------------- Comment By: Georg Schmid (giorgio42) Date: 2002-02-05 13:34 Message: Logged In: YES user_id=437570 What I can do (deadlines approaching) - get a new snapshot (the automated tests show that now the time has come,... done) - see how things work - try David's patch - reduce the real world code to a test case (any hints on where to put it?; I'll try to do like the Romans do) As to the debug log statement: it is already wrapped correctly but as it is inside a tight inner loop, I suggest disabling it (if (false) {...}), at least as long as the quadratic behaviour is there. Now the test suite is running... georg ---------------------------------------------------------------------- Comment By: David Jencks (d_jencks) Date: 2002-02-04 09:05 Message: Logged In: YES user_id=60525 I think I have a fix for this. I don't see a way to attach a file here so I will send it to the forum thread. Could you try it out to see how well it works? A test case for the testsuite would be great! david jencks ---------------------------------------------------------------------- Comment By: marc fleury (mnf999) Date: 2002-02-04 08:48 Message: Logged In: YES user_id=174548 Giorgio why don't you submit 2 patches. First the debugs should be "if log.debugEnabled() then..." And then your idea to make sure it doesn't grow with numbers. I'll get you rw ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=376685&aid=512826&group_id=22866 _______________________________________________ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
