My gut says, "find out how hard the DBMS is working to add one item". I think that would be consistent with the increasing time spent in Context.commit(). I suspect (but can't yet prove or disprove) that there are some places where the choice of columns to index should be reviewed w.r.t. large repositories. I don't even slightly resemble a DBMS performance expert, but I've been looking at performance issues with a much smaller repo -- around 22,000 items*. It looks to me at the moment as though Pg is doing quite a lot more I/O than I would have expected.
I've been collecting detailed logs of SQL statement execution times from Pg, but 8.0 apparently didn't log what was needed while 8.2 is logging so much every day that the analysis tool I'm using runs out of memory no matter how high I set the limit. I'm going to try carving the logs into smaller pieces.... If you can identify some statements that are exercised by batch submission, opening up the database with 'psql' and entering 'EXPLAIN some statement' should show where the time is going and why. Then you get to stare at it and ask yourself, "why did the query planner do it *that* way?" That submission runs sometimes work at both memory limits but more often at the higher limit suggests that we are leaking objects somewhere in the batch submission path. That would be a separate issue. ------------- * One thing that is helping is that I'd followed a rule of thumb and given Tomcat far too *much* memory, so Tomcat and Postgres were paging each other out. I reduced Tomcat's heap maximum and I think performance has smoothed out quite a bit. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Friends don't let friends publish revisable-form documents.
pgpWRjSWhukaF.pgp
Description: PGP signature
------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects
_______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
