Hi,
recently we hit a performance issue using the JDBCFeatureStore
removeFeatures method.
The problem is in the JDBCDataStore ensureAuthorization call that is
checking if all the features can be removed (no locks on them).
We launched a removeFeatures with a Filter.INCLUDE filter on a 1.000.000
records table, that should simply remove all the features, so we thought
that should be quite fast (if the database is fast enough), but:
- the ensureAuthorization executes a sql to extract all the records and
check for locks record by record
- if prepared statements are not used a setFetchSize is not executed, so
on some databases (for example PostGIS) the entire recordset is loaded into
memory (and probably throwing an OOM on big datasets)
I was thinking of some ways to improve this stuff. My ideas would be:
- first of all we should use setFetchSize also on simple Statemenst, not
only on PreparedStatements (this would avoid the OOM)
- we should try to optimize, when possible the locks check, for example:
* if there are no locks on the featuretype the record by record check is
not needed
* we could try to only extract records that match the given filter
combined with a filter from the fids locked on the given featuretype and
check only those records
Both would require some changes to InProcessLockingManager class, for
example making the locks method public (currently is protected).
What do you think?
Thanks
Mauro
--
==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
==
Dott. Mauro Bartolomeoli
@mauro_bart
Senior Software Engineer
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
