Thanks Niels, some comments inline, assume this is for GSIP-132 <https://github.com/geoserver/geoserver/wiki/GSIP-132> (unless that is completed already).
On 7 October 2015 at 05:28, Niels Charlier <[email protected]> wrote: > Hi Jody, Gabriel, Kevin > > I have been porting all modules to use the resources system consistently > and only use files when necessary (usually external library). I still > stumbled upon two minor questions/issues I wanted to discuss. > > 1. Usage of the "data" directory. At the moment the import from data > directory -> jdbc store ignores the "data" directory. In a clustered > environment, this directory thus remains instance specific, and it would be > up to the user to refer to shared files. Are you talking about GEOSERVER_DATA_DIRECTORY/data? If so that is only a convention, I have made data directories that used "raster" and "vector" folders for example. For storing spatial data (GeoTIFF, Shapefile, Image Mosaic here) I had the idea of doing something like JNDI but for referencing an external folder used for this purpose. This could both provide an "ignore" list (so "data" was not hard coded) and allow for a cluster with RAID storage mapped to a specific mount. > At this moment, there is no reason why we couldn't include the data dir in > the jdbcstore and cache it before loading the geotools datastore. This is > actually what my modified version of the rest service already does because > it uses resources everywhere. For configuration files this is what we want. > Another idea, was to program the jdbcstore to return file based resources > only when the "data" directory is used, so that it definitely will never > store those files in the database unnecessarily. > Okay pretty sure you are talking about GEOSERVER_DATA_DIRECTORY/data now. Q: Is it worth removing the files that have been imported into JDBCConfig from GEOSERVER_DATA_DIRECTORY? This would prevent confusion, and allow GEOSERVER_DATA_DIRECTORY to work strictly as a cache (for the few things that require a file to be unpacked on to disk). 2. In the jdbcstore, should the children of a directory be cached when > dir() is called? Cached is on import (so yes). Should the resources be unpacked (staged) to the file system when dir() is called? Yes > The DataDirectory class uses the dir() method to know the root of the data > directory, causing the whole data directory to be cached at once multiple > times unnecessarily, since the root dir is usually requested just to know > the path for some reason (all code where it actually needs files in the > data dir, have been replaced by resources). This is a bug, such logic should be replaced. There is another method to get the root of the GEOSERVER_DATA_DIRECTORY. While we may hard code some things now it would be wide to have an extension point for modules (such as geowebcache) to mark off working directories that should not be cached. Using dir() to determine the root of GEOSERVER_DATA_DIRECTORY is a bad idea, in addition to breaking the design of dir() we are trying to avoid duplicating code contain data directory structure logic. > We now always want to use resources as long as possible, only calling > file() at the last moment if necessary. As a consequence the dir() method > is actually hardly used for the purpose or getting all the files inside > that dir. I would suggest on calling dir() only to create the dir if it > doesn't exist yet and not cache its children. There is only one part of > code left where that would pose a problem, the community module > "validation", which passes on a whole dir to its geotools counterpart. This > however could be changed in the geotools module to pass on a collection of > files instead. > Validation only needed one "validation" folder, so that code could be changed to use resource("validation").dir(). After this change, I wonder if we should make a doc page on the proper > practices of using the Resource API in order to be clustering-safe. > Yep, could add to the developers guide under "file access".
------------------------------------------------------------------------------
_______________________________________________ Geoserver-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-devel
