On Fri, 18 Oct 2019 at 10:17, Jody Garnett <jody.garn...@gmail.com> wrote:
> That is ... unexpected. > > So there are other solutions to notify tile cache state across the cluster > right. Are they more appropriate then depending on file system polling > here? > I would suspect so. The catalog does so it should be possible to do the same with the tile layer catalog. Meanwhile I wouldn't mind this improvement. > > On Thu, Oct 17, 2019 at 11:27 PM Gabriel Roldan <gabriel.rol...@gmail.com> > wrote: > >> >> >> On Thu, 17 Oct 2019 at 13:46, Jody Garnett <jody.garn...@gmail.com> >> wrote: >> >>> Bad news is FileSystemWatcher is killing the machine. For each instance >>>> it gets one thread eating 100% of one CPU all the time. Reported this >>>> before, just sharing my pain. >>>> >>>> Been looking at replacing the internals of FileSystemWatcher to use >>>> java.nio.file.WatchService, and although at a first glance it looks pretty >>>> straightforward, it's not gonna be that simple. >>>> >>> >>> Darn, that was the point of making the interface (since we could not use >>> WatchService on Java 7 yet). >>> >>> >>>> There are a number of situations where it just won't work, basically >>>> when the data dir is on any sort of mapped filesystem (NFS, SAMBA, Docker >>>> bind mounts), for which the Linux implementation doesn't work as it relies >>>> on inotify. >>>> >>>> The MacOS implementation is a polling one, but it's gonna be better >>>> than the current custom one anyways. And for Windows, I don't know yet. >>>> >>> >>> Still this was only supposed to be used for a few files like the >>> security settings which require active notification? If a watcher is being >>> setup beyond that ... is it a mistake? >>> >>> In any case it's clear that anyone that has a large catalog will most >>>> probably have it on a shared filesystem to be used by several instances. >>>> So this might require a GSIP, which I'm willing to start, in order to >>>> tackle on the different scenarios, as it can get even more involved as >>>> above described. >>>> >>> >>> Before you go that far can we talk about how FileSystemWatcher is being >>> used... >>> >> >> Right. It's DefaultTileLayerCatalog listening to changes to gwc-layers/* >> in order to update tile layer configs when they're updated externally on a >> cluster (added at commit 966a8af90fc6b68725a78b09829c3cbc7360242e). >> >> For the time being, I managed to reduce the polling runtime from over 2 >> minutes to ~350 milliseconds, the CPU usage from constant 100% to peaks >> bellow 25% (or so says htop). Memory usage: old gen up to 477MB from 450MB, >> eden space peaks and down quickly of 1GB instead of fixed at around 850MB. >> >> Got some clean up to do before issueing a PR, but here it is: >> >> https://github.com/groldan/geoserver/commit/0ba1614a7f9b1b2bea5647160441f3bd534c9979 >> >> Gabriel >> >>> >>> -- >>> Jody >>> >> >> >> -- >> Gabriel Roldán >> > -- > -- > Jody Garnett > -- Gabriel Roldán
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
