On Thu, 17 Oct 2019 at 13:46, Jody Garnett <[email protected]> wrote:
> Bad news is FileSystemWatcher is killing the machine. For each instance it >> gets one thread eating 100% of one CPU all the time. Reported this before, >> just sharing my pain. >> >> Been looking at replacing the internals of FileSystemWatcher to use >> java.nio.file.WatchService, and although at a first glance it looks pretty >> straightforward, it's not gonna be that simple. >> > > Darn, that was the point of making the interface (since we could not use > WatchService on Java 7 yet). > > >> There are a number of situations where it just won't work, basically when >> the data dir is on any sort of mapped filesystem (NFS, SAMBA, Docker bind >> mounts), for which the Linux implementation doesn't work as it relies on >> inotify. >> >> The MacOS implementation is a polling one, but it's gonna be better than >> the current custom one anyways. And for Windows, I don't know yet. >> > > Still this was only supposed to be used for a few files like the security > settings which require active notification? If a watcher is being setup > beyond that ... is it a mistake? > > In any case it's clear that anyone that has a large catalog will most >> probably have it on a shared filesystem to be used by several instances. >> So this might require a GSIP, which I'm willing to start, in order to >> tackle on the different scenarios, as it can get even more involved as >> above described. >> > > Before you go that far can we talk about how FileSystemWatcher is being > used... > Right. It's DefaultTileLayerCatalog listening to changes to gwc-layers/* in order to update tile layer configs when they're updated externally on a cluster (added at commit 966a8af90fc6b68725a78b09829c3cbc7360242e). For the time being, I managed to reduce the polling runtime from over 2 minutes to ~350 milliseconds, the CPU usage from constant 100% to peaks bellow 25% (or so says htop). Memory usage: old gen up to 477MB from 450MB, eden space peaks and down quickly of 1GB instead of fixed at around 850MB. Got some clean up to do before issueing a PR, but here it is: https://github.com/groldan/geoserver/commit/0ba1614a7f9b1b2bea5647160441f3bd534c9979 Gabriel > > -- > Jody > -- Gabriel Roldán
_______________________________________________ Geoserver-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-devel
