On Fri, Apr 24, 2020 at 12:23:14PM +1000, Paul Ripke wrote: > On Thu, Apr 23, 2020 at 06:14:00PM +0200, tlaro...@polynum.com wrote: > > On Thu, Apr 23, 2020 at 01:30:58PM +0100, Mike Pumford wrote: > > > > > > > > > On 23/04/2020 11:48, tlaro...@polynum.com wrote: > > > >[Please forgive me to give supplementary information gathered for all > > > >answers and "before". And thank you to all for answering.] > > > > > > > >To be more precise: > > > > > > > >The node is not a desktop node but a server one (so there is no X11 > > > >running, so no firefox or whatever), with two main functions: > > > > > > > >1) A file server; > > > >2) A processing node for data (GIS) manipulations, since there can be > > > >huge amounts of data to process, so it is more efficient to process > > > >where it is saved due to the network capabilities. > > > > > > > In my case the build machines main purpose is as a build server (hence the > > > jenkins server which is a java process). I use it as a desktop when > > > developing things as its my most powerful BSD build machine and its easier > > > to develop when you can use a browser to check API references etc so its > > > desktop use is intermittent. So they were just examples of memory hungry > > > processes that got swapped out unexpectedly rather than the only ones. > > > > > > >The node' uptime is 225 days (since some disk replacements) serving > > > >during office time files to 2 to 6 nodes. And it is processing data > > > >(sometimes during office time; generally outside office time). > > > > > > > >It has 3862 MB of avail memory. > > > > > > > >When using a program to convert coordinates, this program being a Linux > > > >binary, the program bailed out, 3 times in a row, after processing 20M > > > >records. Splitting the input data under this amount, I managed to > > > >process everything. > > > > > > > Could you be running into the data size limits for the process? > > > > No I think that you gave the answer in your previous mail. This is the > > sysctl variable vm.execmax. > > > > It is set to 30 (30%) and it was reaching the limit. Since there was > > till now no need for more, the swap space is limited (only 128MB). > > > > So the solution is indeed to play with the vm.* variables and here to > > increase vm.execmax (and I will too add some swap since I have spare > > partitions on some disks). > > To the best of my knowledge, vm.execmax just defines the maximum amount > of memory for mapped executable pages, and won't cause processes to be > killed. Since mapped executable pages are generally read-only, they can > just be dropped from RAM and paged in again from the executables as > needed. > > You probably just need to decrease vm.filemax, so that the file cache > doesn't put so much pressure on anonymous memory, and allows processes > to grow larger.
That makes indeed sense. Thanks! -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ http://www.sbfa.fr/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C