‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, January 27, 2019 5:37 PM, teor <t...@riseup.net> wrote:
> On January 27, 2019 1:07:12 PM UTC, notatorservernotatorser...@protonmail.com > wrote: > > > I am running tor 4.0.1-alpha inside a docker container with a memory > > limit (6GB). Tor runs out of memory and aborts or crashes periodically. > > Tor assumes that 64-bit machines have 8 GB of RAM, unless the relevant APIs > return a lower value. > > How does docker implement memory limits? > Does it modify the values returned by the Linux RAM APIs? > Docker sets up a cgroup with a memory limit. I guess you could check the value: `cat /sys/fs/cgroup/memory/memory.limit_in_bytes`. The value reported in `cat /proc/meminfo` is the same for every container, i.e. host system value. So no, it does not modify the values. > > Usually I see an error message like this: > > Could not mmap file "/var/lib/tor/data/diff-cache/1149": Out of memory > > ...(repeated some number of times) > > Please paste your log onhttps://paste.debian.net here's a sample of logs: http://paste.debian.net/1062943/ I can capture some more if these are helpful. Retention period is short > > How many times are these messages repeated? Depends but it can be anywhere from 1 to ~20. > Is the diff number the same, or different? diff number is increasing > How many files are in that directory? > currently 259 files (~123MB) > > followed by segfault > > Sometimes I see a message: > > Out of memory on malloc(). Dying > > followed by abort. > > Am I correct to assume the diff-cache is the issue here? Looking at the > > files it seems they are all pretty small (~500K). Is some badly behaved > > client requesting 12,000 of these diffs causing my relay to mmap them > > all at once? > > How many times are these messages repeated? > > > or is it just expensive to generate and generating 30-60 > > at once is enough to use all the memory? > > The diffs are generated once an hour, then cached for requests. > Do the errors happen at generation time, or request time? I guess at generation time if there is no way for a client to request to generate many diffs. Looking at mtime of these files it seems they are created in bursts: / # stat /var/lib/tor/data/diff-cache/* | grep Modify | sort | uniq -c ... 24 Modify: 2019-01-28 17:15:30.000000000 6 Modify: 2019-01-28 17:15:31.000000000 15 Modify: 2019-01-28 17:15:32.000000000 6 Modify: 2019-01-28 17:15:33.000000000 12 Modify: 2019-01-28 17:15:34.000000000 21 Modify: 2019-01-28 17:15:35.000000000 1 Modify: 2019-01-28 17:20:27.000000000 2 Modify: 2019-01-28 18:03:27.000000000 21 Modify: 2019-01-28 18:03:28.000000000 6 Modify: 2019-01-28 18:03:29.000000000 9 Modify: 2019-01-28 18:03:30.000000000 18 Modify: 2019-01-28 18:03:31.000000000 3 Modify: 2019-01-28 18:03:32.000000000 6 Modify: 2019-01-28 18:03:33.000000000 18 Modify: 2019-01-28 18:03:34.000000000 18 Modify: 2019-01-28 18:03:35.000000000 > > > Are there any config options to reject expensive queries or otherwise > > limit concurrency? > > Try > MaxMemInQueues 4 GB > I already have max mem set to 5GB but I will set to 4GB. > If that doesn't work, try > NumCPUs 2 > and please let us know, because we'd like to fix these kinds of bugs. > I will try this also. Thanks > T > > ---------------------------------------------------------------------------------------------------------------------------------------- > > teor > > ----- > > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays