I have to research a bit more and get a local build of mono building
but there are definitely some places here:
https://github.com/mono/mono/blob/master/mcs/class/System/System.Net/EndPointListener.cs#L85
That can break the async loop causing the entire lsitener to stop
accepting requests until its restarted.
Cheers,
Greg
On Wed, Mar 2, 2016 at 4:58 PM, Greg Young wrote:
> We have found a situation where HttpListener on mono (reproduced in
> linux/OSX) will permanently stop accepting requests.
>
> The issue involves ulimit. Basically if you get a connection flood
> greater than ulimit the server will stop accepting all requests until
> it is restarted.
>
> After the connection flood you can see the connections were all
> properly killed (I won't put all the ones that were there during the
> flood!):
>
> greg@clown:~$ lsof -p 16651
> lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing
> Output information may be incomplete.
> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> eventstor 16651 greg cwdDIR8,2 4096 2105316
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0
> eventstor 16651 greg rtdDIR8,2 40962 /
> eventstor 16651 greg txtREG8,2 37443029 2105361
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/eventstored
> eventstor 16651 greg memREG8,2 92504 12849525
> /lib/x86_64-linux-gnu/libgcc_s.so.1
> eventstor 16651 greg memREG8,2 51736 12845148
> /lib/x86_64-linux-gnu/libnss_files-2.21.so
> eventstor 16651 greg memREG8,2 47616 12845159
> /lib/x86_64-linux-gnu/libnss_nis-2.21.so
> eventstor 16651 greg memREG8,2 97224 12845153
> /lib/x86_64-linux-gnu/libnsl-2.21.so
> eventstor 16651 greg memREG8,2 35688 12845149
> /lib/x86_64-linux-gnu/libnss_compat-2.21.so
> eventstor 16651 greg memREG8,2 8464208 9444549
> /usr/lib/locale/locale-archive
> eventstor 16651 greg memREG8,2 1869392 12845162
> /lib/x86_64-linux-gnu/libc-2.21.so
> eventstor 16651 greg memREG8,2142080 12845083
> /lib/x86_64-linux-gnu/libpthread-2.21.so
> eventstor 16651 greg memREG8,2 14592 12845158
> /lib/x86_64-linux-gnu/libdl-2.21.so
> eventstor 16651 greg memREG8,2 31680 12845075
> /lib/x86_64-linux-gnu/librt-2.21.so
> eventstor 16651 greg memREG8,2 1084840 12845161
> /lib/x86_64-linux-gnu/libm-2.21.so
> eventstor 16651 greg memREG8,2154376 12845156
> /lib/x86_64-linux-gnu/ld-2.21.so
> eventstor 16651 greg memREG 0,21 4096 88 /dev/shm/mono.16651
> eventstor 16651 greg0u CHR 136,1 0t04 /dev/pts/1
> eventstor 16651 greg1u CHR 136,1 0t04 /dev/pts/1
> eventstor 16651 greg2u CHR 136,1 0t04 /dev/pts/1
> eventstor 16651 greg3r 0,11 0 7966 anon_inode
> eventstor 16651 greg4u REG8,2 8 2624038
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/writer.chk
> eventstor 16651 greg5u REG8,2 8 2624039
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chaser.chk
> eventstor 16651 greg6u REG8,2 8 2624040
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/epoch.chk
> eventstor 16651 greg7u REG8,2 8 2624041
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/truncate.chk
> eventstor 16651 greg8r CHR1,9 0t0 11 /dev/urandom
> eventstor 16651 greg9u REG8,2 268435712 2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-00.00
> eventstor 16651 greg 10r REG8,2 268435712 2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-00.00
> eventstor 16651 greg 11r REG8,2 268435712 2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-00.00
> eventstor 16651 greg 12r REG8,2 268435712 2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-00.00
> eventstor 16651 greg 13r REG8,2 268435712 2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-00.00
> eventstor 16651 greg 14r REG8,2 268435712 2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-00.00
> eventstor 16651 greg 15u IPv4 279150 0t0 TCP
> localhost:1113 (LISTEN)
> eventstor 16651 greg 16u 0,11 0 7966 anon_inode
> eventstor 16651 greg 17u IPv4 278173 0t0 TCP
> localhost:2113 (LISTEN)
>
>
> My guess is that the too many files kills the asynchronous loop around
> the accept in the httplistener code and will take a look there.
> Perhaps someone else has some ideas.
>
> Cheers,
>
> Greg
>
> --
> Studying for the Turing test
--
Studying for the Turing test
___