Hi all, Recently my system has been hit with a "too many open files" error. Attached is the result of running netstat -apn .
The setup is a Play Framework 2.4.2 node sending message to sharded actors using ClusterSharding(system).startProxy(...). The region is provided by a class called RouteeProvider that is binded in Guice asEagerSingleton(). The sharded actor is located at two different nodes at two separate server. My question is, does the proxy open a new socket every time it tries to connect to the sharded actor? Does the error has anything to do with https://github.com/akka/akka/issues/16346 (because we are seeing some sort of anomaly in CPU usage after deployment which may cause network partition)? Akka version is 2.4.0. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1635/nginx.conf tcp 0 0 172.16.10.2:2550 0.0.0.0:* LISTEN 17638/java tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1346/sshd tcp 0 0 0.0.0.0:2552 0.0.0.0:* LISTEN 17638/java tcp 0 0 172.16.10.2:2552 172.16.10.11:37406 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:38412 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37280 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:35407 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:34483 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:38991 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37734 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37038 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:59513 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37057 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:36765 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:36945 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:39207 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:35263 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37787 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37408 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:59982 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37213 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:58685 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:33696 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37491 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:59581 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:36442 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:59061 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:38058 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37407 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37077 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:33830 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:56231 SYN_RECV - tcp 0 0 172.16.10.2:2552 172.16.10.11:37383 SYN_RECV - tcp 0 0 127.0.0.1:17123 0.0.0.0:* LISTEN 1395/python tcp 502 0 172.16.10.2:2552 172.16.10.11:58090 CLOSE_WAIT - tcp 0 0 172.16.10.2:59308 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 127.0.0.1:36135 127.0.0.1:17123 TIME_WAIT - tcp 499 0 172.16.10.2:2552 172.16.10.11:57793 CLOSE_WAIT - tcp 432 0 172.16.10.2:2552 172.16.10.11:57177 CLOSE_WAIT - tcp 0 0 172.16.10.2:44857 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:45240 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 6 172.16.10.2:44583 172.16.10.8:3306 FIN_WAIT1 - tcp 491 0 172.16.10.2:2552 172.16.10.11:43727 CLOSE_WAIT - tcp 495 0 172.16.10.2:2552 172.16.10.11:58533 CLOSE_WAIT - tcp 0 6 172.16.10.2:44582 172.16.10.8:3306 FIN_WAIT1 - tcp 497 0 172.16.10.2:2552 172.16.10.11:54152 CLOSE_WAIT - tcp 503 0 172.16.10.2:2552 172.16.10.11:56950 CLOSE_WAIT - tcp 502 0 172.16.10.2:2552 172.16.10.11:56675 CLOSE_WAIT - tcp 0 6 172.16.10.2:44580 172.16.10.8:3306 FIN_WAIT1 - tcp 0 0 172.16.10.2:44855 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 0 127.0.0.1:36149 127.0.0.1:17123 TIME_WAIT - tcp 484 0 172.16.10.2:2552 172.16.10.11:58532 CLOSE_WAIT - tcp 0 0 172.16.10.2:2550 172.16.10.4:45975 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:44879 172.16.10.8:3306 ESTABLISHED 17638/java tcp 499 0 172.16.10.2:2552 172.16.10.11:56824 ESTABLISHED - tcp 0 1 172.16.10.2:2552 172.16.10.11:58550 LAST_ACK - tcp 0 0 172.16.10.2:2552 172.16.10.11:35175 ESTABLISHED 17638/java tcp 473 0 172.16.10.2:2552 172.16.10.11:56063 CLOSE_WAIT - tcp 0 0 172.16.10.2:36534 169.254.169.254:80 ESTABLISHED 1802/python tcp 464 0 172.16.10.2:2552 172.16.10.11:57520 CLOSE_WAIT - tcp 379 0 172.16.10.2:2552 172.16.10.11:57394 CLOSE_WAIT - tcp 499 0 172.16.10.2:2552 172.16.10.11:43772 CLOSE_WAIT - tcp 0 0 172.16.10.2:45241 172.16.10.9:9300 ESTABLISHED 17638/java tcp 502 0 172.16.10.2:2552 172.16.10.11:57393 CLOSE_WAIT - tcp 487 0 172.16.10.2:2552 172.16.10.11:58534 CLOSE_WAIT - tcp 503 0 172.16.10.2:2552 172.16.10.11:57255 CLOSE_WAIT - tcp 0 0 127.0.0.1:36147 127.0.0.1:17123 TIME_WAIT - tcp 496 0 172.16.10.2:2552 172.16.10.11:56064 CLOSE_WAIT - tcp 501 0 172.16.10.2:2552 172.16.10.11:41218 CLOSE_WAIT - tcp 0 0 172.16.10.2:2552 172.16.10.11:45451 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:53237 54.236.68.122:514 ESTABLISHED 940/rsyslogd tcp 0 0 172.16.10.2:45244 172.16.10.9:9300 ESTABLISHED 17638/java tcp 390 0 172.16.10.2:2552 172.16.10.11:43731 CLOSE_WAIT - tcp 0 0 172.16.10.2:44386 169.254.169.254:80 ESTABLISHED 15816/python tcp 0 0 172.16.10.2:44360 169.254.169.254:80 TIME_WAIT - tcp 0 6 172.16.10.2:44581 172.16.10.8:3306 FIN_WAIT1 - tcp 614 0 172.16.10.2:2552 172.16.10.11:58817 CLOSE_WAIT - tcp 0 6 172.16.10.2:44587 172.16.10.8:3306 FIN_WAIT1 - tcp 0 0 172.16.10.2:45250 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:80 172.16.10.11:52369 TIME_WAIT - tcp 0 0 127.0.0.1:36142 127.0.0.1:17123 TIME_WAIT - tcp 500 0 172.16.10.2:2552 172.16.10.11:57395 CLOSE_WAIT - tcp 497 0 172.16.10.2:2552 172.16.10.11:56948 CLOSE_WAIT - tcp 495 0 172.16.10.2:2552 172.16.10.11:54467 CLOSE_WAIT - tcp 0 0 172.16.10.2:44853 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:2552 172.16.10.11:52856 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:44877 172.16.10.8:3306 ESTABLISHED 17638/java tcp 503 0 172.16.10.2:2552 172.16.10.11:45792 CLOSE_WAIT - tcp 0 1 172.16.10.2:2552 172.16.10.11:34331 LAST_ACK - tcp 0 0 172.16.10.2:44856 172.16.10.8:3306 ESTABLISHED 17638/java tcp 503 0 172.16.10.2:2552 172.16.10.11:57100 CLOSE_WAIT - tcp 0 1 172.16.10.2:2552 172.16.10.11:58399 LAST_ACK - tcp 0 0 172.16.10.2:44875 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 0 127.0.0.1:36131 127.0.0.1:17123 TIME_WAIT - tcp 0 0 172.16.10.2:44852 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 1 172.16.10.2:2552 172.16.10.11:34503 LAST_ACK - tcp 0 6 172.16.10.2:44586 172.16.10.8:3306 FIN_WAIT1 - tcp 0 1 172.16.10.2:2552 172.16.10.11:58549 LAST_ACK - tcp 502 0 172.16.10.2:2552 172.16.10.11:58387 CLOSE_WAIT - tcp 0 0 172.16.10.2:45246 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:22 180.245.49.8:59095 ESTABLISHED 14965/sshd: gobox-p tcp 0 0 172.16.10.2:45239 172.16.10.9:9300 ESTABLISHED 17638/java tcp 500 0 172.16.10.2:2552 172.16.10.11:58966 CLOSE_WAIT - tcp 402 0 172.16.10.2:2552 172.16.10.11:54569 ESTABLISHED - tcp 490 0 172.16.10.2:2552 172.16.10.11:58821 CLOSE_WAIT - tcp 496 0 172.16.10.2:2552 172.16.10.11:58086 CLOSE_WAIT - tcp 483 0 172.16.10.2:2552 172.16.10.11:57911 ESTABLISHED - tcp 499 0 172.16.10.2:2552 172.16.10.11:58535 CLOSE_WAIT - tcp 496 0 172.16.10.2:2552 172.16.10.11:57519 CLOSE_WAIT - tcp 0 0 172.16.10.2:44854 172.16.10.8:3306 ESTABLISHED 17638/java tcp 503 0 172.16.10.2:2552 172.16.10.11:57521 CLOSE_WAIT - tcp 0 0 172.16.10.2:45238 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:2550 172.16.10.7:35601 ESTABLISHED 17638/java tcp 0 6 172.16.10.2:44584 172.16.10.8:3306 FIN_WAIT1 - tcp 0 0 172.16.10.2:2552 172.16.10.11:33811 ESTABLISHED 17638/java tcp 484 0 172.16.10.2:2552 172.16.10.11:58819 CLOSE_WAIT - tcp 502 0 172.16.10.2:2552 172.16.10.11:57256 CLOSE_WAIT - tcp 382 0 172.16.10.2:2552 172.16.10.11:58967 CLOSE_WAIT - tcp 0 0 172.16.10.2:44817 172.16.10.8:3306 ESTABLISHED 17638/java tcp 503 0 172.16.10.2:2552 172.16.10.11:43582 CLOSE_WAIT - tcp 0 0 127.0.0.1:36139 127.0.0.1:17123 TIME_WAIT - tcp 0 0 172.16.10.2:44872 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 1 172.16.10.2:2552 172.16.10.11:34332 LAST_ACK - tcp 0 0 172.16.10.2:44383 169.254.169.254:80 ESTABLISHED 1810/python tcp 502 0 172.16.10.2:2552 172.16.10.11:45791 CLOSE_WAIT - tcp 0 0 127.0.0.1:36133 127.0.0.1:17123 TIME_WAIT - tcp 0 0 172.16.10.2:2550 172.16.10.3:50776 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:58273 169.254.169.254:80 CLOSE_WAIT 1810/python tcp 432 0 172.16.10.2:2552 172.16.10.11:44600 CLOSE_WAIT - tcp 0 0 172.16.10.2:44876 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:44859 172.16.10.8:3306 ESTABLISHED 17638/java tcp 494 0 172.16.10.2:2552 172.16.10.11:43445 CLOSE_WAIT - tcp 0 452 172.16.10.2:33756 54.164.8.26:514 ESTABLISHED 940/rsyslogd tcp 1 0 172.16.10.2:58286 169.254.169.254:80 CLOSE_WAIT 1802/python tcp 0 0 172.16.10.2:2550 172.16.10.5:36063 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:44858 172.16.10.8:3306 ESTABLISHED 17638/java tcp 499 0 172.16.10.2:2552 172.16.10.11:58386 CLOSE_WAIT - tcp 0 0 172.16.10.2:45247 172.16.10.9:9300 ESTABLISHED 17638/java tcp 500 0 172.16.10.2:2552 172.16.10.11:56400 CLOSE_WAIT - tcp 496 0 172.16.10.2:2552 172.16.10.11:54567 ESTABLISHED - tcp 0 6 172.16.10.2:44585 172.16.10.8:3306 FIN_WAIT1 - tcp 0 18564 172.16.10.2:22 175.106.8.178:58566 ESTABLISHED 15595/sshd: gobox-p tcp 0 6 172.16.10.2:44598 172.16.10.8:3306 FIN_WAIT1 - tcp 0 0 172.16.10.2:2550 172.16.10.6:59795 ESTABLISHED 17638/java tcp 565 0 172.16.10.2:2552 172.16.10.11:56228 CLOSE_WAIT - tcp 0 0 172.16.10.2:45248 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:44878 172.16.10.8:3306 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:45243 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:45245 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 172.16.10.2:45242 172.16.10.9:9300 ESTABLISHED 17638/java tcp 496 0 172.16.10.2:2552 172.16.10.11:57910 CLOSE_WAIT - tcp 0 0 127.0.0.1:36137 127.0.0.1:17123 TIME_WAIT - tcp 496 0 172.16.10.2:2552 172.16.10.11:43583 CLOSE_WAIT - tcp 498 0 172.16.10.2:2552 172.16.10.11:58085 CLOSE_WAIT - tcp 410 0 172.16.10.2:2552 172.16.10.11:57099 CLOSE_WAIT - tcp 0 1 172.16.10.2:2552 172.16.10.11:60517 LAST_ACK - tcp 0 0 172.16.10.2:45249 172.16.10.9:9300 ESTABLISHED 17638/java tcp 0 0 127.0.0.1:36145 127.0.0.1:17123 TIME_WAIT - tcp 493 0 172.16.10.2:2552 172.16.10.11:56947 CLOSE_WAIT - tcp6 0 0 :::22 :::* LISTEN 1346/sshd udp 0 0 0.0.0.0:37728 0.0.0.0:* 733/dhclient udp 0 0 127.0.0.1:8125 0.0.0.0:* 1394/python udp 0 0 0.0.0.0:68 0.0.0.0:* 733/dhclient udp 0 0 172.16.10.2:123 0.0.0.0:* 2089/ntpd udp 0 0 127.0.0.1:123 0.0.0.0:* 2089/ntpd udp 0 0 0.0.0.0:123 0.0.0.0:* 2089/ntpd udp 0 0 0.0.0.0:33962 0.0.0.0:* 17638/java udp6 0 0 :::65519 :::* 733/dhclient udp6 0 0 :::123 :::* 2089/ntpd Active UNIX domain sockets (servers and established) Proto RefCnt Flags Type State I-Node PID/Program name Path unix 2 [ ACC ] STREAM LISTENING 11034 1256/acpid /var/run/acpid.socket unix 2 [ ACC ] STREAM LISTENING 10338 322/cgmanager /sys/fs/cgroup/cgmanager/sock unix 2 [ ACC ] STREAM LISTENING 10258 1/init @/com/ubuntu/upstart unix 2 [ ACC ] STREAM LISTENING 11153 1390/python /opt/datadog-agent/run/datadog-supervisor.sock.1386 unix 2 [ ACC ] SEQPACKET LISTENING 8357 427/systemd-udevd /run/udev/control unix 2 [ ACC ] STREAM LISTENING 12488 1657/uuidd /run/uuidd/request unix 2 [ ACC ] STREAM LISTENING 10192 919/dbus-daemon /var/run/dbus/system_bus_socket unix 15 [ ] DGRAM 1767 940/rsyslogd /dev/log unix 2 [ ] STREAM CONNECTED 256333570 17638/java unix 2 [ ] DGRAM 1855 919/dbus-daemon unix 3 [ ] STREAM CONNECTED 1760 919/dbus-daemon /var/run/dbus/system_bus_socket unix 2 [ ] DGRAM 1823 1256/acpid unix 3 [ ] STREAM CONNECTED 275812072 14965/sshd: gobox-p unix 2 [ ] DGRAM 12573 1802/python unix 3 [ ] STREAM CONNECTED 10559 606/upstart-socket- unix 2 [ ] DGRAM 11471 1394/python unix 3 [ ] STREAM CONNECTED 10561 1/init @/com/ubuntu/upstart unix 2 [ ] DGRAM 12542 1754/sshguard unix 3 [ ] DGRAM 8391 427/systemd-udevd unix 3 [ ] STREAM CONNECTED 8346 417/upstart-udev-br unix 3 [ ] STREAM CONNECTED 275812077 14965/sshd: gobox-p unix 3 [ ] STREAM CONNECTED 8657 1/init @/com/ubuntu/upstart unix 2 [ ] STREAM CONNECTED 256334619 17638/java unix 2 [ ] DGRAM 8789 1342/python unix 2 [ ] DGRAM 275866870 15861/sudo unix 3 [ ] STREAM CONNECTED 275813725 15734/0 unix 2 [ ] DGRAM 1908 1396/python unix 3 [ ] STREAM CONNECTED 10206 1/init unix 3 [ ] STREAM CONNECTED 275812071 15116/1 unix 3 [ ] STREAM CONNECTED 8752 919/dbus-daemon /var/run/dbus/system_bus_socket unix 3 [ ] STREAM CONNECTED 275812830 919/dbus-daemon /var/run/dbus/system_bus_socket unix 3 [ ] STREAM CONNECTED 2019 1635/nginx.conf unix 3 [ ] STREAM CONNECTED 10887 886/upstart-file-br unix 2 [ ] DGRAM 11189 1395/python unix 3 [ ] DGRAM 8390 427/systemd-udevd unix 3 [ ] STREAM CONNECTED 8355 1/init @/com/ubuntu/upstart unix 2 [ ] DGRAM 13314 1810/python unix 3 [ ] STREAM CONNECTED 2018 1635/nginx.conf unix 3 [ ] STREAM CONNECTED 11079 1282/systemd-logind unix 2 [ ] DGRAM 11782 2089/ntpd unix 3 [ ] STREAM CONNECTED 275813726 15595/sshd: gobox-p unix 2 [ ] DGRAM 275813707 15595/sshd: gobox-p unix 3 [ ] STREAM CONNECTED 1753 919/dbus-daemon unix 2 [ ] STREAM CONNECTED 275866868 15861/sudo unix 2 [ ] DGRAM 275812808 14965/sshd: gobox-p unix 3 [ ] STREAM CONNECTED 275813728 15595/sshd: gobox-p unix 3 [ ] STREAM CONNECTED 275864637 919/dbus-daemon /var/run/dbus/system_bus_socket unix 3 [ ] STREAM CONNECTED 1754 919/dbus-daemon
