Finally, I think that the problem in /usr/include/linux/posix_types.h, #define __FD_SETSIZE 1024
So, to solve the problem it's necessary to use several instances of policyd or use method different from select(). Artem Bokhan ?????: > May developers test policyd with about 2k simultaneous tcp connections? > Just open them. > > Tobias J. Kreidl ?????: > >> Hello, Artem: >> >> On Wed, 31 Oct 2007, Artem Bokhan wrote: >> >> >> >>> Hello again, I would like to return to the problem. >>> >>> After creating a lot of connections policyd stops to receive new >>> connections and use 100% of cpu. >>> Connections was simulated with JMeter (1500 connections). Anybody can >>> confirm the same behavior? >>> >>> >>> >> I don't think we tsted above around 1000 smtp connections, since postfix >> is normlly limited in out case to 100 connections on each server. Our >> machine has 8 GB of RAM. It was still running, though pretty slowly at >> that point. >> >> Also, you didn't mention of you actually ran out of memory and/or swap >> space. Some operating systems, like Solaris, also have a maximum user >> process parameter set in the kernel -- maxuprc. Is policyd runnning as >> "root" or a different user? You might just write a simple program that >> spawns 1500 or so processes and see if you come up against a process >> limit, memory limit, or other impediment. If nothing else, it sounds like >> you may want to rate-limit the number of connections to postfix or >> whatever your mail server is. If that doesn't keep your mail flow going, >> you may need additional servers. >> >> --Tobias >> >> >> >>> Ubuntu Dapper, uname -a >>> Linux 2.6.15-26-amd64-server #1 SMP Thu Aug 3 03:32:26 UTC 2006 x86_64 >>> GNU/Linux >>> >>> >>> >>> strace: >>> >>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 12 (in >>> [81 482]]) >>> >>> >>> sysctl fs.file-max >>> fs.file-max = 767080 >>> >>> >>> ulimit -n >>> 100000 >>> >>> >>> Compile-time configs: >>> >>> /* CONFIGS */ >>> #define PROJECT "policyd" >>> #define VERSION "v1.82" >>> >>> /* Miscellaneous constants */ >>> #define LISTENQ 1023 /* 2nd argument to listen() */ >>> #define MAXLINE 1023 /* max text line length */ >>> #define BUFFSIZE 8191 /* buffer size for reads and writes */ >>> #define BUFSIZE 4095 >>> #ifndef MAXFDS >>> #define MAXFDS 20479 /* max file descriptors */ >>> #endif >>> >>> >>> CC := gcc >>> CPPFLAGS := -O $(inc) >>> CFLAGS := -g -W -Wall -DMAXFDS=20479 >>> OS_NAME := $(shell uname | tr [A-Z] [a-z]) >>> LDLIBS = $(lib) -lmysqlclient -lz >>> >>> >>> Cami Sardinha ?????: >>> >>> >>>> Artem Bokhan wrote: >>>> >>>> >>>> >>>>> I will play with ulimit (was 1024 :( ) first and wait until bots will >>>>> wake up again :) >>>>> May be it's possible to increase the limits by policyd instead of shell?.. >>>>> >>>>> >>>>> >>>> There is no guarantee that Policyd's attempt to raise it will work. >>>> >>>> I do recommend you do some kernel tuning as well as raising the >>>> ulimit's. I can see from your previous post that Policyd is in fact >>>> built to use 4096 file descriptors, however, if you're limiting >>>> Policyd to 1024 in ulimit, can you really blame Policyd for its >>>> behavior? >>>> >>>> Cami >>>> >>>> ------------------------------------------------------------------------- >>>> This SF.net email is sponsored by: Splunk Inc. >>>> Still grepping through log files to find problems? Stop. >>>> Now Search log events and configuration files using AJAX and a browser. >>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ >>>> _______________________________________________ >>>> policyd-users mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/policyd-users >>>> >>>> >>>> >>> >>> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. >> Still grepping through log files to find problems? Stop. >> Now Search log events and configuration files using AJAX and a browser. >> Download your FREE copy of Splunk now >> http://get.splunk.com/ >> _______________________________________________ >> policyd-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/policyd-users >> >> > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > policyd-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/policyd-users > ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ policyd-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/policyd-users
