I tried to change #define __FD_SETSIZE 1024
to #ifndef __FD_SETSIZE #define __FD_SETSIZE 1024 #endif in /usr/include/bits/typesizes.h and #define __FD_SETSIZE 20480 in policyd.h As I could see, that resolved the problem.... And I think these changes are not correct.... :) May somebody predict possible problems caused by this change? select(1570, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 295 1160 1285 1300 1492 1568 1569], [], NULL, NULL) = 1 (in [9]) select(1570, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 295 1160 1285 1300 1492 1568 1569], [], NULL, NULL) = 1 (in [90]) Artem Bokhan ?????: > Finally, I think that the problem in > /usr/include/linux/posix_types.h, #define __FD_SETSIZE 1024 > > So, to solve the problem it's necessary to use several instances of > policyd or use method different from select(). > > Artem Bokhan ?????: > >> May developers test policyd with about 2k simultaneous tcp connections? >> Just open them. >> >> Tobias J. Kreidl ?????: >> >> >>> Hello, Artem: >>> >>> On Wed, 31 Oct 2007, Artem Bokhan wrote: >>> >>> >>> >>> >>>> Hello again, I would like to return to the problem. >>>> >>>> After creating a lot of connections policyd stops to receive new >>>> connections and use 100% of cpu. >>>> Connections was simulated with JMeter (1500 connections). Anybody can >>>> confirm the same behavior? >>>> >>>> >>>> >>>> >>> I don't think we tsted above around 1000 smtp connections, since postfix >>> is normlly limited in out case to 100 connections on each server. Our >>> machine has 8 GB of RAM. It was still running, though pretty slowly at >>> that point. >>> >>> Also, you didn't mention of you actually ran out of memory and/or swap >>> space. Some operating systems, like Solaris, also have a maximum user >>> process parameter set in the kernel -- maxuprc. Is policyd runnning as >>> "root" or a different user? You might just write a simple program that >>> spawns 1500 or so processes and see if you come up against a process >>> limit, memory limit, or other impediment. If nothing else, it sounds like >>> you may want to rate-limit the number of connections to postfix or >>> whatever your mail server is. If that doesn't keep your mail flow going, >>> you may need additional servers. >>> >>> --Tobias >>> >>> >>> >>> >>>> Ubuntu Dapper, uname -a >>>> Linux 2.6.15-26-amd64-server #1 SMP Thu Aug 3 03:32:26 UTC 2006 x86_64 >>>> GNU/Linux >>>> >>>> >>>> >>>> strace: >>>> >>>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 10 () >>>> select(1025, [9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 >>>> 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 >>>> 54 55 56 57 58 59 60 61 62 63 64 65 66 67 69 70 71 72 73 74 75 76 77 78 >>>> 79 80 81 82 83 84 85 416 417 473 478 482 484], [], NULL, NULL) = 12 (in >>>> [81 482]]) >>>> >>>> >>>> sysctl fs.file-max >>>> fs.file-max = 767080 >>>> >>>> >>>> ulimit -n >>>> 100000 >>>> >>>> >>>> Compile-time configs: >>>> >>>> /* CONFIGS */ >>>> #define PROJECT "policyd" >>>> #define VERSION "v1.82" >>>> >>>> /* Miscellaneous constants */ >>>> #define LISTENQ 1023 /* 2nd argument to listen() */ >>>> #define MAXLINE 1023 /* max text line length */ >>>> #define BUFFSIZE 8191 /* buffer size for reads and writes */ >>>> #define BUFSIZE 4095 >>>> #ifndef MAXFDS >>>> #define MAXFDS 20479 /* max file descriptors */ >>>> #endif >>>> >>>> >>>> CC := gcc >>>> CPPFLAGS := -O $(inc) >>>> CFLAGS := -g -W -Wall -DMAXFDS=20479 >>>> OS_NAME := $(shell uname | tr [A-Z] [a-z]) >>>> LDLIBS = $(lib) -lmysqlclient -lz >>>> >>>> >>>> Cami Sardinha ?????: >>>> >>>> >>>> >>>>> Artem Bokhan wrote: >>>>> >>>>> >>>>> >>>>> >>>>>> I will play with ulimit (was 1024 :( ) first and wait until bots will >>>>>> wake up again :) >>>>>> May be it's possible to increase the limits by policyd instead of >>>>>> shell?.. >>>>>> >>>>>> >>>>>> >>>>>> >>>>> There is no guarantee that Policyd's attempt to raise it will work. >>>>> >>>>> I do recommend you do some kernel tuning as well as raising the >>>>> ulimit's. I can see from your previous post that Policyd is in fact >>>>> built to use 4096 file descriptors, however, if you're limiting >>>>> Policyd to 1024 in ulimit, can you really blame Policyd for its >>>>> behavior? >>>>> >>>>> Cami >>>>> >>>>> ------------------------------------------------------------------------- >>>>> This SF.net email is sponsored by: Splunk Inc. >>>>> Still grepping through log files to find problems? Stop. >>>>> Now Search log events and configuration files using AJAX and a browser. >>>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ >>>>> _______________________________________________ >>>>> policyd-users mailing list >>>>> policyd-users@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/policyd-users >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>> ------------------------------------------------------------------------- >>> This SF.net email is sponsored by: Splunk Inc. >>> Still grepping through log files to find problems? Stop. >>> Now Search log events and configuration files using AJAX and a browser. >>> Download your FREE copy of Splunk now >> http://get.splunk.com/ >>> _______________________________________________ >>> policyd-users mailing list >>> policyd-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/policyd-users >>> >>> >>> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. >> Still grepping through log files to find problems? Stop. >> Now Search log events and configuration files using AJAX and a browser. >> Download your FREE copy of Splunk now >> http://get.splunk.com/ >> _______________________________________________ >> policyd-users mailing list >> policyd-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/policyd-users >> >> > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > policyd-users mailing list > policyd-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/policyd-users > ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ policyd-users mailing list policyd-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/policyd-users
