Re: Fwd: Re: [qmailtoaster] qq soft reject errors on high load
eric i did not yet simulate it using your script these are errors in my live production server during peak hours today and there were mails with and without attachments. the max processes i got was 47 only. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Mon, 4 Sep 2017 11:19:35 -0600 Subject: Did you hit it with email containing attachments as well? On 9/4/2017 10:58 AM, Rajesh M wrote: > eric > > the max i saw was 47 connections using the shell script you gave me. > > there were a few errors related to qq soft reject but they got delivered. > > rajesh > > > > - Original Message - > From: Eric Broch [mailto:ebr...@whitehorsetc.com] > To: qmailtoaster-list@qmailtoaster.com > Sent: Sat, 2 Sep 2017 12:35:14 -0600 > Subject: > > Rajesh, > > I'm still not sure the process limit is being reached. > > I'm testing a COS6/QMT server now and haven't been able to bring about > 'qq soft reject' or failure of any sort. I'm hitting it with email using > a delivery script from two servers with attachment size of 320KB. > > try the script below (psmem) to monitor: > > > > #!/bin/bash > ps -C $1 -O rss | gawk '{ count ++; sum += $2 }; END {count --; print > "Number of processes =",count; print "Memory usage per process > =",sum/1024/count, "MB"; print "Total memory usage =", sum/1024, "MB" ;};' > > > > # chmod 755 psmem > > # watch -n 1 ./psmem simscan > > > Not sure I should change anything until we figure out what's going on. > > Eric > > > > On 9/2/2017 12:03 PM, Rajesh M wrote: >> eric >> >> i could not get the bash script working with either of the two options. >> still gives error >>line 23: `return-limits': not a valid identifier. >> >> >> >> can the limits issue in simscan be fixed ? >> >> thanks, >> rajesh >> >> - Original Message - >> From: Eric Broch [mailto:ebr...@whitehorsetc.com] >> To:qmailtoaster-list@qmailtoaster.com >> Sent: Fri, 1 Sep 2017 23:46:15 -0600 >> Subject: >> >> Rajesh, >> >> It has something to do with #!/bin/bash or #!/bin/sh. >> >> At top of script, when I use #!/bin/sh I get the same error you do, but >> when I use #!/bin/bash script works fine. >> >> >> Eric >> >> >> On 9/1/2017 11:07 PM, Rajesh M wrote: >>> eric >>> >>> 1) is their any resolution for this issue ? >>> >>> 2) if i run the script it throws error as such >>> . >>> /limits.sh: line 23: `return-limits': not a valid identifier >>> >>> #!/bin/bash >>> if [ "$#" -ne "1" ]; then >>> echo "" >>> echo -e "\033[01;32mLimit checker\033[00m" >>> echo -e "\033[01;37mUsage:\033[01;33m $0 process_name\033[00m" >>> echo "" >>> exit 0 >>> fi >>> >>> return-limits(){ >>> for process in $@; do >>> process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2` >>> >>> if [ -z $@ ]; then >>> echo "[no $process running]" >>> else >>> for pid in $process_pids; do >>>echo "[$process #$pid -- limits]" >>>cat /proc/$pid/limits >>> done >>> fi >>> done >>> } >>> return-limits $1 >>> >>> rajesh >>> >>> >>> >>> - Original Message - >>> From: Eric Broch [mailto:ebr...@whitehorsetc.com] >>> To:qmailtoaster-list@qmailtoaster.com >>> Sent: Fri, 1 Sep 2017 17:28:40 -0600 >>> Subject: >>> >>> I sent myself a large file so that I could examine simscan with the >>> below script (limits.sh). Here's the output: >>> >>> [root@pet105 simscan-1.4.0]# ./limits simscan >>> [simscan #31535 -- limits] >>> Limit   >>>    >>>    >>>    >>>    >>>    >>>   Soft Limit >>>    >>>    >>>    Hard >>> Limit   >>>    >>>    >>>  Units >>> Max cpu time   >>>    >>>    >>>    >>>  unlimited unlimited >>>    >>>    >>>  ÂÂÂÂ
Re: Fwd: Re: [qmailtoaster] qq soft reject errors on high load
eric i got 3 types of errors like earlier. failure to fork, ripmime and /var/qmail/simscan/1simscan: check_spam had an error ret: -1 what exactly would be /var/qmail/simscan/1simscan ... there is no such file do want me to send you a few of my log files directly to your email id ? rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Mon, 4 Sep 2017 11:19:04 -0600 Subject: What kind of qq soft reject errors were there? Failure to fork or ripmime? On 9/4/2017 10:58 AM, Rajesh M wrote: > eric > > the max i saw was 47 connections using the shell script you gave me. > > there were a few errors related to qq soft reject but they got delivered. > > rajesh > > > > - Original Message - > From: Eric Broch [mailto:ebr...@whitehorsetc.com] > To: qmailtoaster-list@qmailtoaster.com > Sent: Sat, 2 Sep 2017 12:35:14 -0600 > Subject: > > Rajesh, > > I'm still not sure the process limit is being reached. > > I'm testing a COS6/QMT server now and haven't been able to bring about > 'qq soft reject' or failure of any sort. I'm hitting it with email using > a delivery script from two servers with attachment size of 320KB. > > try the script below (psmem) to monitor: > > > > #!/bin/bash > ps -C $1 -O rss | gawk '{ count ++; sum += $2 }; END {count --; print > "Number of processes =",count; print "Memory usage per process > =",sum/1024/count, "MB"; print "Total memory usage =", sum/1024, "MB" ;};' > > > > # chmod 755 psmem > > # watch -n 1 ./psmem simscan > > > Not sure I should change anything until we figure out what's going on. > > Eric > > > > On 9/2/2017 12:03 PM, Rajesh M wrote: >> eric >> >> i could not get the bash script working with either of the two options. >> still gives error >>line 23: `return-limits': not a valid identifier. >> >> >> >> can the limits issue in simscan be fixed ? >> >> thanks, >> rajesh >> >> - Original Message - >> From: Eric Broch [mailto:ebr...@whitehorsetc.com] >> To:qmailtoaster-list@qmailtoaster.com >> Sent: Fri, 1 Sep 2017 23:46:15 -0600 >> Subject: >> >> Rajesh, >> >> It has something to do with #!/bin/bash or #!/bin/sh. >> >> At top of script, when I use #!/bin/sh I get the same error you do, but >> when I use #!/bin/bash script works fine. >> >> >> Eric >> >> >> On 9/1/2017 11:07 PM, Rajesh M wrote: >>> eric >>> >>> 1) is their any resolution for this issue ? >>> >>> 2) if i run the script it throws error as such >>> . >>> /limits.sh: line 23: `return-limits': not a valid identifier >>> >>> #!/bin/bash >>> if [ "$#" -ne "1" ]; then >>> echo "" >>> echo -e "\033[01;32mLimit checker\033[00m" >>> echo -e "\033[01;37mUsage:\033[01;33m $0 process_name\033[00m" >>> echo "" >>> exit 0 >>> fi >>> >>> return-limits(){ >>> for process in $@; do >>> process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2` >>> >>> if [ -z $@ ]; then >>> echo "[no $process running]" >>> else >>> for pid in $process_pids; do >>>echo "[$process #$pid -- limits]" >>>cat /proc/$pid/limits >>> done >>> fi >>> done >>> } >>> return-limits $1 >>> >>> rajesh >>> >>> >>> >>> - Original Message - >>> From: Eric Broch [mailto:ebr...@whitehorsetc.com] >>> To:qmailtoaster-list@qmailtoaster.com >>> Sent: Fri, 1 Sep 2017 17:28:40 -0600 >>> Subject: >>> >>> I sent myself a large file so that I could examine simscan with the >>> below script (limits.sh). Here's the output: >>> >>> [root@pet105 simscan-1.4.0]# ./limits simscan >>> [simscan #31535 -- limits] >>> Limit   >>>    >>>    >>>    >>>    >>>    >>>   Soft Limit >>>    >>>    >>>    Hard >>> Limit   >>>    >>>    >>>  Units >>> Max cpu time   >>>    >>>    >>>    >>>  unlimited unlimited >>>    >>> ÂÂÂÂ
RE: Fwd: Re: [qmailtoaster] qq soft reject errors on high load
eric the max i saw was 47 connections using the shell script you gave me. there were a few errors related to qq soft reject but they got delivered. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Sat, 2 Sep 2017 12:35:14 -0600 Subject: Rajesh, I'm still not sure the process limit is being reached. I'm testing a COS6/QMT server now and haven't been able to bring about 'qq soft reject' or failure of any sort. I'm hitting it with email using a delivery script from two servers with attachment size of 320KB. try the script below (psmem) to monitor: #!/bin/bash ps -C $1 -O rss | gawk '{ count ++; sum += $2 }; END {count --; print "Number of processes =",count; print "Memory usage per process =",sum/1024/count, "MB"; print "Total memory usage =", sum/1024, "MB" ;};' # chmod 755 psmem # watch -n 1 ./psmem simscan Not sure I should change anything until we figure out what's going on. Eric On 9/2/2017 12:03 PM, Rajesh M wrote: > eric > > i could not get the bash script working with either of the two options. still > gives error > line 23: `return-limits': not a valid identifier. > > > > can the limits issue in simscan be fixed ? > > thanks, > rajesh > > - Original Message - > From: Eric Broch [mailto:ebr...@whitehorsetc.com] > To:qmailtoaster-list@qmailtoaster.com > Sent: Fri, 1 Sep 2017 23:46:15 -0600 > Subject: > > Rajesh, > > It has something to do with #!/bin/bash or #!/bin/sh. > > At top of script, when I use #!/bin/sh I get the same error you do, but > when I use #!/bin/bash script works fine. > > > Eric > > > On 9/1/2017 11:07 PM, Rajesh M wrote: >> eric >> >> 1) is their any resolution for this issue ? >> >> 2) if i run the script it throws error as such >> . >> /limits.sh: line 23: `return-limits': not a valid identifier >> >> #!/bin/bash >> if [ "$#" -ne "1" ]; then >> echo "" >> echo -e "\033[01;32mLimit checker\033[00m" >> echo -e "\033[01;37mUsage:\033[01;33m $0 process_name\033[00m" >> echo "" >> exit 0 >> fi >> >> return-limits(){ >> for process in $@; do >> process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2` >> >> if [ -z $@ ]; then >>echo "[no $process running]" >> else >>for pid in $process_pids; do >> echo "[$process #$pid -- limits]" >> cat /proc/$pid/limits >> done >> fi >> done >> } >> return-limits $1 >> >> rajesh >> >> >> >> - Original Message - >> From: Eric Broch [mailto:ebr...@whitehorsetc.com] >> To:qmailtoaster-list@qmailtoaster.com >> Sent: Fri, 1 Sep 2017 17:28:40 -0600 >> Subject: >> >> I sent myself a large file so that I could examine simscan with the >> below script (limits.sh). Here's the output: >> >> [root@pet105 simscan-1.4.0]# ./limits simscan >> [simscan #31535 -- limits] >> Limit        >>         >>     Soft Limit   >>        Hard >> Limit        >>   Units >> Max cpu time       >>       unlimited >> unlimited       >>     seconds >> Max file size       >>      unlimited unlimited >>         >>   bytes >> Max data size       >>      unlimited unlimited >>         >>   bytes >> Max stack size 10485760 unlimited    >>        bytes >> Max core file size      >>  0 unlimited      >>      bytes >> Max resident set      >>    unlimited unlimited   >>         >> bytes >> Max processes 1024 31121     >>         >>   processes >> Max open files       >> ÂÂÂ
RE: Fwd: Re: [qmailtoaster] qq soft reject errors on high load
eric thanks for the info. the script you sent works the server processes roughly around 8 emails per day during week days and around 80 percent of it comes during around 12 hours. So during peak hours it is around 8000 emails per hour. i will test this out monday morning peak hours and revert. thanks rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Sat, 2 Sep 2017 12:35:14 -0600 Subject: Rajesh, I'm still not sure the process limit is being reached. I'm testing a COS6/QMT server now and haven't been able to bring about 'qq soft reject' or failure of any sort. I'm hitting it with email using a delivery script from two servers with attachment size of 320KB. try the script below (psmem) to monitor: #!/bin/bash ps -C $1 -O rss | gawk '{ count ++; sum += $2 }; END {count --; print "Number of processes =",count; print "Memory usage per process =",sum/1024/count, "MB"; print "Total memory usage =", sum/1024, "MB" ;};' # chmod 755 psmem # watch -n 1 ./psmem simscan Not sure I should change anything until we figure out what's going on. Eric On 9/2/2017 12:03 PM, Rajesh M wrote: > eric > > i could not get the bash script working with either of the two options. still > gives error > line 23: `return-limits': not a valid identifier. > > > > can the limits issue in simscan be fixed ? > > thanks, > rajesh > > - Original Message - > From: Eric Broch [mailto:ebr...@whitehorsetc.com] > To:qmailtoaster-list@qmailtoaster.com > Sent: Fri, 1 Sep 2017 23:46:15 -0600 > Subject: > > Rajesh, > > It has something to do with #!/bin/bash or #!/bin/sh. > > At top of script, when I use #!/bin/sh I get the same error you do, but > when I use #!/bin/bash script works fine. > > > Eric > > > On 9/1/2017 11:07 PM, Rajesh M wrote: >> eric >> >> 1) is their any resolution for this issue ? >> >> 2) if i run the script it throws error as such >> . >> /limits.sh: line 23: `return-limits': not a valid identifier >> >> #!/bin/bash >> if [ "$#" -ne "1" ]; then >> echo "" >> echo -e "\033[01;32mLimit checker\033[00m" >> echo -e "\033[01;37mUsage:\033[01;33m $0 process_name\033[00m" >> echo "" >> exit 0 >> fi >> >> return-limits(){ >> for process in $@; do >> process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2` >> >> if [ -z $@ ]; then >>echo "[no $process running]" >> else >>for pid in $process_pids; do >> echo "[$process #$pid -- limits]" >> cat /proc/$pid/limits >> done >> fi >> done >> } >> return-limits $1 >> >> rajesh >> >> >> >> - Original Message - >> From: Eric Broch [mailto:ebr...@whitehorsetc.com] >> To:qmailtoaster-list@qmailtoaster.com >> Sent: Fri, 1 Sep 2017 17:28:40 -0600 >> Subject: >> >> I sent myself a large file so that I could examine simscan with the >> below script (limits.sh). Here's the output: >> >> [root@pet105 simscan-1.4.0]# ./limits simscan >> [simscan #31535 -- limits] >> Limit        >>         >>     Soft Limit   >>        Hard >> Limit        >>   Units >> Max cpu time       >>       unlimited >> unlimited       >>     seconds >> Max file size       >>      unlimited unlimited >>         >>   bytes >> Max data size       >>      unlimited unlimited >>         >>   bytes >> Max stack size 10485760 unlimited    >>        bytes >> Max core file size      >>  0 unlimited      >>      bytes >> Max resident set      >>    unlimited unlimited   >>         >> bytes >> Max processes 1024 31121     >>    ÂÂÂ