Re: [users@httpd] Segmentation Fault - too many proxy balancers

2014-07-26 Thread Jeff Trawick 
On Fri, Jul 25, 2014 at 2:00 PM, Paul Beckett 
wrote:

> My apache server has started segmentation faulting all the time (seems to
> log a segmentation fault every few requests to the apache error log):
>
> [Fri Jul 25 06:25:42.046752 2014] [core:notice] [pid 11226:tid
> 140006078953216] AH00052: child pid 11715 exit signal Segmentation fault
> (11)
>

Can you post a backtrace from the crash?


>
> This appears to be due to the number of proxy balancers I have configured
> (problem isn't related to any one specific proxy balancer, adding /
> removing any of the proxy balancers causes the problem to
> appear/disappear). I'm using Apache HTTPD as a reverse proxy for a lot of
> load-balanced (by apache httpd) application servers. My googling so far
> hasn't found any specific limit on the number of proxy, or how I can
> increase this.
>
> I am running Apache HTTPD 2.4.9, built from source on RHEL6.
>
> I would be very grateful if anyone can shed more light on this, and
> assuming I'm right about a limit: point my in the right direction as to how
> I can increase this.
>
> Thanks,
> Paul
>



-- 
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/

AW: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

2014-07-26 Thread Ulrich.Herbst 
Hi,

Is it possible that mod_deflate is configured, perhaps unexpectedly, to 
decompress the response from the backend via an input filter?



ð  Yes, for sure. mod_deflate is enabled and we compress the (uncompressed) 
response from the backend before sending to the clients.

See 
http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
  (new with 2.4.10)


ð  So, you think, that our compressed body is too large sometimes, and we got 
“no space” because of this ?

Uli


Von: Jeff Trawick [mailto:traw...@gmail.com]
Gesendet: Freitag, 25. Juli 2014 17:10
An: users@httpd.apache.org
Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] 
(11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: 
error reading response

On Fri, Jul 25, 2014 at 10:49 AM, Tom Evans 
mailto:tevans...@googlemail.com>> wrote:
On Fri, Jul 25, 2014 at 11:37 AM,  
mailto:ulrich.her...@t-systems.com>> wrote:
> Hi,
>
> we have apaches as reverse proxies for some tomcat and wso2-application
> servers.
>
> We get this error message on a high-load-webserver:
>
> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily
> unavailable: [client 172.31.0.142:21815] AH01110: 
> error reading response
>
> And we have no clue, which resource is unavailable and what to do against
> it.
>
> Apache-2.4.10 / apr-1.5.1
>
> Ulimit of apache-user:
> core file size  (blocks, -c) 0
> data seg size   (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size   (blocks, -f) unlimited
> pending signals (-i) 4
> max locked memory   (kbytes, -l) 655360
> max memory size (kbytes, -m) unlimited
> open files  (-n) 655360
> pipe size(512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority  (-r) 0
> stack size  (kbytes, -s) unlimited
> cpu time   (seconds, -t) unlimited
> max user processes  (-u) 655360
> virtual memory  (kbytes, -v) unlimited
> file locks  (-x) unlimited
>
> $ cat /proc/sys/kernel/threads-max
> 2065067
>
>
> So, I don’t see any visible shortage of anything.
>
>
> Any ideas how to find the reason for this error ?
>
> Uli
>
Running out of shm?

How are you proxying to tomcat (http/ajp/jk2)?

The message comes from mod_proxy_http, so "http" it is.

Looking at how to get the ENOSPC error in that particular context, I see that 
several pieces of code can return ENOSPC based on exceeding some configured 
limit, but most don't apply to the path where that message is issued.

Is it possible that mod_deflate is configured, perhaps unexpectedly, to 
decompress the response from the backend via an input filter?

See 
http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
  (new with 2.4.10)  That is part of a fix to vulnerability CVE-2014-0118.



Cheers

Tom

-
To unsubscribe, e-mail: 
users-unsubscr...@httpd.apache.org
For additional commands, e-mail: 
users-h...@httpd.apache.org



--
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/

AW: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

2014-07-26 Thread Ulrich.Herbst 
Running out of shm?

=> Yes, possible. But: How to check this ?

Ulimit -l and ulimit -u show nothing interesting (that is: The usage (ulimit 
-u) is much less than the limits from ulimit -l)

Uli


T-Systems International GmbH
Production / CSS / GCU Automotive & Finance
Ulrich Herbst
Fachbereich Automotive / Daimler AS Open Systems
Head of Application Services Central
Bannwaldallee 42, D-76185 Karlsruhe
+49 721 990-2342 (Tel.)
+49 151 16 20 73 13 (Mobil)
E-Mail: ulrich.her...@t-systems.com
Internet: http://www.t-systems.de

Die gesetzlichen Pflichtangaben finden Sie unter:
http://www.t-systems.de/pflichtangaben


-Ursprüngliche Nachricht-
Von: Tom Evans [mailto:tevans...@googlemail.com] 
Gesendet: Freitag, 25. Juli 2014 16:50
An: users@httpd.apache.org
Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] 
(11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: 
error reading response

On Fri, Jul 25, 2014 at 11:37 AM,   wrote:
> Hi,
>
> we have apaches as reverse proxies for some tomcat and 
> wso2-application servers.
>
> We get this error message on a high-load-webserver:
>
> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource 
> temporarily
> unavailable: [client 172.31.0.142:21815] AH01110: error reading 
> response
>
> And we have no clue, which resource is unavailable and what to do 
> against it.
>
> Apache-2.4.10 / apr-1.5.1
>
> Ulimit of apache-user:
> core file size  (blocks, -c) 0
> data seg size   (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size   (blocks, -f) unlimited
> pending signals (-i) 4
> max locked memory   (kbytes, -l) 655360
> max memory size (kbytes, -m) unlimited
> open files  (-n) 655360
> pipe size(512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority  (-r) 0
> stack size  (kbytes, -s) unlimited
> cpu time   (seconds, -t) unlimited
> max user processes  (-u) 655360
> virtual memory  (kbytes, -v) unlimited
> file locks  (-x) unlimited
>
> $ cat /proc/sys/kernel/threads-max
> 2065067
>
>
> So, I don’t see any visible shortage of anything.
>
>
> Any ideas how to find the reason for this error ?
>
> Uli
>

Running out of shm?

How are you proxying to tomcat (http/ajp/jk2)?

Cheers

Tom

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

AW: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

2014-07-26 Thread Ulrich.Herbst 
Or inotify perhaps:

http://unix.stackexchange.com/questions/65278/httpd-no-space-left-on-device

=> Not sure about this: We had never issues with tail...

But nevertheless: I changed these values and we just wait for new errors.

Uli

-Ursprüngliche Nachricht-
Von: Tom Evans [mailto:tevans...@googlemail.com] 
Gesendet: Freitag, 25. Juli 2014 16:51
An: users@httpd.apache.org
Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] 
(11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: 
error reading response

On Fri, Jul 25, 2014 at 3:49 PM, Tom Evans  wrote:
> On Fri, Jul 25, 2014 at 11:37 AM,   wrote:
>> Hi,
>>
>> we have apaches as reverse proxies for some tomcat and 
>> wso2-application servers.
>>
>> We get this error message on a high-load-webserver:
>>
>> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource 
>> temporarily
>> unavailable: [client 172.31.0.142:21815] AH01110: error reading 
>> response
>>
>> And we have no clue, which resource is unavailable and what to do 
>> against it.
>>
>> Apache-2.4.10 / apr-1.5.1
>>
>> Ulimit of apache-user:
>> core file size  (blocks, -c) 0
>> data seg size   (kbytes, -d) unlimited
>> scheduling priority (-e) 0
>> file size   (blocks, -f) unlimited
>> pending signals (-i) 4
>> max locked memory   (kbytes, -l) 655360
>> max memory size (kbytes, -m) unlimited
>> open files  (-n) 655360
>> pipe size(512 bytes, -p) 8
>> POSIX message queues (bytes, -q) 819200
>> real-time priority  (-r) 0
>> stack size  (kbytes, -s) unlimited
>> cpu time   (seconds, -t) unlimited
>> max user processes  (-u) 655360
>> virtual memory  (kbytes, -v) unlimited
>> file locks  (-x) unlimited
>>
>> $ cat /proc/sys/kernel/threads-max
>> 2065067
>>
>>
>> So, I don’t see any visible shortage of anything.
>>
>>
>> Any ideas how to find the reason for this error ?
>>
>> Uli
>>
>
> Running out of shm?
>
> How are you proxying to tomcat (http/ajp/jk2)?
>

Or inotify perhaps:

http://unix.stackexchange.com/questions/65278/httpd-no-space-left-on-device

Cheers

Tom

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org