Need information about Apache and Module development.
Dear All, We are planning for an enhancement of our servers.We are looking out if we write Apache module and use Apache to support the below requirements. We would like to use Apache as a routing layer that accepts both structured documents (XML docs) also some other predefined data blocks in addition to key/value paris via http and then routes those transactions based on a key available in the request such that the required keys and sub keys are all routed to the same rules instance - this will require the original request to be broadcast to multiple instances of the rules engine ( for example, one instance may be tracking the IP, another instance may be tracking by GUID or Screenname, and yet another may be tracking by some Signature.) A rules execution engine that is able to accept the request, evaluate a set of ops defined rules and execute various responses. There are preference of using DROOLS fusion.Any functionality in Apache based on Rules ? What are the approaches need to be taken for dynamic load balancing.Like suppose I have 3 instances of Apache is running and due to some issue one of the instance goes down.I would expect the traffic should be balanced properly by the existing 2 instances. For load balancing apart from mod_proxy_balancer any other Apache modules can be worth looking into? As and alert broadcast engine that has the ability to distribute an events to multiple end sources. As a storage layer that allows for data persistence for long term tracking of keys and values. We have the target of good performance.Is it going to be good using Apache or any other webserver would be suggestion. Inputs would be very much appreciated. Many thanks in advance -A
Need information about Apache module development.
Dear All, We are planning for an enhancement of our servers.We are looking out if we write Apache module and use Apache to support the below requirements. We would like to use Apache as a routing layer that accepts both structured documents (XML docs) also some other predefined data blocks in addition to key/value paris via http and then routes those transactions based on a key available in the request such that the required keys and sub keys are all routed to the same rules instance - this will require the original request to be broadcast to multiple instances of the rules engine ( for example, one instance may be tracking the IP, another instance may be tracking by GUID or Screenname, and yet another may be tracking by some Signature.) A rules execution engine that is able to accept the request, evaluate a set of ops defined rules and execute various responses. There are preference of using DROOLS fusion.Any functionality in Apache based on Rules ? What are the approaches need to be taken for dynamic load balancing.Like suppose I have 3 instances of Apache is running and due to some issue one of the instance goes down.I would expect the traffic should be balanced properly by the existing 2 instances. For load balancing apart from mod_proxy_balancer any other Apache modules can be worth looking into? As and alert broadcast engine that has the ability to distribute an events to multiple end sources. As a storage layer that allows for data persistence for long term tracking of keys and values. We have the target of good performance.Is it going to be good using Apache or any other webserver would be suggestion. Inputs would be very much appreciated. Many thanks in advance -A
Re: Catching apache restart signal
Apache restart means,child process are killed by the process.So you have to write a program perhaps apache module would be the best.Catch the required signal and do the job.The module can be loaded from the httpd.conf. Writing a application putting it under usr/bin and putting the path in httpd.conf can also make you achieve this. Thanks -A On Thu, Jan 29, 2009 at 1:13 PM, Paras Fadte plf...@gmail.com wrote: Hi, How can one catch a signal indicating apache restart in an external program specified in httpd.conf file . Example would be rotatelogs utility or some other utility to which logs are piped . Thank you . -Paras
Re: Catching apache restart signal
surely.That would be the best. -A On Thu, Jan 29, 2009 at 3:27 PM, Paras Fadte plf...@gmail.com wrote: Can APR be used ? On Thu, Jan 29, 2009 at 2:40 PM, Arnab Ganguly agangul...@gmail.com wrote: Apache restart means,child process are killed by the process.So you have to write a program perhaps apache module would be the best.Catch the required signal and do the job.The module can be loaded from the httpd.conf. Writing a application putting it under usr/bin and putting the path in httpd.conf can also make you achieve this. Thanks -A On Thu, Jan 29, 2009 at 1:13 PM, Paras Fadte plf...@gmail.com wrote: Hi, How can one catch a signal indicating apache restart in an external program specified in httpd.conf file . Example would be rotatelogs utility or some other utility to which logs are piped . Thank you . -Paras
Query on ap_lingering_close
Hi All, I am getting lot of CLOSE_WAIT ,SYNC_RCV and TIME_WAIT state when I do a netstat and result it is Apache pause. What would be the ideal approach to solve the issue.I am planning for explicit call of ap_lingering_close after reading the client request. Also I see a #ifdef CORE_PRIVATE under httpd_connection.h where ap_lingering_close is defined and the macro #define CORE_PRIVATE is defined in mod_proxy.h.But I don't use mod_proxy so is it going to impact anything? For TIME_WAIT plan to reduce tcp_max_tw_buckets value.Is it going to be impacted? For SYNC_RCV Increasing net.ipv4.tcp_max_syn_backlog, decreasing tcp_synack_retries and enabling tcp_syncookies. If you have any suggestion then please let me know. Thanks in advance. -A
Re: Apache and CLOSE_WAIT state
Issue it that I am using Apache internal system calls like [ap_setup_client_block, ap_should_client_block , ap_get_client_block] for client communication and the socket descriptors are not available at this level, as per the apache doc ,when the Apache pool is destroyed the sockets are also going to closed.But seems to be not happening or may be I am doing something silly. How do I make sure the sockets are being closed once the client disconnects. Thanks in advance. Arnab On Wed, Sep 3, 2008 at 10:03 PM, Peter A. Friend [EMAIL PROTECTED] wrote: Arnab Ganguly wrote: Hi All, My Apache module hangs when I do a lsof -i:listening port output gives lots of CLOSE_WAIT. Initially the state comes out as ESTABLISHED but as the CLOSE_WAIT grows my server hangs. What would be procedure in order to prevent this. Apache Webserver version is 2.2.8 with MPM=worker and OS=Red-Hat Release 3.0 Any help would be very much appreciated. Thanks and regards Arnab How to fix it depends on what your module is attempting to do. It helps to understand what CLOSE_WAIT actually means, so grab your copy of Stevens... When a TCP socket receives a FIN (other end closed the socket) from it's peer the socket is then half closed. The FIN from the local socket to the peer won't be sent until the application issues the close(). Hence the name of the TCP state, CLOSE_WAIT, since TCP is waiting for the application to close its end of the connection. When CLOSE_WAIT piles up and doesn't go away it's usually an indication of a descriptor leak bug. Something is either preventing progress to occur in the HTTP session (we are stuck so never end up calling close), or some bug has been introduced that prevents the socket from being closed. There are a number of ways this can happen. Peter
Apache and CLOSE_WAIT state
Hi All, My Apache module hangs when I do a lsof -i:listening port output gives lots of CLOSE_WAIT. Initially the state comes out as ESTABLISHED but as the CLOSE_WAIT grows my server hangs. What would be procedure in order to prevent this. Apache Webserver version is 2.2.8 with MPM=worker and OS=Red-Hat Release 3.0 Any help would be very much appreciated. Thanks and regards Arnab
Issues with my Apache
Hi All, I am getting an issue with my Apache webserver.My configuration is MPM=worker,Apache version 2.2.8 and Red-Hat 3.0. My observation is under heavy load it pauses for sometime and then it starts processing.During the time of pause I see requests handled by apache is being returned with 500 response.I am not using rotate log module. Can it be a problem with my configuration or i am doing something wrong in my module? Any suggestions or help would be very much appreciated. Thanks and regards Arnab
TCP/IP handshake issue
Hi All, I am getting a TCP/IP handshake issue.I have a server component (In Linux) which acts as a router and it is an apache module.What ever data it receives it just passes to client like component (Windows component). Linux component listens to a particular port.Windows component does the initial handshaking fine.But after sometime under the heavy load what we observe is Server hangs in accept().. it waits for the client to connect and on the other hand client calls connect() and gets connection refused. I am wondering what could be the problem. Any help or suggestion would be very much appreciated. Thanks Arnab
Seems to be apache hanging, any idea on this
Hi All, I am using Apache 2.2 Worker in Red-Hat 3.0. Following are the behaviors observed in my access logs. 1) Sometime for a particular second there is no entry in the access log although the server was up and it was fed by HTTP request. 2) Time stamp in the access log entry sometimes come in irregular way, like the time of entry for 40th sec is before than the the entry happened in 30th sec. 3)When I do a tail in the access log , sometime my terminal hangs , means I don't get any entry in the access log but again after sometime I see entries.Even the file size also does not seems to be growing during this particular instance. Is Apache is hanging because of heavy traffic???What would be the work around for this? Any help would be very much appreciated. Thanks in advance. Regards -A
help on the signal handler off topic
Hi All, I have printf statement inside the infinite while loop.I want to write a alarm such that once the timer expires, it goes to the signal handler registered function and again it should come back to the while loop back.It seems not happening to me.Any help would be appreciated. Thanks Arnab
Apache log procedure
Hi All, Can you explain me in brief of how the apache logs gets written.Does it have any buffer mechanism while writing it into the file?I am sing Apache 2.2 with Worker model.When my webserver is overloaded i don't see my access logs file growing for at least 1 min and again later on it grows.Later when I check the entry in the access log I do see entry of the logs during the time when the logs file size was not growing. Thanks in advance. Regards -A
Re: Query on deletion of Request pool
Hi All,
Thanks a lot for such a detailed help.It simple awesome!.Will update you
about the behavior once I do the testing.
Regards
-A
On Sat, Mar 29, 2008 at 10:53 AM, Chris Kukuchka [EMAIL PROTECTED]
wrote:
From: Arnab Ganguly [EMAIL PROTECTED]
It up to the OS to mark the freed areas as free or use it as a
filesystem buffer or whatever buffer, as long as the memory isn't
needed
by applications.
Thanks for the update.Actually when I do top -p on the process id I do
see
memory consumed by Apache is very less but over the time when I do free
-m
the RAM gets reduced.
As Robert indicated, a running Linux system will attempt to make use of
all physical memory. This is because it is undesirable to leave memory
unused (wasted) when it can be put to good use for things such as disk
cache.
Google linux memory management for more information.
I wonder can be this case happen free -m is 0 and the
machine will crash or something
Typically, you will see free memory get close to 0 and stay there. It
will usually not go past that point unless you have your machine overloaded
in some fashion. In most cases, even if free memory dips to 0, you will
still have your swap memory available. At that point, system performance
will start to degrade, but you will still be running.
I was thinking may be the Apache was eating up the RAM
To see how much physical memory (in kilobytes) is being used by Apache,
use this command sequence:
ps -e -o rss,comm | fgrep httpd | awk '{sum+=$1} END {print sum}'
It is not unusual to see memory go up as an Apache process matures. This
is especially true if you are running modules like PHP or Perl which have
the potential to load many helper modules. A quick review of a handful of
running machines available to me show 2-18MB per process is not unusual. Of
those, PHP users are typically 10-12MB higher than non-PHP users. YMMV.
Regards,
Chris Kukuchka
Sequoia Group, Inc.
Re: Query on deletion of Request pool
It sounds like you're tracking the wrong numbers if memory isn't free when the application terminates. Not clear to me.Can you please add some more lights to it. Thanks A On Thu, Mar 27, 2008 at 7:54 PM, Eric Covener [EMAIL PROTECTED] wrote: On Wed, Mar 26, 2008 at 10:51 PM, Arnab Ganguly [EMAIL PROTECTED] wrote: In the particular box only Apache is running no other application process is running.Also one more observation was when the Apache is stopped the free -m doesn't result to the original memory restore.We have to reboot the box to restore the original RAM.Should I try using smaller value for MaxMemFree? Thanks and regards -A -- Eric Covener [EMAIL PROTECTED]
Query on deletion of Request pool
Hi All, I am getting a serious memory issue with my Apache webserver. Initially I was allocating buffer from by using apr_palloc from the request pool assuming the allocated memory is going to be released but not sure what is the problem the memory grows infinitely. I then tried with own malloc and then added a clean up function on apr_pool_cleanup_run.Which on debugging showed the free is being called for the allocated memory but still the behavior is same.On each request there is a increase in memory. Is there any way to release explicitly the memory of the request pool?I tried with apr_pool_cleanup_register and added a clean up function internally I was calling apr_pool_destroy(request_rec-pool).For the first request it worked correctly but on the later there was a crash and restart on the Apache process. It seems I may be doing something silly.Any help would be very much appreciated.BTW my webserver is heavily loaded and it is MPM=Worker Apache version 2.2.8 and OS is Red-Hat 3.0. What are the other possibilities where memory leak can looked into. Looking forward for response. Thanks -A
Need help on memory related issues
Hi All, I am getting a memory leak issue in Apache 2.2.8 on Red-Hat 5.0. I tried out the following options 1) MaxRequestPerChild settings to non zero 2)I was allocating memory by using apr_palloc from the request pool but later I replaced to normal malloc and added a clean handler in apr_pool_cleanup_run. 3)Tried out the MaxMemFree to some lower value so that apache process consumes less memory, not sure as it is a correct approach or not? Above changes didn't resulted any improvements.For each request there is a substantial increase in RSS value and this memory is never returned back to RAM it seems.I may be doing something major mistake in my module any help would be highly appreciated. Just curious any issue with the ap_get_client_block() function.I am using this function many times so can it be a problem and what is the work around for this. Thanks in advance. -A
Re: Need help on memory related issues
Hi , Thanks for the email.Yes the same behavior is observerd without my module.Also the webserver is handling heavy traffic and it remains busy through out. Just to update you on the statistics it gets 1000 hits per minute. How do I give time to stabilize Apache as it is always heavily loaded and server status pages show idle worker is 0.Let me know on this. Regards -A On Sat, Mar 15, 2008 at 1:04 AM, Eric Covener [EMAIL PROTECTED] wrote: On Thu, Mar 13, 2008 at 9:15 PM, Arnab Ganguly [EMAIL PROTECTED] wrote: For each request there is a substantial increase in RSS value and this memory is never returned back to RAM it seems.I may be doing something major mistake in my module any help would be highly appreciated. Does it happen without your module? If you're using a threaded MPM, are you giving Apache enough time to stabilize? RSS should level out after each thread has had a chance to run your hungriest request. -- Eric Covener [EMAIL PROTECTED]
Query on MaxClient and ListenBackLog
Hi All, I am using MPM= worker and Apache version is 2.2 on Red-Hat 3.0. I am getting lots of connection timeout from the client request as due the busy server.I have kept the value of MaxCLient 128 and ListenBackLog to default 511. So in order to reduce the connection timeouts which one makes more sense to increase like MaxClient or ListenBackLog. Any help would be appreciated. Thanks and regards -A
Analyzing access and error logs
Hi All, Is there any free tool available that analyzes Apache error and access logs? Thanks in advance. -A
Re: [EMAIL PROTECTED] Invalid URI in request
Hi Krushna, Thanks for the quick update.It comes in my error_log.I am using Apache 2.2Worker in Red-hat 3. Regards -A On Dec 4, 2007 6:04 PM, Rath, Krushna [EMAIL PROTECTED] wrote: Can u provide bit more info…..where r u getting Invalid URI in request.?in ethereal? Can u provide the trace and the log files? Regards, ~Krushna -- *From:* Arnab Ganguly [mailto:[EMAIL PROTECTED] *Sent:* Tuesday, December 04, 2007 5:55 PM *To:* [EMAIL PROTECTED]; dev@httpd.apache.org *Subject:* [EMAIL PROTECTED] Invalid URI in request Hi All, I am getting lot of Invalid URI in request.How to resolve this.I have checked the DocumentRoot and Directory they are all pointing to same path. Any help would be highly appreciated. Thanks and regards -A
Invalid URI in request
Hi All, I am getting lot of Invalid URI in request.How to resolve this.I have checked the DocumentRoot and Directory they are all pointing to same path. Any help would be highly appreciated. Thanks and regards -A
Re: Segmentation fault( SSL enable Apache 2.2.6(64 bit)
Try loading only the required modules.Are you aware of the content length of
your request, like if it the contents are dynamic keep the KeepAlive Off in
your conf file.
Thanks
-A
On 10/26/07, Renu Tiwari [EMAIL PROTECTED] wrote:
Hi,
From googling I got the solution to delete -O (caps o) from Makefile of
openssl.
I did it and this time make test got passed.
But still I am getting segmentation fault after starting Apache server.
Please suggest.
Thanks.
-Original Message-
From: Niklas Edmundsson [mailto:[EMAIL PROTECTED]
Sent: Thursday, October 25, 2007 4:23 PM
To: 'dev@httpd.apache.org'
Subject: RE: Segmentation fault( SSL enable Apache 2.2.6(64 bit)
On Thu, 25 Oct 2007, Renu Tiwari wrote:
No, when I tried doing make test, it failed.
But when I tried to run make install and make, there I didn't
get any error.
make install doesn't do the test.
If your openssl doesn't pass make test, then it's broken. Fix that
first.
Also I have copied the same SSL-enabled Apache webserver on AIX
5.3(64 bit) there it is working perfectly fine.
Does this depend on the kernel also. As AIX5.3 is 64-bit kernel and
AIX 5.2 is 32-bit kernel m/c.
But our application is running as 64 bit application.
Shouldn't matter, could be that you're hitting some bug that's been
fixed. You might want to check your C runtime patch levels, and other
patches too for that matter.
-Original Message-
From: Niklas Edmundsson [mailto:[EMAIL PROTECTED]
Sent: Thursday, October 25, 2007 2:36 PM
To: 'dev@httpd.apache.org'
Subject: Re: Segmentation fault( SSL enable Apache 2.2.6(64 bit)
On Thu, 25 Oct 2007, Renu Tiwari wrote:
Hi,
We have configured Apache 2.2.6(64 bit) with openssl-0.9.8g on AIX5.2(64
bit).
Build openssl source after setting BUILD_MODE=64.
The issue is, when we start the Apache web server(./apachectl start),
we are getting segmentation fault in error_log.
This issue is coming only when openssl is coming into the picture.
What cud be the possible reason? Please reply.
Did your openssl 64bit build pass make test (or make check, whatever
it's called)?
/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se |
[EMAIL PROTECTED]
---
If I wanted your opinion, I would have given you one
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
CAUTION - Disclaimer *
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
solely for the use of the addressee(s). If you are not the intended
recipient, please notify the sender by e-mail and delete the original
message. Further, you are not to copy, disclose, or distribute this e-mail
or its contents to any other person and any such actions are unlawful. This
e-mail may contain viruses. Infosys has taken every reasonable precaution to
minimize this risk, but is not liable for any damage you may sustain as a
result of any virus in this e-mail. You should carry out your own virus
checks before opening the e-mail or attachment. Infosys reserves the right
to monitor and review the content of all messages sent to or from this
e-mail address. Messages sent to or from this e-mail address may be stored
on the Infosys e-mail system.
***INFOSYS End of Disclaimer INFOSYS***
/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se |
[EMAIL PROTECTED]
---
Taken as a whole, the universe is absurd
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
