Axton, our firewall is a Cisco ASA 5520. Ken =============== Your Previous related msg:
This "can" be normal behavior for a firewall. It all depends on how you allow packets to create a state or if you use state at all. You can create a rule that is stateless so that all packets from host X to host Y are allowed. You should explore creating the rule to do a quick pass of the packets and not use state for these connections. Basically, the rules would look like this: pass in quick on $if_1 inet proto tcp from $mt_host_1 to $ar_host_1 pass out quick on $if_2 inet proto tcp from $ar_host_1 to $mt_host_1 Instead of a state based or policy based ruleset. You can also change the type of packet on the firewall so that syn/syn ack packets are not the only type of packet that can insert a state into the state table. This is a little funky and is probably not the best way to do this, but it can be done this way too. Axton Grams -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arsl...@arslist.org] On Behalf Of Axton Sent: Wednesday, November 18, 2009 5:09 PM To: arslist@ARSLIST.ORG Subject: Re: Firewall TCP Timeouts Midtier timeout has to do with the user session timeout. Once arserver is started there is a persistent socket connection to the arserver. Can I ask what type of firewall this is? Axton The opinions, statements, and/or suggested courses of action expressed in this E-mail do not necessarily reflect those of BMC Software, Inc. My voluntary participation in this forum is not intended to convey a role as a spokesperson, liaison or public relations representative for BMC Software, Inc. On Wed, Nov 18, 2009 at 4:05 PM, Grooms, Frederick W <frederick.w.gro...@xo.com> wrote: > Since you are using MidTier I would suggest you do the following... > > Add a specified port for your AR Server to listen on (The AR Server runs just > fine with portmapper and a specified port). > Set the MidTier to use the specified port. > > Also set the MidTier Session TimeOut value to be less than your firewall time > out setting. This way Mid-Tier will drop old connections before the firewall > does. You should be able to set the MIdTier TimeOut from the configuration > pages on your MidTier server. > > Fred > > > -----Original Message----- > From: Action Request System discussion list(ARSList) > [mailto:arsl...@arslist.org] On Behalf Of Leihkauff, Kenneth > Sent: Wednesday, November 18, 2009 3:51 PM > To: arslist@ARSLIST.ORG > Subject: Re: Firewall TCP Timeouts > > Thanks, Conny. Are you saying the libkeepalive was implemented on the > ARserver -- not the MidTier linux server? Our midtier and arserver are linux > with a firewall in between. > > Unfortunately, the firewall is not an application level firewall so the > firewall rule changes Axton proposed cannot be implemented according to our > network engineer. > > Ken > > -----Original Message----- > From: Action Request System discussion list(ARSList) > [mailto:arsl...@arslist.org] On Behalf Of Conny Martin > Sent: Wednesday, November 18, 2009 3:30 PM > To: arslist@ARSLIST.ORG > Subject: AW: Firewall TCP Timeouts > > We had the same issue. We solved this by using libkeepalive > (http://libkeepalive.sourceforge.net/) on the arserver machine. But it would > work only if you are using Linux. > > HTH > > Kind Regards Conny > > -----Ursprüngliche Nachricht----- > Von: Action Request System discussion list(ARSList) > [mailto:arsl...@arslist.org] Im Auftrag von Leihkauff, Kenneth > Gesendet: Mittwoch, 18. November 2009 18:18 > An: arslist@ARSLIST.ORG > Betreff: Re: Firewall TCP Timeouts > > Thanks for your time, Axton. I spoke with our network engineer and he > studied the firewall logs and sees quit a few denials and some resets, but is > confident things should be set up correctly on the firewall. He only sees > these denials/resets for the Midtier application, not some of our other apps. > > Here is what we're observing on the Linux Midtier server. Below is an > example of 3 existing tcp connections that remain intact indefinitely or > until midtier is restarted. The firewall knows about these 3 tcp > connections, but if the connections are idle for more than 60 minutes > (default firewall setting), the firewall will drop these idle tcp > connections. Then, when a user gets on midtier (after the idle period), > midtier will attempt to use one or more of these tcp connections and the > firewall responds with a Deny (since it has aged off the inactive > connection). This is normal behavior for a firewall as I understand things, > but the application (midtier in this case) should ideally make use of the tcp > keepalive or inactivetly timeout supported by Linux. Apparently, midtier > does not take advantage of this. > > I suspect other customers are having this problem IF their topology utilizes > a firewall between Midtier and ARS and there are long periods of inactivity. > The reason, however, they may not be complaining is because once you get the > RCP failure (arerr 91), then subsequent connections seem to work fine. For > the example below, once the arerr 91 timeout finally occurred, the associated > tcp connection was terminated on the midtier server and a new one was created. > > $netstat -antopp|grep 2031 > Proto Recv-Q Send-Q Local Address Foreign Address > State Timer > tcp 0 0 ::ffff:172.16.0.129:54310 ::ffff:10.1.1.141:2031 > ESTABLISHED 24977/java off (0.00/0/0) > tcp 0 864 ::ffff:172.16.0.129:54309 ::ffff:10.1.1.141:2031 > ESTABLISHED 24977/java on (3.59/12/0) > tcp 0 1416 ::ffff:172.16.0.129:43717 ::ffff:10.1.1.141:2031 > ESTABLISHED 24977/java on (16.19/11/0) > > After a very long wait trying to login in, the user will receive this message: > ARERR [91] > RPC call failed : ONC/RPC call timed out > > > Ken > > > -----Original Message----- > From: Action Request System discussion list(ARSList) > [mailto:arsl...@arslist.org] On Behalf Of Axton > Sent: Tuesday, November 17, 2009 3:24 PM > To: arslist@ARSLIST.ORG > Subject: Re: Firewall TCP Timeouts > > You need to configure your firewall properly. With a firewall, you can > define what types of packets can create a state entry on the firewall. > Typical is syn, syn ack. Even if there is no state entry, a new packet > should create a new state (not be dropped). If a new packet does not create > a new state, change the firewall rules so that it does. A network dump will > tell you what type of packet is going out; the firewall logs should tell you > what type of packet was rejected. > > I could see that this would be a problem if the midtier servers were behind a > NAT. Is this the case? > > Axton Grams > > On Tue, Nov 17, 2009 at 12:07 PM, Leihkauff, Kenneth > <kenneth.g.leihka...@saic.com> wrote: >> ** >> >> Hello, >> >> We have a firewall between our MidTier server and ARS system. The >> firewall is configured to drop tcp connections after being idle for 60 >> minutes (typical/default firewall setting). Several MidTier user >> sessions will make use of a shared tcp connection so you might have >> 100 sessions but significantly fewer tcp connections. During idle >> times (like at night), the firewall will discard these idle tcp >> connections but the MidTier server will still retain these tcp >> references (this can be seen by using "netstat -anto"). So, when >> users get back on the system, MidTier apparently is trying to utilize >> one of these defunct tcp connections so you end up with problems like >> ARERR 91 rpc timeouts because these tcp connections are broken pipes. >> >> Is there a MidTier/Tomcat or other setting that you have found >> addresses this problem? >> >> Thanks. >> >> Ken >> >> Background: >> Version 7.5 patch 3 >> MidTier - Linux, Tomcat JSP >> ARS - Linux, Oracle 10g >> > > _______________________________________________________________________________ > UNSUBSCRIBE or access ARSlist Archives at www.arslist.org > Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are" > _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are"