No NATing is involved. It is simply a firewall between our DMZ and internal networks -- network address translation is not used.
Ken -----Original Message----- From: Action Request System discussion list(ARSList) [mailto:arsl...@arslist.org] On Behalf Of Axton Sent: Thursday, November 19, 2009 10:05 AM To: arslist@ARSLIST.ORG Subject: Re: Firewall TCP Timeouts What pieces of the AR infrastructure are in a NAT? Are the arservers in one subnet behind a NAT and the midtier in a different subnet behind a NAT? Axton On Thu, Nov 19, 2009 at 8:14 AM, Leihkauff, Kenneth <kenneth.g.leihka...@saic.com> wrote: > Axton, our firewall is a Cisco ASA 5520. > > Ken > =============== > Your Previous related msg: > > This "can" be normal behavior for a firewall. It all depends on how you > allow packets to create a state or if you use state at all. You can create a > rule that is stateless so that all packets from host X to host Y are allowed. > You should explore creating the rule to do a quick pass of the packets and > not use state for these connections. > Basically, the rules would look like this: > > pass in quick on $if_1 inet proto tcp from $mt_host_1 to $ar_host_1 pass out > quick on $if_2 inet proto tcp from $ar_host_1 to $mt_host_1 > > Instead of a state based or policy based ruleset. > > You can also change the type of packet on the firewall so that syn/syn ack > packets are not the only type of packet that can insert a state into the > state table. This is a little funky and is probably not the best way to do > this, but it can be done this way too. > > Axton Grams > > > -----Original Message----- > From: Action Request System discussion list(ARSList) > [mailto:arsl...@arslist.org] On Behalf Of Axton > Sent: Wednesday, November 18, 2009 5:09 PM > To: arslist@ARSLIST.ORG > Subject: Re: Firewall TCP Timeouts > > Midtier timeout has to do with the user session timeout. Once > arserver is started there is a persistent socket connection to the > arserver. > > Can I ask what type of firewall this is? > > Axton > > The opinions, statements, and/or suggested courses of action expressed > in this E-mail do not necessarily reflect those of BMC Software, Inc. > My voluntary participation in this forum is not intended to convey a > role as a spokesperson, liaison or public relations representative for > BMC Software, Inc. > > On Wed, Nov 18, 2009 at 4:05 PM, Grooms, Frederick W > <frederick.w.gro...@xo.com> wrote: >> Since you are using MidTier I would suggest you do the following... >> >> Add a specified port for your AR Server to listen on (The AR Server runs >> just fine with portmapper and a specified port). >> Set the MidTier to use the specified port. >> >> Also set the MidTier Session TimeOut value to be less than your firewall >> time out setting. This way Mid-Tier will drop old connections before the >> firewall does. You should be able to set the MIdTier TimeOut from the >> configuration pages on your MidTier server. >> >> Fred >> >> >> -----Original Message----- >> From: Action Request System discussion list(ARSList) >> [mailto:arsl...@arslist.org] On Behalf Of Leihkauff, Kenneth >> Sent: Wednesday, November 18, 2009 3:51 PM >> To: arslist@ARSLIST.ORG >> Subject: Re: Firewall TCP Timeouts >> >> Thanks, Conny. Are you saying the libkeepalive was implemented on the >> ARserver -- not the MidTier linux server? Our midtier and arserver are linux >> with a firewall in between. >> >> Unfortunately, the firewall is not an application level firewall so the >> firewall rule changes Axton proposed cannot be implemented according to our >> network engineer. >> >> Ken >> >> -----Original Message----- >> From: Action Request System discussion list(ARSList) >> [mailto:arsl...@arslist.org] On Behalf Of Conny Martin >> Sent: Wednesday, November 18, 2009 3:30 PM >> To: arslist@ARSLIST.ORG >> Subject: AW: Firewall TCP Timeouts >> >> We had the same issue. We solved this by using libkeepalive >> (http://libkeepalive.sourceforge.net/) on the arserver machine. But it would >> work only if you are using Linux. >> >> HTH >> >> Kind Regards Conny >> >> -----Ursprüngliche Nachricht----- >> Von: Action Request System discussion list(ARSList) >> [mailto:arsl...@arslist.org] Im Auftrag von Leihkauff, Kenneth >> Gesendet: Mittwoch, 18. November 2009 18:18 >> An: arslist@ARSLIST.ORG >> Betreff: Re: Firewall TCP Timeouts >> >> Thanks for your time, Axton. I spoke with our network engineer and he >> studied the firewall logs and sees quit a few denials and some resets, but >> is confident things should be set up correctly on the firewall. He only >> sees these denials/resets for the Midtier application, not some of our other >> apps. >> >> Here is what we're observing on the Linux Midtier server. Below is an >> example of 3 existing tcp connections that remain intact indefinitely or >> until midtier is restarted. The firewall knows about these 3 tcp >> connections, but if the connections are idle for more than 60 minutes >> (default firewall setting), the firewall will drop these idle tcp >> connections. Then, when a user gets on midtier (after the idle period), >> midtier will attempt to use one or more of these tcp connections and the >> firewall responds with a Deny (since it has aged off the inactive >> connection). This is normal behavior for a firewall as I understand things, >> but the application (midtier in this case) should ideally make use of the >> tcp keepalive or inactivetly timeout supported by Linux. Apparently, >> midtier does not take advantage of this. >> >> I suspect other customers are having this problem IF their topology utilizes >> a firewall between Midtier and ARS and there are long periods of inactivity. >> The reason, however, they may not be complaining is because once you get >> the RCP failure (arerr 91), then subsequent connections seem to work fine. >> For the example below, once the arerr 91 timeout finally occurred, the >> associated tcp connection was terminated on the midtier server and a new one >> was created. >> >> $netstat -antopp|grep 2031 >> Proto Recv-Q Send-Q Local Address Foreign Address >> State Timer >> tcp 0 0 ::ffff:172.16.0.129:54310 ::ffff:10.1.1.141:2031 >> ESTABLISHED 24977/java off (0.00/0/0) >> tcp 0 864 ::ffff:172.16.0.129:54309 ::ffff:10.1.1.141:2031 >> ESTABLISHED 24977/java on (3.59/12/0) >> tcp 0 1416 ::ffff:172.16.0.129:43717 ::ffff:10.1.1.141:2031 >> ESTABLISHED 24977/java on (16.19/11/0) >> >> After a very long wait trying to login in, the user will receive this >> message: >> ARERR [91] >> RPC call failed : ONC/RPC call timed out >> >> >> Ken >> >> >> -----Original Message----- >> From: Action Request System discussion list(ARSList) >> [mailto:arsl...@arslist.org] On Behalf Of Axton >> Sent: Tuesday, November 17, 2009 3:24 PM >> To: arslist@ARSLIST.ORG >> Subject: Re: Firewall TCP Timeouts >> >> You need to configure your firewall properly. With a firewall, you can >> define what types of packets can create a state entry on the firewall. >> Typical is syn, syn ack. Even if there is no state entry, a new packet >> should create a new state (not be dropped). If a new packet does not create >> a new state, change the firewall rules so that it does. A network dump will >> tell you what type of packet is going out; the firewall logs should tell you >> what type of packet was rejected. >> >> I could see that this would be a problem if the midtier servers were behind >> a NAT. Is this the case? >> >> Axton Grams >> >> On Tue, Nov 17, 2009 at 12:07 PM, Leihkauff, Kenneth >> <kenneth.g.leihka...@saic.com> wrote: >>> ** >>> >>> Hello, >>> >>> We have a firewall between our MidTier server and ARS system. The >>> firewall is configured to drop tcp connections after being idle for 60 >>> minutes (typical/default firewall setting). Several MidTier user >>> sessions will make use of a shared tcp connection so you might have >>> 100 sessions but significantly fewer tcp connections. During idle >>> times (like at night), the firewall will discard these idle tcp >>> connections but the MidTier server will still retain these tcp >>> references (this can be seen by using "netstat -anto"). So, when >>> users get back on the system, MidTier apparently is trying to utilize >>> one of these defunct tcp connections so you end up with problems like >>> ARERR 91 rpc timeouts because these tcp connections are broken pipes. >>> >>> Is there a MidTier/Tomcat or other setting that you have found >>> addresses this problem? >>> >>> Thanks. >>> >>> Ken >>> >>> Background: >>> Version 7.5 patch 3 >>> MidTier - Linux, Tomcat JSP >>> ARS - Linux, Oracle 10g >>> >> >> _______________________________________________________________________________ >> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org >> Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are" >> > > _______________________________________________________________________________ > UNSUBSCRIBE or access ARSlist Archives at www.arslist.org > Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are" > > _______________________________________________________________________________ > UNSUBSCRIBE or access ARSlist Archives at www.arslist.org > Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are" > _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are" _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor:rmisoluti...@verizon.net ARSlist: "Where the Answers Are"
