Adam,
On Wed, 26 Jan 2011, Adam Greene wrote:
Pete,
Thanks ... we ran some tests this evening, disabling NAT entirely, and saw
the same results, so I think we can safely say that NAT is not causing the
issue.
The situation we are facing is that the customer appears to be unable to
route asymmetric traffic. At least that's what we think the problem is so
far. They get full routes from two providers (one of them being us), and
announce their IP block through both providers. When traffic comes in through
us and goes out through their other provider, it's blocked. ICMP traffic
doesn't seem to mind. But TCP traffic is definitely not working.
"not working" as in "the connection does not complete and hangs" or as in "can
not get the the SYN in the direction from/to them?"
I'd probe at a more granular level to see whether the problem is related to
something blocking due to a confused state machine on a middlebox somewhere,
*or* if it is indeed genuine ACL.
We allow their traffic through our network asymmetrically (we have ASA's at
our edges but have enabled tcp-state-bypass on them). I am suspecting the
customer's other upstream provider (or ours) may have some asymmetry block in
place. But ... that also seems unlikely, since I assume most IP carriers do
not.
The part that seems unlikely to me is the fact that ICMP behaves differently
from TCP - if I were to put in uRPF-like measures, I'd have them identical for
ICMP and TCP.
So we're currently stumped.
I'd debug it this way: first, pick a server somewhere on the 'net where you can
initiate the connections to, from, and do the tcpdump on (to see in details what
happens).
There are several points where it makes sense to put the probes:
1) customer's host (internal).
2) both their firewalls (internal)
3) both their firewalls (external)
4) your cloud closer to customer
5) your cloud closer to the server
6) the server itself
Then you can try a TCP connection from the customer to that server, and see
where things break - and try the TCP connection in the other direction, and
compare how the TCP connection looks at each point when sniffed.
The above might be a bit laborious to do all at once so it can be
split in multiple iterations with less probe points - but just to outline the
principle.
I notice on github there's now also another potentially useful tool for
debugging - https://github.com/enki/muXTCP - but that'd be a much longer story
as I suspect that code had been affected by bitrot :-)
cheers,
andrew
Thanks,
Adam
On 1/24/2011 2:56 PM, Pete Lumbis wrote:
Adam,
I realized (with the help of an off-list post) I mis-read your
original post. I thought this was on two different devices, instead of
two connections on the same device.
For a single box the NAT lookups are done when traffic arrives on any
nat inside/outside interface*. If we create a translation for a packet
exiting f0/0 (for example) and the response arrives on f0/1, we will
see the packet arriving on a NAT outside interface, do the NAT lookup
and match the existing translation that was created by the first
outbound packet.
What kind of problems are you seeing? Is traffic slow or not arriving at
all?
*based on NAT order of ops when traffic arrives on a NAT interface and
is destined for a NAT interface
-Pete
On Fri, Jan 21, 2011 at 6:05 PM, Pete Lumbis<alum...@gmail.com> wrote:
NAT could definitely be causing issues. Generally you could use
something like Stateful NAT (SNAT) between the two BGP speakers to
make sure they sync their NAT tables, but this this feature has had a
number of challenges/issues and development and started moving it to
end of life.
On Fri, Jan 21, 2011 at 4:09 PM, Adam Greene<maill...@webjogger.net>
wrote:
Hi guys,
I have a multihomed customer who receives full BGP routes from both us
and
another provider and load balances between the two connections. Things
are
working fine until the traffic becomes asymmetric (i.e. inbound through
one
provider, outbound through the other).
The block they are announcing to their providers is NATed on their BGP
router. In other words, all their internal hosts are on private IP space.
The internal interface is designated "ip nat inside" and both WAN
interfaces
are designated "ip nat outside". The actual NAT configurations do not
reference any interfaces, just pools.
Could the NAT be prohibiting asymmetric traffic in this case? i.e. if the
inbound traffic is NATed coming in on one interface, will the router
refuse
to NAT the outbound traffic through the other interface?
If the NAT is the problem, I suppose they could do the NAT on a loopback
interface instead ... but I understand that the traffic will all be
process-switched if we do that, and performance will probably suffer.
Thanks for your insight,
Adam
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
