Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-28 Thread Stuart Henderson 
On 2006/11/28 18:07, Michael Lockhart wrote:
> Set net.inet.tcp.rfc1323=0 in /etc/sysctl.conf and that should resolve
> the issue.

that's not a fix though, it just avoids the conditions which cause the
problem to occur. better to ensure the ruleset is completely sane. if so,
then test cases need to be found to isolate the problem.

if anyone wants an example of wonderful source code commenting,
the pf_norm.c sections relating to rfc1323 are particularly good.

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-28 Thread Michael Lockhart 
Set net.inet.tcp.rfc1323=0 in /etc/sysctl.conf and that should resolve
the issue.  We've been testing in house with OpenBSD for Vista (we have
700+/- systems in the field) and this seems to resolve the issue.

Regards,
Mike Lockhart
 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Mike Lockhart[Systems Engineering & Operations]
StayOnline, Inc
http://www.stayonline.net/
mailto: [EMAIL PROTECTED]
GPG: 8714 6F73 3FC8 E0A4 0663  3AFF 9F5C 888D 0767 1550
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf
Of Stuart Henderson
Sent: Tuesday, November 28, 2006 5:46 PM
To: Reverend Deuce
Cc: misc@openbsd.org
Subject: Re: Baffling problem with OBSD-protected servers and Windows
Vista...

On 2006/11/28 14:32, Reverend Deuce wrote:
> Okay guys, I posted that long message about Firefox/etc on Windows
> Vista a couple of days ago.

this would be easier if you just posted pf.conf rather than non-linear
snippets; however..

>  a) there is a default block policy

I didn't notice you posting anything showing a default block for
outgoing
packets, check this and if not, add one.

> block in  log from any to any
label "DefaultBlock"

> block in  log on { $ext_if }   all label
"DefaultBlock"
> block return-rst  in  log on { $ext_if } proto tcp all label
"DefaultBlock"
> block return-icmp in  log on { $ext_if } proto udp all label
"DefaultBlock"

fwiw, you can simplify these if you like:
'block return in log on { $ext_if } label "DefaultBlock"'

> I have heard it said that it makes no sense to filter on two
> interfaces, best to pass on one and block on the other.

that advice is usually given in relation to filtering bridges.

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-28 Thread Stuart Henderson 
On 2006/11/28 14:32, Reverend Deuce wrote:
> Okay guys, I posted that long message about Firefox/etc on Windows
> Vista a couple of days ago.

this would be easier if you just posted pf.conf rather than non-linear
snippets; however..

>  a) there is a default block policy

I didn't notice you posting anything showing a default block for outgoing
packets, check this and if not, add one.

> block in  log from any to any label 
> "DefaultBlock"

> block in  log on { $ext_if }   all label "DefaultBlock"
> block return-rst  in  log on { $ext_if } proto tcp all label "DefaultBlock"
> block return-icmp in  log on { $ext_if } proto udp all label "DefaultBlock"

fwiw, you can simplify these if you like:
'block return in log on { $ext_if } label "DefaultBlock"'

> I have heard it said that it makes no sense to filter on two
> interfaces, best to pass on one and block on the other.

that advice is usually given in relation to filtering bridges.

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-28 Thread Reverend Deuce 

Okay guys, I posted that long message about Firefox/etc on Windows
Vista a couple of days ago.

After I re-read my post and looked at the tcpdump output, and chatting
with a friend of mine who also runs several OBSD firewalls at his
company which exhibited the same EXACT problem when my Vista installs
attempted to connect... I think we've figured it out.

I remember that Vista (and presumably Longhorn Server) have a
completely re-written TCP stack by Microsoft. They've put in all kinds
of new stuff. One of which is Receive Window Auto-Tuning.

I noticed in my tcpdump the following lines:


From Opera/Firefox:


20:40:45.824144 my.workstation.ip.49370 > remote.server.ip.80: S
1215871830:1215871830(0) win 8192  (DF)
20:38:25.198320 remote.server.ip.80 > my.workstation.ip.49357: S
852828096:852828096(0) ack 643900712 win 64240 


From IE 7:


20:39:08.834465 my.workstation.ip.49358 > remote.server.ip.80: S
4155969795:4155969795(0) win 8192  (DF)
20:39:08.835095 remote.server.ip.80 > my.workstation.ip.49358: S
3294485308:3294485308(0) ack 4155969796 win 64240 

Notice the window scale is 2 for IE, and 8 for Firefox/Opera.


From the following MS blog @

http://www.microsoft.com/technet/community/columns/cableguy/cg1105.mspx
:

Note: Some Internet gateway devices and firewalls block packet flows
because they do not correctly interpret the scaling factor used in TCP
connections. Because of this, Internet Explorer in Windows Vista uses
an initial scaling factor of 2. Other applications use a default
initial scaling factor of 8.

After doing the new Vista-equivilent of sudo and elevating my command
shell to Administrator mode, I was able to use the following command
to disable window scaling completely:

C:\windows\system32> netsh interface tcp set global autotuninglevel=disabled

Once I performed this command, Firefox/Opera/Remote Desktop Connection
all function once more as expected.

Now, as I am clearly looking at the window scale issue here, I had
found a thread at
http://archive.openbsd.nu/?ml=openbsd-pf&a=2006-07&t=2147873 where
Daniel Hartmeier comments there are three things that need to be done
to have state created correctly:

 a) there is a default block policy
 b) all 'pass' rules that can match TCP have 'flags S/SA'
 c) all 'pass' rules have 'keep state'

Here is what I am seeing inside PF with the connections in question:

Nov 26 23:09:36.970856 rule 80/(match) pass in on fxp1:
my.workstation.ip.59970 > remote.server.ip.443: [|tcp] (DF)

Then if I pull the 80th rule out:

@80 pass in log quick on fxp1 inet proto tcp from any to
remote.server.ip port = https flags S/SA keep state label "ExchangeIn"

Now, I can easily see that I am matching B and C of Daniel's list,
however A is a bit more in question from my point of view.

The rules I do have are:

@47 block drop in log on fxp1 all label "DefaultBlock"

@48 block return-rst in log on fxp1 proto tcp all label "DefaultBlock"

@49 block return-icmp(port-unr, port-unr) in log on fxp1 proto udp all
label "DefaultBlock"

So, it appears I have condition A matched as well. I do have a line regarding:

@95 pass in quick on fxp0 inet proto tcp from  to any flags
S/SA keep state label "lanOUT"

That should not come into play here at all, as again it is creating
state on a Syn, not a Syn Ack.

However, after testing on this system, I am thinking I am filtering
wrong here. Here is what I have found as the full story of what is
going on:

Connection is open:

Nov 27 12:09:07.978281 rule 80/(match) pass in on fxp1:
my.workstation.ip.62658 > remote.server.ip.443: [|tcp] (DF)

Two state entries are created:

all tcp remote.server.ip:443 <- remote.server.ip:443 <-
my.workstation.ip:62658   ESTABLISHED:ESTABLISHED
  [4265902579 + 65535]  [1356591875 + 65535]
  age 00:01:08, expires in 119:59:09, 12:9 pkts, 2014:5401 bytes, rule 80
  id: 453e890500b00c1e creatorid: 19ad04b2
all tcp my.workstation.ip:62658 <- remote.server.ip:443
ESTABLISHED:ESTABLISHED
  [1356591875 + 65535]  [4265902579 + 65535]
  age 00:01:08, expires in 119:59:09, 9:11 pkts, 5401:1966 bytes, rule 96
  id: 453e890500b00c1f creatorid: 19ad04b2

Rules that match the state entries:

@80 pass in log quick on fxp1 inet proto tcp from any to
remote.server.ip port = https flags S/SA keep state label "ExchangeIn"
 [ Evaluations: 5000  Packets: 204   Bytes: 50906   States: 5 ]
 [ Inserted: uid 0 pid 806 ]
@95 pass in quick on fxp0 inet proto tcp from  to any flags
S/SA keep state label "lanOUT"
 [ Evaluations: 417330Packets: 60770372  Bytes: 36986041704  States: 771  ]
 [ Inserted: uid 0 pid 6307 ]
@96 pass in quick on fxp0 from  to any keep state label "lanOUT"
 [ Evaluations: 429312Packets: 9560597   Bytes: 5957780712  States: 135   ]
 [ Inserted: uid 0 pid 6307 ]

Now, it is looking to me like the issue is a second state entry is
created by that rule 96. When it is modified to be only protocol UDP,
traffic through the FW stops due to the rules:

block in  log

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-27 Thread Christian Ruediger Bahls 
[2006-11-27 10:43] Claudio Jeker <[EMAIL PROTECTED]> wrote:
> Both Firefox and Opera use a wscale of 8 whereas IE uses a wscale of 2.
> In my opinion this sounds like the typical problem where states are not
> created on the initial SYN packet.
sounds like a window scaling problem as described in:
http://inodes.org/blog/2006/09/06/tcp-window-scaling-and-kernel-2617/

there have been some very opinionated mail on LKML about it
(they had the same problem mit linux kernel 2.6.17+)
(about striping the scaling factor ..
 .. or ignoring it when calculation windows)
though i am not able to find it now


christian bahls

-- 
personal email reaches me at gmx.de
[EMAIL PROTECTED]

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-27 Thread Christian M. Bernard 
Hi there

I had the exact same strange (kind of) problem.

All clients could connect to my (own OpenBSD) web server, only my main PC
(sorry linux gentoo machine) could not. The packets match what you show
below. It stops because the initial http) packet does't arrive at your
VistaPC. [Fire up wireshark and load the tcpdump into that -> TCP previous
segment lost, it told me]

After 2 days and nights shuffling around all Hardware and software
(Firewall, PC, Ethernetcards, Gateway, kernels, ...)
this was the problem:
On the OpenBSD www server (pf enabled), the one rule had no 'keep state'
not good:
pass  in quick on $external inet proto tcp  from any  
   to $http_server  port = $http_port  label "l_http_in"
good:
pass  in quick on $external inet proto tcp  from any  
   to $http_srv  port = http   keep statelabel "l_http_in"
 # queue q_rest_out
pass out quick on $external inet proto tcp  from any  
   to anyport = http   keep state
(I added the pass out rule, not sure if needed)

Why on earth this only produced problems with 1/100 of all clients, I
cannot say... (It should always fail or never...)
For example with linux kernel 2.6.16.x it worked, with kernel 2.6.17.x it
didn't. Some IP/networking behavior must have been changed in defaults
...?)

Hope that helps
-cmb


[ long part of email  removed]

>
>
> Opera 9:
>
> 20:40:45.824144 my.workstation.ip.49370 > remote.server.ip.80: S
> 1215871830:1215871830(0) win 8192  8,nop,nop,sackOK> (DF)
> 20:40:45.824646 207.218.64.33.80 > my.workstation.ip.49370: S
> 2582857930:2582857930(0) ack 1215871831 win 64240  0,nop,nop,sackOK>
> 20:40:45.878361 my.workstation.ip.49370 > 207.218.64.33.80: . ack 1 win
> 260 (DF)
> 20:40:45.904597 my.workstation.ip.49370 > 207.218.64.33.80: P
> 1:384(383) ack 1 win 260 (DF)
> 20:40:46.058234 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
> win 63857 (DF)
> 20:40:46.061253 my.workstation.ip.49370 > 207.218.64.33.80: P
> 1:384(383) ack 1 win 260 (DF)
> 20:40:46.061726 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
> win 63857 (DF)
> (at this point, the connection is hung -- the Vista workstation
> receives no further communcations -- it's like it just drops the
> replies)
>
>
>
> Firefox:
>
> 20:38:25.197691 my.workstation.ip.49357 > remote.server.ip.80: S
> 643900711:643900711(0) win 8192 
> (DF)
> 20:38:25.198320 remote.server.ip.80 > my.workstation.ip.49357: S
> 852828096:852828096(0) ack 643900712 win 64240  0,nop,nop,sackOK>
> 20:38:25.244540 my.workstation.ip.49357 > remote.server.ip.80: . ack 1
> win 260 (DF)
> 20:38:25.251037 my.workstation.ip.49357 > remote.server.ip.80: P
> 1:403(402) ack 1 win 260 (DF)
> 20:38:25.567602 my.workstation.ip.49357 > remote.server.ip.80: P
> 1:403(402) ack 1 win 260 (DF)
> 20:38:25.568042 remote.server.ip.80 > my.workstation.ip.49357: . ack
> 403 win 63838 (DF)
> (same deal -- it just seems to die right here)
>

[ removed ... ]

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-27 Thread Claudio Jeker 
On Sun, Nov 26, 2006 at 09:19:25PM -0600, Reverend Deuce wrote:
> (This is very long email because it's a very complicated problem...
> I've included some tcpdump logs below to assist...)
> 


 
> Here are some tcpdumps from the master FW during connection attempts
> with a browser:
> 
> 
> 
> Opera 9:
> 
> 20:40:45.824144 my.workstation.ip.49370 > remote.server.ip.80: S
> 1215871830:1215871830(0) win 8192  8,nop,nop,sackOK> (DF)
> 20:40:45.824646 207.218.64.33.80 > my.workstation.ip.49370: S
> 2582857930:2582857930(0) ack 1215871831 win 64240  0,nop,nop,sackOK>
> 20:40:45.878361 my.workstation.ip.49370 > 207.218.64.33.80: . ack 1 win 260 
> (DF)
> 20:40:45.904597 my.workstation.ip.49370 > 207.218.64.33.80: P
> 1:384(383) ack 1 win 260 (DF)
> 20:40:46.058234 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
> win 63857 (DF)
> 20:40:46.061253 my.workstation.ip.49370 > 207.218.64.33.80: P
> 1:384(383) ack 1 win 260 (DF)
> 20:40:46.061726 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
> win 63857 (DF)
> (at this point, the connection is hung -- the Vista workstation
> receives no further communcations -- it's like it just drops the
> replies)
> 
> 
> 
> Firefox:
> 
> 20:38:25.197691 my.workstation.ip.49357 > remote.server.ip.80: S
> 643900711:643900711(0) win 8192 
> (DF)
> 20:38:25.198320 remote.server.ip.80 > my.workstation.ip.49357: S
> 852828096:852828096(0) ack 643900712 win 64240  0,nop,nop,sackOK>
> 20:38:25.244540 my.workstation.ip.49357 > remote.server.ip.80: . ack 1
> win 260 (DF)
> 20:38:25.251037 my.workstation.ip.49357 > remote.server.ip.80: P
> 1:403(402) ack 1 win 260 (DF)
> 20:38:25.567602 my.workstation.ip.49357 > remote.server.ip.80: P
> 1:403(402) ack 1 win 260 (DF)
> 20:38:25.568042 remote.server.ip.80 > my.workstation.ip.49357: . ack
> 403 win 63838 (DF)
> (same deal -- it just seems to die right here)
> 
> 
> 
> IE 7:
> 
> 20:39:08.834465 my.workstation.ip.49358 > remote.server.ip.80: S
> 4155969795:4155969795(0) win 8192  2,nop,nop,sackOK> (DF)
> 20:39:08.835095 remote.server.ip.80 > my.workstation.ip.49358: S
> 3294485308:3294485308(0) ack 4155969796 win 64240  0,nop,nop,sackOK>
> 20:39:08.892057 my.workstation.ip.49358 > remote.server.ip.80: . ack 1
> win 16685 (DF)
> 20:39:08.904548 my.workstation.ip.49358 > remote.server.ip.80: P
> 1:472(471) ack 1 win 16685 (DF)
> 20:39:08.907010 remote.server.ip.80 > my.workstation.ip.49358: .
> 1:1381(1380) ack 472 win 63769 (DF)
> 20:39:08.907135 remote.server.ip.80 > my.workstation.ip.49358: .
> 1381:2761(1380) ack 472 win 63769 (DF)
> 20:39:08.959016 my.workstation.ip.49358 > remote.server.ip.80: . ack
> 2761 win 16685 (DF)
> 20:39:08.959740 remote.server.ip.80 > my.workstation.ip.49358: .
> 2761:4141(1380) ack 472 win 63769 (DF)
> 20:39:08.959750 remote.server.ip.80 > my.workstation.ip.49358: .
> 4141:5521(1380) ack 472 win 63769 (DF)
> 20:39:08.959911 remote.server.ip.80 > my.workstation.ip.49358: .
> 5521:6901(1380) ack 472 win 63769 (DF)
> 20:39:09.010614 my.workstation.ip.49358 > remote.server.ip.80: . ack
> 6901 win 16685 (DF)
> 20:39:09.011323 remote.server.ip.80 > my.workstation.ip.49358: .
> 6901:8281(1380) ack 472 win 63769 (DF)
> 20:39:09.011333 remote.server.ip.80 > my.workstation.ip.49358: .
> 8281:9661(1380) ack 472 win 63769 (DF)
> 20:39:09.011447 remote.server.ip.80 > my.workstation.ip.49358: .
> 9661:11041(1380) ack 472 win 63769 (DF)
> 20:39:09.011571 remote.server.ip.80 > my.workstation.ip.49358: .
> 11041:12421(1380) ack 472 win 63769 (DF)
> 20:39:09.058459 my.workstation.ip.49358 > remote.server.ip.80: . ack
> 9661 win 16685 (DF)
> 20:39:09.059165 remote.server.ip.80 > my.workstation.ip.49358: .
> 12421:13801(1380) ack 472 win 63769 (DF)
> 20:39:09.059289 remote.server.ip.80 > my.workstation.ip.49358: P
> 13801:15131(1330) ack 472 win 63769 (DF)
> 20:39:09.064831 my.workstation.ip.49358 > remote.server.ip.80: . ack
> 12421 win 16685 (DF)
> 20:39:09.097561 my.workstation.ip.49359 > remote.server.ip.80: S
> 3924198291:3924198291(0) win 8192  2,nop,nop,sackOK> (DF)
> 20:39:09.098564 remote.server.ip.80 > my.workstation.ip.49359: S
> 4022470982:4022470982(0) ack 3924198292 win 64240  0,nop,nop,sackOK>
> (But IE 7 is just dandy! It loads the whole page and just keeps on
> tickin, goddamnit!)
> 
> 
> I should note that the only connections that seem to have trouble are
> TCP connections. It seems that OpenVPN clients can traverse the
> firewall without issue (our OpenVPN server sites behind the firewall).
> Inbound DNS queries have no issues either. ICMP is good also, for the
> hosts that have permission to ping.
> 
> Again, I need to stress this -- I've NOT overlooked the obvious here.
> Remember, the problem **only** happens in Vista. It has happened on
> several desktops/workstations, both off-site from our office, inside a
> VM, directly on hardware, etc. I've tried about a dozen configs so
> far. This is not a "try swapping the RAM" situation.
> 
> There are no strange VPNs, static routes, oddball topologies, etc

Re: Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-26 Thread Bill Maas 
I'm not sure if this will be of any help, but at least the Firefox issue
sounds like FF is able to connect, but never receives any return
traffic. I've had that with misconfigured netmasks I believe. Does Vista
use some sort of net group or certificate based access scheme (e.g. "if
it's not a Vista box talking to me, I won't talk back")? May sound
stupid, but you never know. Who on earth knows what MS does with network
traffic?

Bill

On Sun, 2006-11-26 at 21:19 -0600, Reverend Deuce wrote:
> (This is very long email because it's a very complicated problem...
> I've included some tcpdump logs below to assist...)
> 
> The last week and days I've been working with the RTM version of Vista
> obtained through my MSDN license. This is the "gold" version of
> Windows Vista, BTW. It's done. It's been shipped to manufacturing
> (hence RTM, "release to manufacturing").
> 
> Okay, so I've installed this thing and am testing out all the bells
> and whistles.
> 
> I install Firefox, OpenVPN, putty, the Java JRE from Sun, etc.
> 
> I start to tool around and I notice that none of our company's web
> sites will load in Firefox any longer. Firefox's status bar says
> "Waiting for www.site.tld..." and eventually will time out. It does
> this for every single web site we host.
> 
> I fire up IE 7. No problems with any site.
> 
> I go back to Firefox and it's still having issues -- but *only* with
> sites hosted behind our OpenBSD firewalls.
> 
> I fire up "telnet" (after enabling it through the control panel -- no
> idea why MS did this). I can telnet to our web servers via port 80,
> issue "GET" requests, receive responses. No trouble with Telnet.exe.
> 
> Putty however, has trouble. Wont work period. Port 80 telnet, ssh port
> 22, etc. None of them.
> 
> So now I am thinking that it might just be a Firefox problem... but
> it's not. Microsoft's own Remote Desktop Connection (terminal services
> client, rdp client, etc) wont connect to our datacenter servers -- and
> they are accessed via an openvpn point to point VPN that terminates on
> the OpenBSD firewall which acts strictly as a routed tunnel between
> our two networks.
> 
> I turn off as much of the Vista "security" features as I can. This does 
> nothing.
> 
> Since our OBSD firewalls were of the older variety (3.6), I figured I
> might try an upgrade to 4.0 to see what happens. No dice.
> 
> To summarize:
> 
> This **only** is affecting Windows Vista (have not tried the latest
> betas of Longhorn Server). Windows XP, Windows 2000, Free/OpenBSD,
> CentOS, and our four Mac users with OSX have zero trouble. None. Nada.
> They work flawlessly.
> 
> Okay, so we can blame Vista -- that would be fine with me, but let's
> face it -- this going to be big come January. I have a month to fix
> this damn thing and I am really out of ideas.
> 
> Our network:
> 
> 100mbit dedicated inet connection through AT&T, terminates to a big
> Cisco setup owned by our datacenter.
> 
> Firewalls are now OBSD 4.0, single-proc Xeon 2.4gHz, 1GB RAM, etc. --
> they are decent systems with six gigabit NICs each.
> 
> They are all configured with CARP and pfsync. This has worked very,
> very well since day 1 in 2004! CARP rocks!
> 
> They connect to an HP ProCurve 5400ZL modular switch, configured with
> various port VLANs, etc. Everything is gigabit, 'cept for a few
> databases using 10-gigabit CX4.
> 
> Here are some tcpdumps from the master FW during connection attempts
> with a browser:
> 
> 
> 
> Opera 9:
> 
> 20:40:45.824144 my.workstation.ip.49370 > remote.server.ip.80: S
> 1215871830:1215871830(0) win 8192  8,nop,nop,sackOK> (DF)
> 20:40:45.824646 207.218.64.33.80 > my.workstation.ip.49370: S
> 2582857930:2582857930(0) ack 1215871831 win 64240  0,nop,nop,sackOK>
> 20:40:45.878361 my.workstation.ip.49370 > 207.218.64.33.80: . ack 1 win 260 
> (DF)
> 20:40:45.904597 my.workstation.ip.49370 > 207.218.64.33.80: P
> 1:384(383) ack 1 win 260 (DF)
> 20:40:46.058234 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
> win 63857 (DF)
> 20:40:46.061253 my.workstation.ip.49370 > 207.218.64.33.80: P
> 1:384(383) ack 1 win 260 (DF)
> 20:40:46.061726 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
> win 63857 (DF)
> (at this point, the connection is hung -- the Vista workstation
> receives no further communcations -- it's like it just drops the
> replies)
> 
> 
> 
> Firefox:
> 
> 20:38:25.197691 my.workstation.ip.49357 > remote.server.ip.80: S
> 643900711:643900711(0) win 8192 
> (DF)
> 20:38:25.198320 remote.server.ip.80 > my.workstation.ip.49357: S
> 852828096:852828096(0) ack 643900712 win 64240  0,nop,nop,sackOK>
> 20:38:25.244540 my.workstation.ip.49357 > remote.server.ip.80: . ack 1
> win 260 (DF)
> 20:38:25.251037 my.workstation.ip.49357 > remote.server.ip.80: P
> 1:403(402) ack 1 win 260 (DF)
> 20:38:25.567602 my.workstation.ip.49357 > remote.server.ip.80: P
> 1:403(402) ack 1 win 260 (DF)
> 20:38:25.568042 remote.server.ip.80 > my.workstation.ip.49357: . ack
> 403 win 63838 (DF)
> (s

Baffling problem with OBSD-protected servers and Windows Vista...

2006-11-26 Thread Reverend Deuce 

(This is very long email because it's a very complicated problem...
I've included some tcpdump logs below to assist...)

The last week and days I've been working with the RTM version of Vista
obtained through my MSDN license. This is the "gold" version of
Windows Vista, BTW. It's done. It's been shipped to manufacturing
(hence RTM, "release to manufacturing").

Okay, so I've installed this thing and am testing out all the bells
and whistles.

I install Firefox, OpenVPN, putty, the Java JRE from Sun, etc.

I start to tool around and I notice that none of our company's web
sites will load in Firefox any longer. Firefox's status bar says
"Waiting for www.site.tld..." and eventually will time out. It does
this for every single web site we host.

I fire up IE 7. No problems with any site.

I go back to Firefox and it's still having issues -- but *only* with
sites hosted behind our OpenBSD firewalls.

I fire up "telnet" (after enabling it through the control panel -- no
idea why MS did this). I can telnet to our web servers via port 80,
issue "GET" requests, receive responses. No trouble with Telnet.exe.

Putty however, has trouble. Wont work period. Port 80 telnet, ssh port
22, etc. None of them.

So now I am thinking that it might just be a Firefox problem... but
it's not. Microsoft's own Remote Desktop Connection (terminal services
client, rdp client, etc) wont connect to our datacenter servers -- and
they are accessed via an openvpn point to point VPN that terminates on
the OpenBSD firewall which acts strictly as a routed tunnel between
our two networks.

I turn off as much of the Vista "security" features as I can. This does nothing.

Since our OBSD firewalls were of the older variety (3.6), I figured I
might try an upgrade to 4.0 to see what happens. No dice.

To summarize:

This **only** is affecting Windows Vista (have not tried the latest
betas of Longhorn Server). Windows XP, Windows 2000, Free/OpenBSD,
CentOS, and our four Mac users with OSX have zero trouble. None. Nada.
They work flawlessly.

Okay, so we can blame Vista -- that would be fine with me, but let's
face it -- this going to be big come January. I have a month to fix
this damn thing and I am really out of ideas.

Our network:

100mbit dedicated inet connection through AT&T, terminates to a big
Cisco setup owned by our datacenter.

Firewalls are now OBSD 4.0, single-proc Xeon 2.4gHz, 1GB RAM, etc. --
they are decent systems with six gigabit NICs each.

They are all configured with CARP and pfsync. This has worked very,
very well since day 1 in 2004! CARP rocks!

They connect to an HP ProCurve 5400ZL modular switch, configured with
various port VLANs, etc. Everything is gigabit, 'cept for a few
databases using 10-gigabit CX4.

Here are some tcpdumps from the master FW during connection attempts
with a browser:



Opera 9:

20:40:45.824144 my.workstation.ip.49370 > remote.server.ip.80: S
1215871830:1215871830(0) win 8192  (DF)
20:40:45.824646 207.218.64.33.80 > my.workstation.ip.49370: S
2582857930:2582857930(0) ack 1215871831 win 64240 
20:40:45.878361 my.workstation.ip.49370 > 207.218.64.33.80: . ack 1 win 260 (DF)
20:40:45.904597 my.workstation.ip.49370 > 207.218.64.33.80: P
1:384(383) ack 1 win 260 (DF)
20:40:46.058234 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
win 63857 (DF)
20:40:46.061253 my.workstation.ip.49370 > 207.218.64.33.80: P
1:384(383) ack 1 win 260 (DF)
20:40:46.061726 207.218.64.33.80 > my.workstation.ip.49370: . ack 384
win 63857 (DF)
(at this point, the connection is hung -- the Vista workstation
receives no further communcations -- it's like it just drops the
replies)



Firefox:

20:38:25.197691 my.workstation.ip.49357 > remote.server.ip.80: S
643900711:643900711(0) win 8192 
(DF)
20:38:25.198320 remote.server.ip.80 > my.workstation.ip.49357: S
852828096:852828096(0) ack 643900712 win 64240 
20:38:25.244540 my.workstation.ip.49357 > remote.server.ip.80: . ack 1
win 260 (DF)
20:38:25.251037 my.workstation.ip.49357 > remote.server.ip.80: P
1:403(402) ack 1 win 260 (DF)
20:38:25.567602 my.workstation.ip.49357 > remote.server.ip.80: P
1:403(402) ack 1 win 260 (DF)
20:38:25.568042 remote.server.ip.80 > my.workstation.ip.49357: . ack
403 win 63838 (DF)
(same deal -- it just seems to die right here)



IE 7:

20:39:08.834465 my.workstation.ip.49358 > remote.server.ip.80: S
4155969795:4155969795(0) win 8192  (DF)
20:39:08.835095 remote.server.ip.80 > my.workstation.ip.49358: S
3294485308:3294485308(0) ack 4155969796 win 64240 
20:39:08.892057 my.workstation.ip.49358 > remote.server.ip.80: . ack 1
win 16685 (DF)
20:39:08.904548 my.workstation.ip.49358 > remote.server.ip.80: P
1:472(471) ack 1 win 16685 (DF)
20:39:08.907010 remote.server.ip.80 > my.workstation.ip.49358: .
1:1381(1380) ack 472 win 63769 (DF)
20:39:08.907135 remote.server.ip.80 > my.workstation.ip.49358: .
1381:2761(1380) ack 472 win 63769 (DF)
20:39:08.959016 my.workstation.ip.49358 > remote.server.ip.80: . ack
2761 win 16685 (DF)
20:39:08.959740 remote