Hi Jean-Laurent,
Op 25-6-2016 om 18:46 schreef Jean-Laurent Ivars:
You’re perfectly right i only gave extracts of the logs :) next time I’m going
to be more precise and make the capture from the beginning...
Its not that extracts are bad, just knowing what part to extract to make
it easier to compare on 1 screen.
I cut your tcpdumps a bit shorter below after a few packets ;) should
still have all relevant info for the current issue, if anyone else wants
to go over them.
Anyway, as you suppose, I continue to see the same [S] paquet repeating again
and again…
The client tries to connect, but never gets the [S.] reply packet. As
such its does the right thing to try again a few times..
These captures have been done directly on a WAN interface (not on the LAN one)
the reason you see a private address is because there is a double NAT...
Double nat is in general a bad thing. I assumed LAN because of private
ip's.. Anyway are all locations home/business connections where you
physically manage the router that connects to the isp? Not like inside a
OVH datacenter.? (i know OVH does some strange things in the DC..) Just
for ruling some problem in them out you've tried restarting all those
router/modem devices? Might not help at all, if it does whel that would
indicate a issue there..
Also i assume all the pfSense boxes can properly use the internet
themselves without any issue?
As you ask, I am going to send you :
- tcpdump on a client for the port 2223
- tcpdump on the pfsense on the port 2223 choosing the interface that is
directly connected to the modem-router
1- From the pfsense WAN interface point of view with a not working gateway from
the begining until the end (time out from the client side) :
[2.3.1-RELEASE][r...@pfsense.concorde-pereire.loc]/root: tcpdump -i re0 port
2223
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on re0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:01:51.206180 IP 196.222.21.93.rev.sfr.net.51548 > 192.168.101.254.2223:
Flags [S], seq 3384870708, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS
val 333778453 ecr 0,sackOK,eol], length 0
18:01:51.206358 IP 192.168.101.254.2223 > 196.222.21.93.rev.sfr.net.51548:
Flags [S.], seq 2254053748, ack 3384870709, win 65228, options [mss
1460,nop,wscale 7,sackOK,TS val 1269164786 ecr 333778453], length 0
18:01:52.214077 IP 196.222.21.93.rev.sfr.net.51548 > 192.168.101.254.2223:
Flags [S], seq 3384870708, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS
val 333779453 ecr 0,sackOK,eol], length 0
18:01:52.214177 IP 192.168.101.254.2223 > 196.222.21.93.rev.sfr.net.51548:
Flags [S.], seq 2254053748, ack 3384870709, win 65228, options [mss
1460,nop,wscale 7,sackOK,TS val 1269164786 ecr 333779453], length 0
The question is where/why does this reply [S.] packet get dropped
between pfsense and client machine.
Can you try above tcpdump 1. again with -en flags?
*tcpdump -en -i re0 port 2223 *
Only 4 packets should be enough.. Basically for checking if
source/destination MAC addresses are the same..
2- From the client point of view to the not working gateway from the begining
until the end (time out from the client side) :
root@MacBook-de-Jean-Laurent:~$ tcpdump -i en0 port 2223
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:01:51.196869 IP 192.168.10.50.51548 >
20-98-190-109.dsl.ovh.fr.rockwell-csp3: Flags [S], seq 3384870708, win 65535,
options [mss 1460,nop,wscale 5,nop,nop,TS val 333778453 ecr 0,sackOK,eol], length 0
18:01:52.206194 IP 192.168.10.50.51548 >
20-98-190-109.dsl.ovh.fr.rockwell-csp3: Flags [S], seq 3384870708, win 65535,
options [mss 1460,nop,wscale 5,nop,nop,TS val 333779453 ecr 0,sackOK,eol], length 0
Client never gets the response packets.. As such keeps trying again for
a while.
3- From the pfsense WAN interface point of view with a working gateway from the
begining until the end (connexion then a few commands and deconnexion) :
[2.3.1-RELEASE][r...@pfsense.concorde-pereire.loc]/root: tcpdump -i re0 port
2223
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on re0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:09:12.648018 IP 196.222.21.93.rev.sfr.net.51615 > 192.168.101.254.2223:
Flags [S], seq 4029864830, win 65535, options [mss 1460,wscale 6,sackOK,eol],
length 0
18:09:12.648159 IP 192.168.101.254.2223 > 196.222.21.93.rev.sfr.net.51615:
Flags [S.], seq 2831647839, ack 4029864831, win 65228, options [mss
1460,nop,wscale 7,sackOK,eol], length 0
18:09:12.687456 IP 196.222.21.93.rev.sfr.net.51615 > 192.168.101.254.2223:
Flags [.], ack 1, win 8192, length 0
18:09:12.687607 IP 196.222.21.93.rev.sfr.net.51615 > 192.168.101.254.2223:
Flags [P.], seq 1:22, ack 1, win 8192, length 21
18:09:12.687653 IP 192.168.101.254.2223 > 196.222.21.93.rev.sfr.net.51615:
Flags [.], ack 22, win 513, length 0
18:09:12.737138 IP 192.168.101.254.2223 > 196.222.21.93.rev.sfr.net.51615:
Flags [P.], seq 1:22, ack 22, win 513, length 21
4- From the client point of view to the working WAN access from the begining
until the end (connexion then a few commands and deconnexion) :
root@MacBook-de-Jean-Laurent:~$ tcpdump -i en0 port 2223
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:09:12.626154 IP 192.168.10.50.51615 >
184.196.154.77.rev.sfr.net.rockwell-csp3: Flags [S], seq 4029864830, win 65535,
options [mss 1460,nop,wscale 5,nop,nop,TS val 334217159 ecr 0,sackOK,eol], length 0
18:09:12.666015 IP 184.196.154.77.rev.sfr.net.rockwell-csp3 >
192.168.10.50.51615: Flags [S.], seq 2831647839, ack 4029864831, win 65535,
options [mss 1460,wscale 8,sackOK,eol], length 0
18:09:12.666099 IP 192.168.10.50.51615 >
184.196.154.77.rev.sfr.net.rockwell-csp3: Flags [.], ack 1, win 8192, length 0
18:09:12.666537 IP 192.168.10.50.51615 >
184.196.154.77.rev.sfr.net.rockwell-csp3: Flags [P.], seq 1:22, ack 1, win 8192,
length 21
18:09:12.705914 IP 184.196.154.77.rev.sfr.net.rockwell-csp3 >
192.168.10.50.51615: Flags [.], ack 22, win 513, length 0
18:09:12.754171 IP 184.196.154.77.rev.sfr.net.rockwell-csp3 >
192.168.10.50.51615: Flags [P.], seq 1:22, ack 22, win 513, length 21
Both sides from captures 3 and 4 show each packet arriving as such it
all works fine ;)
I hope these complete captures can tell you more what’s happening ?
I have no special packages installed, had only iperf and nmap for testing
purposes… and just uninstalled but it changed nothing :(
I have absolutely not any exotic or special configuration like limiters/shapers
and absolutely no VPN at the moment, no IPSEC, OPENVPN nor anything else… on
the other pfsense box I have openvpn installed but the symptoms aren’t
different so I don’t think it’s related.
I really don’t understand what’s happening here but the fact that on both my
pfsenses I have some WAN redirections or DMZ that are working and that are not
the only factor that changes here is the internet provider and maybe it’s not
hardware related but I would think that it’s some kind of network setting
somewhere that could help me. And we should not forget that before the upgrade
on the version 2.3 it was working on one of the box for sure ! (the other
wasn’t installed at the moment)
In my opinion it’s something silly like mtu for exemple but making my life a
hell !
Mtu can be troublesome, but i think the syn-ack would make it through
even if the mtu is a bit off.. Anyway MTU seems to be a proper 1500 at
the ovh side.
I know it’s a long long mail with a lot of informations but if someone can help
me with that I would be so much grateful !!!
Thank you for those who are reading :)
Best regards,
Jean-Laurent Ivars
Responsable Technique | Technical Manager
22, rue Robert - 13007 Marseille
Tel: 09 84 56 64 30 - Mobile: 06.52.60.86.47
Linkedin <http://fr.linkedin.com/in/jlivars/> | Viadeo
<http://www.viadeo.com/fr/profile/jean-laurent.ivars> | www.ipgenius.fr
<https://www.ipgenius.fr/>
Not really sure what the issue is just yet.
Hope some of my comments help get you closer ;)
You might also try calling ovh and try to see if maybe they are blocking
something.?.
Regards,
PiBa-NL
Le 25 juin 2016 à 16:47, PiBa <pba_...@yahoo.com> a écrit :
Hi Jean-Laurent,
Op 25-6-2016 om 10:37 schreef Jean-Laurent Ivars:
This is logs generated by tcpdump from the same client machine when I try to
access the firewall thru working internet access provider :
port 2223
16:55:04.501509 IP 46.105.230.225.39304 > 192.168.101.254.2223: Flags [P.], seq
29:701, ack 22, win 32844, length 672
16:55:04.501652 IP 192.168.101.254.2223 > 46.105.230.225.39304: Flags [P.], seq
22:910, ack 701, win 508, length 888
port 10000
16:58:51.821691 IP 192.168.101.254.10000 > 46.105.230.225.5829: Flags [P.], seq
209411:210119, ack 2393, win 513, length 708
16:58:52.058014 IP 46.105.230.225.5829 > 192.168.101.254.10000: Flags [.], ack
210119, win 32673, length 0
Both these captures are in the 'middle' of a already working connection, but
does show it 'works' at that time.. To compare it to the capture below better
start capturing before initiation the connection :) .
And there the same command output when I try to access from one that is not
working :
Port 2223
16:53:13.240166 IP 46.105.230.225.19480 > 192.168.101.254.2223: Flags [S], seq
3864438539, win 8192, options [mss 1460,nop,nop,sackOK], length 0
16:53:13.240306 IP 192.168.101.254.2223 > 46.105.230.225.19480: Flags [S.], seq
2492220538, ack 3864438540, win 65228, options [mss 1460,nop,wscale 7,sackOK,eol],
length 0
Port 10000
16:56:39.864021 IP 46.105.230.225.41932 > 192.168.101.254.10000: Flags [S], seq
2837326484, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:56:39.864169 IP 192.168.101.254.10000 > 46.105.230.225.41932: Flags [S.],
seq 1993261464, ack 2837326485, win 65228, options [mss 1460,nop,wscale
7,sackOK,eol], length 0
These SYN [S] and SYN-ACK [S.] packets are the normal start of a tcp
connection. Sofar nothing strange during connection initiation, there should
follow a 3rd ACK [.] packet after which there would follow some useful data
exchange like in the first working captures.. But i guess you keep seeing these
same two [S] and [S.] packets repeating.?
You have performed these captures on the LAN interface, could you repeat them
on the WAN interface to confirm that the [S.] is properly send back on that
same interface the client send the request on? Also if its possible to capture
on the client device it would be interesting to see if the [S.] arrives there
properly.
I use pcengine APU system, the model is AMD G-T40E Processor with 3 NIC ( I
believe It could be something related to a NIC setting somewhere but really
don’t know)
Is someone encounter the same issue than me ? maybe it’s just a setting in the
NIC driver ?
I don't expect it to be hardware/driver related at this moment..
Do you have any packages installed? Snort or Suricata can sometimes
unexpectedly block traffic you do want.. Or other configurations like
limiters/shapers or openvpn/ipsec networks can possibly interfere..
Regards,
PiBa-NL
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
