This is interesting!
I have diagnosed the issue thus far:
ftpproxy has issues with some directories..... that are large from FTP
servers.
I thought I had run into one of these:
CAVEATS
pf(4) does not allow the ruleset to be modified if the system is
running
at a securelevel(7) higher than 1. At that level ftp-proxy cannot add
rules to the anchors and FTP data connections may get blocked.
Negotiated data connection ports below 1024 are not allowed.
The negotiated IP address for active modes is ignored for security
reasons. This makes third party file transfers impossible.
Since ftp-proxy acts as a man-in-the-middle it breaks explicit FTP TLS
connections (RFC 4217).
however, it doesn't seem to be the case; I think it is something to do
with a timeout somewhere in that the server closes the connection early
before the client has a chance to list the directory:
-t timeout
Number of seconds that the control connection can be idle,
before
the proxy will disconnect. The maximum is 86400 seconds,
which
is also the default. Do not set this too low, because the
control connection is usually idle when large data
transfers are
taking place.
-> though it seems not the ftp-proxy timeout as that is set to 86400 by
default.
Not using the proxy is completely possible as PF seems to handle
outbound client FTP sessions fine, however, tracking the port usage, it
looks as though even if port 21 is opened within the firewall the FTP
connection can be made but if other ports are being negotiated in the
background eg:
# tcpdump -eni pflog0 net 129.250.47.99
tcpdump: WARNING: snaplen raised from 116 to 160
tcpdump: listening on pflog0, link-type PFLOG
11:13:23.613052 rule 823/(match) pass out on pppoe0: <***>.52444 >
129.250.47.99.21: S 1808394633:1808394633(0) win 29200 <mss
1460,sackOK,timestamp 21801766 0,nop,wscale 7>
11:13:25.026958 rule 823/(match) pass out on pppoe0: <***>.56190 >
129.250.47.99.1070: S 1729383519:1729383519(0) win 29200 <mss
1460,sackOK,timestamp 21802190 0,nop,wscale 7>
of course the dynamic port allocation won't work if everything else is
being blocked.
This leads to the question: is there a way to handle opening up extra
ports in PF after the condition of "pass in quick to port 21" has been met??
passive [on | off]
Toggle passive mode. If passive mode is turned on
(default
is on), ftp will send a EPSV command for all data
connections
instead of the usual PORT command. The PASV command
requests
that the remote server open a port for the data connection
and return the address of that port. The remote server
listens on that port and the client connects to it. When
using the more traditional PORT command, the client
listens
on a port and sends that address to the remote server, who
connects back to it. Passive mode is useful when
using ftp
through a gateway router or host that controls the
directionality of traffic. (Note that though FTP
servers are
required to support the PASV command by RFC 1123, some do
not.)
Or simply attempt to use Active ftp on port 20 - though this seems to be
now legacy....
---K
On 01/29/2017 04:14 PM, Kaya Saman wrote:
Hi,
as I have now managed to get back into my OBSD system I have tested
ftp again from clients yet still even with the recent upgrade I seem
to be unable to list certain directories??
I have checked the option flags for ftpproxy (man ftpproxy) however,
couldn't find anything of help.
An example is the Shrubbery ftp site:
this directory lists fine: ftp://ftp.shrubbery.net/pub/astraceroute/
while this one doesn't list at all?? ftp://ftp.shrubbery.net/pub/rancid/
tcpdump -eni doesn't show anythng specific:
16:01:40.755991 10:0b:a9:5c:a4:2c 00:25:90:d0:ba:00 0800 66:
<***>.54958 > 129.250.47.99.21: . ack 2344340832 win 229
<nop,nop,timestamp 1070904 3015288354> (DF)
16:01:40.756004 00:25:90:d0:ba:00 10:0b:a9:5c:a4:2c 0800 66:
129.250.47.99.21 > <***>.54958: . ack 1 win 271 <nop,nop,timestamp
3015288444 0>
16:01:42.938689 10:0b:a9:5c:a4:2c 00:25:90:d0:ba:00 0800 74:
<***>.54962 > 129.250.47.99.21: S 3566867037:3566867037(0) win 29200
<mss 1460,sackOK,timestamp 1071558 0,nop,wscale 7> (DF)
16:01:42.938725 00:25:90:d0:ba:00 10:0b:a9:5c:a4:2c 0800 78:
129.250.47.99.21 > <***>.54962: S 3753545844:3753545844(0) ack
3566867038 win 16384 <mss 1460,nop,nop,sackOK,nop,wscale
6,nop,nop,timestamp 2390749407 1071558>
<snip>
16:02:04.356644 00:25:90:d0:ba:00 10:0b:a9:5c:a4:2c 0800 120:
129.250.47.99.21 > <***>.54974: P 401:455(54) ack 112 win 271
<nop,nop,timestamp 2400821070 1077862>
16:02:04.357548 10:0b:a9:5c:a4:2c 00:25:90:d0:ba:00 0800 66:
<***>.54974 > 129.250.47.99.21: . ack 455 win 229 <nop,nop,timestamp
1077984 2400821070> (DF)
--> then the report waits
I'm just wondering as the last statement is (DF) - the Do Not Fragment
flag, could this be an MTU issue??
Though the odd thing is that this was working fine until two upgrades
ago which are still quite recent 2017 dated.
In PF I have the standard rule:
# Proxy rules
anchor "ftp-proxy/*"
pass in quick on $int_if inet proto tcp to any port ftp \
divert-to 127.0.0.1 port 8021
And as I'm on a VDSL2 line I have:
net.inet.tcp.mssdflt=1452
in sysctl.conf
and the MTU set on the PPPoE interface as: mtu 1492
All other web based traffic operates fine apart from certain ftp
transactions??
Would anyone be able to suggest anything?
Many Thanks.
Kaya
