Re: Checking refusal of a network connection

[Peter J. Holzer]

On 2019-06-03 14:54:29 +0200, Markus Elfring wrote:
> > How would this conversion take place?  Localhost is 127.0.0.1.
> > Localhost6 is ::1.  They are different
> 
> My configuration file “/etc/hosts” provides the following information
> as usual.
> 
> “…
> ::1 localhost ipv6-localhost ipv6-loopback
> …”

The name doesn't matter on the TCP/IP level it is only used to get the
correct IP address(es). 

The line above looks like something from Debian or Ubuntu. If so, you'll
have another line

127.0.0.1   localhost localhost.localdomain

above,

So if you pass the name "localhost" to socket.connect, it will check
/etc/hosts (and possibly DNS and other data sources) and get back a list
of IP addresses, ['127.0.0.1', '::1'] in this case (the order may be
different). It will then try to connect to each of these IP addresses
in turn. 

But if you pass it an IP address (like '::1'), it will connect only to
the IP address you gave it. It won't try to find out if there is a name
associated with that address and whether this name is also associated
with other addresses and try to connect to those.

And the address ::1 is clearly distinct from the address 127.0.0.1, even
if the name "localhost" refers to both and both are bound to the same
interface. 

This is not unusual. If you do a DNS lookup on yahoo.com. you will see
that that name refers to 6 IPv4 and 6 IPv6 addresses - 12 different
addresses for the same name. OTOH I frequently run several webservers on
the same host and when I can, I give them different IP addresses, too. I
think the maximum I've had was over 50 IP addresses on the same
(physical) interface. Naturally when you connect to 192.0.2.23 you don't
want to get connected to the webserver listening on 192.0.2.42 just
because that's the same interface.


> > and you cannot route between the two.
> 
> I got other expectations for the corresponding software behaviour.

You might have to adjust your expectations.


> > What I can see is that your server binds to localhost6 and your client
> > is trying to connect to localhost.
> 
> I am curious to clarify the circumstances further if such a combination
> can also work finally.

Users generally use names, not IP addresses. When you connect to Google,
you use the URL https://google.com, not https://172.217.23.238 or
https://[2a00:1450:4014:80d::200e]. The server listens on both
addresses, the client will try both addresses if IPv6 is available, or
only the IPv4 address if IPv6 isn't available.

(Not sure if this answers your question since I'm not sure what your
question is)


> If my software test client would pass the IPv6 address family for a
> connection, both processes would use the same network protocol
> version.

Yes.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

> How would this conversion take place?  Localhost is 127.0.0.1.
> Localhost6 is ::1.  They are different

My configuration file “/etc/hosts” provides the following information
as usual.

“…
::1 localhost ipv6-localhost ipv6-loopback
…”


> and you cannot route between the two.

I got other expectations for the corresponding software behaviour.


> What I can see is that your server binds to localhost6 and your client
> is trying to connect to localhost.

I am curious to clarify the circumstances further if such a combination
can also work finally.

If my software test client would pass the IPv6 address family for a connection,
both processes would use the same network protocol version.

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Cameron Simpson]

On 01Jun2019 12:57, Dennis Lee Bieber  wrote:

On Sat, 1 Jun 2019 15:33:43 +0200, Markus Elfring 
declaimed the following:

I don't know "strace" (I'd likely be running WireShark to capture all
traffic for investigation).


Sure, but that means you need to winnow it from any other traffic. The 
advantage of strace is that it watches the programme itself, and shows 
(in this case) the network system calls. It is great for seeing what the 
programme is doing/trying to do at the basic level. And, like wireshark, 
language independent - all runtimes have to go through the OS to get 
stuff done.  I've debugged third party java apps this way because the 
issue was a basic one like this.



connect(4, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = 0



connect(3, {sa_family=AF_INET, sin_port=htons(37351), 
sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)


Without seeing the code, I'd be suspicious of that difference. The
latter is asking for a TCP/UDP family connection, whereas the first is a
UNIX domain socket. To my knowledge they are not compatible types.


They're certainly distinct address spaces. In other regards they're 
pretty compatible - you listen/connect the same way. A UNIX socket is 
just IPC within the kernel instead of over the network.


However, the former connect is to the OS name service (hostname lookup - 
it will mediate to the /etc/hosts file, DNS, NIS or whatever other name 
services may be set up). The latter is his connection attempt to his 
service.


So this difference is expected and on the surface it looks correct.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Peter J. Holzer]

On 2019-06-01 20:44:29 +0200, Markus Elfring wrote:
> > Which specific information in that man page contradicts what I wrote?
> 
> We can agree that the mentioned IP addresses are distinct.
> But the corresponding functionality should be equivalent.
> 
> 
> > If you think of
> >
> > | IPv4 connections can be handled with the v6 API by using the
> > | v4-mapped-on-v6 address type; thus a program needs to support only
> > | this API  type to  support  both  protocols.
> >
> > please note that 127.0.0.1 mapped to IPv6 is ::7f00:1, not ::1.

Oops, that should have been :::7f00:1.


> I find another information like “This is handled transparently by
> the address handling functions in the C library.” also interesting.

"Handled transparently" means that an ipv6 server can handle connections
from ipv4 clients without doing anything special. They just appear to
come from a specific IPv6 address range. It doesn't mean the OS performs
random address translations according to user's expectations of
"equivalence".


> > So you still need to bind to two addresses.
> 
> I am unsure about this conclusion.

Well, we don't study theology here. We don't have to theorize (no pun
intended), we can experiment. Why don't you just try it out?


> Under which circumstances will the Python programming interfaces
> support the direct usage of the identification “::1”?

I'm not sure I understand the question. They do.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

> It looks like the service isn't listening at the time the so.connect is 
> called.

* I get an other impression from the programs “/usr/bin/netstat” and 
“/usr/bin/ss”.

* The data transmission seems to work also for my small script 
“socket-send_test_data1.tcl”
  (even when the identification “::1” was passed as a command parameter).

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

> Which specific information in that man page contradicts what I wrote?

We can agree that the mentioned IP addresses are distinct.
But the corresponding functionality should be equivalent.


> If you think of
>
> | IPv4 connections can be handled with the v6 API by using the
> | v4-mapped-on-v6 address type; thus a program needs to support only
> | this API  type to  support  both  protocols.
>
> please note that 127.0.0.1 mapped to IPv6 is ::7f00:1, not ::1.

I find another information like “This is handled transparently by
the address handling functions in the C library.” also interesting.


> So you still need to bind to two addresses.

I am unsure about this conclusion.

Under which circumstances will the Python programming interfaces
support the direct usage of the identification “::1”?

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Peter J. Holzer]

On 2019-06-01 20:22:39 +0200, Markus Elfring wrote:
> >> I would expect that the IPv4 address from such a connection attempt
> >> would be automatically converted to a IPv6 loopback address.
> >
> > You haven't said which OS you are using, but as far as I know this
> > expectation will be frustrated at least on Linux: There ::1 and
> > 127.0.0.1 are distinct addresses.
> 
> How does this view fit to information from the Linux programmer's manual?
> See also: command “man 7 ipv6”

Which specific information in that man page contradicts what what I
wrote?

If you think of 

| IPv4 connections can be handled with the v6 API by using the
| v4-mapped-on-v6 address type; thus a program needs to support only
| this API  type to  support  both  protocols.

please note that 127.0.0.1 mapped to IPv6 is ::7f00:1, not ::1. So you
still need to bind to two addresses.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

>> I would expect that the IPv4 address from such a connection attempt
>> would be automatically converted to a IPv6 loopback address.
>
> You haven't said which OS you are using, but as far as I know this
> expectation will be frustrated at least on Linux: There ::1 and
> 127.0.0.1 are distinct addresses.

How does this view fit to information from the Linux programmer's manual?
See also: command “man 7 ipv6”

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Michael Torrie]

On 06/01/2019 11:15 AM, Markus Elfring wrote:
>>> connect(3, {sa_family=AF_INET, sin_port=htons(37351), 
>>> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)
>>
>>  Without seeing the code, I'd be suspicious of that difference.
> 
> I would expect that the IPv4 address from such a connection attempt
> would be automatically converted to a IPv6 loopback address.

How would this conversion take place?  Localhost is 127.0.0.1.
Localhost6 is ::1.  They are different and you cannot route between the two.

What I can see is that your server binds to localhost6 and your client
is trying to connect to localhost.

> Unfortunately, the direct specification “… socket-send_json_data.py 
> --server_id ::1 …”
> does not work at the moment because of the error message “socket.gaierror: 
> [Errno -9]
> Address family for hostname not supported”.

No idea on that one.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Peter J. Holzer]

On 2019-06-01 19:15:28 +0200, Markus Elfring wrote:
> >> connect(3, {sa_family=AF_INET, sin_port=htons(37351), 
> >> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection 
> >> refused)
> > Without seeing the code, I'd be suspicious of that difference.
> 
> I would expect that the IPv4 address from such a connection attempt
> would be automatically converted to a IPv6 loopback address.

You haven't said which OS you are using, but as far as I know this
expectation will be frustrated at least on Linux: There ::1 and
127.0.0.1 are distinct addresses. If you want to accept connections on
both, you have to listen on both (or on ::, which does accept
connections on all IP addresees - IPv6 and IPv4).

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

>> connect(3, {sa_family=AF_INET, sin_port=htons(37351), 
>> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)
>
>   Without seeing the code, I'd be suspicious of that difference.

I would expect that the IPv4 address from such a connection attempt
would be automatically converted to a IPv6 loopback address.

Unfortunately, the direct specification “… socket-send_json_data.py --server_id 
::1 …”
does not work at the moment because of the error message “socket.gaierror: 
[Errno -9]
Address family for hostname not supported”.

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

> Also, it can be very useful to strace the client process, eg:

Do you find the following background information more helpful
for the desired clarification of unexpected software behaviour?

elfring@Sonne:~/Projekte/Python> LANG=C strace -e trace=network 
/usr/bin/python3 socket-send_json_data.py --server_id localhost --server_port 
37351
Using Python version:
3.7.2 …
socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 3
socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 5
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4
connect(4, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = 0
sendto(4, "\2\0\0\0\r\0\0\0\6\0\0\0hosts\0", 18, MSG_NOSIGNAL, NULL, 0) = 18
recvmsg(4, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="hosts\0", 
iov_len=6}, {iov_base="\310O\3\0\0\0\0\0", iov_len=8}], msg_iovlen=2, 
msg_control=[{cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, 
cmsg_data=[5]}], msg_controllen=20, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_CMSG_CLOEXEC) = 14
connect(3, {sa_family=AF_INET, sin_port=htons(37351), 
sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)
Traceback …:
…
  File "socket-send_json_data.py", line 17, in send_data
so.connect((args.server_id, args.server_port))
ConnectionRefusedError: [Errno 111] Connection refused
+++ exited with 1 +++


> You can also strace the running service process:

I do not observe additional function calls for the TCP client connection
attempt here.


> Also, on the service side it isn't enough to create the service socket,
> you also need to do an accept IIRC.

This should be performed by my implementation of the C++ function “setup”.

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Cameron Simpson]

On 31May2019 17:35, Markus Elfring  wrote:

I can start a service as desired.

elfring@Sonne:~/Projekte/Bau/C++/test-statistic-server1/local> 
./test-statistic-server2 & /usr/bin/ss -t -l -p -H|grep test
[1] 8961
waiting for connections
server_id: localhost
server_port: 35529
LISTEN 0   123  [::1]:35529  [::]:*  
users:(("test-statistic-",pid=8961,fd=3))
elfring@Sonne:~/Projekte/Bau/C++/test-statistic-server1/local> 0 connections 
were handled.


But I wonder about the following error message then.

elfring@Sonne:~/Projekte/Python> /usr/bin/python3 
~/Projekte/Python/socket-send_json_data.py --server_id localhost 
--server_port 35529

Using Python version:
3.7.2 …
Traceback …:
…
 File "/home/elfring/Projekte/Python/socket-send_json_data.py", line 17, in 
send_data
   so.connect((args.server_id, args.server_port))
ConnectionRefusedError: [Errno 111] Connection refused

How should this inter-process communication difficulty be resolved?


It looks like the service isn't listening at the time the so.connect is 
called. Are you doing it before the service is ready?


Otherwise you need to print out the server_id and port, and examine the 
system to see if that address/port is in LISTEN state. Running "netstat 
-an" on the system running the service is a useful way to do this.


Hmm, look like your "ss" command effectively does that.

I'd fall back to the connect then: check that it really is using the 
correct address/port. Print them out.


Also, it can be very useful to strace the client process, eg:

 strace -e trace=network /usr/bin/python3 
~/Projekte/Python/socket-send_json_data.py --server_id localhost --server_port 
35529

You can also strace the running service process:

 strace -e trace=network -p pid-of-service-process-here

to see if it is responding in any way to the client connect.

Also, on the service side it isn't enough to create the service socket, 
you also need to do an accept IIRC. If you're using Python's socket 
library the service classes do that for you.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking refusal of a network connection

[Markus Elfring]

>   Well, providing minimal code samples that produce the problem would be 
> a start.

I prefer an other approach to clarify relevant software configuration 
differences.


>   Otherwise we are just guessing...

I can offer other data before.


> Maybe you have a firewall problem.

I hope not.

I can try another server variant out as expected.


elfring@Sonne:~/Projekte/Python> /usr/bin/python3 test-server2.py &
[1] 14067
elfring@Sonne:~/Projekte/Python> /usr/bin/ss -t -l -p -H|grep python
LISTEN0  5  127.0.0.1:search-agent 0.0.0.0:*
 users:(("python3",pid=14067,fd=3))
elfring@Sonne:~/Projekte/Python> /usr/bin/python3 socket-send_json_data.py 
--server_id localhost --server_port 1234
Using Python version:
3.7.2 (default, Dec 30 2018, 16:18:15) [GCC]
elfring@Sonne:~/Projekte/Python> Result:
…


Can connections work also with a network service address like “[::1]:35529”
(which would be used by the C++ server implementation so far)?
How does the software situation look like for the support of the IPv6 loopback 
address?

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list

Checking refusal of a network connection

[Markus Elfring]

Hello,

I can start a service as desired.

elfring@Sonne:~/Projekte/Bau/C++/test-statistic-server1/local> 
./test-statistic-server2 & /usr/bin/ss -t -l -p -H|grep test
[1] 8961
waiting for connections
server_id: localhost
server_port: 35529
LISTEN 0   123  [::1]:35529  [::]:* 
 users:(("test-statistic-",pid=8961,fd=3))
elfring@Sonne:~/Projekte/Bau/C++/test-statistic-server1/local> 0 connections 
were handled.


But I wonder about the following error message then.

elfring@Sonne:~/Projekte/Python> /usr/bin/python3 
~/Projekte/Python/socket-send_json_data.py --server_id localhost --server_port 
35529
Using Python version:
3.7.2 …
Traceback …:
…
  File "/home/elfring/Projekte/Python/socket-send_json_data.py", line 17, in 
send_data
so.connect((args.server_id, args.server_port))
ConnectionRefusedError: [Errno 111] Connection refused


How should this inter-process communication difficulty be resolved?

Regards,
Markus
-- 
https://mail.python.org/mailman/listinfo/python-list