Re: sockaddr_storage

[Eric S. Raymond]

Hal Murray :
> 
> e...@thyrsus.com said:
> >> Are there any places left in the code that are storing addresses in
> >> packed-4-octets or ints?
> > These are areas I will have to investigate.  
> 
> I think the storage is all sockaddr_u
> 
> I think all the printout goes through socktoa in libntp/socktoa.c

That's what I think, too, but I haven't verified it yet.

> I will punt any DNS changes until after your work in this area.

I don't think you need to, or should.

"Need to" because the DNS-threading stuff is pretty well sparated from
anywhere the IPv4/IPv6 stuff matters.  "Should" because removing sockaddr_u
has fallen way down my priority list. Because of the footprint issue I
probably won't do it all except as part of the immediate run-up to a
Go translation of the code, assuming we decide to go that route.

> If ntpdig is no longer around, is there any reason for libntp?

Two minor ones.  One is that part of libntp is used to build the Python
extension used by ntpq, etc. The other is that there's a separation
between ntpd and libntp test code parallel to the library split -
not a problem per se but it increases the friction cost of reorganizing.

It's on my to-do list to try moving one of the libntp modules that isn't
used in the Python extension over to ntpd/ as a test of how much trouble it
is.  If it's no big deal I'll move everything not used in the Python
extension.

BTW, I've successfully turfed out the ISC MD5 code.  This dropped about
300LOC.  Still need to get rid of the local SHA-1 and remove the
libsodium dependency.
-- 
http://www.catb.org/~esr/;>Eric S. Raymond
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Hal Murray]

e...@thyrsus.com said:
>> Are there any places left in the code that are storing addresses in
>> packed-4-octets or ints?
> These are areas I will have to investigate.  

I think the storage is all sockaddr_u

I think all the printout goes through socktoa in libntp/socktoa.c

The input side is 2 types.  The first is numeric host addresses. The second 
is IPv4 and IPv6 variants of DNS name lookup.  They come from the command 
line and server and pool lines in ntp.conf.  There may be others that I'm 
overlooking.

I will punt any DNS changes until after your work in this area.  (or get back 
to you when I'm ready)  The thread stuff is scattered wider than I had 
anticipated.  I'll have to poke around a bit before I actually start changing 
anything.

-

There are various comments about ntpd or ntpdig.  I assume the ntpdig part 
can be cleaned up.

If ntpdig is no longer around, is there any reason for libntp?


-- 
These are my opinions.  I hate spam.



___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Eric S. Raymond]

Mark Atwood :
> I like the idea of removing more IPv4 specific code.
> 
> If we do, can we still get arrival timestamp from the kernel?

I'm pretty sure the answer is "yes".  Checking...it doesn't look like
that code has any dependency on the address family.

> How do standard display functions display v6 mapped v4 addresses?   How do
> we want them displayed?
> 
> Are there any places left in the code that are storing addresses in
> packed-4-octets or ints?

These are areas I will have to investigate. 
-- 
http://www.catb.org/~esr/;>Eric S. Raymond
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Mark Atwood]

I like the idea of removing more IPv4 specific code.

If we do, can we still get arrival timestamp from the kernel?

How do standard display functions display v6 mapped v4 addresses?   How do
we want them displayed?

Are there any places left in the code that are storing addresses in
packed-4-octets or ints?

..m

On Fri, Jan 27, 2017 at 12:16 PM Gary E. Miller  wrote:

> Yo Eric!
>
> On Fri, 27 Jan 2017 15:01:19 -0500
> "Eric S. Raymond"  wrote:
>
> > Gary E. Miller :
> > > Yes, but NTP does not have to.  NTP can just open an IPv6 socket and
> > > shove all IPv6 and IPv4 in through that socket.  Apache does this,
> > > sendmail does this, nginx does this, postfix does this.  Very
> > > standard and very easy.
> >
> > That's interesting.  Does this mean we could throw out all
> > IPV4-specific code and use IPV6 logic everywhere?
>
> Yes.
>
> > I didn't think an IPv6 accept() would take incoming IPv4 connections
>
> Yes.
>
> > - is there a way to make it take both?
>
> It does by default, if you only open an IPv6 socket.
>
> Look at 'man 7 ipv6'
>
> Or here: http://man7.org/linux/man-pages/man7/ipv6.7.html
>
> "IPv4 connections can be handled with the v6 API by using the
> v4-mapped-on-v6 address type; thus a program needs to support
> only this API type to support both protocols.  This is handled
> transparently by the address handling functions in the C library."
>
>
> RGDS
> GARY
> ---
> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
> g...@rellim.com  Tel:+1 541 382 8588 <(541)%20382-8588>
>
> Veritas liberabit vos. -- Quid est veritas?
> "If you can’t measure it, you can’t improve it." - Lord Kelvin
> ___
> devel mailing list
> devel@ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Gary E. Miller]

Yo Eric!

On Fri, 27 Jan 2017 15:01:19 -0500
"Eric S. Raymond"  wrote:

> Gary E. Miller :
> > Yes, but NTP does not have to.  NTP can just open an IPv6 socket and
> > shove all IPv6 and IPv4 in through that socket.  Apache does this, 
> > sendmail does this, nginx does this, postfix does this.  Very
> > standard and very easy.  
> 
> That's interesting.  Does this mean we could throw out all
> IPV4-specific code and use IPV6 logic everywhere?

Yes.

> I didn't think an IPv6 accept() would take incoming IPv4 connections

Yes.

> - is there a way to make it take both?

It does by default, if you only open an IPv6 socket.

Look at 'man 7 ipv6'

Or here: http://man7.org/linux/man-pages/man7/ipv6.7.html

"IPv4 connections can be handled with the v6 API by using the
v4-mapped-on-v6 address type; thus a program needs to support
only this API type to support both protocols.  This is handled
transparently by the address handling functions in the C library."


RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgpXXbVTIZkPh.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Eric S. Raymond]

Gary E. Miller :
> Yes, but NTP does not have to.  NTP can just open an IPv6 socket and
> shove all IPv6 and IPv4 in through that socket.  Apache does this, 
> sendmail does this, nginx does this, postfix does this.  Very standard
> and very easy.

That's interesting.  Does this mean we could throw out all IPV4-specific code
and use IPV6 logic everywhere?  I didn't think an IPv6 accept() would take
incoming IPv4 connections - is there a way to make it take both?
-- 
http://www.catb.org/~esr/;>Eric S. Raymond


signature.asc
Description: PGP signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Gary E. Miller]

Yo Hal!

On Fri, 27 Jan 2017 01:14:25 -0800
Hal Murray  wrote:

> fallenpega...@gmail.com said:
> > I prefer :: tagged IPv6 addresses for IPv4 addresses.   No need
> > for a redundant flag.   
> 
> I don't think that actually works.  We need to know if it really is
> an IPv4 address or is an IPv4 address reached via IPv6.

There is no difference between those two.  Just a different way
to encode the same thing.

>  Somebody has
> to switch the packet between IPv6 and IPv4.

Yes, but NTP does not have to.  NTP can just open an IPv6 socket and
shove all IPv6 and IPv4 in through that socket.  Apache does this, 
sendmail does this, nginx does this, postfix does this.  Very standard
and very easy.

>  We need to know if that
> is us or some router/gateway out on the internet.

Which is the job of the local OS routing table.  Daemons do not need to
know routing.  Unless, of coure, they are routing daemons.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgp8_QNb2vhfp.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Hal Murray]

fallenpega...@gmail.com said:
> I prefer :: tagged IPv6 addresses for IPv4 addresses.   No need for a
> redundant flag. 

I don't think that actually works.  We need to know if it really is an IPv4 
address or is an IPv4 address reached via IPv6.  Somebody has to switch the 
packet between IPv6 and IPv4.  We need to know if that is us or some 
router/gateway out on the internet.



-- 
These are my opinions.  I hate spam.



___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Mark Atwood]

I prefer :: tagged IPv6 addresses for IPv4 addresses.   No need for a
redundant flag.

On Mon, Jan 23, 2017 at 12:48 PM Gary E. Miller  wrote:

> Yo Hal!
>
> On Sun, 22 Jan 2017 20:10:36 -0800
> Hal Murray  wrote:
>
> > All the MRU list needs is a flag to specify if the IP address
> > is 4 or 6, the address, and the port number.
>
> Or just use IPv4 addresses mapped into IPv6.  From wikipedia:
>
> https://en.wikipedia.org/wiki/IPv6_address
>
> You place the prefix an IPv4 address with :::
>
> 192.0.2.128 becomes :::c000:0280 also written as :::192.0.2.128
>
> RGDS
> GARY
> ---
> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
> g...@rellim.com  Tel:+1 541 382 8588 <(541)%20382-8588>
>
> Veritas liberabit vos. -- Quid est veritas?
> "If you can’t measure it, you can’t improve it." - Lord Kelvin
> ___
> devel mailing list
> devel@ntpsec.org
> http://lists.ntpsec.org/mailman/listinfo/devel
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Gary E. Miller]

Yo Hal!

On Sun, 22 Jan 2017 20:10:36 -0800
Hal Murray  wrote:

> All the MRU list needs is a flag to specify if the IP address
> is 4 or 6, the address, and the port number.

Or just use IPv4 addresses mapped into IPv6.  From wikipedia:

https://en.wikipedia.org/wiki/IPv6_address

You place the prefix an IPv4 address with ::: 

192.0.2.128 becomes :::c000:0280 also written as :::192.0.2.128

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgp0Hl9YKnJmN.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Eric S. Raymond]

Hal Murray <hmur...@megapathdsl.net>:
> 
> > First, to eliminate a union from our source code. Second, to move from a
> > private type to a POSIX-standardized one that is likely to have a binding in
> > any language we move to. 
> 
> Just because it's POSIX doesn't mean it's any good.  Unless there is stylized 
> code for using sockaddr_storage, I would try hard to avoid it.  There is 
> nothing in there to help the type checker.

Good point.

> If the only problem with space is in the mrulist, we can fix that easily.  
> All the MRU list needs is a flag to specify if the IP address is 4 or 6, the 
> address, and the port number.  There is extra cruft in the sockaddr_in6 so it 
> wouldn't cost any space to use a non-union there.  We could save the space of 
> the IPv4 address with a bit of kludgery.

This might be worth doing for space-reduction reasons.  I'll poke at it.

> Barf.  The example URL you provided looked horrible to me.

This is me not arguing with your revulsion. :-)

> I forget what they called it, but Mesa had a sane way of handling things like 
> network packets.  The basic idea is that there was a way to select which 
> branch of the union you wanted to use, and you told the compiler about it.  
> Then you could do a select and each branch of the select could only access 
> the appropriate fields.

It's called a tagged union.  Rust has these. Go doesn't.
-- 
http://www.catb.org/~esr/;>Eric S. Raymond
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Hal Murray]

> First, to eliminate a union from our source code. Second, to move from a
> private type to a POSIX-standardized one that is likely to have a binding in
> any language we move to. 

Just because it's POSIX doesn't mean it's any good.  Unless there is stylized 
code for using sockaddr_storage, I would try hard to avoid it.  There is 
nothing in there to help the type checker.

If the only problem with space is in the mrulist, we can fix that easily.  
All the MRU list needs is a flag to specify if the IP address is 4 or 6, the 
address, and the port number.  There is extra cruft in the sockaddr_in6 so it 
wouldn't cost any space to use a non-union there.  We could save the space of 
the IPv4 address with a bit of kludgery.


e...@thyrsus.com said:
>> unions are a reasonably clean way of handling things like
>> network packets.   How does Go do that?
> With an opaque object structure. I don't know if the packet storage is 

Barf.  The example URL you provided looked horrible to me.

I forget what they called it, but Mesa had a sane way of handling things like 
network packets.  The basic idea is that there was a way to select which 
branch of the union you wanted to use, and you told the compiler about it.  
Then you could do a select and each branch of the select could only access 
the appropriate fields.


-- 
These are my opinions.  I hate spam.



___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: sockaddr_storage

[Eric S. Raymond]

Hal Murray <hmur...@megapathdsl.net>:
> It wasn't obvious from your message that getting rid of sockaddr_u involved 
> replacing it with sockaddr_storage.  Why would you want to use it?  It looks 
> like a step backwards in terms of type checking.

First, to eliminate a union from our source code. Second, to move from
a private type to a POSIX-standardized one that is likely to have a binding
in any language we move to.

> If the goal is to eliminate unions, you might be able to do it with your own 
> clone of sockaddr_storage that is only as long as you need it.
> 
> You could do it squeaky clean without unions by having both ipv4 and ipv6 
> fields in the same struct.  With only a bit of kludgery you could put the 
> ipv4 fields in ipv6 slots.

That's an interesting possibility...

> unions are a reasonably clean way of handling things like network packets.  
> How does Go do that?

With an opaque object structure. I don't know if the packet storage is
unioned internallly or not.  That can't be done in Go itself, but there's
a strange hack with interfaces

http://stackoverflow.com/questions/21553398/best-practice-for-unions-in-go

that can approximate it at the cost of some type safety.

> PS: HAVE_STRUCT_SOCKADDR_STORAGE isn't used by anything.

I'll remove it.  Anything thar can reduce the complexity of the waf recipe
makes me happy.
-- 
http://www.catb.org/~esr/;>Eric S. Raymond
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

sockaddr_storage

[Hal Murray]

e...@thyrsus.com said:
> * Remove sockaddr_storage from our sockaddr_u union of sockaddr,
>   sockaddr_in, and sockaddr_in6, shaving about 100 bytes from its size
>   and substantially decreasing MRU entry memory consumption.

> I interpret this to mean that if we replaced that trailing sockaddr_u with
> sockaddr_storage, the size of an MRU entry would increase by about 100
> bytes. 

I probably went off on a wild goose chase.

It wasn't obvious from your message that getting rid of sockaddr_u involved 
replacing it with sockaddr_storage.  Why would you want to use it?  It looks 
like a step backwards in terms of type checking.

If the goal is to eliminate unions, you might be able to do it with your own 
clone of sockaddr_storage that is only as long as you need it.

You could do it squeaky clean without unions by having both ipv4 and ipv6 
fields in the same struct.  With only a bit of kludgery you could put the 
ipv4 fields in ipv6 slots.

--

unions are a reasonably clean way of handling things like network packets.  
How does Go do that?


PS: HAVE_STRUCT_SOCKADDR_STORAGE isn't used by anything.


-- 
These are my opinions.  I hate spam.



___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel