Hi Ingo, et al,
I really appreciate all the positive attitudes towards analying this
particular item. This advances OpenBSD with each small step.
On 15/03/2025 20:00, Ingo Schwarze wrote:
Hello,
Van Dung Ha wrote on Wed, Mar 12, 2025 at 02:25:02PM +0100:
[...]
Recently, I was parsing a few IP addresses from the snort logs
to populate a pf table and encountered something counter-intuitive.
Here is an example source list.
[...]
77.224.14.2
77.224.14.21
77.224.14.18
77.224.14.21
When you sort this list using '| sort -u', you will end up with the
following, expected list.
[...]
The weird thing occurs that when you filter the same source list
using '| sort -hu', you end up with this shorter list
[...]
Notice that this list is missing 77.224.14.2 and 77.224.14.21!
Is this by design?
Others explained why this behaviour is required by POSIX and i just
committed a patch improving our manual page, explaining what the key
used by -u is for the three numerical sort options.
My first try to get what you want was
$ sort -nut . -k1 -k2 -k3 -k4 test.in
but that does not work either, among other things resulting in the
wrong ordering
77.224.14.18
77.224.14.2
77.224.14.21
apparently because both OpenBSD sort(1) and GNU coreutils gsort(1)
intepret these lines as two fields of one floating point number each,
with 14.18 < 14.2 < 14.21, and ignore the -k3 and -k4 options.
I fail to see how our sort(1) manual describes that behaviour,
or how the POSIX specification mandates it, and i do not see an
explanation in the info(1) manual of sort(1) for GNU coreutils
either. So our sort(1) manual can still be improved.
The easiest way i found to do what you want is
$ tr . s < test.in | sort -nuts -k1 -k2 -k3 -k4 | tr s .
After some head scratching, I finally understand what you are doing
here. Please correct me if I am wrong, but it looks like you chop the IP
address into for pieces and use 'sort -nuts' to filter them and finally
reassemble it back to an IP address. Simply brilliant!
nuts in the Karakorum!
Yours,
Ingo
