Iván Ezequiel Rodriguez <[email protected]> writes: > join -i and uniq -i used memcasecmp, which only folds ASCII A-Z, so > in UTF-8 locales É and é did not match despite -i. That matches the > long-standing FIXME in join's keycmp. > > In multibyte locales, compare c32tolower of each mcel character > (same approach as gnulib exclude). Unibyte locales keep the historical > memcasecmp + length behavior. As before, -i does not use LC_COLLATE. > > This is simple codepoint folding, not full Unicode case mapping: > É matches é, but ß and SS remain distinct. > > * src/memcasecmp-mcel.h (memcasecmp_fields): New shared helper. > * src/join.c (keycmp): Use it for -i. > * src/uniq.c (different): Likewise. > * src/local.mk: List the new header. > * tests/join/join.pl: Cover É/é join and ß/SS non-join. > * tests/uniq/uniq.pl: Cover É/é collapse and ß/SS distinct. > * NEWS: Mention the improvement. > > Signed-off-by: Iván Ezequiel Rodriguez <[email protected]> > --- > NEWS | 5 ++++ > src/join.c | 17 +++++--------- > src/local.mk | 1 + > src/memcasecmp-mcel.h | 54 +++++++++++++++++++++++++++++++++++++++++++ > src/uniq.c | 6 ++--- > tests/join/join.pl | 18 +++++++++++++++ > tests/uniq/uniq.pl | 13 ++++++++++- > 7 files changed, 99 insertions(+), 15 deletions(-) > create mode 100644 src/memcasecmp-mcel.h
I think I messaged an equivelent patch for 'uniq -i' privately (or maybe publically, I forget) to Padraig a few months ago. But we decided to hold off for now. Collin
