[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-12-10 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Caroline Cyr La Rose  changed:

   What|Removed |Added

  Documentation||Caroline Cyr La Rose
contact||
 Resolution|--- |FIXED
 Status|Needs documenting   |RESOLVED
  Documentation||https://gitlab.com/koha-com
 submission||munity/koha-manual/-/merge_
   ||requests/1142

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-11-22 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Nick Clemens (kidclamp)  changed:

   What|Removed |Added

 Blocks||41287


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41287
[Bug 41287] Using locale sorting may have a negative impact on search speeds
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-08-01 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Fridolin Somers  changed:

   What|Removed |Added

 Status|Pushed to stable|Needs documenting

--- Comment #98 from Fridolin Somers  ---
Enhancement not pushed to 24.11.x

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-24 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #97 from Paul Derscheid  ---
Nice work everyone!

Pushed to 25.05.x

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-24 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Paul Derscheid  changed:

   What|Removed |Added

 Version(s)|25.11.00|25.11.00,25.05.02
released in||
 Status|Pushed to main  |Pushed to stable

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-11 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Lucas Gass (lukeg)  changed:

   What|Removed |Added

 Status|Passed QA   |Pushed to main
 Version(s)||25.11.00
released in||

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-11 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #96 from Lucas Gass (lukeg)  ---
Nice work everyone!

Pushed to main for 25.11

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Status|Signed Off  |Passed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #95 from Martin Renvoize (ashimema) 
 ---
Created attachment 183887
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183887&action=edit
Bug 36947: (QA follow-up) Add FacetSortingLocale to Search.t mock preferences

The test was failing because the new FacetSortingLocale system preference
added was not included in the mock preference handler in
t/db_dependent/Search.t.
This caused warnings about the unknown preference when the _sort_facets_zebra
function was called.

This commit adds the FacetSortingLocale preference to the mock, returning
'default' as the value, which matches the expected fallback behavior of
the new unicode collation sorting feature.

Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Attachment #183818|0   |1
is obsolete||

--- Comment #94 from Martin Renvoize (ashimema) 
 ---
Created attachment 183886
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183886&action=edit
Bug 36947: (QA follow-up) Add skippable unit test for fi_FI locale

This patch adds a skippable unit test for the fi_FI locale to prove
that the patches work well for other locales that don't use
the default Unicode collation. It will run on koha-testing-docker,
but will be skipped on systems lacking it.

This patch also fleshes out the existing unit tests a bit more
to prove that the patches also improves the sorting of names
with diacritics generally. That is, the default Unicode sort
is better than the existing stringwise/bytewise sort even for
English and French locales.

Test plan:
0. prove -v t/Koha/SearchEngine/Elasticsearch/Search.t \
t/db_dependent/Search_FacetSorting.t

Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Attachment #183568|0   |1
is obsolete||

--- Comment #93 from Martin Renvoize (ashimema) 
 ---
Created attachment 183885
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183885&action=edit
Bug 36947: (QA follow-up) Fix locale-list to show selected values

Signed-off-by: David Cook 
Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Attachment #183567|0   |1
is obsolete||

--- Comment #92 from Martin Renvoize (ashimema) 
 ---
Created attachment 183884
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183884&action=edit
Bug 36947: (QA follow-up) Fix hash key name for locale-list syspref type

Signed-off-by: David Cook 
Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Attachment #183566|0   |1
is obsolete||

--- Comment #91 from Martin Renvoize (ashimema) 
 ---
Created attachment 183883
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183883&action=edit
Bug 36947: (QA follow-up) Add missing POD documentation to Koha::I18N

The Koha::I18N module was missing POD documentation. This adds complete
documentation for all exported functions including usage examples and
descriptions.

Signed-off-by: David Cook 
Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Attachment #183565|0   |1
is obsolete||

--- Comment #90 from Martin Renvoize (ashimema) 
 ---
Created attachment 183882
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183882&action=edit
Bug 36947: Add system preference for configurable locale-based facet sorting

This follow-up adds two major enhancements to the locale-based facet sorting
implementation:

1. **System Preference Integration**
   - Adds FacetSortingLocale system preference for configurable locale
selection
   - Dynamically detects available system locales using Koha::I18N
   - Provides user-friendly dropdown with locale descriptions
   - Supports both Elasticsearch and Zebra search engines

2. **Zebra Search Engine Support**
   - Extends locale-based sorting to Zebra (in addition to existing
Elasticsearch)
   - Implements _sort_facets_zebra() function in C4::Search
   - Maintains consistent Unicode-aware sorting behavior across search engines

Key Features:
- Dynamic locale detection via Koha::I18N::available_locales()
- Proper module architecture with comprehensive unit tests
- Clean fallback chain: preference → system LC_COLLATE → default
- Admin UI integration in System Preferences → I18N/L10N
- Universal facet sorting for international library systems

Test plan:
1. Apply patch and restart services
2. Install/run database update
3. Navigate to Administration → System Preferences → I18N/L10N
4. Verify "Sort facet names using" dropdown shows available system locales
5. Test facet sorting with different locales and both search engines
6. Run: prove t/Koha/I18N.t t/Koha/SearchEngine/Elasticsearch/Search.t
t/db_dependent/Search_FacetSorting.t

Signed-off-by: David Cook 
Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-09 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Martin Renvoize (ashimema)  changed:

   What|Removed |Added

 Attachment #183564|0   |1
is obsolete||

--- Comment #89 from Martin Renvoize (ashimema) 
 ---
Created attachment 183881
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183881&action=edit
Bug 36947: Do a locale-based sort for ES facet names

This change uses a configurable locale-based collator to sort
the ES facet names.

Test plan:
0. Apply the patch
1. vi /etc/locale.gen
2. Uncomment the locale you want to generate (e.g. fi_FI.UTF-8 UTF-8)
3. locale-gen
4. vi "/etc/default/koha-common"
5. Add the following to the bottom of the file:
export LC_ALL=fi_FI.UTF-8
6. koha-plack --restart koha-common
7. Setup some test records with authors with accented and unaccented names,
and different cases for the lead letter
e.g. Aa author, Åa author2, aa author, étienne
8. Switch to using Elasticsearch and reindex
koha-elasticsearch -b -v --rebuild kohadev
9. Do a test search
e.g. http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
10. Confirm the facet names are sorted in ascending order following
Finnish collation rules
e.g.
aa author
Aa author
étienne
Farley, David
Humble, Jez
Martin, Robert C.
Åa author

NOTE: Any collation and language can be used. Finnish is just
an example of a Latin-based script which has a different
alphabetical ordering than just A-Z

Signed-off-by: Martin Renvoize 
Signed-off-by: David Cook 
Signed-off-by: Martin Renvoize 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-06 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

David Cook  changed:

   What|Removed |Added

 Attachment #183817|0   |1
is obsolete||

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-06 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

David Cook  changed:

   What|Removed |Added

 Status|In Discussion   |Signed Off

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-06 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #88 from David Cook  ---
Created attachment 183818
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183818&action=edit
Bug 36947: (QA follow-up) Add skippable unit test for fi_FI locale

This patch adds a skippable unit test for the fi_FI locale to prove
that the patches work well for other locales that don't use
the default Unicode collation. It will run on koha-testing-docker,
but will be skipped on systems lacking it.

This patch also fleshes out the existing unit tests a bit more
to prove that the patches also improves the sorting of names
with diacritics generally. That is, the default Unicode sort
is better than the existing stringwise/bytewise sort even for
English and French locales.

Test plan:
0. prove -v t/Koha/SearchEngine/Elasticsearch/Search.t \
t/db_dependent/Search_FacetSorting.t

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-06 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #87 from David Cook  ---
Created attachment 183817
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183817&action=edit
Bug 36947: Add skippable unit test for fi_FI locale

This patch adds a skippable unit test for the fi_FI locale to prove
that the patches work well for other locales that don't use
the default Unicode collation. It will run on koha-testing-docker,
but will be skipped on systems lacking it.

This patch also fleshes out the existing unit tests a bit more
to prove that the patches also improves the sorting of names
with diacritics generally. That is, the default Unicode sort
is better than the existing stringwise/bytewise sort even for
English and French locales.

Test plan:
0. prove -v t/Koha/SearchEngine/Elasticsearch/Search.t \
t/db_dependent/Search_FacetSorting.t

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-06 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

David Cook  changed:

   What|Removed |Added

 Attachment #183727|0   |1
is obsolete||

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-04 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #86 from Tomás Cohen Arazi (tcohen)  ---
(In reply to David Cook from comment #84)
> I think you do raise a good point though about the unit tests. I hope that
> Tomas can give us a hand with getting out a new KTD that adds the fi_FI
> locale, so that we can do a unit test that proves this same thing.

Done! Thanks for adding me to the loop BTW.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-03 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #85 from David Cook  ---
Another reason why you'd use Unicode::Collate::Locale and not "use locale" is
so that 1 Koha server can provide Koha instances with different locales.

Technically, you could use both "use locale" and "setlocale()" together I
suppose, but again Unicode::Collate::Locale is a nice core library designed for
this exact purpose that we're using it for.

(In reply to David Cook from comment #57)
> I just finished writing and polishing this...
> 
> And I realize the LC_COLLATE thing is controversial, since it's a
> system-level thing and not an instance-level thing.
> 
> A system preference or a koha-conf.xml config is probably the way to go.
> We'd look it up in Koha/SearchEngine/Elasticsearch/Search.pm and pass it
> into the _sort_facets() function using the "locale" parameter.
> 
> --
> 
> I was going to say that since it's a system level thing and something that
> shouldn't be changed by librarians, I thought koha-conf.xml would be better.
> 
> But... if we used the system preference, then people could override it using
> VirtualHost directives in Apache, so that different front-ends use different
> locales.
> 
> I'm really not too fussed... the main reason I did this patch was out of the
> goodness of my heart (or perhaps intellectual vanity or perhaps both), so
> I'm happy to be led by others here on the configuration side of things.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-03 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #84 from David Cook  ---
(In reply to Jonathan Druart from comment #81)
> Are you sure using Sort::Naturally is not enough? At least the tests pass.
> If it's not enough please provide a test that makes this last patch fail.

As per my above comment, we can't do an automated unit test within KTD
currently that makes the last patch fail without interfering with the local KTD
system. However, we can do a manual test.

To test:
0. apt-get install libsort-naturally-perl
1. Perform all the test plans from
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947#c71

As expected, the "Use ncmp" patch fails the advanced test when the fi_FI locale
is selected using FacetSortingLocale. The search results return the following
which are incorrectly sorted for the Finnish language:

aa author
Aa author
Åa author2
étienne
Farley, David
Humble, Jez
Martin, Robert C.

If you revert that patch (or checkout the one before it) and perform the test
again, you get the correct result for the Finnish language:

aa author
Aa author
étienne
Farley, David
Humble, Jez
Martin, Robert C.
Åa author2

--

Is that enough proof?

I think you do raise a good point though about the unit tests. I hope that
Tomas can give us a hand with getting out a new KTD that adds the fi_FI locale,
so that we can do a unit test that proves this same thing.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-03 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

David Cook  changed:

   What|Removed |Added

 CC||[email protected]

--- Comment #83 from David Cook  ---
(In reply to Jonathan Druart from comment #81)
> Are you sure using Sort::Naturally is not enough? At least the tests pass.
> If it's not enough please provide a test that makes this last patch fail.

Yes, I am reasonably sure that Sort::Naturally is not enough. If you read
through the comments, you'll see that I've previously indicated that the tests
are flawed in the current koha-testing-docker because of the available locales.
This could be fixed, but would take some coordination with whoever manages the
ktd images. I'm guessing Tomas?

I'll explain in a bit more detail below.

--

Unicode::Collate::Locale is core Perl. Sort::Naturally is a CPAN module which
isn't currently available in Koha. 

Sort::Naturally appears to just use "use locale" under the hood. If you look
back on the comments, you can see that has already been explored. Maybe I
wasn't clear in past comments abouts its limitations, but from memory it's
limited to the system locale or the locale as specified in the runtime
environment of that process at startup (I can't recall if it's compile time or
run time though). Unicode::Collate::Locale allows us to very easily work with
locales supplied at runtime, which creates many wonderful possibilities for
adapting to the language needs of users. Due to limited resources, Martin has
specified the locale via a system preference, but there's really a lot more
potential here, if you're interested in this topic.

Finally, the unit tests are constrained by the locales available on
koha-testing-docker. If you run `locale -a`, you'll see the following:

C
C.utf8
en_US.utf8
fr_FR.utf8
POSIX

All of these locales use the same underlying Unicode collator, so they're all
always going to have the same result. 

In order for Martin or myself to "provide a test that makes this last patch
fail", we'd have to change the koha-testing-docker to include a locale like
fi_FI which has a different Unicode collator. 

Now I think it would be a great idea to add fi_FI to koha-testing-docker for
this purpose alone. I think we haven't done that because of the obstacle that
presents, but yeah... that's an option. Probably a good one in fact. 

I'm adding Tomas as CC. I'll send him a message on Mattermost too. If we can
get a new KTD image with fi_FI as an available locale, then we can write a test
to prove how good Unicode::Collate::Locale is and why we need it.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-03 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Jonathan Druart  changed:

   What|Removed |Added

 Attachment #183726|0   |1
is obsolete||

--- Comment #82 from Jonathan Druart  ---
Created attachment 183727
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183727&action=edit
Bug 36947: Use ncmp

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-03 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

Jonathan Druart  changed:

   What|Removed |Added

 CC||[email protected]
 Status|Signed Off  |In Discussion

--- Comment #81 from Jonathan Druart  ---
Are you sure using Sort::Naturally is not enough? At least the tests pass. If
it's not enough please provide a test that makes this last patch fail.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-07-03 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

--- Comment #80 from Jonathan Druart  ---
Created attachment 183726
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183726&action=edit
Bug 36947: Use ncmp

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


[Koha-bugs] [Bug 36947] Sort Elasticsearch/Zebra facets according to configurable locale instead of using Perl's stringwise/bytewise sort

2025-06-26 Thread bugzilla-daemon--- via Koha-bugs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36947

David Cook  changed:

   What|Removed |Added

Summary|Sort Elasticsearch facets   |Sort Elasticsearch/Zebra
   |according to system locale  |facets according to
   |instead of using Perl's |configurable locale instead
   |stringwise/bytewise sort|of using Perl's
   ||stringwise/bytewise sort

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/