[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Caroline Cyr La Rose changed: What|Removed |Added Documentation||Caroline Cyr La Rose contact|| Documentation||https://gitlab.com/koha-com submission||munity/koha-manual/-/merge_ ||requests/953 CC||caroline.cyr-la-rose@inlibr ||o.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added CC|martin.renvoize@ptfs-europe | |.com| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Katrin Fischer changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=37137 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Laura O'Neil changed: What|Removed |Added CC||la...@bywatersolutions.com --- Comment #51 from Laura O'Neil --- +1 for backport to 23.05, this would be very helpful to a lot of libraries. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #50 from Nick Clemens --- Current patches apply cleanly to 23.05.x and work in my testing - can this be considered for backport? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 David Cook changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=5534 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Fridolin Somers changed: What|Removed |Added Keywords||rel_23_05_candidate --- Comment #49 from Fridolin Somers --- I'll work on it -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #48 from Nick Clemens --- (In reply to Fridolin Somers from comment #47) > (In reply to Fridolin Somers from comment #45) > > Enhancement not pushed to 23.05.x > > Sure, I'll try backport. > Any ES feature is welcome ;) Ping? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #47 from Fridolin Somers --- (In reply to Fridolin Somers from comment #45) > Enhancement not pushed to 23.05.x Sure, I'll try backport. Any ES feature is welcome ;) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Status|Needs documenting |Pushed to master Severity|enhancement |normal --- Comment #46 from Nick Clemens --- I marked as 'normal' now - while this adds new functionality I think it resolves a bug for users where records are not being found. Reconsider? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Fridolin Somers changed: What|Removed |Added Keywords||release-notes-needed Status|Pushed to master|Needs documenting --- Comment #45 from Fridolin Somers --- Enhancement not pushed to 23.05.x -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #44 from Tomás Cohen Arazi --- Pushed to master for 23.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Tomás Cohen Arazi changed: What|Removed |Added Version(s)||23.11.00 released in|| Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Status|Signed Off |Passed QA --- Comment #43 from Martin Renvoize --- This adds a nice flexible and configurable way of achieving the desired goal. No regressions found, the QA scripts are happy and unit tests are all passing. Passing QA -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Attachment #155517|0 |1 is obsolete|| --- Comment #42 from Martin Renvoize --- Created attachment 157737 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157737&action=edit Bug 27153: Adjust kohastructure for new primary key Signed-off-by: Danielle M Elder Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Attachment #155516|0 |1 is obsolete|| --- Comment #41 from Martin Renvoize --- Created attachment 157736 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157736&action=edit Bug 27153: Adjust mappings to add title without punctuation This patch finally adjust the default mappings to add a new field to the elastic index with the title stripped of punctuation This method optionally allows the library to place the filtered version in the same search field, or a new search field. The default mappings will include the filtered version in a keyword search, but not a targeted search To test: 1 - Import some affected records via z3950, e.g.: Carrie's war 1,000 Japanese words 2 - Search (using Elasticsearch) for the titles without including punctuation Carries war 1000 Japanese words 3 - No results 4 - Reload mappings and reindex perl misc/search_tools/rebuild_elasticsearch.pl -v -r 5 - Search again 6 - Success! 7 - Search title specifically: ti:Carries war 8 - No results 9 - Adjust mappings.yaml to add second mapping for 245 to title index: - facet: '' marc_field: 245abp marc_type: marc21 sort: 1 suggestible: 1 filter: punctuation 10 - Reload mappings and reindex 11 - Repeat 7 12 - Success Signed-off-by: Danielle M Elder Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Attachment #155515|0 |1 is obsolete|| --- Comment #40 from Martin Renvoize --- Created attachment 157735 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157735&action=edit Bug 27153: Add filter option to Elasticsearch indexing This patch uses the filter option from the mappings to add a value_callback to reord processing for indexing. Fields defined with 'punctuation' filter will have all punctuation stripped when conveted to documents. Tests are updated. Signed-off-by: Danielle M Elder Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Attachment #155514|0 |1 is obsolete|| --- Comment #39 from Martin Renvoize --- Created attachment 157734 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157734&action=edit Bug 27153: Interface updates This patch adds a new column to the 'Bibliographic records' tab in Administration > Search engine configuration (Elasticsearch) To test: 1 - Confirm the new 'filter' column shows 2 - Update an existing field to set filter to punctuation 3 - Confirm it can be saved 4 - Add a new field 5 - Confirm it saves correctly 6 - Unset filter for a field 7 - Confirm it saves Signed-off-by: Danielle M Elder Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Attachment #155512|0 |1 is obsolete|| --- Comment #38 from Martin Renvoize --- Created attachment 157733 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157733&action=edit Bug 27153: DO NOT PUSH: Schema updates Signed-off-by: Danielle M Elder Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Attachment #155145|0 |1 is obsolete|| --- Comment #37 from Martin Renvoize --- Created attachment 157732 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157732&action=edit Bug 27153: DB Updates We start by adding the field to the table It is possible a library would want two versions of the same field in the indexing. We need to make the filter part of the primary key in order to allow this Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added QA Contact||martin.renvoize@ptfs-europe ||.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Tomás Cohen Arazi changed: What|Removed |Added CC||tomasco...@gmail.com Keywords||rel_23_11_candidate -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Michelle Spinney changed: What|Removed |Added CC||mspin...@clamsnet.org -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Dani Elder changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Attachment #155513|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Attachment #155511|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #36 from ByWater Sandboxes --- Created attachment 155517 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155517&action=edit Bug 27153: Adjust kohastructure for new primary key Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #35 from ByWater Sandboxes --- Created attachment 155516 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155516&action=edit Bug 27153: Adjust mappings to add title without punctuation This patch finally adjust the default mappings to add a new field to the elastic index with the title stripped of punctuation This method optionally allows the library to place the filtered version in the same search field, or a new search field. The default mappings will include the filtered version in a keyword search, but not a targeted search To test: 1 - Import some affected records via z3950, e.g.: Carrie's war 1,000 Japanese words 2 - Search (using Elasticsearch) for the titles without including punctuation Carries war 1000 Japanese words 3 - No results 4 - Reload mappings and reindex perl misc/search_tools/rebuild_elasticsearch.pl -v -r 5 - Search again 6 - Success! 7 - Search title specifically: ti:Carries war 8 - No results 9 - Adjust mappings.yaml to add second mapping for 245 to title index: - facet: '' marc_field: 245abp marc_type: marc21 sort: 1 suggestible: 1 filter: punctuation 10 - Reload mappings and reindex 11 - Repeat 7 12 - Success Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155510|0 |1 is obsolete|| --- Comment #34 from ByWater Sandboxes --- Created attachment 155515 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155515&action=edit Bug 27153: Add filter option to Elasticsearch indexing This patch uses the filter option from the mappings to add a value_callback to reord processing for indexing. Fields defined with 'punctuation' filter will have all punctuation stripped when conveted to documents. Tests are updated. Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155509|0 |1 is obsolete|| --- Comment #33 from ByWater Sandboxes --- Created attachment 155514 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155514&action=edit Bug 27153: Interface updates This patch adds a new column to the 'Bibliographic records' tab in Administration > Search engine configuration (Elasticsearch) To test: 1 - Confirm the new 'filter' column shows 2 - Update an existing field to set filter to punctuation 3 - Confirm it can be saved 4 - Add a new field 5 - Confirm it saves correctly 6 - Unset filter for a field 7 - Confirm it saves Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155502|0 |1 is obsolete|| --- Comment #32 from ByWater Sandboxes --- Created attachment 155513 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155513&action=edit Bug 27153: Adjust kohastructure for new primary key Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155508|0 |1 is obsolete|| --- Comment #31 from ByWater Sandboxes --- Created attachment 155512 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155512&action=edit Bug 27153: DO NOT PUSH: Schema updates Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155149|0 |1 is obsolete|| --- Comment #30 from ByWater Sandboxes --- Created attachment 155511 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155511&action=edit Bug 27153: Adjust mappings to add title without punctuation This patch finally adjust the default mappings to add a new field to the elastic index with the title stripped of punctuation This method optionally allows the library to place the filtered version in the same search field, or a new search field. The default mappings will include the filtered version in a keyword search, but not a targeted search To test: 1 - Import some affected records via z3950, e.g.: Carrie's war 1,000 Japanese words 2 - Search (using Elasticsearch) for the titles without including punctuation Carries war 1000 Japanese words 3 - No results 4 - Reload mappings and reindex perl misc/search_tools/rebuild_elasticsearch.pl -v -r 5 - Search again 6 - Success! 7 - Search title specifically: ti:Carries war 8 - No results 9 - Adjust mappings.yaml to add second mapping for 245 to title index: - facet: '' marc_field: 245abp marc_type: marc21 sort: 1 suggestible: 1 filter: punctuation 10 - Reload mappings and reindex 11 - Repeat 7 12 - Success Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155148|0 |1 is obsolete|| --- Comment #29 from ByWater Sandboxes --- Created attachment 155510 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155510&action=edit Bug 27153: Add filter option to Elasticsearch indexing This patch uses the filter option from the mappings to add a value_callback to reord processing for indexing. Fields defined with 'punctuation' filter will have all punctuation stripped when conveted to documents. Tests are updated. Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155147|0 |1 is obsolete|| --- Comment #28 from ByWater Sandboxes --- Created attachment 155509 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155509&action=edit Bug 27153: Interface updates This patch adds a new column to the 'Bibliographic records' tab in Administration > Search engine configuration (Elasticsearch) To test: 1 - Confirm the new 'filter' column shows 2 - Update an existing field to set filter to punctuation 3 - Confirm it can be saved 4 - Add a new field 5 - Confirm it saves correctly 6 - Unset filter for a field 7 - Confirm it saves Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155146|0 |1 is obsolete|| --- Comment #27 from ByWater Sandboxes --- Created attachment 155508 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155508&action=edit Bug 27153: DO NOT PUSH: Schema updates Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 ByWater Sandboxes changed: What|Removed |Added Attachment #155230|0 |1 is obsolete|| --- Comment #26 from ByWater Sandboxes --- Created attachment 155502 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155502&action=edit Bug 27153: Adjust kohastructure for new primary key Signed-off-by: Danielle M Elder -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #25 from Nick Clemens --- Created attachment 155230 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155230&action=edit Bug 27153: Adjust kohastructure for new primary key -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=34695 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Attachment #139969|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Attachment #114234|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Assignee|jonathan.field@ptfs-europe. |n...@bywatersolutions.com |com | -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #24 from Nick Clemens --- Created attachment 155149 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155149&action=edit Bug 27153: Adjust mappings to add title without punctuation This patch finally adjust the default mappings to add a new field to the elastic index with the title stripped of punctuation This method optionally allows the library to place the filtered version in the same search field, or a new search field. The default mappings will include the filtered version in a keyword search, but not a targeted search To test: 1 - Import some affected records via z3950, e.g.: Carrie's war 1,000 Japanese words 2 - Search (using Elasticsearch) for the titles without including punctuation Carries war 1000 Japanese words 3 - No results 4 - Reload mappings and reindex perl misc/search_tools/rebuild_elasticsearch.pl -v -r 5 - Search again 6 - Success! 7 - Search title specifically: ti:Carries war 8 - No results 9 - Adjust mappings.yaml to add second mapping for 245 to title index: - facet: '' marc_field: 245abp marc_type: marc21 sort: 1 suggestible: 1 filter: punctuation 10 - Reload mappings and reindex 11 - Repeat 7 12 - Success -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #23 from Nick Clemens --- Created attachment 155148 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155148&action=edit Bug 27153: Add filter option to Elasticsearch indexing This patch uses the filter option from the mappings to add a value_callback to reord processing for indexing. Fields defined with 'punctuation' filter will have all punctuation stripped when conveted to documents. Tests are updated. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #22 from Nick Clemens --- Created attachment 155147 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155147&action=edit Bug 27153: Interface updates This patch adds a new column to the 'Bibliographic records' tab in Administration > Search engine configuration (Elasticsearch) To test: 1 - Confirm the new 'filter' column shows 2 - Update an existing field to set filter to punctuation 3 - Confirm it can be saved 4 - Add a new field 5 - Confirm it saves correctly 6 - Unset filter for a field 7 - Confirm it saves -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #21 from Nick Clemens --- Created attachment 155146 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155146&action=edit Bug 27153: DO NOT PUSH: Schema updates -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #20 from Nick Clemens --- Created attachment 155145 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=155145&action=edit Bug 27153: DB Updates We start by adding the field to the table It is possible a library would want two versions of the same field in the indexing. We need to make the filter part of the primary key in order to allow this -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Nick Clemens changed: What|Removed |Added Status|In Discussion |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Katrin Fischer changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=21357 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #19 from David Cook --- (In reply to David Cook from comment #18) > Regarding Zebra, I don't think we can get full feature parity here. However, > we could potentially add a Title-punc_removed index and update > ./etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl to strip punctuation for it, > and then add that into C4::Search::_build_weighted_query() (or elsewhere). Alternatively, we could store a punctuation removed version into the same index. That way you end up indexing both "Carrie's war" and "Carries War". But then do you do that for all indexes? That would double the size of your index... -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Dani Elder changed: What|Removed |Added CC||danielle.elder@law.utexas.e ||du -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Fridolin Somers changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=31695 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #18 from David Cook --- I tried to understand this without applying the patches, but it's too hard without the context, so let's see... I assume that the second patch "POC" is an alternative patch and not an additional patch... First patch: - adds apostrophe filter to "analyzer_standard" which is used for all default searches... - apostrophe strips out apostrophes Second patch: - Adds a "punc_removed" field (which uses the analyzer_stdno analyzer which already has a punctuation filter) to "default" under "search". If I understood Elasticsearch and Koha's integration better, I would probably understand this, but I don't currently. - Reading through Koha/SearchEngine/Elasticsearch.pm and https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html it looks like "phrase", "raw", "ci_raw" and "punc_removed" are "fields" only used when queried specifically - In Koha/SearchEngine/Elasticsearch/QueryBuilder.pm adds title.punc_removed field to all Elastic queries. We do something similar in Zebra in C4::Search::_build_weighted_query() where we add title fields to search. I wonder a bit if adding title.punc_removed there will have unintended consequences but it seems similar to Zebra so might not be a big drama. Regarding Zebra, I don't think we can get full feature parity here. However, we could potentially add a Title-punc_removed index and update ./etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl to strip punctuation for it, and then add that into C4::Search::_build_weighted_query() (or elsewhere). -- I think the "POC" patch would need some testing to make sure there aren't any unintended consequences, but overall it sounds like a reasonable proposition. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #17 from Jonathan Field --- I think the POC is a good compromise. Difficult are this but we need to do something to address it as it comes up constantly. We're happy to go with this. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #16 from Nick Clemens --- (In reply to David Cook from comment #15) > (In reply to Katrin Fischer from comment #7) > > Could we treat it ' as a space and 'no space' at the same time maybe? > Without language-specific analysis, this might be the best way to go... That's sort of what the POC does, but just for "as apostrophe" "as no space" - we could add a third field for converting apostrophes to spaces > > (In reply to Nick Clemens from comment #13) > > Created attachment 139969 [details] [review] [review] > > Bug 27153: POC > I'm not very familiar with the Elasticsearch integration, but that POC > doesn't touch the indexing, does it? It does, it adds an extra field, with a punctuation filter to the indexes for 'default' fields - that filter removes all ASCII punctuation characters > Isn't it just modifying the search? It then adds that field to the fields being searched in a 'keyword' search > If you searched "carries war" but it had been indexed as "carrie s war", > then you still wouldn't get a hit? Not with this one, but "carrie's war" or "carries war" would work The nice thing is, adding an extra index means we can generically search all versions of the field, but can also target specific ones: title.punc_removed: carries title.raw: carrie's So staff can be specific, while making standard search more friendly -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #15 from David Cook --- (In reply to Katrin Fischer from comment #7) > Could we treat it ' as a space and 'no space' at the same time maybe? Without language-specific analysis, this might be the best way to go... (In reply to Nick Clemens from comment #13) > Created attachment 139969 [details] [review] > Bug 27153: POC > > This is a small example to show that we can add an extra field to our > default indexing that > removes punctuation. > > If we then add that field to the fields we push, we can sarch both the > regular and the punctuation stripped version I'm not very familiar with the Elasticsearch integration, but that POC doesn't touch the indexing, does it? Isn't it just modifying the search? (I don't know that it's touching apostrophes either?) If you searched "carries war" but it had been indexed as "carrie s war", then you still wouldn't get a hit? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #14 from David Cook --- I had a librarian (from an English-only library) asking for this the other day, and I was wondering why we have apostrophes replaced with a space (both for CHR and ICU indexing with Zebra). But Frido's example makes a lot of sense: (In reply to Fridolin Somers from comment #6) > Elision removes the text : l'europe = europe > This apostrophe filter creates : l'europe = leurope > > Wrong in french but surely ranking will bet better for the exact match. And in English, we'd have "father's" become "fathers", which means "father" wouldn't match unless you're using right truncation (which Koha typically does out of the box I suppose). But then there's a Ukrainian word під'їзд which if you broke it into під їзд, you'd get hits for під which is a totally unrelated word. -- I know people have praised Google here, but it's not perfect either. Try searching "l'arbre під'їзд" and try searching "під'їзд l'arbre", and you'll get wildly different results. It seems that Google tries to determine the language of the search query (possibly based off the first word), and then analyze the search string based off that. For "l'arbre під'їзд", you get many results with "L'Arbre", "des arbres", "arbre", etc. Most results seem to include Russian rather than Ukrainian. But for "під'їзд l'arbre", you only get a few results that have the string "L'Arbre"and most of the results are Ukrainian. So Google is likely determining the language of the search string and then applying a language-specific analyzer. "My mom" in Chinese is 我的妈妈. 的 is the character that denotes possession of mom (妈妈) by me/I (我). You might think then you could replace 的 with a space to separate the 2 nouns... except 的 isn't always used that way. The word 目的 means "goal". Doing anything to 的 would compromise the word, unless you're able to understand the context that it's used in. If you google "我的妈妈", eventually it strips off "我的" and just searches for "妈妈" since that's the main noun in the phrase. Search is hard. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #13 from Nick Clemens --- Created attachment 139969 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=139969&action=edit Bug 27153: POC This is a small example to show that we can add an extra field to our default indexing that removes punctuation. If we then add that field to the fields we push, we can sarch both the regular and the punctuation stripped version To complete this we would need to add a 'search variations' field to the ES config page, for the user to decide which fields should be searched in all variations, and which shouldn't Or we could add this to all 'simple searches' from the search bar, and only search exact on advanced searches. I am happy to explore either option -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Jennifer Millikan changed: What|Removed |Added CC||jmilli...@sja1840.org --- Comment #12 from Jennifer Millikan --- I am brand new to Koha, but I was told this bug is why I have to include apostrophes while searching. I tried searching for Don't Hate the Player title and came up with zero results. I did not use the apostrophe because I have never had to before in multiple catalog systems. I repeated the search with the apostrophe and found the title. I agree with this and think the searching should work both ways with or without the apostrophe. Thank you. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Janet McGowan changed: What|Removed |Added CC||janet.mcgowan@ptfs-europe.c ||om -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Fridolin Somers changed: What|Removed |Added Status|Needs Signoff |In Discussion --- Comment #11 from Fridolin Somers --- (In reply to David Cook from comment #10) > It might be a good idea to harmonize the normalization used between Zebra > and Elasticsearch just for consistency... +1 So we just need to change replacement with a space character ? This will require a unit test I bet. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au --- Comment #10 from David Cook --- It might be a good idea to harmonize the normalization used between Zebra and Elasticsearch just for consistency... -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #9 from Fridolin Somers --- Actually Zebra as a rule in words-icu.xml : But it may not be optimal to do the exact same thing in Elasticsearch. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Barbara Johnson changed: What|Removed |Added CC||barbara.johnson@bedfordtx.g ||ov --- Comment #8 from Barbara Johnson --- This is a constant problem for our patrons who are used to searching Amazon or Google without having to worry about apostrophes. Fixing this issue would really improve the ability for our patrons to find items in our catalog. The status of this bug is listed as Needs Signoff. Is there a test plan? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #7 from Katrin Fischer --- Could we treat it ' as a space and 'no space' at the same time maybe? And how do we currently handle this with Zebra? I guess we'd want Don't Dont don t Pierre d'Anjou Pierre dAnjou Pierre d Anjou to be the same? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #6 from Fridolin Somers --- (In reply to Katrin Fischer from comment #4) > I don't mind a simple solution, but can you explain the difference? Elision removes the text : l'europe = europe This apostrophe filter creates : l'europe = leurope Wrong in french but surely ranking will bet better for the exact match. I have to test combined with ellision. But : Maybe it is bad for authorities : "Pierre d'Anjou" and Pierre Danjou". -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #5 from Nick Clemens --- With this simple apostrophe filter: Dont let the pigeon drive the batmobile will find: Don't let the pigeon drive the batmobile but, you get no results for: ti:"Don't let the pigeon drive the batmobile" I like the idea of a simple solution, but I am not sure it works for everyone. While elisions are different than contractions I believe they both use an apostrophe, so this will affect both. We are planning on testing the 'english' analyzer with one of our partners to help with this issue, I suspect that this may not have a one-size-fits-all solution - but will require the user to decide on which configuration works best in their library -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #4 from Katrin Fischer --- I don't mind a simple solution, but can you explain the difference? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Abbey Holt changed: What|Removed |Added CC||ah...@dubuque.lib.ia.us -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Version|20.11 |master -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added CC||fridolin.som...@biblibre.co ||m -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added CC||n...@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Assignee|koha-b...@lists.koha-commun |jonathan.field@ptfs-europe. |ity.org |com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added CC||martin.renvoize@ptfs-europe ||.com Resolution|DUPLICATE |--- Status|RESOLVED|REOPENED --- Comment #3 from Martin Renvoize --- In my understanding elisions and apostrophes are not equal and as such distinct bugs can/should be opened to handle the two cases. This case is for apostrophes only and leaves elisions, which appear to be more difficult to reach a consensus, to the other bug. As such, I don't think this is a duplicate at all.. I've uploaded Jonathans patch. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Martin Renvoize changed: What|Removed |Added Status|REOPENED|Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 --- Comment #2 from Martin Renvoize --- Created attachment 114234 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=114234&action=edit Bug 27153: Default to apostrophe blind search Keyword searching in elasticsearch should be apostrophe blind; this patch updates the default configuration to filter apostrophe's. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 27153] ElasticSearch should search keywords apostrophe blind
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27153 Katrin Fischer changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |DUPLICATE --- Comment #1 from Katrin Fischer --- Hi Jonathan, I believe this to be a duplicate of bug 21357 - maybe you could add your voice to that bug for a simplified solution (ignore ') *** This bug has been marked as a duplicate of bug 21357 *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
