On 10/30/13 12:08 PM, wxjmfa...@gmail.com wrote:
Le mercredi 30 octobre 2013 13:44:47 UTC+1, Ned Batchelder a écrit :
On 10/30/13 4:49 AM, wxjmfa...@gmail.com wrote:
Le mardi 29 octobre 2013 06:24:50 UTC+1, Steven D'Aprano a écrit :
On Mon, 28 Oct 2013 09:23:41 -0500, Tim Chase wrote:
On 2013-10-28 07:01, wxjmfa...@gmail.com wrote:
Simply ignoring diactrics won't get you very far.
Right. As an example, these four French words : cote, côte, coté, côté
.
Distinct words with distinct meanings, sure.
But when a naïve (naive? ☺) person or one without the easy ability to
enter characters with diacritics searches for "cote", I want to return
possible matches containing any of your 4 examples. It's slightly
fuzzier if they search for "coté", in which case they may mean "coté" or
they might mean be unable to figure out how to add a hat and want to
type "côté". Though I'd rather get more results, even if it has some
that only match fuzzily.
The right solution to that is to treat it no differently from other fuzzy
searches. A good search engine should be tolerant of spelling errors and
alternative spellings for any letter, not just those with diacritics.
Ideally, a good search engine would successfully match all three of
"naïve", "naive" and "niave", and it shouldn't rely on special handling
of diacritics.
------
This is a non sense. The purpose of a diacritical mark is to
make a letter a different letter. If a tool is supposed to
match an ô, there is absolutely no reason to match something
else.
jmf
jmf, Tim Chase described his use case, and it seems reasonable to me.
I'm not sure why you would describe it as nonsense.
--Ned.
--------
My comment had nothing to do with Python, it was a
general comment. A diacritical mark just makes a letter
a different letter; a "ï " and a "i" are "as
diferent" as a "a" from a "z". A diacritical mark
is more than a simple ornementation.
Yes, we understand that. Tim outlined a need that had to do with users'
informal typing. In his case, he needs to deal with that sloppiness.
You can't simply insist that users be more precise.
Unicode is a way to represent text, and text gets used in many different
ways. Each of us has to acknowledge that our text needs may be
different than someone else's. jmf, I'm guessing from your comments
over the last few months that you are doing detailed linguistic work
with corpora in many languages. That work leads to one style of Unicode
use. In your domain, it is "nonsense" to ignore diacriticals.
Other people do different kinds of work with Unicode, and that leads to
different needs. In Tim's system, it is important to ignore
diacriticals. You might not have a use personally for Tim's system.
That doesn't make it nonsense.
--Ned.
From a unicode perspective.
Unicode.org "knows", these chars a very important, that's
the reason why they exist in two forms, precomposed and
composed forms.
From a software perspective.
Luckily for the end users, all the serious software
are considering all these chars in an equal way. They
are all belonging to the BMP plane. An "Ą" is treated
as an "ê", same memory consumption, same performance,
==> very smooth software.
jmf
--
https://mail.python.org/mailman/listinfo/python-list
