[issue5200] unicode.normalize gives wrong result for some characters

Tue, 10 Feb 2009 12:04:28 -0800 

Peter Landgren <peter.tal...@telia.com> added the comment:

Thanks for the fast response.

I understand that python follows the unicode specification. I think the unicode 
standard 
is not correct in this case for the Swedish letters. I have asked unicode.org 
for an 
explanation. 

Should not the Danish letter "Ø" be normalized as "O"? I get "Ø" for all 
NFC/NFD/NFKC/NFKD 
normalizations?

Regards,
Peter Landgren

Added file: http://bugs.python.org/file13018/unnamed

_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue5200>
_______________________________________
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" 
"http://www.w3.org/TR/REC-html40/strict.dtd";>
<html><head><meta name="qrichtext" content="1" /><style type="text/css">
p, li { white-space: pre-wrap; }
</style></head><body style=" font-family:'Sans Serif'; font-size:10pt; 
font-weight:400; font-style:normal;">
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">Thanks for the fast response.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; 
margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;"></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I 
understand that python follows the unicode specification. I think the unicode 
standard is not correct in this case for the Swedish letters. I have asked 
unicode.org for an explanation. </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; 
margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;"></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">Should not the Danish letter "Ø" be normalized as "O"? I 
get "Ø" for all NFC/NFD/NFKC/NFKD normalizations?</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; 
margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;"></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">Regards,</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Peter 
Landgren</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; 
margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;"></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
Martin v. Löwis &lt;mar...@v.loewis.de&gt; added the comment:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
It is not true that normalize produces "aaoAAO". Instead, it produces</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
u'a\u030aa\u0308o\u0308A\u030aA\u0308O\u0308'</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
This is the correct result, according to the Unicode specification. It</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
would be incorrect to normalize them unchanged under the Unicode Normal</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
Form D (for decomposed); the decomposed character for 'LATIN SMALL</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
LETTER A WITH RING ABOVE' (for example) is 'LATIN SMALL LETTER A' +</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
'COMBINING RING ABOVE'.</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
The wikipedia article is irrelevant; refer to the Unicode specification</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
for a normative reference.</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
Closing as invalid.</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
----------</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
nosy: +loewis</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
resolution:  -&gt; invalid</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
status: open -&gt; closed</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
_______________________________________</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
Python tracker &lt;rep...@bugs.python.org&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
&lt;http://bugs.python.org/issue5200&gt;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 
_______________________________________</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">-- 
</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Peter 
Landgren</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">Talken Hagen   </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">671 
94  BRUNSKOG</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">0570-530 21</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">070-635 4719</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">peter.tal...@telia.com</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; 
margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;">Skype: pgl4820.2</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; 
margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; 
-qt-user-state:0;"></p></body></html>
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to