Re: [Zope-dev] Catalog and Unicode

2005-08-10 Thread Dieter Maurer 
Florent Guillaume wrote at 2005-8-9 17:18 +0200:
We're seeing problems in one application here due to the catalog and  
interactions with Unicode. Here's what happens:

- an object is indexed with a Unicode title, so in the catalog the  
metadata tuple has for instance (u'cafe',)
- later that title is changed to latin-1, so the new metadata tuple  
would be ('caf\xe9',)

The problem is that Catalog.py has in updateMetadata() the code:

 if data.get(index, 0) != newDataRecord:
 data[index] = newDataRecord
 try:
 changed = data.get(index, 0) != newDataRecord
 except UnicodeDecodeError:
 changed = True
if changed:
 data[index] = newDataRecord

Objections ?

I fear, you will get similar problems in the indexes.

You should avoid mixed unicode/non-unicode in fields or indexes
(or the the default encoding appropriately).

-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Catalog and Unicode

2005-08-09 Thread Florent Guillaume 
We're seeing problems in one application here due to the catalog and  
interactions with Unicode. Here's what happens:


- an object is indexed with a Unicode title, so in the catalog the  
metadata tuple has for instance (u'cafe',)
- later that title is changed to latin-1, so the new metadata tuple  
would be ('caf\xe9',)


The problem is that Catalog.py has in updateMetadata() the code:

if data.get(index, 0) != newDataRecord:
data[index] = newDataRecord

The simple comparison in the first line provokes a  
UnicodeDecodeError, you can reproduce by a simple:


python -c u'e' == '\xe9'

This understandable, but in the case of the catalog really not  
helpful. I propose to change the code above to:


try:
changed = data.get(index, 0) != newDataRecord
except UnicodeDecodeError:
changed = True
   if changed:
data[index] = newDataRecord

Objections ?

Florent

--
Florent Guillaume, Nuxeo (Paris, France)   CTO, Director of RD
+33 1 40 33 71 59   http://nuxeo.com   [EMAIL PROTECTED]


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )