yeah,maybe you are right,
but I'm not professional in mysql
I just use the SQLAlchemy default
create the table and field
the field is utf8_general_ci
my problem like that
import sqlalchemy
sqlalchemy.__version__
'0.6beta1'
from sqlalchemy import create_engine
engine = create_engine('mysql://root:@localhost/dev')
from sqlalchemy import Table, Column, Integer, String, Unicode,
MetaData, ForeignKey
metadata = MetaData()
users_table = Table('users', metadata,
... Column('id', Integer, primary_key=True),
... Column('name', Unicode(24)),
... )
metadata.create_all(engine)
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)
session = Session()
class User(object):
... def __init__(self, name):
... self.name = name
...
from sqlalchemy.orm import mapper
mapper(User, users_table)
Mapper at 0x101b81510; User
name1 = '中文中文'
name1
'\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87'
len(name1)
12
ed_user = User(name1.decode('utf8'))
session.add(ed_user)
session.commit()
session.query(User).get(1).name
u'\u4e2d\u6587\u4e2d\u6587'
name2 = '中文中文中文中文'
name2
'\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87'
ed_user = User(name2.decode('utf8'))
session.add(ed_user)
session.commit()
session.query(User).get(2).name
'\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87'
name3 = '中文中a文中文中文'
name3
'\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87\xe4\xb8\xad\xe6\x96\x87'
ed_user = User(name3.decode('utf8'))
session.add(ed_user)
session.commit()
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/SQLAlchemy-0.6beta1dev-py2.6.egg/sqlalchemy/engine/default.py:207:
Warning: Data truncated for column 'name' at row 1
cursor.execute(statement, parameters)
session.query(User).get(3).name
Traceback (most recent call last):
...
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 22-23:
unexpected end of data
I know the last Chinese char is not complete
the end byte \x87 is not save to database.
the \xe6\x96\x87 is a complete Chinese char
and now is \xe6\x96 so when use the sqlalchemy
query It's can't decode by utf8. my solution
will ignore the end char \x96 and \xe6
the decode is work,and haven't Garbled.
if you don't care this you can ignore this issue
because it's not will appear in normal.
btw,in my phpMyAdmin the field is garbled,
but i use the django model hasn't this issue
2010/1/13 Gaetan de Menten gdemen...@gmail.com
On Wed, Jan 13, 2010 at 10:16, 诚子 zhicheng1...@gmail.com wrote:
detial please see
http://my.unix-center.net/~WeiZhicheng/2010/01/13/sqlalchemy-0-6-unicodedecodeerror-bug-patch/http://my.unix-center.net/%7EWeiZhicheng/2010/01/13/sqlalchemy-0-6-unicodedecodeerror-bug-patch/
Well, if I understand correctly, you would like a workaround for a bad
truncate algorithm in mysql (which apparently truncates in the middle
of a multi-byte character). Are you sure it's not a configuration
problem of mysql? I would be amazed if they don't provide a solution
to this problem (though that wouldn't be the first time I'm amazed at
mysql... sigh...).
Now, as to solve your problem (assuming it's not solvable on mysql's
side), I think passing another UnicodeDecodingError behavior for the
decoder would be a better generic solution. This is not yet possible
with the generic types on current SQLAlchemy.
See http://www.sqlalchemy.org/trac/ticket/1257
In the meantime, you are free to use a custom type, or type decorator
to provide whatever custom decoding mechanism you want. We can't
accept your patch though, as it is too specific to your case and would
provide a very unexpected behavior in some cases (what if the decoding
error is not at the end of the string?).
Hope it helps,
--
Gaëtan de Menten
http://openhex.org
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.comsqlalchemy%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
my.unix-center.net/~WeiZhicheng
--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
