Re: [sqlalchemy] Re: exception message encoded in utf8
I think I can help reproduce this, but one has to configure the base system
in a non C or english locale.
E.g. the system I work with is in french: it's default locale is
fr_FR.UTF-8, hence the postgresql server
I installed on it runs with that locale too, by default. One can check with
the `SHOW lc_messages;` SQL query.
Hence, with any error I trigger on the server side, like an IntegrityError
or a ProgrammingError, comes
with a message from the server, which is encoded in that locale. For
example many of messages translated
in french, come with «» (french *guillemets*) instead of double quotes
which have psycopg or SQLAlchemy
choke on it. I would agree that is looks more like psycopg issue. It also
rarely impacts production environments,
which are generally not set with fancy locale (at least I don't do that).
For reference, some ways to mitigate this are:
- reconfigure your postgresql serveur and have it use another default
(requires "administrative" access to reconfigure the server);
- reconfigure your session so it use another session (cf. code snippet
below, requires to be superuser which is rarely convenient);
Regarding the required priveleges cf.
https://www.postgresql.org/docs/current/static/runtime-config-client.html.
@listens_for(_engine, 'connect')
def first_connect_callback(dbapi_connection, connection_record):
# Registers composite types so psycopg2 know how to
# handle them when they appear in resultsets.
cursor = dbapi_connection.cursor()
cursor.execute("SET lc_messages TO 'C';")
cursor.execute("COMMIT;")
cursor.execute("BEGIN;") # Make sure you start a new transaction (may
not always be appropriate)
Le jeudi 20 décembre 2012 18:35:13 UTC+1, Michael Bayer a écrit :
>
> OK as I said earlier, I'm not able to reproduce this.So I'd need that
> reproduction case in order to do anything. To be honest it sounds more
> like a psycopg2 bug, since psycopg2 does the decoding in most cases
> nowadays and even works with Python 3, so for it to be raising an exception
> with the "bytes" type for the message is certainly a bug. But would need
> to see a real world example to get a feel for it.
>
>
> On Dec 20, 2012, at 7:03 AM, Sylvain Prat wrote:
>
> Sorry to ressurrect this thread but the problem is still there. Since
> SQLAlchemy knows the encoding used to communicate with the database, it can
> properly decode the error strings returned by the database to unicode. So,
> I think it should be SQLAlchemy's responsibility to convert the error
> strings to unicode, not the user's responsibility. Could we open a bug for
> that in the tracker?
>
> Sylvain
>
>
> Le vendredi 7 mars 2008 02:39:56 UTC+1, jean-philippe dutreve a écrit :
>>
>> On 7 mar, 02:29, Michael Bayer wrote:
>>
>> > logging module itself throws UnicodeDecodeError ?
>> yes, in logging.format: ... = "%s" % msg
>> with msg the exception message encoded in utf8 and the default
>> encoding is ascii.
>>
>> > are you sending exception messages using logging.debug() or similar ?
>> exactly: log.error("... : %s", e.message)
>>
>> my impression
>> > is that you'd want to decode those manually doing something like
>> > string.decode('utf-8').
>> yes, it works fine, but it's pain to do this in each try/except.
>> Another solution is setting utf8 as the default encoding in
>> sitecustomize.py.
>> It's better centralized, but has sitepackage effect.
>>
>> But a better way IMHO is that the DB driver or SA returns unicode
>> exception message.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/sqlalchemy/-/OOEvbKoo63cJ.
> To post to this group, send email to sqlal...@googlegroups.com
> .
> To unsubscribe from this group, send email to
> sqlalchemy+...@googlegroups.com .
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
>
>
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Re: exception message encoded in utf8
Sorry to ressurrect this thread but the problem is still there. Since
SQLAlchemy knows the encoding used to communicate with the database, it can
properly decode the error strings returned by the database to unicode. So,
I think it should be SQLAlchemy's responsibility to convert the error
strings to unicode, not the user's responsibility. Could we open a bug for
that in the tracker?
Sylvain
Le vendredi 7 mars 2008 02:39:56 UTC+1, jean-philippe dutreve a écrit :
On 7 mar, 02:29, Michael Bayer mike...@zzzcomputing.com wrote:
logging module itself throws UnicodeDecodeError ?
yes, in logging.format: ... = %s % msg
with msg the exception message encoded in utf8 and the default
encoding is ascii.
are you sending exception messages using logging.debug() or similar ?
exactly: log.error(... : %s, e.message)
my impression
is that you'd want to decode those manually doing something like
string.decode('utf-8').
yes, it works fine, but it's pain to do this in each try/except.
Another solution is setting utf8 as the default encoding in
sitecustomize.py.
It's better centralized, but has sitepackage effect.
But a better way IMHO is that the DB driver or SA returns unicode
exception message.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To view this discussion on the web visit
https://groups.google.com/d/msg/sqlalchemy/-/OOEvbKoo63cJ.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: exception message encoded in utf8
OK as I said earlier, I'm not able to reproduce this.So I'd need that
reproduction case in order to do anything. To be honest it sounds more like a
psycopg2 bug, since psycopg2 does the decoding in most cases nowadays and even
works with Python 3, so for it to be raising an exception with the bytes type
for the message is certainly a bug. But would need to see a real world
example to get a feel for it.
On Dec 20, 2012, at 7:03 AM, Sylvain Prat wrote:
Sorry to ressurrect this thread but the problem is still there. Since
SQLAlchemy knows the encoding used to communicate with the database, it can
properly decode the error strings returned by the database to unicode. So, I
think it should be SQLAlchemy's responsibility to convert the error strings
to unicode, not the user's responsibility. Could we open a bug for that in
the tracker?
Sylvain
Le vendredi 7 mars 2008 02:39:56 UTC+1, jean-philippe dutreve a écrit :
On 7 mar, 02:29, Michael Bayer mike...@zzzcomputing.com wrote:
logging module itself throws UnicodeDecodeError ?
yes, in logging.format: ... = %s % msg
with msg the exception message encoded in utf8 and the default
encoding is ascii.
are you sending exception messages using logging.debug() or similar ?
exactly: log.error(... : %s, e.message)
my impression
is that you'd want to decode those manually doing something like
string.decode('utf-8').
yes, it works fine, but it's pain to do this in each try/except.
Another solution is setting utf8 as the default encoding in
sitecustomize.py.
It's better centralized, but has sitepackage effect.
But a better way IMHO is that the DB driver or SA returns unicode
exception message.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To view this discussion on the web visit
https://groups.google.com/d/msg/sqlalchemy/-/OOEvbKoo63cJ.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: exception message encoded in utf8
On Mar 6, 2008, at 6:35 PM, jean-philippe dutreve wrote:
Hi all,
I use SQLAlchemy-0.4.2p3, postgreSQL 8.2.4 (UTF8 configured) and
psycopg2.
I have no issue with unicode DATA in and out of the database.
My problem is that when an IntegrityError is thrown, the exception
message is a string encoded in utf8.
And the logging module throws an UnicodeDecodeError.
Is there a way that the exception message is type of unicode instead
of string?
logging module itself throws UnicodeDecodeError ? are you sending
exception messages using logging.debug() or similar ? my impression
is that you'd want to decode those manually doing something like
string.decode('utf-8').
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
