[google-appengine] UnicodeEncodeError but still saves to the Datastore
Hello all,
My datastore model has an attribute of class string. According to the
documentation, the String class should be of type str or unicode.
The data I'm trying to save is unicode (I confirmed this by printing out
the type and the original page was encoded in utf-8).
My data is successfully saved to the datastore but it still raises the
following exception 'ascii' codec can't encode character u'\u2019' in
position 6: ordinal not in range(128)
I'm curious as to why an exception is being raised even though the data is
unicode and why even with the exception, the data is still saved
successfully.
Apart from specifically trapping this error using something like {exception
UnicodeEncodeError: pass}, is there a way to prevent this error?
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] UnicodeEncodeError but still saves to the Datastore
On Sun, Jun 2, 2013 at 4:18 PM, NP wrote:
> Hello all,
>
> My datastore model has an attribute of class string. According to the
> documentation, the String class should be of type str or unicode.
>
> The data I'm trying to save is unicode (I confirmed this by printing out
> the type and the original page was encoded in utf-8).
>
> My data is successfully saved to the datastore but it still raises the
> following exception 'ascii' codec can't encode character u'\u2019' in
> position 6: ordinal not in range(128)
>
When does this exception happen? From the error message, you're trying to
encode u'\u2019' (RIGHT SINGLE QUOTATION MARK) with the 'ascii' codec. Here
is an example:
>>> u = u'\u2019'
>>> u.encode('ascii')
Traceback (most recent call last):
File "", line 1, in
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in
position 0: ordinal not in range(128)
>>> u.encode('utf-8') # you can get an utf-8 encoded binary string
'\xe2\x80\x99'
>
> I'm curious as to why an exception is being raised even though the data is
> unicode and why even with the exception, the data is still saved
> successfully.
>
> Apart from specifically trapping this error using something like
> {exception UnicodeEncodeError: pass}, is there a way to prevent this error?
>
I don't recommend doing that. I recommend you check where the exception
occurs and fix it.
-- Takashi
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
Takashi Matsuo | Developers Programs Engineer | tmat...@google.com
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] UnicodeEncodeError but still saves to the Datastore
What i do in my application is setting unicode literals in my scripts. All
of them start with:
from __future__ import unicode_literals
Once GAE and Jinja, my template system, work just fine with this
configuration, i have no hadaches with strings.
Good Lucky,
Renzo
On Monday, June 3, 2013 3:08:42 PM UTC-3, Takashi Matsuo (Google) wrote:
>
>
>
>
> On Sun, Jun 2, 2013 at 4:18 PM, NP >wrote:
>
>> Hello all,
>>
>> My datastore model has an attribute of class string. According to the
>> documentation, the String class should be of type str or unicode.
>>
>> The data I'm trying to save is unicode (I confirmed this by printing out
>> the type and the original page was encoded in utf-8).
>>
>> My data is successfully saved to the datastore but it still raises the
>> following exception 'ascii' codec can't encode character u'\u2019' in
>> position 6: ordinal not in range(128)
>>
>
> When does this exception happen? From the error message, you're trying to
> encode u'\u2019' (RIGHT SINGLE QUOTATION MARK) with the 'ascii' codec. Here
> is an example:
>
> >>> u = u'\u2019'
> >>> u.encode('ascii')
> Traceback (most recent call last):
> File "", line 1, in
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in
> position 0: ordinal not in range(128)
> >>> u.encode('utf-8') # you can get an utf-8 encoded binary string
> '\xe2\x80\x99'
>
>
>>
>> I'm curious as to why an exception is being raised even though the data
>> is unicode and why even with the exception, the data is still saved
>> successfully.
>>
>> Apart from specifically trapping this error using something like
>> {exception UnicodeEncodeError: pass}, is there a way to prevent this error?
>>
>
> I don't recommend doing that. I recommend you check where the exception
> occurs and fix it.
>
> -- Takashi
>
>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to google-appengi...@googlegroups.com .
>> To post to this group, send email to
>> google-a...@googlegroups.com
>> .
>> Visit this group at http://groups.google.com/group/google-appengine?hl=en
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
>
> --
> Takashi Matsuo | Developers Programs Engineer | tma...@google.com
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
