I'm not sure what's wrong -- I cannot reproduce it on an Ubuntu 12.04
desktop or server.
Anthony
On Friday, September 27, 2013 4:03:00 PM UTC-4, viniciusban wrote:
>
> A new situation.
>
> Before calling code.interact():
> >>> len("á") # len() == 2 is OK
> 2
> >>> "á" # 2 bytes: ok
> '\xc3\xa1'
> >>> code.interact()
> Python 2.7.3 (default, Apr 10 2013, 06:20:15)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> (InteractiveConsole)
> >>> len("á") # len() == 4 is not OK
> 4
> >>> "á" # 4 bytes: strange.
> '\xc3\x83\xc2\xa1'
> >>> # here I pressed CTRL-D to return from code.interact()
> >>> "á" # 2 bytes, ok again.
> '\xc3\xa1'
>
>
> Any ideas?
>
>
>
>
> On Fri, Sep 27, 2013 at 4:53 PM, Vinicius Assef
> <vinic...@gmail.com<javascript:>>
> wrote:
> > Anthony, I think you're in the right direction.
> > See:
> > # ---------------- begin
> > $ python
> > Python 2.7.3 (default, Apr 10 2013, 06:20:15)
> > [GCC 4.6.3] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> print "á"
> > á
> >>>> import code
> >>>> print "á"
> > á
> >>>> code.interact()
> > Python 2.7.3 (default, Apr 10 2013, 06:20:15)
> > [GCC 4.6.3] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > (InteractiveConsole)
> >>>> print "á"
> > á
> >>>> exit()
> > # ---------------- end
> >
> > The problem occurs after code.interact() call.
> >
> >
> >
> >
> > On Fri, Sep 27, 2013 at 4:38 PM, Anthony <abas...@gmail.com<javascript:>>
> wrote:
> >> What happens if you run a Python file that includes the following:
> >>
> >> import code
> >> code.interact()
> >>
> >> Anthony
> >>
> >>
> >> On Friday, September 27, 2013 3:33:45 PM UTC-4, viniciusban wrote:
> >>>
> >>> Anthony, I'm using Ubuntu server 12.04 inside a Vagrant machine with
> >>> provided by Virtualbox.
> >>>
> >>> Michele, my $LANG is exactly en_US.UTF-8, like yours.
> >>>
> >>> Would it be a problem with my OS? As I explained in first message, it
> >>> works with standard Python shell, without web2py environment loaded.
> >>>
> >>> The problem occurs only when I use web2py from shell.
> >>>
> >>>
> >>>
> >>> On Fri, Sep 27, 2013 at 4:08 PM, Michele Comitini
> >>> <michele....@gmail.com> wrote:
> >>> > On your terminal you have to look at locales setup. Issue the
> following:
> >>> >
> >>> > $ locales
> >>> >
> >>> > You should have utf-8 based locale. For instance:
> >>> >
> >>> > LANG=en_US.UTF-8
> >>> >
> >>> > On the browser look on the prefereces, see if you have
> locale/language
> >>> > settings supporting UTF-8.
> >>> > If you have UTF-8 both on terminal and browser, then there is a
> >>> > problem...
> >>> >
> >>> >
> >>> > 2013/9/27 Vinicius Assef <vinic...@gmail.com>
> >>> >>
> >>> >> I was making a maintenance in table data through shell.
> >>> >>
> >>> >> Despite that, I'm still curious to discover why this behaviour is
> >>> >> different from natural Python behaviour.
> >>> >>
> >>> >>
> >>> >> On Fri, Sep 27, 2013 at 3:53 PM, Richard Vézina
> >>> >> <ml.richa...@gmail.com> wrote:
> >>> >> > Ok, but why do you need to do that in web2py? I remember have
> read
> >>> >> > something
> >>> >> > about encoding in the book that was saying don't need to concern
> >>> >> > about
> >>> >> > that
> >>> >> > in web2py.
> >>> >> >
> >>> >> > Richard
> >>> >> >
> >>> >> >
> >>> >> > On Fri, Sep 27, 2013 at 2:46 PM, Vinicius Assef <
> vinic...@gmail.com>
> >>> >> > wrote:
> >>> >> >>
> >>> >> >> On Fri, Sep 27, 2013 at 3:37 PM, Richard Vézina
> >>> >> >> <ml.richa...@gmail.com> wrote:
> >>> >> >> > Ha ok, I was miss leading by the linux shell... Could it be
> that
> >>> >> >> > you
> >>> >> >> > need
> >>> >> >> > u('éà') ??
> >>> >> >>
> >>> >> >> No. the "áéíóú".decode("utf-8") should do that.
> >>> >> >> And really does, in standard Python shell, outside web2py.
> >>> >> >>
> >>> >> >>
> >>> >> >> >
> >>> >> >> >
> >>> >> >> > On Fri, Sep 27, 2013 at 1:47 PM, Vinicius Assef
> >>> >> >> > <vinic...@gmail.com>
> >>> >> >> > wrote:
> >>> >> >> >>
> >>> >> >> >> It wasn't a bash script, Richard.
> >>> >> >> >>
> >>> >> >> >> I made exactly as I described: straight from terminal.
> >>> >> >> >>
> >>> >> >> >> On Fri, Sep 27, 2013 at 2:23 PM, Richard Vézina
> >>> >> >> >> <ml.richa...@gmail.com> wrote:
> >>> >> >> >> > Could it be the encoding of the file that content your bash
> >>> >> >> >> > script?
> >>> >> >> >> >
> >>> >> >> >> > Richard
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> > On Fri, Sep 27, 2013 at 12:23 PM, Vinicius Assef
> >>> >> >> >> > <vinic...@gmail.com>
> >>> >> >> >> > wrote:
> >>> >> >> >> >>
> >>> >> >> >> >> Hey all.
> >>> >> >> >> >>
> >>> >> >> >> >> I'm facing a strange problem concerning special
> characters.
> >>> >> >> >> >>
> >>> >> >> >> >> Using web2py shell, through command line (linux shell), I
> do
> >>> >> >> >> >> this:
> >>> >> >> >> >>
> >>> >> >> >> >> # ---------------- start
> >>> >> >> >> >> $ python web2py.py -S welcome
> >>> >> >> >> >> web2py Web Framework
> >>> >> >> >> >> Created by Massimo Di Pierro, Copyright 2007-2013
> >>> >> >> >> >> Version 2.6.4-stable+timestamp.2013.09.22.01.43.37
> >>> >> >> >> >> Database drivers available: SQLite(sqlite3),
> MySQL(pymysql),
> >>> >> >> >> >> PostgreSQL(pg8000), IMAP(imaplib)
> >>> >> >> >> >> WARNING:web2py:import IPython error; use default python
> shell
> >>> >> >> >> >> Python 2.7.3 (default, Apr 10 2013, 06:20:15)
> >>> >> >> >> >> [GCC 4.6.3] on linux2
> >>> >> >> >> >> Type "help", "copyright", "credits" or "license" for more
> >>> >> >> >> >> information.
> >>> >> >> >> >> (InteractiveConsole)
> >>> >> >> >> >> >>> print "áéíóú" # note the strange chars!
> >>> >> >> >> >> áéÃóú
> >>> >> >> >> >> >>> from unicodedata import normalize
> >>> >> >> >> >> >>> normalize("NFKD",
> >>> >> >> >> >> >>> "áéuíóú".decode("utf-8")).encode("ASCII",
> >>> >> >> >> >> >>> "ignore")
> >>> >> >> >> >> >>> # again!
> >>> >> >> >> >> 'AAuAA3Ao'
> >>> >> >> >> >> # ---------------- end
> >>> >> >> >> >>
> >>> >> >> >> >> Note the noisy characters on the last line.
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >> Then, at the same machine, I enter into the naked Python
> shell
> >>> >> >> >> >> and:
> >>> >> >> >> >> # ---------------- start
> >>> >> >> >> >> $ python
> >>> >> >> >> >> Python 2.7.3 (default, Apr 10 2013, 06:20:15)
> >>> >> >> >> >> [GCC 4.6.3] on linux2
> >>> >> >> >> >> Type "help", "copyright", "credits" or "license" for more
> >>> >> >> >> >> information.
> >>> >> >> >> >> >>> print "áéíóú" # looks ok
> >>> >> >> >> >> áéíóú
> >>> >> >> >> >> >>> from unicodedata import normalize
> >>> >> >> >> >> >>> normalize("NFKD",
> >>> >> >> >> >> >>> "áéuíóú".decode("utf-8")).encode("ASCII",
> >>> >> >> >> >> >>> "ignore")
> >>> >> >> >> >> >>> # ok again.
> >>> >> >> >> >> 'aeiou'
> >>> >> >> >> >> # ---------------- end
> >>> >> >> >> >>
> >>> >> >> >> >> Everything works fine.
> >>> >> >> >> >>
> >>> >> >> >> >> The same code.
> >>> >> >> >> >>
> >>> >> >> >> >> I found this problem because I'm working to strip special
> >>> >> >> >> >> characters
> >>> >> >> >> >> off of names, with unicodedata.normalize().
> >>> >> >> >> >>
> >>> >> >> >> >> If it wan't strange enough, the unicodedata.normalize()
> works
> >>> >> >> >> >> fine
> >>> >> >> >> >> from web environment, throug app admin. There the
> >>> >> >> >> >> normalization
> >>> >> >> >> >> works
> >>> >> >> >> >> as expected (the same way as the standard naked Python
> shell).
> >>> >> >> >> >>
> >>> >> >> >> >> What could be the root of this situation?
> >>> >> >> >> >>
> >>> >> >> >> >> --
> >>> >> >> >> >> Resources:
> >>> >> >> >> >> - http://web2py.com
> >>> >> >> >> >> - http://web2py.com/book (Documentation)
> >>> >> >> >> >> - http://github.com/web2py/web2py (Source code)
> >>> >> >> >> >> - https://code.google.com/p/web2py/issues/list (Report
> Issues)
> >>> >> >> >> >> ---
> >>> >> >> >> >> You received this message because you are subscribed to
> the
> >>> >> >> >> >> Google
> >>> >> >> >> >> Groups
> >>> >> >> >> >> "web2py-users" group.
> >>> >> >> >> >> To unsubscribe from this group and stop receiving emails
> from
> >>> >> >> >> >> it,
> >>> >> >> >> >> send
> >>> >> >> >> >> an
> >>> >> >> >> >> email to web2py+un...@googlegroups.com.
> >>> >> >> >> >> For more options, visit
> >>> >> >> >> >> https://groups.google.com/groups/opt_out.
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> > --
> >>> >> >> >> > Resources:
> >>> >> >> >> > - http://web2py.com
> >>> >> >> >> > - http://web2py.com/book (Documentation)
> >>> >> >> >> > - http://github.com/web2py/web2py (Source code)
> >>> >> >> >> > - https://code.google.com/p/web2py/issues/list (Report
> Issues)
> >>> >> >> >> > ---
> >>> >> >> >> > You received this message because you are subscribed to the
> >>> >> >> >> > Google
> >>> >> >> >> > Groups
> >>> >> >> >> > "web2py-users" group.
> >>> >> >> >> > To unsubscribe from this group and stop receiving emails
> from
> >>> >> >> >> > it,
> >>> >> >> >> > send
> >>> >> >> >> > an
> >>> >> >> >> > email to web2py+un...@googlegroups.com.
> >>> >> >> >> > For more options, visit
> >>> >> >> >> > https://groups.google.com/groups/opt_out.
> >>> >> >> >>
> >>> >> >> >> --
> >>> >> >> >> Resources:
> >>> >> >> >> - http://web2py.com
> >>> >> >> >> - http://web2py.com/book (Documentation)
> >>> >> >> >> - http://github.com/web2py/web2py (Source code)
> >>> >> >> >> - https://code.google.com/p/web2py/issues/list (Report
> Issues)
> >>> >> >> >> ---
> >>> >> >> >> You received this message because you are subscribed to the
> >>> >> >> >> Google
> >>> >> >> >> Groups
> >>> >> >> >> "web2py-users" group.
> >>> >> >> >> To unsubscribe from this group and stop receiving emails from
> it,
> >>> >> >> >> send
> >>> >> >> >> an
> >>> >> >> >> email to web2py+un...@googlegroups.com.
> >>> >> >> >> For more options, visit
> https://groups.google.com/groups/opt_out.
> >>> >> >> >
> >>> >> >> >
> >>> >> >> > --
> >>> >> >> > Resources:
> >>> >> >> > - http://web2py.com
> >>> >> >> > - http://web2py.com/book (Documentation)
> >>> >> >> > - http://github.com/web2py/web2py (Source code)
> >>> >> >> > - https://code.google.com/p/web2py/issues/list (Report
> Issues)
> >>> >> >> > ---
> >>> >> >> > You received this message because you are subscribed to the
> Google
> >>> >> >> > Groups
> >>> >> >> > "web2py-users" group.
> >>> >> >> > To unsubscribe from this group and stop receiving emails from
> it,
> >>> >> >> > send
> >>> >> >> > an
> >>> >> >> > email to web2py+un...@googlegroups.com.
> >>> >> >> > For more options, visit
> https://groups.google.com/groups/opt_out.
> >>> >> >>
> >>> >> >> --
> >>> >> >> Resources:
> >>> >> >> - http://web2py.com
> >>> >> >> - http://web2py.com/book (Documentation)
> >>> >> >> - http://github.com/web2py/web2py (Source code)
> >>> >> >> - https://code.google.com/p/web2py/issues/list (Report Issues)
> >>> >> >> ---
> >>> >> >> You received this message because you are subscribed to the
> Google
> >>> >> >> Groups
> >>> >> >> "web2py-users" group.
> >>> >> >> To unsubscribe from this group and stop receiving emails from
> it,
> >>> >> >> send
> >>> >> >> an
> >>> >> >> email to web2py+un...@googlegroups.com.
> >>> >> >> For more options, visit https://groups.google.com/groups/opt_out.
>
> >>> >> >
> >>> >> >
> >>> >> > --
> >>> >> > Resources:
> >>> >> > - http://web2py.com
> >>> >> > - http://web2py.com/book (Documentation)
> >>> >> > - http://github.com/web2py/web2py (Source code)
> >>> >> > - https://code.google.com/p/web2py/issues/list (Report Issues)
> >>> >> > ---
> >>> >> > You received this message because you are subscribed to the
> Google
> >>> >> > Groups
> >>> >> > "web2py-users" group.
> >>> >> > To unsubscribe from this group and stop receiving emails from it,
> >>> >> > send
> >>> >> > an
> >>> >> > email to web2py+un...@googlegroups.com.
> >>> >> > For more options, visit https://groups.google.com/groups/opt_out.
>
> >>> >>
> >>> >> --
> >>> >> Resources:
> >>> >> - http://web2py.com
> >>> >> - http://web2py.com/book (Documentation)
> >>> >> - http://github.com/web2py/web2py (Source code)
> >>> >> - https://code.google.com/p/web2py/issues/list (Report Issues)
> >>> >> ---
> >>> >> You received this message because you are subscribed to the Google
> >>> >> Groups
> >>> >> "web2py-users" group.
> >>> >> To unsubscribe from this group and stop receiving emails from it,
> send
> >>> >> an
> >>> >> email to web2py+un...@googlegroups.com.
> >>> >> For more options, visit https://groups.google.com/groups/opt_out.
> >>> >
> >>> >
> >>> > --
> >>> > Resources:
> >>> > - http://web2py.com
> >>> > - http://web2py.com/book (Documentation)
> >>> > - http://github.com/web2py/web2py (Source code)
> >>> > - https://code.google.com/p/web2py/issues/list (Report Issues)
> >>> > ---
> >>> > You received this message because you are subscribed to the Google
> >>> > Groups
> >>> > "web2py-users" group.
> >>> > To unsubscribe from this group and stop receiving emails from it,
> send
> >>> > an
> >>> > email to web2py+un...@googlegroups.com.
> >>> > For more options, visit https://groups.google.com/groups/opt_out.
> >>
> >> --
> >> Resources:
> >> - http://web2py.com
> >> - http://web2py.com/book (Documentation)
> >> - http://github.com/web2py/web2py (Source code)
> >> - https://code.google.com/p/web2py/issues/list (Report Issues)
> >> ---
> >> You received this message because you are subscribed to the Google
> Groups
> >> "web2py-users" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to web2py+un...@googlegroups.com <javascript:>.
> >> For more options, visit https://groups.google.com/groups/opt_out.
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
