Unicode issue on Windows cmd line
Having issue on Windows cmd. > Python.exe >>>a = u'\xf0' >>>print a This gives a unicode error. Works fine in IDLE, PythonWin, and my Macbook but I need to run this from a windows batch. Character should look like this "ð". Please help! -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode issue on Windows cmd line
On Feb 11, 2:35 pm, Albert Hopkins wrote: > On Wed, 2009-02-11 at 10:35 -0800, jeffg wrote: > > Having issue on Windows cmd. > > > Python.exe > > >>>a = u'\xf0' > > >>>print a > > > This gives a unicode error. > > > Works fine in IDLE, PythonWin, and my Macbook but I need to run this > > from a windows batch. > > > Character should look like this "ð". > > > Please help! > > You forgot to paste the error. The error looks like this: File " File "C:\python25\lib\encodings\cp437.py", line 12, in encode return codecs.charmap_encode(input,errors,encoding_map) UnicodeEncodeError: 'charmap' codec can't encode character u'\xf0' in position 0 : character maps to Running Python 2.5.4 on Windows XP -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode issue on Windows cmd line
On Feb 11, 3:57 pm, "Martin v. Löwis" wrote:
> > Having issue on Windows cmd.
> >> Python.exe
> a = u'\xf0'
> print a
>
> > This gives a unicode error.
>
> > Works fine in IDLE, PythonWin, and my Macbook but I need to run this
> > from a windows batch.
>
> > Character should look like this "ð".
>
> > Please help!
>
> Well, your terminal just cannot display this character by default; you
> need to use a different terminal program, or reconfigure your terminal.
>
> For example, do
>
> chcp 1252
>
> and select Lucida Console as the terminal font, then try again.
>
> Of course, this will cause *different* characters to become
> non-displayable.
>
> Regards,
> Martin
Thanks, I ended up using encode('iso-8859-15', "replace")
Perhaps more up to date than cp1252...??
It still didn't print correctly, but it did write correctly, which was
my main problem.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unicode issue on Windows cmd line
On Feb 11, 6:30 pm, "Martin v. Löwis" wrote:
> > Thanks, I ended up using encode('iso-8859-15', "replace")
> > Perhaps more up to date than cp1252...??
>
> > It still didn't print correctly, but it did write correctly, which was
> > my main problem.
>
> If you encode as iso-8859-15, but this is not what your terminal
> expects, it certainly won't print correctly. To get correct printing,
> the output encoding must be the same as the terminal encoding. If the
> terminal encoding is not up to date (as you consider cp1252), then
> the output encoding should not be up to date, either.
>
> If you want a modern encoding that supports all of Unicode, and you
> don't care whether the output is legible, use UTF-8.
>
> Regards,
> Martin
I did try UTF-8 but it produced the upper case character instead of
the proper lower case, so the output was incorrect for the unicode
supplied.
I think both 8859-15 and cp1252 produced the correct output, but I
figured 8859-15 would have additional character support (though not
sure this is the case - if it is not, please let me know and I'll use
1252). I'm dealing with large data sets and this just happend to be
one small example. I want to have the best ability to write future
unicode characters properly based on running from the windows command
line (unless there is a better way to do it on windows).
--
http://mail.python.org/mailman/listinfo/python-list
Another optimization request :-)
If anyone wants to take this on... I would really really like to have
the spring_layout modified to support multi-threading if at all
possible.
My test data is 20,000, which makes this process 20,000 x 20,000 or
400,000,000 (400 million) calculations. This is taking somewhere
between 2-3 hours an iteration.
I plan to plot out over 1,000,000 data points or more, which would put
this at 1,000,000,000,000 (1 trillion) calculations. Any help in
making this more efficient would be much appreciated.
def spring_layout(G, iterations=50, dim=2, node_pos=None,
verbose=False):
"""Spring force model layout"""
if node_pos==None : # set the initial positions randomly in 1x1
box
vpos=random_layout(G, dim=dim)
else:
vpos=node_pos
if iterations==0:
return vpos
if G.order()==0:
k=1.0
else:
k=N.sqrt(1.0/G.order()) # optimal distance between nodes
disp={} # displacements
# initial "temperature" (about .1 of domain area)
# this is the largest step allowed in the dynamics
# linearly step down by dt on each iteration so
# on last iteration it is size dt.
t=0.1
dt=0.1/float(iterations+1)
for i in range(0,iterations):
for v in G:
if verbose==True:
print("Interation: " + str(i + 1) + ", Calculating: "
+ str(v.encode('iso-8859-15', "replace")))
disp[v]=N.zeros(dim)
for u in G:
delta=vpos[v]-vpos[u]
dn=max(sqrt(N.dot(delta,delta)),0.01)
# repulsive force between all
deltaf=delta*k**2/dn**2
disp[v]=disp[v]+deltaf
# attractive force between neighbors
if G.has_edge(v,u):
deltaf=-delta*dn**2/(k*dn)
disp[v]=disp[v]+deltaf
# update positions
for v in G:
l=max(sqrt(N.dot(disp[v],disp[v])),0.01)
vpos[v]=vpos[v]+ disp[v]*t/l
t-=dt
return vpos
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unicode issue on Windows cmd line
On Feb 11, 10:00 pm, "Gabriel Genellina"
wrote:
> En Wed, 11 Feb 2009 23:11:37 -0200, jeffg escribió:
>
>
>
> > On Feb 11, 6:30 pm, "Martin v. Löwis" wrote:
> >> > Thanks, I ended up using encode('iso-8859-15', "replace")
> >> > Perhaps more up to date than cp1252...??
> >> If you encode as iso-8859-15, but this is not what your terminal
> >> expects, it certainly won't print correctly. To get correct printing,
> >> the output encoding must be the same as the terminal encoding. If the
> >> terminal encoding is not up to date (as you consider cp1252), then
> >> the output encoding should not be up to date, either.
> > I did try UTF-8 but it produced the upper case character instead of
> > the proper lower case, so the output was incorrect for the unicode
> > supplied.
> > I think both 8859-15 and cp1252 produced the correct output, but I
> > figured 8859-15 would have additional character support (though not
> > sure this is the case - if it is not, please let me know and I'll use
> > 1252). I'm dealing with large data sets and this just happend to be
> > one small example. I want to have the best ability to write future
> > unicode characters properly based on running from the windows command
> > line (unless there is a better way to do it on windows).
>
> As Martin v. Löwis already said, the encoding used by Python when writing
> to the console, must match the encoding the console expects. (And you also
> should use a font capable of displaying such characters).
>
> windows-1252 and iso-8859-15 are similar, but not identical. This table
> shows the differences (less than 30 printable characters):
> http://en.wikipedia.org/wiki/Western_Latin_character_sets_(computing)
> If your script encodes its output using iso-8859-15, the corresponding
> console code page should be 28605.
> "Western European" (whatever that means exactly) Windows versions use the
> windows-1252 encoding as the "Ansi code page" (GUI applications), and
> cp850 as the "OEM code page" (console applications) -- cp437 in the US
> only.
>
> C:\Documents and Settings\Gabriel>chcp 1252
> Tabla de códigos activa: 1252
>
> C:\Documents and Settings\Gabriel>python
> Python 2.6 (r26:66721, Oct 2 2008, 11:35:03) [MSC v.1500 32 bit (Intel)]
> on win
> 32
> Type "help", "copyright", "credits" or "license" for more information.
> py> unichr(0x0153).encode("windows-1252")
> '\x9c'
> py> print _
> œ
> py> ^Z
>
> C:\Documents and Settings\Gabriel>chcp 28605
> Tabla de códigos activa: 28605
>
> C:\Documents and Settings\Gabriel>python
> Python 2.6 (r26:66721, Oct 2 2008, 11:35:03) [MSC v.1500 32 bit (Intel)]
> on win
> 32
> Type "help", "copyright", "credits" or "license" for more information.
> py> unichr(0x0153).encode("iso-8859-15")
> '\xbd'
> py> print _
> œ
> py> unichr(0x0153).encode("latin9")
> '\xbd'
>
> --
> Gabriel Genellina
Thanks, switched it to windows-1252.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Another optimization request :-)
On Feb 11, 9:56 pm, "andrew cooke" wrote:
> sorry, that was stupid. there is no inversion (apart from 1/m), just the
> integration.
>
> still, improving the integration would allow larger steps.
>
> andrew
>
> andrew cooke wrote:
>
> > why are you dong this point by point? surely you can express the physics
> > as a set of equations and invert the matrix? wouldn't that be a lot
> > faster? you'd replace the iteration over all combinations of points with
> > a faster matrix inversion.
>
> > see for example
> >http://www.medwelljournals.com/fulltext/ajit/2006/324-338.pdfpage 331
> > onwards.
>
> > there's a very nice into to the verlet integration mentioned here -
> >http://teknikus.dk/tj/gdc2001.htm
>
> > andrew
>
> > jeffg wrote:
> >> If anyone wants to take this on... I would really really like to have
> >> the spring_layout modified to support multi-threading if at all
> >> possible.
> >> My test data is 20,000, which makes this process 20,000 x 20,000 or
> >> 400,000,000 (400 million) calculations. This is taking somewhere
> >> between 2-3 hours an iteration.
> >> I plan to plot out over 1,000,000 data points or more, which would put
> >> this at 1,000,000,000,000 (1 trillion) calculations. Any help in
> >> making this more efficient would be much appreciated.
>
> >> def spring_layout(G, iterations=50, dim=2, node_pos=None,
> >> verbose=False):
> >> """Spring force model layout"""
> >> if node_pos==None : # set the initial positions randomly in 1x1
> >> box
> >> vpos=random_layout(G, dim=dim)
> >> else:
> >> vpos=node_pos
> >> if iterations==0:
> >> return vpos
> >> if G.order()==0:
> >> k=1.0
> >> else:
> >> k=N.sqrt(1.0/G.order()) # optimal distance between nodes
> >> disp={} # displacements
>
> >> # initial "temperature" (about .1 of domain area)
> >> # this is the largest step allowed in the dynamics
> >> # linearly step down by dt on each iteration so
> >> # on last iteration it is size dt.
> >> t=0.1
> >> dt=0.1/float(iterations+1)
> >> for i in range(0,iterations):
> >> for v in G:
> >> if verbose==True:
> >> print("Interation: " + str(i + 1) + ", Calculating: "
> >> + str(v.encode('iso-8859-15', "replace")))
> >> disp[v]=N.zeros(dim)
> >> for u in G:
> >> delta=vpos[v]-vpos[u]
> >> dn=max(sqrt(N.dot(delta,delta)),0.01)
> >> # repulsive force between all
> >> deltaf=delta*k**2/dn**2
> >> disp[v]=disp[v]+deltaf
> >> # attractive force between neighbors
> >> if G.has_edge(v,u):
> >> deltaf=-delta*dn**2/(k*dn)
> >> disp[v]=disp[v]+deltaf
>
> >> # update positions
> >> for v in G:
> >> l=max(sqrt(N.dot(disp[v],disp[v])),0.01)
> >> vpos[v]=vpos[v]+ disp[v]*t/l
> >> t-=dt
> >> return vpos
> >> --
> >>http://mail.python.org/mailman/listinfo/python-list
>
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
>
To be honest, this is not my code and I'm new to python. It's part of
the open source project NetworkX, but I'm using this one call
extensively. I'm also not that familiar with the math behind the
physics. I'll read the documents and see if I can figure it
out. :-) Thank you for the links and suggestions. I really need to
get this code performing at peak levels.
--
http://mail.python.org/mailman/listinfo/python-list
