On 5/11/2011 11:44 PM, harrismh777 wrote:
Steven D'Aprano wrote:
You need to understand the difference between characters and bytes.
http://www.joelonsoftware.com/articles/Unicode.html
is also a good resource.
Thanks for being patient guys, here's what I've done:
astr="pound sign"
asym=" \u00A3"
afile=open("myfile", mode='w')
afile.write(astr + asym)
12
afile.close()
When I edit "myfile" with vi I see the 'characters' :
pound sign £
... same with emacs, same with gedit ...
When I hexdump myfile I see this:
0000000 6f70 6375 2064 6973 6e67 c220 00a3
This is *not* what I expected... well it is (little-endian) right up to
the 'c2' and that is what is confusing me....
I did not open the file with an encoding of UTF-8... so I'm assuming
UTF-16 by default (python3) so I was expecting a '00A3' little-endian as
'A300' but what I got instead was UTF-8 little-endian 'c2a3' ....
See my problem?... when I open the file with emacs I see the character
pound sign... same with gedit... they're all using UTF-8 by default. By
default it looks like Python3 is writing output with UTF-8 as default...
and I thought that by default Python3 was using either UTF-16 or UTF-32.
So, I'm confused here... also, I used the character sequence \u00A3
which I thought was UTF-16... but Python3 changed my intent to 'c2a3'
which is the normal UTF-8...
If you open a file as binary (bytes), you must write bytes, and they are
stored without transformation. If you open in text mode, you must write
text (string as unicode in 3.2) and Python will encode to bytes using
either some default or the encoding you specified in the open statement.
It does not matter how Python stored the unicode internally. Does this
help? Your intent is signalled by how you open the file.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
