2011/6/22 Saul Spatz <saul.sp...@gmail.com>: > Hi, > > I'm just starting to learn a bit about Unicode. I want to be able to read a > utf-8 encoded file, and print out the codepoints it encodes. After many > false starts, here's a script that seems to work, but it strikes me as > awfully awkward and unpythonic. Have you a better way? > > def codePoints(s): > ''' return a list of the Unicode codepoints in the string s ''' > answer = [] > skip = False > for k, c in enumerate(s): > if skip: > skip = False > answer.append(ord(s[k-1:k+1])) > continue > if not 0xd800 <= ord(c) <= 0xdfff: > answer.append(ord(c)) > else: > skip = True > return answer > > if __name__ == '__main__': > s = open('test.txt', encoding = 'utf8', errors = 'replace').read() > code = codePoints(s) > for c in code: > print('U+'+hex(c)[2:]) > > Thanks for any help you can give me. > > Saul > > > -- > http://mail.python.org/mailman/listinfo/python-list >
Hi, what functionality should codePoints(...) add over just iterating through the characters in the unicode string directly (besides filtering out the surrogates)? It seems, that you can just use s = open(r'C:\install\filter-utf-8.txt', encoding = 'utf8', errors = 'replace').read() for c in s: print('U+'+hex(ord(c))[2:]) or eventually add the condition before the print: if not 0xd800 <= ord(c) <= 0xdfff: you can also use string formatting to do the hex conversion and a more usual zero padding; the print(...) calls would be: "older style formatting" print("U+%04x"%(ord(c),)) or the newer, potentially more powerful way using format(...) print("U+{:04x}".format(ord(c))) hth, vbr -- http://mail.python.org/mailman/listinfo/python-list