Steffen Daode Nurpmeso <sdao...@googlemail.com> added the comment: I wonder what this normalize_encoding() does! Here is a pretty standard version of mine which is a bit more expensive but catches match more cases! This is stripped, of course, and can be rewritten very easily to Python's needs (i.e. using char[32] instead of char[11].
* @@li If a character is either ::s_char_is_space() or ::s_char_is_punct(): * @@li Replace with ASCII space (0x20). * @@li Squeeze adjacent spaces to a single one. * @@li Else if a character is ::s_char_is_alnum(): * @@li ::s_char_to_lower() characters. * @@li Separate groups of alphas and digits with ASCII space (0x20). * @@li Else discard character. * E.g. "ISO_8859---1" becomes "iso 8859 1" * and "ISO8859-1" also becomes "iso 8859 1". s_textcodec_normalize_name(s_CString *_name) { enum { C_NONE, C_WS, C_ALPHA, C_DIGIT } c_type = C_NONE; char *name, c; auto s_CString input; s_cstring_swap(s_cstring_init(&input), _name); _name = s_cstring_reserve(_name, 31, s_FAL0); name = s_cstring_cstr(&input); while ((c = *(name++)) != s_NUL) { s_si8 sep = s_FAL0; if (s_char_is_space(c) || s_char_is_punct(c)) { if (c_type == C_WS) continue; c_type = C_WS; c = ' '; } else if (s_char_is_alpha(c)) { sep = (c_type == C_DIGIT); c_type = C_ALPHA; c = s_char_to_lower(c); } else if (s_char_is_digit(c)) { sep = (c_type == C_ALPHA); c_type = C_DIGIT; } else continue; do _name = s_cstring_append_char(_name, (sep ? ' ' : c)); while (--sep >= s_FAL0); } s_cstring_destroy(&input); return _name; } ---------- nosy: +sdaoden _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue11303> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com