Moin Juerd, Am 08.03.2010 um 16:15 schrieb Juerd Waalboer:
> Michael Ludwig skribis 2010-03-08 15:55 (+0100): >> Okay. But unless I'm completely misled, you can tell whether a >> string is supposed to contain characters (<- Encode::decode) or >> bytes (<- Encode::encode) > > The result of decode is a character string. > > The result of encode is a byte string. Thanks for confirming. > However, apart from looking at the source code and deducing the > intentions of the programmer, there is no way to tell whether a given > string is meant as a character or byte string, simply because there is > no technical representation of this intent in the string or its > metadata. > > Note that "characters" are the general case: a string is made of > characters. When every character value fits in a single byte, the string > can be used as a byte string. And clarifying further. > This bug forces us to look at the internal encoding and flags to come to > the conclusion that it is indeed a bug. Don't mistake this as a sign > that looking at the internal encoding or flags should ever happen in > actual code. Even if you work around the bug, make sure that you don't > make anything conditional on the current formatting of the string. > > Instead, coerce it to whatever you need by using utf8::downgrade or > utf8::upgrade. In your specific case, concatenation of two separate > parts is probably the most sane thing to do. Good. >>>> Am I mistaken in my expectation that while "\xa0" should be >>>> a byte, "\x{a0}" and "\x{00a0}" should be characters? > > Yes. These three escapes are supposed to be exactly the same. They > create a U+00A0 character, which happens to be perfectly usable as the > A0 byte when used as such, in a string that doesn't contain any > character greater than U+00FF. Okay. Let me try to see if I have understood correctly. Without the utf8 pragma in scope, "so\xa0ein\xa0Käse" with a-Umlaut stored as a sequence of two bytes in my source code will be stored internally as a sequence of 12 integers. With the utf8 pragma in scope, only 11 integers. I know I shouldn't care about the internals, but sometimes grokking the internals is helpful as an aide-mémoire, because it puts things into perspective that otherwise seem more arbitrary. -- Michael.Ludwig (#) XING.com