The cleanes interface for the JSON parser/serializer would be to
receive and produce UnicodeStrings and let the programmer worry about
encoding.
I see. An alternative is, in the case when you read "\uXXXX", to just
return Strings. To add a UnicodeCharacter to a String stream, you just use
aStream display: aCharacter
A full implementation would probably require adding a method like this:
PositionableStream >> encoding
^collection encoding
and I can take care of a more complete implementation of Stream encoding.
There are many ways to specify encoding, for example the following:
1) add a #on:encoding: constructor where the encoding defaults to
'UTF-8'. When creating a String to be returned, use the same encoding
as the input.
2) use the aforementioned PositionableStream >> encoding method; when
creating a String to be returned, use the same encoding as the input.
3) use the aforementioned PositionableStream >> encoding method and add
a #on:outputEncoding: constructor, where the encoding defaults to the
same encoding as the input.
4) use the aforementioned PositionableStream >> encoding method and
always return UnicodeStrings. In this case, you will never find
Characters whose value is >= 128 in the input (you'll find
UnicodeCharacters instead!).
Hm, I agree that hasing Strings in their UTF-8 encoded form is a good
approximation.
Which will of course horribly break if someone chooses to use eg. german
"umlaute"
in the source code in latin-1 encoding, or maybe not. How is the encoding of a
literal string determined?
It is not so far, and unless one is interested in using Strings and
UnicodeStrings interchangeably for hashing, you should not care. Do you
have example of prior art for other languages?
ASCII characters and UTF-8 please. :-) I'm also from a Latin-1 country,
but I try to think as international as possible. :-)
That Smalltalk source code literals come in UTF-8 encoded form is a bold
assumption (which is increasingly right these days on Linux and other OSs :-)
Yes.
Paolo
_______________________________________________
help-smalltalk mailing list
[email protected]
http://lists.gnu.org/mailman/listinfo/help-smalltalk
