On 09 Apr 2014, at 19:35, itli...@schrievkrom.de wrote: > Ok, forget the JSON stuff - it has nothing to do with the "problem". > > Other way round: > > My whole database and internal processing is done in UTF8. This is the > most important point here to mention.
Why ? This means you forgo almost all String functionality, since UTF8 is a variable length encoding not really suitable to character by character processing. > Now the request comes into Zinc as mentioned below (the content of the > request is a JSON string only): > > HTML-Request (charset=UTF-8) =(sends)=> ZINC HTTP > > Now Zinc sees the content of the body, knows that it is coded in UTF8 > and creates a ZnStringEntity with UTF8Encoder. > > Zinc HTTP =(builds)=> ZnStringEntity (with UTF8Encoder) > > The instance of ZnRequest and its entity value is an instance of > ZnStringEntity (with its encoder attribute is set to an instance to > ZnUTF8Encoder). Yes, of course, UTF-8 (a variable length binary encoding) is converted into native Pharo Strings (possibly WideStrings) containing Characters, each of which is encoded using a Unicode code point value. > I checked the content of the string attribute of the ZnStringEntity and > this string is NOT encoded in UTF8 any more, but in either ISO8859-? > or WIN1252. Here you lose me (again) ;-) > I think, that this is ok for almost all people, because they work with > some CodePages - but my internal processing assumes UTF8. No, nobody works with code pages or any encoding, just native [Wide]Strings in pure Unicode. > I just fixed this for me by changing ZnStringEntity>>initializeEncoder > to ALWAYS set the encoder attribute to ZnNullEncoder and now everthing > is ok again. This means of course, that all apllication running with > that source code work in UTF 8 only ... OK, I think I understand, you want UTF-8 to remain UTF-8. What you did is one solution, but I think it is wrong to use a String to represent bytes. This case is actually already implemented server side for Seaside: ZnZincServerAdaptor>>#configureServerForBinaryReading "Seaside wants to do its own text conversions" server reader: [ :stream | ZnRequest readBinaryFrom: stream ] The #reader: option is used here to read everything binary, without decoding to Strings. You will get ZnZnByteArrayEntity objects back, containing the original binary representation. BTW, I think this is an interesting discussion. Regards, Sven > Marten > > Am 09.04.2014 18:42, schrieb Sven Van Caekenberghe: >> Marten, >> >> On 09 Apr 2014, at 18:25, itli...@schrievkrom.de wrote: >> >>> Ok, if the browser sends POST/PUT request with a JSON structure it also >>> sends charset = utf8 (in my case). That's ok, because for JSON this is >>> more or less the default charset. >>> >>> Zinc now seems to notice, that UTF8 charset is needed and creates a >>> ZnStringEntity with an UTF8Encoder. >>> >>> Now when my application tries to get the JSON string of that >>> ZnStringEntity and builds the structure out of that string - and the >>> strings are NOT UTF8, but converted to (?) ISO8859 ? >> >> (NeoJSONReader fromString: >> (ZnEntity with: (NeoJSONWriter toString: { #message -> 'An der schönen >> blauen Donau' } asDictionary))) >> at: #message. >> >> You must be doing something possibly wrong when you <<get the JSON string of >> that ZnStringEntity and builds the structure out of that string>> (how do >> you do that, BTW), so please write some code that demonstrates what is not >> right according to you. >> >> Sven >> > > > -- > Marten Feldtmann >