On Thu, 28 Feb 2002, Bernd Eidenschink wrote: > Hi, > > I don't know how to set up a combination of the latest AOLserver, > using the nsd8x Interpreter, and a Postgres 7.2 database, that allows > me to safely work with a charset of iso8859-1. Please don't throw > stones, I know this has been discussed very often ;-) > > How to do it? > > The problems I run into during my own tests and the problems that > other people have (I read through some threads on various boards) are: > > A. > Using the latest server "out of the box", working with most European > characters will almost always fail using the typical string and regexp > functions that internally use utf-8 ... _IF_ you are returning (in http- > header or meta-tags) a charset of e.g. iso8859-1 (you have to > know what comes from forms/submits; you try to return "umlauts" > or any language depending chars and tell the browser of it). > > B. > I got the tip of using an undocumented parameter in the config > file, that maps the iso8859-1 charset to ".adp" files (last year on this > list). > But this does not guarantee that all characters that _leave_ an > adp will be in iso8859-1 encoding (e.g. if I use combinations of > "ns_adp_parse -file" (where strings from the DB are regexped and > stringed and whatever) and return the string with "ns_return"). > At least if all what comes in goes iso-ed to the DB, you could do > a workaround and translate all outgoing chars to "{" html code. > (There was an example of a ns_adp_puts function that does this > given by Harray Moreau on the list)... If you "escape" all the characters > that way, I assume, you would not no longer have to return a > charset header or charset-meta-tag. > > C. > The uncool way: Using "charset=utf-8" outgoing, then also > expecting it incoming. A special character should come in as unicode > and tcl should treat it this way. The database must be installed with > unicode-encoding. You will run into performace problems and/or, maybe, > some unresolved topics of Postgres unicode-implementation as well. > I have not tried this yet, I merely assume this would work. What's > your opinion on that? > > D. > Using an AD-patched version of AOLserver. The Problem: It's an > older version of the server, will it be kept up to date in the future? > (Of course, there's a large user base running it) > > E. > I did not try --enable-recode and setting up a charset table for > utf8 -> latin1 and latin1 -> utf8 for Postgres. Maybe this would > work if you can guarantee all charsets coming from AOLserver > are coming as utf8 or iso8859x. Maybe, don't know. > I tried putting "SET ENCODING TO 'UNICODE'" resp. "LATIN1" in front > of every SQL statement in my test api (and took into account that it may > make a > difference if you are SELECTing or INSERTing) for telling the Postgres > server that the client uses unicode or latin with and without the > undocumented feature of (B). This only lead to error messages > noticing me of failed encoding translations. Every insert _always_ > was logged with german special characters (umlauts) and never > as unicode characters (don't know if this is correct). > > Will AOLserver 4 come with I18N support that solves all the problems > and what to do if you need a solution here and now? > Solution B? C? > > > Thanks for reading through this one... > > Bernd. >
Kriston from AOL assured me that AOLserver 4 will have full international support. But I haven't inspected availabale code and verified that yet. Now I successfully run on 3.4.x dealing with iso8859-2 in the following way (I'm sure that you can do this the same way with iso8859-1 = latin1). * AOLserver scripts I use only .adp scripts with undocumented config ns_section "ns/encodings" ns_param "adp" iso8859-2 Of course I use *.tcl libraries with proper encoding set by [encoding system iso8859-2]. I don't use *.tcl pages at all because as far as I remeber ns_return* and ns_write don't do proper encoding translation. If you have to use *.tcl scripts (for example you are ACS user) I think you should use ArsDigita version today. * Databases If I use Postgres - I configure my database to use UTF-8 encoding createdb -E unicode database_name_in_utf8 and I set in the AOLserver process enviroment PGCLIENTENCODIGN=latin2 for postgres libraries. I suggest using UTF-8 for Postgres database rather than using 8-bit encoding in database (ex. latin2) and translating TCL strings which are send in unicode to some 8-bit encoding (it is possible by postgres configure option --enable-unicode-conversion). Main reason is that when running database in latin2 I had problems with encoding in very usefull Postgres procedural languages: pltcl & pltclu. You may also encounter problems with char expansion which I discovered with Oracle. Anyway for Oracle I successfully used database in latin2 encoding with CharExpansion=4 parameter added to ArsDigita Oracle Driver 2.6 as a result of my proposal. * Other stuff With 3.4 I use the following simple patch for ns_urlencode. This is not general solution, but rather dirty fix. [tkosiak@mule nsd]$ diff urlencode.c.orig urlencode.c 72a73,81 > Tcl_DString ds; > char* encname; > > Tcl_DStringInit(&ds); > > encname = Ns_ConfigGet(NS_CONFIG_PARAMETERS, "urlencoding"); > if ( encname != NULL ) { > string = NsUtf2Ext(NsGetEnc(encname),string,&ds); > } Than you have to configure ns_urlencode: ns_section "ns/parameters" ns_param urlencoding iso8859-2 In 3.4 ns_urldecode works fine in adp scripts with properly configured encoding. You should also inspect and probably fix procs defined in shared TCL library: ns_httpopen in aolserver/modules/tcl/http.tcl ns_sendmail in aolserver/modules/tcl/sendmail.tcl They should use [fconfigure -encoding %$fd] to properly handle encodings. But this is optional if you don't use these features. --tkosiak