On Tue, Jul 28, 2009 at 2:45 AM, Ryan Culpepper<ry...@ccs.neu.edu> wrote: > Robby Findler wrote: >> >> Hi all (Ryan?): I've got a question about pgsql. From what I can tell, >> string data is stored in the database in the latin-1 encoding >> (sql-data.ss line 191), but is then retrieved from the database in the >> utf-8 encoding (io.ss line 205). Am I getting that right? >> >> This doesn't mean much, but I changed planet's copy of io.ss to use >> bytes->string/latin-1 instead of bytes->string/utf-8, and I was able >> to avoid crashing (but the latin-1 encoding might not have any >> unencodable octets, so that isn't really saying too too much). > > I don't know of any reason for latin-1 to show up in string code... maybe > for "bytea" code. It might be a mistake from the port to 200, when string > and bytes split. So the change sounds good. > > Were you able to get a crash before? Or data corruption?
Yes, the current planet database contains the bytes #"Fl\341vio Cruz" as the name of one of the planet contributors. This is not a valid utf-8 encoding (and it does appear to be a latin-1 encoded string). When it is retrieved, I get the following error: bytes->string/utf-8: string is not a well-formed UTF-8 encoding: #"Fl\341vio Cruz" === context === parse:DataRow /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/p3.ss:87:5 /local/svn/plt/collects/scheme/promise.ss:94:4: loop1 /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:71:6: recv method in base% /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:469:6: query1:data-loop method in ...rivate/connection.ss:371:4 /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:402:10: loop /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:850:6: -map method in ...rivate/connection.ss:759:4 get-all-users /local/svn/plt/collects/scheme/private/misc.ss:68:7 > If you send me some code to produce it, I'll add it to the test suite. Unfortunately, I only have a database that has what appears to be bad data in it; I don't know how to make it happen with an empty database, but I guess that if you could get the line I listed to run, you'd make it happen. > Since the line numbers > you list don't correspond to recent spgsql releases, it's possible this bug > has already been fixed and planet is just using an old version. Oh, I was using version (2 3) of spgsql, sorry. Robby _________________________________________________ For list-related administrative tasks: http://list.cs.brown.edu/mailman/listinfo/plt-dev
