RE: Problem with DB and Char size (2)
Actually... there is a "native" Oracle driver, or rather, one that uses the OCI. It's usable, but certainly not complete. http://cvs.sourceforge.net/viewcvs.py/haskell-libs/libs/takusen/src/ However, I've only tested it with bog-standard ASCII Cstrings (my NLS_LANG setting specifies WE8ISO8859P1), so I've no idea how well it works with Unicode. Have you tried fiddling with your NLS_LANG environment variable? This determines the charset your data is converted to when sent from the DBMS to the client. You can set it either in a .bat script (just for that session), or modify the registry setting in: HKEY_LOCAL_MACHINE\SOFTWARE\Oracle\HOME0 What's it currently set to? Alistair. > -Original Message- > From: Santoemma Enrico [mailto:[EMAIL PROTECTED] > Sent: 27 January 2005 16:49 > To: Krasimir Angelov > Cc: glasgow-haskell-users@haskell.org > Subject: R: Problem with DB and Char size (2) > > This is exactly what happens: > (btw, the same thing happens in Python, so the trouble must > be in the Oracle odbc driver. But this is also what happens > today to any Haskell business application which connects to > Oracle, as - to my knowledge - there is no native Oracle > driver for Haskell) > > ... > stmt0 <- query c $ "select 'Ã' as TEST from dual" > stmt1 <- query c $ "select '" ++ chr(195):chr(131):[] ++ "' > as TEST from dual" > stmt2a<- query c $ "select '" ++ [chr(195*256 + 131)] ++ "' > as TEST from dual" > stmt2b<- query c $ "select '" ++ [chr(131)] ++ "' as TEST from dual" > stmt3 <- query c $ "select chr(195*256 + 131) as TEST from dual" > ... > > after execution on a UTF-8 Oracle9i instance: > 0: "TEST", "\195\131" > 1: "TEST", "\195\131\198\146" > 2a: "TEST", "\198\146" > 2b: "TEST", "\198\146" > 3: "TEST", "\195\131" > > 0 is the reference: UTF-8 encoding of an A with a tilde on the top. > 1 is UTF-8 encoding (by Oracle) of an already encoded UTF-8 string > 2a and 2b show that the high byte is stripped: \198\146 is > the UTF-8 encoding of chr(131) > 3 is the only (almost useless) workaround I've found. > > Of course, with 3 byte chars things get even more confused. > > Enrico > > > > -Messaggio originale- > Da: Krasimir Angelov [mailto:[EMAIL PROTECTED] > Inviato: giovedì 27 gennaio 2005 15.41 > A: glasgow-haskell-users@haskell.org > Oggetto: Re: Problem with DB and Char size (2) > > > On Thu, 27 Jan 2005 06:34:40 -0800, John Meacham > <[EMAIL PROTECTED]> wrote: > > On Thu, Jan 27, 2005 at 04:23:52PM +0200, Krasimir Angelov wrote: > > > HSQL uses withCString internally. withCString strips the > higher order > > > bytes from Char. > > > > You should be able to replace withCString with > withUTF8String from my > > CWStringBasic module, which you can get from here: > > http://repetae.net/john/recent/src/HsLocale/CWStringBasic.hs > > > > and is part of my bigger HsLocale project > http://repetae.net/john/recent/out/HsLocale.html > > but just that one file should be enough if all you need is UTF8. > >John > > Santoemma Enrico said that Oracle ODBC driver expects UCS-2 instead of > UTF-8 so this will not help. > > Krasimir - * Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person(s) or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer. * ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Problem with DB and Char size (2)
On Thu, 27 Jan 2005 06:34:40 -0800, John Meacham <[EMAIL PROTECTED]> wrote: > On Thu, Jan 27, 2005 at 04:23:52PM +0200, Krasimir Angelov wrote: > > HSQL uses withCString internally. withCString strips the higher order > > bytes from Char. > > You should be able to replace withCString with withUTF8String from my > CWStringBasic module, which you can get from here: > http://repetae.net/john/recent/src/HsLocale/CWStringBasic.hs > > and is part of my bigger HsLocale project > http://repetae.net/john/recent/out/HsLocale.html > but just that one file should be enough if all you need is UTF8. >John Santoemma Enrico said that Oracle ODBC driver expects UCS-2 instead of UTF-8 so this will not help. Krasimir ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Problem with DB and Char size (2)
On Thu, Jan 27, 2005 at 04:23:52PM +0200, Krasimir Angelov wrote: > HSQL uses withCString internally. withCString strips the higher order > bytes from Char. You should be able to replace withCString with withUTF8String from my CWStringBasic module, which you can get from here: http://repetae.net/john/recent/src/HsLocale/CWStringBasic.hs and is part of my bigger HsLocale project http://repetae.net/john/recent/out/HsLocale.html but just that one file should be enough if all you need is UTF8. John > > > On Thu, 27 Jan 2005 15:11:38 +0100, Santoemma Enrico > <[EMAIL PROTECTED]> wrote: > > Oops... I hit the wrong key, sending a partial post. Sorry :) > > > > Again: > > I have a problem with strings and unicode chars, when writing slq > > statements on Oracle ODBC driver through HSQL. > > I'm writing here because I suspect that a fix could come from writing 8 bit > > strings, if it is possible somehow. > > > > I'm sorry, the problem is very deep into it's context, and I know it's not > > easy to follow the reasoning. > > These are the facts: > > - I execute an SQL statement, an INSERT, where some values come from a web > > browser, and are UTF-8 encoded > > - Oracle ODBC expects UCS-2, so it converts the values into UTF-8, of > > course scrambling the data > > - when I convert, in Haskell, UTF-8 to UCS-2, I see that someone (perhaps > > HSQL) strips one byte from the two Char's bytes, so again Oracle receives > > bad data > > > > As I haven't found how to force the driver not to strip the byte, and also > > I don't like to convert data two times, I'd try to send 8 bit strings, but > > don't know how. > > Is Word8 a solution? If it is, what is the contstructor?, as w = W8# 1 > > doesn't compile. > > > > I'm weak on low level Haskell. Where do I find references? On the Report? > > > > Thanks, > > Enrico > > > > ___ > > Glasgow-haskell-users mailing list > > Glasgow-haskell-users@haskell.org > > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users > > > -- > Glasgow-haskell-users mailing list > Glasgow-haskell-users@haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users > -- John Meacham - ârepetae.netâjohnâ ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Problem with DB and Char size (2)
As I haven't found how to force the driver not to strip the byte, and also I don't like to convert data two times, I'd try to send 8 bit strings, but don't know how. Is Word8 a solution? If it is, what is the contstructor?, as w = W8# 1 doesn't compile. I'm weak on low level Haskell. Where do I find references? On the Report? I don't know about the rest of your problem but use fromIntegral or toEnum to get Word8 values, eg charToWord8 :: Char -> Word8 charToWord8 = toEnum . fromEnum ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Problem with DB and Char size (2)
HSQL uses withCString internally. withCString strips the higher order bytes from Char. On Thu, 27 Jan 2005 15:11:38 +0100, Santoemma Enrico <[EMAIL PROTECTED]> wrote: > Oops... I hit the wrong key, sending a partial post. Sorry :) > > Again: > I have a problem with strings and unicode chars, when writing slq statements > on Oracle ODBC driver through HSQL. > I'm writing here because I suspect that a fix could come from writing 8 bit > strings, if it is possible somehow. > > I'm sorry, the problem is very deep into it's context, and I know it's not > easy to follow the reasoning. > These are the facts: > - I execute an SQL statement, an INSERT, where some values come from a web > browser, and are UTF-8 encoded > - Oracle ODBC expects UCS-2, so it converts the values into UTF-8, of course > scrambling the data > - when I convert, in Haskell, UTF-8 to UCS-2, I see that someone (perhaps > HSQL) strips one byte from the two Char's bytes, so again Oracle receives bad > data > > As I haven't found how to force the driver not to strip the byte, and also I > don't like to convert data two times, I'd try to send 8 bit strings, but > don't know how. > Is Word8 a solution? If it is, what is the contstructor?, as w = W8# 1 > doesn't compile. > > I'm weak on low level Haskell. Where do I find references? On the Report? > > Thanks, > Enrico > > ___ > Glasgow-haskell-users mailing list > Glasgow-haskell-users@haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users > ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
