Re: [sqlite] Re: Unicode Help

2006-12-08 Thread Da Martian
It looks fine to me. To help check it, one thing you can try is writing the result of FieldAsString directly to a file as raw bytes, then in notepad open that with encoding set to Unicode. E.g. something logically equivalent to: size := Length(field) * 2; SetLength(buffer, size );

Re: [sqlite] Re: Unicode Help

2006-12-08 Thread Da Martian
EUREKA! Ok I got it working now. It turns out my source was UTF-8 Encoded, so even when i used the utf-16 functions it wasnt comming out right. I am now doing a converstion in delphi from UTF-8 to UTF16 and using all UTF-16 sqlite functions as recommended. Thanks a million for all your help, it

Re: [sqlite] Re: Unicode Help

2006-12-08 Thread Kees Nuyt
On Fri, 8 Dec 2006 15:54:45 +, you wrote: How do you set Notepad to Ecnoding = Unicode. I cant see an option for that ? Perhaps it listens to a BOM? http://unicode.org/unicode/faq/utf_bom.html#22 It would mean you have to initialize your textfile before editing with some utility like awk:

Re: [sqlite] Re: Unicode Help

2006-12-08 Thread Trevor Talbot
On 12/8/06, Kees Nuyt [EMAIL PROTECTED] wrote: On Fri, 8 Dec 2006 15:54:45 +, you wrote: How do you set Notepad to Ecnoding = Unicode. I cant see an option for that ? Perhaps it listens to a BOM? It does, and will also try heuristics to detect the encoding if no BOM is present. But,

Re: [sqlite] Re: Unicode Help

2006-12-07 Thread Da Martian
I think std function for convertions would be very helpful. I am still having issues trying to get my charaters standardizsed. I spent all of yesterday playing with ideas but it still in the dark. Part of my problem is I dont have a clue what my source data is encoded as. Does anyone know of a

Re: [sqlite] Re: Unicode Help

2006-12-07 Thread Trevor Talbot
On 12/7/06, Da Martian [EMAIL PROTECTED] wrote: I am still having issues trying to get my charaters standardizsed. I spent all of yesterday playing with ideas but it still in the dark. Whatever you were doing the first time was fine: So if I look at a name with umlaughts in the database via

Re: [sqlite] Re: Unicode Help

2006-12-07 Thread Da Martian
I am still having issues trying to get my charaters standardizsed. I spent all of yesterday playing with ideas but it still in the dark. Whatever you were doing the first time was fine: I have been having that very thought! So if I look at a name with umlaughts in the database via

Re: [sqlite] Re: Unicode Help

2006-12-07 Thread Trevor Talbot
On 12/7/06, Da Martian [EMAIL PROTECTED] wrote: Yeah I am currently using VirtualTree from Mikes Delphi Gems. Its fully unicode enabled (I beleive). I use WideStrings through out the entire pipeline from xml I recieve into SQLite via the prepare16 back out through column_text16 into virtual

Re: [sqlite] Re: Unicode Help

2006-12-07 Thread Chris Hanson
On Dec 5, 2006, at 8:42 AM, Igor Tandetnik wrote: Da Martian [EMAIL PROTECTED] wrote: So if I look at a name with umlaughts in the database via sqlite3.exe I get: Städt. Klinikum Neunkirchen gGmbH -- | an a with two dots on top A with umlaut is represented as two bytes in UTF-8.

Re: [sqlite] Re: Unicode Help

2006-12-06 Thread Marten Feldtmann
Ulrich Schöbel schrieb: SQLite includes a Tcl API. Tcl does all these conversions with ease. See the encoding convertto/convertfrom commands and fconfigure But Tcl is not part of SQLite (and this is good) - this is just an add-on. The idea with the additional functions are pretty good !

Re: [sqlite] Re: Unicode Help

2006-12-06 Thread John Stanton
Marten Feldtmann wrote: Ulrich Schöbel schrieb: SQLite includes a Tcl API. Tcl does all these conversions with ease. See the encoding convertto/convertfrom commands and fconfigure But Tcl is not part of SQLite (and this is good) - this is just an add-on. The idea with the additional

Re: [sqlite] Re: Unicode Help

2006-12-06 Thread Nicolas Williams
On Wed, Dec 06, 2006 at 10:06:12AM -0600, John Stanton wrote: Marten Feldtmann wrote: But Tcl is not part of SQLite (and this is good) - this is just an add-on. The idea with the additional functions are pretty good ! How does Sqlite become Sqbloated? By function creep, one little step at

Re: [sqlite] Re: Unicode Help

2006-12-06 Thread John Stanton
Nicolas Williams wrote: On Wed, Dec 06, 2006 at 10:06:12AM -0600, John Stanton wrote: Marten Feldtmann wrote: But Tcl is not part of SQLite (and this is good) - this is just an add-on. The idea with the additional functions are pretty good ! How does Sqlite become Sqbloated? By function

[sqlite] Re: Unicode Help

2006-12-05 Thread Igor Tandetnik
Da Martian [EMAIL PROTECTED] wrote: So if I look at a name with umlaughts in the database via sqlite3.exe I get: Städt. Klinikum Neunkirchen gGmbH -- | an a with two dots on top A with umlaut is represented as two bytes in UTF-8. sqlite3.exe just dumps these bytes onto the console,

Re: [sqlite] Re: Unicode Help

2006-12-05 Thread Marten Feldtmann
Igor Tandetnik schrieb: Da Martian [EMAIL PROTECTED] wrote: So if I look at a name with umlaughts in the database via sqlite3.exe I get: Städt. Klinikum Neunkirchen gGmbH -- | an a with two dots on top A with umlaut is represented as two bytes in UTF-8. sqlite3.exe just dumps these

Re: [sqlite] Re: Unicode Help

2006-12-05 Thread Nicolas Williams
On Tue, Dec 05, 2006 at 06:53:28PM +0100, Marten Feldtmann wrote: Perhaps it would be nice to change sqlite3 in that way, that (when columns with storage class text) these columns are converted to the host platform code page. But actually even in that situation you may have strings, which are

Re: [sqlite] Re: Unicode Help

2006-12-05 Thread Ralf Junker
Hello Nicolas Williams, No, but having built-in functions that can do codeset conversion would be nice. SQLiteSpy can do this: http://www.yunqa.de/delphi/sqlitespy/ - To unsubscribe, send email to [EMAIL PROTECTED]

Re: [sqlite] Re: Unicode Help

2006-12-05 Thread Ulrich Schöbel
SQLite includes a Tcl API. Tcl does all these conversions with ease. See the encoding convertto/convertfrom commands and fconfigure. On Tuesday 05 December 2006 20:42, Nicolas Williams wrote: On Tue, Dec 05, 2006 at 06:53:28PM +0100, Marten Feldtmann wrote: Perhaps it would be nice to change