Re: [sqlite] Problems opening db in win9x and utf8 filename

[John Stanton]

Obviously one has to have file names which do not clash with the rules 
of the underlying file system.  If you need to map a name to suit the OS 
you can detect the Windows OS version in your application and enforce 
compatibility by having a lookup table or by mangling.  As the old 
saying goes "In computer science any problem can be solved by yet 
another level of indirection".

Costas Stergiou wrote:
Hi John,
Have you tried to use sqlite3_open with a path that contains non-ascii chars
and make it work at the same time in Win9x and win2K? 
The 2 apps I mentioned before (sqLiteExplorer and SQLiteSpy) both fail the
above test (and for a good reason)
Costas
P.S. As I said, you can make an app work on both of these OSs, but with
external manipulation. 


-----Original Message-----
From: John Stanton [mailto:[EMAIL PROTECTED]
Sent: Tuesday, August 08, 2006 11:13 AM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Problems opening db in win9x and utf8 filename

Our Sqlite applications work not only on Win98 and Win2000 but also on
Linux, AIX and Solaris.  Where did we go wrong?

Costas Stergiou wrote:

Hi all,
I saw no more comments on this suggestion. It is very simple to program
around this issue in user code, but I would like to see it fixed in the
library level. Unless someone has made this work-around in his code, an
application cannot work at the same time in Win9x and Win2k if there is

any

ansii char in the filepath.
Costas




-----Original Message-----
From: Costas Stergiou [mailto:[EMAIL PROTECTED]
Sent: Saturday, August 05, 2006 11:47 PM
To: sqlite-users@sqlite.org
Subject: RE: [sqlite] Problems opening db in win9x and utf8 filename




I no longer have a win98 system to test with, but based on my
understanding...

os_win.c attempts to convert the filename from UTF-8 to UTF-16.  If it
succeeds, it calls CreateFileW;

Actually, there is a flag there that caused the convertion to UTF-16 to
'fail' (it doesn't really fail, just that the utf8ToUnicode returns 0).



if it fails, it calls CreateFileA with
the original string.

Exactly



CreateFileW is a nonfunctional stub on win98, so
when you pass a UTF-8 filename sqlite takes that codepath and fails.
An ANSI filename won't pass the UTF-8 conversion, so CreateFileA is
called with the ANSI string.

Actually, in Win98 it will pass the conversion, but as I said above, the
function fails by a check: "if (!isNT())"



That doesn't necessarily explain win2k though.  Perhaps the current
user locale does not match the ANSI encoding of the filename you're
passing in?  Internally win2k only uses the Unicode version, so
CreateFileA must do an implict conversion to Unicode using the current
user codepage.

Now that I checked the code, it actually does.
Unfortunately, the way the code is setup makes it necessary for the

caller

to check in which OS it runs and either use UTF8 paths or ansii ones. I
think this is not a good technique (and not actually intended from what

I

have read in the docs) since the sqlite3_open does not give a truly
uniform
interface to the caller.

My suggestion is this:
The sqlite3_open should always expect a utf8 path (as the docs say). If

in

win2k everything works fine. If in win98 it should convert the path to
utf16
and THEN convert it to ansii using the CP_ACP (current ansii code page).
This will work for 99.9% cases since in non-English win9x OS, almost

99.9%

ansii strings are in the system's locale.
I think this is also the expected behavior (and what I have programmed

my

app to do, until I tested it in win98).

To make these changes, all the logic of os_win.c should change to
accommodate the above. I would certainly say that the way it currently
works
is wrong (bug).
Of course, there is the problem of breaking existing code (since many
win9x
user will not have read the docs, or else someone would have mentioned
this
behavior looong time agoe).
To maintain compatibility (e.g. to accept ansii non-utf8 paths), a check
can
be made on whether the supplied path is in utf8 (heuristically this has
almost 100% success) and then do the above.

Costas





MSLU does provide a functional CreateFileW wrapper for win9x, but I
don't believe the stock sqlite binaries are built with it.


On 8/5/06, Peter Cunderlik <[EMAIL PROTECTED]> wrote:



I think you will never succeed using UTF-8 encoded filenames on those
systems. I don't know how it can be done programmatically, but each
file or directory name has its 8.3 name as well, i.e. "Program Files"
would be "progra~1". I think this is the safest way how to pass
filenames to SQLite. It should work on Win 9x as well as 2K and XP.

NTFS can have 8.3 shortname creation disabled.  Systems running
without it are not common but do exist, so you should avoid relying on
them if at all possible.