On Thu, Oct 16, 2014 at 4:33 PM, k <kmccord0...@gmail.com> wrote:
> Hi all,
> this is my first reply-to post to this mailing list (using gmane nttp
> interface) so I hope this post passes moderation ok and is correctly
> threaded and not duplicated...).
>
> Regarding the inability to use databases created on EBCDIC systems on
> 'normal' systems, my initial thoughts are that sqlite should at least
> create databases with the magic number 'SQLite format 3' correctly - ie the
> magic number should be encoded in the source as a hex literal rather than a
> string.
>
I did this on a port of 3.8.7 that I am working on. It turns out that this
is a const char which is initialized by a C macro variable called
SQLITE_FILE_HEADER. I created a "do_config.sh" file to contain my
./configure. It looks like:
export CFLAGS='-O3 -V -DSQLITE_MAX_MMAPSIZE=1048576 -qTARG=zOSV1R12 '
export CFLAGS="${CFLAGS} -qLANG=EXTC99 -qFLOAT=IEEE -qnolist -qnosource "
export CFLAGS="${CFLAGS} -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600"
export CFLAGS="${CFLAGS} -DSQLITE_ENABLE_COLUMN_METADATA"
export CFLAGS="${CFLAGS} -DSQLITE_ENABLE_SQLLOG"
export CFLAGS="${CFLAGS} -DHAVE_POSIX_FALLOCATE=0"
export CFLAGS="${CFLAGS}
-DSQLITE_FILE_HEADER='\"\\\\123\\\\121\\\\114\\\\151\\\\164\\\\145\\\\040\\\\146\\\\157\\\\162\\\\155\\\\141\\\\164\\\\040\\\\063\\\\000\"'"
# The above is the octal encoding for the phrase "SQLite format 3\0" in
ASCII.
# This is the first phase of being compatible with the ASCII version of the
sqlite data base file.
# Each octal value requires four \ in front of it due to multiple shell
evaluations. When presented
# to the compiler, it will look like a single \ as it really should.
./configure CC=xlc CPP="xlc -E" CXX=xlc++ CXXPP="xlc++ -E"
--prefix=$PWD/sqlite-run
Guess what? I can use "sqlite3" on either Linux or z/OS to create a test
sqlite3 data base file. I can ftp it, in binary, to the other system. And
on that system, I can access it without an error message. !! HOWEVER !! Due
to the differences in encoding, neither side can see any tables created by
the other side. I have not had any time to check up on why this is so. I
remember some of the code does a check to validate any "name". And a "name"
encoded in EBCDIC does not validate when accessed on an ASCII system. And
vice versa. "fixing" this, first of all, would likely be a major task. And,
more importantly, likely above my current C knowledge.
>
> Regarding codepage conversions, how does this normally work? As Teg said
> (or alluded to), should it not be the application's responsibly to do the
> codepage conversion and pass to the sqlite engine the text in the
> appropriate Unicode encoding (UTF-8/UTF-16) as required. I'm not sure
> though, if there is a distinction here between using the bind functions of
> the API vs supplying literal text in the SQL. ** The documentation on the C
> API does not say a lot about how encodings are handled for text (bound or
> in the SQL) **. I would expect in any case, if the application is locale
> aware it should respect the settings of the respective LC_* environment
> variables and perform the necessary conversions before calling the sqlite
> functions??
>
> Regarding the specific case of performing EBCDIC<->ASCII conversions on
> zOS, this is (IMHO) not the approach to take, since (outside of the unix
> subsystem of zOS) any EBCDIC codepage can be in use, and as per the above,
> a locale aware application should take care of this, rather than leaving it
> to the sqlite engine.
>
> All the above of course would (again IMHO) be moot for blobs, where
> everything would be stored as is.
>
> On the subject of support for sqlite on zOS, has anyone investigated the
> possibility on making the command line interface, or the engine itself (via
> a custom vfs) support the MVS (record orientated) filesystem?
>
> Please share your thoughts,
> Thanks in advance,
>
> k
>
>
To address all of "k"'s requirements, I _think_ that it would require a
change in SQLite's architecture. SQLite would really need to work in,
perhaps, UTF-8 for all its internal information. I say UTF-8 because most
of the data kept in z/OS EBCDIC (IBM-1047) characters can be "round trip"
converted to/from UTF-8. And using UTF-8 would likely _not_ impact any
current ASCII users. But that is up to Dr. Hipp to decide. Definitely above
my pay grade.
--
The temperature of the aqueous content of an unremittingly ogled
culinary vessel will not achieve 100 degrees on the Celsius scale.
Maranatha! <><
John McKown
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
