Re: [sqlite] Best practices for forward conversion of database formats?

2018-08-30 Thread Keith Medcalf
Or you can store the version in a database table itself. Though I suppose the user_version pragma is kinda like a table: create table version(user_version); and then putting a single integer in there. All the "good" applications ship with an upgrader from *any* version ever to the current ve

Re: [sqlite] sqlidiff --schema question

2018-08-30 Thread Richard Hipp
On 8/7/18, Mark Wagner wrote: > I was surprised to see sqldiff --schema not report column constraints as > schema differences. Or am I missing something? > > $ echo .schema | sqlite3 /tmp/f1.db > CREATE TABLE t (foo text unique); > > $ echo .schema | sqlite3 /tmp/f2.db > CREATE TABLE t (foo text)

Re: [sqlite] sqlidiff --schema question

2018-08-30 Thread Mark Wagner
Just pining on this in case anyone knows more... ? On Tue, Aug 7, 2018 at 2:00 PM Mark Wagner wrote: > > I was surprised to see sqldiff --schema not report column constraints as > schema differences. Or am I missing something? > > $ echo .schema | sqlite3 /tmp/f1.db > CREATE TABLE t (foo text u

Re: [sqlite] sqlite on IBM z/OS Unix

2018-08-30 Thread Don V Nielsen
This is what I was trying to remember. It is for calling LE Cobol, but it demonstrates the use of the CEEENTRY and CEETERM macro that are used to establish the environment without having to use a LE C stub program. I used to use the Cobol stub method in the past. https://www.ibm.com/support/knowle

[sqlite] sqlite on IBM z/OS Unix

2018-08-30 Thread David Jackson
Thanks for the various replies to this question. I have my link working now. It starts off wit a basic 3 line c program (to establish the LE enclave) that then calls an Asm program that passes the SQL that then loads and branches to a c program to do the grunt work. There is an example IBM program

Re: [sqlite] beginners question

2018-08-30 Thread Jürgen Palm
Jens Alfke wrote: On Aug 30, 2018, at 12:10 PM, Richard Hipp wrote: So, perhaps the solution is for Klaus to compile it himself from sources? What about `sudo apt-get install sqlite3` ? —Jens ___ sqlite-users mailing list sqlite-users@mailinglis

Re: [sqlite] beginners question

2018-08-30 Thread Jens Alfke
> On Aug 30, 2018, at 12:10 PM, Richard Hipp wrote: > > So, perhaps the solution is for Klaus to compile it himself from sources? What about `sudo apt-get install sqlite3` ? —Jens ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org ht

Re: [sqlite] beginners question

2018-08-30 Thread Jürgen Palm
Richard Hipp wrote: On 8/30/18, Jürgen Palm wrote: Richard Hipp wrote: On 8/30/18, Klaus Maas wrote: Same issues on Xubuntu 16.04 and 18.04 US international keyboard layout Actually 2 questions: 1) The precompiled binary for Linux of sqlite3 does not run, but sqldiff and sqlite3_analyze

Re: [sqlite] beginners question

2018-08-30 Thread Richard Hipp
On 8/30/18, Jürgen Palm wrote: > > > Richard Hipp wrote: >> On 8/30/18, Klaus Maas wrote: >>> Same issues on Xubuntu 16.04 and 18.04 >>> >>> US international keyboard layout >>> >>> Actually 2 questions: >>> >>> 1) The precompiled binary for Linux of sqlite3 does not run, but >>> sqldiff and sql

Re: [sqlite] beginners question

2018-08-30 Thread Jürgen Palm
Richard Hipp wrote: On 8/30/18, Klaus Maas wrote: Same issues on Xubuntu 16.04 and 18.04 US international keyboard layout Actually 2 questions: 1) The precompiled binary for Linux of sqlite3 does not run, but sqldiff and sqlite3_analyzer do. ./sqlite3: error while loading shared libraries

Re: [sqlite] beginners question

2018-08-30 Thread Richard Hipp
On 8/30/18, Klaus Maas wrote: > Same issues on Xubuntu 16.04 and 18.04 > > US international keyboard layout > > Actually 2 questions: > > 1) The precompiled binary for Linux of sqlite3 does not run, but > sqldiff and sqlite3_analyzer do. > ./sqlite3: error while loading shared libraries: libz.so.

Re: [sqlite] Best practices for forward conversion of database formats?

2018-08-30 Thread Scott Robison
On Thu, Aug 30, 2018 at 11:10 AM Roger Binns wrote: > > On 30/08/18 09:51, Randall Smith wrote: > > is how to convert existing data from one DB format to another, given some > > arbitrary set of changes in the database schema in the interim. > > I use SQLite's user pragma. It starts at zero. > >

Re: [sqlite] Best practices for forward conversion of database formats?

2018-08-30 Thread Simon Slavin
On 30 Aug 2018, at 5:51pm, Randall Smith wrote: > Are there accepted or best practices in the industry for handling this in > general, or with SQLite in particular? Can anyone who has implemented this > make useful suggestions? Or are there published resources I am missing? Roger has provide

[sqlite] beginners question

2018-08-30 Thread Klaus Maas
Same issues on Xubuntu 16.04 and 18.04 US international keyboard layout Actually 2 questions: 1) The precompiled binary for Linux of sqlite3  does not run, but sqldiff and sqlite3_analyzer do. ./sqlite3: error while loading shared libraries: libz.so.1: cannot open shared object file: No such

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread David Raymond
I know the CLI has the .stats option and the .scanstats option. I don't exactly what they implement, but here's an example output from a very quick query on something after turning both of those on. I think the page cache numbers are probably what you're most interested in. (I vaguely recall tha

Re: [sqlite] Best practices for forward conversion of database formats?

2018-08-30 Thread Roger Binns
On 30/08/18 09:51, Randall Smith wrote: > is how to convert existing data from one DB format to another, given some > arbitrary set of changes in the database schema in the interim. I use SQLite's user pragma. It starts at zero. https://sqlite.org/pragma.html#pragma_user_version My code ends u

[sqlite] Best practices for forward conversion of database formats?

2018-08-30 Thread Randall Smith
This is not strictly a SQLite question, though in my case it is. Any application that involves a "persistent" database, i.e., one where the data is long-lived and expected to exist and evolve over time, sooner or later has the issue of moving customers from a V1 database to a V2 database. Obvio

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread t...@qvgps.com
It would be interesting to "measure" the effect of these ideas during the process of optimizing. I can profile and measure the execution times, but also interesting would be to know, how much pages are involved in a specific query. Is there maybe a way to get the count of pages currently used?

Re: [sqlite] sqlite on IBM z/OS Unix

2018-08-30 Thread John McKown
On Thu, Aug 30, 2018 at 11:12 AM Don V Nielsen wrote: > "Having successfully ported sqlite to z/OS Unix as a 32 bit app" > > Totally Awesome! Do I have a solution? No. But I'll bet John McKown will. I > believe he is a guru with the mainframe. > I don't know about being a "guru". Personally, all

Re: [sqlite] sqlite on IBM z/OS Unix

2018-08-30 Thread Don V Nielsen
"Having successfully ported sqlite to z/OS Unix as a 32 bit app" Totally Awesome! Do I have a solution? No. But I'll bet John McKown will. I believe he is a guru with the mainframe. It is not SqlLite. It is that communication mechanism between the non-LE program calling into the LE environment. A

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread David Raymond
People being very specific about "integer primary key" is about another optimization by the way, and not because using "long" or "int" as a your data type is wrong in any way. Simply that "integer" primary key is needed for the optimization. See: https://www.sqlite.org/lang_createtable.html#row

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread Richard Hipp
On 8/30/18, t...@qvgps.com wrote: > Will change to INT PRIMARY KEY now. It must be INTEGER PRIMARY KEY - spelled out. INT PRIMARY KEY won't work. This is a quirk of SQLite. -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread t...@qvgps.com
Thanks guys for quick and competent answers! After first sight, this "Z-Order curve" looks very promising, will give it a try. The LONG PRIMARY KEY is because I need a 64-bit integer (osm-id). I only learned now, that sqlite-int is also 64 bit long. Will change to INT PRIMARY KEY now. Tom --

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread Richard Hipp
On 8/30/18, t...@qvgps.com wrote: > > Structure is simple: > CREATE TABLE Lines(Id LONG PRIMARY KEY, Label VARCHAR(50), Coordinates > BLOB, Flags INT, StyleId INT); > And an rtree-index: > CREATE VIRTUAL TABLE Lines_bb USING rtree (Id LONG PRIMARY KEY, X0 > FLOAT, X1 FLOAT, Y0 FLOAT, Y1 FLOAT, Z0

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread Clemens Ladisch
t...@qvgps.com wrote: > The larger the db, the slower is the fetching! > > My assumption is, that in the big db, these 1000 lines are just spread over a > much higher count of pages. > So more page-loads resulting in more time. Correct. > We changed page_size to the maximum value of 64k and it b

[sqlite] Strategies to reduce page-loads?

2018-08-30 Thread t...@qvgps.com
Hi Guys, we are using SQlite for storing OpenStreetMap ways (lines). Once filled its readonly. Then the lines is queried for specific areas to draw a map. Structure is simple: CREATE TABLE Lines(Id LONG PRIMARY KEY, Label VARCHAR(50), Coordinates BLOB, Flags INT, StyleId INT); And an rtree-inde