Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread t...@qvgps.com
-side the only options is to have page size as big as possible and line-data in the blob-field as much compressed as possible. -- Originalnachricht -- Von: "Simon Slavin" An: "SQLite mailing list" Gesendet: 31.08.2018 19:07:36 Betreff: Re: [sqlite] Strategies

Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread Simon Slavin
On 31 Aug 2018, at 2:46pm, J Decker wrote: > There was a voxel engine that was claiming they were going to move to a > morton encoding; and I was working with a different engine, so I built a > simulator to test averge lookup distances; it was far more efficient to > keep sectors of voxels

Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread J Decker
On Thu, Aug 30, 2018 at 4:48 AM Richard Hipp wrote: > On 8/30/18, t...@qvgps.com wrote: > > > > Structure is simple: > > CREATE TABLE Lines(Id LONG PRIMARY KEY, Label VARCHAR(50), Coordinates > > BLOB, Flags INT, StyleId INT); > > And an rtree-index: > > CREATE VIRTUAL TABLE Lines_bb USING

Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread Richard Hipp
On 8/31/18, t...@qvgps.com wrote: > Ok, then WITHOUT ROWID will most properly fit best in our use case. > Then I can fill the PRIMARY KEY with the z-order and store the osm-id > just in another column. I would think that your best approach is to make the INTEGER PRIMARY KEY be the Morton code

Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread t...@qvgps.com
. At which point during insert are the pages actually written? -- Originalnachricht -- Von: "Richard Hipp" An: "SQLite mailing list" Gesendet: 31.08.2018 15:10:15 Betreff: Re: [sqlite] Strategies to reduce page-loads? >On 8/31/18, t...@qvgps.com wrote: >&

Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread Richard Hipp
On 8/31/18, t...@qvgps.com wrote: > > So is it just the value of the primary key controlling in which page the > row is stored? The page on which content is stored is determine (approximately) by the value of the ROWID, which is the same as the INTEGER PRIMARY KEY as long as you declare the

Re: [sqlite] Strategies to reduce page-loads?

2018-08-31 Thread t...@qvgps.com
> >(3) Consider deliberating choosing INTEGER PRIMARY KEY values based on >a "Morton code" or "Z-Order curve" of the coordinates. >(https://en.wikipedia.org/wiki/Z-order_curve) That will cause >features that are close together geographically to tend to be close >together within the file. My

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread David Raymond
-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of t...@qvgps.com Sent: Thursday, August 30, 2018 12:28 PM To: SQLite mailing list Subject: Re: [sqlite] Strategies to reduce page-loads? It would be interesting to "measure" the effect of these ideas during the process of o

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread t...@qvgps.com
ently used? -- Originalnachricht -- Von: "Richard Hipp" An: "SQLite mailing list" Gesendet: 30.08.2018 13:48:30 Betreff: Re: [sqlite] Strategies to reduce page-loads? >On 8/30/18, t...@qvgps.com wrote: >> >>Structure is simple: >>CREAT

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread David Raymond
s://www.sqlite.org/lang_createtable.html#rowid -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of t...@qvgps.com Sent: Thursday, August 30, 2018 8:18 AM To: SQLite mailing list Subject: Re: [sqlite] Strategies to reduce page-loads? Tha

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread Richard Hipp
On 8/30/18, t...@qvgps.com wrote: > Will change to INT PRIMARY KEY now. It must be INTEGER PRIMARY KEY - spelled out. INT PRIMARY KEY won't work. This is a quirk of SQLite. -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread t...@qvgps.com
now. Tom -- Originalnachricht -- Von: "Richard Hipp" An: "SQLite mailing list" Gesendet: 30.08.2018 13:48:30 Betreff: Re: [sqlite] Strategies to reduce page-loads? >On 8/30/18, t...@qvgps.com wrote: >> >>Structure is simple: >>CREATE TABLE Lines(

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread Richard Hipp
On 8/30/18, t...@qvgps.com wrote: > > Structure is simple: > CREATE TABLE Lines(Id LONG PRIMARY KEY, Label VARCHAR(50), Coordinates > BLOB, Flags INT, StyleId INT); > And an rtree-index: > CREATE VIRTUAL TABLE Lines_bb USING rtree (Id LONG PRIMARY KEY, X0 > FLOAT, X1 FLOAT, Y0 FLOAT, Y1 FLOAT,

Re: [sqlite] Strategies to reduce page-loads?

2018-08-30 Thread Clemens Ladisch
t...@qvgps.com wrote: > The larger the db, the slower is the fetching! > > My assumption is, that in the big db, these 1000 lines are just spread over a > much higher count of pages. > So more page-loads resulting in more time. Correct. > We changed page_size to the maximum value of 64k and it

[sqlite] Strategies to reduce page-loads?

2018-08-30 Thread t...@qvgps.com
Hi Guys, we are using SQlite for storing OpenStreetMap ways (lines). Once filled its readonly. Then the lines is queried for specific areas to draw a map. Structure is simple: CREATE TABLE Lines(Id LONG PRIMARY KEY, Label VARCHAR(50), Coordinates BLOB, Flags INT, StyleId INT); And an