[sqlite] sqlidiff --schema question

2018-08-07 Thread Mark Wagner
I was surprised to see sqldiff --schema not report column constraints as schema differences. Or am I missing something? $ echo .schema | sqlite3 /tmp/f1.db CREATE TABLE t (foo text unique); $ echo .schema | sqlite3 /tmp/f2.db CREATE TABLE t (foo text); $ sqldiff --schema /tmp/f1.db /tmp/f2.db

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-07 Thread Graham Holden
Tuesday, August 07, 2018, 7:36:26 PM, Abramo Bagnarawrote: > I've verified that not only substr(data, start, len) loads entire row, > but with my surprise that also retrieving a sibling column (I've tried > to add another column "info blob not null") loads entire row (i.e. > including whole data

[sqlite] Save text file content in db: lines or whole file?

2018-08-07 Thread Abramo Bagnara
Il 07/08/2018 19:41, Abroży Nieprzełoży ha scritto: > substr(data, start, len) loads entire value and then substrs it. > sqlite3_blob_read reads the desired parts. I've verified that not only substr(data, start, len) loads entire row, but with my surprise that also retrieving a sibling column

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-07 Thread Abroży Nieprzełoży
substr(data, start, len) loads entire value and then substrs it. sqlite3_blob_read reads the desired parts. 2018-08-06 20:59 GMT+02:00, Abramo Bagnara : > Il 04/08/2018 07:07, Abramo Bagnara ha scritto: >> Il 03/08/2018 23:53, Abroży Nieprzełoży ha scritto: >>> -- One table with whole files >>>

Re: [sqlite] Database locks

2018-08-07 Thread Keith Medcalf
By "constantly writing" I presume you mean "periodically writing". For example doing one independent INSERT every millisecond where there is no dependency from between inserts is "periodic writing". "Constantly writing", from a database perspective, means a single transaction that never ends

Re: [sqlite] Database locks

2018-08-07 Thread Keith Medcalf
Make sure to wrap your transactions in BEGIN / COMMIT. Use BEGIN IMMEDIATE for transactions that you know are going to write, and plain BEGIN for read-only transactions. You can omit the explicit BEGIN / COMMIT if each transaction consists of only a single statement since autocommit will do

Re: [sqlite] Database locks

2018-08-07 Thread David Raymond
I think what he's saying is "my reads are slow, and I don't want them to block my writes" which would mean he use WAL mode. i.e. the long reads are going to cause a traffic jam of backed up writes in rollback journal mode, but will be ok in WAL mode. -Original Message- From:

Re: [sqlite] Database locks

2018-08-07 Thread Simon Slavin
On 7 Aug 2018, at 3:04pm, Gerlando Falauto wrote: > What might happen in my case is that reads could take a very long time to > consume/process all the data (30s for instance), and I believe the lock is > held until all data is consumed. > I believe in that case the read would see all the data

Re: [sqlite] Database locks

2018-08-07 Thread Gerlando Falauto
On Tue, Aug 7, 2018 at 3:28 PM, Simon Slavin wrote: > On 7 Aug 2018, at 1:54pm, Gerlando Falauto > wrote: > > > I just realized I'm using default settings... perhaps I should use WAL > mode > > instead? > > How important is it to you that SELECT gets up-to-date information ? If a > read

Re: [sqlite] Database locks

2018-08-07 Thread Simon Slavin
On 7 Aug 2018, at 1:54pm, Gerlando Falauto wrote: > I just realized I'm using default settings... perhaps I should use WAL mode > instead? How important is it to you that SELECT gets up-to-date information ? If a read happens at the same time as a write: Normal mode: SELECT waits until the

Re: [sqlite] Database locks

2018-08-07 Thread David Raymond
Correct. In rollback journal mode when one connection says "I'm ready to write now" it blocks any new transactions from being made, but it can't do anything about existing read transactions. It has to wait for them to finish their reads and end their transactions before it can do any actual

Re: [sqlite] Database locks

2018-08-07 Thread Gerlando Falauto
I just realized I'm using default settings... perhaps I should use WAL mode instead? Thanks, Gerlando On Tue, Aug 7, 2018 at 2:22 PM, Gerlando Falauto wrote: > Hi Simon, > than you for your answer. > > On Tue, Aug 7, 2018 at 2:09 PM, Simon Slavin wrote: > >> On 7 Aug 2018, at 12:55pm,

Re: [sqlite] Database locks

2018-08-07 Thread Simon Slavin
On 7 Aug 2018, at 1:22pm, Gerlando Falauto wrote: > Hmm... are you saying the writer could potentially block for up to 10 > seconds? I should have been clearer. The 10 second time is purely for diagnostic purposes, to see if the error goes away. It was chosen to be far longer than any

Re: [sqlite] Database locks

2018-08-07 Thread Gerlando Falauto
Hi Simon, than you for your answer. On Tue, Aug 7, 2018 at 2:09 PM, Simon Slavin wrote: > On 7 Aug 2018, at 12:55pm, Gerlando Falauto > wrote: > > > I'm trying to implement a logging system based on SQLite, using python3 > > package apsw. > > There's one process constantly writing and another

Re: [sqlite] Database locks

2018-08-07 Thread Simon Slavin
On 7 Aug 2018, at 12:55pm, Gerlando Falauto wrote: > I'm trying to implement a logging system based on SQLite, using python3 > package apsw. > There's one process constantly writing and another one reading. > From time to time I get an exception from the writer, complaining the > database is

Re: [sqlite] [EXTERNAL] Database locks

2018-08-07 Thread Hick Gunter
Your impression is only correct if you are running in WAL journal_mode. In all other modes, readers will block the writer(s) and yo need to set a busy timeout or a busy handler. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag

[sqlite] Database locks

2018-08-07 Thread Gerlando Falauto
Hi, I'm trying to implement a logging system based on SQLite, using python3 package apsw. There's one process constantly writing and another one reading. From time to time I get an exception from the writer, complaining the database is locked. I'm pretty sure there's no other process writing, and

[sqlite] Pragma notnull for integer primary keys?

2018-08-07 Thread Dirkjan Ochtman
Yesterday I started a new project with SQLite. I wanted to create some simple integer-based primary keys and used an ORM to generate code. I then noticed that the primary keys were generated as Nullable types. I asked on Gitter and someone pointed me to this bit of documentation: "According to