Re: [sqlite] Memoization in sqlite json1 functions

2017-03-25 Thread Jens Alfke
> On Mar 24, 2017, at 4:48 PM, Deon Brewis wrote: > > Yeah ok, if you take I/O hits then things like memory pre-fetching makes zero > difference. We're more in the business of "You take a page fault" == "You buy > more memory". Different level of performance requirements. (And glad that > SQL

[sqlite] Best settings for concurency

2017-03-25 Thread Max Terentiev
Hi, I developing app server with sqlite as storage engine. Sqlite have many settings combinations affecting concurrency please help me to chose right settings for my task: 1. SQLITE_THREADSAFE 1 (serialized) vs SQLITE_THREADSAFE 2 (multithreaded) My app server will execute each client request i

Re: [sqlite] Best settings for concurency

2017-03-25 Thread Simon Slavin
On 25 Mar 2017, at 7:14pm, Max Terentiev wrote: > Sqlite have many settings combinations affecting concurrency please help me > to chose right settings for my task 1. Leave SQLITE_THREADSAFE to the default setting. 2. Leave shared cache and uncommitted reads to the default settings. Use WAL m

Re: [sqlite] Best settings for concurency

2017-03-25 Thread Keith Medcalf
On Saturday, 25 March, 2017 13:15, Max Terentiev wrote: > I developing app server with sqlite as storage engine. > Sqlite have many settings combinations affecting concurrency please help > me to chose right settings for my task: > 1. SQLITE_THREADSAFE 1 (serialized) vs SQLITE_THREADSAFE 2 (

[sqlite] How does one block a reader connection?

2017-03-25 Thread petern
I would like to construct a SQLite database for one writer and one or more reader connections. The writer will be updating various data tables which occasionally trigger one or more row inserts in a command table named 'cmd'. This command table is being polled by the readers for new commands. To

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Simon Slavin
On 25 Mar 2017, at 10:52pm, petern wrote: > CREATE TABLE cmd(opcode TEXT, params TEXT); > > Also assume each reader is in a different process which maintains its own > open db connection over which it periodically executes the following > command retrieval query, > > SELECT * FROM cmd WHERE ro

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-25 Thread Deon Brewis
> Page faults aren’t necessarily due to swapping. If you read a memory-mapped > file, the first access to any page will fault to disk. Since SQLite supports > memory-mapping, I’d assume this can occur during a query — the column data > returned by sqlite may point into mmap’ed pages. (Is that co

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread petern
All good points. Yes, query by 'rowid > $lastCmdRowid' was the intent. Is there something that can be done by the writer, like holding a BEGIN EXCLUSIVE TRANSACTION open with PRAGMA read_uncommitted=0? Would that block all readers or would they continue to get empty results from the command tabl

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Richard Damon
On 3/25/17 6:52 PM, petern wrote: I would like to construct a SQLite database for one writer and one or more reader connections. The writer will be updating various data tables which occasionally trigger one or more row inserts in a command table named 'cmd'. This command table is being polled

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Simon Slavin
On 26 Mar 2017, at 2:35am, petern wrote: > Is there something that can be done by the writer, like holding a BEGIN > EXCLUSIVE TRANSACTION open with PRAGMA read_uncommitted=0? Would that > block all readers or would they continue to get empty results from the > command table? I suppose I could

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Keith Medcalf
On Saturday, 25 March, 2017 19:35, petern wrote: > All good points. Yes, query by 'rowid > $lastCmdRowid' was the intent. > Is there something that can be done by the writer, like holding a BEGIN > EXCLUSIVE TRANSACTION open with PRAGMA read_uncommitted=0? Would that > block all readers or wo

Re: [sqlite] Reporting Solutions that work with SQLite / VB 2015?

2017-03-25 Thread Cousin Stanley
James K. Lowden wrote: > I recently wrote a utility to produce simple reports. > > https://github.com/jklowden/sqlrpt > > Numeric columns are aligned on the decimal point and formatted > with the thousands separator consistent with the locale. > > Wide text columns are formatted to fit nic

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread petern
Thanks Simon. Yes, the actual command table has other columns like a time stamp to allow readers to join and restart asynchronously without losing state. The concept of 'done' in this system is also not so clear to the readers because command execution depends on the state of independent systems

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Jay Kreibich
On Mar 25, 2017, at 5:52 PM, petern wrote: > So finally, here is the question. Is there a SQLite API way for reader > connections to block and wait for a meaningful change, like a new row, in > the 'cmd' table instead of madly polling and using up database concurrency > resources? [Block with

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Keith Medcalf
Saturday, 25 March, 2017 23:44. petern wrote: > Can anybody explain the purpose of > http://sqlite.org/c3ref/busy_handler.html > ? It seems the only practical use would be to allow the caller to give > the engine a suggested lock deadline before SQLITE_BUSY is returned by > sqlite3_exec or sqli

Re: [sqlite] How does one block a reader connection?

2017-03-25 Thread Jens Alfke
> On Mar 25, 2017, at 3:52 PM, petern wrote: > > So finally, here is the question. Is there a SQLite API way for reader > connections to block and wait for a meaningful change, like a new row, in > the 'cmd' table instead of madly polling and using up database concurrency > resources? [Block w