Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-05 Thread E.Pasma
A question was if a sort is also programmatically interruptable. With the knowledge that the command tool calls sqlite3_interrupt upon the first CONTROL-C, this is easily tested. From the timings below it appears to be so. $ sqlite3 SQLite version 3.25.2 2018-09-25 19:08:10 Enter ".help" for us

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread Keith Medcalf
linglists.sqlite.org >Subject: Re: [sqlite] Using sqlite3_interrupt with a timeout > >On Mon, 31 Dec 2018 14:25:41 -0700 >"Keith Medcalf" wrote: > >> def run_query_with_timeout(db, query, timeout, whizround) >> stmt = prepare(db, query) >> create_thr

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread Richard Hipp
On 1/2/19, James K. Lowden wrote: > > can I call sqlite3_interrupt from a > signal handler? > Yes. It was designed for that very purpose, and more specifically to catch Ctrl-C in the CLI and stop the running query rather than kill off the whole process. -- D. Richard Hipp d...@sqlite.org __

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread Jens Alfke
> On Jan 2, 2019, at 12:22 PM, James K. Lowden wrote: > > If I don't want to use threads, can I call sqlite3_interrupt from a > signal handler? It looks like you can, since all the implementation does is set a boolean flag inside the sqlite3 struct. However, if SQLITE_ENABLE_API_ARMOR is d

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread James K. Lowden
On Mon, 31 Dec 2018 14:25:41 -0700 "Keith Medcalf" wrote: > def run_query_with_timeout(db, query, timeout, whizround) > stmt = prepare(db, query) > create_thread A interrupt_function(db, stmt, timeout, > whizround) while sqlite3_step(stmt) == SQLITE_ROW > ... process the

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread Jens Alfke
> On Dec 31, 2018, at 1:49 PM, Simon Slavin wrote: > > If you have a 100 Gig table with no indexes suited to your clauses, creating > a temporary index might take a minute or two. But you really only have > yourself to blame. […] > Since the one possibly-long operation happens at the very be

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread Keith Medcalf
On Wednesday, 2 January, 2019 03:06, Dominique Devienne wrote: >On Mon, Dec 31, 2018 at 10:31 PM Keith Medcalf wrote: >>> I don't think the interrupt call will actually terminate a step >>> that is actually being processed, but only mark that no more steps >>> should happen. In other words, I

Re: [sqlite] Using sqlite3_interrupt with a timeout

2019-01-02 Thread Dominique Devienne
On Mon, Dec 31, 2018 at 10:31 PM Keith Medcalf wrote: > >I don't think the interrupt call will actually terminate a step that > >is actually being processed, but only mark that no more steps should > >happen. In other words, I don't think SQLite is spending time > >checking a flag to stop in the

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Simon Slavin
On 31 Dec 2018, at 10:44pm, Jesse Rittner wrote: > Simon Slavin-3 wrote > >> it can be long only if you're a bad SQL programmer > > To be fair, the query in question might not get run frequently enough to > warrant the overhead of maintaining an index. Fair point. Simon.

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Jesse Rittner
Simon Slavin-3 wrote > it can be long only if you're a bad SQL programmer To be fair, the query in question might not get run frequently enough to warrant the overhead of maintaining an index. -- Sent from: http://sqlite.1065341.n5.nabble.com/ ___ sql

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Simon Slavin
On 31 Dec 2018, at 9:10pm, Jesse Rittner wrote: > Simon Slavin-3 wrote > >> You can calculate an "end time" yourself, check it each time you're ready to >> call sqlite3_step(), and jump straight to sqlite3_finalize() if your time is >> up. > > I'm not familiar with the inner workings of sqlit

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Keith Medcalf
>I don't think the interrupt call will actually terminate a step that >is actually being processed, but only mark that no more steps should >happen. In other words, I don't think SQLite is spending time >checking a flag to stop in the middle of processing a step to allow the >processing to terminat

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Richard Damon
On 12/31/18 4:10 PM, Jesse Rittner wrote: > Simon Slavin-3 wrote >> You can calculate an "end time" yourself, check it each time you're ready >> to call sqlite3_step(), and jump straight to sqlite3_finalize() if your >> time is up. > I'm not familiar with the inner workings of sqlite3_step, but if

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Keith Medcalf
That is, of course (I forgot the stmt argument to sqlite3_stmt_busy in all the whizing around): def interrupt_function(db, stmt, timeout, whizround) while whizround and !sqlite3_stmt_busy(stmt) /* whizround waiting for statement to start */ sleep(0.001) sleep(time

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Keith Medcalf
On Monday, 31 December, 2018 13:48, Simon Slavin wrote: >On 31 Dec 2018, at 8:18pm, Jesse Rittner wrote: >> I'm trying to write a function to run a query with a timeout. If >> the timeout expires, the query must stop execution within a "reasonable" >> amount >> of time. >There is no rule that

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Keith Medcalf
On Monday, 31 December, 2018 13:19, Jesse Rittner wrote: >Keith Medcalf wrote >> What are you trying to accomplish? Perhaps what you really want is >> a progress callback? > I'm trying to write a function to run a query with a timeout. If the > timeout expires, the query must stop execution

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Jesse Rittner
Simon Slavin-3 wrote > You can calculate an "end time" yourself, check it each time you're ready > to call sqlite3_step(), and jump straight to sqlite3_finalize() if your > time is up. I'm not familiar with the inner workings of sqlite3_step, but if that itself takes time, then I'd like to interru

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Simon Slavin
On 31 Dec 2018, at 8:18pm, Jesse Rittner wrote: > I'm trying to write a function to run a query with a timeout. If the timeout > expires, the query must stop execution within a "reasonable" amount of time. There is no rule that you must continue to call sqlite3_step() until it runs out of rows.

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Jesse Rittner
Keith Medcalf wrote > What are you trying to accomplish? Perhaps what you really want is a > progress callback? I'm trying to write a function to run a query with a timeout. If the timeout expires, the query must stop execution within a "reasonable" amount of time. To use a progress callback, it

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Keith Medcalf
>To be clear, is it sqlite3_step returning SQLITE_DONE that marks it >as "not running", or calling sqlite3_reset/sqlite3_finalize? Well, "running" means that execution has commenced (the first call to sqlite3_step has been made on the statement) and the execution has not yet completed. Executi

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Jesse Rittner
To be clear, is it sqlite3_step returning SQLITE_DONE that marks it as "not running", or calling sqlite3_reset/sqlite3_finalize? Also, is there any way to mark a statement as "running" other than calling sqlite_step on it? Otherwise, it sounds like I'll have to wait until after sqlite3_step gets c

Re: [sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Keith Medcalf
y a Stairway to Heaven says a lot about anticipated traffic volume. >-Original Message- >From: sqlite-users [mailto:sqlite-users- >boun...@mailinglists.sqlite.org] On Behalf Of Jesse Rittner >Sent: Monday, 31 December, 2018 09:03 >To: sqlite-users@mailinglists.sqlite.org >Subje

[sqlite] Using sqlite3_interrupt with a timeout

2018-12-31 Thread Jesse Rittner
Consider the following pseudo-code. void interrupt_timeout(sqlite3* db, int timeout) { sleep(timeout); sqlite3_interrupt(db); } int main() { sqlite3* db = sqlite3_open_v2(...); sqlite3_stmt* stmt = sqlite3_prepare_v2(db, ...); ... pthread_create(interrupt_timeout, db, ti