Re: SQLite and callbacks

2011-02-16 Thread Branko Čibej
On 16.02.2011 20:26, Mark Phippard wrote: > 2011/2/16 Branko Čibej : > >> My not very humble opinion -- we can play silly buggers trying to >> optimize this bit of the query, but effort would be better spent in >> merging NODES and ACTUAL_NODE, which in turn would allow us to drop the >> second que

Re: SQLite and callbacks

2011-02-16 Thread Stefan Sperling
On Wed, Feb 16, 2011 at 02:26:51PM -0500, Mark Phippard wrote: > 2011/2/16 Branko Čibej : > > > My not very humble opinion -- we can play silly buggers trying to > > optimize this bit of the query, but effort would be better spent in > > merging NODES and ACTUAL_NODE, which in turn would allow us

Re: SQLite and callbacks

2011-02-16 Thread Mark Phippard
2011/2/16 Branko Čibej : > My not very humble opinion -- we can play silly buggers trying to > optimize this bit of the query, but effort would be better spent in > merging NODES and ACTUAL_NODE, which in turn would allow us to drop the > second query altogether and halve the total time needed to

Re: SQLite and callbacks

2011-02-16 Thread Branko Čibej
On 16.02.2011 19:25, Julian Foad wrote: > On Tue, 2011-02-15, Stefan Sperling wrote: >> [[[ >> Improve performance of svn proplist in a similar way as was done in r1039808. >> But, this time, avoid problems with callbacks invoked during sqlite >> transactions by storing results in a temporary table

Re: SQLite and callbacks

2011-02-16 Thread Daniel Shahaf
Julian Foad wrote on Wed, Feb 16, 2011 at 18:25:35 +: >AND NOT (SELECT 1 FROM actual_node > WHERE wc_id = ?1 AND local_relpath = ?2) > > (I'm not sure whether "NOT (SELECT 1 ...)" is the correct or best way to > say "this selection is empty", but you get the idea.) >

Re: SQLite and callbacks

2011-02-16 Thread Julian Foad
On Tue, 2011-02-15, Stefan Sperling wrote: > [[[ > Improve performance of svn proplist in a similar way as was done in r1039808. > But, this time, avoid problems with callbacks invoked during sqlite > transactions by storing results in a temporary table and invoking > callbacks during a query on th

Re: SQLite and callbacks

2011-02-16 Thread Branko Čibej
On 16.02.2011 11:52, Philip Martin wrote: > Stefan Sperling writes: > >> [[[ >> Improve performance of svn proplist in a similar way as was done in r1039808. >> But, this time, avoid problems with callbacks invoked during sqlite >> transactions by storing results in a temporary table and invoking

Re: SQLite and callbacks

2011-02-16 Thread Philip Martin
Stefan Sperling writes: > [[[ > Improve performance of svn proplist in a similar way as was done in r1039808. > But, this time, avoid problems with callbacks invoked during sqlite > transactions by storing results in a temporary table and invoking > callbacks during a query on the temporary table

Re: SQLite and callbacks

2011-02-15 Thread Stefan Sperling
On Tue, Feb 15, 2011 at 03:35:43PM +, Hyrum K Wright wrote: > I've not reviewed the patch, but to the question about bumping the wc > format, I think you're safe to do so (so long as the upgrade code is > properly implemented / tested, etc). > > The current working copy much more gracefully ha

Re: SQLite and callbacks

2011-02-15 Thread Hyrum K Wright
On Tue, Feb 15, 2011 at 2:53 PM, Stefan Sperling wrote: > On Tue, Feb 15, 2011 at 01:30:45PM +0100, Stefan Sperling wrote: >> On Mon, Feb 14, 2011 at 09:48:35PM +0100, Branko Čibej wrote: >> > On 14.02.2011 13:37, Stefan Sperling wrote: >> > > On Tue, Feb 08, 2011 at 11:50:36PM +0100, Branko Čibej

Re: SQLite and callbacks

2011-02-15 Thread Stefan Sperling
On Tue, Feb 15, 2011 at 01:30:45PM +0100, Stefan Sperling wrote: > On Mon, Feb 14, 2011 at 09:48:35PM +0100, Branko Čibej wrote: > > On 14.02.2011 13:37, Stefan Sperling wrote: > > > On Tue, Feb 08, 2011 at 11:50:36PM +0100, Branko Čibej wrote: > > >> Well, here it is, I fixed the thinko in the act

Re: SQLite and callbacks

2011-02-15 Thread Stefan Sperling
On Mon, Feb 14, 2011 at 09:48:35PM +0100, Branko Čibej wrote: > On 14.02.2011 13:37, Stefan Sperling wrote: > > On Tue, Feb 08, 2011 at 11:50:36PM +0100, Branko Čibej wrote: > >> Well, here it is, I fixed the thinko in the actual_props query and got > >> all prop_tests to pass with this version. >

Re: SQLite and callbacks

2011-02-14 Thread Branko Čibej
On 14.02.2011 13:37, Stefan Sperling wrote: > On Tue, Feb 08, 2011 at 11:50:36PM +0100, Branko Čibej wrote: >> Well, here it is, I fixed the thinko in the actual_props query and got >> all prop_tests to pass with this version. > Just FYI, the patch doesn't apply to HEAD. I'll try to adjust it when

Re: SQLite and callbacks

2011-02-14 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 11:50:36PM +0100, Branko Čibej wrote: > Well, here it is, I fixed the thinko in the actual_props query and got > all prop_tests to pass with this version. Just FYI, the patch doesn't apply to HEAD. I'll try to adjust it when I find time. But maybe you'll be quicker than me?

Re: SQLite and callbacks

2011-02-09 Thread Ivan Zhakov
On Wed, Feb 9, 2011 at 23:06, Stefan Sperling wrote: > On Wed, Feb 09, 2011 at 10:56:35PM +0300, Ivan Zhakov wrote: >> On Wed, Feb 9, 2011 at 17:40, Ivan Zhakov wrote: >> > On Windows there is FILE_ATTRIBUTE_TEMPORARY [1] attribute, which >> > prevents file to be written to disk: >> > [[[ >> > Sp

Re: SQLite and callbacks

2011-02-09 Thread Stefan Sperling
On Wed, Feb 09, 2011 at 10:56:35PM +0300, Ivan Zhakov wrote: > On Wed, Feb 9, 2011 at 17:40, Ivan Zhakov wrote: > > On Windows there is FILE_ATTRIBUTE_TEMPORARY [1] attribute, which > > prevents file to be written to disk: > > [[[ > > Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file s

Re: SQLite and callbacks

2011-02-09 Thread Ivan Zhakov
v@subversion.apache.org >>> Subject: Re: SQLite and callbacks >>> >>> On 07.02.2011 21:51, Stefan Sperling wrote: >>> >> A lot of wc databases out there will be >>> >> so small that the user will hardly notice the memory increase. >>>

Re: SQLite and callbacks

2011-02-09 Thread Ivan Zhakov
On Tue, Feb 8, 2011 at 12:50, Bert Huijben wrote: > > >> -Original Message- >> From: Branko Čibej [mailto:br...@xbc.nu] On Behalf Of Branko Cibej >> Sent: dinsdag 8 februari 2011 4:39 >> To: dev@subversion.apache.org >> Subject: Re: SQLite and callbac

Re: SQLite and callbacks

2011-02-08 Thread Daniel Shahaf
ssh ne...@svn-qavm.apache.org Use the ssh key you use for people.apache.org. (And please add it to svn in the designated location; see pmc/machines/notes/ for the URL.) Daniel Neels Hofmeyr wrote on Wed, Feb 09, 2011 at 02:38:49 +0100: > On Tue, 2011-02-08 at 20:27 +0100, Stefan Sperling wrote:

Re: SQLite and callbacks

2011-02-08 Thread Neels Hofmeyr
On Tue, 2011-02-08 at 20:27 +0100, Stefan Sperling wrote: > > Also instead of nebulous handwaving about "performance is bad", it > > would be nice to have actually datasets and actual numbers. We have a > > VM at the ASF which could be used for hosting a set of benchmarking > > data; we just need

Re: SQLite and callbacks

2011-02-08 Thread Philip Martin
Branko Čibej writes: > Using Serf 0.7.1 definitely helps, the checkout of > ^/subversion/tags/ebcdic was now successful, took 19 minutes and got svn > memory usage up to 360MB. I'm not too happy about the last two points, > but at least it works. I checked out ^/subversion/branches using HEAD ov

Re: SQLite and callbacks

2011-02-08 Thread Branko Čibej
On 08.02.2011 23:50, Branko Čibej wrote: > Well, here it is, I fixed the thinko in the actual_props query and got > all prop_tests to pass with this version. Did I say that the way > ACTUAL_NODE is separate from NODE makes these kinds of WC operations > (that merge results from both tables) quite c

Re: SQLite and callbacks

2011-02-08 Thread Branko Čibej
On 08.02.2011 23:57, Branko Čibej wrote: > On 08.02.2011 23:47, Stefan Sperling wrote: >> On Tue, Feb 08, 2011 at 11:37:16PM +0100, Branko Čibej wrote: >>> On 08.02.2011 18:13, Hyrum K Wright wrote: It shouldn't be difficult to create a large working copy ('svn co ^/subversion') and use t

Re: SQLite and callbacks

2011-02-08 Thread Branko Čibej
On 08.02.2011 23:47, Stefan Sperling wrote: > On Tue, Feb 08, 2011 at 11:37:16PM +0100, Branko Čibej wrote: >> On 08.02.2011 18:13, Hyrum K Wright wrote: >>> It shouldn't be difficult to create a large working copy ('svn co >>> ^/subversion') and use that for testing, but getting more insight than

Re: SQLite and callbacks

2011-02-08 Thread Branko Čibej
On 07.02.2011 21:57, Stefan Sperling wrote: > On Mon, Feb 07, 2011 at 09:32:48PM +0100, Branko Čibej wrote: >> On 07.02.2011 17:10, Stefan Sperling wrote: >>> The bug is probabaly in the following query. >>> Maybe the INSERT OR REPLACE doesn't work as intended? >>> And why is COMMIT TRANSACTION com

Re: SQLite and callbacks

2011-02-08 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 11:37:16PM +0100, Branko Čibej wrote: > On 08.02.2011 18:13, Hyrum K Wright wrote: > > It shouldn't be difficult to create a large working copy ('svn co > > ^/subversion') and use that for testing, but getting more insight than > > just our own dataset would be very nice. >

Re: SQLite and callbacks

2011-02-08 Thread Branko Čibej
On 08.02.2011 18:13, Hyrum K Wright wrote: > It shouldn't be difficult to create a large working copy ('svn co > ^/subversion') and use that for testing, but getting more insight than > just our own dataset would be very nice. Except that, as I noted elsewhere in this thread, even 'svn co ^/subver

Re: SQLite and callbacks

2011-02-08 Thread Hyrum K Wright
On Tue, Feb 8, 2011 at 7:59 PM, Stefan Sperling wrote: > On Tue, Feb 08, 2011 at 07:40:03PM +, Hyrum K Wright wrote: >> One of my greater concerns is that we don't have a concrete answer to >> "we'll release when " for the performance question.  What is good >> enough?  Which operations?  

Re: SQLite and callbacks

2011-02-08 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 07:40:03PM +, Hyrum K Wright wrote: > One of my greater concerns is that we don't have a concrete answer to > "we'll release when " for the performance question. What is good > enough? Which operations? How much better than 1.6.x? Having a > concrete answer, and

Re: SQLite and callbacks

2011-02-08 Thread Hyrum K Wright
On Tue, Feb 8, 2011 at 7:27 PM, Stefan Sperling wrote: > On Tue, Feb 08, 2011 at 05:13:50PM +, Hyrum K Wright wrote: >> On Tue, Feb 8, 2011 at 4:15 PM, Stefan Sperling wrote: >> > On Tue, Feb 08, 2011 at 04:34:52PM +0100, Bert Huijben wrote: >> >> There is nobody actively working on status an

Re: SQLite and callbacks

2011-02-08 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 05:13:50PM +, Hyrum K Wright wrote: > On Tue, Feb 8, 2011 at 4:15 PM, Stefan Sperling wrote: > > On Tue, Feb 08, 2011 at 04:34:52PM +0100, Bert Huijben wrote: > >> There is nobody actively working on status and there are no open > >> issues on status to block branching.

Re: SQLite and callbacks

2011-02-08 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 06:02:28PM +0100, Branko Čibej wrote: > On 08.02.2011 16:34, Bert Huijben wrote: > > An even better solution would be that SQLite tries to do things completely > > in memory and only *creates* a tempfile when needed. (It seems it now > > creates the file anyway; but doesn'

Re: SQLite and callbacks

2011-02-08 Thread Hyrum K Wright
On Tue, Feb 8, 2011 at 4:15 PM, Stefan Sperling wrote: > On Tue, Feb 08, 2011 at 04:34:52PM +0100, Bert Huijben wrote: >> There is nobody actively working on status and there are no open >> issues on status to block branching... > > There's the general wc-ng performance issue (but I don't think it

Re: SQLite and callbacks

2011-02-08 Thread Branko Čibej
On 08.02.2011 16:34, Bert Huijben wrote: > An even better solution would be that SQLite tries to do things completely in > memory and only *creates* a tempfile when needed. (It seems it now creates > the file anyway; but doesn't use it until needed. Introducing a heavy > performance penalty on N

Re: SQLite and callbacks

2011-02-08 Thread Philip Martin
Stefan Sperling writes: > Ah, so status performs well using per-node queries? > I thought it was still worse compared to 1.6.x for some use cases. > See http://svn.haxx.se/dev/archive-2010-09/0526.shtml > I think we should try to get trunk to perform at least as well as 1.6.x > for all uses cases

Re: SQLite and callbacks

2011-02-08 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 04:34:52PM +0100, Bert Huijben wrote: > There is nobody actively working on status and there are no open > issues on status to block branching... There's the general wc-ng performance issue (but I don't think it has an issue number attached to it right now). > So if statu

RE: SQLite and callbacks

2011-02-08 Thread Bert Huijben
> -Original Message- > From: Stefan Sperling [mailto:s...@elego.de] > Sent: dinsdag 8 februari 2011 15:18 > To: Bert Huijben > Cc: 'Branko Čibej'; dev@subversion.apache.org > Subject: Re: SQLite and callbacks > > On Tue, Feb 08, 2011 at 10

Re: SQLite and callbacks

2011-02-08 Thread Stefan Sperling
On Tue, Feb 08, 2011 at 10:50:46AM +0100, Bert Huijben wrote: > > > > -Original Message- > > From: Branko Čibej [mailto:br...@xbc.nu] On Behalf Of Branko Cibej > > Sent: dinsdag 8 februari 2011 4:39 > > To: dev@subversion.apache.org > > Subject:

RE: SQLite and callbacks

2011-02-08 Thread Bert Huijben
> -Original Message- > From: Branko Čibej [mailto:br...@xbc.nu] On Behalf Of Branko Cibej > Sent: dinsdag 8 februari 2011 4:39 > To: dev@subversion.apache.org > Subject: Re: SQLite and callbacks > > On 07.02.2011 21:51, Stefan Sperling wrote: > >> A lot of

Re: SQLite and callbacks

2011-02-07 Thread Branko Čibej
On 07.02.2011 21:51, Stefan Sperling wrote: >> A lot of wc databases out there will be >> so small that the user will hardly notice the memory increase. > All we'd be doing is allowing sqlite to flush data to disk if needed. > Even with a temporary table backed by a file, most operations happen in

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 09:32:48PM +0100, Branko Čibej wrote: > On 07.02.2011 17:10, Stefan Sperling wrote: > > The bug is probabaly in the following query. > > Maybe the INSERT OR REPLACE doesn't work as intended? > > And why is COMMIT TRANSACTION commented, BTW? Is this the problem? > > > > -- ST

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 09:23:23PM +0100, Johan Corveleyn wrote: > I've been wondering about the question "how about storing/buffering > the entire query results in memory?" Would this really be a problem, > even for very large working copies? > > I have a quite large working copy checked out here

Re: SQLite and callbacks

2011-02-07 Thread Branko Čibej
On 07.02.2011 21:23, Johan Corveleyn wrote: > On Mon, Feb 7, 2011 at 5:28 PM, Stefan Sperling wrote: >> On Mon, Feb 07, 2011 at 05:18:57PM +0100, Stefan Sperling wrote: >>> On Mon, Feb 07, 2011 at 10:55:47AM -0500, Mark Phippard wrote: On Mon, Feb 7, 2011 at 10:52 AM, Stefan Sperling wrote:

Re: SQLite and callbacks

2011-02-07 Thread Branko Čibej
On 07.02.2011 17:10, Stefan Sperling wrote: > The bug is probabaly in the following query. > Maybe the INSERT OR REPLACE doesn't work as intended? > And why is COMMIT TRANSACTION commented, BTW? Is this the problem? > > -- STMT_REPLACE_ACTUAL_PROPS_IN_CACHE > INSERT OR REPLACE INTO temp_query_cache

Re: SQLite and callbacks

2011-02-07 Thread Johan Corveleyn
On Mon, Feb 7, 2011 at 5:28 PM, Stefan Sperling wrote: > On Mon, Feb 07, 2011 at 05:18:57PM +0100, Stefan Sperling wrote: >> On Mon, Feb 07, 2011 at 10:55:47AM -0500, Mark Phippard wrote: >> > On Mon, Feb 7, 2011 at 10:52 AM, Stefan Sperling wrote: >> > >> > > Where is the temporary table stored?

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 05:18:57PM +0100, Stefan Sperling wrote: > On Mon, Feb 07, 2011 at 10:55:47AM -0500, Mark Phippard wrote: > > On Mon, Feb 7, 2011 at 10:52 AM, Stefan Sperling wrote: > > > > > Where is the temporary table stored? Is it back by a file or memory? > > > If backed by memory, d

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 10:55:47AM -0500, Mark Phippard wrote: > On Mon, Feb 7, 2011 at 10:52 AM, Stefan Sperling wrote: > > > Where is the temporary table stored? Is it back by a file or memory? > > If backed by memory, do we have to worry about memory consumption for > > large working copies? >

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 04:52:30PM +0100, Stefan Sperling wrote: > I like this patch a lot and think we should commit it after fixing > the test failure. The test is failing because your code lists both the BASE properties and the ACTUAL properties for a node which had its props changed: sqlite>

Re: SQLite and callbacks

2011-02-07 Thread Mark Phippard
On Mon, Feb 7, 2011 at 10:52 AM, Stefan Sperling wrote: > Where is the temporary table stored? Is it back by a file or memory? > If backed by memory, do we have to worry about memory consumption for > large working copies? The patch says it is backed by a file. -- Thanks Mark Phippard http://

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 12:43:41PM +0100, Stefan Sperling wrote: > On Mon, Feb 07, 2011 at 10:30:15AM +0100, Branko Čibej wrote: > > On 07.02.2011 02:52, Branko Čibej wrote: > > > Note that I didn't run the tests with this patch, so I'm not claiming it > > > to be bug-free. In fact there's one bug

Re: SQLite and callbacks

2011-02-07 Thread Stefan Sperling
On Mon, Feb 07, 2011 at 10:30:15AM +0100, Branko Čibej wrote: > On 07.02.2011 02:52, Branko Čibej wrote: > > Note that I didn't run the tests with this patch, so I'm not claiming it > > to be bug-free. In fact there's one bug in the recursive proplist of the > > WC root, where the root props show u

Re: SQLite and callbacks

2011-02-07 Thread Branko Čibej
On 07.02.2011 02:52, Branko Čibej wrote: > Note that I didn't run the tests with this patch, so I'm not claiming it > to be bug-free. In fact there's one bug in the recursive proplist of the > WC root, where the root props show up twice -- there's an issue with the > filter in the query there. But

Re: SQLite and callbacks

2011-02-06 Thread Branko Čibej
On 05.02.2011 21:55, Johan Corveleyn wrote: > On Sat, Feb 5, 2011 at 9:37 PM, Johan Corveleyn wrote: >> On Sat, Feb 5, 2011 at 7:14 PM, Stefan Sperling wrote: >>> On Sat, Feb 05, 2011 at 06:47:35PM +0100, Branko Čibej wrote: I would not worry about existing clients -- simply mark the existin

Re: SQLite and callbacks

2011-02-05 Thread Johan Corveleyn
On Sat, Feb 5, 2011 at 9:37 PM, Johan Corveleyn wrote: > On Sat, Feb 5, 2011 at 7:14 PM, Stefan Sperling wrote: >> On Sat, Feb 05, 2011 at 06:47:35PM +0100, Branko Čibej wrote: >>> I would not worry about existing clients -- simply mark the existing >>> APIs as deprecated, but keep them and do no

Re: SQLite and callbacks

2011-02-05 Thread Johan Corveleyn
On Sat, Feb 5, 2011 at 7:14 PM, Stefan Sperling wrote: > On Sat, Feb 05, 2011 at 06:47:35PM +0100, Branko Čibej wrote: >> I would not worry about existing clients -- simply mark the existing >> APIs as deprecated, but keep them and do not attempt to improve their >> performance. > > Neglecting per

Re: SQLite and callbacks

2011-02-05 Thread Stefan Sperling
On Sat, Feb 05, 2011 at 06:47:35PM +0100, Branko Čibej wrote: > I would not worry about existing clients -- simply mark the existing > APIs as deprecated, but keep them and do not attempt to improve their > performance. Neglecting performance of backwards compat code is an interesting idea. It all

Re: SQLite and callbacks

2011-02-05 Thread Branko Čibej
On 05.02.2011 15:35, Stefan Sperling wrote: > On Sat, Feb 05, 2011 at 03:29:23PM +0100, Stefan Sperling wrote: >> I think we should strongly consider revving affected callbacks in the >> 1.7 API and document restriction they have to heed. Then we can bring >> the r1039808 code back. We can keep the

Re: SQLite and callbacks

2011-02-05 Thread Stefan Sperling
On Sat, Feb 05, 2011 at 03:29:23PM +0100, Stefan Sperling wrote: > I think we should strongly consider revving affected callbacks in the > 1.7 API and document restriction they have to heed. Then we can bring > the r1039808 code back. We can keep the code added in r1066541 for > backwards-compatibi

Re: SQLite and callbacks

2011-02-05 Thread Stefan Sperling
On Sat, Feb 05, 2011 at 01:11:28PM +0100, Stefan Sperling wrote: > I'll look at profiler output. It has much better granularity. I took 3 profiled runs of "svn pl -v -R" on an svn-trunk working copy with and without the patch. The results were pretty much the same for all runs. The numbers below a

Re: SQLite and callbacks

2011-02-05 Thread Stefan Sperling
On Fri, Feb 04, 2011 at 03:29:37AM +0200, Daniel Shahaf wrote: > (test script attached --- hope it goes through) > > Stefan Sperling wrote on Wed, Feb 02, 2011 at 18:53:39 +0100: > > I've made svn proplist issue per-directory queries in r1066541. > > Reviews of this change are most welcome. > > Al

Re: SQLite and callbacks

2011-02-03 Thread Daniel Shahaf
(test script attached --- hope it goes through) Stefan Sperling wrote on Wed, Feb 02, 2011 at 18:53:39 +0100: > I've made svn proplist issue per-directory queries in r1066541. > Reviews of this change are most welcome. > Also, if someone has a nice way of testing performance impacts of > this chan

Re: SQLite and callbacks

2011-02-02 Thread Stefan Sperling
On Thu, Jan 13, 2011 at 01:25:27PM +, Stefan Sperling wrote: > This is about callbacks in the libsvn_client(!) and libsvn_wc APIs. > Those are widely used by third parties so we should really make > the right decision. > > Do we trust third parties to heed restrictions we place on callbacks? >

Re: SQLite and callbacks

2011-01-13 Thread C. Michael Pilato
On 01/13/2011 10:10 AM, Stefan Sperling wrote: > If we decide that the new approach violates our backwards compat rules, > we'd have to provide a separate implementation to continue supporting > semantics of old APIs. Ah, okay. This is precise what I did in BDB recently with the svn_fs_get_locks(

Re: SQLite and callbacks

2011-01-13 Thread Stefan Sperling
On Thu, Jan 13, 2011 at 09:03:10AM -0500, C. Michael Pilato wrote: > On 01/13/2011 08:25 AM, Stefan Sperling wrote: > > On Wed, Jan 12, 2011 at 09:46:47AM -0500, C. Michael Pilato wrote: > >> Nothing to add to the immediate discussion at this time. Just wanted to > >> note that we have essentially

Re: SQLite and callbacks

2011-01-13 Thread C. Michael Pilato
On 01/13/2011 08:25 AM, Stefan Sperling wrote: > On Wed, Jan 12, 2011 at 09:46:47AM -0500, C. Michael Pilato wrote: >> Nothing to add to the immediate discussion at this time. Just wanted to >> note that we have essentially the same problems to deal with in the BDB >> backend. The solution we've

Re: SQLite and callbacks

2011-01-13 Thread Stefan Sperling
On Wed, Jan 12, 2011 at 09:46:47AM -0500, C. Michael Pilato wrote: > Nothing to add to the immediate discussion at this time. Just wanted to > note that we have essentially the same problems to deal with in the BDB > backend. The solution we've taken there is "don't drive public API > callbacks f

Re: SQLite and callbacks

2011-01-12 Thread C. Michael Pilato
Nothing to add to the immediate discussion at this time. Just wanted to note that we have essentially the same problems to deal with in the BDB backend. The solution we've taken there is "don't drive public API callbacks from inside BDB transactions". It's not always the best for performance, bu

Re: SQLite and callbacks

2011-01-12 Thread Stefan Sperling
On Tue, Nov 30, 2010 at 12:33:38PM +, Philip Martin wrote: > "Hyrum K. Wright" writes: > > > Stefan's patch to make a recursive proplist much more performant > > highlights the great benefit that our sqlite-backed storage can have. > > However, he reverted it due to concerns about the potenti

Re: SQLite and callbacks

2010-12-11 Thread Stefan Sperling
On Tue, Nov 30, 2010 at 03:52:06PM +0100, Stefan Sperling wrote: > On Tue, Nov 30, 2010 at 03:40:01PM +0100, Stefan Sperling wrote: > > We could forbid callbacks from calling any other libsvn_client function, > > and provide all information they will ever need as arguments. Careful > > consideratio

Re: SQLite and callbacks

2010-11-30 Thread Hyrum K. Wright
On Tue, Nov 30, 2010 at 9:20 AM, Philip Martin wrote: > Florian Weimer writes: > >> * Philip Martin: >> >>> An SQLite read blocks SQlite writes by other threads/processes, so >>> the long lived select may cause writes to fail that would otherwise >>> succeed. >> >> AFAIK, such blocking only happe

Re: SQLite and callbacks

2010-11-30 Thread Philip Martin
Florian Weimer writes: > * Philip Martin: > >> An SQLite read blocks SQlite writes by other threads/processes, so >> the long lived select may cause writes to fail that would otherwise >> succeed. > > AFAIK, such blocking only happens inside SQLite if you are not in WAL > mode and the write trans

Re: SQLite and callbacks

2010-11-30 Thread Florian Weimer
* Philip Martin: > An SQLite read blocks SQlite writes by other threads/processes, so > the long lived select may cause writes to fail that would otherwise > succeed. AFAIK, such blocking only happens inside SQLite if you are not in WAL mode and the write transaction has outgrown its cache size.

Re: SQLite and callbacks

2010-11-30 Thread Stefan Sperling
On Tue, Nov 30, 2010 at 03:40:01PM +0100, Stefan Sperling wrote: > We could forbid callbacks from calling any other libsvn_client function, > and provide all information they will ever need as arguments. Careful > consideration of possible use cases for an API would allow us to do this. > E.g. for

Re: SQLite and callbacks

2010-11-30 Thread Stefan Sperling
On Mon, Nov 29, 2010 at 07:43:40PM -0600, Hyrum K. Wright wrote: > We use callbacks extensively throughout our code as a means of > providing streamy feedback to callers. It's a pretty good paradigm, > and one that has served us well. We don't put many restrictions on > what the callbacks can do

Re: SQLite and callbacks

2010-11-30 Thread Stefan Sperling
On Tue, Nov 30, 2010 at 01:32:27PM +, Philip Martin wrote: > Stefan Sperling writes: > > > If you run this version of your test, it will run in an endless loop. > > The callback inserts new values, the caller will see those and invoke > > the callback again. > > Hmm, I expected the select to

Re: SQLite and callbacks

2010-11-30 Thread Philip Martin
Stefan Sperling writes: > If you run this version of your test, it will run in an endless loop. > The callback inserts new values, the caller will see those and invoke > the callback again. Hmm, I expected the select to block the write causing it to return SQLITE_BUSY. That is what happens if t

Re: SQLite and callbacks

2010-11-30 Thread Stefan Sperling
On Mon, Nov 29, 2010 at 07:43:40PM -0600, Hyrum K. Wright wrote: > We use callbacks extensively throughout our code as a means of > providing streamy feedback to callers. It's a pretty good paradigm, > and one that has served us well. We don't put many restrictions on > what the callbacks can do

Re: SQLite and callbacks

2010-11-30 Thread Philip Martin
"Hyrum K. Wright" writes: > Stefan's patch to make a recursive proplist much more performant > highlights the great benefit that our sqlite-backed storage can have. > However, he reverted it due to concerns about the potential for > database contention. The theory was that the callback might try

SQLite and callbacks

2010-11-29 Thread Hyrum K. Wright
We use callbacks extensively throughout our code as a means of providing streamy feedback to callers. It's a pretty good paradigm, and one that has served us well. We don't put many restrictions on what the callbacks can do in terms of fetching more information or calling other functions. Enter