Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
Without additional information, SQLite guesses that the data_idx index will narrow down the search to about 7 entries in the table. This is, of course, a guess, but it is a reasonable guess for most indices. The primary key, even though it is unique, has an IN clause with 50 entries, it SQLite g

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
On 06/07/2013 12:46 PM, Simon Slavin wrote: On 7 Jun 2013, at 5:37pm, Brad House wrote: I've modified my code to run an Analyze on startup to work around this, but it obviously takes time to run and slows down startup. I can't answer your question about why this happens in the f

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
I would expect all queries which specify the primary key components in the WHERE clause to use the Primary Key in the query plan, regardless of if ANALYZE has been run or not. SQLite examines many different strategies for evaluating each query. For each strategy it tries to estimate the total

[sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
I would expect all queries which specify the primary key components in the WHERE clause to use the Primary Key in the query plan, regardless of if ANALYZE has been run or not. I would also think it would assume any index which covers the most where-clause components would be the most efficient if

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug?? -- test case!

2012-05-01 Thread Brad House
On 04/28/2012 09:36 AM, Richard Hipp wrote: On Sat, Apr 28, 2012 at 8:24 AM, Black, Michael (IS) wrote: Should another "disadvantage" of WAL mode be added to http://www.sqlite.org/draft/wal.html Something that says rolled back transactions will cause an abort on any reads in progress if sha

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug?? -- test case!

2012-04-27 Thread Brad House
On 04/27/2012 04:38 PM, Richard Hipp wrote: Disable shared cache mode and you should be good to go. If two database connections share the same cache, and one connection rolls back, that means it will be changing cache content out from under the other database connection, so any queries ongoing i

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug?? -- test case!

2012-04-27 Thread Brad House
Ok, I guess attachments don't come through. I've uploaded it here: http://www.brad-house.com/other/sqlite_test.c -Brad On 04/27/2012 03:50 PM, Brad House wrote: On 04/27/2012 02:00 PM, Brad House wrote: Only the connection that does the rollback has its queries aborted. That

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug?? -- test case!

2012-04-27 Thread Brad House
On 04/27/2012 02:00 PM, Brad House wrote: Only the connection that does the rollback has its queries aborted. That is not the behavior I am seeing in 3.7.11, but was the behavior I saw in 3.7.10. If you are seeing other connections get queries aborted, that is something new that I have not

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug??

2012-04-27 Thread Brad House
Only the connection that does the rollback has its queries aborted. That is not the behavior I am seeing in 3.7.11, but was the behavior I saw in 3.7.10. If you are seeing other connections get queries aborted, that is something new that I have not seen before and will need to investigate.

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug??

2012-04-26 Thread Brad House
On 04/26/2012 04:38 PM, Jos Groot Lipman wrote: As far as I understand this means: you will not see changes made by other connections (committed or uncommited) after your transaction started. If another connections commits a change, you will not see it. I would expect: If another connections roll

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug??

2012-04-26 Thread Brad House
On 04/26/2012 05:11 PM, Richard Hipp wrote: On Thu, Apr 26, 2012 at 4:38 PM, Jos Groot Lipman wrote: From the docs: 'The default isolation level for SQLite is SERIALIZABLE' As far as I understand this means: you will not see changes made by other connections (committed or uncommited) after y

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug??

2012-04-26 Thread Brad House
On 04/26/2012 04:09 PM, Black, Michael (IS) wrote: Would WAL mode prevent this? We're using WAL mode :) -Brad ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug??

2012-04-26 Thread Brad House
On 04/26/2012 03:44 PM, Richard Hipp wrote: On Thu, Apr 26, 2012 at 3:34 PM, Brad House wrote: I've got 2 threads with different connections to the same database. This is approximately what I am seeing: Thread 1: SELECT bar,baz FROM foo WHERE ...; Thread 2: BEGIN IMME

[sqlite] SQLite 3.7.11 behavioral change from 3.7.10 with concurrency -- bug??

2012-04-26 Thread Brad House
We just noticed a behavioral change in SQLite 3.7.11 that caused a regression in our software. This change did not exist in 3.7.10. Looking back at the changelog, I notice this: "Pending statements no longer block ROLLBACK. Instead, the pending statement will return SQLITE_ABORT upon next acces

[sqlite] 3.7.6.1 build error on platforms without pread/pwrite

2011-04-16 Thread Brad House
It appears there is a bad check when setting up the pread (and pwrite) callback: #if defined(USE_PREAD) || defined(SQLITE_ENABLE_LOCKING_STYLE) { "pread",(sqlite3_syscall_ptr)pread, 0 }, #else { "pread",(sqlite3_syscall_ptr)0, 0 }, #endif SQLite forces SQLIT

[sqlite] 3.7.6.1 build error on platforms without pread/pwrite

2011-04-15 Thread Brad House
It appears there is a bad check when setting up the pread (and pwrite) callback: #if defined(USE_PREAD) || defined(SQLITE_ENABLE_LOCKING_STYLE) { "pread",(sqlite3_syscall_ptr)pread, 0 }, #else { "pread",(sqlite3_syscall_ptr)0, 0 }, #endif SQLite forces SQLIT

Re: [sqlite] Most wanted features of SQLite ?

2009-09-21 Thread Brad House
Simon Slavin wrote: > >> I definitely don't agree here as we're talking about these additional >> locks existing _only_ in memory, not on disk. > > Which requires client/server architecture. Which SQLite3 doesn't > have. Once you require concurrent access features in your DBMS (i.e. > multi

Re: [sqlite] Most wanted features of SQLite ?

2009-09-21 Thread Brad House
>> It could probably benefit a large number of integrations to >> have finer grained locking even if it could not be implemented for all >> integration types. > > It makes the system a great deal slower, since you need to keep > checking all the levels of lock you have implemented. For instance

Re: [sqlite] Most wanted features of SQLite ?

2009-09-21 Thread Brad House
>> Fine for me. It seems to be everybody else that wants their favorite >> feature imbedded in the core :-) >> >> Fine grained locking would be a great "asset" I feel. Notice I did >> not >> request a "feature." > > > On my to-do list is to write a paper that explains why fine-grain > locki

[sqlite] SQLITE_MUTEX_APPDEF doc issue

2009-05-18 Thread Brad House
As of SQLite 3.6, SQLITE_MUTEX_APPDEF is no longer valid, but it is still referenced here: http://www.sqlite.org/c3ref/mutex_alloc.html Probably want to make that change in the docs, specifically the section which states: "If SQLite is compiled with the SQLITE_MUTEX_APPDEF preprocessor macro defi

Re: [sqlite] Break compatibility with linux 2.4 in SQLite 3.6.7?

2008-11-23 Thread Brad House
>> When you said breaking compatibility with 2.4, you meant >> NPTL vs LinuxThreads, right? > > My thought as well. There are still some architectures which do not > support NPTL, even with 2.6 kernels (hppa comes to my mind). But lack > of NPTL support causes pain in other areas, too, so it's p

Re: [sqlite] Break compatibility with linux 2.4 in SQLite 3.6.7?

2008-11-22 Thread Brad House
determine it as well, but I'm not sure. -Brad Brad House wrote: > I'm pretty sure both RHEL 2 & 3 both use 2.4 kernels and are > still actively supported by RedHat if that sways your decision. > I know I have clients that are on RHEL3 still so I'd prefer this > c

Re: [sqlite] Break compatibility with linux 2.4 in SQLite 3.6.7?

2008-11-22 Thread Brad House
I'm pretty sure both RHEL 2 & 3 both use 2.4 kernels and are still actively supported by RedHat if that sways your decision. I know I have clients that are on RHEL3 still so I'd prefer this change not to be made if there will be negative impact. We do share connections across threads, but not duri

Re: [sqlite] sqlite 3.5.8 database corruption

2008-07-23 Thread Brad House
D. Richard Hipp wrote: > On Jul 23, 2008, at 1:08 PM, Brad House wrote: > >> I'm just investigating an issue now. This is the first ever >> incident of a corrupt database we've had on a few thousand >> installations, > > Have you read the background infor

Re: [sqlite] sqlite 3.5.8 database corruption

2008-07-23 Thread Brad House
Guess the mailing list stripped the attachment, I've uploaded it here: http://www.monetra.com/~brad/integrity_check.txt.bz2 Brad House wrote: > I'm just investigating an issue now. This is the first ever > incident of a corrupt database we've had on a few thousand > insta

[sqlite] sqlite 3.5.8 database corruption

2008-07-23 Thread Brad House
I'm just investigating an issue now. This is the first ever incident of a corrupt database we've had on a few thousand installations, though most of our installations are on SQLite 3.4, our latest release is now using 3.5.8. We have deployments on just about every OS... The OS that experienced t

Re: [sqlite] 64bit Version

2008-07-18 Thread Brad House
> Hi Brad > Thanks for the prompt reply the error I am getting is that the sqlite files > (see below) could not be copied to the system32 folder? I have tried all > sorts, > I do have administrative rights but just have no luck with this... > > Sqliteodbc.dll > Sqliteodbcu.dll > Sqlite3.odbc.dll

Re: [sqlite] 64bit Version

2008-07-18 Thread Brad House
Yes, we utilize 64bit versions of SQLite on: Linux (x86_64) FreeBSD (x86_64) Windows (x64 -- XP, Vista -- should also work on 2008) Solaris (sparc64) AIX (ppc64) All work fine. -Brad Nic wrote: > Hi All > > Has anyone got Sqlite to run on a 64bit Operating System ? I am having issues > with se

Re: [sqlite] SQLite version 3.5.9

2008-05-14 Thread Brad House
> You may see some performance increase by setting pragma page_size to a > larger value so that SQLite transfers fewer, but larger, blocks across > the network. I would try benchmark tests with page sizes of 8K and 32K > to see if there is a substantial difference. Good point Dennis, though you

Re: [sqlite] SQLite version 3.5.9

2008-05-14 Thread Brad House
> I would disagree with this, unless I misunderstand. File copies (from the > Finder under OS X) to/from our Xserve run at about 50 MBytes/s or about 50% > of theoretical max on our Gbit LAN, whereas reading the records from the > same file via SQLite is 20-25x slower (—2MB/sec at best, terrible >

Re: [sqlite] Cannot get amalgamation built from CVS to compile

2008-05-06 Thread Brad House
We ran into the same problem here. It seems as though maybe the amalgamation is hand-edited for distribution to remove the contents of the config.h to be system agnostic. When we built ours from CVS, we just did the same hand-edit and packaged it and it compiled fine on the dozen or so OS's we di

Re: [sqlite] restricting access to sqlite database

2008-04-22 Thread Brad House
I'm assuming you're using mod_php. For a virtual-hosted environment, I don't think that can be made 'secure'. You probably need to switch to suexec and fastcgi php. That way the php scripts are run as your user rather than the 'www' user. You might glance at this: http://www.k4ml.com/wiki/serve

Re: [sqlite] [newbie] SQLite and VB.Net?

2008-03-02 Thread Brad House
> I don't know anything about .Net, and I'd like to build a quick app > with Visual Studio 2005 or 2008 to check how well it performs with > SQLite. If performance and deployment prove to be good enough, we'll > use VB.Net for new projects and finally dump VB6. I have almost no experience wi

Re: [sqlite] How to compile SQLite for Windows 32bit & 64bit?

2008-02-18 Thread Brad House
> Please Help > > Is there a good tutorial that shows how to compile SQLite for both > Windows 32 bit & 64bit? > > If ICU is required then I need the files to be statically linked rather > than having any external dependencies. > > Anyone know of a good tutorial for building both a 32bit &

Re: [sqlite] VFS memory leak : During lock / unlock operations

2008-02-08 Thread Brad House
>> they have put traces and identified that for 1000 lock calls >> there are only 950 unlock calls, which is a shortage of 50 unlock >> calls. > The os_unix.c backend to SQLite makes no attempt to match lock/unlock > calls, because posix does not requires such. If you are running > on an operatin

[sqlite] error building amalgamation from 3.5.6 source tree

2008-02-06 Thread Brad House
I have a few patches that I apply to SQLite and need to rebuild the amalgamation, but I get an error saying it can't find tsrc/fts3.h I verified on an un-patched 3.5.6 release that the same error occurs, both on Linux and MacOSX 10.5.1. Am I missing a step somewhere? If I remove all the fts3 ref

Re: [sqlite] SQLite 3.5 threading & vfs discussion

2008-02-05 Thread Brad House
Crap, sent the wrong (older/bad) diff... Attached is the right one. Brad House wrote: It appears as though os_unix.c references pthread routines directly when SQLITE_THREADSAFE is defined (and not 0). I think those routines should probably be abstracted like the mutex routines. The problem

[sqlite] SQLite 3.5 threading & vfs discussion

2008-02-05 Thread Brad House
It appears as though os_unix.c references pthread routines directly when SQLITE_THREADSAFE is defined (and not 0). I think those routines should probably be abstracted like the mutex routines. The problem lies if an alternate threading implementation is preferred on a unix-like system (as is th

Re: [sqlite] thread concurrency, inserts using transactions, bug?

2007-06-26 Thread Brad House
[EMAIL PROTECTED] wrote: >> It appears that if 2 threads start transactions at the same time, >> both inserting into the same table, neither thread can finish until >> one has rolled back. > The behavior is deliberate because it gives you, the programmer, > more control and better concurrency in so

Re: [sqlite] thread concurrency, inserts using transactions, bug?

2007-06-26 Thread Brad House
> Here are my results after modifying the "begin transaction" to a "begin > exclusive" > Begin transaction is a bit Lazy in that the lock escalation doesnt occur > until the pager escalates the lock due to a write. > > You'll see that the begin exclusive acquires a lock immediately and avoid

[sqlite] thread concurrency, inserts using transactions, bug?

2007-06-26 Thread Brad House
I've read http://www.sqlite.org/cvstrac/wiki?p=MultiThreading under the "Case in point: a benchmark application I've written for this purpose" and found that current releases of SQLite do not appear to behave in this manner. I cannot find any documentation which clearly states the intended behavio