[sqlite] MMAP performance with databases over 2GB

2015-08-06 Thread Dan Kennedy
On 08/06/2015 09:53 PM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > In my testing I am finding that using SQLite's mmap support with > larger databases (final db size is ~10GB) to create the database (WAL > mode) is considerably (about three times) slower than no mmap.

[sqlite] Determine query type

2015-08-06 Thread Stephan Beal
On Thu, Aug 6, 2015 at 6:57 PM, Ben Newberg wrote: > Excellent. This is exactly what I was looking for. > Great :). Now that i have some code in front of me i can say, more concretely, that sqlite3_column_count() is what you want. https://www.sqlite.org/c3ref/column_count.html -- - stepha

[sqlite] Determine query type

2015-08-06 Thread Stephan Beal
On Thu, Aug 6, 2015 at 6:46 PM, Ben Newberg wrote: > Is there a good way to determine if a sql query is either attempting to > modify a database, or simply querying it? > Check the column count of the prepared statement: it will be >0 for a select or select-like pragma. Anything else is an updat

[sqlite] Sorting by greatest count amount

2015-08-06 Thread Reinhard Max
On Thu, 6 Aug 2015 at 17:51, Reinhard Max wrote: > On Thu, 6 Aug 2015 at 17:41, jose isaias cabrera wrote: > >> SELECT pmuk, count(*) FROM LSOpenJobs WHERE pmuk LIKE '% %' GROUP BY pmuk; > > For ordered results you need an ORDER BY clause, and if you want to > order by a calculated column (count

[sqlite] Sorting by greatest count amount

2015-08-06 Thread Reinhard Max
On Thu, 6 Aug 2015 at 17:41, jose isaias cabrera wrote: > SELECT pmuk, count(*) FROM LSOpenJobs WHERE pmuk LIKE '% %' GROUP BY pmuk; For ordered results you need an ORDER BY clause, and if you want to order by a calculated column (count in this case), you have to give it a name: SELECT pmuk,

[sqlite] Determine query type

2015-08-06 Thread Nelson, Erik - 2
> users-bounces at mailinglists.sqlite.org] On Behalf Of Scott Hess wrote on > Thursday, August 06, 2015 1:11 PM > > Also consider https://www.sqlite.org/c3ref/stmt_readonly.html > > On Thu, Aug 6, 2015 at 10:08 AM, Stephan Beal > wrote: > > > On Thu, Aug 6, 2015 at 6:57 PM, Ben Newberg > wro

[sqlite] Read data from database slow the first time but then very fast thereafter. How can I improve the speed for the first time read?

2015-08-06 Thread Stephan Beal
On Thu, Aug 6, 2015 at 4:54 PM, Wade, William wrote: > I'm an SQLite novice, so if I'm wrong about this, I hope someone will > correct me. > > The first query will be slower than later queries for the reason Simon > mentioned. Pedantically speaking, "will" is not quite right. That will _normall

[sqlite] Read data from database slow the first time but then very fast thereafter. How can I improve the speed for the first time read?

2015-08-06 Thread Wade, William
I'm an SQLite novice, so if I'm wrong about this, I hope someone will correct me. The first query will be slower than later queries for the reason Simon mentioned. However, it may still be possible to get some speed up of the first query, if it can avoid reading stuff not in the result set and

[sqlite] Sqlite ticket f51d9501800de5a0fb69d5048ce6662981b461ec

2015-08-06 Thread Bernhard Schommer
Hi, the warning which is mentioned in the ticket still occurs also with newer gcc versions. The ticket was closed after a gcc bug report was opened. The gcc bug was closed due to a missing testcase. I had a quick look at the problem and it seems that the warning is right since in certain context a

[sqlite] Regarding root in transitive closure extension of SQLite

2015-08-06 Thread Sairam Gaddam
I have enabled the transitive closure extension and I followed the steps regarding querying the virtual table mentioned in closure.c file as follows CREATE VIRTUAL TABLE ct1 USING transitive_closure( tablename='group1', idcolumn='groupId', parentcolumn='parentId' ); CR

[sqlite] Sorting by greatest count amount

2015-08-06 Thread jose isaias cabrera
"Reinhard Max" wrote... > On Thu, 6 Aug 2015 at 17:51, Reinhard Max wrote: > >> On Thu, 6 Aug 2015 at 17:41, jose isaias cabrera wrote: >> >>> SELECT pmuk, count(*) FROM LSOpenJobs WHERE pmuk LIKE '% %' GROUP BY >>> pmuk; >> >> For ordered results you need an ORDER BY clause, and if you want to

[sqlite] Sorting by greatest count amount

2015-08-06 Thread jose isaias cabrera
"Reinhard Max" wrote... > > On Thu, 6 Aug 2015 at 17:41, jose isaias cabrera wrote: > >> SELECT pmuk, count(*) FROM LSOpenJobs WHERE pmuk LIKE '% %' GROUP BY >> pmuk; > > For ordered results you need an ORDER BY clause, and if you want to order > by a calculated column (count in this case), you

[sqlite] Determine query type

2015-08-06 Thread Ben Newberg
Excellent. This is exactly what I was looking for. Thanks. On Thu, Aug 6, 2015 at 11:50 AM, Stephan Beal wrote: > On Thu, Aug 6, 2015 at 6:46 PM, Ben Newberg wrote: > > > Is there a good way to determine if a sql query is either attempting to > > modify a database, or simply querying it? > > >

[sqlite] Bugs/improvements to sqlite constraints

2015-08-06 Thread sqlite-mail
Hello ! After submitting several emails with subject "Bug in sqlite3_trace/trigger/delete" and only one reply to then so far I decided to look a bit more deep on the problem I'm facing using sqlite3 with one specific database and created a simpler database that can show several problems and oppo

[sqlite] Determine query type

2015-08-06 Thread Ben Newberg
Is there a good way to determine if a sql query is either attempting to modify a database, or simply querying it? In the past, I have used sqlite3_update_hook with good results, but in this current project, I am looking for a way to determine the type of query even before the prepared statement is

[sqlite] Sorting by greatest count amount

2015-08-06 Thread jose isaias cabrera
Greetings. I have this query, SELECT pmuk, count(*) FROM LSOpenJobs WHERE pmuk LIKE '% %' GROUP BY pmuk; this works but it is not sorted. I tried things like SELECT pmuk, count(*) FROM LSOpenJobs WHERE pmuk LIKE '% %' GROUP BY count(*) || '-' || pmuk; and SELECT pmuk, count(*) FROM LSOpenJ

[sqlite] Determine query type

2015-08-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/06/2015 09:46 AM, Ben Newberg wrote: > Is there a good way to determine if a sql query is either > attempting to modify a database, or simply querying it? You most like want to use the authoriser. It tells you exactly what the query is doing, a

[sqlite] MMAP performance with databases over 2GB

2015-08-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/06/2015 09:27 AM, Dan Kennedy wrote: >> Is it using more CPU cycles in mmap mode or just taking longer? >> If the former, does [time] attribute them to "user" or "system"? It is taking longer. I have 3 XML dumps which I turn into JSON (incremen

[sqlite] Read data from database slow the first time but then very fast thereafter. How can I improve the speed for the first time read?

2015-08-06 Thread John McKown
If the OP shows us the data base schema, and the select, then perhaps its performance could be enhanced by changing something. One thing which I sometimes do is "denormalize" a table by splitting it into two, or more, tables which are related via a shared UUID value. I put the often used columns in

[sqlite] System.Data.SQLite encryption

2015-08-06 Thread Marco Bambini
Anyone know which encryption algorithm is used in System.Data.SQLite? Seems like it doesn?t use the standard SEE sqlite extension. Thanks a lot. -- Marco Bambini http://www.sqlabs.com http://twitter.com/sqlabs http://instagram.com/sqlabs

[sqlite] Determine query type

2015-08-06 Thread Scott Hess
Also consider https://www.sqlite.org/c3ref/stmt_readonly.html -scott On Thu, Aug 6, 2015 at 10:08 AM, Stephan Beal wrote: > On Thu, Aug 6, 2015 at 6:57 PM, Ben Newberg wrote: > > > Excellent. This is exactly what I was looking for. > > > > Great :). Now that i have some code in front of me i c

[sqlite] Bug in sqlite3_trace/trigger/delete

2015-08-06 Thread sqlite-mail
Hello ! After waiting for any help on this issue I've tried to see what "explain query plan delete from res_users where id=7" shows and found that on this particular database because the res_users.id is a foreign key on almost all other tables and sometimes on more than one field (most of then w

[sqlite] Regarding root in transitive closure extension of SQLite

2015-08-06 Thread Dominique Devienne
On Thu, Aug 6, 2015 at 8:55 AM, Sairam Gaddam wrote: > I have enabled the transitive closure extension and I followed the steps > regarding querying the virtual table mentioned in closure.c file as follows > Given that SQLite supports CTE's now, why use that extension? AFAIK, it was to add hier

[sqlite] MMAP performance with databases over 2GB

2015-08-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In my testing I am finding that using SQLite's mmap support with larger databases (final db size is ~10GB) to create the database (WAL mode) is considerably (about three times) slower than no mmap. The default max mmap limit is 2GB (sadly). Has anyon

[sqlite] System.Data.SQLite encryption

2015-08-06 Thread Kevin Benson
On Thu, Aug 6, 2015 at 4:16 AM, Marco Bambini wrote: > Anyone know which encryption algorithm is used in System.Data.SQLite? > Seems like it doesn?t use the standard SEE sqlite extension. > https://www.mail-archive.com/sqlite-users at sqlite.org/msg87770.html -- -- -- --???--