Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Gabriel Corneanu
I admit I didn't think (or didn't even read in detail) about technical implementation. This is an extract from analyzer: *** Table AE_DATA Percentage of total database.. 99.89% Number of entries. 1030371 Bytes of

[sqlite] MMIO and VFS Obfuscation

2013-06-07 Thread Paul Vercellotti
Hi there, We use a vfs-based obfuscation system using the old xRead and xWrite methods.   I'm wondering if these can be adapted to work with the new memory-mapped i/o functionality in a way that still has the advantages of memory-mapped i/o? The current system does in-place decryption on read

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Simon Slavin
On 7 Jun 2013, at 8:57am, Gabriel Corneanu gabrielcorne...@gmail.com wrote: BTW I found this by opening some file over network, which of course made everything worse. [...] Not that I really need, but I have to support specified data rates up to 100k records / second. Maximum speed of a

[sqlite] regexp.c

2013-06-07 Thread Philip Bennefall
Hello all, I was having a quick look at the extensions provided in ext/misc in the source tree, and I am pleasantly surprised at the number of useful things that I found in there that I didn't know about. I was wondering about the regular expression parser in particular. Does it support

Re: [sqlite] Header Application-ID list

2013-06-07 Thread Eduardo Morras
On Thu, 6 Jun 2013 08:15:57 -0400 Richard Hipp d...@sqlite.org wrote: On Thu, Jun 6, 2013 at 8:05 AM, Eduardo Morras emorr...@yahoo.es wrote: Hi, Is there an official list of assigned application id sqlite header? If exist, How can I register my application-id? The official list

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Gabriel Corneanu
This is not at all my case ... I don't obviously write 1 by 1, but using blocks of data ( array of struct ), virtual tables wrappers, and insert ... select. This way I can achieve 200k rec/s, or at least 100k when having some more fields. Right now I'm completely CPU bound, it's 100% load at

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Eduardo Morras
On Thu, 6 Jun 2013 10:53:55 -0400 Richard Hipp d...@sqlite.org wrote: On Thu, Jun 6, 2013 at 10:52 AM, Gabriel Corneanu gabrielcorne...@gmail.com wrote: Strange is, count(*) uses the cover index for a but select count(a) does NOT use the same cover index... count(a) has to check for

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Clemens Ladisch
Eduardo Morras wrote: where t.a = NULL where t.a IS NULL (NULL compares as not equal to any value, including itself.) Regards, Clemens ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] regexp.c

2013-06-07 Thread Richard Hipp
On Fri, Jun 7, 2013 at 5:30 AM, Philip Bennefall phi...@blastbay.comwrote: Hello all, I was having a quick look at the extensions provided in ext/misc in the source tree, and I am pleasantly surprised at the number of useful things that I found in there that I didn't know about. I was

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Eduardo Morras
On Fri, 07 Jun 2013 13:12:14 +0200 Clemens Ladisch clem...@ladisch.de wrote: Eduardo Morras wrote: where t.a = NULL where t.a IS NULL (NULL compares as not equal to any value, including itself.) OPppss you're right. Thought too fast and wrote even faster :( Regards, Clemens ---

[sqlite] FW: problem sqlite3 c++ please

2013-06-07 Thread Maxime Gerum
Hello, i’m french, sorry if i make errors of langage In my school project, i use sqlite3 with c++, i can make any request such as create table or select *from, no problems. But i would export my database in a csv file, in sqlite3 .mode csv

[sqlite] FW: problem sqlite3 c++ please

2013-06-07 Thread Maxime Gerum
Hello, i’m french, sorry if i make errors of langage In my school project, i use sqlite3 with c++, i can make any request such as create table or select *from, no problems. But i would export my database in a csv file, in sqlite3 .mode csv

Re: [sqlite] FW: problem sqlite3 c++ please

2013-06-07 Thread fnoyanisi
Is this a repeated post? You cannot use shell commands as Sql statements in c api. Write your own export routine. Here is one I wrote. https://github.com/fnoyanisi/sqlite3_capi_extensions On 06/06/2013, at 11:02 PM, Maxime Gerum pilot@hotmail.fr wrote: Hello, i’m french, sorry if i

Re: [sqlite] problem sqlite3 c++ please

2013-06-07 Thread Simon Slavin
On 6 Jun 2013, at 2:30pm, Maxime Gerum pilot@hotmail.fr wrote: but in c++, if i make request[4]= ”.mode csv” Commands starting with . are specially built into the SQLite shell tool application. They are not part of SQLite itself and cannot be used just by using API calls. Sorry.

Re: [sqlite] MMIO and VFS Obfuscation

2013-06-07 Thread Drake Wilson
Quoth Paul Vercellotti pverce...@yahoo.com, on 2013-06-07 01:07:35 -0700: The current system does in-place decryption on read (no copying), which adds almost no overhead to the operation.    Now in-place decryption with a memory-mapped file seems like it would dirty the page, which could

Re: [sqlite] MMIO and VFS Obfuscation

2013-06-07 Thread Drake Wilson
Quoth Drake Wilson dr...@dasyatidae.net, on 2013-06-07 08:14:27 -0500: If you really want, you might be able to implement xFetch to allocate a shadow buffer, decrypt from the map into that, and return that pointer. Since it's designed for accessing maps directly, though, I don't see it

Re: [sqlite] regexp.c

2013-06-07 Thread Philip Bennefall
- Original Message - From: Richard Hipp To: phi...@blastbay.com ; General Discussion of SQLite Database Sent: Friday, June 07, 2013 1:14 PM Subject: Re: [sqlite] regexp.c On Fri, Jun 7, 2013 at 5:30 AM, Philip Bennefall phi...@blastbay.com wrote: Hello all, I

Re: [sqlite] MMIO and VFS Obfuscation

2013-06-07 Thread Drake Wilson
Quoth Drake Wilson dr...@dasyatidae.net, on 2013-06-07 08:18:05 -0500: Actually, I dropped a paragraph on the floor, sorry. It's probably better to use xRead for this, since in that case SQLite will manage its own memory for the cache of decrypted pages. The loss in that case, if you

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Ryan Johnson
On 06/06/2013 10:52 AM, Gabriel Corneanu wrote: In my opinion, count(*) is the same as count(rowid) (I see that even count() is accepted); I could say it's even the same as count(x) (any other field). Not quite... count(x) only counts rows having non-NULL x. Granted, that's not a problem for

Re: [sqlite] select count(*) does not use primary key??

2013-06-07 Thread Richard Hipp
On Fri, Jun 7, 2013 at 11:51 AM, Ryan Johnson ryan.john...@cs.utoronto.cawrote: On 06/06/2013 10:52 AM, Gabriel Corneanu wrote: In my opinion, count(*) is the same as count(rowid) (I see that even count() is accepted); I could say it's even the same as count(x) (any other field). Not

[sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
I would expect all queries which specify the primary key components in the WHERE clause to use the Primary Key in the query plan, regardless of if ANALYZE has been run or not. I would also think it would assume any index which covers the most where-clause components would be the most efficient

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Richard Hipp
On Fri, Jun 7, 2013 at 12:37 PM, Brad House b...@monetra.com wrote: I would expect all queries which specify the primary key components in the WHERE clause to use the Primary Key in the query plan, regardless of if ANALYZE has been run or not. SQLite examines many different strategies for

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Simon Slavin
On 7 Jun 2013, at 5:37pm, Brad House b...@monetra.com wrote: I've modified my code to run an Analyze on startup to work around this, but it obviously takes time to run and slows down startup. I can't answer your question about why this happens in the first place, but I can tell you that the

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
I would expect all queries which specify the primary key components in the WHERE clause to use the Primary Key in the query plan, regardless of if ANALYZE has been run or not. SQLite examines many different strategies for evaluating each query. For each strategy it tries to estimate the total

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
On 06/07/2013 12:46 PM, Simon Slavin wrote: On 7 Jun 2013, at 5:37pm, Brad House b...@monetra.com wrote: I've modified my code to run an Analyze on startup to work around this, but it obviously takes time to run and slows down startup. I can't answer your question about why this happens in

Re: [sqlite] An unable to open database file error that has nothing to do with opening database file

2013-06-07 Thread Warren Young
On 6/6/2013 21:56, Philip Goetz wrote: I think the problem is that the Cygwin distribution has the wrong version of SQLite, one built for unix. Nope. And even if true, it wouldn't be the right explanation. There are two major ways to build SQLite on Cygwin: 1. By default, building SQLite

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Richard Hipp
On Fri, Jun 7, 2013 at 12:56 PM, Brad House b...@monetra.com wrote: I would expect all queries which specify the primary key components in the WHERE clause to use the Primary Key in the query plan, regardless of if ANALYZE has been run or not. SQLite examines many different strategies for

Re: [sqlite] Query on primary key not using primary key?

2013-06-07 Thread Brad House
Without additional information, SQLite guesses that the data_idx index will narrow down the search to about 7 entries in the table. This is, of course, a guess, but it is a reasonable guess for most indices. The primary key, even though it is unique, has an IN clause with 50 entries, it SQLite

Re: [sqlite] Not reading data in the -wal file? (Not Reproducible)

2013-06-07 Thread Jerry Krinock
Thank you to Igor and Richard. I've studied this issue more, and still don't have an answer, although I've not been able to reproduce it either. I'm not using shared cache, and even if I did leave a database connection open, which seems impossible since sqlite3_open(), sqlite3_finalize() and