Re: [sqlite] Reading compressed database files

2014-05-28 Thread Roger Binns
ter - instead you need to measure and tweak multiple places. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlOGMfgACgkQmOOfHg372QT9IACfVvhc1LWG4X2IFBC0rKKNnrdw UFIAoNhtFdh1EZKEo3fx7Kj9bkdKJRW4 =02fs -END PGP SIGNATURE- ___

Re: [sqlite] Reading compressed database files

2014-05-28 Thread Roger Binns
size), or by internal compression with cerod: http://www.hwaci.com/sw/sqlite/cerod.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlOFpvsACgkQmOOfHg372QRSRACfcDqTprcD//n9yYXcGPl9yQfo sTIAoLkIaQHR4JAwk1LbuRzCyQsx/5aN =tYeT -END P

Re: [sqlite] warning for partial indexes with "X AND Y"?

2014-05-23 Thread Roger Binns
e years: http://www.sqlite.org/src/tktview?name=25e09aa2ab The SQLite team rejected it in March. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlN/2XsACgkQmOOfHg372QRtnACeOCNixsgztnpZSqk/4T4fWqZE fN4An0t4mC2VQOXwXeCnV3qYrUTULHRx =qk0x -E

Re: [sqlite] Pre-preparing querys

2014-05-20 Thread Roger Binns
utside bit was because of having to do UTF8 conversion from native string representation, and inside because statement preparation takes a while - it involves parsing, many memory allocations and lots of other fiddly stuff. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlN7rRM

Re: [sqlite] Pre-preparing querys

2014-05-19 Thread Roger Binns
have to have a copy of the SQL statement. SQLite already has to deal with all these issues, including keeping a copy of the statement so SQLite implementing the cache would save memory. Roger ___ sqlite-users mailing list sqlite-users@sqlite.org http:/

Re: [sqlite] Pre-preparing querys

2014-05-19 Thread Roger Binns
There is a ticket including pointers to previous mailing list discussion: http://www.sqlite.org/src/tktview?name=ee4b2b48f5 Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlN6dfEACgkQmOOfHg372QQmcACfX46ZvXwKt/Y9DnBL0BRxH9QM lFsAoLRh/hjZai4SC5ie/DCpd2+D3NA2 =1Ve9

Re: [sqlite] How do I know the python functions registered on SQLite ?

2014-05-08 Thread Roger Binns
m the public API. SQLite could provide the information as a virtual table or similar. A ticket was created 7 years ago asking for it, and closed 2 months ago by the team with resolution "Rejected": https://www.sqlite.org/src/tktview?name=5896edbe46 Roger -BEGIN PGP SIGNAT

Re: [sqlite] Most efficient storage for arrays

2014-04-22 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22/04/14 15:00, Neville Dastur wrote: > On 22 Apr 2014, at 21:58, Roger Binns wrote: >> Your data is from MongoDB :) Note they do have an extended JSON to >> deal with types like ObjectId, binary and dates: > Yes, it is.

Re: [sqlite] Most efficient storage for arrays

2014-04-22 Thread Roger Binns
tored. You'll have to write them to understand your data shape and queries. There was a now defunct project UnQL that was mixing together JSON like data, SQL like queries and bit of SQLite. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlNW19UACgkQmOOfHg372QTNvgCgltU2vNzzVuRfTuPd

Re: [sqlite] sqlite dump makes wrong "CREATE VIEW" order

2014-04-17 Thread Roger Binns
out dependency order. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlNP58cACgkQmOOfHg372QT0RwCg5gXqYYmQ0Ws1LqJaFBhdpvwJ dLAAn2+8hvf+nl9jTR45Z2TYQsT6mAlB =APvL -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-

Re: [sqlite] INSERT several rows

2014-04-03 Thread Roger Binns
erything up so Python being used was not visible (nor relevant) to the user. This approach worked fine on Windows, Linux and Mac. [1] distutils - a standard part of python - is used under the hood which has multiple subcommands each of which can be given flags. pip etc ultimately call into that.

Re: [sqlite] How good is pragma integrity_check

2014-03-23 Thread Roger Binns
nail down the cause for. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iEYEARECAAYFAlMvwWMACgkQmOOfHg372QSnOQCdEpBWBvcNsntkZ6WPvDs0yAju fc0AoJzagj56DyoYrhmeE73rwHhe+D2f =ZAfw -END PGP SIGNATURE- ___ sqlite-users m

Re: [sqlite] How good is pragma integrity_check

2014-03-21 Thread Roger Binns
the data, just the structure of the data. There was a feature request ticket for several years for checksums to at least catch unexpected changes to the data itself: https://www.sqlite.org/src/tktview?name=72b01a982a Sadly it was rejected a few weeks ago without explanation. Roger -BEG

Re: [sqlite] SQLITE_MAX_PAGE_COUNT

2014-03-05 Thread Roger Binns
levant code including any code that calls that code, to audit for overflows, to provide a new api and tests for that. And to ensure that stays maintained for the future lifetime of SQLite 3. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU

Re: [sqlite] Why would batched write operations NOT be faster than individual ones

2014-03-03 Thread Roger Binns
aspx You can mitigate it by having a setup that doesn't have failures such as using battery backup. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iEYEARECAAYFAlMUxBMACgkQmOOfHg372QTxewCgjuqKWh4m+pz2JRtQWznPA83o YEcAnjDuMMULpMX14VVlLsQ4NmJbD6PA =Dp0Y

Re: [sqlite] Compiling SQLite on VxWorks 6.3 (DKM)

2014-03-01 Thread Roger Binns
.@126.com http://catb.org/~esr/faqs/smart-questions.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iEUEARECAAYFAlMSPxoACgkQmOOfHg372QQXIQCfT7Sa9kempGXWTYs+L6aot98I Rw8AmJ2dp9jR0bN0FThp98ab/ZygeD0= =T0I3 -END PGP

Re: [sqlite] New

2014-02-28 Thread Roger Binns
d the files I would > greatly appreciate it I think the file ext. is a plist. Live, love & > laugh. In addition to those, read this: http://catb.org/~esr/faqs/smart-questions.html Roger -BEGIN PGP SIGNATURE- Vers

Re: [sqlite] Solving the Sudoku with SQlite 3.8.3 trunk

2014-02-06 Thread Roger Binns
actually have. It also supports very old versions of SQLite - I believe it will work correctly with 3.0.0! > If we wish to have SQLite + Python combination You'll find the python sqlite community at https://groups.google.com/forum/#!forum/python-sqlite Roger -BEGIN PGP SIGNATURE

Re: [sqlite] fsync on iOS

2014-01-16 Thread Roger Binns
with other files/directories/data in flight. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iEYEARECAAYFAlLYT00ACgkQmOOfHg372QTw3QCfQNoOjJHrs9+pTSNmAx5BCoXk essAoM0wmW454CL54bf0u2XTqV9vNxXP =DY1e -END PGP SIGNATURE- ___ sqlite-us

Re: [sqlite] Saw something interesting in the debugger...

2014-01-09 Thread Roger Binns
nularity of one second. The blocking thread could have finished after 10ms, but you'll still be stuck in the busy handlers for another 990ms. Simply ensure HAVE_USLEEP is defined when building sqlite3.c. Or add your own busy handler that sleeps for sub-second amounts of time. Roger

Re: [sqlite] Is that same between rebinding every parameter and

2013-12-30 Thread Roger Binns
n that SQLite should have a statement cache built in rather than everyone having to implement their own: https://www.sqlite.org/src/tktview?name=ee4b2b48f5 Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iEYEARECAAYFAlLB3aYACgkQmOOfHg372QSsFgCgwLkmzGqOrVL9YRLy6gG1

Re: [sqlite] Does not detect invalid column name when subexpression optimized away

2013-12-19 Thread Roger Binns
column: nonsense sqlite> select nonsense and 0; 0 Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iEYEARECAAYFAlKzo/8ACgkQmOOfHg372QTyVACfbNlMOQR5nxfzgUg7tY5pF8il d0sAnjIp9K/aAIFXZcbc+pZvDdK6tN0x =Siia -END PGP SIGNATURE- __

Re: [sqlite] "Last Modified" file attribute is not updated

2013-12-09 Thread Roger Binns
detect changes since you last looked at the file. You can use a trigger internally at the SQL level to track changes in a meaningful way too. Finally you can disable the use of memory mapping - have a look at the Windows VFS. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4

Re: [sqlite] Concrete example of corruption

2013-12-01 Thread Roger Binns
for many years to at least make detecting a mismatch between the database and a (possibly non-existent) journal possible: https://www.sqlite.org/src/tktview?name=61d35ac210 Roger ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:808

Re: [sqlite] Need suggestion for database scheama

2013-11-07 Thread Roger Binns
brainz http://musicbrainz.org/doc/MusicBrainz_Database/Schema Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlJ8ZWQACgkQmOOfHg372QQN/wCg0EyCpIs8pl9MVOzerfshJKxc JwAAoN+E6Wa8T3zWZ+r5ghLpeK1a8dem =23xE -END

Re: [sqlite] Android - Occasional Error when closing a database

2013-10-29 Thread Roger Binns
nd logCat shows the SQLITE_BUSY > error code. On Android I use the SQLiteDatabase class, and the insert method it provides and haven't encountered this issue. https://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html How are you inserting the data? Roger -BEGIN

Re: [sqlite] Write-ahead logging issue on Android

2013-10-07 Thread Roger Binns
id.com/reference/android/content/Context.html#getCacheDir() Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlJTb3YACgkQmOOfHg372QTbUwCguSVoQ7AcJrGCk7qwwhzFQOqQ z5YAoIFNQxhOTeP8/0Aq2lEpOD0Lc+GT =VUgP -END PGP SIGNATURE-

Re: [sqlite] Namespacing sqlite3

2013-09-24 Thread Roger Binns
API. http://apidoc.apsw.googlecode.com/hg/pysqlite.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlJCZOcACgkQmOOfHg372QRaYwCgqoR4C9DmXg27SCbOXg2a4GNe vyUAnRYFf8Uej21hrJTkhwXJWddkvYWi =bKyX -END PGP SIGNATURE- __

Re: [sqlite] Namespacing sqlite3

2013-09-24 Thread Roger Binns
(eg CoreData on Mac likes to do that). As another bonus it is also a bit faster too as the compiler ends up inlining SQLite code into your methods that call it. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlJCBoIACgkQmOOfHg372QRRsQCePzIDcnfY

Re: [sqlite] Compare Similar Cells

2013-09-22 Thread Roger Binns
you gave in your example, truncating to 30 characters, and whatever else is relevant for your data. You can now do matching against the normalised title column for each title. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iE

Re: [sqlite] Hints for the query planner

2013-09-12 Thread Roger Binns
SQLite go off and do whatever it deems necessary to gather to make the query perform well in the future. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIyQagACgkQmOOfHg372QT8jgCgtSROjcL1dyrHo+yP2leh1ffV xBEAoKEOTIVqz3vlrVrlVeJ130Wru/Mg =+8TU -

Re: [sqlite] help needed for major SQLite problem

2013-09-02 Thread Roger Binns
that pysqlite parses your SQL and tries to second guess what is going with transactions, and starts them behind your back. That is possibly causing problems for you too. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIk8J8ACgkQmOOfHg372QTYCQCgi79D27GRMc6Tgjc

Re: [sqlite] help needed for major SQLite problem

2013-09-01 Thread Roger Binns
elease. Note that the APSW API is similar to pysqlite, but not the same. It behaves the SQLite way rather than the DBAPI way. http://apidoc.apsw.googlecode.com/hg/pysqlite.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIj0vYACgkQmOOfH

Re: [sqlite] help needed for major SQLite problem

2013-09-01 Thread Roger Binns
t can be caused by other rare sequences of api calls. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIj0AIACgkQmOOfHg372QTU5QCfeGcle8r3m8nDs3dDgI7zb6BK qWQAoLWkeKarpubX4S1SzcBxaT5LT1yr =sjZY -END PGP SIGNATURE- _

Re: [sqlite] Path Length Limit on Windows

2013-08-23 Thread Roger Binns
bout this for several years. http://www.sqlite.org/src/tktview?name=c060923a54 Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIXpJkACgkQmOOfHg372QQDFQCgn56WYKUg3oxbiZ0Kp886flWa H+sAoLMWH6Q462K1Lb5pTs7WwlZyxlnf =Uj9e -END PGP

Re: [sqlite] Database locking without locking database file

2013-08-14 Thread Roger Binns
, which is why it turns out to be the least exercised part of protocols and where bugs/quirks lie. [1] http://www.rogerbinns.com/visionfs.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIMB3wACgkQmOOfHg372QTyGgCgxB/7XxUibw+p0T/aI0cNecE4 ZegAoMDoh7RKb6h

Re: [sqlite] sqlite3 error string size!!!

2013-08-12 Thread Roger Binns
tc can also generate arbitrary messages. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIIrLsACgkQmOOfHg372QS2pACeK5gt9bOdk9dX3gM+kxhYt6xA MxQAnRLb4yMHdNEV6RpV0MbFUQVteNJ/ =gJrr -END PGP SIGNATURE- ___ sqlite

Re: [sqlite] Slow Query on large database Help

2013-08-07 Thread Roger Binns
results of EXPLAIN QUERY from > sqliteman. Attachments get stripped from the mailing list. You can put them somewhere like Dropbox. Did you run analyze? Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlICmtMACgkQmOOfHg372QSRLwCgkb

Re: [sqlite] Guidance with Python and nested cursors

2013-07-17 Thread Roger Binns
thonic than the SQLite wrappers (disclosure: I'm the author of one of them). Code can use iterators and look something like this: for id,one,two in db.cursor().execute("select id, one, two from XXX"): three=one+two db.cursor().execute("insert into ", (three,

Re: [sqlite] function auxiliary data

2013-07-17 Thread Roger Binns
e you can immediately rerun the statement with the same bindings. However that isn't exposed in APSW so you can edit the code to clear the bindings after each reset call. It doesn't do that at the moment as a form of being lazy - only clearing bindings when the statement is re-executed. R

Re: [sqlite] function auxiliary data

2013-07-17 Thread Roger Binns
te3_finalize is only called when the statement is being destroyed which includes eviction from the statement cache, and on getting SQLITE_SCHEMA error. I do wish SQLite had a standard statement cache available. Pretty much all wrappers implement their own which is duplicate work and tricky to get right

Re: [sqlite] UnQLite

2013-07-07 Thread Roger Binns
ut would be ugly. It will be interesting to see who wins the query syntax mindshare. There does seem to be a desire to stay with SQL like syntax - for example Cassandra also uses it. Another thing I like about the Mongo syntax is that it is trivial to build up queries programmatically. Roger -BEGIN PG

Re: [sqlite] UnQLite

2013-07-07 Thread Roger Binns
data are substantially similar - it is very similar to Query By Example. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlHaGJwACgkQmOOfHg372QTXaQCgkiy49SE5bHyvRr8gn+Oecgve kSMAoLcq1ZXRg4Z5KXdU3HsfT9CkyqhE =J/3c -END PGP SIGNATURE- _

Re: [sqlite] BUG REPORT: function instr() with chinese characters

2013-07-06 Thread Roger Binns
???', '') as x from a; 1 sqlite> select instr('', '') as x from a; 3 sqlite> select instr('????', '') as x from a; 5 sqlite> select instr('', '') as x from a

Re: [sqlite] DLL Size differences and other info requested

2013-07-03 Thread Roger Binns
599k mark. You probably want -Os http://www.sqlite.org/footprint.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlHUyqQACgkQmOOfHg372QTkQACgkKeNo1Qk5G4sgdkLJZPKV/Ai 0nkAn34ULSWYCsNk6I5WHvDViQOeTRTJ

Re: [sqlite] Another 2 questions about SQLite

2013-06-30 Thread Roger Binns
QLite. It is already designed and implemented to do the least amount of work - you haven't found a way to do less because SQLite is already doing the least! It only calculates the next (or first) result row when you ask for it, and not before. Roger -BEGIN PGP SIGNATURE- Version

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Roger Binns
xtensions: http://msdn.microsoft.com/en-us/library/windows/desktop/aa378870(v=vs.85).aspx Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlHN+ugACgkQmOOfHg372QTTqgCeN3hNpHGON/CaoEx95y9605Qz GMIAnAtmuJzgo3wwLrZdGOIbA2yWwuRP =5SeP -

Re: [sqlite] Is there available sample "featured" SQLite database file?

2013-06-28 Thread Roger Binns
create table ""("" ""); If your code runs on a platform that is UTF-16 (or worse UCS-16) make sure you test with code points above 65535. For performance testing this could be helpful with the --dump-sql arg: https://code.google.com/p/apsw/source/browse/tools/

Re: [sqlite] Should I do analyze?

2013-06-26 Thread Roger Binns
predictable so the query solutions chosen tend that way. An upcoming release will have a new query planner that should work better when there are a lot of joins. http://www.sqlite.org/draft/queryplanner-ng.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4

Re: [sqlite] First Day of Week Inconsistency

2013-06-19 Thread Roger Binns
the latter. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlHB29kACgkQmOOfHg372QTxwQCfc3qGW5PtGljBm/rvgEOGVtA/ DzwAoI6cIiZ+SzMmX7NE2gcwEfDR0w2W =y1z6 -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

2013-06-16 Thread Roger Binns
e has not had any changes in two years. Also if you want to use SQLite and not pretend that it is the same as other databases then this will be of interest: http://apidoc.apsw.googlecode.com/hg/pysqlite.html (Disclosure: I am the author of APSW) Roger -BEGIN PGP SIGNATURE- Version: GnuP

Re: [sqlite] Prepared statement invariants

2013-06-14 Thread Roger Binns
;t used: http://apidoc.apsw.googlecode.com/hg/execution.html#apsw-trace There is a sqlite3_trace function but I don't use it because it expands bindings which means you can't lookup the queries in your code. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEAR

Re: [sqlite] Feature request: add support for COMMENT statement

2013-06-14 Thread Roger
Would this not work for you? http://stackoverflow.com/questions/7426205/sqlite-adding-comments-to-tables-and-columns .schema On 13-06-14 11:00 AM, Dave Wellman wrote: Thanks Clemens, that is probably a workable option (at least for me). As someone else noted, the PRAGMA user_version will no

Re: [sqlite] Limit of attached databases

2013-06-04 Thread Roger Binns
SQLite then decides to use the index you can then create it on the fly. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlGuQQMACgkQmOOfHg372QRxxwCfV1QvfO6Fsky0x3krTLe08+Nm EN4AniM6kkOHgcTm/mtREY3iD4QAF9o+ =

Re: [sqlite] Import skip first line

2013-06-01 Thread Roger Binns
ection phase. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlGqUEUACgkQmOOfHg372QRVXACdG1ZDSUzrL2TGMfaE1cQuSUgN 6vMAmQHwoWKQ+/zhAuZUNwM2U2Om4SAB =P6JF -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite

Re: [sqlite] Import skip first line

2013-05-31 Thread Roger Binns
r each column while the second pass actually imports the data. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlGpIoEACgkQmOOfHg372QRSqwCg3WTRwifRKC+tK55BmTbomVyo PHsAnR8v79nKkpVZ7WYWydeTYxsHbZoE =c95H -END PGP SIGNATURE- _

Re: [sqlite] How people test db failures

2013-05-02 Thread Roger Binns
even show up under pragma check. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlGDT0QACgkQmOOfHg372QSkKwCgysgQWSsCWgSN1zMS0p9uOOWb TL8An1MuVaYjysden5anmCnKk2G3+b5w =8HDa -END PGP SIGNATURE- ___ sqlite-users m

Re: [sqlite] How people test db failures

2013-05-02 Thread Roger Binns
an change bits in memory. Filesystems without checksums can have bit flips on the storage or in transmission. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlGDTf0ACgkQmOOfHg372QSuhgCfZaiQPow1ioBbZnez569/oVPt 3s0AoLdwcTPuEB8apk5Dv1VNpLS2bMXC =VYvk ---

Re: [sqlite] How people test db failures

2013-05-01 Thread Roger Binns
w.c#217 Here is the corresponding test: https://code.google.com/p/apsw/source/browse/tests.py#7562 (I use 1/0 to ensure that there will always be an exception, and if it turns out to be ZeroDivision then I know the previous line didn't cause one and should have.) Roger -BEGIN PGP S

Re: [sqlite] Writing in a blob

2013-04-22 Thread Roger Binns
This is discussed in: http://www.sqlite.org/c3ref/blob_open.html It mentions using zeroblob() so you can create the blob of the length you want and then use blob_open/read/write/close to modify the contents. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (

Re: [sqlite] SQLite version 3.7.16.2

2013-04-13 Thread Roger Binns
http://www.sqlite.org/howtocorrupt.html (section 2.1) Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlFp78AACgkQmOOfHg372QRcWQCfcPTuzBjSGNUeYR6xwQSFyvDe G8kAnim8px5ZTDow9KcbqebHBu3Nod/2 =llAx -END PGP SIGNATURE- _

Re: [sqlite] Sqlite in Google Summer of Code 2013

2013-04-12 Thread Roger Binns
The reason is because I can put arbitrary JSON objects in and get the same arbitrary JSON objects out. Queries have the same "shape" as the JSON objects (unlike the defunct unql which tries to use SQL). Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.

Re: [sqlite] Database modeling question

2013-04-10 Thread Roger Binns
ir mangled data and ids to make future matching easier - - almost every system like this ends up "intelligently" duplicating all the user records at some point. Think about prevention and how the user recovers (undo from earlier is a great help) Roger

[sqlite] Trivial doc issue for functions

2013-03-30 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 http://www.sqlite.org/lang_corefunc.html The hex line is out of alphabetical order and should be two lines earlier. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux

Re: [sqlite] Timeline for full ALTER TABLE support?

2013-03-27 Thread Roger Binns
ables, with triggers for insert to redirect columns as appropriate - - convince/pay someone to make a patch that implements what you need and maintain your fork forwards - - convince/pay the SQLite team to add it to the core, keeping in mind that they care about library size and the '

Re: [sqlite] Query optimizer suboptimal planning with virtual tables

2013-03-18 Thread Roger Binns
ows better. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlFGzgAACgkQmOOfHg372QSzoQCfWxH+ACpo3FbeYFli7DEZVGpO G+IAn2twe1Lva3sVE05jedQi/7VL3lpk =CFQH -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-us

Re: [sqlite] SQLite 3.7.16 beta

2013-03-12 Thread Roger Binns
te data. My test suite extensively covers the SQLite C API. Everything looks good and I don't even remember the last release there was a problem. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlE/2oUACgkQmOOfHg372QS3CwCfY/P+4IM

Re: [sqlite] Announcement of Copy - A cross platform syncing app, that uses SQLite as its backend

2013-03-01 Thread Roger Binns
SQLite thinks it wrote is not what is later returned: http://www.sqlite.org/src/tktview?name=72b01a982a Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlExZhYACgkQmOOfHg372QTLBgCgsbidy6oQfmAeS4OWq4OBSmFI zxEAn04lneghgvr+ww76AQWzycZ3x+Q

Re: [sqlite] Announcement of Copy - A cross platform syncing app, that uses SQLite as its backend

2013-03-01 Thread Roger Binns
ing TEXT, sync_status INTEGER, _natsort_name TEXT COLLATE NOCASE); CREATE TABLE pending_uploads (_id INTEGER PRIMARY KEY AUTOINCREMENT, class TEXT, data TEXT); dropbox._data is the path to a local copy of the file, not the file contents. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (G

Re: [sqlite] .dump - what isn't preserved?

2013-03-01 Thread Roger Binns
TLDR: good question and there are a *lot* of issues under the surface depending on how you intend to do the restore and what things matter to your code Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEwsFoACgkQmOOfHg372QSsTQCffwvaQFk1c/XAhZHJpPxCHflE 2PUAn1R

Re: [sqlite] Thread sync issue

2013-02-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/02/13 00:14, Roger Binns wrote: > On 23/02/13 21:23, Ashok Pitambar wrote: >> Sqlite doc says it thread safe ... > > http://www.sqlite.org/threadsafe.html > > What makes you think the doc is wrong? One big gotcha: er

Re: [sqlite] Thread sync issue

2013-02-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/02/13 21:23, Ashok Pitambar wrote: > Sqlite doc says it thread safe ... http://www.sqlite.org/threadsafe.html What makes you think the doc is wrong? Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAl

Re: [sqlite] Do not read File change counter

2013-02-21 Thread Roger Binns
up in the operating system cache and so perform at the speed of memory (no seeks, no disk). http://en.wikipedia.org/wiki/Amdahl's_law Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEmTxAACgkQmOOfHg372QSq/QCgu6iT0Y+XBpLyczh

Re: [sqlite] import TXT file

2013-02-21 Thread Roger Binns
append "';". (That is essentially what the .import command is doing behind the scenes, but it tries a lot harder to find separators, count/enforce number of columns, deal with end of line etc.) Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEAR

Re: [sqlite] "default value of column [name] is not constant." error in table creation when using double quoted string literal in parenthesis

2013-02-10 Thread Roger Binns
uot;. http://www.sqlite.org/src/info/25e09aa2ab Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEYS6EACgkQmOOfHg372QR+kgCePgJMHJ55coumXdEIsIOlOhJr Ns8An1bmLOFeJ/5vYMBa8nQJSw31MmLC =czW1 -END PGP SIGNATURE- ___ sqlite-

Re: [sqlite] using the same sqlite parameter more than once causes premature memory deallocation

2013-02-09 Thread Roger Binns
ck=full --leak-resolution=high --show-reachable=yes Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEWueoACgkQmOOfHg372QQtaQCeNfCzwMp2UzNslMjoI538tjBf 0SEAoNEfzvG70jqtWeY7T2LcVfjcFYAM =jmof -END PGP SIGNATURE- ___

Re: [sqlite] sqlite3 Python interface

2013-02-07 Thread Roger Binns
fault (disclosure I am the author). The recommended build instructions embed the SQLite amalgamation inside the extension so it doesn't affect anything else. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlET+T8ACgkQmOOfHg372QQZw

Re: [sqlite] Load a Sqlite Database on Disk as an In-memory Database

2013-02-05 Thread Roger Binns
tings but does work on every other modern operating system.) Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEQxrAACgkQmOOfHg372QTlLgCeLI2dZP/v6Pw4VlgcgRGUKqHB xcwAnjM1RAHNzGgvi9nnGnP4uoYG6vqV =DCOk -END

Re: [sqlite] Determine if an index has been created

2013-02-01 Thread Roger Binns
exist then you a row per column in the index. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlELmyUACgkQmOOfHg372QSkCwCgvIzw59DcPMytdRssRs37YPRB IUMAoKmyDMkKhdubaMqdmp/H9WLrkYAW =WCNM -END PGP SIGNATURE- ___

Re: [sqlite] SQLite3 Bug Report: The shell accepts some dot commands ending in semicolons while rejecting others.

2013-01-28 Thread Roger Binns
to the GIGO > principle is perfectly adequate. Note the change the SQLite team have made: http://www.sqlite.org/src/info/b4d94947fc Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEHZEMACgkQmOOfHg372QTUTgCeOLdm5gpZQDdE6WIWZHZ

Re: [sqlite] identify virtual tables?

2013-01-26 Thread Roger Binns
3(content /* ab */ TEXT) Of course this is an implementation detail and not specified anywhere, but it is good enough. Your like query has the problem that it will match rows where virtual appears anywhere. Roger -BEGIN PGP SIGNATURE- Version: G

Re: [sqlite] using the same sqlite parameter more than once causes premature memory deallocation

2013-01-26 Thread Roger Binns
. It looks like SQLite is the victim of some other memory mismanagement in your app and changing the SQL has just changed what code will fall victim to it. It is a very good idea to run valgrind before proceeding. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1

Re: [sqlite] Substring Search Across Many Columns

2013-01-25 Thread Roger Binns
#x27; 'hi ' 'hi t' 'i ' 'i t' 'i th' ' t' ' th' ' the' 'th' 'the' 'ther' 'he' 'her' 'here' 'er' 'ere' 're' You can possibly al

Re: [sqlite] Substring Search Across Many Columns

2013-01-25 Thread Roger Binns
OR desc LIKE "%red%" OR title LIKE > "%soup%"...' > > Will creating a composite index ... If you are concerned about performance then SQLite already has a solution for you: http://www.sqlite.org/fts3.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG

Re: [sqlite] using the same sqlite parameter more than once causes premature memory deallocation

2013-01-25 Thread Roger Binns
s often in powers of two. It could be that row had a string needing 33 bytes while others needed less which then caused memory to come out of a different bucket which then changes where the victim of the actual bug shows up. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linu

Re: [sqlite] using the same sqlite parameter more than once causes premature memory deallocation

2013-01-24 Thread Roger Binns
C/ARC and C. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlEBu6sACgkQmOOfHg372QSR5gCfU+FAhhp8QuzAB6q1sP8xq2T4 j3wAoMUx3i4dFzzIldgyGJghNtXmsJR6 =+wFc -END PGP SIGNATURE- ___ sqlite-users mailing list sql

Re: [sqlite] SQLITE starvation.

2013-01-17 Thread Roger Binns
usleep). { 1, 2, 5, 10, 15, 20, 25, 25, 25, 50, 50, 100 } Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlD4erkACgkQmOOfHg372QQWPwCfZlMikT1DnuiflZDa58pIdnD9 vwYAoKRI+Z5UQWl6NP6rm8K1eyrRxYht =vuw2 -END PGP SIG

Re: [sqlite] sqlite version of android 4.0.3

2013-01-09 Thread Roger Binns
:31 8609a15dfad23a7c5311b52617d5c4818c0b8d1e. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlDuIA0ACgkQmOOfHg372QS9yACfZFFwvjkqUhcDB0zXRiRzv8NK LTsAn2nTH/sVr8BAHHogRixfXUfeQkVQ =+1Ku -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-us

Re: [sqlite] Using sqlite in multi fork() without threads

2013-01-07 Thread Roger Binns
om/p/apsw/source/browse/src/apsw.c#704 In my benchmark tests I measured a slowdown of 1%. ie if your code did nothing but SQLite calls then you can expect it to be about 1% slower. I strongly recommend you do something like this to ensure that no developer accidentally has databases used across f

Re: [sqlite] 1.1GB database - 7.8 million records

2012-12-31 Thread Roger Binns
e which makes things slower. Journaling off definitely worked for me when I benchmarked it. I was working with a 15GB dump from postgres on Linux. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlDiABEACgkQmOOfHg372QQesQCgjTJbDb3Yt2iyC/7vUEJAFuTq 1T0AoIPDu/fpd

Re: [sqlite] 1.1GB database - 7.8 million records

2012-12-31 Thread Roger Binns
creating the initial database then you can turn journalling etc off until the database, indices etc are fully created. This will get you a little more speed too. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlDh7mkACgkQmOOfHg372QR/dwCfVhcMaYJIr6pTFKsL1LbaFiVJ

Re: [sqlite] How to verify referential integrity of SQLite database

2012-12-12 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/12 12:36, Richard Hipp wrote: > Does anybody know of any reason why we should not do this? I would love if this was combined with an optional per page checksum that detects corruption early. Roger -BEGIN PGP SIGNATURE- Vers

Re: [sqlite] table backup

2012-12-11 Thread Roger Binns
;ll probably find that quite hard. A simpler alternative is to temporarily attach another database and copy the table into it. attach 'backup.db' as backup; drop table if exists backup.important; create table backup.important as select * from important; detach backup; Roger -

Re: [sqlite] Unql

2012-12-10 Thread Roger Binns
s/Riak-Search---Querying/ MongoDB uses JSON with extra magic operators: http://docs.mongodb.org/manual/applications/read/ Personally I like MongoDB approach where queries strongly resemble the underlying stored data which means very little translation between the two. Roger -BEGIN PGP SIG

Re: [sqlite] table backup

2012-12-10 Thread Roger Binns
ble is to use a virtual table and do the backup during writes/sync. This is all considerably more work than figuring out why you are getting corruption in the first place. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlDGF2IACgkQmOOfHg372QTo9gCfSQQwreSvsa9lrV/

Re: [sqlite] table backup

2012-12-09 Thread Roger Binns
rom implementing undo by using triggers to save old values, through using a virtual table that backs up data somewhere else. The SQLite backup API only works on the whole database. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Li

Re: [sqlite] Bug: Segmentation fault in libsqlite3.so.0.8.6[b69a4000+ac000]

2012-12-04 Thread Roger Binns
C based process. Also if glib/evolution/libcamel play silly games with memory (eg using their own allocators and pools) then it will completely obscure what is going on from valgrind. You'll most likely get better insight in a group devoted to evolution. Roger -BEGIN PGP SIGNATURE

Re: [sqlite] list: Please allow signed messages.

2012-12-02 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/12/12 13:47, Paul Menzel wrote: > please allow signed messages to be sent to the list. > > I got: »The message's content type was not explicitly allowed« This works. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4

[sqlite] Deprecation of asyncvfs

2012-12-02 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The documentation also needs to say asyncvfs is out of favour, in particular on this page: http://www.sqlite.org/asyncvfs.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux

Re: [sqlite] Windows (slow) vs. iOS/OSX (fast) Performance

2012-11-30 Thread Roger Binns
27;t shouldn't using a current Windows 7/8 on a fresh machine today. Run msinfo32 and then Components > Storage > Disks to find the relevant partition and its starting offset/alignment. (This is unlikely to be your problem, but if present does result in the kind of performance degradation y

<    1   2   3   4   5   6   7   8   9   10   >