Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

2015-02-12 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/10/2015 07:03 PM, Jay Kreibich wrote: ... VFS is unlikely to make the cut. ... similar things about the xBestIndex() and xFilter() functions I haven't read the book, but one thing that may help is not using C for these. I think it is easier

Re: [sqlite] sqlite3 mailing list broken

2015-02-12 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/12/2015 05:15 AM, Cal Leeming wrote: I've been having a problem where iterdump() is exporting in a format which is unsuitable for MySQL, iterdump is part of pysqlite, and has no code from the official SQLite project. iterdump itself is just

Re: [sqlite] SQLCipher - Full Database Encryption for SQLite

2015-02-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/09/2015 02:54 AM, Dominique Devienne wrote: Adding PRAGMAs is not possible to a true SQLite extension I thought, It is however possible to add functions. eg encryption_mode could be added and called like this: select

Re: [sqlite] equivalent for JOIN LATERAL

2015-02-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/08/2015 03:32 PM, Simon Slavin wrote: For those like me who hadn't heard of it, here's a reference: Here is a presentation referenced Modern SQL in PostgreSQL, with title Still using Windows 3.1? So why stick to SQL-92? Lots of nice

Re: [sqlite] Interrupting custom collation processing

2015-02-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/07/2015 05:55 PM, Dan Ackroyd wrote: Due to the nature of PHP, it would be expected that the processing of the collation should stop immediately. However I can't see how to indicate to SQLite that an error has occurred, and so the function

Re: [sqlite] Possible to get table size (in bytes)?

2015-02-04 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/04/2015 10:26 AM, Rael Bauer wrote: Is it possible to get the information of how many bytes a table is taking up in the database? What is the underlying problem you are trying to solve with this information? There may be alternate

Re: [sqlite] Fsync or fdatasync performs slowly

2015-01-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/20/2015 01:12 AM, Wei, Catherine wrote: What's strange is that when I remove the disk, the difference still exists. It takes long time for fsync working with no disks, which has confused me for several days. What exactly do you mean by no

Re: [sqlite] Fsync or fdatasync performs slowly

2015-01-19 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/18/2015 06:49 PM, Wei, Catherine wrote: takes too much time when it executes fsync or fdatasync function. Note that although the documentation says only the file handle is synced, in practise many filesystems actually sync the whole filesystem

Re: [sqlite] sqlite3 tool bug

2015-01-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/17/2015 12:20 PM, Graham Holden wrote: I would echo this: it's good at mangling the command-line, but I've not been aware of it ever mangling data sent to a file/stream (other than the binary/text mode conversions). In text mode (the default

Re: [sqlite] sqlite3 tool bug

2015-01-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/16/2015 01:05 PM, Simon Slavin wrote: Why on earth would an operating system programmer bother to put any translation into piping You have a system with a bunch of apps installed. You then upgrade to a new version of the operating system and

Re: [sqlite] sqlite3 tool bug

2015-01-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/16/2015 02:35 PM, James K. Lowden wrote: I'm skeptical of the notion that cmd.exe is diddling with your data en route to the pipe. Almost certainly the reason is that stdout and stdin are in character mode. It requires extra code to put them

Re: [sqlite] damaged database recovery

2015-01-15 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/15/2015 12:52 PM, Dave Dyer wrote: Of course that's possible, but .dump produced what superficially appeared to be a perfectly consistent text file. Note that .dump writes the output and then on encountering problems attempts the table

Re: [sqlite] sqlite3_exec and returned error

2015-01-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2015 11:43 PM, Valery Reznic wrote: I was planing to use this function like following: --- int res; char *errmsg; I'd recommend you explicitly set that to NULL here. res = sqlite3_exec( db, zSql, NULL,

Re: [sqlite] VACUUM requires 6.7 times space ?

2015-01-05 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2015 10:25 AM, Jim Wilcoxson wrote: It's easy to do this in Python using iterdump(), Be aware that the pysqlite dump implementation is incomplete in many ways. Some issues: - - Doesn't dump in a transaction so will have irregularities if

Re: [sqlite] PHP: squelch warning and error messages

2015-01-05 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2015 04:53 PM, Lev wrote: I'm using the PHP bindings for SQLite3. How can I squelch error and warning messages? I do error checking, but the failing call emits the messages. Messages like: Warning: SQLite3::prepare(): Unable to

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/08/2014 05:36 PM, David Barrett wrote: *Re: a simple way is to sleep in the progress callback* -- Can you tell me more about this? Are you referring to the callback provided to sqlite3_exec(), or something else?

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/07/2014 04:43 PM, David Barrett wrote: so I'm curious if you can think of a way using the API (or any other way) to essentially nice the process by inserting a short sleep into whatever loop runs inside the VACUUM command. Using OS provided

Re: [sqlite] Table names length and content

2014-12-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/08/2014 10:30 AM, jose isaias cabrera wrote: Hmmm... what I am looking for it is not there. If the string length defined there is what defines the length of the name of a table, I am in business. :-) However, there is nothing about

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/08/2014 01:35 PM, Max Vlasov wrote: I wonder whether I/O sleeping possible in the first place. In this particular case the OP wants to vacuum while the machine is doing other I/O activity unrelated to the vacuum. Having more sleeping during

Re: [sqlite] Search for text in all tables

2014-12-05 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/05/2014 01:24 AM, Max Vlasov wrote: I once implemented a virtual table allvalues that outputs all database values with (hope self-explaining) fields TableName, TableRowId, FieldName, Value Could you expand on how you coped with the

Re: [sqlite] Search for text in all tables

2014-12-04 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/03/2014 11:44 PM, Baruch Burstein wrote: Is it possible to somehow search for/replace a string in all columns of all tables? (Disclosure: I am the APSW author) The APSW shell includes a .find command that does the searching bit. You also get

Re: [sqlite] Search for text in all tables

2014-12-04 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/04/2014 11:59 AM, Petite Abeille wrote: On Dec 4, 2014, at 8:44 AM, Baruch Burstein bmburst...@gmail.com wrote: Is it possible to somehow search for/replace a string in all columns of all tables? .dump | sed ’s/old/new/g' | .read ?

Re: [sqlite] Locking errors on network

2014-11-10 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/10/2014 09:41 AM, Mike McWhinney wrote: Please let know if there are any other solutions to this database locking problem as used on a network. Yes. Do not do it. See the FAQ: https://www.sqlite.org/faq.html#q5 Roger -BEGIN PGP

Re: [sqlite] Locking errors on network

2014-11-10 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/10/2014 10:22 AM, Mike McWhinney wrote: So SQLite shouldn't be used at all on a network? Aren't there any other provisions to handled the locking errors if/when they occur? Network filesystems do not implement locking and other operations

Re: [sqlite] Is sqlite thread-safety sufficient for use with Go language ?

2014-11-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/05/2014 02:05 PM, nicolas riesch wrote: This means that EACH SUCCESSIVE function in the sequence above can be processed on a DIFFERENT OS THREAD. That works just fine with SQLite, with one caveat. You should also make sure the wrapper itself

Re: [sqlite] sqlite3.c in a library - api rename

2014-10-29 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/28/2014 08:18 PM, Ward Willats wrote: I am using the amalgamation in a C++ library statically linked into other people's applications. Is there a way to namespace and/or macro and/or let C++ do its name-mangling thing to all the

Re: [sqlite] Will someone be able to explain this weird outcome...

2014-10-10 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/10/2014 01:18 PM, jose isaias cabrera wrote: I was able to figure out that comma's are more important than just a 1000 number delemeter, so I received the right answer by taking the commas out: To help avoid this in the future, be aware that

Re: [sqlite] 50% faster than 3.7.17

2014-09-24 Thread Roger Binns
On 24/09/14 06:19, Simon Slavin wrote: How much max is max ? Not giving up ACID. But for example stat4 is better than the default stat1. Memory mapping (especially on 64 bit) is great. So is WAL. All are off by default. If you want to give up ACID then you should really be on your own to

Re: [sqlite] 50% faster than 3.7.17

2014-09-23 Thread Roger Binns
On 22/09/14 10:48, Richard Hipp wrote: But if you have any new ideas on how we can further reduce the I/O, we'd love to hear from you. The single biggest problem for me is defaults. SQLite supports memory mapped i/o which has many advantages. The stat4 analyze does a really good job. WAL

Re: [sqlite] An order by problem, maybe a bug?

2014-09-19 Thread Roger Binns
On 19/09/14 07:58, James K. Lowden wrote: I wonder what problems you're talking about. Do you think the IRS, the Social Security Administration, the DMV, the passport agency, your birth certificate, and your local bank are just doing it wrong? You do realise there are more people in the US

Re: [sqlite] sqlite3 hangs

2014-09-19 Thread Roger Binns
On 19/09/14 17:40, Bokassa wrote: Hi all, I see my query hanging with this stack: dybagme-where #0 0x00332b00ee00 in __fsync_nocancel () from /lib64/libpthread.so.0 #1 0x0041b418 in full_fsync (fd=6, fullSync=0, dataOnly=0) at sqlite3.c:27735 ... The program itself is

Re: [sqlite] Automatic conversion between keywords and literals

2014-09-16 Thread Roger Binns
On 15/09/14 19:28, lchis...@paradise.net.nz wrote: I have been caught several times when a previously working piece of SQL has silently failed, due to a column or view change in the database not matched with a Delphi code change, an inadvertent character injection into a column name, or

Re: [sqlite] presentation about ordering and atomicity of filesystems

2014-09-13 Thread Roger Binns
On 12/09/14 17:07, Simon Slavin wrote: Programmers don't expect file services to support transactions because file services have never supported transactions. Ever hear of Windows and Transactional NTFS :-) http://msdn.microsoft.com/en-us/magazine/cc163388.aspx It turns out that adding

Re: [sqlite] Python - database disk image is malformed

2014-09-08 Thread Roger Binns
On 08/09/14 03:29, Andres Riancho wrote: In my project we use the database to store data during the process life, and then remove it when the process finishes. It sounds like what you could use temporary tables and let SQLite do the work for you. With this in mind, sync=OFF still looks like

Re: [sqlite] Python - database disk image is malformed

2014-09-08 Thread Roger Binns
On 08/09/14 03:49, Andres Riancho wrote: Off-list some guys contacted me and mentioned APSW [0], another wrapper around sqlite for python, and said that it might be worth giving it a try. Do you guys believe that a change in wrapper could improve my situation? Thanks! (Disclosure: I am the

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-08 Thread Roger Binns
On 08/09/14 05:35, Richard Hipp wrote: See the essay at: http://www.sqlite.org/affcase1.html Comments, criticism, and feedback are welcomed. BTW historically Microsoft used a file system for Office files before the XML stuff (ie even in the first versions from over 20 years ago). Back

Re: [sqlite] Can't figure out how to report a bug

2014-09-07 Thread Roger Binns
On 07/09/14 05:20, Joe Mucchiello wrote: So I'm posting it here. For the record, this wiki page explains how to report a SQLite bug (first google result too): https://www.sqlite.org/src/wiki?name=Bug+Reports Your issue is covered in the FAQ. Roger

Re: [sqlite] Request to change int parameter to size_t parameter / potential bug on iOS (64 bit)

2014-09-07 Thread Roger Binns
On 07/09/14 10:02, skywind mailing lists wrote: I have seen that SQLite uses normally parameters of type int to pass the size of a variable Correct. It should be using size_t or ssize_t, but the SQLite developers chose not to do that, especially as at the time of the decision those weren't

Re: [sqlite] Request to change int parameter to size_t parameter / potential bug on iOS (64 bit)

2014-09-07 Thread Roger Binns
On 07/09/14 11:19, Richard Hipp wrote: Please use a cast to silence the compiler warnings. (int)sizeof(...) instead of just sizeof(...). That isn't safe for correctly written 64 bit apps. For example they could end up with data items that are bigger than 2GB correctly using (s)size_t. The

Re: [sqlite] SQLITE_THREADSAFE question

2014-09-07 Thread Roger Binns
On 04/09/14 06:59, Neo Anderson wrote: I'm building a custom library wrapper for Cocoa. I want to handle database threading issue myself. One gotcha not documented in the threading pages is that the SQLite error handling APIs are not threadsafe, and the only correct way of handling errors with

Re: [sqlite] Python - database disk image is malformed

2014-09-07 Thread Roger Binns
On 07/09/14 19:11, Andres Riancho wrote: * I'm setting [4] PRAGMA synchronous=OFF for increased performance. Can this trigger malformed errors? Read the doc: https://sqlite.org/pragma.html#pragma_synchronous TLDR: yes To improve write performance use WAL:

Re: [sqlite] Cannot retrieve SQLite Db Data Immediately After Application Startup

2014-09-02 Thread Roger Binns
On 29/08/14 12:55, Bob Moran wrote: The return code (rc) is SQLITE_OK, but stmnt is NULL (0) if I start the application and wait for at least 1 minute, everything works. You get NULL back from prepare with SQLITE_OK if the statement doesn't do anything. Examples are empty strings or

Re: [sqlite] how best to determine # of rows in a table

2014-08-27 Thread Roger Binns
On 27/08/14 06:51, Mark Halegua wrote: I can do that in pysqlite, but I don't see a method for determining I'm at the end of the file The best way of thinking about how SQLite works internally is that it does the least amount of work to get you the next row of results. That also happens to

Re: [sqlite] Avoiding content duplication when extracting results

2014-08-13 Thread Roger Binns
On 13/08/14 06:25, Nathaniel Trellice wrote: Is there any way to avoid unnecessarily duplicating the data coming from the following functions? You will need to duplicate. You can use the memory allocation/copying scheme of your choice. I'm rather fond of the hierarchical/pool based ones like

Re: [sqlite] use sqlite3 in ios

2014-08-13 Thread Roger Binns
On 13/08/14 02:31, YAN HONG YE wrote: When I add sqlite3.c and sqlite3.h to xcode ios cocoa project, and compiled, the error msg is: Sqlite3 class is not a objective class, who have any cocoa sqlite source, I don't know how to do. iOS already includes SQLite as a shared library. Unless

Re: [sqlite] Views as Virtual Tables -- Command line vs. Called Interface

2014-08-03 Thread Roger Binns
Disclosure: I am the apsw author On 08/02/2014 10:19 AM, Jim Callahan wrote: I got apsw to work, but it had a curious side-effect -- it clobbered my IPython prompt (replaced prompt with smiley faces). APSW certainly didn't do that. It doesn't do anything - you have to make calls and get

Re: [sqlite] Handling Timezones

2014-07-30 Thread Roger Binns
On 29/07/14 17:23, Will Fong wrote: Ah! I have not explained my issue properly :) I'm very sorry about that. I'm using SQLite as a backend to a small website and I have users in multiple timezones. When users login, their timezone is retrieved from the user table. Why do you even need to

Re: [sqlite] Handling Timezones

2014-07-30 Thread Roger Binns
On 30/07/14 10:05, Nico Williams wrote: Users travel; they don't have a single timezone. What matters is: the TZ when a user posted / did something, so you can have a vague idea of when they might be sleeping / unavailable. I'm not sure if you are disagreeing or agreeing with me. A clearer

Re: [sqlite] Handling Timezones

2014-07-30 Thread Roger Binns
On 30/07/14 10:51, Nico Williams wrote: I find that somewhat obnoxious. I often prefer absolute time It depends on the content being shown. We go for human friendly relative times (eg 13 hours ago) and then have a tooltip that gives the full timestamp. Doing maths on times and dates is

Re: [sqlite] Vacuum command fails

2014-07-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/07/14 10:42, veeresh kumar wrote: When i execute the command PRAGMA temp_store, it returned me 0. What is the ideal value that needs to be set? A quick google search would have found the answer:

Re: [sqlite] Sometimes when my process restarts, it returns error database is locked

2014-07-15 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/07/14 22:39, Mayank Kumar (mayankum) wrote: The file system is ext3. I am calling this api sqlite3_config(SQLITE_CONFIG_LOG, errorLogCallback, NULL); [...] Is my understanding correct since my callback is not getting called ? Did you check

Re: [sqlite] Brief intro to SQLite in Python

2014-07-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/07/14 14:05, Simon Slavin wrote: But the examples of SQLite-via-Python code are clear and well written and may be useful for Python users who want to learn SQLite. Sadly it neglects one huge surprising area. The sqlite3 module shipped with

Re: [sqlite] SQLite JDBC column count

2014-07-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/07/14 22:51, Manoj wrote: Is there any workaround available for this? https://sqlite.org/limits.html#max_column Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlO8B14ACgkQmOOfHg372QRr6QCfZfbcFkz/lowVT8uBFy92FY/7

Re: [sqlite] Sqlite detect change in column

2014-07-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/07/14 23:22, Micka wrote: I know that there is a callback that can be called when a column is updated. Are you sure about that? SQLITE_API void *sqlite3_update_hook( [...] I tried to find when this function is called in the sqlite3.c .

Re: [sqlite] detect database/table/field use

2014-06-27 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 27/06/14 11:24, big stone wrote: I notice some of your examples are written in Python2-only syntax. Would it be possible to make their syntax more Python2/3 compatible, at least for the print function ? How does that help you?

Re: [sqlite] detect database/table/field use

2014-06-26 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/06/14 12:58, Nelson, Erik - 2 wrote: I'd like to record which databases/tables/fields are accessed. Is there any not-too-difficult way of doing this? The authorizer interface will address your issue. You can just record what it tells you,

Re: [sqlite] How to get access to SQLite Test Harness #3(TH3)

2014-06-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/06/14 23:09, Kishore Reddy wrote: a)I want to use sqlite library in my project software.In the website it is listed as TH3 achieves 100% branch test coverage. The SQLite software as released has already been tested with TH3 and passed. You

Re: [sqlite] Sequential numbers

2014-06-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/06/14 13:02, Dave Wellman wrote: I have some rows in a table (not very many, typically less than 20) and I want to generate a unique, sequential number for each row. http://www.sqlite.org/autoinc.html Roger -BEGIN PGP SIGNATURE-

Re: [sqlite] Suggestion for SHELL impovement (built-in scripting)

2014-06-18 Thread Roger Binns
On 06/18/2014 02:47 PM, to...@acm.org wrote: But you need bash, or TCL, or Perl, or Python, or whatever other than sqlite3.exe So, you're suggesting that an innocent SQLite user should install any of those programming packages just to run SQLite. Hmm... no, thanks! Yes. Quite simply you'll

Re: [sqlite] Creating a 'SQL text' Backup of a SQlite database with the mere sqlite.dll tool (and a Python 3)

2014-06-02 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/06/14 14:15, big stone wrote: No doubt that ASPW is much much better than SQLite3 standard library module. Just wanted to make sure you knew :-) Also you are welcome (and encouraged) to appropriate whatever pieces of APSW code would be

Re: [sqlite] Fwd: [fossil-users] DRH's PGCon 2014 Keynote (with Fossil sighting!)

2014-06-02 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There is a blog posting showing what a Postgres developer attending learned: http://use-the-index-luke.com/blog/2014-05/what-i-learned-about-sqlite-at-a-postgresql-conference There is also a link to the slides:

Re: [sqlite] Creating a 'SQL text' Backup of a SQlite database with the mere sqlite.dll tool (and a Python 3)

2014-06-01 Thread Roger Binns
On 06/01/2014 07:30 AM, big stone wrote: There is indeed an iterdump function in sqlite3 module, that I didn't notice. The one in APSW is far more thorough. If you just have some regular data tables then it won't make any difference. However if you are about correctness then be wary of the

Re: [sqlite] Reading compressed database files

2014-05-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/05/14 00:25, Hadashi, Rinat wrote: My databases are very big (almost 100 GB). I am looking for a compression solution. Did anyone have an experience with reading a compressed database? It would be helpful if you characterise your data and

Re: [sqlite] Reading compressed database files

2014-05-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/05/14 02:26, Hadashi, Rinat wrote: I have 13 tables, of which 2 are huge, 2 are medium and the rest are very small. My huge tables have 3 columns: numeric, numeric and varchar with millions of rows. I keep an index on the numeric columns.

Re: [sqlite] warning for partial indexes with X AND Y?

2014-05-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/05/14 05:26, Török Edwin wrote: Would it be possible to show a message when someone creates a useless index There was a ticket from two years requesting a lint mode to catch various issues that keep cropping up over the years:

Re: [sqlite] Pre-preparing querys

2014-05-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/05/14 21:09, James K. Lowden wrote: I took statement cache to mean that execution plans would persist either past sqlite3_exec() or that many plan would be kept, in case later useful, when sqlite3_step() recompiles according to passed

Re: [sqlite] Pre-preparing querys

2014-05-19 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/05/14 15:27, Richard Hipp wrote: The TCL interface for SQLite caches the N most recent prepared statements (where N defaults to 10 but is configurable) and reuses those prepared statements if the same queries are run again. That approach

Re: [sqlite] Pre-preparing querys

2014-05-19 Thread Roger Binns
On 05/19/2014 02:26 PM, Simon Slavin wrote: Are there any advantages to this other than increased speed ? Nope. However I've yet to see anyone complaining that SQLite is too fast and shouldn't be faster :-) I believe that on average an app using SQLite will have a distribution where a few

Re: [sqlite] How do I know the python functions registered on SQLite ?

2014-05-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/05/14 09:34, big stone wrote: Is there a way to get the list of all the 'external' functions created in a SQLite connexion ? This is only known internally within SQLite and there is no way to get at the information from the public API.

Re: [sqlite] Most efficient storage for arrays

2014-04-22 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22/04/14 05:24, Neville Dastur wrote: I am looking for some advice on storing and searching data that comes from an external JSON source and needs to be stored on device in a Sqlite3 database. Your data is from MongoDB :) Note they do have an

Re: [sqlite] Most efficient storage for arrays

2014-04-22 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22/04/14 15:00, Neville Dastur wrote: On 22 Apr 2014, at 21:58, Roger Binns rog...@rogerbinns.com wrote: Your data is from MongoDB :) Note they do have an extended JSON to deal with types like ObjectId, binary and dates: Yes, it is. But I

Re: [sqlite] sqlite dump makes wrong CREATE VIEW order

2014-04-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/04/14 00:43, Tyumentsev Alexander wrote: Is it the user responsibility to follow all dependencies and recreate VIEW tree ? The dumping is happening in the order that the views were created. This approach generally works, but fails in your

Re: [sqlite] INSERT several rows

2014-04-03 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/04/14 14:37, David King wrote: ... because of the author's opinions of the standard Python ways to require packages As said author, the problem is that pip etc authors chose to make it impossible to provide arguments to parts of the install

Re: [sqlite] How good is pragma integrity_check

2014-03-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21/03/14 15:24, Simon Slavin wrote: Checksums stored with the page index lists, SQLite already has the ability to carve out data on each page for other uses. For example the encryption extension uses this. Nevertheless, the basic SQLite engine

Re: [sqlite] How good is pragma integrity_check

2014-03-21 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 20/03/14 18:06, Simon Slavin wrote: All useful as far as SQLite itself goes, and better than nothing. Unfortunately, failing hard disks do weird things in weird orders. And the interaction between the physical hard disk and the on-board cache

Re: [sqlite] SQLITE_MAX_PAGE_COUNT

2014-03-05 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/03/14 10:59, Raheel Gupta wrote: If you point out to me the changes required I will do it and have it reviewed. The changes required are to update the test suites (there are several) to hit/cross the current limit, to modify all relevant

Re: [sqlite] Why would batched write operations NOT be faster than individual ones

2014-03-03 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/03/14 03:00, Simon Slavin wrote: What the heck ? Is this a particular implementation of RAID ... The technical term is write hole and can occur at many RAID levels: http://www.raid-recovery-guide.com/raid5-write-hole.aspx You can mitigate

Re: [sqlite] Compiling SQLite on VxWorks 6.3 (DKM)

2014-03-01 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/02/14 06:37, deltuo wrote: i compile sqlite 3.8.3 to vxworks 6.9, i first compile sqlite in dkm and get xx.a lib file, and then test it in vip project, but meet disk i/o error, can you help me ? thank you , my email is del...@126.com

Re: [sqlite] New

2014-02-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/02/14 16:54, Ashleigh wrote: I'm trying to view files from my iphone backup I'm not sure which program it is it says sqlite it is a black box like the windows command If any one knows a better way to read and understand the files I would

Re: [sqlite] Solving the Sudoku with SQlite 3.8.3 trunk

2014-02-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/02/14 23:15, big stone wrote: APSW looks indeed great for specialised installations. The intention behind APSW is a Python wrapper for SQLite3. It does everything the SQLite way where applicable. It advances with SQLite meaning new versions

Re: [sqlite] fsync on iOS

2014-01-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16/01/14 11:43, Ward Willats wrote: So it looks like fsync() is taking more than the 5 second timeout I've set. This is not uncommon on mobile devices using flash based storage. There is a lot of volatility in read and write performance. I

Re: [sqlite] Saw something interesting in the debugger...

2014-01-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/01/14 08:53, Ward Willats wrote: I found the UI thread and a worker thread, both in the DB, both in the default busy handler, both taking a 1 second sleep. I expected to see a third thread in the DB doing some work while the other two

Re: [sqlite] Is that same between rebinding every parameter and

2013-12-30 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30/12/13 06:18, Igor Tandetnik wrote: sqlite3_clear_binding is very rarely needed, in my experience. In fact, I have not yet encountered a reason to use it. I use it in my Python wrapper (APSW). The reason is because I have an automatic

Re: [sqlite] Does not detect invalid column name when subexpression optimized away

2013-12-19 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/12/13 15:36, RSmith wrote: With this query you essentially ask the RDBMS to evaluate and supply you with the result of (X and 0) - my guess is the optimiser pounces directly on the fact that (X and 0) will always be 0 no matter what X is so

Re: [sqlite] Last Modified file attribute is not updated

2013-12-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/12/13 10:30, Felipe Farinon wrote: I'm sorry to repost, but I just want to confirm that there is no interest in fixing this, so that I can handle this with a workaround in my application. Note that the effect will be to make SQLite slower

Re: [sqlite] Concrete example of corruption

2013-12-01 Thread Roger Binns
On 01/12/13 06:10, L. Wood wrote: D. Richard Hipp, are there any plans to make this more robust in the future, so reading/writing a corrupt database (with no -journal file available) will return an *error* instead of causing further damage? There has been a ticket languishing for many years

Re: [sqlite] Need suggestion for database scheama

2013-11-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/11/13 19:47, James K. Lowden wrote: You might guess from my email domain name that I take an interest in posts like yours. And it's pretty good first cut, no pun intended. ;-) It is also worthwhile looking at musicbrainz

Re: [sqlite] Android - Occasional Error when closing a database

2013-10-29 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 29/10/13 06:23, Matthew Dumbleton wrote: I have noticed that occasionally, after the data is entered and the transaction ended, the close method call produces an error 'unable to close due to unfinalised statements' and logCat shows the

Re: [sqlite] Write-ahead logging issue on Android

2013-10-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/10/13 18:37, Sascha Sertel wrote: I have since tried to find out what the correct way is to point SQLite to the right place for creating temporary files in Android, with no luck.

Re: [sqlite] Namespacing sqlite3

2013-09-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/09/13 12:56, Neville Dastur wrote: Searching around on Google it seems that namespacing in c / obj-c is not possible for the sqlite3 library. There is another approach that I use. I produce a Python C extension. The final shared library with

Re: [sqlite] Namespacing sqlite3

2013-09-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/09/13 16:37, Neville Dastur wrote: That's a very interesting concept. I've been doing it since 2004, and support all versions of Python from 2.3 onwards including 3.x. Are you using http://www.cython.org/ or something else to create the

Re: [sqlite] Compare Similar Cells

2013-09-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22/09/13 19:41, Kristopher Roy wrote: I have a table of songs, several have similar titles I can't find where to get started. I tried this but its not right. Select SongTitle, COUNT(SongTitle) AS LIKE_COUNT FROM Songs I did work with a database

Re: [sqlite] Hints for the query planner

2013-09-12 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/09/13 05:03, Gabor Grothendieck wrote: Perhaps indexing the expression in question would be an alternative that would keep the performance info separate from the select. I'd rather just do 'ANALYZE query' and have SQLite go off and do

Re: [sqlite] help needed for major SQLite problem

2013-09-02 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/09/13 08:58, C M wrote: If you think APSW would provide more details about just what went wrong other than SQLite logic error or missing database (which, I feel, doesn't tell me much at all), then I could potentially try it. I'm in the

Re: [sqlite] help needed for major SQLite problem

2013-09-01 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/09/13 14:34, C M wrote: Do you know how I can do that with Python? For example, I tried this: status = cursor.execute(some SQL statement here) print The status is: , status But it prints the cursor object: The status is

Re: [sqlite] help needed for major SQLite problem

2013-09-01 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 He needs to give the exception traceback which will show what is happening at the time. A common mistake with newish Python programmers is to catch all exceptions, and then keep going which also hides the exception tracebacks. On 01/09/13 15:12,

Re: [sqlite] Path Length Limit on Windows

2013-08-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22/08/13 07:38, Markus Schaber wrote: http://www.mail-archive.com/sqlite-users@sqlite.org/msg18707.html seems to indicate that SQLite is not using those APIs. On the other hand, that post is of 2006. There has been an open ticket about this for

Re: [sqlite] Database locking without locking database file

2013-08-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/08/13 06:06, Ralf Ramsauer wrote: Neither cifs. I worked on a CIFS server (visionfs)[1]. They are a convoluted complicated mess. During the OLE2 era, Microsoft's apps abused locking as a means of inter-process communication. It got very

Re: [sqlite] sqlite3 error string size!!!

2013-08-12 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/08/13 01:45, techi eth wrote: What is the maximum size error string returned by sqlite3_errmsg() function You need to make a copy of the string since it can be changed at any time, including by other threads. In practise it will usually be

Re: [sqlite] Slow Query on large database Help

2013-08-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/08/13 11:54, Christopher W. Steenwyk wrote: I have been working on a large database and its queries now for several weeks and just can't figure out why my query is so slow. I've attached the schema, my query, and the results of EXPLAIN QUERY

  1   2   3   4   5   6   7   8   9   10   >