Re: [sqlite] Whish List for 2015

2014-12-23 Thread Nico Williams
On Tue, Dec 23, 2014 at 05:26:00PM +0200, Tony Papadimitriou wrote: Proposal: Minimal stored procedure functionality -- or, maybe call it stored It's already there. It's called TRIGGER. I use triggers to hold stored procedures, and WHERE conditions on DMLs in the trigger body as

[sqlite] ALTER TABLE .. ADD COLUMN .. IF NOT EXISTS ..?

2014-12-16 Thread Nico Williams
I have a habit of putting schema definitions in a file that's always safe to read and execute against a DB connection. This means that I DROP some things IF EXISTS and CREATE all things IF NOT EXISTS. But if I have to ALTER TABLE... there's no IF NOT EXISTS .. equivalent for ALTER TABLE. Funny

Re: [sqlite] ALTER TABLE .. ADD COLUMN .. IF NOT EXISTS ..?

2014-12-16 Thread Nico Williams
On Tue, Dec 16, 2014 at 11:40:22PM +, Simon Slavin wrote: If 'ALTER TABLE ... ADD COLUMN ...' fails it fails harmlessly, with But it doesn't fail so harmlessly: $ sqlite3 db 'alter table toy add column foo text; select 5;' || echo fail SQL Error: duplicate column name: foo fail $ Note

Re: [sqlite] Query Planner for Virtual Tables: link table evaluation transitive property of constraints not used

2014-12-15 Thread Nico Williams
On Mon, Dec 15, 2014 at 06:23:31PM +0700, Dan Kennedy wrote: It's tricky. As you say, xBestIndex() will currently be invoked twice - once with no constraints usable and once with both b.id=? and b.linkid=? usable. I guess the reason it is not invoked in the other ways you suggest is that that

Re: [sqlite] sqlite bugreport : unique index causes valid updates to fail

2014-12-08 Thread Nico Williams
Ideally there would be something like DEFERRED foreign key checking for uniqueness constraints... You can get something like that by using non-unique indexes (but there would also go your primary keys) and then check that there are no duplicates before you COMMIT. (Doing this reliably would

Re: [sqlite] sqlite bugreport : unique index causes valid updates to fail

2014-12-08 Thread Nico Williams
On Mon, Dec 8, 2014 at 9:01 AM, Dan Kennedy danielk1...@gmail.com wrote: You could hack SQLite to do enforce unique constraints the same way as FKs. When adding an entry to a UNIQUE index b-tree, you check for a duplicate. If one exists, increment a counter. Do the opposite when removing

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Nico Williams
On Sun, Nov 23, 2014 at 4:26 PM, James K. Lowden jklow...@schemamania.org wrote: Metadata updates to Posix filesystems are seen as so costly that fsync(2) on the datafile descriptor doesn't update them. A separate sync on the directory is required. Compared to an in-memory update (of

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Nico Williams
My advice is to borrow from other clustered filesystems' experience. If you want to adhere to POSIX semantics then st_mtime and st_size visibility will be a particular headache, especially since you don't know when it's OK to lie (i.e., which callers of stat() are using st_mtime/st_size for

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Nico Williams
BTW, the experience with dedup is that doing things off-line means never catching up, while doing them online means going slow. You might cache as much as you can in memory then go slow when you miss the cache... In practice I think it's best to separate data and metadata devices so that you can

Re: [sqlite] SQLite as a meta database

2014-11-05 Thread Nico Williams
IIRC there's a GNOME interface to databases, libgda, that works this way. The fundamental problem with using SQLite3 virtual tables to access remote/high-latency resources is that the SQLite3 VM and the generated bytecode are very serial, therefore the VM can't make progress on other parts of a

Re: [sqlite] Ordering of group_concat values using subselect in 3.8.7

2014-10-28 Thread Nico Williams
On Tue, Oct 28, 2014 at 2:33 PM, James Earl ja...@truckhardware.ca wrote: After upgrading from SQLite 3.8.6 to 3.8.7, I am no longer able to order group_concat values by using a subselect. For example the following query with 3.8.6 will give me an ordered string of items.image values based on

Re: [sqlite] Circular References [Was: Re: [RFE bug] Improve error reporting for foreign keys]

2014-10-27 Thread Nico Williams
On Mon, Oct 27, 2014 at 4:19 AM, Simon Slavin slav...@bigfraud.org wrote: A foreign key which points to the same table had not occurred to me, outside of abstract mathematical exercises. I think you've come up with an acceptable example of why I felt reluctant to condemn the practise. If

Re: [sqlite] quasi-bug related to locking, and attached databases....

2014-10-27 Thread Nico Williams
On Mon, Oct 27, 2014 at 3:23 AM, Hick Gunter h...@scigames.at wrote: SQLite treats each attached database as a separate entity. Attaching the same file twice is just asking for problems. Well, it could do something to detect duplicates, but it may not be easy (or even possible) to portably

Re: [sqlite] quasi-bug related to locking, and attached databases....

2014-10-27 Thread Nico Williams
On Mon, Oct 27, 2014 at 3:17 PM, Stephan Beal sgb...@googlemail.com wrote: That's conceptually the same problem we're trying to solve here: keep the public db names stable, regardless of where/how they're actually attached. Yes, I think that's desirable. If it's not too much to ask for then

Re: [sqlite] quasi-bug related to locking, and attached databases....

2014-10-27 Thread Nico Williams
On Mon, Oct 27, 2014 at 5:12 PM, Simon Slavin slav...@bigfraud.org wrote: On 27 Oct 2014, at 9:49pm, Nico Williams n...@cryptonector.com wrote: If it's not too much to ask for then SQLite3 ought to: a) check for duplicates by canonicalized path (but keep in mind that this can be difficult

Re: [sqlite] JSON indexing

2014-10-22 Thread Nico Williams
On Tue, Oct 21, 2014 at 4:01 PM, Mike Jarmy mja...@gmail.com wrote: something like that seems like its probably a necessary first step to being able to build a JSON index on disk. To be fair, there are other ways it could be done. But I would prefer a mechanism that allows for external

Re: [sqlite] Index on expressions

2014-10-22 Thread Nico Williams
On Wed, Oct 22, 2014 at 3:42 PM, Philippe Riand p...@riand.com wrote: Hello, If there any plan to get this in? (see: http://sqlite.org:8080/cgi-bin/mailman/private/sqlite-users/2014-September/055065.html ) You'll notice that the SQLite3 developers don't often discuss what features they will

Re: [sqlite] JSON indexing

2014-10-21 Thread Nico Williams
On Tue, Oct 21, 2014 at 10:14 AM, Dominique Devienne ddevie...@gmail.com wrote: On Tue, Oct 21, 2014 at 3:38 PM, Mike Jarmy mja...@gmail.com wrote: Has there been any discussion of adding JSON indexing to sqlite, similar to the way Full Text Search and R-Tree are available? Postgres 9.4 beta

Re: [sqlite] JSON indexing

2014-10-21 Thread Nico Williams
On Tue, Oct 21, 2014 at 2:13 PM, Mike Jarmy mja...@gmail.com wrote: I could probably use an Entity-Attribute-Value approach to modeling JSON Yes. I can sort of work around needing to query embedded data structures like lists and json objects, so EAV would more or less work for me. Sqlite's

Re: [sqlite] Stored Procedures

2014-10-09 Thread Nico Williams
I use triggers, particularly INSTEAD OF INSERT triggers on VIEWs. The values of the columns of the rows to be inserted are the stored procedure's arguments. I use WHERE clauses judiciously to make up for the lack of IFs. It works well enough. I've used this in combination with recursive

Re: [sqlite] Detecting multiple CHECK failures

2014-10-08 Thread Nico Williams
On Wed, Oct 8, 2014 at 12:23 PM, Peter Haworth p...@lcsql.com wrote: Actually, I do get the CHECK constraint name returned to me in the error message otherwise, as you say, it would be impossible to find out what failed. I use a translation table in my application to reformat the SQLite error

Re: [sqlite] Detecting multiple CHECK failures

2014-10-08 Thread Nico Williams
On Wed, Oct 8, 2014 at 8:01 AM, Clemens Ladisch clem...@ladisch.de wrote: Peter Haworth wrote: I'm a great believer in using CHECK constraints to do as much validation as possible within the database rather than code it in my application. However, I think I'm right in saying that as soon as a

Re: [sqlite] I'm trying to figure out how to ...

2014-09-18 Thread Nico Williams
If you do a select with the an ORDER BY ... DESC (or ASC) LIMIT 1, with the order-by clause matching one of the indexes on that table, then you'll get the last row without having to use a rowid. For example: CREATE TABLE person (firstname TEXT, lastname TEXT, stuff TEXT, PRIMARY KEY (lastname,

Re: [sqlite] presentation about ordering and atomicity of filesystems

2014-09-15 Thread Nico Williams
On Fri, Sep 12, 2014 at 6:47 PM, James K. Lowden jklow...@schemamania.org wrote: On Fri, 12 Sep 2014 19:38:53 +0100 Simon Slavin slav...@bigfraud.org wrote: I don't think it can be done by trying to build it on top of an existing file system. I think we need a file system (volume format,

Re: [sqlite] presentation about ordering and atomicity of filesystems

2014-09-15 Thread Nico Williams
On Fri, Sep 12, 2014 at 7:21 PM, Richard Hipp d...@sqlite.org wrote: On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin slav...@bigfraud.org wrote: one thing that annoys me about SQLite is that it needs to make a journal file which isn't part of the database file. Why ? Why can't it just write

Re: [sqlite] presentation about ordering and atomicity of filesystems

2014-09-15 Thread Nico Williams
On Sat, Sep 13, 2014 at 10:39 AM, Richard Hipp d...@sqlite.org wrote: I say that a filesystem is an eventually-consistent key/value database. Yes! The keys are the filenames and the values are all big BLOBs, specifically the file content. Filesystems also have a hierarchical keyspace, which

Re: [sqlite] presentation about ordering and atomicity of filesystems

2014-09-12 Thread Nico Williams
On ZFS datasets with sync disabled fsync() functions as osync(), as a write barrier without durability and without the associated penalty. The obvious problem is that really do need osync() and fsync(); just one or the other is not a reasonable compromise. Nico --

Re: [sqlite] presentation about ordering and atomicity of filesystems

2014-09-12 Thread Nico Williams
On Fri, Sep 12, 2014 at 1:18 PM, Howard Chu h...@symas.com wrote: Nico Williams wrote: On ZFS datasets with sync disabled fsync() functions as osync(), as a write barrier without durability and without the associated penalty. The obvious problem is that really do need osync() and fsync

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-09 Thread Nico Williams
On Mon, Sep 8, 2014 at 11:46 PM, Stephan Beal sgb...@googlemail.com wrote: FWIW: http://fossil.wanderinghorse.net/wikis/cson/?page=cson_sqlite3 Yup, I've written that sort of wrapper in Python too. Of course, in the preceding sub-thread I'm talking about something rather different, but

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-08 Thread Nico Williams
And now that SQLite3 has recursive queries, building a mapping of XML-SQL is relatively easy, which might make it even easier to switch to SQLite3. (Speaking of which, a XPath to SQL compiler would be really nice. I haven't sat down to think about whether that'd be feasible, but my impression is

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-08 Thread Nico Williams
It'd also be good to have JSON and XML support, possibly as an loadable extension. That way XPath and similar expressions matching document snippets in SQL string values could be used in SQL queries. I've played with building a JSON extension for SQLite3 using jq's excellent JSON C library. The

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-08 Thread Nico Williams
On Mon, Sep 8, 2014 at 6:16 PM, Simon Slavin slav...@bigfraud.org wrote: On 8 Sep 2014, at 11:45pm, Nico Williams n...@cryptonector.com wrote: It'd also be good to have JSON and XML support, possibly as an loadable extension. That way XPath and similar expressions matching document snippets

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-08 Thread Nico Williams
On Mon, Sep 8, 2014 at 8:05 PM, John McKown john.archie.mck...@gmail.com wrote: Hum, why not a URI data type instead? ref: Because we're talking about a purely internal type, with internal linkage. Externally it would appear as TEXT or BLOB. You'd use the TEXT type to store JSON, XML, ...,

Re: [sqlite] What if OpenDocument were built atop SQLite?

2014-09-08 Thread Nico Williams
On Mon, Sep 8, 2014 at 9:44 PM, Keith Medcalf kmedc...@dessus.com wrote: Not really very difficult, and it does not require another type. You just need a scalar function to process the URI passed to the function and return the result -- just like the readfile() (in the fileio.c extension,

Re: [sqlite] Index on expressions

2014-09-04 Thread Nico Williams
On Thu, Sep 4, 2014 at 3:59 AM, Dominique Devienne ddevie...@gmail.com wrote: On Thu, Sep 4, 2014 at 2:04 AM, Nico Williams n...@cryptonector.com wrote: [...] but there's a gotcha: SELECT * on a table source with computed columns might result in much more work being done than the user might

Re: [sqlite] Index on expressions

2014-09-03 Thread Nico Williams
On Wed, Sep 3, 2014 at 6:16 PM, Simon Slavin slav...@bigfraud.org wrote: I'm used to calling them 'computed columns' but yes, they should be relatively easy to implement, as long as users accept a bunch of restructions on what they can refer to, roughly equivalent to the restrictions on what

Re: [sqlite] Index on expressions

2014-09-03 Thread Nico Williams
Also, of course, MERGE is very convenient, syntactically and semantically. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Handling Timezones

2014-07-30 Thread Nico Williams
On Wed, Jul 30, 2014 at 11:53 AM, Roger Binns rog...@rogerbinns.com wrote: Why do you even need to store their timezone? The only time it would matter is if you are showing one user what another users local time is. Users travel; they don't have a single timezone. What matters is: the TZ when

Re: [sqlite] Handling Timezones

2014-07-30 Thread Nico Williams
On Wed, Jul 30, 2014 at 12:34 PM, Roger Binns rog...@rogerbinns.com wrote: On 30/07/14 10:05, Nico Williams wrote: Users travel; they don't have a single timezone. What matters is: the TZ when a user posted / did something, so you can have a vague idea of when they might be sleeping

Re: [sqlite] ISO time leap second.

2014-07-29 Thread Nico Williams
On Tue, Jul 29, 2014 at 10:02 AM, Simon Slavin slav...@bigfraud.org wrote: You're never going to get non-scientific programmers to do this properly anyway. Every financial programmer knows that there are exactly 60*60*24 = 86,400 seconds in a day. You've never going to get them to use

Re: [sqlite] Handling Timezones

2014-07-29 Thread Nico Williams
You can haz per-connection TZ setting: use a temp table and join with it. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] ISO time leap second.

2014-07-28 Thread Nico Williams
On Mon, Jul 28, 2014 at 12:20 PM, Keith Medcalf kmedc...@dessus.com wrote: It may or may not be a valid timestamp depending on what your time representation is. SQLite does not use UTC (which is an artificial timescale maintained by a bunch of atomic clocks). SQLite (and most other things

Re: [sqlite] ISO time leap second.

2014-07-28 Thread Nico Williams
On Mon, Jul 28, 2014 at 7:54 PM, James K. Lowden jklow...@schemamania.org wrote: On Mon, 28 Jul 2014 16:32:42 -0500 Nico Williams n...@cryptonector.com wrote: Raising errors is not that useful here: most users who ever run into such timestamps will not really know what to do with them. I

Re: [sqlite] Suggestion for SHELL impovement (built-in scripting)

2014-06-18 Thread Nico Williams
My suggestion is to have a sqlite_... table in which to start statements to run at DB open time, so as to: - automatically CREATE temp tables, indexes, views - automatically ATTACH related DBs - automatically load extensions (this should require explicit acquiescence from the API caller

Re: [sqlite] Building multiple indexes in one scan over a table

2014-06-10 Thread Nico Williams
On Tue, Jun 10, 2014 at 4:25 PM, Simon Slavin slav...@bigfraud.org wrote: On 10 Jun 2014, at 7:22pm, Fabian Giesen fabi...@radgametools.com wrote: Not possible in current SQLite. No standard syntax for doing it in SQL. A quick google suggests that no other popular DBMS implements it. If the

Re: [sqlite] DELETE INSERT vs. REPLACE

2014-05-29 Thread Nico Williams
I tend to use an UPDATE ... WHERE EXISTS ...; followed by an INSERT ... WHERE NOT EXISTS ...;, which gives me insert or update semantics. Nico -- ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] DELETE INSERT vs. REPLACE

2014-05-29 Thread Nico Williams
On May 29, 2014 4:59 PM, J Decker d3c...@gmail.com wrote: If you have constraints between your tables (like cascade on delete) then deleting a guitar entry would remove its related entirs in other tables referencing that same ID. That's what deferred foreign key constraints are for, iiuc. But

Re: [sqlite] very slow fdsync() calls

2014-03-19 Thread Nico Williams
On Wed, Mar 19, 2014 at 8:03 PM, Jono Poff jonathan.p...@taitradio.com wrote: On 12/03/2014 12:01 p.m., Nico Williams wrote: ZFS recordsize Hi Nico, I have one (or two) last question(s)... All I need to do is prevent a single process ('process A') from being time bound by making fsync

Re: [sqlite] very slow fdsync() calls

2014-03-11 Thread Nico Williams
On Mon, Mar 10, 2014 at 7:01 PM, Jono Poff jonathan.p...@taitradio.com wrote: I'm using Solaris 10 with zfs and I may be able to disable write caching on the entire disk, but would prefer not to! Any ideas appreciated. The way ZFS works, if you don't have a ZIL (ZFS Intent Log) device then all

Re: [sqlite] Still trying to track down loadable extensions

2014-02-20 Thread Nico Williams
On Thu, Feb 20, 2014 at 5:19 PM, Peter Haworth p...@lcsql.com wrote: I know nothing about git so can someone please point me in the right direction to find these loadable extensions. SQLite3 is version controlled with Fossil anyways, not git... Besides the ones that SQLite3 comes with (see

Re: [sqlite] Proposed enhancement to the sqlite3.exe command-line shell

2014-02-11 Thread Nico Williams
On Mon, Feb 10, 2014 at 10:30 PM, Simon Slavin slav...@bigfraud.org wrote: On 11 Feb 2014, at 4:06am, James K. Lowden jklow...@schemamania.org wrote: ${HOME}/.sqlite/db would be my choice. Since the objective is not to let a naive user unexpectedly lose the data, it might seem a bad idea to

Re: [sqlite] Memory-mapped I/O [was: SQLite 3.7.17 preview - 2x faster?]

2014-02-10 Thread Nico Williams
On Sun, Feb 9, 2014 at 5:03 PM, Richard Hipp d...@sqlite.org wrote: On Sun, Feb 9, 2014 at 5:49 PM, James K. Lowden jklow...@schemamania.orgwrote: I suspect that adding msync() calls would wipe out any speed advantage for using memory-mapped I/O. And since speed is the only advantage to

Re: [sqlite] Memory-mapped I/O [was: SQLite 3.7.17 preview - 2x faster?]

2014-02-10 Thread Nico Williams
On Sat, Feb 8, 2014 at 7:26 AM, Richard Hipp d...@sqlite.org wrote: OpenBSD lacks a coherent filesystem cache. That is to say, changes to a file made using write() are not necessarily reflected in mmap-ed memory right away. And change to a mmap-ed segment are not necessarily reflected in

Re: [sqlite] Proposed enhancement to the sqlite3.exe command-line shell

2014-02-10 Thread Nico Williams
On Mon, Feb 10, 2014 at 10:06 PM, James K. Lowden jklow...@schemamania.org wrote: James proposes that when no DB is named on the command-line argument list then a [user-specific] default be used, and that the user be told. I like it. I admit that I often rely on the shell keeping no state when

Re: [sqlite] Common Table Expression

2014-01-17 Thread Nico Williams
On Fri, Jan 17, 2014 at 2:05 PM, Richard Hipp d...@sqlite.org wrote: On Fri, Jan 17, 2014 at 2:24 PM, Petite Abeille petite.abei...@gmail.comwrote: On Jan 17, 2014, at 7:47 PM, big stone stonebi...@gmail.com wrote: - I just did my first recursive CTE under Ipython notebook. Finally! We can

Re: [sqlite] SQLite 2013 retrospective

2014-01-01 Thread Nico Williams
On Tue, Dec 31, 2013 at 10:37 AM, Richard Hipp d...@sqlite.org wrote: New tested and supported extensions added to the source tree: 2. The transitive_closure virtual table: www.sqlite.org/src/artifact/6360243 Funny you should mention this. I've written transitive closure computations

Re: [sqlite] SQLite version 3.8.2 running 2.5x faster for some queries.

2013-11-25 Thread Nico Williams
If a function is deterministic for then it can be executed once and its result can be memoized for the rest of the statement (or transaction even). If a function is idempotent for a statement then it can be executed once per-statement, and its result(s) can be memoized and reused for the life of

Re: [sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-19 Thread Nico Williams
On Tue, Nov 19, 2013 at 6:35 AM, RSmith rsm...@rsweb.co.za wrote: I also do not think (or imagine) that unique constraints ought to be required - I am saying that in the case of SQLite or any mainstream RDBMS it becomes necessary to have some unique reference to single out a row should you

Re: [sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-19 Thread Nico Williams
On Tue, Nov 19, 2013 at 11:50 AM, RSmith rsm...@rsweb.co.za wrote: Hi Nico, thanks for the discussion - although this thread is dangerously close to becoming a stale debate rather than a helpful forum discussion. In this light, I will try to be brief and then fight the urge to reply again.

[sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-18 Thread Nico Williams
It seems obvious at first glance that WITHOUT ROWID should require a table to have a PRIMARY KEY, but on second glance I don't get it. SQL itself doesn't require tables to have primary keys, so why couldn't an RDBMS have primary key-less tables? Obviously a B-Tree-based table will need *some*

Re: [sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-18 Thread Nico Williams
I should add that WITHOUT ROWID ended up making a very small difference in bulk-load performance of my application, but a bigger and noticeable difference in incremental operation (where it's most welcomed and desired anyways) performance. Nico -- ___

Re: [sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-18 Thread Nico Williams
I can also imagine a day when PRAGMA without_rowid_by_default = 1; will cause tables created subsequently to lack implicit rowid columns. It'd be easier to enable that if the selection of columns for the b-tree index were less consequential than requiring a PRIMARY KEY with its attendant

Re: [sqlite] Intended use case for 'without rowid'?

2013-11-18 Thread Nico Williams
On Fri, Nov 15, 2013 at 5:14 AM, Pepijn Van Eeckhoudt pep...@vaneeckhoudt.net wrote: I've been looking into the upcoming 'without rowid' feature implementation to assess if it will have any impact on the OGC GeoPackage specification. One of the things I was wondering is what the intended use

Re: [sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-18 Thread Nico Williams
On Mon, Nov 18, 2013 at 8:57 PM, RSmith rsm...@rsweb.co.za wrote: Can you honestly use tables without a single unique reference to them? Plain piles of data abound in quantities so voluminous that there's no time to index them (of course, one would not apply SQLite3 to such a dataset, I'm only

Re: [sqlite] Why must WITHOUT ROWID tables have PRIMARY KEYs?

2013-11-18 Thread Nico Williams
Tomorrow I may describe two real-world cases where logical uniqueness constraints are difficult or extremely difficult to enforce naturally using standard SQL uniqueness constraints. Both are or can be generalizations of the id,attr,val model, one of them using a timeseries approach to permit

Re: [sqlite] Feature Request: Binding Arrays

2013-10-31 Thread Nico Williams
On Wed, Oct 16, 2013 at 07:28:04AM -0400, Richard Hipp wrote: Please see http://www.sqlite.org/src/artifact/2ece66438?ln=13-75 I've been thinking for a while -ever since I happened upon jq(1)- that a marriage of jq and SQLite3 would be wonderful. jq is a JSON query language. It's a functional

Re: [sqlite] Feature Request: Binding Arrays

2013-10-31 Thread Nico Williams
Oh, and jq is at: https://stedolan.github.io/jq ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQlite Memory Leakage with 65536 Page Size

2013-10-28 Thread Nico Williams
On Tue, Oct 29, 2013 at 01:22:39AM +0530, Raheel Gupta wrote: Sir, is there any way to not allow malloc to hold memory ? I mean shouldnt free(), be freeing the memory ? No. The way malloc() typically works is that it allocates large chunks of memory from the OS's kernel, then it allocates

Re: [sqlite] Hints for the query planner

2013-10-02 Thread Nico Williams
On Wednesday, October 2, 2013, Richard Hipp wrote: In the current beta (at http://www.sqlite.org/download.html) the function is either unlikely(X) or likelihood(X,Y). In the second form, Y must be a floating point constant between 0.0 and 1.0, inclusive. The first form is equivalent to

Re: [sqlite] Hints for the query planner

2013-10-01 Thread Nico Williams
On Tuesday, September 10, 2013, Richard Hipp wrote: SURVEY QUESTION: The question for today is what to call this magic hint function: (1) unlikely(EXPR) (2) selective(EXPR) (3) seldom(EXPR) (4) seldom_true(EXPR) (5) usually_not_true(EXPR) (1), on account of: it's already used in

Re: [sqlite] BETWEEN and explicit collation assignment

2013-09-03 Thread Nico Williams
On Thu, Aug 29, 2013 at 6:35 PM, James K. Lowden jklow...@schemamania.org wrote: On Tue, 27 Aug 2013 21:00:50 -0500 Nico Williams n...@cryptonector.com wrote: Of course, lacking a syntax for associating collations with string literals there will be times when some, or even all of the operands

Re: [sqlite] BETWEEN and explicit collation assignment

2013-08-27 Thread Nico Williams
On Mon, Aug 26, 2013 at 2:03 PM, Roman Fleysher roman.fleys...@einstein.yu.edu wrote: Certainly associating a type conversion with a lexical instance of an operator, applying to all of the operator's operands (or perhaps even just to some, while having to explicitly cast others) would work, at

Re: [sqlite] BETWEEN and explicit collation assignment

2013-08-22 Thread Nico Williams
On Aug 22, 2013 8:18 PM, James K. Lowden jklow...@schemamania.org wrote: Collation is not a property of the value. Strings are just strings. They have an *encoding*, necessarily, but not a collation. Collation therefore must, by process of elimination, be a property of the type. And a good

Re: [sqlite] UnQLite

2013-07-07 Thread Nico Williams
On Sun, Jul 7, 2013 at 8:40 PM, Roger Binns rog...@rogerbinns.com wrote: On 07/07/13 16:19, Simon Slavin wrote: http://unqlite.org What has generally become important to me is being able to supply arbitrary JSON as a record, be able to query it, and get the same arbitrary JSON back out. In

Re: [sqlite] Prepared statement invariants

2013-06-14 Thread Nico Williams
IMO you should provide a function to invalidate the cache and also one to check the schema version number, then leave it to the application to decide when or if to bother with this. ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] SQLite and integer division 1/2=0

2013-05-13 Thread Nico Williams
On May 12, 2013 11:36 PM, James K. Lowden jklow...@schemamania.org wrote: I'd add also that syntactically the key need is to distinguish use floating point arithmetic from use integer arithmetic where no other type information is available, specifically in numeric constant literals. Having a

Re: [sqlite] B-Tree usage

2013-05-11 Thread Nico Williams
SQLite3 only uses B-trees, for tables and indexes. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] B-Tree usage

2013-05-11 Thread Nico Williams
It'd help if you posted a simplified schema and statements that are not planned correctly by SQLite3. Nico -- ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] B-Tree usage

2013-05-11 Thread Nico Williams
On Sat, May 11, 2013 at 1:12 AM, Igor Korot ikoro...@gmail.com wrote: On Fri, May 10, 2013 at 11:07 PM, Nico Williams n...@cryptonector.comwrote: It'd help if you posted a simplified schema and statements that are not planned correctly by SQLite3. I'm not saying that my statements

Re: [sqlite] Suspected bug in SELECT EXCEPT ORDER COLLATE

2013-05-07 Thread Nico Williams
On Mon, May 6, 2013 at 9:39 PM, Keith Medcalf kmedc...@dessus.com wrote: I read this: SELECT a FROM t1 EXCEPT SELECT a FROM t2 ORDER BY a; as saying select all a from t1 that are not in t2, *then* order that by that a. No, that is not what it says, actually. The translation of your

Re: [sqlite] How do sqlite3_reset_last_insert_rowid() or functionalequivalent?

2013-05-07 Thread Nico Williams
On Tue, May 7, 2013 at 7:14 AM, Mikael mikael.tr...@gmail.com wrote: A generalized way of getting if an ID was inserted on the last performed query and if so which, is of value for instance when inserting a row into a table with an AUTOINCREMENT key e.g. This can get tricky if you have BEFORE

Re: [sqlite] Suspected bug in SELECT EXCEPT ORDER COLLATE

2013-05-06 Thread Nico Williams
On Mon, May 6, 2013 at 4:22 PM, Keith Medcalf kmedc...@dessus.com wrote: It is difficult to determine what is correct behaviour. I would think that the order by clause applies to the set operation, therefore any collation being applied has to be applied also to the component sets before the

Re: [sqlite] Bug report: query parser should trigger an error if a subquery selects unknown columns from table

2013-05-01 Thread Nico Williams
On Wed, May 1, 2013 at 2:23 PM, Anderson Medeiros Gomes amg1...@gmail.com wrote: Hi. I think I found a bug in SQLite, so I'm reporting it in this message. I do't think it's a bug. Correlated sub-queries can refer to columns from table sources outside them. Your example query is silly, no

Re: [sqlite] Query optimizer bug?

2013-04-30 Thread Nico Williams
On Tue, Apr 30, 2013 at 11:59 AM, Richard Hipp d...@sqlite.org wrote: http://www.sqlite.org/draft/queryplanner-ng.html That's quite interesting. Should the user have a way to influence the query planner? Perhaps by indicating a cost for each table source? SQL is supposed to let the RDBMS pick

Re: [sqlite] I/O errors with WAL on ZFS

2013-04-24 Thread Nico Williams
On Wed, Apr 24, 2013 at 11:21 AM, Sašo Kiselkov skiselkov...@gmail.com wrote: ZFS has been the filesystem of choice for SunOS-based systems for about the last 5 years now, is becoming that for FreeBSD as we speak, and is More like 8 years :) quickly gaining ground on Linux. The absence of

Re: [sqlite] I/O errors with WAL on ZFS

2013-04-24 Thread Nico Williams
On Wed, Apr 24, 2013 at 3:25 PM, Richard Hipp d...@sqlite.org wrote: On Wed, Apr 24, 2013 at 12:21 PM, Sašo Kiselkov skiselkov...@gmail.comwrote: ... SQLite is using posix_fallocate() to allocate space for a region of shared memory obtained using mmap(). If the space cannot be preallocated,

Re: [sqlite] 64bit compatibility warnings

2013-04-09 Thread Nico Williams
On Mon, Apr 8, 2013 at 8:52 AM, Alexandr Němec a.ne...@atlas.cz wrote: The first warning is harmless and results from a prior datatype change. Dan has already fixed that one. The other four appear to be due to an MSVC compiler bug, since every (i64%int) operation will always yield a value

Re: [sqlite] 64bit compatibility warnings

2013-04-09 Thread Nico Williams
On Mon, Apr 8, 2013 at 9:10 AM, Michael Black mdblac...@yahoo.com wrote: Which is why...IMHOto avoid all the repeats of this question in the future (and from the past)one should simply to do the cast to int and put a comment on the line that says % int always fits in an int.

Re: [sqlite] 64bit compatibility warnings

2013-04-09 Thread Nico Williams
On Tue, Apr 9, 2013 at 6:28 PM, David Empson demp...@emptech.co.nz wrote: No, this is a compiler bug. It is not a compiler bug. It is a failure of the compiler to deduce that the warning is unnecessary. [...] C's usual arithmetic conversions specify that if either operand of a binary

Re: [sqlite] SQLite 3.7.17 preview - 2x faster?

2013-04-04 Thread Nico Williams
On Thu, Apr 4, 2013 at 8:19 AM, Howard Chu h...@symas.com wrote: This is why OpenLDAP LMDB uses a read-only mmap by default. User bugs get an immediate SEGV, and usually the bug becomes obvious and easy to fix. There are many reasons to want to use read-only mmap()s (with MAP_SHARED though) and

Re: [sqlite] SQLite 3.7.17 preview - 2x faster?

2013-04-04 Thread Nico Williams
On Thu, Apr 4, 2013 at 11:44 AM, Drake Wilson dr...@dasyatidae.net wrote: Repeating these steps, but compiling the application with the sqlite3.c from the 201304040051 snapshot amalgamation that uses unprotected mmap, causes the entire kvserv process to die with SIGBUS as soon as a query tries

Re: [sqlite] SQLite 3.7.17 preview - 2x faster?

2013-04-04 Thread Nico Williams
On Thu, Apr 4, 2013 at 4:45 PM, Drake Wilson dr...@dasyatidae.net wrote: Quoth Nico Williams n...@cryptonector.com, on 2013-04-04 16:08:24 -0500: This is very sad. But really, the OS should cause kvserv to hang waiting for I/O from the device to complete (and you should get some indication

Re: [sqlite] Implementing Save As... functionality for Application File Format usecase

2013-04-03 Thread Nico Williams
On Wed, Apr 3, 2013 at 4:11 PM, Tiago Rodrigues wtrm...@gmail.com wrote: I'm writing a small simulation app and for it I would like to use SQLite3 as an application file format, as suggested by the Appropriate uses for SQLite page in sqlite.org. More specifically, the page suggests calling

Re: [sqlite] Implementing Save As... functionality for Application File Format usecase

2013-04-03 Thread Nico Williams
As a user I prefer continuous saving + infinite undo. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing Save As... functionality for Application File Format usecase

2013-04-03 Thread Nico Williams
On Wed, Apr 3, 2013 at 5:08 PM, Simon Slavin slav...@bigfraud.org wrote: On 3 Apr 2013, at 10:52pm, Nico Williams n...@cryptonector.com wrote: As a user I prefer continuous saving + infinite undo. Undo is difficult with SQLite. For instance, to undo an UPDATE command you need to know

Re: [sqlite] sqlite3 and sqlite4 disk I/O

2013-03-28 Thread Nico Williams
On Thu, Mar 28, 2013 at 3:02 PM, Dan Kennedy danielk1...@gmail.com wrote: So sqlite4 is using more CPU. But in this particular case is faster overall because the way the database file is written is more efficient (i.e. more sequential). It's probably writing a bit less too. One way to reduce

Re: [sqlite] sqlite3 and sqlite4 disk I/O

2013-03-28 Thread Nico Williams
On Thu, Mar 28, 2013 at 6:06 PM, Simon Slavin slav...@bigfraud.org wrote: On 28 Mar 2013, at 9:54pm, Nico Williams n...@cryptonector.com wrote: Question: would it be better to optimize key and value encodings more for CPU and less for space? Don't forget that SQLite is not solely intended

Re: [sqlite] SQLite strong-typing [WAS: inner vs. outer join inconsistency]

2013-03-08 Thread Nico Williams
On Fri, Mar 8, 2013 at 1:26 PM, James K. Lowden jklow...@schemamania.org wrote: On Thu, 7 Mar 2013 19:20:44 +0100 Petite Abeille petite.abei...@gmail.com wrote: Yeah? 'cool' is not necessarily how I would describe it? having a check constraint 'magically' coerce - change! - the inserted data

Re: [sqlite] SQLite strong-typing [WAS: inner vs. outer join inconsistency]

2013-03-07 Thread Nico Williams
On Thu, Mar 7, 2013 at 7:36 AM, Michael Black mdblac...@yahoo.com wrote: Personally I think this behavior is horrid. Is there some scenario where this wouldn't be a latent bug? I don't like it either. I also share Simon's (and yours!) opinion regarding your patch: if cast be fixed at all (and

Re: [sqlite] SQLite strong-typing [WAS: inner vs. outer join inconsistency]

2013-03-07 Thread Nico Williams
On Thu, Mar 7, 2013 at 11:12 AM, Ryan Johnson ryan.john...@cs.utoronto.ca wrote: I would argue that, if a column has type affinity, CHECK should work with the value that would actually get stored, not the one that was assigned. But then you couldn't check the value that was attempted to store.

  1   2   3   4   >