Re: [sqlite] Doing math in sqlite

2018-12-20 Thread Shevek
For the cost of a single table scan, you may be better with: select max(foo) - min(foo) where etc. S. -- Anyone using floating point for financial computations needs their head examined. On 12/20/18 4:32 PM, Jens Alfke wrote: On Dec 20, 2018, at 4:21 PM, Jungle Boogie wrote: select

[sqlite] Either bad constant or bad comment in os_unix.c

2018-08-13 Thread Shevek
In the nolock IOMETHODS: https://github.com/mackyle/sqlite/blob/7bd4fc81a71bdc777151c747b2e6d3ee58994251/src/os_unix.c#L5203 Either the constant should be '1' or the comment should permit mmap. I suspect the constant should be 1? S. ___

Re: [sqlite] mmap, madvise, mlock and performance

2018-08-13 Thread Shevek
On 08/03/2018 12:55 PM, Simon Slavin wrote: On 3 Aug 2018, at 8:36pm, Shevek wrote: We are running a 100Gb sqlite database, which we mmap entirely into RAM. We are having trouble with parts of the disk file being evicted from RAM during periods of low activity causing slow responses

[sqlite] mmap, madvise, mlock and performance

2018-08-03 Thread Shevek
Hi, We are running a 100Gb sqlite database, which we mmap entirely into RAM. We are having trouble with parts of the disk file being evicted from RAM during periods of low activity causing slow responses, particularly before 9am. Has anybody played with mlock and/or madvise within the

Re: [sqlite] random rows

2018-06-01 Thread Shevek
You may find it faster to do: select c from t where rowid in (list-of-constants) and generate the list of constants using something like a blackrock permutation generator. That takes linear time, whereas all the order-by variants are n.log(n). You need some sort of row-id generator function,

Re: [sqlite] [EXTERNAL] unexpected error with "GROUP BY 0"

2018-05-31 Thread Shevek
SQL overall is phenomenally ambiguous about group-by: 1) select a0 + 1 as a0 from a group by a0 has a different behaviour in different SQL implementations, depending on whether the group-by prefers the underlying column or the projection alias. 2) As you rightly point out, group by integer

Re: [sqlite] [EXTERNAL] Window functions

2018-04-26 Thread Shevek
About storing the whole result set: Note that in postgresql-derivatives (and Oracle? maybe Teradata?), this is valid: select lag(a0, a0) over () from a; whereas many other servers (which I won't name) require the second argument of lag() to be constant. If it is constant (even in

Re: [sqlite] UPSERT available in pre-release

2018-04-19 Thread Shevek
Opinion: Of all the DBMS's UPSERT/MERGE semantics, postgresql's is the least useful because it's very limited: It can only do a check against a constraint, and the cost of evaluating that constraint has to be carried by all other statements which mutate the table. Oracle/Teradata MERGE is a

Re: [sqlite] Are you getting spam when you post to sqlite-users ?

2018-04-18 Thread Shevek
It has to be a subscriber. I just got spam in the form of an almost-immediate reply to my last message to the list, including message-ids. Web wouldn't bother including that metadata, and is unlikely to be quite so real-time. On 04/18/2018 11:35 PM, Simon Slavin wrote: On 18 Apr 2018, at

Re: [sqlite] JDBC driver experience

2018-04-18 Thread Shevek
1. Xerial 2. A year or so, relatively heavily. 3. * MAX_MMAP_SIZE is too small by a few hundred megabytes. It's safe to set to around a terabyte. * No way to effectively use multiple threads, even on a read-only mmap'd database. * Planner sometimes misses plans - does it need better STAT

Re: [sqlite] High performance and concurrency

2018-03-01 Thread Shevek
Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Shevek Gesendet: Donnerstag, 01. März 2018 09:10 An: SQLite mailing list <sqlite-users@mailinglists.sqlite.org>; Simon Slavin <slav...@bigfraud.org> Betreff: [EXTERNAL] Re: [

Re: [sqlite] High performance and concurrency

2018-03-01 Thread Shevek
On 02/28/2018 11:45 PM, Simon Slavin wrote: On 1 Mar 2018, at 7:24am, Shevek <sql...@anarres.org> wrote: What I think is happening is that either a pthread mutex or a database lock is serializing the accesses, so each thread blocks the others. To be specific, I'm concerned

[sqlite] High performance and concurrency

2018-02-28 Thread Shevek
Hi, I would like to have truly concurrent access to an sqlite database, that is, the ability for multiple connections to read from the database simultaneously. I'm using Java with xerial's sqlite-jdbc, customized to let me mmap the entire database into RAM, and with additional debugging

[sqlite] Missing several important tricks in partial indexes

2018-02-26 Thread Shevek
If I create a partial index: create table a (a0, a1) create index idx on a (a0) where a1 is null; Then we have several issues: 1) This should be a covering index for select a0, a1 from a where a1 is null; It isn't. It's a great index, but we still defer to the table to look up the (always