Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
Normally there is a small "Memory Mapped" buffer into the FileSystem cache that maintains a constant size. This area is "mapped" into the Cache Working Set. As you read/write different parts of files in the cache, the "mapped" are is "moved" so that the I/O can be carried out by the pager.

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
That will do nothing. I explicitly set the cache mode. I always disable all "Mystical Microsoft Magical Behaviour" wherever I can find it -- though there are a very few "autotuning" things that work properly by themselves, just not many (unless they originated with an actual Engineer

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
According to Mark Russinovich's fine RAMMap tool (part of the SysInternals Suite) Windows is "converting" the file from a "standard" FileSystem cached file to a "Memory Mapped" file and back again. "Memory Mapped" files occupy RAM (as in allocate RAM so it cannot be used for other purposes)

Re: [sqlite] .timer

2018-06-15 Thread Simon Slavin
On 15 Jun 2018, at 11:21pm, Keith Medcalf wrote: > And you are correct, it appears to be related to "reading backwards" somehow. If you're comparing the two directions of reading the same file, there are ways to fake out Windows' read-ahead tricks and see if they're what's messing up your

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
Ok, I have been able to reproduce this but have not figured out what is happening. And you are correct, it appears to be related to "reading backwards" somehow. It is unaffected by the WIN32 cache mode that is set (I tested all modes and they all behave the same -- except that I could not

Re: [sqlite] Recursive query

2018-06-15 Thread R Smith
I confirm your query determining the fully expanded list of A_ids works as I explicitly tried it. Now. I have actually simplified the problem before presenting it here in the list. But I just tried to extrapolate the solution back to the original problem and failed miserably :-( Yes, that

[sqlite] Mistake in documentation about xCreate/xConnect vtab methods

2018-06-15 Thread sqlite
The documentation for the xCreate and xConnect methods for virtual tables give the incorrect type. It says "char**argv" but the actual type should be "const char*const*argv". ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Recursive query

2018-06-15 Thread David Raymond
with recursive foo (X_id) as ( values (21) union select Aelements.X_id from X inner join Aelements on X.A_id = Aelements.object_id where X_id is not null ) select Aelements.* from X inner join Aelements on X.A_id = Aelements.object_id where X.id in foo and Aelements.B_id is not

Re: [sqlite] SQLite query planner doesn't seem to know when a partial index is COVERING

2018-06-15 Thread David Raymond
I've seen similar. I think the issue is that since a isn't an indexed column then it's not technically covering, despite being given a in the where clause. sqlite> create index Foo_partial_with_a on Foo (a, b, c) where a = 1; sqlite> analyze; sqlite> explain query plan select b from Foo where

Re: [sqlite] [OT} Posting through Gmane

2018-06-15 Thread Igor Tandetnik
On 6/15/2018 9:33 AM, Lifepillar wrote: I use Gmane to follow this mailing list and Fossil's. I have subscribed to both lists. While I can post through Gmane to this list, however, I cannot post to fossil-users: messages keep bouncing back with "Address not found, ..., The response was 550

Re: [sqlite] Back on-line. Was: Mailing list shutting down...

2018-06-15 Thread J. King
On June 15, 2018 12:17:31 PM EDT, dmp wrote: >> Mailing lists are now back on-line and once again accepting >> subscriptions. I have implemented measures to block the subscription >> robots and to better log subscription activity to better detect >future >> mischief. > >> I consider this to be a

Re: [sqlite] Recursive query

2018-06-15 Thread MM
> > On 2018/06/15 3:19 PM, MM wrote: > I have table A of rows with column "object_id", and 2 NULL columns A_id > and B_id. > Each row of A (among rows with the same object_id) either references > table A itself or table B, ie columns A_id and B_id are mutually > exclusive. > e.g. > Table A >

[sqlite] SQLite query planner doesn't seem to know when a partial index is COVERING

2018-06-15 Thread Deon Brewis
Looks like a missed optimization opportunity here. Well, more than that - there doesn't appear to be a way to get SQLITE to automatically use a partial index if a similar non-partial index exists. E.g. create table Foo(a,b,c,d,e); create index Foo_inx on Foo(a,b,c); create index Foo_partial_inx

Re: [sqlite] .timer

2018-06-15 Thread x
Thanks Keith. I did look on the performance page but didn’t realise clicking the items on the left brought up different info. I am on windows 10 pro. I haven’t had time to try creating the ‘personalised’ sqlite3.exe so the following relates to my own app run under the original conditions (i.e.

Re: [sqlite] Back on-line. Was: Mailing list shutting down...

2018-06-15 Thread dmp
> Mailing lists are now back on-line and once again accepting > subscriptions. I have implemented measures to block the subscription > robots and to better log subscription activity to better detect future > mischief. > I consider this to be a stop-gap measure that will buy me some time > to

Re: [sqlite] Recursive query

2018-06-15 Thread R Smith
On 2018/06/15 3:19 PM, MM wrote: I have table A of rows with column "object_id", and 2 NULL columns A_id and B_id. Each row of A (among rows with the same object_id) either references table A itself or table B, ie columns A_id and B_id are mutually exclusive. e.g. Table A object_id A_id

Re: [sqlite] Recursive query

2018-06-15 Thread David Raymond
Something like this? Your seed number will go into the values statement at the start. with recursive foo (A_id) as ( values (1) union select A.A_id from foo inner join A on foo.A_id = A.object_id where A.A_id is not null ) select distinct B.data from foo inner join A on

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
Your other "right", the one on the left :) --- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume. >-Original Message- >From: sqlite-users [mailto:sqlite-users- >boun...@mailinglists.sqlite.org] On Behalf Of Keith Medcalf

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
You are using Windows 10? It is on the "Performance" tab, select the wee graph on the right for "Memory". In the detail, right underneath "Available" and beside "Committed" at the bottom where all the text is. Oh, you have to be in "more details" view, not in the "simple" default view ...

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
>You spoon fed me on a previous thread how to load extensions using a >core_init function placed at the end of the sqlite3.c code. I do have >the series.c in my core_init so it is available to me and works fine >in my cpp code. I don’t see how that relates to sqlite3.exe though. >How do you get

[sqlite] [OT} Posting through Gmane

2018-06-15 Thread Lifepillar
I use Gmane to follow this mailing list and Fossil's. I have subscribed to both lists. While I can post through Gmane to this list, however, I cannot post to fossil-users: messages keep bouncing back with "Address not found, ..., The response was 550 unknown user". Is there any difference in the

[sqlite] Recursive query

2018-06-15 Thread MM
I have table A of rows with column "object_id", and 2 NULL columns A_id and B_id. Each row of A (among rows with the same object_id) either references table A itself or table B, ie columns A_id and B_id are mutually exclusive. e.g. Table A object_id A_id B_id ... 1 NULL

Re: [sqlite] .timer

2018-06-15 Thread curmudgeon
PS I can't find 'cache' in task manager. -- Sent from: http://sqlite.1065341.n5.nabble.com/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] .timer

2018-06-15 Thread x
Keith, You spoon fed me on a previous thread how to load extensions using a core_init function placed at the end of the sqlite3.c code. I do have the series.c in my core_init so it is available to me and works fine in my cpp code. I don’t see how that relates to sqlite3.exe though. How do

Re: [sqlite] .timer

2018-06-15 Thread David Raymond
2 MB, not 2 GB. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Warren Young Sent: Thursday, June 14, 2018 5:54 PM To: SQLite mailing list Subject: Re: [sqlite] .timer By default, SQLite will only cache a smidge under 2 GiB, so

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
The increase/decrease in memory is almost certainly the cache (after running the command once and before flushing look and see what Task Manager says for "Cached", then look again after you do the flush and see if it releases it. This is memory that would otherwise be unused being used by

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
Yes, that is correct. Then compile with the preprocessor symbol SQLITE_WIN32_FILE_RANDOM defined. generate_series is an extension module located in ext/misc/series.c https://www.sqlite.org/src/dir?ci=567e09ef2a8cd84a=ext/misc By default I load almost every one of those extensions in every

Re: [sqlite] .timer

2018-06-15 Thread x
>Actually I cannot reproduce even if I turn off forced RANDOM mode for the >cache, reduce the size of the >sqlite3 cache to the default, and make sure the >temp_store is on disk (not my default). Are you rebooting or flushing the cache between the commands? I don’t have any problems (except

[sqlite] Just noticed ... fileio.c cannot be compiled as an extension ...

2018-06-15 Thread Keith Medcalf
It looks for an external sqlite3_win32_utf8_to_unicode function. Although this is an exported API function the linker on Windows cannot resolve it at compile time. Even if it could, the loader trampoline could not link it back to the API in the original (loading) sqlite3 code (especially not

Re: [sqlite] .timer

2018-06-15 Thread x
Keith, I can find no instance of 2966 in my sqlite.c code. I have found the code you mentioned if( isDelete ){ #if SQLITE_OS_WINCE dwFlagsAndAttributes = FILE_ATTRIBUTE_HIDDEN; isTemp = 1; #else dwFlagsAndAttributes = FILE_ATTRIBUTE_TEMPORARY

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
Actually I cannot reproduce even if I turn off forced RANDOM mode for the cache, reduce the size of the sqlite3 cache to the default, and make sure the temp_store is on disk (not my default). I do note, however, that the actual CPU used is relatively constant (USER+SYS) and that it is the

Re: [sqlite] .timer

2018-06-15 Thread Keith Medcalf
Yes. Search your sqlite3.c for #2966 (there is only one instance and it is in the winOpen function, somewhere around line 44847). The code that is there will be an #ifdef that forces RANDOM_ACCESS only for WINCE. Just change it so that you can compile it with RANDOM_ACCESS set and see if

Re: [sqlite] .timer

2018-06-15 Thread x
Thanks for all the replies and great suggestions. I’m just up but will shortly investigate all. After reading this post by Clemens Ladisch http://sqlite.1065341.n5.nabble.com/This-is-driving-me-nuts-tp102034p102105.html I was guessing the answer might be something along the lines of what