Re: [sqlite] The mailing list is deprecated...... [was: Re: [EXTERNAL] No such column error]

2020-03-24 Thread Hick Gunter
list is deprecated.. [was: Re: [EXTERNAL] No such column error] OK i must have must the posts from the 12th of March till the end of that week 😉, being busy with other things. On 24-3-2020 09:19, Hick Gunter wrote: > See announcement on the mailing list dated march 12th > >

Re: [sqlite] The mailing list is deprecated...... [was: Re: [EXTERNAL] No such column error]

2020-03-24 Thread Hick Gunter
See announcement on the mailing list dated march 12th -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Richard Hipp Gesendet: Donnerstag, 12. März 2020 21:18 An: General Discussion of SQLite Database Betreff: [EXTERNAL] [sql

Re: [sqlite] [EXTERNAL] No such column error

2020-03-23 Thread Hick Gunter
The mailing list is deprecated. You need to go to https://sqlite.org/forum/ for the sqlite forum. Can you replicate the problem while using the sqlite shell? Are you checking column names returned from the second statement? Note that a.BIRTH.YYY from your example looks a bit weird for a qualifi

Re: [sqlite] [EXTERNAL] Re: COALESCE() does not short-circuit sometimes

2020-03-12 Thread Hick Gunter
Exactly what I gained from the EXPLAIN output. The SQL "compiler" is extracting the constant expression ABS(...) and evaluating it in the program prolog (where schema is checked and locks taken). See instructions 11 and 12 asql> explain select coalesce((SELECT 'hello'),ABS(-9223372036854775808)

Re: [sqlite] [EXTERNAL] Re: COALESCE() does not short-circuit sometimes

2020-03-11 Thread Hick Gunter
While ORACLE does state that COALESCE will short circuit, SQLite does not. May I suggest implementing your own user defined function to do this instead. void THROW_IF_NULL( sqlite3_context *ctx, int argc, sqlite3_value**argv) { int ii; for( ii == 0; ii < argc; ii++) {

Re: [sqlite] [EXTERNAL] Re: COALESCE() does not short-circuit sometimes

2020-03-11 Thread Hick Gunter
It is possible to infer, from the EXPLAIN output, that the SQLite program generator attempts to isolate constant expressions and evaluates them first, before it enters the COALESCE loop. From my experience in reading SQL Programs, the general structure is GOTO INIT START: - load constant values

Re: [sqlite] [EXTERNAL] Inconsistency of CREATE/DROP TABLE with attached DBs

2020-03-04 Thread Hick Gunter
-Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Hick Gunter Gesendet: Donnerstag, 5. März 2020 08:48 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] Inconsistency of CREATE/DROP TABLE with attached DBs I don't se

Re: [sqlite] [EXTERNAL] Inconsistency of CREATE/DROP TABLE with attached DBs

2020-03-04 Thread Hick Gunter
I don't see any inconsistency here. 1) implicit attach of a.sqlite as main and create a.t1 2) implcit attach b.sqlite as main, attach a.sqlite as a and create (main).t1 (in b.sqlite) 3) implicit attach c.sqlite as main, attaxh a.sqlite as a and drop the only table named t1 from a c.sqlite never

Re: [sqlite] [EXTERNAL] Sql update script. check for existing rows before inserting...

2020-02-23 Thread Hick Gunter
SQLite is not a procedural language. IF is not a programming construct, it is part of an expression. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Andy KU7T Gesendet: Montag, 24. Februar 2020 05:17 An: SQLite mailing list

Re: [sqlite] [EXTERNAL] rtrim and round functions unexpected result

2020-02-20 Thread Hick Gunter
Round(1299.6) returns the floating point number 1300.0, passing 1300.0 to the rtrim function converts it tot he string '1300.0' removing all '.' and '0' characters from '1300.0' yields 13 This is no suprise -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglis

Re: [sqlite] [EXTERNAL] question about INTEGER PRIMARY KEY AUTOINCREMENT

2020-02-20 Thread Hick Gunter
The next value for an INTEGER PRIMARY KEY AUTOINCREMENT does not depend on the current contents of the table, only its history. While ROWIDs are monotnically increasing, there may be gaps in the sequence, caused by rows that failed to insert due to constraint violations. However, ROWIDs that get

Re: [sqlite] [EXTERNAL] Optimizer limitation with partial indexes

2020-02-12 Thread Hick Gunter
>Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im >Auftrag von Jens Alfke >> On Feb 12, 2020, at 5:30 AM, Hick Gunter wrote: >> >> This is documented here https://sqlite.org/partialindex.html >> <https://sqlite.org/partialindex.htm

Re: [sqlite] [EXTERNAL] Optimizer limitation with partial indexes

2020-02-12 Thread Hick Gunter
This is documented here https://sqlite.org/partialindex.html and here https://sqlite.org/queryplanner.html Specifically, SQLIte does not prove theorems in first-order logic. To have a chance of using the partial indices, you would need to have your query translator formulate (expr1>val1 AND exp

Re: [sqlite] [EXTERNAL] Re: Patch: VTable Column Affinity Question and Change Request

2020-02-06 Thread Hick Gunter
>Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im >Auftrag von Dominique Devienne > >On Thu, Feb 6, 2020 at 9:32 AM Hick Gunter wrote: >> >Of course, it may be that the writer of the VTable should know what they >> >are doing and genera

Re: [sqlite] [EXTERNAL] Re: Patch: VTable Column Affinity Question and Change Request

2020-02-06 Thread Hick Gunter
list Betreff: Re: [sqlite] [EXTERNAL] Re: Patch: VTable Column Affinity Question and Change Request Hick Gunter wrote on Thursday, February 6, 2020 3:32 AM >We are almost exclusively using virtual tables to allow queries against >our internal data sources, which are C language structs an

Re: [sqlite] [EXTERNAL] Re: Patch: VTable Column Affinity Question and Change Request

2020-02-06 Thread Hick Gunter
>Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im >Auftrag von Keith Medcalf >Betreff: [EXTERNAL] Re: [sqlite] Patch: VTable Column Affinity Question and >Change Request > > >Patch to Fix Column Affinity not applied to Virtual Columns. > >In expr.c function sqlite3ExprCo

Re: [sqlite] [EXTERNAL] Unsubscribe from Sqlite users list

2020-02-05 Thread Hick Gunter
Visit the link given at the bottom of every message, including this one -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Ainhoa B Gesendet: Mittwoch, 5. Februar 2020 15:11 An: SQLite mailing list Betreff: [EXTERNAL] [sqlite]

Re: [sqlite] [EXTERNAL] Re: Please increase the default for SQLITE_MAX_VARIABLE_NUMBER

2020-02-05 Thread Hick Gunter
WHERE x IN carray($PTR, $DIM) With $PTR being the address of the array and $DIM ist cardinality. Should be tons faster than parsing and binding a gazillion parameters. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Deon B

Re: [sqlite] [EXTERNAL] Re: Default isolation_level for sqlite3.connect?

2020-01-30 Thread Hick Gunter
"Autocommit" means that each SQL Statement executes in it's own transaction. Just as if you were to execute "begin; ; commit;" each time. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Peng Yu Gesendet: Freitag, 31. Januar

Re: [sqlite] [EXTERNAL] Either a bug or I don't understand SQL update

2020-01-30 Thread Hick Gunter
You have fallen into the double quote trap. SQLite uses double quotes to denote COLUMN NAMES, and single quotes to delimit STRING CONSTANTS. When asking for "M" or "G", you get the contents of the column named m and g respectively (column names are case insensitive). When asking for "P" or "R"

Re: [sqlite] [EXTERNAL] Is mutliple-execute-one-commit slower than multiple single-execute-single-commit?

2020-01-28 Thread Hick Gunter
If you could provide more information then maybe someone can suggest a reason or even a solution for the effect you are seeing. Some of the following may be helpful. What schema are you using? Which journal mode is your database running in? What kind of statements are executed? How are you contr

Re: [sqlite] [EXTERNAL] Re: Row length in SQLITE

2020-01-27 Thread Hick Gunter
You are missing maxsize += _varIntSize_(maxsize) fort he size varint at the begin oft he header just before the return -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Keith Medcalf Gesendet: Montag, 27. Januar 2020 12:43 A

Re: [sqlite] [EXTERNAL] Re: Row length in SQLITE

2020-01-27 Thread Hick Gunter
As previously mentioned, SQLite uses a compressed format to store rows. You would have to reverse engineer at least the calculation -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Deon Brewis Gesendet: Samstag, 25. Januar 2

Re: [sqlite] [EXTERNAL] Row length in SQLITE

2020-01-22 Thread Hick Gunter
SQLite uses a compressed format to store records (be it rows of a table or entries in an index), so the length of a specific record depends on its contents. See https://sqlite.org/fileformat.html Storing a row of (NULL, NULL, NULL, NULL) takes just 5 bytes, whereas (1024, 1.234, 'some string',

Re: [sqlite] [EXTERNAL] Re: 18 minutes 41 seconds

2020-01-02 Thread Hick Gunter
> Obviously the character(s) responsible for dates etc were NOT C programmers! No, they still using Roman Numerals instead of Indian Numbers and were oblivious of the number 0. As indeed Abu Dschaʿfar Muhammad ibn Musa al-Chwārizmī published his book "De numero Indorum" (the earliest latin tr

Re: [sqlite] [EXTERNAL] Only enter higher values in table

2019-12-27 Thread Hick Gunter
You need an UPDATE trigger for this, since the comparison requires knowledge of the old and new values. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Cecil Westerhof Gesendet: Freitag, 27. Dezember 2019 13:05 An: SQLite m

Re: [sqlite] [EXTERNAL] Compiling SQLite without the database storage?

2019-12-22 Thread Hick Gunter
No. You need the BTree and table code to handle the sqlite3_master table. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Jens Alfke Gesendet: Samstag, 21. Dezember 2019 19:50 An: SQLite mailing list Betreff: [EXTERNAL] [sq

Re: [sqlite] [EXTERNAL] Implementing a statement cache

2019-12-16 Thread Hick Gunter
Any statement that has been stepped but not to completion will hold open the transaction on a connection. This may interfere with your expectations. Clearing bindings as a precaution will prevent inadvertent re-use of old bindings. The statement may have to be reset first, see documentation. St

Re: [sqlite] [EXTERNAL] Difference between hex notation and string notation

2019-12-15 Thread Hick Gunter
The X'' notation returns a blob. LIKE works with strings. Comparing a string to a blob of the same content always returns false. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Sascha Ziemann Gesendet: Freitag, 13. Dezember

Re: [sqlite] [EXTERNAL] Re: Result set column names

2019-12-10 Thread Hick Gunter
Think about same column names in distinct tables within the same select and then throw in a couple of AS clauses and maybe an SQL parameter. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Doug Gesendet: Dienstag, 10. Dezem

Re: [sqlite] [EXTERNAL] virtual tables vs. expression-based indexes

2019-11-28 Thread Hick Gunter
If your external data store can maintain an index on some expression, then exposing that index as a computed field is the way to go with a virtual table. Alternatively, you can expose the index as a separate virtual table with a "foreign key" that references the original virtual table and join t

Re: [sqlite] [EXTERNAL] Slow joining of tables with indexes

2019-11-26 Thread Hick Gunter
You are using text columns as primary keys and referencing them directly in foreign keys. This is probably not what you want, because it duplicates the text key. Also, with foreign keys enabled, your join is not accomplishing anything more than a direct select from joining_table, just with more

Re: [sqlite] [EXTERNAL] Symlink to update a database table

2019-11-21 Thread Hick Gunter
What is the use case? The statement you give will set the value of the "column" field of table "table" to the whole contents of file.txt in each and every row that matches SQLite does not have a symbolic link type. You can store the text of a symbolic link, but accessing the contents would st

Re: [sqlite] [EXTERNAL] [Importing CSV] Empty colums != NULL?

2019-11-21 Thread Hick Gunter
CSV is unable to represent the NULL value. The best it can do is "empty string", which gets converted to - drumroll - an empty string (or the value 0 for numeric fields). The field names you are using suggest that you may have not sufficiently normalized your data, since you have duplicated pos

Re: [sqlite] [EXTERNAL] Re: Question about: Adding a record to a table with select failure

2019-11-19 Thread Hick Gunter
The Magic Max at work, forcing the query to return at least one record. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Jose Isaias Cabrera Gesendet: Montag, 18. November 2019 20:11 An: 'SQLite mailing list' Betreff: [EXTER

Re: [sqlite] [EXTERNAL] Re: Question about: Adding a record to a table with select failure

2019-11-17 Thread Hick Gunter
Nothing. The select returns no rows so no rows are inserted. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Doug Gesendet: Freitag, 15. November 2019 17:42 An: 'SQLite mailing list' Betreff: [EXTERNAL] Re: [sqlite] Questio

Re: [sqlite] [EXTERNAL] Re: Things you shouldn't assume when you store names

2019-11-14 Thread Hick Gunter
>> A growing number of organisations now ask me for my DOB or my >> postcode, rather than my name, when looking me up. I think you just >> explained why. In my country we have an increasing number of foreign >> family names, which probably helps it along. > >UK postcodes are incredibly fine-grain

Re: [sqlite] [EXTERNAL] Specific sqlite_autoindex_* missing in recent SQLite versions

2019-11-14 Thread Hick Gunter
Maybe you are confusing the autoindex logic by including superflous attributes: ... Id INTEGER NOT NULL PRIMARY KEY UNIQUE ... NOT NULL is enforced for WITHOUT ROWID tables and a single field PRIMARY KEY already implies UNIQUE, so no autoindex is required for Id This leaves only the autoindex r

Re: [sqlite] [EXTERNAL] Re: select for power-meter accumulated total readings

2019-10-20 Thread Hick Gunter
>What about if I want 1 hour granity? (to plot a graph of daily consumption for >example) For a (meaningful, as opposed to "all interpolated values") granularity of 1 hour, information theory states that you need a sample every 30 minutes or less. The desire to charge consumers more for "peak

Re: [sqlite] [EXTERNAL] Roadmap?

2019-10-20 Thread Hick Gunter
The "virtual table playground gadget" was our primary reason for selecting SQLite in the first place, because none of our production data sources are native SQLite tables. Instead, we have about 20 virtual table modules that implement about 1000 virtual table instances. -Ursprüngliche Nachr

Re: [sqlite] [EXTERNAL] Re: Limit on number of columns in SQLite table

2019-10-18 Thread Hick Gunter
to construct one "original" row will be faster? So not sure if I understand why reading and decoding cells in over multiple columns is so much slower than reading and decoding cells in over multiple rows? Mitar On Thu, Oct 17, 2019 at 3:38 PM Hick Gunter wrote: > > I have the

Re: [sqlite] [EXTERNAL] Re: Limit on number of columns in SQLite table

2019-10-17 Thread Hick Gunter
multiple columns is so much slower than reading and decoding cells in over multiple rows? Mitar On Thu, Oct 17, 2019 at 3:38 PM Hick Gunter wrote: > > I have the impression that you still do not grasp the folly of a 100k column > schema. > > See the example below, which only

Re: [sqlite] [EXTERNAL] Re: Limit on number of columns in SQLite table

2019-10-17 Thread Hick Gunter
I have the impression that you still do not grasp the folly of a 100k column schema. See the example below, which only has 6 fields. As you can see, each field requires a Column opcode and arguments (about 10 bytes) and a "register" to hold the value (48 bytes), which for 100k columns uses abou

Re: [sqlite] [EXTERNAL] Re: Limit on number of columns in SQLite table

2019-10-17 Thread Hick Gunter
Since your data is at least mostly opaque in the sense that SQLite is not expected to interpret the contents, why not split your data into "stuff you want to query ins SQLite" and "stuff you want to just store"? The former means individual columns, whereas the latter could be stored in a single

Re: [sqlite] [EXTERNAL] Limit on number of columns in SQLite table

2019-10-16 Thread Hick Gunter
Decker Gesendet: Donnerstag, 17. Oktober 2019 08:24 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] Limit on number of columns in SQLite table On Wed, Oct 16, 2019 at 11:03 AM Mitar wrote: > Hi! > > On Wed, Oct 16, 2019 at 3:16 PM Hick Gunter wrote: > > 100k distinct colu

Re: [sqlite] [EXTERNAL] Limit on number of columns in SQLite table

2019-10-16 Thread Hick Gunter
019 20:03 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] Limit on number of columns in SQLite table Hi! On Wed, Oct 16, 2019 at 3:16 PM Hick Gunter wrote: > 100k distinct column names? Or is that 1 repeats of 10 attributes? 100k distinct names. Like each column a different g

Re: [sqlite] [EXTERNAL] Limit on number of columns in SQLite table

2019-10-16 Thread Hick Gunter
100k distinct column names? Or is that 1 repeats of 10 attributes? -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Mitar Gesendet: Mittwoch, 16. Oktober 2019 14:57 An: sqlite-users@mailinglists.sqlite.org Betreff: [EXTER

Re: [sqlite] [EXTERNAL] Re: Last record

2019-10-15 Thread Hick Gunter
Then the first peanut may well be the last one, irrespective of the cardinality of the tin. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Don V Nielsen Gesendet: Dienstag, 15. Oktober 2019 21:52 An: SQLite mailing list B

Re: [sqlite] [EXTERNAL] Last record

2019-10-15 Thread Hick Gunter
The order of rows returned by a query is undefined - i.e. from the point of view of the application, a random member of the result set will be returned last - unless you include an ORDER BY clause that uniquely defines the order of the records to be returned. Given the latter, it is easy to defi

Re: [sqlite] [EXTERNAL] Possible bug in storing text values in numeric columns

2019-10-13 Thread Hick Gunter
You are getting exactly what is documented and exactly what you asked for. Declaring a column NUMERIC means you intend to store NUMBERS. Leading zeros do not change the value of a number. 0012 == 12 unless you have a convention of interpreting a leading zero as indicating octal base. If you nee

Re: [sqlite] [EXTERNAL] Re: Date time input

2019-10-08 Thread Hick Gunter
What it boils down to is asking the data storage layer to perform a presentation layer task. If you insist on solving the problem inside an SQL statement, you can always write your own extension function to "easily" perform the necessary conversion. -Ursprüngliche Nachricht- Von: sqlite

Re: [sqlite] [EXTERNAL] Re: Newbie Issues with COLLATE

2019-10-01 Thread Hick Gunter
I'm guessing that LOCALE_NOCASE will probably be causing things that collate distinct in NOCASE to collate equal, so the risk of breaking UNIQUE constraints seems rather small -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag vo

Re: [sqlite] [EXTERNAL] The LIKE operator and Swift

2019-09-26 Thread Hick Gunter
You can't have a variable inside a pattern. Use like '%' || ? || '%' -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Daniel Odom Gesendet: Donnerstag, 26. September 2019 15:26 An: sqlite-users@mailinglists.sqlite.org Betreff

Re: [sqlite] [EXTERNAL] Group-by and order-by-desc does not work as expected

2019-09-20 Thread Hick Gunter
g list Betreff: Re: [sqlite] [EXTERNAL] Group-by and order-by-desc does not work as expected On Thu, Sep 19, 2019 at 6:15 PM Hick Gunter wrote: > -Ursprüngliche Nachricht- > Von: sqlite-users > [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > Im Auftrag von Fredrik La

Re: [sqlite] [EXTERNAL] Group-by and order-by-desc does not work as expected

2019-09-19 Thread Hick Gunter
-Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Fredrik Larsen Gesendet: Donnerstag, 19. September 2019 17:29 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] Group-by and order-by-desc does not work as expected ...

Re: [sqlite] [EXTERNAL] Group-by and order-by-desc does not work as expected

2019-09-19 Thread Hick Gunter
An ORDER BY clause will omit sorting only if the visitation order exactly fulfills the clause. A GROUP BY clause is able to avoid creating a temporary table if the visitation order exactly fulfills the clause. If a SELECT references only fields present in an index, that (covering) index may be

Re: [sqlite] [EXTERNAL] Re: How to do setdiff but update the original table by the result?

2019-09-18 Thread Hick Gunter
The mathlab function setdiff(a,b) returns the rows from a that are not in b. The equivalent SQL (assuming identical tables a and b) would be SELECT FROM a EXCEPT SELECT FROM b You can then INSERT INTO b SELECT * FROM a WHERE IN (SELECT FROM a EXCEPT SELECT FROM b); DELETE FROM a WHERE IN

Re: [sqlite] [EXTERNAL] Comparison of incompatible types

2019-09-15 Thread Hick Gunter
This is well documented in https://sqlite.org/datatypes.html and https://sqlite.org/datatype3.html -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von kapil Gesendet: Samstag, 14. September 2019 11:15 An: sqlite-users@mailingl

Re: [sqlite] [EXTERNAL] Fastest way to SELECT on a set of keys?

2019-09-13 Thread Hick Gunter
WITH list (key) AS (VALUES (mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Jens Alfke Gesendet: Freitag, 13. September 2019 18:39 An: SQLite mailing list Betreff: [EXTERNAL] [sqlite] Fastest way to SELECT on a set of keys? If I have a set of primary keys (let's say a few hun

Re: [sqlite] [EXTERNAL] Re: How to increase performance when inserting a lot of small data into table using indices

2019-09-10 Thread Hick Gunter
While a write transaction is open, SQLite needs to keep the changed pages in memory. When the size of a transaction (measured in changed pages) exceeds the available memory, SQLite starts to spill the transaction to disk. The optimal transaction size would be just before this occurs, but there i

Re: [sqlite] [EXTERNAL] Re: insert: how to force application to provide value for int primary key?

2019-09-10 Thread Hick Gunter
BTrees as per concept are aware of sorted load vs random load and will adjust their node splitting algorithm accordingly (e.g. 90/10 split for ordered and 50/50 for random load). The rationale being that an ordered load tends to indicate that new data is unlikely or added at the end, whereas a r

Re: [sqlite] [EXTERNAL] Endless loop in update recursive query with UNION ALL

2019-09-06 Thread Hick Gunter
Does your "parent" relationship contain (at least one) loop(s)? UNION will break the loop by eliminating already visited rows, whereas UNION ALL will run faster precisely because it does not keep track of the visited rows. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-

Re: [sqlite] [EXTERNAL] What concurrency level is of sqlite?

2019-09-04 Thread Hick Gunter
Just the same. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Peng Yu Gesendet: Dienstag, 03. September 2019 22:14 An: SQLite mailing list Betreff: [EXTERNAL] [sqlite] What concurrency level is of sqlite? Hi, In other wo

Re: [sqlite] [EXTERNAL] How to create a new table or overwrite an existing table in one command?

2019-09-04 Thread Hick Gunter
There is only IF NOT EXISTS in the CREATE TABLE command. This assumes that you may want to keep a pre-existing table and the data it contains. If you don't care about any old table or ist contents, just issue DROP TABLE IF EXISTS and CREATE TABLE in a single transaction. -Ursprüngliche Nach

Re: [sqlite] [EXTERNAL] Re: http://roaringbitmap.org/

2019-09-02 Thread Hick Gunter
[mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Dominique Devienne Gesendet: Montag, 02. September 2019 13:50 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] Re: http://roaringbitmap.org/ On Mon, Sep 2, 2019 at 12:08 PM Hick Gunter wrote: > Back in 201

Re: [sqlite] [EXTERNAL] INSERT vs BEGIN

2019-09-02 Thread Hick Gunter
For batch loading via script, you should limit the number of values per statement (SQLite compiles each statement into memory) and per transaction (SQLite needs to write to disk after a certain number of pages are modified). For batch loading via program, you can prepare the insert statement for

Re: [sqlite] [EXTERNAL] char(0) with SQLite

2019-09-02 Thread Hick Gunter
Dimensions are ignored by SQLite. A field defined CHAR(0) can hold any length (up to the internal limit) of string. SQlite will only store the actual length of the string plus its contents, no space is wasted. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mail

Re: [sqlite] [EXTERNAL] Re: http://roaringbitmap.org/

2019-09-02 Thread Hick Gunter
Back in 2011 I implemented a virtual table using the "fastbit" library by John Wu of the Lawrence Berekely National Laboratory. This allowed selects of the form SELECT ... FROM WHERE rowid IN (SELECT rowid FROM WHERE ); provided that the data had been inserted before by running INSERT INTO

Re: [sqlite] [EXTERNAL] Non-keyword quoted identifiers parsed as string literals

2019-09-01 Thread Hick Gunter
This is documented behaviour. Use single quotes for literal strings. SQLite will assume you meant 'literlal' if your write "literal" and there is no column of that name. There is no need to quote names in SQLite unless the name contains non-alpha characters. -Ursprüngliche Nachricht- Vo

Re: [sqlite] [EXTERNAL] ORDER BY is ignored during INSERT INTO

2019-08-25 Thread Hick Gunter
Why would you want to do this? If you require a SELECT to return rows in a certain order, you need to specify ORDER BY on the SELECT statement. And not rely on ascending insert time or any other visitation order effect. Additionally - unless specific precautions are taken - sorted insert result

Re: [sqlite] [EXTERNAL] Attached databases and union view.

2019-08-25 Thread Hick Gunter
I think you are looking for UNION ALL to avoid creating an ephemeral table to implement the implied DISTINCT -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Peter da Silva Gesendet: Donnerstag, 22. August 2019 17:28 An: SQL

Re: [sqlite] [EXTERNAL] Re: Schema updates across threads in WAL & multithread mode

2019-08-19 Thread Hick Gunter
As already stated, this looks like you have at least one transaction underways. Your schema change will become visible only after 1) they are committed on ther "writer" connection AND 2) a new transaction is started on the "reader" connection If your "readers" are failing to reset or finalize an

Re: [sqlite] [EXTERNAL] Re: Getting a notification when a write lock is released.

2019-08-16 Thread Hick Gunter
Maybe you are looking for semaphores. These can be tricky to use correctly in the case of cooperating processes, where you have to handle the case of the current owner of the semaphore terminating within the monitored section of code. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:s

Re: [sqlite] [EXTERNAL] Re: Documentation update request

2019-08-16 Thread Hick Gunter
Reminds me of "... two mice ran up the clock, the clock struck one, and the other escaped with minor injuries" -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Kevin Benson Gesendet: Donnerstag, 15. August 2019 20:40 An: SQL

Re: [sqlite] [EXTERNAL] Re: Programming methodology (was DEF CON (wasL A license plate of NULL))

2019-08-13 Thread Hick Gunter
How about #define is_true(tf) ((uintptr_t)0 != (uintptr_t)(tf)) -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Don V Nielsen Gesendet: Dienstag, 13. August 2019 22:42 An: SQLite mailing list Betreff: [EXTERNAL] Re: [sqlit

Re: [sqlite] [EXTERNAL] Re: DEF CON (wasL A license plate of NULL)

2019-08-13 Thread Hick Gunter
But surely any compiler worth ist salt would optimize away all of that code and just use the result of the expression given as argument in the call ;) -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von James K. Lowden Gesendet

Re: [sqlite] [EXTERNAL] Re: Correct use of sqlite3_vtab_nochange/sqlite3_value_nochange

2019-08-13 Thread Hick Gunter
ble api which multiple modules are then built on top of, it doesn't know which columns are large/expensive, so the idea was just to use call sqlite3_vtab_nochange for all of them, which includes the primary key. > On 13 Aug 2019, at 13:00, Hick Gunter wrote: > > Very strange and

Re: [sqlite] [EXTERNAL] Correct use of sqlite3_vtab_nochange/sqlite3_value_nochange

2019-08-13 Thread Hick Gunter
Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Kevin Martin Gesendet: Dienstag, 13. August 2019 13:23 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] Correct use of sqlite3_vtab_nochange/sqlite3_value_nochange > On 12 Aug 2019, a

Re: [sqlite] [EXTERNAL] Why does WAL prevent the need for async IO?

2019-08-13 Thread Hick Gunter
I don't think so. Async IO module creates a queue of pages that will be written to the database file on disk according to available IO bandwidth. WAL mode creats a queue of pages from committed transactions that are written to the database file on disk according to available IO bandwidth. Both

Re: [sqlite] [EXTERNAL] SQLITE_BUSY (SQLITE_BUSY_SNAPSHOT) due to a simple SELECT query -- Documentation needed!

2019-08-12 Thread Hick Gunter
So how do you propose to have consistency and isolation if SELECT does not create an automatic transaction if no explicit transaction exists? Consider: SELECT FROM ; BEGIN; UPDATE SET field = +1; COMMIT; If the SELECT and UPDATE statements are not part of the same transaction, there is no g

Re: [sqlite] [EXTERNAL] SQLITE_BUSY (SQLITE_BUSY_SNAPSHOT) due to a simple SELECT query -- Documentation needed!

2019-08-12 Thread Hick Gunter
Works as intended. "our code base does not use transactions at all" does NOT mean that there are no transactions, just that SQLite uses *implicit* transactions, i.e. every statement is in it's own transaction. "we can share a connection between threads as long as we don't read/write into the s

Re: [sqlite] [EXTERNAL] Correct use of sqlite3_vtab_nochange/sqlite3_value_nochange

2019-08-11 Thread Hick Gunter
To correctly determine what SQLite is asking of your xUpdate routine requires looking at argc, argv[0] and possibly argv[1] (if argc > 1). You did not state your argc and argv[0] values, so looking at the documentation would suggest that SQLite is actually asking for an INSERT into a WITHOUT ROW

Re: [sqlite] [EXTERNAL] select for power-meter accumulated total readings

2019-08-08 Thread Hick Gunter
I see two subproblems in this query a) estimating total electricity consumption for points in time that do not have an entry b) generating regular points in time ad a) assume a linear consumption of power between two measurements So for a time tx that is between ta and tb with values of pa and

Re: [sqlite] [EXTERNAL] Re: Problem with int and DateTime types with EntityFrameWorkCore.Sqlite

2019-08-08 Thread Hick Gunter
There is no decimal type in SQLite, and you are lucky that they are converted to string instead of real, because you cannot do proper (implied) decimal (point) arithmetic with real (ieee binary floating point) values. The rounding errors intruduced by binary floating point not beeing able to rep

Re: [sqlite] [EXTERNAL] mode insert dumps

2019-08-02 Thread Hick Gunter
You need to .mode insert SELECT * FROM ; Repeat for all your tables. -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Luca Ferrari Gesendet: Freitag, 02. August 2019 10:04 An: SQLite Betreff: [EXTERNAL] [sqlite] mode inse

Re: [sqlite] [EXTERNAL] Error in recover sqlite3 database

2019-08-01 Thread Hick Gunter
The error is due to a full disk. You should not be deleting files associated with an SQLite db file. Have you tried running pragma integrity_check(); before the disk actually becomes full? Other than corruption of the file, the two candidates are internal fragmentation (doing lots of INSERT an

Re: [sqlite] [EXTERNAL] Error in recover sqlite3 database

2019-07-31 Thread Hick Gunter
What is this tbllog table? -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von bhandari_nikhil Gesendet: Donnerstag, 01. August 2019 07:39 An: sqlite-users@mailinglists.sqlite.org Betreff: [EXTERNAL] [sqlite] Error in recover sq

Re: [sqlite] [EXTERNAL] storing blobs in a separate table

2019-07-31 Thread Hick Gunter
SQLite stores rows in a compressed format that requires decoding. To access the nth field, all the fields that come before it need to be decoded. If there is a large blob stoed in a blob field, any field after that will suffer a performance penalty (unless, of course, both fields are required).

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-31 Thread Hick Gunter
s the best way to go for me, not for the sqlite library, that's why I writing to the sqlite library. Il giorno mar 30 lug 2019 alle ore 15:50 Hick Gunter ha scritto: > f) There are exactly 2 documented functions in your code. Did you not > read their documentation??? > > S

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Hick Gunter
f) There are exactly 2 documented functions in your code. Did you not read their documentation??? See https://sqlite.org/c3ref/column_blob.html " After a type conversion, the result of calling sqlite3_column_type() is undefined, though harmless. Future versions of SQLite may change the behavior

Re: [sqlite] [EXTERNAL] sqlite3_exec without ubiqitous text conversions

2019-07-30 Thread Hick Gunter
What you are doing is a very bad idea indeed. a) you are circumventing the intended interface b) you are breaking encapsulation, because columnMem returns a pointer to an internal type, which is useless to you, unless you have made public all the SQLite internals c) you are assuming that type co

Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect

2019-07-29 Thread Hick Gunter
y own code. From: sqlite-users on behalf of Hick Gunter Sent: Monday, July 29, 2019 10:53:02 AM To: 'SQLite mailing list' Subject: Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect Let's go back to your example statement with your join of two tables. selec

Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect

2019-07-29 Thread Hick Gunter
Let's go back to your example statement with your join of two tables. select b,c from tbl0 join tbl1 where tbl0.a = tbl1.a and tbl0.a > ?1; SQLIte should ask the authorizer the following questions: 1) SELECT 2) READ tbl0 3) READ tbl1 4) READ tbl0 field a 5) READ tbl1 field a 6) READ tbl0 field b

Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect

2019-07-29 Thread Hick Gunter
That strikes me as purely procedural thinking. Does the set of allowed operations really depend on the order of the requests (which probably depends on the query plan)? E.g. "you can update this field of this table only if you read this other field from that other table *first*"? -Ursprüngl

Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect

2019-07-28 Thread Hick Gunter
Note that parsing debug output is not a stable method of analysis (meaning SQlite Dev can change anything at whim), whereas the authorizer interface is documented. Your implicit claim is "not all instances of column reference are reported to the authorizer, notably those inside a USING clause".

Re: [sqlite] [EXTERNAL] Help with sqlite3TreeViewSelect

2019-07-28 Thread Hick Gunter
WITH is basically syntactic sugar that allows you to name the result set of a certain select and refer to it by name, so that select has to appear in the generated bytecode and also in the query resolution tree. Guessing what an element of the query resolution tree does would be very much easie

Re: [sqlite] [EXTERNAL] Virtual Tables xConnect Called When?

2019-07-24 Thread Hick Gunter
This assumes that there is some kind of backing store that needs to be created once (xCreate) but may be conncted to (xConnect) later. CREATE VIRTUAL TABLE calls the xCreate function (and DROP TABLE will call xDestroy), whereas queries will call the xConnect function. IIRC connecting to an SQLi

Re: [sqlite] [EXTERNAL] Re: Estimated Costs and Memory DBs

2019-07-24 Thread Hick Gunter
f: Re: [sqlite] [EXTERNAL] Re: Estimated Costs and Memory DBs On Wed, Jul 24, 2019 at 10:45 AM Hick Gunter wrote: > The speed of a virtual table depends on the backing store and software > used to implement it. > [DD] Sure. virtual-tables can also access the disk and do expensive things

Re: [sqlite] [EXTERNAL] Re: Estimated Costs and Memory DBs

2019-07-24 Thread Hick Gunter
The speed of a virtual table depends on the backing store and software used to implement it. We have virtual tables that reference CTree files as well as virtual tables that reference memory sections here. The advantage is that the VT implementation can adjust it's answers in the xBestIndex fun

Re: [sqlite] [EXTERNAL] Determining valid statement placeholders.

2019-07-22 Thread Hick Gunter
This cannot be determined programatically. The query generator stores an OP_Variable opcode when the SQL program needs to access the contents of an SQL parameter, and keeps track of the highest parameter number used, which determines the size of the parameter array. Even if you were to examine

  1   2   3   4   5   6   7   8   9   10   >