Re: [sqlite] Maintaing a Last updated timestamp column using triggers

2020-03-12 Thread David Raymond
> Alternatively, you can use the new GENERATED ALWAYS AS (...) STORED to make > it an automatically updated stored field and you do not need triggers at all, > just a version of SQLite3 that does generated columns (version 3.31.0 from > 2020-01-22 or later). > > create table MyData > ( > id

Re: [sqlite] Maintaing a Last updated timestamp column using triggers

2020-03-12 Thread David Raymond
> What stops the > UPDATE ports SET timeofday = CURRENT_TIMESTAMP WHERE id = NEW.id ; > from also triggering the AFTER UPDATE ON recursively? > > Perhaps a pragma or inSQLite are triggers non-recursive by default? > > I am using (now I have by semi-colons right) > CREATE TRIGGER tgrUpdate AFTER

Re: [sqlite] How to open an sqlite3 for reading without being blocked by process writing to it?

2020-03-09 Thread David Raymond
I see this. What does it mean? Does it mean even when the sqlite3 session is closed there is still -wal and -shm left on the disk? """ There is an additional quasi-persistent "-wal" file and "-shm" shared memory file associated with each database, which can make SQLite less appealing for use as

Re: [sqlite] After column add, what should be done to update the schema?

2020-02-28 Thread David Raymond
"I read somewhere that the DDL is really just a copy of the text when teh table was created" It starts that way, but the ALTER TABLE command will update that string when it runs so that it reflects the new column. Did you commit your changes after the alter table and before looking for the

Re: [sqlite] Fwd: inserting new data only

2020-02-25 Thread David Raymond
Awesome---exactly what's needed. The monotonicity of the time key variable is assured by how the data is collected---but is there a way to express that in sqlite? create table data ( key text primary key check (julianday(key) > julianday(select max(key) from data), data integer not null);

Re: [sqlite] Fwd: inserting new data only

2020-02-25 Thread David Raymond
A before trigger which uses the raise function would stop it from getting inserted in the first place. create trigger cull before insert on tbl when new.value = (select value from tbl order by time desc limit 1) begin select raise(ignore); end; Or if you want it to actually return an error to

Re: [sqlite] JSON_EACH + recursive query = unexpected performance degradation

2020-02-12 Thread David Raymond
Not necessarily related to the question itself, but how did the attachments actually come through with this mail? Every single other person to try and attach something to this list has had it stripped off. -Original Message- From: sqlite-users On Behalf Of ? Sent:

Re: [sqlite] UPSERT documentation question

2020-02-07 Thread David Raymond
> CREATE TABLE vocabulary(word TEXT PRIMARY KEY, count INT DEFAULT 1); > INSERT INTO vocabulary(word) VALUES('jovial') > ON CONFLICT(word) DO UPDATE SET count=count+1; > > Shouldn't that actually be written as "vocabulary.count+1"? Nope. Unqualified names there refer to the one and only record

Re: [sqlite] Unsubscribe from Sqlite users list

2020-02-05 Thread David Raymond
Follow the link at the bottom of every email. The bottom box/button on the page is for unsubscribing. http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users -Original Message- From: sqlite-users On Behalf Of Ainhoa B Sent: Wednesday, February 5, 2020 9:11 AM To: SQLite

Re: [sqlite] Table has different rootpage in EXPLAIN and sqlite_master

2020-02-03 Thread David Raymond
They shouldn't be different, no. Do you have a copy of the weird version of the database still, or have a copy of the explain text and the sqlite_master contents? Was there another index or table whose root page is what was listed in the explain output? For example, were you expecting it to use

Re: [sqlite] log() in sqlite3

2020-01-31 Thread David Raymond
The core functions of the library are fairly unlikely to get expanded upon at this point, so it's up to the application using the SQLite library to expand on them. You can load an extension which has those functions. Or the C API gives you means to add your own functions. For Python, the

Re: [sqlite] Default isolation_level for sqlite3.connect?

2020-01-31 Thread David Raymond
> Is there a complete list that defines what are "the statements that > modify the database"? What is the difference between "the statements > that modify the database" and "Data Modification Language (DML) > statement"? Are they the same? If you dig into their code you can attempt to find what

Re: [sqlite] conditional insert operations

2020-01-31 Thread David Raymond
e: [sqlite] conditional insert operations h1, h2 pair are unique. I don't quite understand your example. Could you explain what it does in plain English so that I can be sure it does what I want? (What is v?) On 1/30/20, David Raymond wrote: > Is the combo of h1, h2 unique? If so you could do

Re: [sqlite] Is Comment column in 'EXPLAIN' blank?

2020-01-31 Thread David Raymond
To get the comments the library needs to have been compiled with SQLITE_ENABLE_EXPLAIN_COMMENTS flag https://www.sqlite.org/compile.html#_options_to_enable_features_normally_turned_off That setting is off by default, but for the pre-compiled CLI they do include it. I'm not seeing it in the list

Re: [sqlite] conditional insert operations

2020-01-30 Thread David Raymond
d to h1,h2,v a,,X a,A,1 a,B,2 ... On 1/30/20, David Raymond wrote: > I'm not quite following what you're trying to do here. Could you provide a > few examples of "here's what used to be in there", "here's what I want to > insert", "here's what it should like in

Re: [sqlite] Generated columns and COLLATE in the AS parens

2020-01-30 Thread David Raymond
From a Windows installation of 12.0 testing=> select version(); version PostgreSQL 12.0, compiled by Visual C++ build 1914, 64-bit (1 row) Time: 0.283 ms testing=> create table foo ( testing(> a text,

Re: [sqlite] conditional insert operations

2020-01-30 Thread David Raymond
I'm not quite following what you're trying to do here. Could you provide a few examples of "here's what used to be in there", "here's what I want to insert", "here's what it should like in the end" -Original Message- From: sqlite-users On Behalf Of Peng Yu Sent: Thursday, January

Re: [sqlite] Default isolation_level for sqlite3.connect?

2020-01-30 Thread David Raymond
Incorrect. "Not specifying one" is sqlite3.connect(fi) And the connection will still start implicit transactions for you. (with "begin;") "Setting it to None" is sqlite3.connect(fi, isolation_level = None) Which will turn off all implicit transactions, put it in autocommit mode, and you have

Re: [sqlite] Does .commit() ensure the .execute()'s and .executemany()'s called before are run atomically?

2020-01-30 Thread David Raymond
That's just my own personal paranoia wanting to make sure the cursor always gets closed, even on an exception. As I don't think the normal context manager on a cursor closes it when it exits. In the real world it's probably overkill as a) The destructors probably take care of that b) It's the

Re: [sqlite] Default isolation_level for sqlite3.connect?

2020-01-30 Thread David Raymond
https://docs.python.org/3.8/library/sqlite3.html#sqlite3-controlling-transactions "You can control which kind of BEGIN statements sqlite3 implicitly executes via the isolation_level parameter to the connect() call, or via the isolation_level property of connections. If you specify no

Re: [sqlite] Either a bug or I don't understand SQL update

2020-01-30 Thread David Raymond
The mailing list strips all attachments, so you'll have to either provide a link to it on some external source, or give more create table/insert statements like at the bottom. Without having all the data I'll say: Remember to use single quotes for text literals. You have ...and

Re: [sqlite] Does .commit() ensure the .execute()'s and .executemany()'s called before are run atomically?

2020-01-30 Thread David Raymond
-Original Message- From: sqlite-users On Behalf Of Peng Yu Sent: Thursday, January 30, 2020 5:16 AM To: SQLite mailing list Subject: Re: [sqlite] Does .commit() ensure the .execute()'s and .executemany()'s called before are run atomically? Could you show a python example on how to

Re: [sqlite] Is sqlite3 smart enough to remove redundant command?

2020-01-29 Thread David Raymond
It's going to run both since you asked it to. Even if it was in a compiled language where the compiler could look ahead and was looking to implement that sort of optimization, then for example there still might be triggers on the table which would need to be run, or other constraints on the

Re: [sqlite] SQL CASE WHEN THEN ELSE END

2020-01-27 Thread David Raymond
This is technically valid CASE syntax which is why you're not getting an error, it's just not what you're looking for. ... CASE (SELECT c.WYear FROM t2 WHERE pid = a.a) WHEN c.WYear = 2020 THEN “YES” ELSE “NO” END ) AS DIGITAL ... What that is saying is take the value you get from this: (SELECT

Re: [sqlite] Capturing the changes in columns in a table

2020-01-14 Thread David Raymond
A note and a question on subqueries. On reading: select ... (select b from t where a == new.a and idate < new.idate order by idate desc) as oldv, ... My brain started yelling that that needed a "limit 1" on the subquery so that it would only return 1 row. I looked in the SQLite docs though

Re: [sqlite] Unexplained table bloat

2020-01-13 Thread David Raymond
Well, I believe this is the relevant bit from the docs for binding: https://www.sqlite.org/c3ref/bind_blob.html "If a non-negative fourth parameter is provided to sqlite3_bind_text() or sqlite3_bind_text16() or sqlite3_bind_text64() then that parameter must be the byte offset where the NUL

Re: [sqlite] Unexplained table bloat

2020-01-10 Thread David Raymond
Well something's weird anyway. When I open it with the command line tool it queries it just fine. I tried to open it in Python to check all the characters in the strings and got this: Traceback (most recent call last): File "...\Testing4.py", line 8, in cur.execute("select * from

Re: [sqlite] How create database, tables,indices...

2020-01-07 Thread David Raymond
To create a new database you just open the file name you want, and if it doesn't exist it will be created. The actual new file creation is slightly delayed until it has to write something to the disk. There are a couple of options that can only be set at file creation time like page size, text

Re: [sqlite] Size limits

2020-01-07 Thread David Raymond
Along with the other replies already mentioned I'll pipe in with a reminder that any large blob fields should be the final field in a table's definition. Due to how the data is stored, to get the data for any particular field, SQLite has to go through/decode the data for all previous fields in

Re: [sqlite] Difference between hex notation and string notation

2019-12-13 Thread David Raymond
Run... pragma compile_options; ...and see if LIKE_DOESNT_MATCH_BLOBS is in the list that comes up. If it is NOT in the list then both rows will show up. This is the case for the precompiled Windows cli for example. If it IS in the list, then only the second one that was inserted as text will

Re: [sqlite] wal mode

2019-12-06 Thread David Raymond
"So it suffices that I run "PRAGMA journal_mode=WAL;" once from say the sqlite3 cli, for all future connections from any tool will use WAL mode for this database file?" Yup, the journal mode is stored in the database header. So the pragma will update the file's header, and any new connection

Re: [sqlite] Changes in version 3.28.0 makes sql run slower

2019-12-06 Thread David Raymond
Those lines on their own aren't any significantly slower or faster than each other. What's happening is that when you add in a function the planner has a harder time determining if it can use certain optimizations. The "is not null" version is simple enough where the planner says "ohh, I'm ok

Re: [sqlite] Query freezing on latest SQLite 3.30.1 build

2019-12-05 Thread David Raymond
The mailing list strips off all attachments, so you'll have to provide another place to get that. Alternatively could you post the schema here as text, along with the explain query plan output from the slow version and from a fast version? -Original Message- From: sqlite-users On

Re: [sqlite] [EXTERNAL] Re: what is the optimized way to do the vaccum?

2019-12-05 Thread David Raymond
SQLite is pretty good at using free space inside the file. So if inserting something is going to run you out of space, then it's going to run you out of space whether the file was previously vacuumed or not. Also reminder that when vacuum is run, SQLite makes a brand new copy of the database,

Re: [sqlite] PRAGMA for .load

2019-12-03 Thread David Raymond
I _think_ the load_extension() function is what you'll be looking for, though I could be wrong. https://www.sqlite.org/lang_corefunc.html#load_extension -Original Message- From: sqlite-users On Behalf Of Jose Isaias Cabrera Sent: Tuesday, December 3, 2019 3:38 PM To:

Re: [sqlite] [EXTERNAL] Slow joining of tables with indexes

2019-11-26 Thread David Raymond
Not the reason for the slowdown, but note that both of these are redundant: CREATE INDEX IF NOT EXISTS data_table__data_id__pk_idx ON data_table ( data_id ); CREATE INDEX IF NOT EXISTS ignore_me__ignored_id__pk_idx ON ignore_me ( ignored_id ); ...because you declared them as the

Re: [sqlite] Shell commands for controlling headers

2019-11-25 Thread David Raymond
Dr Hipp replied to this 2 days ago with this: Documentation fix https://www.sqlite.org/docsrc/info/a2762f031964e774 will appears in the next release. ".header" is an abbreviation for ".headers" and does exactly the same thing. -Original Message- From: sqlite-users On Behalf Of John

Re: [sqlite] LIKE IN

2019-11-25 Thread David Raymond
names.name like '%'||query||'%' > ); > > On Fri, 22 Nov 2019 at 15:19, David Raymond > wrote: > >> Or alternatively something like: >> >> select * from table >> where exists ( >> select query from queries >> where table.name like query >> ); >>

Re: [sqlite] LIKE IN

2019-11-22 Thread David Raymond
Or alternatively something like: select * from table where exists ( select query from queries where table.name like query ); ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Optimising query with aggregate in subselect.

2019-11-20 Thread David Raymond
"There's a small sidenote (that I'm too lazy too find right now) in the select docs that mentions that, in case of using min or max as aggregate, the non-aggregate columns will come from the row that held the min/max value." Look in https://www.sqlite.org/quirks.html under "6. Aggregate

Re: [sqlite] How to use CASE statement to SUM() some numbers

2019-11-15 Thread David Raymond
So why do you need a case? What will not work with the simple: select sum(Expense) from Expenses where Date between date('now', '-1 months') and date('2019-11-04', '-1 days'); ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Specific sqlite_autoindex_* missing in recent SQLite versions

2019-11-14 Thread David Raymond
Apparently it got smarter about "primary key unique" in 3.20.0 and stopped making the extra index when it's a without rowid table. Don't see anything about it in the release notes though. Even on the current release "primary key unique" will still make an extra index for the unique if it's a

Re: [sqlite] Why do these 2 updates give different results?

2019-11-13 Thread David Raymond
Why not change it to something along the lines of: UPDATE QR3PARAMS SET ED = 1 WHERE ED is not 1 AND EXISTS ( SELECT 1 FROM CURRENT_MED WHERE ID = QR3PARAMS.ID AND ( TERM_TEXT GLOB 'Sildenafil*' OR TERM_TEXT GLOB 'Tadalafil*' OR TERM_TEXT GLOB 'Vardenafil*' ) );

Re: [sqlite] Why do these 2 updates give different results?

2019-11-13 Thread David Raymond
"Why is ED changed to '' or NULL for ID 5?" When you update to a subquery which returns no rows, then the field gets updated to null. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Deterministic random sampling via SELECT

2019-11-07 Thread David Raymond
ge- From: sqlite-users On Behalf Of Merijn Verstraaten Sent: Thursday, November 7, 2019 2:16 PM To: SQLite mailing list Subject: Re: [sqlite] Deterministic random sampling via SELECT > On 7 Nov 2019, at 19:16, David Raymond wrote: > > Along those lines SQLite includes the rev

Re: [sqlite] Deterministic random sampling via SELECT

2019-11-07 Thread David Raymond
: [sqlite] Deterministic random sampling via SELECT On 7 Nov 2019, at 1:56pm, David Raymond wrote: > Others will correct me if I'm wrong on that. No correction, but I wanted to add something. According to the theory of how SQL (not just SQLite, SQL) works, tables have no order. You

Re: [sqlite] Deterministic random sampling via SELECT

2019-11-07 Thread David Raymond
"So, is this behaviour documented/guaranteed somewhere?" Short version is: Nope. The engine is free to do whatever it wants as long as it gives the correct result in the end. Consider a simple select * from foo where predicate order by non_indexed_field; Since there is no nice ordering of the

Re: [sqlite] Find stmt that's stopping a DB from closing

2019-11-05 Thread David Raymond
I think you're referring to "The SQLITE_STMT Virtual Table": https://www.sqlite.org/stmt.html -Original Message- From: sqlite-users On Behalf Of x Sent: Tuesday, November 5, 2019 2:27 PM To: sqlite-users@mailinglists.sqlite.org Subject: [sqlite] Find stmt that's stopping a DB from

Re: [sqlite] DELETE extremely slow

2019-11-01 Thread David Raymond
Looks like you have triggers going on there. You only gave us the table and index definitions. What are the on delete triggers you have? -Original Message- From: sqlite-users On Behalf Of Thomas Kurz Sent: Thursday, October 31, 2019 6:54 PM To: SQLite mailing list Subject: Re:

Re: [sqlite] DELETE extremely slow

2019-10-31 Thread David Raymond
It's not gonna account for 88 minutes, but out of curiosity is there a way to separately report the "delete and commit" time from the "dang this WAL needs to be checkpointed now" time? -Original Message- From: sqlite-users On Behalf Of Thomas Kurz Sent: Thursday, October 31, 2019

Re: [sqlite] DELETE extremely slow

2019-10-31 Thread David Raymond
"Four observations that may (or may not) explain the problem: - dataset.id declared "unique": useless since the primary key is unique by definition but it may create an index (not checked). A PK is not "more unique" if an additional "unique" constraint is declared." Declaring it as both

Re: [sqlite] Multiple files for a single SQLite database

2019-10-30 Thread David Raymond
"There's a limit, somewhere around ten, I think. Though you can increase it by explicitly messing with limits and compilation options." Default is 10, can be raised to 125. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] What is the default value of isolation_level?

2019-10-25 Thread David Raymond
https://docs.python.org/3.7/library/sqlite3.html#controlling-transactions "If you specify no isolation_level, a plain BEGIN is used, which is equivalent to specifying DEFERRED." I believe the empty string qualifies as "no isolation_level" for this. -Original Message- From:

Re: [sqlite] What does commit actually do?

2019-10-25 Thread David Raymond
"I just use the default .connect() without specifying isolation_level explicitly. Then I am in a transaction?" E, umm, well. Even regular users may not know the answer to that do to "automatic stuff behind the scenes" which may have corner cases or maybe bad documentation. Which is

Re: [sqlite] What is the default value of isolation_level?

2019-10-25 Thread David Raymond
https://www.sqlite.org/lang_transaction.html Depending on how they're implementing it, one could argue that they're just copying the specs for SQLite and saying "if you're not gonna specify it, then we're not gonna specify it, and we're just gonna let the SQLite library do with it as it pleases

Re: [sqlite] What does commit actually do?

2019-10-25 Thread David Raymond
"https://docs.python.org/2/library/sqlite3.html; Also, please consider going to Python 3 instead of 2. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] What does commit actually do?

2019-10-25 Thread David Raymond
This question could span the entire range of experience level from beginner to expert. When you're in a transaction, .commit() says "make everything done in this transaction on this connection permanent." If there are any crashes or improper disconnections in the middle of a transaction then

Re: [sqlite] ALTER TABLE ADD COLUMN

2019-10-23 Thread David Raymond
On disk a record basically looks like: Here are 5 values: value1, value2, value3, value4, value5 If your query is looking for the 6th, 7th or 8th field and the record on the disk only has 5, then it goes " I guess they should be the default values for the missing fields." What that means is

Re: [sqlite] Can SQLite do this in a single query?

2019-10-22 Thread David Raymond
Or something like this using the new filter clause from 3.30. As written it should show it with two decimal points. (not tested for typos) select CITY_TXT as CITY, round(100.0 * total(Employee_Count) filter (where MODE_TXT = 'WALKING') / total(Employee_Count), 2) as WALKING, round(100.0 *

Re: [sqlite] Getting "chunked" output from a large SELECT operation.

2019-10-16 Thread David Raymond
What language/library are you using? In Python for example there's .fetchone() to get just the next result row, .fetchmany(n) to get the next n rows, or .fetchall() to go get them all. In general though at its core SQLite will get and return one row at a time. Though if there's grouping or

Re: [sqlite] Last record

2019-10-16 Thread David Raymond
"Keith, what if one has a peanut allergy?" Well, the maid dutifully logs the changes she makes to the tin, so that in the event of an anaphylactic crash the tin can be returned to its original state. This helps ensure we have ACID peanuts. ___

Re: [sqlite] insertion of a single record taking more than 200 msec some times

2019-10-11 Thread David Raymond
Without more info it's impossible to say. The immediate possibility coming to mind would be if you're running in WAL mode, and that happened to be the commit that triggered a checkpoint, meaning you have both the commit time and the checkpoint time in there. Depending on how much there was to

Re: [sqlite] Opposite of SQLite

2019-10-10 Thread David Raymond
SQLephantine -Original Message- From: sqlite-users On Behalf Of Ned Fleming Sent: Thursday, October 10, 2019 2:55 PM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] Opposite of SQLite > Someone asked: > >>> What the opposite of "Lite”? > SQLessLite -- Ned

Re: [sqlite] SQLite version 3.30.0 in about two weeks.

2019-09-26 Thread David Raymond
Looks good. One request that popped to mind while looking at this: Would you expand the documentation on the index_info and index_xinfo pragmas to include what they return when run on an expression index? -Original Message- From: sqlite-users On Behalf Of Richard Hipp Sent:

Re: [sqlite] Documentation bug: CREATE TABLE diagram misplaces comma

2019-09-18 Thread David Raymond
It is comma then table-constraint. Check the direction of the arrow on that loop. You go past it on the main line, go down into the ,-circle, then follow the arrow left into the table-constraint box, then continue the loop clockwise to meet back up with the main line. -Original

Re: [sqlite] Differentiate between an empty result set and an error using Python3

2019-09-04 Thread David Raymond
If you run a query that returns no results, then cursor.fetchone() will return None, or cursor.fetchall() will return an empty list. If there is an error during the processing then some sort of exception should be raised. -Original Message- From: sqlite-users On Behalf Of Rob Sciuk

Re: [sqlite] How to create a new table or overwrite an existing table in one command?

2019-09-04 Thread David Raymond
> If you delete the database file then make sure you also delete any other > files that might have been associated with it, such as left over journals > and so forth. I never see those extra files in practice. Are they guaranteed to be deleted automatically once an SQLite session is finished?

Re: [sqlite] Window functions

2019-09-04 Thread David Raymond
Kind of annoying that when the author shows a screenshot of the sample data he's using for his queries that he doesn't include 2 of the fields that are in the queries. Makes it harder to "play along at home" For their ntile example (on page2) I don't think I've seen a window function used with

Re: [sqlite] error: unable to read database header

2019-08-30 Thread David Raymond
When you run ".open test.db" if the database doesn't already exist, then the file is created. However *no data is written yet* as there are a few permanent things you could change at the very start (like page size, encoding, etc), so it holds off on writing the first page with the header until

Re: [sqlite] .save always errors: database is locked

2019-08-30 Thread David Raymond
Running sqlite3 will open up an in-memory database to start with. ".open test" will either open the file "test" if it exists, or create the file if it doesn't. You will then be working on that file called "test" which is your database. The help line for .save is: .save FILE Write

Re: [sqlite] Unexpected REINDEX behavior.

2019-08-30 Thread David Raymond
Sorry if my mind is working slowly today, but why are those showing up as a difference when they're exactly the same line? The only difference between both runs: > $ diff run1 run2 > 1260d1259 > < INSERT INTO space_used >

Re: [sqlite] Query for Many to Many

2019-08-27 Thread David Raymond
It does support natural joins. USING needs parenthesis around the column list: ...using (author_id)...using (book_isbn)... -Original Message- From: sqlite-users On Behalf Of Dominique Devienne Sent: Tuesday, August 27, 2019 10:08 AM To: SQLite mailing list Subject: Re: [sqlite]

Re: [sqlite] Query for Many to Many

2019-08-27 Thread David Raymond
The basic query is going to be the below select stuff from books inner join author_books on author_books.book_isbn = books.book_isbn inner join author on author_books.author_id = author.author_id where things; -Original Message- From: sqlite-users On Behalf Of dboland9

Re: [sqlite] Attached databases and union view.

2019-08-22 Thread David Raymond
I don't know how smart the planner is, but as a thought, would UNION ALL make any improvement over just UNION? With just UNION it has to de-duplicate all the subquery results whereas with UNION ALL it would be free to separate all the various subqueries from each other. Or do you actually need

Re: [sqlite] conditional trigger

2019-08-22 Thread David Raymond
Your create view had 2 typos in it by the way. Here's a fixed version for those copying and pasting: CREATE VIEW select_file_dirs AS SELECT dir_paths.dir_path AS dir_path, files.f_name FROM dir_paths INNER JOIN files ON files.dir_id = dir_paths.id;

Re: [sqlite] COMMIT, what locking behaviour when current lock is merely SHARED?

2019-08-09 Thread David Raymond
>> A connection holds a SHARED lock. It did start a DEFERRED transaction then >> read something. Now it executes COMMIT. >You are one of the rollback journaling modes, not WAL mode, right? >Different rules apply for WAL mode. > >In rollback mode, the connection automatically promotes to EXCLUSIVE

Re: [sqlite] THREADSAFE (1, SERIALIZED) vs (2, MULTITHREAD)

2019-08-06 Thread David Raymond
>"But you do need WAL to achieve multiple readers concurrency..." Nope, you can have concurrent readers with rollback journal mode. You just can't have anyone writing while they're all reading. (Or I may just be misunderstanding what you meant) ___

Re: [sqlite] [EXTERNAL] storing blobs in a separate table

2019-08-01 Thread David Raymond
https://www.sqlite.org/fileformat2.html is the page with the nitty gritty for the file format. Overflow pages are stored in a singly linked list of pages, so you have to traverse through all of the pages to get to the end. So while you may know right away that you want the 20th overflow page

Re: [sqlite] Quick way to determine optimal page size?

2019-07-31 Thread David Raymond
Not that I'm aware of no. How much of a difference are you seeing for your database size depending on the page size you try? -Original Message- From: sqlite-users On Behalf Of Tony Papadimitriou Sent: Wednesday, July 31, 2019 3:29 PM To: General Discussion of SQLite Database Subject:

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread David Raymond
mple. How sure are you that "any SELECT that reads from the DB file starts a read transaction"? Does the read transaction read from a snapshot of the entire DB, or are only specific tables in the read snapshot? On Tue, Jul 30, 2019 at 7:14 PM David Raymond wrote: > To get the

Re: [sqlite] Explicit "read transaction" with journal_mode=WAL.

2019-07-30 Thread David Raymond
To get the read lock you're going to need to read something from the database file. I think this page is your best bet: https://www.sqlite.org/lang_transaction.html "Transactions can be deferred, immediate, or exclusive. The default transaction behavior is deferred. Deferred means that no

Re: [sqlite] I can insert multiple rows with the same primary key when one of the value of the PK is NULL ...

2019-07-19 Thread David Raymond
Well, yes and no. I see that as more of a generic question of "why is some rogue process accessing and changing your database?" rather than a problem specific to SQLite. If your data needs foreign keys, or some extension like FTS, R-Tree, etc. then you're going to be controlling what's

Re: [sqlite] I can insert multiple rows with the same primary key when one of the value of the PK is NULL ...

2019-07-18 Thread David Raymond
"integer primary key"s cannot contain a null since they're an alias for the rowid. So when you insert a null into them they act similar to autoincrement and automatically fill it in with an unused id. (Current implementation is 1 more than the _current_ highest rowid. Subject to change) If you

Re: [sqlite] how to delete BLOB object from the data base

2019-07-15 Thread David Raymond
The questions coming to mind at the moment: What is the schema of the table holding the BLOBs? What is the normal size for the blobs? How are you doing the inserts and deletes? What journal mode are you using? I would think normal way to delete a record is the simple delete from blob_table

Re: [sqlite] Grammar police

2019-07-11 Thread David Raymond
Other small ones from the Quirks page: Section 2: "to realize the SQLite is not intended as" to realize [that] SQLite is not intended as Section 3.2: "SQLite as no DATETIME datatype." SQLite [has] no DATETIME datatype Section 5: "Due to an historical oversight" Due to [a] historical

[sqlite] FW: [sqlite-announce] Version 3.29.0

2019-07-11 Thread David Raymond
And there was great rejoicing: "1. Added the SQLITE_DBCONFIG_DQS_DML and SQLITE_DBCONFIG_DQS_DDL actions to sqlite3_db_config() for activating and deactivating the double-quoted string literal misfeature. Both default to "on" for legacy compatibility, but developers are encouraged to turn them

Re: [sqlite] How lock is implemented upon write?

2019-07-02 Thread David Raymond
I don't have good answers for you as I'm not familiar with locking, but I'd suggest reading the comments in SQLite's os.h file (starting around line 91 at the moment) which I found interesting. For Python it looks like it'd be something involving the fcntl module for Unix or the msvcrt module

Re: [sqlite] WAL mode much slower in certain use cases

2019-07-01 Thread David Raymond
When you're inserting in WAL mode everything will basically have to be written twice. First the new page data will be written to the WAL, and then when a checkpoint happens the WAL file will be copied over to the main database file. With rollback journal mode the contents of the original page

Re: [sqlite] the sqlite3 documentation would be pretty good if it wasn't tragic...

2019-06-27 Thread David Raymond
On the documentation page (https://www.sqlite.org/docs.html) under "Overview Documents" is "Books About SQLite" https://www.sqlite.org/books.html > SQLite has, AFAIK, _two_ employees (Richard and Dan) See: https://www.sqlite.org/crew.html -Original Message- From: sqlite-users On

Re: [sqlite] select within transaction

2019-06-14 Thread David Raymond
How are you sending the commands to the cli? If you're doing... sqlite3 myfile.sqlite ".read somefile.sql" ...then you can start the sql file with... .bail on ...and as soon as it hits an error it will stop there and not continue processing lines. So if you get rid of the "or rollback" then

Re: [sqlite] Should SQLite distinguish between +0.0 and -0.0 on output?

2019-06-12 Thread David Raymond
https://www.sqlite.org/fileformat2.html#record_format The storage type of each record is given by an integer. And in the current format, all non-negative integers are used. To me of course that begs the question: Are negative serial types an option? That would of course mean a full 9 bytes per

Re: [sqlite] readfile() enhancement request

2019-05-17 Thread David Raymond
Are you on an OS with a working edit() function? https://www.sqlite.org/cli.html#the_edit_sql_function In Windows using notepad I can do this for example: insert into t values ('simple field', edit('', 'notepad')); You can even use it for multiple fields and it'll open one at a time insert

Re: [sqlite] SETting a value to a field with multiple conditions

2019-05-15 Thread David Raymond
When in doubt tinker. In this case it looks like it's the dot in "set tt.b = 'z'" that it's complaining about. Its thinking is probably "if you're updating a table you can't set a field from a different table, so no qualified field names there" sqlite> UPDATE t as tt set tt.b = 'z' WHERE tt.a

Re: [sqlite] Instr(x, y, z) ?

2019-05-14 Thread David Raymond
Well, one way is that you can make a sub-query that uses a recursive CTE to find it. I've got something that appears to work, but it seems overly convoluted, so I'm going to feel embarrassed when someone posts a simple elegant version in a couple minutes. Say for example you have create

Re: [sqlite] REINDEX causes rows not to be fetched in a WITHOUT ROWIDs table and PRIMARY KEY DESC

2019-05-13 Thread David Raymond
USE_ALLOCA sqlite> -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Luuk Sent: Monday, May 13, 2019 2:11 PM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] REINDEX causes rows not to be fetched in a WITHOUT ROWIDs table and PRIMARY K

Re: [sqlite] REINDEX causes rows not to be fetched in a WITHOUT ROWIDs table and PRIMARY KEY DESC

2019-05-13 Thread David Raymond
What version are you using? Because it seems to be working fine for me on 3.28.0 Windows CLI. SQLite version 3.28.0 2019-04-16 19:49:53 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> create table t0

Re: [sqlite] Series of statements results in a malformed database disk image

2019-05-09 Thread David Raymond
So it happens _before_ the update or replace? That is weird indeed. Using "indexed by" still returns 2 rows from the index, but integrity check reports 1 missing, so I'm curious as to what part of the integrity got broken. SQLite version 3.28.0 2019-04-16 19:49:53 Enter ".help" for usage

Re: [sqlite] What could happen if an app crashes while calling sqlite3_rekey_v2()?

2019-05-03 Thread David Raymond
Random question from a non-C person: What is sqlite3_rekey_v2()? I was curious, so looked for it in the docs and don't see it listed in the C reference... https://www.sqlite.org/c3ref/funclist.html ...and it doesn't get any hits when put into the search box for the web page. -Original

Re: [sqlite] [EXTERNAL] Re: seeking information on the throughput requirement using Sqlite

2019-05-03 Thread David Raymond
Which again goes to the hardware question. If your database is on "spinning rust" as people call it, you can't commit a transaction (that changed the data) any quicker than 2 rotations of the physical disk, because the data needs to get synced to the disk before the next write transaction can

Re: [sqlite] seeking information on the throughput requirement using Sqlite

2019-05-03 Thread David Raymond
Within a single transaction SQLite can do things very quickly. But a reminder that there can be only 1 write transaction happening at a time. So my questions are: What is the planned _transaction_ rate? And how many different connections will be trying to write at once? -Original

  1   2   3   4   5   >