[sqlite] Hot-Journal with VFS

2018-10-18 Thread Bob schwanzer
Hi, I'm seeing hot journal frequently in our SQLite installation. We use VFS which does some checksum and other operations. There are no abnormal scenarios such as power off, process crash, abandoned transactions... DB is opened by 10-20 processes each of which can have multiple threads. Old vers

Re: [sqlite] [SQLITE]ignoring syntax errors during search for sth include "symbols"

2018-10-18 Thread Bob Gailer
On Oct 18, 2018 5:59 AM, "Maziar Parsijani" wrote: > > Hi, > how could I ignore syntax errors like this? > > > *SELECT *,* > > > > * highlight(searchsimpleenhanced, 2, '', '') text* > > > > * FROM searchsimpleenhanced* > > > > * WHERE searchsimpleenhanced MATCH 'sth][';* As I understand the docum

Re: [sqlite] Fill empty space with random

2018-10-14 Thread Bob Gailer
On Oct 14, 2018 7:57 AM, "J Decker" wrote: > > Is there maybe a compile option for sqlite to fill empty space in a db with > random data rather than 0 ? I don't know about compile options. You could do a 1 time update to set all 0 columns to random(1,99), and create a trigger to do the same on i

[sqlite] Looking for SQLite app builder

2018-08-17 Thread Bob Sisson
e shelf) SQLite form app on iPad Pros. We fill in the form, create a PDF, and then email that pdf out all from our iPads. Anyway... can anyone point us at a good form/database app builder please? Bob b...@inspectionsbybob.com Sent from my iPad ___ sqlit

Re: [sqlite] Huge RAM usage when sqlite is started from another thread

2018-07-13 Thread Bob Friesenhahn
rtual memory which could be from a heap allocation (with no subsequent writes to it) or due to memory mapping something such as a file. The cause of the the usage may be deduced by inspecting the /proc/[pid]/smaps content on a Linux system where '[pid]' is the process id of the progra

Re: [sqlite] Huge RAM usage when sqlite is started from another thread

2018-07-13 Thread Bob Friesenhahn
with VmSwap=0) is more interesting since it indicates how much memory is actually being used. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] column types and constraints

2018-06-29 Thread Bob Friesenhahn
On Fri, 29 Jun 2018, Richard Hipp wrote: On 6/29/18, Bob Friesenhahn wrote: Without adding all the necessary safe-guards to ensure that only valid data goes into the database, sqlite puts the using application at risk (security and stability) with its wishy-washy ways. Can you provide an

Re: [sqlite] column types and constraints

2018-06-29 Thread Bob Friesenhahn
e' in 'sqlite3' does not prohibit the same database from being developed by many programers using a variety of programming languages and scripts to solve very large problems. The stability of the sqlite file format and column type integrity are two different things. Bob -- B

Re: [sqlite] column types and constraints

2018-06-29 Thread Bob Friesenhahn
adding all the necessary safe-guards to ensure that only valid data goes into the database, sqlite puts the using application at risk (security and stability) with its wishy-washy ways. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsM

Re: [sqlite] column types and constraints

2018-06-29 Thread Bob Friesenhahn
, why do you think that it is appropriate to pass it the string 'xyzzy'? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ __

Re: [sqlite] building sqlite-src-3240000 I was surprised to see "make test" fail

2018-06-21 Thread Bob Friesenhahn
An Autoconf configure script which uses this macro: AC_USE_SYSTEM_EXTENSIONS will likely expose those prototypes. Solaris has the __EXTENSIONS__ define (e.g. -D__EXTENSIONS__=1) to enable all extensions. Otherwise you could try -D_XOPEN_SOURCE=500 and see if that helps. Bob -- Bob Friesenhahn bfrie..

Re: [sqlite] building sqlite-src-3240000 I was surprised to see "make test" fail

2018-06-21 Thread Bob Friesenhahn
depends on _POSIX_C_SOURCE >= 1 || _XOPEN_SOURCE || _BSD_SOURCE || _SVID_SOURCE || _POSIX_SOURCE but Solaris is proven to be Unix-compliant so the rules could be somewhat different. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagi

Re: [sqlite] .timer

2018-06-14 Thread Bob Friesenhahn
hat is not true. Something else which can take substantial time which is not attributed to the program is memory page faults. These might not be attributed to the program (e.g. as 'sys' time) since a kernel driver performs the I/O for page faults. Bob -- Bob Friesenhahn bfrie...

Re: [sqlite] Mailing list shutting down...

2018-06-13 Thread Bob Friesenhahn
s via SMTP email (I don't remember that it does), then the problem is worse. If only one new subscription is allowed on the list per day, then there is a trivial DOS (no new valid subscriptions are possible) as soon as the one daily subscription has been consumed. Bob -- Bob Friesenhahn bfri

Re: [sqlite] How to convert SQL file into database when a column value is Inf?

2018-06-12 Thread Bob Friesenhahn
is willing to consume? The range of a double value is typically 2.225074e-308 to 1.797693e+308. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] Usage of temporary files of SQLite3 on Android / SQLite commands fail

2018-06-09 Thread Bob Friesenhahn
tried setting the POSIX standard TMPDIR environment variable? This might have useful influence under Android. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] sqlite.org website is now HTTPS-only

2018-06-08 Thread Bob Friesenhahn
on and produce a new HTTPS connection to the remote server. Many HTTPS clients have been found to be doing wrong things. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://ww

Re: [sqlite] sqlite.org website is now HTTPS-only

2018-06-07 Thread Bob Friesenhahn
server certificate checks are disabled. It is much easier to add to the certificates used by the system given that wget already works. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] sqlite.org website is now HTTPS-only

2018-06-07 Thread Bob Friesenhahn
x27;s Encrypt-friendly machine or from the Let's Encrypt site. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sq

Re: [sqlite] sqlite.org website is now HTTPS-only

2018-06-07 Thread Bob Friesenhahn
"wget" are slightly out of date. The certificates used by the browser are usually provided by the browser vendor and so they are not necessarily provided by the OS vendor. If 'wget' does not know about 'Lets Encrypt' then merely waiting is unlikely to solve the prob

Re: [sqlite] Size of the SQLite library

2018-06-06 Thread Bob Friesenhahn
ing of software from source code should always be under the complete control of the person who is performing the build and should inherently support use of local files which may contain local changes. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfri

Re: [sqlite] Filename encoding on Unix platforms

2018-06-05 Thread Bob Friesenhahn
system stores files in a MS-DOS/Windows FAT filesystem then you are subject to the limitations of that filesystem. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] Size of the SQLite library

2018-05-31 Thread Bob Friesenhahn
ents common in laptop PCs. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mai

Re: [sqlite] Fossil Delta Compression in SqLite

2018-05-08 Thread Bob Friesenhahn
umentation (assuming that the dependent software provides any documentation). https://opensource.org/licenses/BSD-2-Clause Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.Graphics

Re: [sqlite] Question about Practicality of Embedding SQLite on Cortex-M4 Processor

2018-03-02 Thread Bob Friesenhahn
If you really only have 160KB of RAM (vs 160MB), then that would be prohibitive. Linux and SQLite are not going to be able to run with 160KB of RAM. Otherwise, it sounds like a fine idea. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen

Re: [sqlite] Vetting SQLite

2018-02-05 Thread Bob Friesenhahn
. The security of SQLite depends on how it is built, the environment in which it is used, and the arguments supplied to it. If arbitrary SQL commands can be sent into SQLite, then good luck and best wishes regarding security. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http

Re: [sqlite] Move to Github!!?

2017-12-27 Thread Bob Friesenhahn
a DVCS, the whole repository normally needs to be duplicated first. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing

Re: [sqlite] Move to Github!!?

2017-12-26 Thread Bob Friesenhahn
there such tools available for Fossil? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlit

Re: [sqlite] Fix for 32bit compilation on Solaris

2017-12-01 Thread Bob Friesenhahn
EA package. Sqlite could fix its copy but you should report this upstream from where it came from so that all Tcl extensions which update their TEA package will benefit. See http://wiki.tcl.tk/327 and https://github.com/tcltk/tclconfig Bob -- Bob Friesenhahn bfrie...@simple.dallas.t

Re: [sqlite] Please remove multiple warnings from compiler about optimisation, variable conversion, signed overflow and many more potential errors.

2017-09-29 Thread Bob Friesenhahn
y get better for all platforms and compilers ) This causes problems given that there are varying configuration methods, but this annoying logic should get rid of the warning: #if defined(SQLITE_4_BYTE_ALIGNED_MALLOC) && SQLITE_4_BYTE_ALIGNED_MALLOC Bob -- Bob Friesenhahn bfrie...@simple

Re: [sqlite] Please remove multiple warnings from compiler about optimisation, variable conversion, signed overflow and many more potential errors.

2017-09-29 Thread Bob Friesenhahn
s unsigned types. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlit

Re: [sqlite] Compiler warning "gethostuuid() is disabled" building SQLite for iOS

2017-08-17 Thread Bob Friesenhahn
. I am still using NFS mounted home directories across all systems here (since 1993). No significant problems have been encountered during that time. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http

Re: [sqlite] group_concat() reverses order given where clause?

2017-08-15 Thread Bob Friesenhahn
On Tue, 15 Aug 2017, Dan Kennedy wrote: On 08/15/2017 10:12 PM, Bob Friesenhahn wrote: select group_concat(name, ' ') AS 'names' from moca_config where enable == 1 order by name; Maybe this: select group_concat(name, ' ') AS 'names' from ( SELECT

[sqlite] group_concat() reverses order given where clause?

2017-08-15 Thread Bob Friesenhahn
#x27;names' from moca_config where enable == 1 order by name; names rf_band verbose moca_core_trace_enable preferred_nc lof_update bonding Notice that adding a 'where' clause has caused the order to be reversed from what was requested in the query. Why is this and what can I d

Re: [sqlite] Packing integer primary key with field bits

2017-08-10 Thread Bob Friesenhahn
/O operations possible within a given amount of time (IOPS) rather than the bandwidth (bytes/second) available. If the database is small enough to fit in the OS filesystem cache, then hardly any I/O operations to underlying store will be needed to satisfy requests. Bob -- Bob Friesenhahn bfrie

Re: [sqlite] Tutorials, books, video about SQLite

2017-08-09 Thread Bob Friesenhahn
On Wed, 9 Aug 2017, Lars Frederiksen wrote: I would appreciate very much if you clever people out there have some booktitles or links to tutorials (websites, video etc) about SQLite. The O'Reilly book "Using SQLite" has been the most useful book for me thus far. Bob --

Re: [sqlite] Feature request: check for 'lib' prefix for load_extension()

2017-08-04 Thread Bob Friesenhahn
On Fri, 4 Aug 2017, Peter Da Silva wrote: On 8/4/17, 8:29 AM, "sqlite-users on behalf of Bob Friesenhahn" wrote: Lazy programmers who request such things are of the same ilk which use programming practices resulting in SQL injection attacks. Sqlite should not promote such

Re: [sqlite] Feature request: check for 'lib' prefix for load_extension()

2017-08-04 Thread Bob Friesenhahn
ted. Lazy programmers who request such things are of the same ilk which use programming practices resulting in SQL injection attacks. Sqlite should not promote such practices. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.sim

Re: [sqlite] Feature request: check for 'lib' prefix for load_extension()

2017-08-03 Thread Bob Friesenhahn
an attacker was able to write into). Apple's OS X and Microsoft Windows always try to load from the current directory. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ _

Re: [sqlite] Poll: Include the recent sqlite3_column_name() fix in the upcoming 3.20.0 release?

2017-07-31 Thread Bob Friesenhahn
will be severe problems. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Database version

2017-07-21 Thread Bob Friesenhahn
l could be using a different sqlite version. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@

Re: [sqlite] marketing - The world's most popular open source database

2017-07-18 Thread Bob Friesenhahn
ve turned to Rust by 2050. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mai

Re: [sqlite] cannot compile sqlite3 with intel compiler (ICC) - log and hints for a possible solution

2017-07-09 Thread Bob Friesenhahn
solution is to just not use icc? Sqlite3 has been cycle-optimized for GCC. You now have 29-days to also cycle-optimize it for ICC. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] Version 3.19.3 containing an important bug fix

2017-06-08 Thread Bob Friesenhahn
for me to do is to send you the modified file privately. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite

Re: [sqlite] Version 3.19.3 containing an important bug fix

2017-06-08 Thread Bob Friesenhahn
his happens, 'pragma integrity_check' never finds a problem with the database. Regardless, it seems that once the problem starts happening, it recurs given the same starting database. The starting database is already a product of VACCUM before any other software starts to use it.

Re: [sqlite] PRAGMA cache_size=0 increases memory usage

2017-05-18 Thread Bob Friesenhahn
Our schema takes a bit over 1MB to load on a 32-bit CPU. The increased usage we are seeing is on the order of 200k so it is not the schema. Regardless, the application is using the database immediately so it must always consume the schema. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us,

Re: [sqlite] PRAGMA cache_size=0 increases memory usage

2017-05-18 Thread Bob Friesenhahn
On Thu, 18 May 2017, Kim Gräsman wrote: On Thu, May 18, 2017 at 6:10 PM, Bob Friesenhahn wrote: Does anyone know why using 'PRAGMA cache_size=0' (or some other small value) to attempt to decrease memory usage (and it is reported as immediately decreased in the shell by .stats

Re: [sqlite] PRAGMA cache_size=0 increases memory usage

2017-05-18 Thread Bob Friesenhahn
On Thu, 18 May 2017, Simon Slavin wrote: On 18 May 2017, at 5:10pm, Bob Friesenhahn wrote: Does anyone know why using 'PRAGMA cache_size=0' (or some other small value) to attempt to decrease memory usage (and it is reported as immediately decreased in the shell by .stats

[sqlite] PRAGMA cache_size=0 increases memory usage

2017-05-18 Thread Bob Friesenhahn
Does anyone know why using 'PRAGMA cache_size=0' (or some other small value) to attempt to decrease memory usage (and it is reported as immediately decreased in the shell by .stats) actually significantly increases heap memory usage? I find this to be an interesting phenomena.

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

2017-04-18 Thread Bob Friesenhahn
adopt this same test. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mailinglists.

Re: [sqlite] "struct Mem" conflicts with namespaces/classes having the same name

2017-04-10 Thread Bob Friesenhahn
ution for everyone. A way to get rid of the annoying 'sqlite::' prefix for plain C functions is to import the functions actually planned to be used into global scope or into the namespace of the using code: namespace MyNamespace { using sqlite:sqlite3_exec; } Bob -- Bob Friese

Re: [sqlite] "struct Mem" conflicts with namespaces/classes having the same name

2017-04-09 Thread Bob Friesenhahn
On Sun, 9 Apr 2017, dip wrote: What happens with mangling then? extern "C" functions inside a namespace just do not get mangled? If so, sounds like a workaround. Yes, extern "C" functions don't get mangled. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, htt

Re: [sqlite] "struct Mem" conflicts with namespaces/classes having the same name

2017-04-09 Thread Bob Friesenhahn
ode or refer to them specifically within the namespace you created. I have done this successfully across many C++ compilers for 19 years already. It would definitely be best if sqlite does not pollute the name space with names not starting from a common prefix. Bob -- Bob Friesenhahn bfrie...@simpl

Re: [sqlite] Sqlite + Dropbox

2017-04-08 Thread Bob Friesenhahn
esses it. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Continuous recovery of journal

2017-04-07 Thread Bob Friesenhahn
On Thu, 6 Apr 2017, Simon Slavin wrote: On 6 Apr 2017, at 2:38pm, Bob Friesenhahn wrote: Is calling sqlite3_close() prior to exiting a requirement in general for sqlite? I was not aware of this requirement. Do you explicitly call sqlite3_shutdown() ? If not, when are you expecting

Re: [sqlite] VT table behavior change between 3.10 and 3.17

2017-04-06 Thread Bob Friesenhahn
is closed. What is the correct expectation? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users

Re: [sqlite] Continuous recovery of journal

2017-04-06 Thread Bob Friesenhahn
problems which may occur in sqlite3_close() if other requests have not been completed properly. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] Incompatibility into configure.ac

2017-04-05 Thread Bob Friesenhahn
hardly a representation of a standards-conformant/enforcing shell. It is popular because of its extensions and because of its interactive behavior. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http

Re: [sqlite] regression: SQLite 3.18.0 - editline configure no longer working

2017-03-31 Thread Bob Friesenhahn
this issue when I sent the sample patch to Richard. If need be, I can help and provide a more portable patch than my sample. Bob On 3/31/17, Lonnie Abelbeck wrote: Hi, This commit: [bf28a55d] http://www.sqlite.org/src/fdiff?v1=cacf2616abf6e4a5&v2=2893b823ecc86cea&sbs=0 Keeps li

Re: [sqlite] VT table behavior change between 3.10 and 3.17

2017-03-29 Thread Bob Friesenhahn
01 28VBegin 0 0 0 vtab:CB170000 29String80 110 bar00 30Goto 0 1 000 Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maint

Re: [sqlite] VT table behavior change between 3.10 and 3.17

2017-03-29 Thread Bob Friesenhahn
improperly constructed and was implemented based on observed behavior at the time. There is inadequate tracing in the code so proper tracing will be added when it is re-written. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintaine

[sqlite] VT table behavior change between 3.10 and 3.17

2017-03-28 Thread Bob Friesenhahn
ow when the 'xOpen' callback is called if it is to support an update transaction (i.e. 'xUpdate' callback will be called)? Thanks, Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.Grap

Re: [sqlite] Developing a SQLite3 DB remotely

2017-03-24 Thread Bob Friesenhahn
lient tries to improve apparent performance by intentionally not obeying the rules. Transaction performance over NFS may not be very good (depending on properties of server and client) but that does not equate to corruption or failed locking. Bob -- Bob Friesenhahn bfrie...@simple.dallas.

Re: [sqlite] Compressed schema in memory?

2017-03-16 Thread Bob Friesenhahn
In sqlite_master I see quite a lot of "sql_autoindex" indexes. Do these auto indexes consume the same RAM as explicit indexes? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.Graphics

Re: [sqlite] Compressed schema in memory?

2017-03-16 Thread Bob Friesenhahn
ess memory should be consumed. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlit

Re: [sqlite] Compressed schema in memory?

2017-03-16 Thread Bob Friesenhahn
not shorten them all to 4 characters? That isn't a lot of space, but it is a start. This seems like a good idea and does not limit human comprehension much, particularly if it is done as a step while preparing the final database. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us,

Re: [sqlite] Compressed schema in memory?

2017-03-16 Thread Bob Friesenhahn
On Thu, 16 Mar 2017, Richard Hipp wrote: On 3/16/17, Bob Friesenhahn wrote: The schema (already stripped to remove white space and comments) for our database has reached 664K Yikes. That's about 10x or 20x what we typically see. Are you able to share your schema with us? I share

[sqlite] Compressed schema in memory?

2017-03-16 Thread Bob Friesenhahn
, the database schema reduces to just 62k so there is a 10X benefit. With 10 processes/connections, almost 6MB could be saved with our database. It is likely that the compression ratio is less when compressing many small fragments of text. Thoughts? Bob -- Bob Friesenhahn bfrie

[sqlite] Shared cache mode and busy wait

2017-03-16 Thread Bob Friesenhahn
ack mechanisms are not supported, then it becomes prohibitive to enable this mode on an existing code base. Given that each sqlite connection duplicates the entire database schema in RAM, this shared cache mode becomes quite useful on limited memory systems. It would be good if it

Re: [sqlite] feature req: PLEASE why the heck COMMENT fields are not supporte in years!!

2017-03-15 Thread Bob Friesenhahn
tly as-is in the SQL field of the schema table (main.sqlite_master) and is easy to parse out later, or use a standard tool Are these comments loaded into memory used by each program which connects to the database? If so, more resources are consumed. Bob -- Bob Friesenhahn bfrie...@simple.dallas.t

Re: [sqlite] confused getting started

2017-03-06 Thread Bob Friesenhahn
under Windows. Based on my experience, Python with the APSW extension is an excellent way to create "write once" code as long as you are able to use Python. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintaine

Re: [sqlite] Massively multithreaded SQLite queries

2017-03-04 Thread Bob Friesenhahn
memory-mapping the database may speed access by decreasing programmed I/O for queries which only do reads. This assumes that there is some commonality among the data used by the queries so there is a high probability that often accessed data is already in RAM. Bob -- Bob Friesenhahn bfrie

Re: [sqlite] Database is locked

2017-03-01 Thread Bob Friesenhahn
On Wed, 1 Mar 2017, Simon Slavin wrote: On 1 Mar 2017, at 7:27pm, Bob Friesenhahn wrote: What is the recommended approach to diagnosing "Database is locked" errors and determining the guilty party? Are you checking the result codes returned by all SQLite commands you give to

Re: [sqlite] Database is locked

2017-03-01 Thread Bob Friesenhahn
have a database connection open all the time. I am assuming that the problem is a hung (not commited or rolled back) transaction. I am hoping that sqlite provides a way to know what process is currently performing an update transaction. Bob -- Bob Friesenhahn bfrie...@simple.dal

[sqlite] Database is locked

2017-03-01 Thread Bob Friesenhahn
o bytes. What is the recommended approach to diagnosing "Database is locked" errors and determining the guilty party? Is there a way to clear the condition which causes "Database is locked" errors? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://w

Re: [sqlite] Beginning of release testing for version 3.17.0

2017-02-27 Thread Bob Friesenhahn
intended to ease certain aspects of SQLite development which are assisted from a portable implementation with no external dependencies. The mere existence of an implementation does not mean that it is a defect. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users

Re: [sqlite] No way to check for CHECK constraint violations a posteriori

2017-02-27 Thread Bob Friesenhahn
approach where table rows are updated one by one so that we can know which table row update failed. This causes other issues since whole-row or inter-table consistency checks may temporarily fail. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen

Re: [sqlite] Thread safety of serialized mode

2017-02-16 Thread Bob Friesenhahn
one has posted a follow-up on. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users

Re: [sqlite] Thread safety of serialized mode

2017-02-16 Thread Bob Friesenhahn
a SQLite3 implementation logic issue (something to do with a deferred moveto). Why should destroying a prepared statement care about a cursor's deferred moveto? Bob On Wed, 15 Feb 2017, Bob Friesenhahn wrote: It turns out that I have more data on the problem. The error message rep

Re: [sqlite] Thread safety of serialized mode

2017-02-15 Thread Bob Friesenhahn
;deferredMoveto = 0; 70278 p->cacheStatus = CACHE_STALE; 70279 return SQLITE_OK; 70280 } Ideas? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ __

Re: [sqlite] Thread safety of serialized mode

2017-02-15 Thread Bob Friesenhahn
other thread? Not if they’re using the same statement. A statement is a stateful object, so using it on multiple threads is probably going to cause problems. To be clear, each thread is using its own prepared statement. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http

Re: [sqlite] Thread safety of serialized mode

2017-02-15 Thread Bob Friesenhahn
On Tue, 14 Feb 2017, Richard Hipp wrote: On 2/14/17, Bob Friesenhahn wrote: Due to memory constraints (at least 1MB is consumed per connection!), only one database connection is used. Any thread may acquire and use this one database connection at any time. This is yet another reason why I

Re: [sqlite] Thread safety of serialized mode

2017-02-14 Thread Bob Friesenhahn
On Wed, 15 Feb 2017, Simon Slavin wrote: On 14 Feb 2017, at 11:51pm, Bob Friesenhahn wrote: One of our Linux programs (not written by me) is reporting errors of the form "SQLITE_CORRUPT: database disk image is malformed database corruption". Is the database actually corrupt

[sqlite] Thread safety of serialized mode

2017-02-14 Thread Bob Friesenhahn
he other thread? In this use case is sqlite3 usage "thread safe" or is behavior unstable due to sqlite3_step(), sqlite3_reset(), and result column accessors accessing/disrupting data from the result set of the other thread? Thanks, Bob -- Bob Friesenhahn bfrie...@

Re: [sqlite] Problem compiling 3.17.0 in MSVS 12

2017-02-14 Thread Bob Friesenhahn
e the most likely cause of the problem. Windows headers are very unclean and its pre-processor definitions often overwrite user code. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.Graphics

Re: [sqlite] Virtual table vs real table query performance

2017-02-08 Thread Bob Friesenhahn
implementation is about as good as it can be. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users

Re: [sqlite] Virtual table vs real table query performance

2017-02-08 Thread Bob Friesenhahn
before xBestIndex and xFilter support was added). Due to the requirements of the implementation, POSIX reader/writer locks are used so there is some low-contention locking overhead. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick

[sqlite] Virtual table vs real table query performance

2017-02-07 Thread Bob Friesenhahn
little additional overhead) vs a native table? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mailing list sqlite-users

Re: [sqlite] extension to run bash

2017-01-11 Thread Bob Friesenhahn
exploit opportunities and now SQL injection exploit opportunities also become shell exploit opportunities. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org

Re: [sqlite] SQLite 3.16.0 enters testing

2016-12-29 Thread Bob Friesenhahn
time it is executed. Any other factor on the machine would impact perf results. It seems that cachegrind produces absolutely consistent results which do not depend on I/O, multi-core, or VM artifacts. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users

Re: [sqlite] SQLite 3.16.0 enters testing

2016-12-29 Thread Bob Friesenhahn
would explain the inflection point and more rapid progress starting at that time. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite

Re: [sqlite] SQLite 3.16.0 enters testing

2016-12-29 Thread Bob Friesenhahn
On Thu, 29 Dec 2016, Richard Hipp wrote: On 12/29/16, Bob Friesenhahn wrote: Is there a way to know how well cachegrind CPU cycles map to real-world CPU usage? Not that I know of. If you have any suggestions, please speak up. I tried to make clear in the docs that I consider cachegrind to

Re: [sqlite] SQLite 3.16.0 enters testing

2016-12-29 Thread Bob Friesenhahn
world CPU usage? If sqlite3 consumes 2X less "CPU cycles" since 2009 is there a way to measure how much less actual CPU time that it takes in order to validate that there is a clear relationship? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users

Re: [sqlite] Error code for VT xColumn()/xRowid() if row does not exist?

2016-12-12 Thread Bob Friesenhahn
The case where the row might no longer exist is for xRowid(), but this function does not seem to be called for the table I am looking at. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagic

[sqlite] Error code for VT xColumn()/xRowid() if row does not exist?

2016-12-12 Thread Bob Friesenhahn
ct code to return? If a cursor is being navigated, then I would prefer that the cursor continue to the next result row. Thanks, Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.Graphics

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

2016-11-15 Thread Bob Friesenhahn
ot storage. The useful mass storage offerings for hardware running MacOS is rather limited. Operating systems where fsync() or fdatasync() do not truely commit data to hardware are broken. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ Grap

Re: [sqlite] Why so much I/O ? Can sqlite be improved ?

2016-10-01 Thread Bob Friesenhahn
You may have an issue with write amplification at the filesystem level. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ sqlite-users mail

[sqlite] sqlite3_bind_text with empty sting....

2016-09-19 Thread Mr Bob
I have a table with a not null column... Some of the values are empty string and some have more lengthy values :-) I prepare a statement ("Select col2 from table where column = ?") and then use bind to that parameter what are the parameters ? From slq browser I can "Select col2 from tab

Re: [sqlite] Problem with rename table

2016-09-07 Thread Bob McFarlane
Please reply if you sent this. Thanks. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp Sent: Wednesday, September 7, 2016 7:49 PM To: SQLite mailing list Subject: Re: [sqlite] Problem with rename table On 9/6/16, R

Re: [sqlite] Query time execution difference between my application and SQLiteBrowser

2016-09-07 Thread Bob McFarlane
Please reply if you sent this. Thanks. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Chris Locke Sent: Wednesday, September 7, 2016 12:12 PM To: SQLite mailing list Subject: Re: [sqlite] Query time execution difference between

  1   2   3   >