Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

2011-02-20 Thread Dan Kennedy
On 02/21/2011 09:38 AM, Simon Slavin wrote: > > On 21 Feb 2011, at 1:47am, Roger Binns wrote: > >> On 02/20/2011 02:48 PM, Todd Shutts wrote: >>> The application >>> never used more than 10MB and it is currently using 57+MB and continues >>> to climb. >> >> The single most likely explanation is thi

Re: [sqlite] SQLite server using execnet ?

2011-02-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2011 06:23 PM, Jay A. Kreibich wrote: > On Sun, Feb 20, 2011 at 05:23:09PM -0800, Roger Binns scratched on the wall: >> If you want to use SQL then use Postfix. > > I might suggest PostgreSQL instead. > (Sorry, Roger, I couldn't resist.)

Re: [sqlite] SQLite server using execnet ?

2011-02-20 Thread Jay A. Kreibich
On Sun, Feb 20, 2011 at 05:23:09PM -0800, Roger Binns scratched on the wall: > If you want a close match to the Python object model then use MongoDB. > > If you want to use SQL then use Postfix. I might suggest PostgreSQL instead. Postgres is likely to give you much better results than th

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

2011-02-20 Thread Simon Slavin
On 21 Feb 2011, at 1:47am, Roger Binns wrote: > On 02/20/2011 02:48 PM, Todd Shutts wrote: >> The application >> never used more than 10MB and it is currently using 57+MB and continues >> to climb. > > The single most likely explanation is this is WAL in action, the memory is > from a memory m

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

2011-02-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2011 02:48 PM, Todd Shutts wrote: > However; memory usage is growing like crazy. It is a very frequent occurrence that what people think is being measured is not what is actually being measured. Make sure you understand exactly what the too

Re: [sqlite] Fwd: Scrolling through results of select

2011-02-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2011 04:47 PM, BareFeetWare wrote: > How can I best scroll though the results of an arbitrary select query? If you have infinite time and memory then there is no problem. You haven't stated your constraints or how arbitrary "arbitrary" reall

Re: [sqlite] Fwd: Scrolling through results of select

2011-02-20 Thread Simon Slavin
On 21 Feb 2011, at 12:47am, BareFeetWare wrote: > How can I best scroll though the results of an arbitrary select query? Suppose the results of the SELECT change between your original decision to do the scrolling and the time the user decides to scroll. Should what's shown on the display refl

Re: [sqlite] SQLite server using execnet ?

2011-02-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2011 04:55 PM, Stef Mientki wrote: > still want to see if a simple client/server setup would solve my current > problems There is no such thing as "simple" client/server. You have to worry about issues like authentication and authorization

Re: [sqlite] SQLite and SIGFPE

2011-02-20 Thread Richard Hipp
On Sun, Feb 20, 2011 at 7:39 PM, Samuel Adam wrote: > [I]f SQLite can *guarantee* no SIGFPE *ever*—under any possible inputs, > period, bar none ... that should also be documented. > SQLite attempts to intercept division by 0.0, replacing the result with NULL, but otherwise trusts the C compiler

[sqlite] SQLite server using execnet ?

2011-02-20 Thread Stef Mientki
hello, knowing that SQllite is not a client/server database, still want to see if a simple client/server setup would solve my current problems for the moment (because I love the simplicity of SQLlite, and planned to go to a client / server database in the future) Now I wonder if anyone has consi

[sqlite] Fwd: Scrolling through results of select

2011-02-20 Thread BareFeetWare
Anyone, anyone, Bueller? Here's a short version: How can I best scroll though the results of an arbitrary select query? Below is the detailed version of my question. Any help appreciated. Thanks, Tom BareFeetWare From: BareFeetWare Date: 16 February 2011 12:05:47 AM AEDT To: General Di

Re: [sqlite] SQLite and SIGFPE

2011-02-20 Thread Samuel Adam
On Sun, 20 Feb 2011 14:51:12 -0500, Nico Williams wrote: > On Sun, Feb 20, 2011 at 6:24 AM, Samuel Adam wrote: [big snip] >> Any numerics experts (which I am not) or fp-software gurus care to chime >> in? *If* I am correct in my inductive hypothesis that the SQLite core >> may >> in rare ci

Re: [sqlite] How to use sqlite and pthread together?

2011-02-20 Thread Samuel Adam
On Sun, 20 Feb 2011 14:46:06 -0500, Nico Williams wrote: > On Sun, Feb 20, 2011 at 6:28 AM, Samuel Adam wrote: >> On Sat, 19 Feb 2011 17:12:31 -0500, Pavel Ivanov >> wrote: >> >>> [snip] On >>> Windows it’s different - process is much more heavy-weight object than >>> thread and involves much

[sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

2011-02-20 Thread Todd Shutts
I inherited an application which used SQLite 3.5.8 running on Windows 2000. I upgrade to version 3.7.5 of the dll and added pragma statement to use WAL. There was an immediate and significant performance increase. However; memory usage is growing like crazy. The application never used more tha

Re: [sqlite] apostrophes in strings...

2011-02-20 Thread Scott Hess
You can also convert: ATTACH DATABASE x AS y KEY z to: SELECT sqlite_attach(x, y, z) where the parameters can be turned into bind arguments. Then embedded quotes won't be an issue. -scott On Sun, Feb 20, 2011 at 11:31 AM, Pavel Ivanov wrote: > I believe doubling the single quote inside th

[sqlite] SQLite GUI comparison (was: ANN: Base 2.0, Mac SQLite GUI)

2011-02-20 Thread BareFeetWare
On 21/02/2011, at 3:20 AM, skywind mailing lists wrote: > in your comparison chart it would also be nice to see which software is able > to support SQLite extension. A couple of them do not support the FTS nor > RTree capabilities of SQLite. Sure, I'd be happy to add that. How do you suggest th

Re: [sqlite] SQLite and SIGFPE

2011-02-20 Thread Nico Williams
On Sun, Feb 20, 2011 at 6:24 AM, Samuel Adam wrote: > I’m not writing anything multithreaded right now.  But next month or next > year, the humble little SQL user functions I now make could grow up and > get plugged into something bigger.  And before I longjmp(), I like to know > with certainty wh

Re: [sqlite] How to use sqlite and pthread together?

2011-02-20 Thread Nico Williams
On Sun, Feb 20, 2011 at 6:28 AM, Samuel Adam wrote: > On Sat, 19 Feb 2011 17:12:31 -0500, Pavel Ivanov > wrote: > >> [snip] On >> Windows it’s different - process is much more heavy-weight object than >> thread and involves much bigger system load to support it. There’s an >> official general adv

Re: [sqlite] apostrophes in strings...

2011-02-20 Thread Pavel Ivanov
I believe doubling the single quote inside the string literal should help. Pavel On Sun, Feb 20, 2011 at 2:23 PM, Sam Carleton wrote: > This is a bit crazy and I know the ideal way would be to not allow the > apostrophy in the first place but, my focus is easy of use for my customers, > as comp

[sqlite] apostrophes in strings...

2011-02-20 Thread Sam Carleton
This is a bit crazy and I know the ideal way would be to not allow the apostrophy in the first place but, my focus is easy of use for my customers, as compared to easy for me... I had a customer that saved their SQLite database here: D:/My Events/President's Day/event.sqlite My software uses ATT

Re: [sqlite] Help with join

2011-02-20 Thread Jeff Archer
>Date: Fri, 18 Feb 2011 07:34:06 -0800 >From: Jim Morris > >A correlated sub-query might work for you. > >SELECT >[Analyzers].[AnalyzerID] >, [Analyzers].[Name] AS [Analyzer] >, [Analysis].[AnalysisID] >, [Analysis].[ScanID] >, [Analysis].[Timestamp] >, [Analysis].[EndTime]

Re: [sqlite] ANN: Base 2.0, Mac SQLite GUI

2011-02-20 Thread skywind mailing lists
Hi Tom, in your comparison chart it would also be nice to see which software is able to support SQLite extension. A couple of them do not support the FTS nor RTree capabilities of SQLite. Hartwig Am 20.02.2011 um 07:19 schrieb BareFeetWare: > Hi Ben, > > In reply to your announcement of Base

[sqlite] owner/permissions on wal/shm and journal files

2011-02-20 Thread thilo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I am using sqlite 3.7.3 and have noticed the following behavior (which cause me some headaches). My DB is owned by a different user (www) and I as root insert some data. Using pragma journal=delete, the journal file gets created as whoever sta

Re: [sqlite] EXT :Re: FTS Append?

2011-02-20 Thread Black, Michael (IS)
Come to think of it that should work. I was storing terms in single rows and had to allow for duplicates. But converting to FTS relieves that as there are two flows in the code where it knows when a record is new or old so I don't have to worry about the "record already exists" for this case..

Re: [sqlite] Is it possible to determine how many openconnections are active for a sqlite database?

2011-02-20 Thread Black, Michael (IS)
I don't know if it works for your data...but you don't need to do all 5.4M in one batch. You should test doing it in different batch sizes -- like 1000 at a time (and let other processes do their thing potentially). That way you won't lock them out. But I think your other selects need to use t

Re: [sqlite] Compiler warnings in R-Tree code under Visual StudioExpress

2011-02-20 Thread Samuel Adam
On Sat, 19 Feb 2011 10:37:42 -0500, Afriza N. Arief wrote: > On Sat, Feb 19, 2011 at 6:27 AM, Samuel Adam wrote: > >> A FAQ[2] isn’t enough, as we can see. >> >> To put it another way: Bug reporters should have probable cause before >> they bug others. A compiler warning is only a reasonable

Re: [sqlite] How to use sqlite and pthread together?

2011-02-20 Thread Samuel Adam
On Sat, 19 Feb 2011 17:12:31 -0500, Pavel Ivanov wrote: > [snip] On > Windows it’s different - process is much more heavy-weight object than > thread and involves much bigger system load to support it. There’s an > official general advice for Windows: better create a new thread in the > same proc

Re: [sqlite] SQLite and SIGFPE

2011-02-20 Thread Samuel Adam
[In general reply to interesting answers by Messrs. Williams and Binns:] The potential SIGFPE in my case is optionally raised by a numeric library as part of its error-handling mechanism; its manual expressly suggests installing a signal handler and longjmp()ing back to find out what went wr

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Simon Slavin
On 20 Feb 2011, at 11:28am, Roger Binns wrote: > And if your customers care then they will already have existing solutions > for encryption and protection which includes dealing with incapacitation of > users, system administration, backups etc. It is not a good idea to defeat > those. And give

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2011 02:32 AM, Robert Hairgrove wrote: > There are certain parts of an SQLite database or page header (the first > 100 bytes, for example) which have known values. I think it is perhaps > even dangerous to encrypt this data, at least with the

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Philip Graham Willoughby
On 20 Feb 2011, at 10:52, Robert Hairgrove wrote: > On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote: >> For this task I would use AES-256 in counter mode with an appropriate nonce >> (the counter is trivially derived from the file offset of the block to be >> read/written). Th

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Philip Graham Willoughby
On 20 Feb 2011, at 10:32, Robert Hairgrove wrote: > On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote: >> On 20 Feb 2011, at 09:10, Robert Hairgrove wrote: >> >>> I am not starting from scratch doing my own encryption; there are enough >>> open source libraries publicly available

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Robert Hairgrove
On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote: > For this task I would use AES-256 in counter mode with an appropriate nonce > (the counter is trivially derived from the file offset of the block to be > read/written). The key should be derived from the user's password using >

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Robert Hairgrove
On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote: > On 20 Feb 2011, at 09:10, Robert Hairgrove wrote: > > > I am not starting from scratch doing my own encryption; there are enough > > open source libraries publicly available which are good enough for my > > purposes. > > And all

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Roger Binns
On 02/20/2011 01:10 AM, Robert Hairgrove wrote: > I saw that, but I find it a little intrusive, programmatically speaking, > from a licensing standpoint. Why don't you ask DRH then since you won't be the first to want to include SEE with QT and a commercial app? > I am not starting from scratch d

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Philip Graham Willoughby
On 20 Feb 2011, at 09:10, Robert Hairgrove wrote: > I am not starting from scratch doing my own encryption; there are enough > open source libraries publicly available which are good enough for my > purposes. And all of them offer approximately no security if you use them incorrectly. For this t

[sqlite] permissions for created database files are too restrictive and do not obey umask setting

2011-02-20 Thread Anders Lennartsson
File permissions on database files created by the library libsqlite3 on Unix/Linux are created with a mask setting that restricts permissions beyond what the user may expect from the current umask setting. In the file src/os_unix.c a mask is defined as: ** Default permissions when creating a new

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

2011-02-20 Thread Robert Hairgrove
On Sat, 2011-02-19 at 17:08 -0800, Roger Binns wrote: > On 02/19/2011 03:40 PM, Robert Hairgrove wrote: > > But before I "try this at home", I thought I would ask if there are any > > caveats I should be aware of? Thanks for any helpful advice! Thanks, Roger ... those are great links! > Unless yo