Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Dan Kennedy]

On 02/21/2011 09:38 AM, Simon Slavin wrote:
>
> On 21 Feb 2011, at 1:47am, Roger Binns wrote:
>
>> On 02/20/2011 02:48 PM, Todd Shutts wrote:
>>> The application
>>> never used more than 10MB and it is currently using 57+MB and continues
>>> to climb.
>>
>> The single most likely explanation is this is WAL in action, the memory is
>> from a memory mapped file and a WAL checkpoint will release it.
>>
>>   http://www.sqlite.org/wal.html
>
> Todd's question, or something like it, has been asked at least seven times on 
> this list in the seven months since WAL was released.  Might I suggest that 
> an explanation of WAL's use of lots of memory be placed higher up on that 
> page and headed something like "Memory usage" ?
>
> Also, something like "May use many megabytes of memory between checkpoints." 
> should be listed in the 'disadvantages' list.  The explanation of 
> checkpointing is fine, but users don't know they need to read it until 
> they've asked here.

The size of the memory mapped file is about 8 bytes per page
in the WAL log, allocated in 32 KB chunks.

So if the poster is using a page size of 1024 bytes, a 47MB
memory mapped file  corresponds to roughly a 6GB WAL log. Or
if he is using the win32 default page size of 4K, a WAL file
four times that large.

WAL file growth has (apparently) been a problem for people with
really busy systems. But so far it has been the actual size of
the file on disk that bothers them, not the memory mapped bit.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite server using execnet ?

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 06:23 PM, Jay A. Kreibich wrote:
> On Sun, Feb 20, 2011 at 05:23:09PM -0800, Roger Binns scratched on the wall:
>> If you want to use SQL then use Postfix.
> 
>   I might suggest PostgreSQL instead.
>   (Sorry, Roger, I couldn't resist.)

Yeah, long night :-)  However, technically SQL over SMTP is possible and
would actually work.  And if anyone is insane enough to try that then using
Postfix and Postgres are a good combination.

>> If you need lots of processes on the network to access data quickly then
>> consider memcached.
> 
>   More seriously, in this category you might also consider Redis.
>   Redis allows your data to have some structure, 

The Python binding pylibmc does structure the data for you automagically.

>   plus it has the
>   ability to persist the data to disk. 

The moment you talk about persistence you then have significant overlap with
databases.  My personal favourite is MongoDB but there are loads of others
such as Cassandra, HBase, Tokyo Cabinet etc.

I like the ones that don't have a schema the best.

>   I've been playing around with connecting SQLite virtual tables to
>   a Redis server, and it is producing some interesting results.

A while back I implemented a virtual table that talks to CouchDB.  I suspect
you'll have similar issues.  You may find the documentation of interest as
you'll likely encounter similar issues.

  http://apidoc.apsw.googlecode.com/hg/couchdb.html

What programming language are you using to implement the virtual tables?

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1h6+wACgkQmOOfHg372QT/DACfQFOCo/Ku/kHXZGQ0eiXqWDcJ
XQwAnRaBR8/uNgSKNBKXKiG5i/y7G1wm
=1v5a
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite server using execnet ?

[Jay A. Kreibich]

On Sun, Feb 20, 2011 at 05:23:09PM -0800, Roger Binns scratched on the wall:

> If you want a close match to the Python object model then use MongoDB.
> 
> If you want to use SQL then use Postfix.

  I might suggest PostgreSQL instead.
  
  Postgres is likely to give you much better results than the
  Postfix SMTP email server.

  (Sorry, Roger, I couldn't resist.)

> If you need lots of processes on the network to access data quickly then
> consider memcached.

  More seriously, in this category you might also consider Redis.
  Redis allows your data to have some structure, plus it has the
  ability to persist the data to disk. 

  I've been playing around with connecting SQLite virtual tables to
  a Redis server, and it is producing some interesting results.

   -j

-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"Intelligence is like underwear: it is important that you have it,
 but showing it to the wrong people has the tendency to make them
 feel uncomfortable." -- Angela Johnson
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Simon Slavin]

On 21 Feb 2011, at 1:47am, Roger Binns wrote:

> On 02/20/2011 02:48 PM, Todd Shutts wrote:
>> The application
>> never used more than 10MB and it is currently using 57+MB and continues
>> to climb.  
> 
> The single most likely explanation is this is WAL in action, the memory is
> from a memory mapped file and a WAL checkpoint will release it.
> 
>  http://www.sqlite.org/wal.html

Todd's question, or something like it, has been asked at least seven times on 
this list in the seven months since WAL was released.  Might I suggest that an 
explanation of WAL's use of lots of memory be placed higher up on that page and 
headed something like "Memory usage" ?

Also, something like "May use many megabytes of memory between checkpoints." 
should be listed in the 'disadvantages' list.  The explanation of checkpointing 
is fine, but users don't know they need to read it until they've asked here.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 02:48 PM, Todd Shutts wrote:
> However; memory usage is growing like crazy. 

It is a very frequent occurrence that what people think is being measured is
not what is actually being measured.  Make sure you understand exactly what
the tool showing you memory consumption is measuring and especially
understand if it includes memory shared with other processes or memory
mapped files.  Additionally you need to understand the difference between
memory as the kernel sees and reports it versus memory within particular
APIs such as C's malloc.

> The application
> never used more than 10MB and it is currently using 57+MB and continues
> to climb.  

The single most likely explanation is this is WAL in action, the memory is
from a memory mapped file and a WAL checkpoint will release it.

  http://www.sqlite.org/wal.html

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1hxDYACgkQmOOfHg372QRikgCdHrEuzE5p71LTaiF+WRHfG6j2
9S0An100kCApkwZI74XGYR6zxczr2m7u
=d0xw
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: Scrolling through results of select

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 04:47 PM, BareFeetWare wrote:
> How can I best scroll though the results of an arbitrary select query?

If you have infinite time and memory then there is no problem.  You haven't
stated your constraints or how arbitrary "arbitrary" really is.

> 1. Is there any significant overhead on SQLite from my selecting from a view 
> representing the original arbitrary select? That is, will SQLite still use 
> any indexes etc correctly? Or do I need to dissect/parse the original select 
> statement, changing the where statement etc?

You can work out the answer yourself by using EXPLAIN and EXPLAIN QUERY PLAN
of some representative examples.

> 2. If the arbitrary select statement already contains an "order by" clause, 
> then I obviously need to use the order by column(s) specified there as the 
> keyColumn for the scrolling, rather than override the order by. Is there any 
> way to get the sort order of an arbitrary select statement?

Results are returned in the order requested or randomly(*) if not.  Given
you can have subqueries with ordering, collations and all sorts of other
things, trying to extract the actual ordering is as difficult as
implementing the SQLite engine itself.  You can even "ORDER BY random()".

(*) In practise it is in btree iteration order but that is not something you
should depend on.

The rest of your questions assume a particular solution.  The only thing
that will reliably work is to reissue the query using skip and limit
assuming no changes in between.  This is if you are trying to save
memory/disk and there is no possibility of changes between scrolling
operations.

If you need to be resilient to that too (implied by "arbitrary" since user
defined functions could have side effects) then the solution is to 'CREATE
TEMP TABLE results AS ...select...'.  This will also work if someone uses
"ORDER BY random()" or any other udf that depends on more than its arguments.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEQEARECAAYFAk1hwsUACgkQmOOfHg372QTmTgCYrErijaVbARjH772SJC9qID2S
hgCYt7OxymRNAUhOjyUBQvDuoluQJw==
=GPYv
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: Scrolling through results of select

[Simon Slavin]

On 21 Feb 2011, at 12:47am, BareFeetWare wrote:

> How can I best scroll though the results of an arbitrary select query?

Suppose the results of the SELECT change between your original decision to do 
the scrolling and the time the user decides to scroll.  Should what's shown on 
the display reflect the data as it originally was, or up-to-date data ?  Or 
could there never be any such changes ?

> 1. Is there any significant overhead on SQLite from my selecting from a view 
> representing the original arbitrary select? That is, will SQLite still use 
> any indexes etc correctly? Or do I need to dissect/parse the original select 
> statement, changing the where statement etc?

A VIEW is a saved SELECT query -- the query, not the results, are saved.  So I 
think you don't need to make the extra effort you describe.

> 2. If the arbitrary select statement already contains an "order by" clause, 
> then I obviously need to use the order by column(s) specified there as the 
> keyColumn for the scrolling, rather than override the order by. Is there any 
> way to get the sort order of an arbitrary select statement?

I don't know of any.

> 3. This method requires that keyColumn is defined as unique (or primary key), 
> otherwise it can skip rows of data. Is there any way to allow for a 
> non-unique keyColumn?

No, but instead of using just keyColumn you could use (keyColumn,rowid).  This 
would ensure your key was always unique, and will work on arbitrary SQLite 
tables unless someone is intentionally messing with how SQLite works.

> 4. If the arbitrary select statement does not specify an order by, how can I 
> least affect the output (ie not impose a sort order) but still facilitate 
> scrolling? For selecting from a table, the best I can think of is to use 
> rowid (or its alias), which seems to be the typical result order when no 
> order is specified. But when selecting from a view (which may contain joins), 
> by which column(s) can I explicitly sort (for the sake of scrolling) that 
> will best mimic the usual SQL output order (which I know is "undefined")?
> 
> 5. I understand that "Rule Number 1" is to "not leave queries open".

Correct.  Don't allow a user to create and close a query just by choosing when 
they want to scroll through a list.

> So what's the best way to minimize the overhead of repeatedly running the 
> same query but with a different where clause and limit (and order if 
> reversing). I'm thinking I would be best to actually keep the query (ie 
> prepared statement) open while live scrolling (eg flicking through rows on an 
> iPhone/iPad), not using a limit clause at all, but instead just keep getting 
> more rows as needed to fill the scrolling, until the user stops scrolling, 
> then finalize, close etc. When they begin scrolling again, fire up a new 
> prepare (with a new maxVisibleKeyValue) .

To get the following or previous line to one which is already being shown, find 
the key for that row (which you should save in memory as you're displaying the 
line) and use

SELECT  FROM  WHERE (keyColumn||rowid)>lastlineKey ORDER BY 
keyColumn,rowid LIMIT 1

to get the following line or

SELECT  FROM  WHERE (keyColumn||rowid)http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite server using execnet ?

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 04:55 PM, Stef Mientki wrote:
> still want to see if a simple client/server setup would solve my current 
> problems 

There is no such thing as "simple" client/server.  You have to worry about
issues like authentication and authorization.  You have to deal with naming.
 You have to think about APIs being synchronous or asynchronous.  You have
to worry about state and if state is maintained across connections or
dropped.  You have to worry about new error codes that couldn't occur
before.  You have to deal with race conditions and latency.  Sweeping all
this under the rug will appear simple until you do real deployments and
start painfully encountering and addressing the issues.  That is why
networked databases are not simple.

If the data is not valuable then all that doesn't matter.

> (and I realize that I'm a great optimist)

Indeed :-)

You should look closely at what it is you actually need.

If you want a close match to the Python object model then use MongoDB.

If you want to use SQL then use Postfix.

If you want a stronger binding to SQLite and the ability to operate with and
without a network then consider using SQLite virtual tables with the backend
talking over the network or locally as needed.

If you need lots of processes on the network to access data quickly then
consider memcached.

If you eventually intend to go for Amazon or Google cloud deployments then
look at what they provide to run locally.

If transactions and ACID matter then carefully research what meets your
needs and deploying using as much redundancy and backup as appropriate.

If you can't make your mind up, write a server that provides your data REST
style and make the clients use HTTP.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1hvnkACgkQmOOfHg372QSFZACfbWZXHwD3+q9xfmfIVAZr9ITO
yHAAn1s3y6w6FV0pW0VPAL1cTfoscB96
=Id/K
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite and SIGFPE

[Richard Hipp]

On Sun, Feb 20, 2011 at 7:39 PM, Samuel Adam  wrote:

> [I]f SQLite can *guarantee* no SIGFPE *ever*—under any possible inputs,
> period, bar none ... that should also be documented.
>

SQLite attempts to intercept division by 0.0, replacing the result with
NULL, but otherwise trusts the C compiler and hardware to "do the right
thing" - which I define to be "don't raise signals".

There is one way to guaranteed that SQLite will never generate a SIGFPE, and
that is to compile with SQLITE_OMIT_FLOATING_POINT, but that is probably not
the answer you were looking for

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] SQLite server using execnet ?

[Stef Mientki]

hello,

knowing that SQllite is not a client/server database,
still want to see if a simple client/server setup would solve my current 
problems for the moment
(because I love the simplicity of SQLlite,
and planned to go to a client / server database in the future)

Now I wonder if anyone has considered  to use Python execnet-module to realize 
a simple SLQlite
client / server application.

If I look at the documentation of execnet,
(and I realize that I'm a great optimist)
it would take between 20 and 50 lines of Python code.

thanks very much for your opinions.
cheers,
Stef Mientki


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Fwd: Scrolling through results of select

[BareFeetWare]

Anyone, anyone, Bueller?

Here's a short version:

How can I best scroll though the results of an arbitrary select query?

Below is the detailed version of my question.

Any help appreciated.

Thanks,
Tom
BareFeetWare

 
From: BareFeetWare 
Date: 16 February 2011 12:05:47 AM AEDT
To: General Discussion of SQLite Database 
Subject: [sqlite] Scrolling through results of select

Hi all,

I'm looking for the best way to scroll through data returned by an arbitrary 
select statement. That is, I want something that will work for any SQL select 
statement string I throw at it at runtime.

I've looked through:

http://www.sqlite.org/cvstrac/wiki?p=ScrollingCursor

and a few posts on this list.

1. It seems that in order to extract, say, the first n rows from a select, I do 
this:

begin
;
create temp view if not exists "Scrolling View" as 
;
select * from "Scrolling View"
order by keyColumn
limit n
;
commit
;

2. And then, to get the next n rows:

maxVisibleKeyValue = last value of keyColumn in visible set (returned above).

begin
;
create temp view if not exists "Scrolling View" as 
;
select * from "Scrolling View"
where keyColumn > :maxVisibleKey
order by keyColumn
limit n
;
commit
;

For the next 10 rows, I should repeat step 2.

For reverse scrolling, I'll run something like:

minVisibleKeyValue = first value of keyColumn in visible set.

begin
;
create temp view if not exists "Scrolling View" as 
;
select * from "Scrolling View"
where keyColumn < :minVisibleKeyValue
order by keyColumn desc
limit n
;
commit
;

Questions:

1. Is there any significant overhead on SQLite from my selecting from a view 
representing the original arbitrary select? That is, will SQLite still use any 
indexes etc correctly? Or do I need to dissect/parse the original select 
statement, changing the where statement etc?

2. If the arbitrary select statement already contains an "order by" clause, 
then I obviously need to use the order by column(s) specified there as the 
keyColumn for the scrolling, rather than override the order by. Is there any 
way to get the sort order of an arbitrary select statement?

3. This method requires that keyColumn is defined as unique (or primary key), 
otherwise it can skip rows of data. Is there any way to allow for a non-unique 
keyColumn?

4. If the arbitrary select statement does not specify an order by, how can I 
least affect the output (ie not impose a sort order) but still facilitate 
scrolling? For selecting from a table, the best I can think of is to use rowid 
(or its alias), which seems to be the typical result order when no order is 
specified. But when selecting from a view (which may contain joins), by which 
column(s) can I explicitly sort (for the sake of scrolling) that will best 
mimic the usual SQL output order (which I know is "undefined")?

5. I understand that "Rule Number 1" is to "not leave queries open". So what's 
the best way to minimize the overhead of repeatedly running the same query but 
with a different where clause and limit (and order if reversing). I'm thinking 
I would be best to actually keep the query (ie prepared statement) open while 
live scrolling (eg flicking through rows on an iPhone/iPad), not using a limit 
clause at all, but instead just keep getting more rows as needed to fill the 
scrolling, until the user stops scrolling, then finalize, close etc. When they 
begin scrolling again, fire up a new prepare (with a new maxVisibleKeyValue) .

Any help appreciated.

Thanks,
Tom
BareFeetWare

--
Comparison of SQLite GUI tools:
http://www.barefeetware.com/sqlite/compare/?ml

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite and SIGFPE

[Samuel Adam]

On Sun, 20 Feb 2011 14:51:12 -0500, Nico Williams   
wrote:

> On Sun, Feb 20, 2011 at 6:24 AM, Samuel Adam  wrote:
[big snip]
>> Any numerics experts (which I am not) or fp-software gurus care to chime
>> in?  *If* I am correct in my inductive hypothesis that the SQLite core  
>> may
>> in rare circumstances trip SIGFPE, I have some suggestions as to
[snip]
>
> Why don’t you try it?

It’s not a question which can be answered by a trivial  
type-numbers-into-the-shell test.  I can immediately think of at least  
two-and-a-half distinct practical reasons why not; a real numerics expert  
with extensive fp programming experience could easily infer what I am  
talking about, then add a few more I’ve probably never heard of.  One  
reason is that different compilers, compiler switches, processor targets,  
processor version “errata” (bugs), and combinations thereof botch up  
floating point calculations in different ways—meaning that your ./sqlite3  
and my sqlite3.exe may give subtly different results in some tests; the  
other reasons, which are more obvious, I will leave as an exercise to the  
reader.

I know enough to understand the problem area conceptually, and to ask some  
precise questions for addressing the practical impact thereof.  In other  
words, with apologies to Einstein, I am sufficiently knowledgeable to  
understand that I know almost nothing.

Thus do I ask, if any Real Mathematicians or fp-calc specialists hit this  
thread, please do feel free to strut your stuff.  As I said:  *If* SQLite  
can in rare circumstances cause a signal to be raised, the result of which  
is officially “implementation-defined” in C, that really ought be  
investigated and documented.  (And if SQLite can *guarantee* no SIGFPE  
*ever*—under any possible inputs, period, bar none—then I would be  
pleasantly quite surprised, and that should also be documented.)

Very truly,

Samuel Adam ◊ 
763 Montgomery Road ◊ Hillsborough, NJ  08844-1304 ◊ United States
Legal advice from a non-lawyer: “If you are sued, don’t do what the
Supreme Court of New Jersey, its agents, and its officers did.”
http://www.youtube.com/watch?v=iT2hEwBfU1g
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to use sqlite and pthread together?

[Samuel Adam]

On Sun, 20 Feb 2011 14:46:06 -0500, Nico Williams   
wrote:

> On Sun, Feb 20, 2011 at 6:28 AM, Samuel Adam  wrote:
>> On Sat, 19 Feb 2011 17:12:31 -0500, Pavel Ivanov 
>> wrote:
>>
>>> [snip] On
>>> Windows it’s different - process is much more heavy-weight object than
>>> thread and involves much bigger system load to support it. There’s an
>>> official general advice for Windows: better create a new thread in the
>>> same process than a new process.
>>
>> Mr. Ivanov explained what I was saying better than I did.  My unclear
>> offhand comment about fork()/exec() was an allusion to why *nix  
>> developed
>> much lighter-weight processes than Windows, viz., decades of a
>> fork()/exec() custom and practice.  (Indeed, I believe that’s precisely
>> why Linux went to the trouble of re-engineering fork() with COW.)  I
>> intended to address the overhead of running, and inadvertently  
>> introduced
>> a red herring about overhead of starting.
>
> You seem to be conflating the weightiness of a notion of process with
> the weightiness of interfaces for creating processes.

I appreciate your extensive (if wildly offtopic) analysis as quoted  
below.  You thoroughly misunderstood what I said, though.  Again, my  
fork()/exec() comment was directed to the same “cultural thing” as you  
spoke about in a different context; and my object thereby was to posit  
__why__ *nix kernel developers have more incentive to make sure processes  
run light.  Winapi doesn’t offer a really equivalent pair of syscalls, nor  
an extensive existing fork-exec practice, so NT kernel developers needn’t  
optimize that use case; whereas *nix kernel folks must of practical  
necessity design their process models to support a typical *nix code  
pattern.  If they do not so do, their users will complain bitterly about  
the overhead of all their daemons’ zillion workers *after* those workers  
are started with the classic fork()/exec().

This being off-topic as it is, I must decline to continue discussing OS  
process practice in front of 10,000 or so people (or so I heard) who tuned  
in for discussion about SQLite.  You said some very interesting stuff,  
though, particularly as to the TLB.  I’d like to leave the door open to  
engaging such discussions in an appropriate venue sometime (ENOTIME for  
the foreseeable future).

Very truly,

Samuel Adam ◊ 
763 Montgomery Road ◊ Hillsborough, NJ  08844-1304 ◊ United States
Legal advice from a non-lawyer: “If you are sued, don’t do what the
Supreme Court of New Jersey, its agents, and its officers did.”
http://www.youtube.com/watch?v=iT2hEwBfU1g


>
> fork() has nothing to do with whether a notion of process is
> light-weight or not.  And quite aside from that, fork() is only as
> light-weight as the writable resident set size of the parent process.
> Long, long ago fork() would copy the parent's address space.  Later on
> fork() implementations started marking what should be writable pages
> as read-only in the MMU page table entries for the process in order to
> catch writes and then copy-on-write.  COW works fine for
> single-threaded processes when the child of fork() intends to exec()
> or exit() immediately and the parent is willing to wait for the child
> to do so.  But for a heavily multi-threaded process with a huge RSS,
> such as a web browser, COW is a performance disaster as it means
> cross-calls to do MMU TLB shoot down, and then incurring a potentially
> large number of page faults in the parent as those threads continue
> executing.  Nowadays it's often simpler and faster to just copy the
> writable portion of the parent's RSS...  vfork(), OTOH, need only
> result in cross-calls to stop the parent's threads, but no page table
> manipulations, TLB shootdowns, data copies, nor page faults need be
> incurred.  And a true posix_spawn() wouldn't even have to stop the
> parent's threads (but using vfork() makes posix_spawn perform so well
> compared to fork() that, for example, Solaris' posix_spawn() just uses
> vfork()).  In Solaris, for example, we've obtained major performance
> improvements by having applications such as web browsers use
> posix_spawn() or vfork() in preference to fork().
>
> In any case, fork() is not an essential attribute of an operating
> system's notion of "process", but an incidental one (related to how
> one creates processes).  In terms of essential attributes, Unix and
> Windows processes compare to each other, and Windows and Unix threads
> (POSIX threads) also compare to each other (roughly anyways, as some
> pthreads implementations have M:N mappings to kernel constructs while
> others have 1:1, and so on).  Yes, Linux has clone(2), which allows
> one to decide just what parts of the parent's various attributes the
> child will share with the parent or get a copy of from the parent, but
> because the standard is pthreads, in practice most developers on Linux
> 

[sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Todd Shutts]

I inherited an application which used SQLite 3.5.8 running on Windows
2000.  I upgrade to version 3.7.5 of the  dll and added pragma statement
to use WAL.  There was an immediate and significant performance
increase.  However; memory usage is growing like crazy.  The application
never used more than 10MB and it is currently using 57+MB and continues
to climb.  

 

I have searched the archives; I don't see anything that applies to
upgrading.I can switch back to the previous version and memory usage
drops and does not grow.  I understand there have been substantial
changes; but I have been unable to find the max usage.  Cache_size is
2000.  Is there any way to limit/control memory or has something changed
related to memory  management where my code needs to change?

Any assistance is appreciated.

 

Thanks.

 


-- 
*
This e-mail may contain confidential or legally privileged information that is 
intended for the individual or entity named as the recipient. 
If you are not the intended recipient, you are hereby notified that any 
disclosure, copying, distribution or reliance upon the contents of this e-mail 
is strictly prohibited. 
If you have received this e-mail in error, please destroy the message 
immediately and contact the sender at Balance Innovations, LLC. Thank you!
*
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] apostrophes in strings...

[Scott Hess]

You can also convert:
  ATTACH DATABASE x AS y KEY z
to:
  SELECT sqlite_attach(x, y, z)
where the parameters can be turned into bind arguments.  Then embedded
quotes won't be an issue.

-scott



On Sun, Feb 20, 2011 at 11:31 AM, Pavel Ivanov  wrote:
> I believe doubling the single quote inside the string literal should help.
>
>
> Pavel
>
> On Sun, Feb 20, 2011 at 2:23 PM, Sam Carleton
>  wrote:
>> This is a bit crazy and I know the ideal way would be to not allow the
>> apostrophy in the first place but, my focus is easy of use for my customers,
>> as compared to easy for me...
>>
>> I had a customer that saved their SQLite database here:
>>
>> D:/My Events/President's Day/event.sqlite
>>
>> My software uses ATTACH to connect to the DB:
>>
>> ATTACH DATABASE 'D:/My Events/President's Day/event.sqlite' AS EventDB;
>>
>> Obviously this isn't working.  Is there some way to escape the apostrophe?
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] SQLite GUI comparison (was: ANN: Base 2.0, Mac SQLite GUI)

[BareFeetWare]

On 21/02/2011, at 3:20 AM, skywind mailing lists wrote:

> in your comparison chart it would also be nice to see which software is able 
> to support SQLite extension. A couple of them do not support the FTS nor 
> RTree capabilities of SQLite.

Sure, I'd be happy to add that. How do you suggest that the feature is worded 
in the table? Is "Supports SQLite extension" accurate? Please let me know what 
value (eg yes or no) I should show for this feature for any apps you know so I 
can add those entries.

Thanks,
Tom
BareFeetWare

 --
Comparison of SQLite GUI tools:
http://www.barefeetware.com/sqlite/compare/?ml

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite and SIGFPE

[Nico Williams]

On Sun, Feb 20, 2011 at 6:24 AM, Samuel Adam  wrote:
> I’m not writing anything multithreaded right now.  But next month or next
> year, the humble little SQL user functions I now make could grow up and
> get plugged into something bigger.  And before I longjmp(), I like to know
> with certainty where I will land.  So I believe I should disable the
> library’s SIGFPE generation and do an ugly mish-mash of error checks
> instead.

Roughly what I recommended.

> For the archives, though, my original question has not been answered.
> SQLite itself does not do anything with or to signals; indeed, it does not
> and should not #include .  Therefore, the default behavior on
> SIGFPE is is “implementation-defined” per C standard; it may be SIG_IGN,
> or it may cause the computer to halt and catch fire.

Libraries in general should not manipulate signal handlers, as it is
difficult to do well and always has the potential to conflict with the
application (since the application can change handlers at any time,
asynchronously to the libraries that care).

> On Fri, 18 Feb 2011 22:32:07 -0500, Roger Binns 
> wrote:
>
>> Using builtins I see NULL or errors being returned:
>>
>>   sqlite> .mode insert
>>   sqlite> select 1/0.0;
>>   INSERT INTO table VALUES(NULL);
>>   sqlite> select abs(-9223372036854775808);
>>   Error: integer overflow
>>
>
> I predict that if I looked at the sources, I’d find that Dr. Hipp made
> SQLite’s divide operator check for a zero underneath it.  But as far as I
> understand, there is no nontrivial way for SQLite to guarantee it can
> prevent generating other floating-point exceptions.  The foregoing does
> not show what will happen if SQLite is fed a calculation resulting in a
> s-NaN, subnormal number, inexact,  and as noted, the default signal
> handler is implementation-defined.  Signed integer overflow is a whole
> different ball of wax (officially “undefined” behavior including
> specifically in  abs(); probably also input-checked in SQLite).
>
> Any numerics experts (which I am not) or fp-software gurus care to chime
> in?  *If* I am correct in my inductive hypothesis that the SQLite core may
> in rare circumstances trip SIGFPE, I have some suggestions as to
> documenting that—including a clear note for idiots that this is not an
> SQLite problem, but rather a general platform-/compiler-dependent
> programming issue which may affect SQLite.

Why don't you try it?

Nico
--
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to use sqlite and pthread together?

[Nico Williams]

On Sun, Feb 20, 2011 at 6:28 AM, Samuel Adam  wrote:
> On Sat, 19 Feb 2011 17:12:31 -0500, Pavel Ivanov 
> wrote:
>
>> [snip] On
>> Windows it’s different - process is much more heavy-weight object than
>> thread and involves much bigger system load to support it. There’s an
>> official general advice for Windows: better create a new thread in the
>> same process than a new process.
>
> Mr. Ivanov explained what I was saying better than I did.  My unclear
> offhand comment about fork()/exec() was an allusion to why *nix developed
> much lighter-weight processes than Windows, viz., decades of a
> fork()/exec() custom and practice.  (Indeed, I believe that’s precisely
> why Linux went to the trouble of re-engineering fork() with COW.)  I
> intended to address the overhead of running, and inadvertently introduced
> a red herring about overhead of starting.

You seem to be conflating the weightiness of a notion of process with
the weightiness of interfaces for creating processes.

fork() has nothing to do with whether a notion of process is
light-weight or not.  And quite aside from that, fork() is only as
light-weight as the writable resident set size of the parent process.
Long, long ago fork() would copy the parent's address space.  Later on
fork() implementations started marking what should be writable pages
as read-only in the MMU page table entries for the process in order to
catch writes and then copy-on-write.  COW works fine for
single-threaded processes when the child of fork() intends to exec()
or exit() immediately and the parent is willing to wait for the child
to do so.  But for a heavily multi-threaded process with a huge RSS,
such as a web browser, COW is a performance disaster as it means
cross-calls to do MMU TLB shoot down, and then incurring a potentially
large number of page faults in the parent as those threads continue
executing.  Nowadays it's often simpler and faster to just copy the
writable portion of the parent's RSS...  vfork(), OTOH, need only
result in cross-calls to stop the parent's threads, but no page table
manipulations, TLB shootdowns, data copies, nor page faults need be
incurred.  And a true posix_spawn() wouldn't even have to stop the
parent's threads (but using vfork() makes posix_spawn perform so well
compared to fork() that, for example, Solaris' posix_spawn() just uses
vfork()).  In Solaris, for example, we've obtained major performance
improvements by having applications such as web browsers use
posix_spawn() or vfork() in preference to fork().

In any case, fork() is not an essential attribute of an operating
system's notion of "process", but an incidental one (related to how
one creates processes).  In terms of essential attributes, Unix and
Windows processes compare to each other, and Windows and Unix threads
(POSIX threads) also compare to each other (roughly anyways, as some
pthreads implementations have M:N mappings to kernel constructs while
others have 1:1, and so on).  Yes, Linux has clone(2), which allows
one to decide just what parts of the parent's various attributes the
child will share with the parent or get a copy of from the parent, but
because the standard is pthreads, in practice most developers on Linux
constrain themselves to using pthreads, thus the concept of clone(2)
is not that relevant here.

> Speaking as a user, by the way, I don’t think I actually have *any*
> Windows applications which use worker processes for concurrency the same
> way my *nix server daemons do.  There’s a reason for that.

It's largely a cultural thing.  Windows NT and up had and promoted
threading from the get-go, while Unix had a very long tradition of
single-threaded processes, and some Unix systems had to catch up to
Windows regarding multi-threading.  There are many other factors
leading to this dichotomy, such as the fact that Unix developers tend
to appreciate isolation, the fact that Window's process spawn API is
so complex and difficult to use, the fact that Windows allows
individual threads of a process to execute with different access
tokens in effect (thus reducing the need to start additional
processes, even if this means losing a lot on the isolation front),
etcetera.  OTOH I've no reason to believe that this split has anything
to do with the weightiness of Windows processes vs. Unix ones (though
the complexity of creating new processes certainly is involved).  But
we do get many heavily multi-threaded applications on Unix nowadays.
Perhaps the Windows model won out... threads _are_ easier to get
started with than processes.

Obtopic: I've successfully used SQLite_2_ with pthreads, and have
every reason to believe that it is possible to safely and productively
use SQLite3 with pthreads.  The key for using SQLite3 in a
multi-threaded way is to adhere to good threaded programming
guidelines while thoroughly understanding the APIs you choose to use.
In particular you should do your utmost to minimize the use of any

Re: [sqlite] apostrophes in strings...

[Pavel Ivanov]

I believe doubling the single quote inside the string literal should help.


Pavel

On Sun, Feb 20, 2011 at 2:23 PM, Sam Carleton
 wrote:
> This is a bit crazy and I know the ideal way would be to not allow the
> apostrophy in the first place but, my focus is easy of use for my customers,
> as compared to easy for me...
>
> I had a customer that saved their SQLite database here:
>
> D:/My Events/President's Day/event.sqlite
>
> My software uses ATTACH to connect to the DB:
>
> ATTACH DATABASE 'D:/My Events/President's Day/event.sqlite' AS EventDB;
>
> Obviously this isn't working.  Is there some way to escape the apostrophe?
>
> Sam
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] apostrophes in strings...

[Sam Carleton]

This is a bit crazy and I know the ideal way would be to not allow the
apostrophy in the first place but, my focus is easy of use for my customers,
as compared to easy for me...

I had a customer that saved their SQLite database here:

D:/My Events/President's Day/event.sqlite

My software uses ATTACH to connect to the DB:

ATTACH DATABASE 'D:/My Events/President's Day/event.sqlite' AS EventDB;

Obviously this isn't working.  Is there some way to escape the apostrophe?

Sam
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Help with join

[Jeff Archer]

>Date: Fri, 18 Feb 2011 07:34:06 -0800
>From: Jim Morris 
>
>A correlated sub-query might work for you.
>
>SELECT
>[Analyzers].[AnalyzerID]
>, [Analyzers].[Name] AS [Analyzer]
>, [Analysis].[AnalysisID]
>, [Analysis].[ScanID]
>, [Analysis].[Timestamp]
>, [Analysis].[EndTime]
>, (SELECT COUNT(*) AS NumDefects FROM Defects d where d.AnalysisID =
Analysis.AnalysisID) as NumDefects,
>, [Analysis].[Result]
>FROM  [Analysis]
>JOIN [Analyzers] ON [Analyzers].[AnalyzerID] = [Analysis].[AnalyzerID]
>ORDER BY [Analysis].[Timestamp];

Thank you, Jim.  This is very handy technique for me know as I am still a
SQL beginner.


--
Jeff Archer
Nanotronics Imaging
jsarc...@nanotronicsimaging.com
<330>819.4615
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] ANN: Base 2.0, Mac SQLite GUI

[skywind mailing lists]

Hi Tom,

in your comparison chart it would also be nice to see which software is able to 
support SQLite extension. A couple of them do not support the FTS nor RTree 
capabilities of SQLite.

Hartwig

Am 20.02.2011 um 07:19 schrieb BareFeetWare:

> Hi Ben,
> 
> In reply to your announcement of Base 2:
> 
>> Just a short message to announce that version 2.0 of Base, our Mac SQLite 
>> GUI is now available.
> 
> The new version looks great. Congrats :-)
> 
>> It's a major upgrade, the highlight of which is the ability for the app to 
>> view, create & alter tables with support for *all* table- and column-level 
>> constraints.
> 
> Neat.
> 
> When I choose "Alter Table", it nicely shows the list of columns, but 
> mistakenly also shows the constraints as if they were columns, with nothing 
> in the Constraints list. I tested a few schemas, including this one:
> 
> create table "Products Detail"
> (
>   Supplier integer
>   not null
>   references Suppliers (ID)
>   on delete restrict
>   on update cascade
> , Code text
>   not null
>   collate nocase
> , Detail text
>   not null
>   collate nocase
> , primary key (Supplier, Code)
> , foreign key (Supplier, Code)
>   references "Products Base" (Supplier, Code)
>   on delete cascade
>   on update cascade
> )
> ;
> 
> which shows in Base 2 as:
> 
>>> Columns:
>>> 
>>> Name  Type Constraints
>>> Supplier  integer  NF
>>> Code  test NC
>>> Detailtext NC
>>> primary   key
>>> 
>>> Constraints:
>>> 
>>> none listed
> 
> The "Alter" panel also shows an "Origin" column, which I think only makes 
> sense in a view.
> 
> When saving a changed table, Base seems to try to explicitly save any auto 
> indexes (which of course fails). For instance, when saving changes to the 
> above table, Base informed me:
> 
>>> There were problems re-creating table indexes. One or more indexes present 
>>> in the table before modification could not be recreated. Their original SQL 
>>> is listed below:
>>> 
>>> CREATE UNIQUE INDEX "sqlite_autoindex_Products Detail_1" ON "Products 
>>> Detail" ("Supplier", "Code");
> 
> I also failed to save the trigger, but that was probably due to it messing up 
> the create table columns (eg adding a column called "primary" etc).
> 
>> You can read the full announcement (with a link to release notes) here: 
>> http://menial.co.uk/2011/02/18/base-2-0/
> 
> I've updated the details for Base 2.0 on my SQLite GUI comparison page at:
> http://www.barefeetware.com/sqlite/compare/?ml
> 
> If there are any corrections or additions you'd like, please let me know.
> 
> Thanks,
> Tom
> BareFeetWare
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] owner/permissions on wal/shm and journal files

[thilo]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
 
Hi,

I am using sqlite 3.7.3 and have noticed the following behavior (which
cause me some headaches).

My DB is owned by a different user (www) and I as root insert some data.

Using pragma journal=delete, the journal file gets created as whoever
starts updating it first,
then the ownership doesn't change (that's ok).

Using wal, the owner of the shared memory and wal file is that of the
current user.
(meaning if root updates the db, www will never be allowed to access
the wal while the root process runs).
Am I right to assume that wal files not only get truncated but also
deleted?

It seems to me that creating the journals should always have a chown
to the owner of the db?
(this would work for root, but not for others on *nix - I understand
that!)
But at least it could be the journal files get created with the same
rw-perms as the db and
if uid==0 then also with the same user-id.

Are there any suggestions on how to make this workable?

cheers thilo
bash-4.1# ls -lnd tst.db* .
drwxr-xr-x  11 1002  1000  3584 Feb 20 17:17 .
- -rw-rw-rw-   1 1002  0 38586368 Feb 20 17:17 tst.db
- -rw-r--r--   1 0 1000 32768 Feb 20 17:17 tst.db-shm
- -rw-r--r--   1 0 1000  2128 Feb 20 17:17 tst.db-wal

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
 
iEYEARECAAYFAk1hPvAACgkQrfkjR68QUaeqIwCffKg8hJDxswjiFnR1W+1ahS0Q
ma0Anjn+U2nNQzhukvxT5sL47/fuw4qT
=YrLC
-END PGP SIGNATURE-

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] EXT :Re: FTS Append?

[Black, Michael (IS)]

Come to think of it that should work.  I was storing terms in single rows and 
had to allow for duplicates.  But converting to FTS relieves that as there are 
two flows in the code where it knows when a record is new or old so I don't 
have to worry about the "record already exists" for this caseit won't exist.

I was just starting to flesh out how to do what I need and was playing with FTS 
to see if it will work for what I need.

Michael D. Black
Senior Scientist
NG Information Systems
Advanced Analytics Directorate




From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
behalf of Puneet Kishor [punk.k...@gmail.com]
Sent: Saturday, February 19, 2011 10:21 AM
To: General Discussion of SQLite Database
Subject: EXT :Re: [sqlite] FTS Append?

On Saturday, February 19, 2011 at 10:17 AM, Black, Michael (IS) wrote:
> sqlite> create virtual table data using fts4(content text);
> sqlite> insert into data values('one two');
> sqlite> insert or replace into data(content) select content||' three four' 
> from data where docid=1;
>
> The docid=1 doesn't work the way I want and I can't seem to figure out how to 
> put in an "as" clause to make it work. Should end up with just one record 
> but, of course, get 2.
>

Shouldn't an UPDATE just work?

UPDATE data
SET content = content || ' three four'
WHERE docid = 1;



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is it possible to determine how many openconnections are active for a sqlite database?

[Black, Michael (IS)]

I don't know if it works for your data...but you don't need to do all 5.4M in 
one batch.
You should test doing it in different batch sizes -- like 1000 at a time (and 
let other processes do their thing potentially).  That way you won't lock them 
out.  But I think your other selects need to use the sqlite3_busy_handler 
function to let them be next in queue.
You will probably not take much longer to insert.

Michael D. Black
Senior Scientist
NG Information Systems
Advanced Analytics Directorate




From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
behalf of Frank Chang [frank_chan...@hotmail.com]
Sent: Saturday, February 19, 2011 11:25 PM
To: sqlite-users@sqlite.org
Subject: EXT :Re: [sqlite] Is it possible to determine how many openconnections 
are active for a sqlite database?

   I wanted to thank Roger Binns for solving my problem. Using sqlite3_request, 
I was able to determine the sqlite database was corrupted when I didn't issue a 
BEGIN EXCLUSIVE before beginning to insert the 5.4 million rows. Evidently, the 
use of BEGIN EXCLUSIVE prevents my transaction from being interrupted by 
another connection from the same process. Thank you.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Compiler warnings in R-Tree code under Visual StudioExpress

[Samuel Adam]

On Sat, 19 Feb 2011 10:37:42 -0500, Afriza N. Arief   
wrote:

> On Sat, Feb 19, 2011 at 6:27 AM, Samuel Adam  wrote:
>
>> A FAQ[2] isn’t enough, as we can see.
>>
>> To put it another way:  Bug reporters should have probable cause before
>> they bug others.  A compiler warning is only a reasonable articulable
>> suspicion.  Note that “probable cause” doesn’t imply computer expertise;
>> “it crashes” is probable cause.  But a compiler warning only means that  
>> a
>> dumb piece of melted sand (i.e., a computer) running a static analysis
>> suggested there might be perhaps something wrong with it, maybe.  Not  
>> that
>> any actual misbehavior was observed.  Relying on a compiler warning  
>> means
>> abdicating wetware to kneel in thrall at the feet of silicon dioxide.
>> It’s wrong and it’s stupid.
>>
>> N.b., I am not accusing hereby Mr. Black of so relying; I just happened  
>> to
>> reply to his message, because the uninitialized-memory trick seemed
>> apropos of his message and I think he as a C coder would duly appreciate
>> the argument (whether or not he agrees).  But the original poster, Mr.
>> Arief, posted an apparent copy-and-paste of such warnings with aught  
>> other
>> said but a helpful link to where we can download MSVC Express.  It  
>> happens
>> here every few months; I am sick of it, ten thousand other list readers
>> are probably sick of it, and it peeves the SQLite team sufficiently that
>> they have a FAQ[2,idem] on the topic.
> [snip]
> Now, I actually did read Testing Process Page[3] a few months back when I
> first knew about SQLite but unfortunately it was not carved into my brain
> since I didn't face any warning when compiling SQLite at that time.
>
> I have googled the warning I found and search the mailing list but did  
> not
> find satisfying result. During my short time searching, I only found the
> FAQ[2] which again, I think should be revised to refer to Testing Process
> Page[3].
>
> Did I mention that I actually read a bit of the code around the warning  
> and
> thought of a way to fix it? Nah, I believe I haven't mention it. I gave  
> up
> because I afraid my fix would actually introduce bugs.
>
> And unfortunately I didn't find/read Testing Process Page[3] during my
> recent search.
>
> Thank you,
>
> Afriza N. Arief

Since you say you checked the sources and found something you thought  
might merit a change, I suggest that you post your suggested fix so  
someone who knows the R-Tree module insides can advise of whether it will  
help or hurt.  Since your original post was even helpful enough to provide  
a link to where people could download MSVC++ Express to reproduce the  
warnings, and you say you had already read the FAQ on compiler warnings,  
may I ask why you made no mention thereby of what you had already found  
reading the source?

SQLite bugs are relatively rare; speaking to experience, the last time I  
reported one on-list, I was promptly shown it had already been found and  
fixed in trunk.  So if you found a bug, I advise you should not hesitate  
to stomp on it.

Very truly,

Samuel Adam ◊ 
763 Montgomery Road ◊ Hillsborough, NJ  08844-1304 ◊ United States
Legal advice from a non-lawyer: “If you are sued, don’t do what the
Supreme Court of New Jersey, its agents, and its officers did.”
http://www.youtube.com/watch?v=iT2hEwBfU1g
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to use sqlite and pthread together?

[Samuel Adam]

On Sat, 19 Feb 2011 17:12:31 -0500, Pavel Ivanov 
wrote:

> [snip] On
> Windows it’s different - process is much more heavy-weight object than
> thread and involves much bigger system load to support it. There’s an
> official general advice for Windows: better create a new thread in the
> same process than a new process.

Mr. Ivanov explained what I was saying better than I did.  My unclear
offhand comment about fork()/exec() was an allusion to why *nix developed
much lighter-weight processes than Windows, viz., decades of a
fork()/exec() custom and practice.  (Indeed, I believe that’s precisely
why Linux went to the trouble of re-engineering fork() with COW.)  I
intended to address the overhead of running, and inadvertently introduced
a red herring about overhead of starting.

Speaking as a user, by the way, I don’t think I actually have *any*
Windows applications which use worker processes for concurrency the same
way my *nix server daemons do.  There’s a reason for that.

Lots to say about threads, but well—that will need await another thread.

Very truly,

Samuel Adam ◊ 
763 Montgomery Road ◊ Hillsborough, NJ  08844-1304 ◊ United States
Legal advice from a non-lawyer: “If you are sued, don’t do what the
Supreme Court of New Jersey, its agents, and its officers did.”
http://www.youtube.com/watch?v=iT2hEwBfU1g
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite and SIGFPE

[Samuel Adam]

[In general reply to interesting answers by Messrs. Williams and Binns:]

The potential SIGFPE in my case is optionally raised by a numeric library  
as part of its error-handling mechanism; its manual expressly suggests  
installing a signal handler and longjmp()ing back to find out what went  
wrong.  It’s a neat functionality, much more convenient and efficient from  
the user’s side than if()ing context bits after every op; most people  
would just do what the manual says, without stopping to think about what  
would happen in multithreaded code.  I guess that’s why Dr. Hipp advises  
people not to use threads.  In this case, the issue is compounded by being  
a tricky race condition (a) unlikely to manifest itself during any kind of  
normal testing, (b) requiring precisely a certain cross-thread interaction  
between seemingly unconnected software components, and (c) severe enough  
to really trash the stack.

I’m not writing anything multithreaded right now.  But next month or next  
year, the humble little SQL user functions I now make could grow up and  
get plugged into something bigger.  And before I longjmp(), I like to know  
with certainty where I will land.  So I believe I should disable the  
library’s SIGFPE generation and do an ugly mish-mash of error checks  
instead.

For the archives, though, my original question has not been answered.   
SQLite itself does not do anything with or to signals; indeed, it does not  
and should not #include .  Therefore, the default behavior on  
SIGFPE is is “implementation-defined” per C standard; it may be SIG_IGN,  
or it may cause the computer to halt and catch fire.

On Fri, 18 Feb 2011 22:32:07 -0500, Roger Binns   
wrote:

> Using builtins I see NULL or errors being returned:
>
>   sqlite> .mode insert
>   sqlite> select 1/0.0;
>   INSERT INTO table VALUES(NULL);
>   sqlite> select abs(-9223372036854775808);
>   Error: integer overflow
>

I predict that if I looked at the sources, I’d find that Dr. Hipp made  
SQLite’s divide operator check for a zero underneath it.  But as far as I  
understand, there is no nontrivial way for SQLite to guarantee it can  
prevent generating other floating-point exceptions.  The foregoing does  
not show what will happen if SQLite is fed a calculation resulting in a  
s-NaN, subnormal number, inexact,  and as noted, the default signal  
handler is implementation-defined.  Signed integer overflow is a whole  
different ball of wax (officially “undefined” behavior including  
specifically in  abs(); probably also input-checked in SQLite).

Any numerics experts (which I am not) or fp-software gurus care to chime  
in?  *If* I am correct in my inductive hypothesis that the SQLite core may  
in rare circumstances trip SIGFPE, I have some suggestions as to  
documenting that—including a clear note for idiots that this is not an  
SQLite problem, but rather a general platform-/compiler-dependent  
programming issue which may affect SQLite.

Very truly,

Samuel Adam ◊ 
763 Montgomery Road ◊ Hillsborough, NJ  08844-1304 ◊ United States
Legal advice from a non-lawyer: “If you are sued, don’t do what the
Supreme Court of New Jersey, its agents, and its officers did.”
http://www.youtube.com/watch?v=iT2hEwBfU1g
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Simon Slavin]

On 20 Feb 2011, at 11:28am, Roger Binns wrote:

> And if your customers care then they will already have existing solutions
> for encryption and protection which includes dealing with incapacitation of
> users, system administration, backups etc.  It is not a good idea to defeat
> those.

And given the actual situation that Robert is writing for, there's a very high 
chance that someone somewhere will write their password down.  Any time someone 
starts talking about the strength of AES-256 outside a need-entry-only 
environment I suspect that time's being wasted.  Yes absolutely: security 
experts have to understand encryption strength and common faults in the use of 
encryption.  But whoever cracks this system isn't going to do it by running a 
brute-force combinatoric attack.

You could write the data to a pen drive rather than the hard disk, and let the 
teacher carry it in a pocket or purse.  That would be more secure than leaving 
the data file on a desktop computer in a school.  If there's any chance the 
teacher will type their password in while students are in the room, you've 
pretty-much blown your security: all it takes is a mobile phone with video 
recording pointing at the teacher and someone can watch shoulder and arm 
movements.  Or even just dust the keyboard soon after a login and find out 
which keys are most greasy.  Or dust the keyboard before a login and find which 
keys don't have dust on any more.  If the teacher has their back to a window, 
have someone shoulder-surf them from outside the window.  Or look at the 
reflection in the window.

It all depends on how much advantage there is to cracking that data.  If the 
students can change their marks, that would be worth cracking.  If the system 
is just used to arrange who gets which lunch-shift, there's less chance anyone 
will bother.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 02:32 AM, Robert Hairgrove wrote:
> There are certain parts of an SQLite database or page header (the first
> 100 bytes, for example) which have known values. I think it is perhaps
> even dangerous to encrypt this data, at least with the same method used
> for the rest of the file.

If the file is not a plain old SQLite file then you must encrypt
*especially* the initial page.  If you do not do this then regular SQLite
can attempt to open the file and depending on luck can reject it, corrupt it
or do other hideous things.

> SEE encrypts the entire file, according to the information on the
> website. 

All of the file contents are encrypted but the entire file is not encrypted
at once.  Instead each page is encrypted separately in order to allow for
random access.  (Note that getting this right is hard and requires careful
design.  For example did you note offset 20 in the SQLite header?)

> So if I leave the headers unencrypted, am I disclosing anything I should
> be (somehow) hiding?

You are seriously wasting your time!  If whole file encryption is ok then
use 7zip and an unmodified SQLite copying across as needed.  If you want the
file encrypted while in use then use SEE.

Any other scheme you come up with will have weaknesses you can drive a truck
through.  In addition there are likely to be bugs and your testing won't be
thorough enough.  Users hate it when you lose their data in the name of
security that doesn't exist :-)

> AES-256 is an accepted
> standard, and AFAICT offers the best openly available encryption today.

Algorithms are not that important - it is how they are initialized,
combined, ordered, padded, randomized, compressed and many other things you
haven't considered.

If you really do still want to proceed then may I suggest just using plain
XOR.  It is trivial to test, hard to implement wrong and good enough.
Anyone who would take the effort to "crack" it would find other ways anyway.
 It is also evidence - ie anyone who gets the plain text contents of the
database had to make an effort to do so and cannot claim to have done so
accidentally.  This would then be sufficiently useful for discipline or
prosecution.

And if your customers care then they will already have existing solutions
for encryption and protection which includes dealing with incapacitation of
users, system administration, backups etc.  It is not a good idea to defeat
those.

Alternatively provide this all SaaS style so that everything lives on
systems you control with a web interface.  That way the data and the
encryption keys will not be living on an arbitrary end user system.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1g+tEACgkQmOOfHg372QQLiQCdHooHlbtW6J+ldqY3ZGROQ4hm
xEEAmwYZ1at5ZroQsQBEUpVhXUNko+PH
=wl8w
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Philip Graham Willoughby]

On 20 Feb 2011, at 10:52, Robert Hairgrove wrote:

> On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote:
>> For this task I would use AES-256 in counter mode with an appropriate nonce 
>> (the counter is trivially derived from the file offset of the block to be 
>> read/written). The key should be derived from the user's password using 
>> 1-iteration PBKDF2 with the SHA-256 hash algorithm as the pluggable hash 
>> function and a suitably long salt.
>> 
>> If you are only doing sequential block writes you can use CBC mode rather 
>> than counter mode - either can be used for random reads.
> 
> Thanks, Phil. This is very helpful to me. AES-256 is an accepted
> standard, and AFAICT offers the best openly available encryption today.

It's still less secure than CBC-mode ROT-13 if you use it incorrectly, and if 
you do not understand why that is you are very likely to do just that.

Best Regards,

Phil Willoughby
-- 
Managing Director, StrawberryCat Limited

StrawberryCat Limited is registered in England and Wales with Company No. 
7234809.

The registered office address of StrawberryCat Limited is:

107 Morgan Le Fay Drive
Eastleigh
SO53 4JH

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Philip Graham Willoughby]

On 20 Feb 2011, at 10:32, Robert Hairgrove wrote:

> On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote:
>> On 20 Feb 2011, at 09:10, Robert Hairgrove wrote:
>> 
>>> I am not starting from scratch doing my own encryption; there are enough
>>> open source libraries publicly available which are good enough for my
>>> purposes.
>> 
>> And all of them offer approximately no security if you use them incorrectly.
> 
> Thanks, I realize this.
> 
> Another question:
> There are certain parts of an SQLite database or page header (the first
> 100 bytes, for example) which have known values. I think it is perhaps
> even dangerous to encrypt this data, at least with the same method used
> for the rest of the file. If I used the same algorithm and key, etc. to
> encrypt the header data as the rest of the file, it might be trivial to
> decrypt it, knowing the published file format (which is explained in
> great detail on the SQLite website).
> 
> SEE encrypts the entire file, according to the information on the
> website. But I'm sure they must have taken this into consideration when
> they designed their library...
> 
> So if I leave the headers unencrypted, am I disclosing anything I should
> be (somehow) hiding?

Any encryption algorithm which is vulnerable to a known plaintext attack is 
considered insecure. Therefore any algorithm which is considered secure does 
not have the problem which you are worrying about.

Best Regards,

Phil Willoughby
-- 
Managing Director, StrawberryCat Limited

StrawberryCat Limited is registered in England and Wales with Company No. 
7234809.

The registered office address of StrawberryCat Limited is:

107 Morgan Le Fay Drive
Eastleigh
SO53 4JH

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Robert Hairgrove]

On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote:
> For this task I would use AES-256 in counter mode with an appropriate nonce 
> (the counter is trivially derived from the file offset of the block to be 
> read/written). The key should be derived from the user's password using 
> 1-iteration PBKDF2 with the SHA-256 hash algorithm as the pluggable hash 
> function and a suitably long salt.
> 
> If you are only doing sequential block writes you can use CBC mode rather 
> than counter mode - either can be used for random reads.

Thanks, Phil. This is very helpful to me. AES-256 is an accepted
standard, and AFAICT offers the best openly available encryption today.

Bob

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Robert Hairgrove]

On Sun, 2011-02-20 at 09:35 +, Philip Graham Willoughby wrote:
> On 20 Feb 2011, at 09:10, Robert Hairgrove wrote:
> 
> > I am not starting from scratch doing my own encryption; there are enough
> > open source libraries publicly available which are good enough for my
> > purposes.
> 
> And all of them offer approximately no security if you use them incorrectly.

Thanks, I realize this.

Another question:
There are certain parts of an SQLite database or page header (the first
100 bytes, for example) which have known values. I think it is perhaps
even dangerous to encrypt this data, at least with the same method used
for the rest of the file. If I used the same algorithm and key, etc. to
encrypt the header data as the rest of the file, it might be trivial to
decrypt it, knowing the published file format (which is explained in
great detail on the SQLite website).

SEE encrypts the entire file, according to the information on the
website. But I'm sure they must have taken this into consideration when
they designed their library...

So if I leave the headers unencrypted, am I disclosing anything I should
be (somehow) hiding?

Bob

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Roger Binns]

On 02/20/2011 01:10 AM, Robert Hairgrove wrote:
> I saw that, but I find it a little intrusive, programmatically speaking,
> from a licensing standpoint.

Why don't you ask DRH then since you won't be the first to want to include
SEE with QT and a commercial app?

> I am not starting from scratch doing my own encryption; there are enough
> open source libraries publicly available which are good enough for my
> purposes.

Yes, there are many libraries, algorithms etc.  Even if you were an expert
in this stuff, there would still be a large possibility of incorrectly using
or combining them.  History is littered with examples.

> Compression is not the same as encryption. 

7zip supports encryption and they have done it right.  For example they have
used key strengthening.  Compressing the data before encryption also helps
since there are fewer patterns.

> I do worry that some student might get hold of the file and try to hack it. 

The students will be able to get physical access so pretty much anything can
be worked around.

> ... hash ... user-supplied passphrase ... value known internally 
> final encryption key ... unique hash value ...

As I said :-)

  Anyone can design a scheme they themselves cannot break. It requires
  far more skill and experience to come up with something that is
  actually strong.

In your situation I would just use SEE working with DRH to ensure
appropriate usage.  If whole file encryption is okay then I would use 7zip
and its encryption features with temporary files plus the backup API in
order to copy the database between the 7zip archive and the regular
filesystem.  Using a 7zip archive also lets you keep older copies etc.

You implementing or using any kind of encryption scheme also means you
defeat good system management practises.  For example if whoever sets the
password is incapacitated then the data cannot be recovered.  Good systems
management practises will typically use encryption systems (eg a filesystem)
that can be accessed both by the user and by appropriate administrators.

> As to the gun, Bruce Schneier already pointed out that this is one of
> the more expensive options in the attack tree. ;)

$60k seems like a lot and he refers to a gang.  My swag is that you could
hire a local thug for a few thousand to wave a gun around, all depending on
the possibility of being caught or observed.  Safes are far more likely to
be somewhere secure and discreet and to contain valuable items.

Roger
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Philip Graham Willoughby]

On 20 Feb 2011, at 09:10, Robert Hairgrove wrote:

> I am not starting from scratch doing my own encryption; there are enough
> open source libraries publicly available which are good enough for my
> purposes.

And all of them offer approximately no security if you use them incorrectly.

For this task I would use AES-256 in counter mode with an appropriate nonce 
(the counter is trivially derived from the file offset of the block to be 
read/written). The key should be derived from the user's password using 
1-iteration PBKDF2 with the SHA-256 hash algorithm as the pluggable hash 
function and a suitably long salt.

If you are only doing sequential block writes you can use CBC mode rather than 
counter mode - either can be used for random reads.

Best Regards,

Phil Willoughby
-- 
Managing Director, StrawberryCat Limited

StrawberryCat Limited is registered in England and Wales with Company No. 
7234809.

The registered office address of StrawberryCat Limited is:

107 Morgan Le Fay Drive
Eastleigh
SO53 4JH

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] permissions for created database files are too restrictive and do not obey umask setting

[Anders Lennartsson]

File permissions on database files created by the library libsqlite3
on Unix/Linux are created with a mask setting that restricts
permissions beyond what the user may expect from the current umask
setting.

In the file src/os_unix.c a mask is defined as:

 ** Default permissions when creating a new file
 */
 #ifndef SQLITE_DEFAULT_FILE_PERMISSIONS
 # define SQLITE_DEFAULT_FILE_PERMISSIONS 0644
 #endif

which is applied when the open function is called to open a database
and create it if necessary. In many cases the user has umask settings
022 and new files are thus created with 644 permissions which are what
the user expects.

However, if the user has an umask setting of 002, to allow write
permissions for the group by giving newly created files permissions as
664, the mask defined in the sqlite source code restricts this to 644
nevertheless, as the default file permission defined above is a value
for masking the permissions.

IMHO this is an error. A valid example of that are on server machines
where web services such as Trac are provided. If several
administrators work together such machines are often configured so
that the SGD bit is set on the directories where Trac-webs are stored,
and the group for them are set to the user name that runs the
web-server. On Debian/Ubuntu systems that is www-data. The intention
of this is to allow the web-server to write in the database file, but
also to allow an administrator to do administration such as deleting a
file created by another administrator.

With Subversion 1.6 there is an essentially identical problem arising
as sqlite3 is used within Subversion repositories in the form of the
file db/rep-cache.db. This file is created when something is put in
the repository and not when the repository is created (with svnadmin
create). Thus everything looks fine and files and directories have the
correct permissions in a newly created repository. But, the database
file that is created when something is put into the repository may not
be writable by the web-server process, as it will have 644
permissions, and the web-server process will then report an error.

A work-around for this is to create an empty database file (e.g. by
touch) and give it correct permissions (664). But it is clearly
tedious and error prone to depend on this type of extra tasks for
something that should be done correctly in the first place.

This problem is reported as Debian bug #608604, but it has come to my
attention that other distributions are also bitten by this and it is
probably best fixed upstream.

I have searched the sqlite archives and this topic sometimes surfaces,
the most recent mentioning seems to be on Jan 10, 2011:
http://comments.gmane.org/gmane.comp.db.sqlite.general/62194
Another thread is started by
http://article.gmane.org/gmane.comp.db.sqlite.general/26395
where some posts indicate a full understanding of the problem.

In summary, I'd like to report this as a bug. A trivial fix exists
which does not change the behaviour for most users but does help in a
common situation where many people cooperate on administrating
important production servers on which sqlite databases are used.

Anders
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Implementing custom xRead() and xWrite() routines for sqlite3_vfs

[Robert Hairgrove]

On Sat, 2011-02-19 at 17:08 -0800, Roger Binns wrote:
> On 02/19/2011 03:40 PM, Robert Hairgrove wrote:
> > But before I "try this at home", I thought I would ask if there are any
> > caveats I should be aware of? Thanks for any helpful advice!

Thanks, Roger ... those are great links!

> Unless your time has no value, I'd suggest using this:
> 
>   http://www.hwaci.com/sw/sqlite/see.html

I saw that, but I find it a little intrusive, programmatically speaking,
from a licensing standpoint. Although SQLite itself is PD licensed, I am
using it coupled with the Qt source code which is LGPL licensed. So I'm
not really not sure how this encryption add-in would play with that if I
have to recompile something I got from Qt in the first place. My
application is a closed-source commercial one, and linking dynamically
to Qt without changing any of the Qt sources is allowed under the LGPL.

If I can hook in my own read/write routines like this at runtime, I
won't have to alter and/or recompile any of the library sources. I
*could* build the Qt SQLite module with SEE as a plug-in, but would
rather link it statically (and the SEE people apparently want it that
way, too). In this case, I would need to treat the SQLite sources the
same as any of the other Qt sources. But "IANAL", so maybe I am being
too cautious here.

> (Note that it is supported, tested and cryptographically sound.  It would
> take you a long time to achieve the same.)

I am not starting from scratch doing my own encryption; there are enough
open source libraries publicly available which are good enough for my
purposes.

> If you just want whole file encryption then I'd recommend using an archive
> tool and storing/extracting as appropriate.  For example 7zip does this well
> and is open source.

Compression is not the same as encryption. The application is primarily
for school teachers (single-user desktop use, which is why SQLite is
ideal for this) and the database might contain stuff like confidential
student reports, grades, etc. Since I am not trying to protect highly
sensitive government secrets or medical data here, although someone
could probably use the app for that as well, I do worry that some
student might get hold of the file and try to hack it. A clever student
would see right away that compression and not encryption was employed --
and a student's time usually DOES come cheap! ;)

> If you really want to do your own thing then beware that the encryption key
> has to be where the data is encrypted/decrypted.  You should carefully study
> exactly what it is you a protecting, who you are protecting it from, how
> long it is protected etc.  These can help:
> 
>   http://www.schneier.com/paper-attacktrees-ddj-ft.html
> 
> If you really do still want to proceed then xRead/xWrite are an appropriate
> place to do it.

Thanks, this is what I needed to know.

>   http://en.wikipedia.org/wiki/Initialization_vector
> 
> Anyone can design a scheme they themselves cannot break.  It requires far
> more skill and experience to come up with something that is actually strong.
> 
> Also consider that what you may actually need is just some obfuscation.  For
> example you could just XOR the database contents with deterministic bytes.
> If you did this then seeing the contents would go from costing a few dollars
> (load the file into the command line shell) into a few hundred or thousand
> (figure out what it is you did).  In any event an attacker could always
> point a gun or use a hardware keylogger if they don't want to be discovered.
>  That would workaround any encryption scheme.

I would hash the user-supplied passphrase with a value known internally
to my program to create the final encryption key, so the hacker would
have to have a copy of the executable of my program in addition to
whatever data was gleaned by using a key logger. And each licensed copy
of my program would have a unique hash value embedded within the
executable.

As to the gun, Bruce Schneier already pointed out that this is one of
the more expensive options in the attack tree. ;)

Bob

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users