Re: [sqlite] Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

[Jörg Knobloch]

On 24/08/2017 13:02, Richard Hipp wrote:

The problem, if my analysis is correct, should be obscure and require
a very specific query to hit.


Thanks, your patch works. I believe our use case is as obscure as you 
can think ;-(


Jörg.

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

[Richard Hipp]

On 8/24/17, Jörg Knobloch  wrote:
> On 24/08/2017 11:40, Clemens Ladisch wrote:
>> Jörg Knobloch wrote:
>>> Mozilla have upgraded to SQLite 3.20.0 and that has caused the
>>> Thunderbird test suite some test failures [1] as follows:
>>>
>>> Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file
>>> sqlite3.c, line 70285
>> Hmm, is there a custom memory allocator somewhere?
>> And one can never rule out memory corruption ...
>>
> Yes, as far as I'm aware Mozilla have their own (twisted and
> complicated) memory management, let me see:
>
> https://dxr.mozilla.org/mozilla-central/source/memory
>
> My position is a little difficult, since I'm the Thunderbird maintainer,
> not a Mozilla core developer. Thunderbird has a small team using Mozilla
> core technology (frequently referred to a "Gecko"). Mozilla core is
> backed by hundreds of employees, some look after the memory management
> and also a component called mozStorage which interfaces with SQLite. The
> maintainers of that component sent me here.

I think the problem is a bug in the new sqlite3_result_pointer()
interface.  I added a proposed patch to the ticket.  But we are still
working on getting a test case here.

The problem, if my analysis is correct, should be obscure and require
a very specific query to hit.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

[Jörg Knobloch]

On 24/08/2017 11:40, Clemens Ladisch wrote:

Jörg Knobloch wrote:

Mozilla have upgraded to SQLite 3.20.0 and that has caused the
Thunderbird test suite some test failures [1] as follows:

Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file sqlite3.c, 
line 70285

Hmm, is there a custom memory allocator somewhere?
And one can never rule out memory corruption ...

Yes, as far as I'm aware Mozilla have their own (twisted and 
complicated) memory management, let me see:


https://dxr.mozilla.org/mozilla-central/source/memory

My position is a little difficult, since I'm the Thunderbird maintainer, 
not a Mozilla core developer. Thunderbird has a small team using Mozilla 
core technology (frequently referred to a "Gecko"). Mozilla core is 
backed by hundreds of employees, some look after the memory management 
and also a component called mozStorage which interfaces with SQLite. The 
maintainers of that component sent me here.


Jörg.


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

[Clemens Ladisch]

Jörg Knobloch wrote:
> Mozilla have upgraded to SQLite 3.20.0 and that has caused the
> Thunderbird test suite some test failures [1] as follows:
>
> Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file sqlite3.c, 
> line 70285

Hmm, is there a custom memory allocator somewhere?
And one can never rule out memory corruption ...

> One of the Mozilla developers was of the opinion that the custom
> tokenizer and a custom ranking function we're using might be at the
> root of the problem [...] might need to be updated in view of the
> pointer interfaces mentioned here: https://sqlite.org/bindptr.html.

That is a new interface; the old method still works.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

[Jörg Knobloch]

Dear SQLite developer,

Mozilla have upgraded to SQLite 3.20.0 and that has caused the 
Thunderbird test suite some test failures [1] as follows:


Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file 
sqlite3.c, line 70285


One of the Mozilla developers was of the opinion that the custom 
tokenizer and a custom ranking function we're using might be at the root 
of the problem and mentioned that the example which we copied 
(https://sqlite.org/fts3.html#appendix_a, very end, aMatchinfo = 
(unsigned int *)sqlite3_value_blob(apVal[0]);) might need to be updated 
in view of the pointer interfaces mentioned here: 
https://sqlite.org/bindptr.html.


Any help would be appreciated.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1392937

--
Jörg Knobloch - jo...@jorgk.com - www.jorgk.com 

Thunderbird Developer 
(Thunderbird 
, Compose 
and Mailnews 
Editor and MIME 
peer) - 
Member of the Thunderbird Council 



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 16/06/13 17:55, Simon Slavin wrote:
>> Here’s a nice Python wrapper:
>> 
>> 
> 
> Woops.  That’s Windows, you’re Ubuntu.  Don’t you have the sqlite3
> Python module

APSW is not Windows.  It is provided as source for all platforms, and also
binaries for Windows because Windows users typically do not have C
compilers available (and even if they do they have to match Python).

The recommended APSW build instructions are below.
(The Windows binaries were built that way too).

  http://apidoc.apsw.googlecode.com/hg/build.html#recommended

In particular note that it downloads and statically includes the SQLite
amalgamation completely ignoring any system SQLite.

pysqlite's setup.py script does have an option to do that, but I doubt it
works any more.  The pysqlite source has not had any changes in two years.

Also if you want to use SQLite and not pretend that it is the same as
other databases then this will be of interest:

  http://apidoc.apsw.googlecode.com/hg/pysqlite.html

(Disclosure: I am the author of APSW)

Roger

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlG+pO8ACgkQmOOfHg372QQLZQCgxhbB/EHUBdAZ3vVX78a4u3eY
9XkAoK3de4S0zjad3978iDdO7zRfrXCv
=eFbl
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Keith Medcalf]

The apsw package compiles and works on Linux just fine.

The python sqlite3 package uses the system libsqlite3.so module.  When you 
compile apsw.so, you can choose to use either the system libsqlite3.so or to 
compile your own sqlite3 amalgamation code into the extension.


---
()  ascii ribbon campaign against html e-mail
/\  www.asciiribbon.org

> -Original Message-
> From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-
> boun...@sqlite.org] On Behalf Of Simon Slavin
> Sent: Sunday, 16 June, 2013 18:56
> To: General Discussion of SQLite Database
> Subject: Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server
> 
> 
> On 17 Jun 2013, at 1:50am, Simon Slavin <slav...@bigfraud.org> wrote:
> 
> >
> > On 17 Jun 2013, at 1:44am, Merton Lister <mrtnl...@gmail.com> wrote:
> >
> >> My application is Python-based, and I'm using py-sqlite module to
> connect
> >> to sqlite. Where should I put the .c and .h files?
> >
> > Here's a nice Python wrapper:
> >
> > <http://code.google.com/p/apsw/>
> 
> Woops.  That's Windows, you're Ubuntu.  Don't you have the sqlite3 Python
> module
> 
> <http://docs.python.org/2/library/sqlite3.html>
> 
>  ?  Try
> 
> import sqlite3 as lite
> 
> does it give an error ?
> 
> Take a look here:
> 
> <http://zetcode.com/db/sqlitepythontutorial/>
> 
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Simon Slavin]

On 17 Jun 2013, at 1:50am, Simon Slavin  wrote:

> 
> On 17 Jun 2013, at 1:44am, Merton Lister  wrote:
> 
>> My application is Python-based, and I'm using py-sqlite module to connect
>> to sqlite. Where should I put the .c and .h files?
> 
> Here’s a nice Python wrapper:
> 
> 

Woops.  That’s Windows, you’re Ubuntu.  Don’t you have the sqlite3 Python module



 ?  Try

import sqlite3 as lite

does it give an error ?

Take a look here:



Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Simon Slavin]

On 17 Jun 2013, at 1:44am, Merton Lister  wrote:

> My use case falls to the third case, i.e. writing an application which
> talks to a sqlite db. You suggested that I should include the .c and .h
> files from the amalgamation in my app. Specifically, how should I do that?
> My application is Python-based, and I'm using py-sqlite module to connect
> to sqlite. Where should I put the .c and .h files?

Here’s a nice Python wrapper:



Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Merton Lister]

Hi Simon,

Thank you for the excellent explanation!

My use case falls to the third case, i.e. writing an application which
talks to a sqlite db. You suggested that I should include the .c and .h
files from the amalgamation in my app. Specifically, how should I do that?
My application is Python-based, and I'm using py-sqlite module to connect
to sqlite. Where should I put the .c and .h files?

Thanks a lot.

Regards,

Merton


On Mon, Jun 17, 2013 at 12:19 AM, Simon Slavin  wrote:

>
> On 17 Jun 2013, at 12:01am, Merton Lister  wrote:
>
> > What is the best way for upgrading sqlite3 on an Ubuntu server (12.04 or
> > 10.04)?
>
> SQLite3 does not exist as a server application.  It’s not a pre-compiled
> library or a daemon which runs in the background.  Instead it comes as
> source code which programmers can build into their own applications.  It’s
> quite possible you have five programs on your computer, each of which is
> using a different version of SQLite.  (It’s tiny.  Having five copies take
> up disk space is easily worth not having to worry about interrelated
> version dependencies.)
>
> Since you mentioned Ubuntu Server rather than just the client what you
> might be talking about is that a SQLite library is part of PHP which is
> part of Apache.  To update the version of SQLite used for that, update PHP
> or Apache.
>
> Alternatively you might want to write an application which talks to SQLite
> databases.  If so, the cannonical answer is to download the latest
> amalgamation of SQLite from
>
> 
>
> This is one .c and one .h file which you include in your app.
>
> A little warning: if you have some application which uses SQLite, don’t
> update the version of SQLite it’s using other than by updating the
> application.  Applications sometimes have work-arounds for bugs in earlier
> versions of SQLite and if you update SQLite the application can stop
> working because a bug it’s depending on is no longer present.
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Simon Slavin]

On 17 Jun 2013, at 12:01am, Merton Lister  wrote:

> What is the best way for upgrading sqlite3 on an Ubuntu server (12.04 or
> 10.04)?

SQLite3 does not exist as a server application.  It’s not a pre-compiled 
library or a daemon which runs in the background.  Instead it comes as source 
code which programmers can build into their own applications.  It’s quite 
possible you have five programs on your computer, each of which is using a 
different version of SQLite.  (It’s tiny.  Having five copies take up disk 
space is easily worth not having to worry about interrelated version 
dependencies.)

Since you mentioned Ubuntu Server rather than just the client what you might be 
talking about is that a SQLite library is part of PHP which is part of Apache.  
To update the version of SQLite used for that, update PHP or Apache.

Alternatively you might want to write an application which talks to SQLite 
databases.  If so, the cannonical answer is to download the latest amalgamation 
of SQLite from



This is one .c and one .h file which you include in your app.

A little warning: if you have some application which uses SQLite, don’t update 
the version of SQLite it’s using other than by updating the application.  
Applications sometimes have work-arounds for bugs in earlier versions of SQLite 
and if you update SQLite the application can stop working because a bug it’s 
depending on is no longer present.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Upgrade/Install sqlite3 on Ubuntu server

[Merton Lister]

Hi,

What is the best way for upgrading sqlite3 on an Ubuntu server (12.04 or
10.04)?

The official package maintained by Ubuntu is way behind the latest version
offered on sqlite.org. I would like to know a way to upgrade sqlite3 as its
new versions get released.

Is there a PPA available? Or do I have to compile from source every time
for a new version? Or there is some other way?

Thanks!

Regards,

Merton
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] change after sqlite upgrade with python

[Richard Hipp]

On Mon, Sep 19, 2011 at 10:22 PM, Jamie Winquist  wrote:

> In my environment, we are connecting to the sqlite file using
> pysqlite2.dbapi2.  After upgrading from sqlite 3.5.4 to 3.7.7.1, iterating
> over the results of a cursor.execute() are yielding different keys than they
> did pre-upgrade.  My query is of the form:
>
> SELECT table.column1, table.column2 FROM table;
>
> With sqlite 3.5.4, when iterating over the cursor, the keys in each result
> row were named "table.column1" and "table.column2" as specified in the
> query.  After the upgrade to sqlite 3.7.7.1, the keys in each result row do
> not include the table name in the key, but are solely the column name.  Is
> this expected?
>

If you use an AS clause to define the names of result columns, then they
will always follow those names.  Ex;

SELECT table.column1 AS result1, table.column2 AS result2 FROM table;

If you omit the AS clause, then the result column names can vary according
to compile-time and run-time settings and which version of SQLite you are
using.  We don't deliberately change things just to frustrate you.  But
changes do happen.  I do not recall any changes to the result column names
between 3.5.4 and 3.7.8, but 3.5.4 was a really long time ago so I might
easily have forgotten something.


>
> Thanks.
>
> Jamie
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] change after sqlite upgrade with python

[Simon Slavin]

On 20 Sep 2011, at 3:22am, Jamie Winquist wrote:

> After upgrading from sqlite 3.5.4 to 3.7.7.1, iterating over the results of a 
> cursor.execute() are yielding different keys than they did pre-upgrade.  My 
> query is of the form:
> 
> SELECT table.column1, table.column2 FROM table;

Read the documentation for



though you should not rely on those PRAGMAs in the long term, or do any 
programming which relies on them existing in future versions of SQLite.

You might want to use those PRAGMAs for a short-term fix which you're 
re-writing your app, perhaps to use 'AS', in the longer term.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] change after sqlite upgrade with python

[Stephan Beal]

On Tue, Sep 20, 2011 at 4:22 AM, Jamie Winquist  wrote:

> With sqlite 3.5.4, when iterating over the cursor, the keys in each result
> row were named "table.column1" and "table.column2" as specified in the
> query.  After the upgrade to sqlite 3.7.7.1, the keys in each result row do
> not include the table name in the key, but are solely the column name.  Is
> this expected?
>

Yes, it's expected and well documented. The names of ANY columns is
UNSPECIFIED unless you use an AS qualifier to give them the names you want.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] change after sqlite upgrade with python

[Jamie Winquist]

In my environment, we are connecting to the sqlite file using pysqlite2.dbapi2. 
 After upgrading from sqlite 3.5.4 to 3.7.7.1, iterating over the results of a 
cursor.execute() are yielding different keys than they did pre-upgrade.  My 
query is of the form:

SELECT table.column1, table.column2 FROM table;

With sqlite 3.5.4, when iterating over the cursor, the keys in each result row 
were named "table.column1" and "table.column2" as specified in the query.  
After the upgrade to sqlite 3.7.7.1, the keys in each result row do not include 
the table name in the key, but are solely the column name.  Is this expected?

Thanks.

Jamie


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Todd Shutts]

I am looking at the private byes counter from Windows (MSDN Description
below) for my process.  Is there another counter or stat I should be
looking at?  

(from MSDN - Private Bytes: 
Displays the current number of bytes this process has allocated that
cannot be shared with other processes.)

I receive similar behavior (although it does not climb as fast-because
it doesn't run as fast) with Journal_mode=delete.  When using WAL; I set
the wal_autocheckpoint to 250 and called WAL_CHECKPOINT without any
measurable result to see if I see any difference.

I have left the application running and the private bytes continues to
increase; I am 150MB and counting  My code has to be leaking; but why
not with the older version?

Thanks for all the help

Todd


-Original Message---
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Roger Binns
Sent: Sunday, February 20, 2011 7:48 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory
usage

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 02:48 PM, Todd Shutts wrote:
> However; memory usage is growing like crazy. 

It is a very frequent occurrence that what people think is being
measured is
not what is actually being measured.  Make sure you understand exactly
what
the tool showing you memory consumption is measuring and especially
understand if it includes memory shared with other processes or memory
mapped files.  Additionally you need to understand the difference
between
memory as the kernel sees and reports it versus memory within particular
APIs such as C's malloc.

> The application
> never used more than 10MB and it is currently using 57+MB and
continues
> to climb.  

The single most likely explanation is this is WAL in action, the memory
is
from a memory mapped file and a WAL checkpoint will release it.

  http://www.sqlite.org/wal.html

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1hxDYACgkQmOOfHg372QRikgCdHrEuzE5p71LTaiF+WRHfG6j2
9S0An100kCApkwZI74XGYR6zxczr2m7u
=d0xw
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Simon Slavin]

On 21 Feb 2011, at 5:56am, Dan Kennedy wrote:

> WAL file growth has (apparently) been a problem for people with
> really busy systems. But so far it has been the actual size of
> the file on disk that bothers them, not the memory mapped bit.

If that's what you think is raising concern then fine, someone needs to write 
something about that.  But looking at the actual posts to this list gives me 
questions about reported memory-usage.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Dan Kennedy]

On 02/21/2011 09:38 AM, Simon Slavin wrote:
>
> On 21 Feb 2011, at 1:47am, Roger Binns wrote:
>
>> On 02/20/2011 02:48 PM, Todd Shutts wrote:
>>> The application
>>> never used more than 10MB and it is currently using 57+MB and continues
>>> to climb.
>>
>> The single most likely explanation is this is WAL in action, the memory is
>> from a memory mapped file and a WAL checkpoint will release it.
>>
>>   http://www.sqlite.org/wal.html
>
> Todd's question, or something like it, has been asked at least seven times on 
> this list in the seven months since WAL was released.  Might I suggest that 
> an explanation of WAL's use of lots of memory be placed higher up on that 
> page and headed something like "Memory usage" ?
>
> Also, something like "May use many megabytes of memory between checkpoints." 
> should be listed in the 'disadvantages' list.  The explanation of 
> checkpointing is fine, but users don't know they need to read it until 
> they've asked here.

The size of the memory mapped file is about 8 bytes per page
in the WAL log, allocated in 32 KB chunks.

So if the poster is using a page size of 1024 bytes, a 47MB
memory mapped file  corresponds to roughly a 6GB WAL log. Or
if he is using the win32 default page size of 4K, a WAL file
four times that large.

WAL file growth has (apparently) been a problem for people with
really busy systems. But so far it has been the actual size of
the file on disk that bothers them, not the memory mapped bit.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Simon Slavin]

On 21 Feb 2011, at 1:47am, Roger Binns wrote:

> On 02/20/2011 02:48 PM, Todd Shutts wrote:
>> The application
>> never used more than 10MB and it is currently using 57+MB and continues
>> to climb.  
> 
> The single most likely explanation is this is WAL in action, the memory is
> from a memory mapped file and a WAL checkpoint will release it.
> 
>  http://www.sqlite.org/wal.html

Todd's question, or something like it, has been asked at least seven times on 
this list in the seven months since WAL was released.  Might I suggest that an 
explanation of WAL's use of lots of memory be placed higher up on that page and 
headed something like "Memory usage" ?

Also, something like "May use many megabytes of memory between checkpoints." 
should be listed in the 'disadvantages' list.  The explanation of checkpointing 
is fine, but users don't know they need to read it until they've asked here.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2011 02:48 PM, Todd Shutts wrote:
> However; memory usage is growing like crazy. 

It is a very frequent occurrence that what people think is being measured is
not what is actually being measured.  Make sure you understand exactly what
the tool showing you memory consumption is measuring and especially
understand if it includes memory shared with other processes or memory
mapped files.  Additionally you need to understand the difference between
memory as the kernel sees and reports it versus memory within particular
APIs such as C's malloc.

> The application
> never used more than 10MB and it is currently using 57+MB and continues
> to climb.  

The single most likely explanation is this is WAL in action, the memory is
from a memory mapped file and a WAL checkpoint will release it.

  http://www.sqlite.org/wal.html

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1hxDYACgkQmOOfHg372QRikgCdHrEuzE5p71LTaiF+WRHfG6j2
9S0An100kCApkwZI74XGYR6zxczr2m7u
=d0xw
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Upgrade from 3.5.8 -> 3.7.5 - increase memory usage

[Todd Shutts]

I inherited an application which used SQLite 3.5.8 running on Windows
2000.  I upgrade to version 3.7.5 of the  dll and added pragma statement
to use WAL.  There was an immediate and significant performance
increase.  However; memory usage is growing like crazy.  The application
never used more than 10MB and it is currently using 57+MB and continues
to climb.  

 

I have searched the archives; I don't see anything that applies to
upgrading.I can switch back to the previous version and memory usage
drops and does not grow.  I understand there have been substantial
changes; but I have been unable to find the max usage.  Cache_size is
2000.  Is there any way to limit/control memory or has something changed
related to memory  management where my code needs to change?

Any assistance is appreciated.

 

Thanks.

 


-- 
*
This e-mail may contain confidential or legally privileged information that is 
intended for the individual or entity named as the recipient. 
If you are not the intended recipient, you are hereby notified that any 
disclosure, copying, distribution or reliance upon the contents of this e-mail 
is strictly prohibited. 
If you have received this e-mail in error, please destroy the message 
immediately and contact the sender at Balance Innovations, LLC. Thank you!
*
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] UPGRADE to 3.6.20 and python

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Cooper, Andrew wrote:
> This has probably been asked a lot of times before but I couldn't find
> an answer in the archives.

You should ask on the python sqlite list :-)  It is jointly maintained by
the authors of the two Python to SQLite bindings:

  http://groups.google.com/group/python-sqlite

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr585IACgkQmOOfHg372QSDjACgmeA8rzqKT0KW9rEzIFGK25lf
Iv0AmwQUWRZiOVaMMKzG1I6SKp6Tge8X
=5KAA
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] UPGRADE to 3.6.20 and python

[Cooper, Andrew]

This has probably been asked a lot of times before but I couldn't find
an answer in the archives.

 

I want to upgrade the version of sqlite in python 2.5 to use 3.6.20(the
latest)

 

What steps do I need to do to achieve this ?

Does pysqlite need to be installed and if so does this support the
latest version of sqlite ?

 

Oh and my target machine is Linux

 

Thanks in advance

 

Andy Cooper

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.9 to 3.6.11

[D. Richard Hipp]

On Mar 13, 2009, at 9:22 PM, Kees Nuyt wrote:

> On Fri, 13 Mar 2009 08:57:35 -0700 (PDT), Joanne Pham
>  wrote:
>
>> Hi All,
>> We have a application using SQLite 3.5.9 now
>> and we will be releasing this product in June.
>>  I am think about upgrading SQLite from 3.5.9 to
>> SQLite 3.6.11 but I don't know what are the
>> impact for the application and is it worth
>> to upgrade SQLite to newest one before the
>> product is releaseed.
>> Would like to have your input on this.
>
> It is hard to have an opinion on this, because we don't know
> your application. I would advise to study the release notes:
> http://www.sqlite.org/changes.html
>
> If you require more detail, there is always the timeline:
> http://www.sqlite.org/cvstrac/timeline
> which documents all solved bug tickets and all code
> checkins.
>
> If you have a testsuite which covers your application very
> well, you can just give it a go with 3.6.11.


Everything that Kees says is correct.  Even so, I say the answer to  
Joanne's question is "Yes".  You should upgrade.  To at least 3.6.11  
if not 3.6.12 (due out in 10 days or so).


D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.5.9 to 3.6.11

[Kees Nuyt]

On Fri, 13 Mar 2009 08:57:35 -0700 (PDT), Joanne Pham
 wrote:

> Hi All,
> We have a application using SQLite 3.5.9 now
> and we will be releasing this product in June.
> I am think about upgrading SQLite from 3.5.9 to
> SQLite 3.6.11 but I don't know what are the
> impact for the application and is it worth
> to upgrade SQLite to newest one before the
> product is releaseed.
> Would like to have your input on this. 

It is hard to have an opinion on this, because we don't know
your application. I would advise to study the release notes:
http://www.sqlite.org/changes.html

If you require more detail, there is always the timeline:
http://www.sqlite.org/cvstrac/timeline
which documents all solved bug tickets and all code
checkins.

If you have a testsuite which covers your application very
well, you can just give it a go with 3.6.11.

> Thank in advance,
> JP
-- 
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Upgrade from 3.5.9 to 3.6.11

[Joanne Pham]

Hi All,
We have a application using SQLite 3.5.9 now and we will be releasing this 
product in June.
 I am think about upgrading SQLite from 3.5.9 to SQLite 3.6.11 but I don't know 
what are the impact for the application and is it worth to upgrade SQLite to 
newest one before the product is releaseed. Would like to have your input on 
this. 
Thank in advance,
JP


  
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7

[Edward J. Yoon]

Thanks for your helpful information!

On Wed, Jan 14, 2009 at 12:48 AM, Griggs, Donald
<donald.gri...@allscripts.com> wrote:
> Subject: [sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7
>
> Hi,
>
> I consider upgrade sqlite 3.3.4 to sqlite 3.6.7. So, I wonder there is
> any change (or problem) of file format.
> ===
>
> Upgrading from version 2 to version 3 (understandably) required a dump
> and restore, but not upgrading from one version 3 to a newer one.
>
> Following is from page: http://sqlite.org/different.html
>
> Stable Cross-Platform Database File
>
> The SQLite file format is cross-platform. A database file written on one
> machine can be copied to and used on a different machine with a
> different architecture. Big-endian or little-endian, 32-bit or 64-bit
> does not matter. All machines use the same file format. Furthermore, the
> developers have pledged to keep the file format stable and backwards
> compatible, so newer versions of SQLite can read and write older
> database files.
>
> Most other SQL database engines require you to dump and restore the
> database when moving from one platform to another and often when
> upgrading to a newer version of the software.
>
>
>
>
> -Original Message-
> From: sqlite-users-boun...@sqlite.org
> [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Edward J. Yoon
> Sent: Tuesday, January 13, 2009 2:45 AM
> To: General Discussion of SQLite Database
> Subject: [sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7
>
> Hi,
>
> I consider upgrade sqlite 3.3.4 to sqlite 3.6.7. So, I wonder there is
> any change (or problem) of file format.
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardy...@apache.org
> http://blog.udanax.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Best Regards, Edward J. Yoon @ NHN, corp.
edwardy...@apache.org
http://blog.udanax.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7

[Griggs, Donald]

Subject: [sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7

Hi,

I consider upgrade sqlite 3.3.4 to sqlite 3.6.7. So, I wonder there is
any change (or problem) of file format.
===

Upgrading from version 2 to version 3 (understandably) required a dump
and restore, but not upgrading from one version 3 to a newer one.

Following is from page: http://sqlite.org/different.html

Stable Cross-Platform Database File

The SQLite file format is cross-platform. A database file written on one
machine can be copied to and used on a different machine with a
different architecture. Big-endian or little-endian, 32-bit or 64-bit
does not matter. All machines use the same file format. Furthermore, the
developers have pledged to keep the file format stable and backwards
compatible, so newer versions of SQLite can read and write older
database files. 

Most other SQL database engines require you to dump and restore the
database when moving from one platform to another and often when
upgrading to a newer version of the software. 


 

-Original Message-
From: sqlite-users-boun...@sqlite.org
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Edward J. Yoon
Sent: Tuesday, January 13, 2009 2:45 AM
To: General Discussion of SQLite Database
Subject: [sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7

Hi,

I consider upgrade sqlite 3.3.4 to sqlite 3.6.7. So, I wonder there is
any change (or problem) of file format.

--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardy...@apache.org
http://blog.udanax.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Upgrade sqlite 3.3.4 to sqlite 3.6.7

[Edward J. Yoon]

Hi,

I consider upgrade sqlite 3.3.4 to sqlite 3.6.7. So, I wonder there is
any change (or problem) of file format.

-- 
Best Regards, Edward J. Yoon @ NHN, corp.
edwardy...@apache.org
http://blog.udanax.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.4.2 to 3.5.9 breaks Tcl code

[Jeff Godfrey]

D. Richard Hipp wrote:
> On Jun 13, 2008, at 12:22 AM, Jeff Godfrey wrote:
>   
>> Thanks for the quick solution.  I've adjusted my code accordingly.   
>> So,
>> do you consider this a buglet or more just the closing of a gap that
>> shouldn't have existed in the first place?
> The column naming algorithm for results  
> without an AS clause has changed in the past and will probably change  
> again in the future.  So make no assumptions.  Always use an AS clause.
>   
Noted for future reference, and code (re)adjusted...  ;^)

Thanks,

Jeff
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.4.2 to 3.5.9 breaks Tcl code

[D. Richard Hipp]

On Jun 13, 2008, at 12:22 AM, Jeff Godfrey wrote:
>>
> Thanks for the quick solution.  I've adjusted my code accordingly.   
> So,
> do you consider this a buglet or more just the closing of a gap that
> shouldn't have existed in the first place?  It's interesting that the
> original code works outside of a proc in 3.5.9, but not inside...
>


The TCL interface assigns values to variables where the name of the  
variable is the "column name" in the result set.  But SQLite makes no  
promises about what a column name is going to be unless you use an  
"AS" clause on each column.  The column naming algorithm for results  
without an AS clause has changed in the past and will probably change  
again in the future.  So make no assumptions.  Always use an AS clause.

D. Richard Hipp
[EMAIL PROTECTED]



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.4.2 to 3.5.9 breaks Tcl code

[Jeff Godfrey]

D. Richard Hipp wrote:
> On Jun 12, 2008, at 11:08 PM, Jeff Godfrey wrote:
>   
>> dbMem eval {select distinct(owner) from lockinfo order by owner} {
>>   
>> }
>>
>> That works correctly in my original app using 3.4.2, but complains  
>> that
>> there is no such variable "owner" when using 3.5.9.
>> 
>
> Solution 1:
>
>dbMem eval {SELECT DISTINCT owner FROM lockinfo ORDER BY owner} {
>   puts $owner
>}
>
> Solution 2:
>
>dbMem eval {SELECT DISTINCT(owner) AS owner FROM lockinfo ORDER BY  
> owner} {
>puts $owner
>}
>
> Solution 3:
>
>dbMem eval {SELECT DISTINCT(owner) FROM lockinfo ORDER BY owner} {
>   puts ${(owner)}
>}
>   
Thanks for the quick solution.  I've adjusted my code accordingly.  So, 
do you consider this a buglet or more just the closing of a gap that 
shouldn't have existed in the first place?  It's interesting that the 
original code works outside of a proc in 3.5.9, but not inside...

Again - thanks.

Jeff
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Upgrade from 3.4.2 to 3.5.9 breaks Tcl code

[D. Richard Hipp]

On Jun 12, 2008, at 11:08 PM, Jeff Godfrey wrote:
>
> dbMem eval {select distinct(owner) from lockinfo order by owner} {
>   
> }
>
> That works correctly in my original app using 3.4.2, but complains  
> that
> there is no such variable "owner" when using 3.5.9.

Solution 1:

   dbMem eval {SELECT DISTINCT owner FROM lockinfo ORDER BY owner} {
  puts $owner
   }

Solution 2:

   dbMem eval {SELECT DISTINCT(owner) AS owner FROM lockinfo ORDER BY  
owner} {
   puts $owner
   }

Solution 3:

   dbMem eval {SELECT DISTINCT(owner) FROM lockinfo ORDER BY owner} {
  puts ${(owner)}
   }



D. Richard Hipp
[EMAIL PROTECTED]



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Upgrade from 3.4.2 to 3.5.9 breaks Tcl code

[Jeff Godfrey]

Hm... I sent this message last night as a non-list member.  I 
received a notification that it would need to be approved by a moderator 
before being made public.  About 15 hours later, I received a "Post was 
rejected... No reason given" message.  Why's that?  So now I'm 
subscribed again.  Anyway...

==

Hi All,

I am upgrading a Tcl/SQLite based application from SQLite 3.4.2 to 
SQLite 3.5.9.  I'll mention that I'm also upgrading from Tcl 8.4.19 to 
Tcl 8.5.2 in case it's relevant, though both versions of Tcl behave the 
same when using SQLite 3.5.9.

Anyway, I have a number of procedures that operate like the following:

dbMem eval {select distinct(owner) from lockinfo order by owner} {
   
}

That works correctly in my original app using 3.4.2, but complains that 
there is no such variable "owner" when using 3.5.9.  Interestingly, I've 
found that the above code works under 3.5.2 when not inside a Tcl 
procedure.  Inside the procedure though, it breaks as described.  Also, 
a few other variations...

Inside a procedure, using 3.5.2, replacing "distinct(owner)" with just 
"owner" works correctly and replacing "distinct(owner) with just "*" 
works correctly, though neither obviously does what I need.

Is this an intended change to SQLite Tcl bindings, a bug, or just 
something else I'm missing?

Thanks for any insight.

Jeff Godfrey



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] upgrade sqlite

[Jason Berkhimer]

I have just instaleld apache 2.2 and php 5.2.5 on a windows xp machine. I would 
also like the current version of sqlite. running sqlite3.exe gives me a verion 
of 3.3.15. Is this the version of sqlite3.exe or of the dqlite database engine? 
If this is the engine version how do I upgrade it? I do not see a sqlite.dll 
file anywhere. 

I have also googled this topic and have come up empty.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] upgrade from sqlite 2.8.11 to sqlite3?

[Julian Qian]

Hi everyone.
  I am thinking about upgrading sqlite version, and I found lots of things
changed in sqlite3, for example, sqliteBtreeUpdateMeta(Btree *pBt, int
*aMeta) changed to sqlite3BtreeUpdateMeta(Btree *p, int idx, u32 iMeta),
What should I put in idx and iMeta? and old code require call
BeginTransaction() before it, will new code need it too?

Anyone did such kind of upgrade before? I could not find information about
upgrade in sqlite webpages or wiki.
Please send me info if you hack it before, I will gather all the info and
update the wiki.

Julian

[sqlite] upgrade from 2 to 3

[Cronos]

I've just upgraded my program from using v2.8.16 of sqlite to using v3.2.1
of sqlite. So I've change the wrapper class that I was using to use the new
prepare() instead of compile() etc. The problem I have is that my program is
now running slower - a lot slower, as in one fiftieth of its previous speed.
Is there something obvious that I have overlooked in the changes ? I've
added indexes to tables and synchronous is now off but its made no
improvement.

Re: [sqlite] upgrade?

[Ulrik Petersen]

Hi Michael,
Michael Hunley wrote:
Hi,
I am currently using SQLite v 2.8.13 for a commercial product for Palm 
Handhelds.  I am noticing some performance issues; most notably when I 
do a database validate() to verify my integrity at app open and 
close.  I see that version 3.0.8 has some code optimizations, but am 
unsure if upgrading from the 2.8 series to the 3.0 series is just a 
drop in or if my code will need to change.  Can anyone tell me if the 
API has changed in a notable way?  
The API has changed slightly.  Mostly it's a matter of changing the 
sqlite_ prefix to sqlite3_, but you also need to change the name of the 
structure which you pass to sqlite3_step, and a few other changes.  I 
can send you two pieces of code that show "before" and "after" use if 
you contact me off-list.


Also, is my performance going to improve with the 3.0.8 over 2.8.13?  
If not, what about 2.8.15?
I experienced a performance increase when I moved from 2.8.13 to 2.8.15, 
but a performance decrease when moving from 2.8.13 to 3.0.7.  A recent 
thread on this list deals with this perceived decrease in performance 
(the thread is from around 5 October 2004, entitled "Degradation of 
performance in SQLite3?" -- you can find it in the archives, which are 
linked to from www.sqlite.org).  PLEASE note that some people seem to 
experience performance *increases* when going from SQLite 2.8 to 3.0, so 
Your Mileage May Vary.

Basically, Dr. Hipp said back then that SQLite 3 uses less disk space 
(and so fewer disk reads) at the expense of using more CPU cycles.  This 
may or may not give you a performance increase, and in my case, it gave 
me a performance decrease, probably because the databases I have are so 
small that the operating system (Linux in my case) can map most of the 
file into virtual memory both for 2.8 and 3.0, and so the usage of more 
CPU cycles in 3.0 gives an overall performance decrease.

However, there are several other good reasons to move to SQLite 3 than 
performance issues, and I'm sure that others can fill in the details.

HTH.
Cheers,
Ulrik

[sqlite] upgrade?

[Michael Hunley]

Hi,
I am currently using SQLite v 2.8.13 for a commercial product for Palm 
Handhelds.  I am noticing some performance issues; most notably when I do a 
database validate() to verify my integrity at app open and close.  I see 
that version 3.0.8 has some code optimizations, but am unsure if upgrading 
from the 2.8 series to the 3.0 series is just a drop in or if my code will 
need to change.  Can anyone tell me if the API has changed in a notable 
way?  Also, is my performance going to improve with the 3.0.8 over 
2.8.13?  If not, what about 2.8.15?

thanks!
Michael Hunley
Managing Partner/Senior Engineer
PocketPurchase Consultants