On Mon, 25 Aug 2008 12:26:18 -0700, you wrote:
>Interesting, I just tried that in my test application and Dennis's and I
>get access violations during the vacuum command execution when trying to
>resize the pages from 1k to 4k with my database or Dennis's test
>database.
I just used the command l
Brown, Daniel wrote:
> Interesting, I just tried that in my test application and Dennis's and I
> get access violations during the vacuum command execution when trying to
> resize the pages from 1k to 4k with my database or Dennis's test
> database.
>
Daniel,
I have found that sqlite works corre
PROTECTED] On Behalf Of Kees Nuyt
Sent: Saturday, August 23, 2008 8:51 AM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
On Thu, 21 Aug 2008 16:29:10 -0700, you wrote:
>How do I rebuild a database file for another page size
>or did th
On Thu, 21 Aug 2008 16:29:10 -0700, you wrote:
>How do I rebuild a database file for another page size
>or did the pragma do that already?
Use PRAGMA page_size={your_new_pagesize} immediately before
a vacuum. It will change the page size of the vacuumed
database. See:
http://www.sqlite.org/pragm
Becker
> Sent: Friday, August 22, 2008 1:31 PM
> To: General Discussion of SQLite Database
> Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
>
> At one point I wrote some vbscript to generate a table declaration and
> insert statements for a csv. I might be able to dig it up
That script would be great :)
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Jeffrey Becker
Sent: Friday, August 22, 2008 1:31 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
At one point I wrote
sqlite?
>
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Dennis Cote
> Sent: Friday, August 22, 2008 7:10 AM
> To: General Discussion of SQLite Database
> Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
>
> Brown, Da
sqlGuruOMeter where name="Ron Wilson";
2
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brown, Daniel
Sent: Friday, August 22, 2008 1:59 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
Brown, Daniel wrote:
> I just ran Dennis's test databases through the test application and
> we're getting similar results:
> 1k Pages (17.4 MB) used 18102 KB High 20416 KB
> 4k Pages (12.2 MB) used 18102 KB, High 26416 KB (not sure why
> the high is higher?)
> My test database however
Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Dennis Cote
Sent: Friday, August 22, 2008 7:10 AM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
Brown, Daniel wrote:
> 2. And the other thing to try would be
Brown, Daniel wrote:
> Ok so after reading your feedback I tried:
> 1. "PRAGMA cache_size =10" no change in memory usage.
> 2. "PRAGMA page_size = 4096" no change in memory usage.
>
> I'm doing both those queries (in C++) after the 'sqlite3_open(
> ":memory:", &m_pDataBase );' in my test but befor
Brown, Daniel wrote:
> 2. And the other thing to try would be if anyone has a fairly meaty test
> database they don't mind sharing that I could fling at my test
> application to try and rule out the data?
>
Daniel,
I can send you copies of the databases I am using for my testing, both
the versi
qlite] Reducing SQLite Memory footprint(!)
On Aug 21, 2008, at 7:33 PM, Brown, Daniel wrote:
> if anyone has a fairly meaty test
> database they don't mind sharing that I could fling at my test
> application...
A project repository with "fossil" is a (meaty) SQLite database.
On Aug 21, 2008, at 7:33 PM, Brown, Daniel wrote:
> if anyone has a fairly meaty test
> database they don't mind sharing that I could fling at my test
> application...
A project repository with "fossil" is a (meaty) SQLite database. You
could (for example) go clone one of the various fossil
ucing SQLite Memory footprint(!)
Nicolas Williams wrote:
>
> I thought the DB was 9MB; forgive me for wasting your time then. If
> it's 17.4MB then the memory usage seems a lot more reasonable.
Daniel, the OP's, database is 9 MB. I don't have his database file, but
I do
gust 21, 2008 3:13 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
On Thu, Aug 21, 2008 at 04:02:32PM -0600, Dennis Cote wrote:
> I built a copy of my test database using a 4096 byte page size and it
> reduced the database file size fro
Nicolas Williams wrote:
>
> I thought the DB was 9MB; forgive me for wasting your time then. If
> it's 17.4MB then the memory usage seems a lot more reasonable.
Daniel, the OP's, database is 9 MB. I don't have his database file, but
I do have his test code. I used a database of my own that is a
On Thu, Aug 21, 2008 at 04:02:32PM -0600, Dennis Cote wrote:
> I built a copy of my test database using a 4096 byte page size and it
> reduced the database file size from 17.5 MB to 12.2 MB. When I repeat
> the tests using this database file I get the same 18102 KB of memory
> used, but a slight
Nicolas Williams wrote:
>
> I wonder too, what does the page cache do when doing full table scans?
> If the cache has an LRU/LFU page eviction algorithm then full table
> scans should not be a big deal. Ideally it should not allow pages read
> during a full table scan to push out other pages, but
Jeffrey Becker wrote:
> Just out of curiosity what happens if you call
> "PRAGMA page_size=4096"
> before running the import?
>
As I expected, it has no effect. The page size pragma only effects the
:memory: database he is copying into. The page size of the database file
was set when it was cre
On Thu, Aug 21, 2008 at 10:32:23AM -0400, Jeffrey Becker wrote:
> Just out of curiosity what happens if you call
> "PRAGMA page_size=4096"
> before running the import?
I wonder too, what does the page cache do when doing full table scans?
If the cache has an LRU/LFU page eviction algorithm then fu
Just out of curiosity what happens if you call
"PRAGMA page_size=4096"
before running the import?
On Thu, Aug 21, 2008 at 9:52 AM, Dennis Cote <[EMAIL PROTECTED]> wrote:
> Brown, Daniel wrote:
>>
>> I just tried with rebuilt data from SQLite Analyzer in SQLite 3.6.1, I'm
>> still seeing a memory u
Brown, Daniel wrote:
>
> I just tried with rebuilt data from SQLite Analyzer in SQLite 3.6.1, I'm
> still seeing a memory usage that is roughly three times the size of the
> source database file, looking at your changes to my test there doesn't
> seem to be any fixes that would resolve that.
>
N
Brown, Daniel wrote:
Perhaps, when is the next release due? I'd be interested to see the
differences, if an upgrade reduces memory overhead that significantly it
would be most excellent :)
3.6.2 is (tentatively) due this Monday :)
___
sqlite-users mai
5 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
Brown, Daniel wrote:
> I just upgraded to the latest version (3.6.1) of the pre-processed C
> source code from the website, running the test again gives me similar
> results of 22.2 MB use
Brown, Daniel wrote:
> I just upgraded to the latest version (3.6.1) of the pre-processed C
> source code from the website, running the test again gives me similar
> results of 22.2 MB used and 24.55 MB high water from the same 9 MB file.
> Is there any way it could be the file that is causing the
esday, August 20, 2008 2:43 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
On Aug 20, 2008, at 5:30 PM, Brown, Daniel wrote:
> I just upgraded to the latest version (3.6.1) of the pre-processed C
> source code from the website, running the
On Aug 20, 2008, at 5:30 PM, Brown, Daniel wrote:
> I just upgraded to the latest version (3.6.1) of the pre-processed C
> source code from the website, running the test again gives me similar
> results of 22.2 MB used and 24.55 MB high water from the same 9 MB
> file.
> Is there any way it cou
27;ll try rebuilding it next.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brown, Daniel
Sent: Wednesday, August 20, 2008 1:50 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
That is interesting, all
m the site.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of D. Richard Hipp
Sent: Wednesday, August 20, 2008 12:14 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
On Aug 20, 2008, at 2:56 PM, Brown, Da
On Aug 20, 2008, at 2:56 PM, Brown, Daniel wrote:
> Thank you, I imported our data from the source Excel file (.xls) using
> the third party SQLite Analyzer application
> (http://www.kraslabs.com/sqlite_analyzer.php) if that makes any
> difference?
>
> The size of the SQLite database on disc is 9
. Richard Hipp
Sent: Wednesday, August 20, 2008 11:44 AM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
On Aug 20, 2008, at 2:22 PM, Brown, Daniel wrote:
> sqlite3_memory_highwater() ~ 25673060
> sqlite3_memory_used() ~ 23222709
>
On Aug 20, 2008, at 2:22 PM, Brown, Daniel wrote:
> sqlite3_memory_highwater() ~ 25673060
> sqlite3_memory_used() ~ 23222709
>
OK. I'll have a look
D. Richard Hipp
[EMAIL PROTECTED]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http:/
More over, the memory usage reported by process explorer is subject to
the vagrancies of the windows memory allocator. Generally the memory
usage you see reported in tools is quite a bit higher than the actual
memory usage of your application. First off windows manages memory in
fixed size chunks
ed();
printf("%s %d KB High %d KB", pfilename, (memUsed/1024),
(memHigh/1024));
sqlite3_close( pDataBase );
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of D. Richard Hipp
Sent: Wednesday, August 20, 2008 11:00 AM
To: General Discussion
On Aug 20, 2008, at 1:53 PM, Brown, Daniel wrote:
> Looking in process explorer on XP after the disc database detached
> should a memory size change of 28 MB of RAM in the test application, I
> assumed this was the size of the database in memory.
That would be the peak memory usage by the appli
Sent: Wednesday, August 20, 2008 10:44 AM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory footprint(!)
On Aug 19, 2008, at 6:24 PM, Brown, Daniel wrote:
> SQLite is very memory
> intensive compared to our current solution (although SQLite is faster
On Aug 19, 2008, at 6:24 PM, Brown, Daniel wrote:
> SQLite is very memory
> intensive compared to our current solution (although SQLite is faster
> and more feature rich), e.g. 9MB for our current solution versus 28 MB
> for SQLite with the same source data.
Where did you get the 28MB figure?
%
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of D. Richard Hipp
Sent: Tuesday, August 19, 2008 3:43 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Reducing SQLite Memory
On Aug 19, 2008, at 6:24 PM, Brown, Daniel wrote:
> Good afternoon list,
>
> Are there any known techniques for reducing SQLite's memory footprint?
Construct your database into a file (say "test.db"). Then run the
sqlite3_analyzer utility (available for download on the SQLite
website) over
Good afternoon list,
Are there any known techniques for reducing SQLite's memory footprint?
I am currently evaluating SQLite as a possible replacement to our
current proprietary database solution. However SQLite is very memory
intensive compared to our current solution (although SQLite is faster
41 matches
Mail list logo