Re: [sqlite] Memory usage when increasing SQLITE_MAX_VARIABLE_NUMBER to 32768

On Tuesday, 14 January, 2020 16:04, Simon Slavin wrote: >On 14 Jan 2020, at 10:56pm, Alexandre Doubov >wrote: >> 1) Does the act of bumping this limit up have an effect on memory at >all (assuming that no more than 999 arguments are passed into >statements)? >Section 9: >

Re: [sqlite] Memory usage when increasing SQLITE_MAX_VARIABLE_NUMBER to 32768

On 14 Jan 2020, at 10:56pm, Alexandre Doubov wrote: > 1) Does the act of bumping this limit up have an effect on memory at all > (assuming that no more than 999 arguments are passed into statements)? Section 9: If I read this correctly, memory is reserved only

[sqlite] Memory usage when increasing SQLITE_MAX_VARIABLE_NUMBER to 32768

Hello, I have a few questions with regards to memory impact when bumping the default SQLITE_MAX_VARIABLE_NUMBER limit from 999 to 32768. For reference, this is the issue that I created that brought me to this mailing list: https://github.com/requery/sqlite-android/issues/124 1) Does the act of b

Re: [sqlite] memory usage after close database on linux arm

On 2 Jun 2017, at 10:18am, Stephane Guibert wrote: > sqlite3_finalize(... > sqlite3_close_v2(... > }while First, check out the result returned by sqlite3_close() as Clemens wrote. If that doesn’t give a useful result code, continue. After closing the file, execute sqlite3_shutdown(). Does

Re: [sqlite] memory usage after close database on linux arm

Stephane Guibert wrote: > I get memory usage not freed at close. > > the return is SQLITE_OK at each step: > > do{ > sqlite3_open_v2(...,&db,SQLITE_OPEN_READWRITE,0); > sqlite3_exec(db,"PRAGMA journal_mode = MEMORY",0,0,0); > sqlite3_prepare_v2(db... > sqlite3_step( statement Begin //BEGIN EXCLUSI

[sqlite] memory usage after close database on linux arm

Hi SQLite creators, for embedded Linux 2.6.35.3 on armV5, I get memory usage not freed at close(ltib compilation environment). with this following test ( only one connection to the database ): 1) open database 2) prepare statement 3) SQL request inside begin commit 4) finalize statement 5) cl

Re: [sqlite] Memory Usage

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 03:12 PM, David Muchene wrote: > I was wondering what options I can tune to make sqlite use more > memory. Have you tried using memory mapping (there is a pragma). https://sqlite.org/mmap.html Roger -BEGIN PGP SIGNATURE- Ver

Re: [sqlite] Memory Usage

David Muchene wrote: > I was wondering what options I can tune to make sqlite use more memory. You did not mention PRAGMA cache_size, so I guess you are not using it? Regards, Clemens ___ sqlite-use

[sqlite] Memory Usage

Hi, I was wondering what options I can tune to make sqlite use more memory. We are currently using the memsys5 allocator and giving it a 2G buffer, but it doesn't seem to be using any more than 32MB. Thanks, Dave M ___ sqlite-users mailing list sqlite

Re: [sqlite] Memory usage of sqlite3

> Some changes, if blob is bigger than a few bytes, you should normalize them. > If 2 blobs are equal, their id must be equal and you don't waste time > comparing nor memory joining blob content. So you get: They are quite small (max ~70 bytes...) > DROP TABLE IF EXISTS tour_blob; > CREATE TAB

Re: [sqlite] Memory usage of sqlite3

On Wed, 17 Jul 2013 12:04:52 +0200 Paolo Bolzoni wrote: > On Tue, Jul 16, 2013 at 8:13 PM, Eduardo wrote: > > > Can you show us the query and/or schemas? If not: > Sure, I appended everything in the bottom of this email. > Unfortunately gmail will mess-up the layout, I hope it will be > readabl

Re: [sqlite] Memory usage of sqlite3

On Tue, Jul 16, 2013 at 8:13 PM, Eduardo wrote: > Can you show us the query and/or schemas? If not: Sure, I appended everything in the bottom of this email. Unfortunately gmail will mess-up the layout, I hope it will be readable. (See here, it seems google does not know the mean of "should") htt

Re: [sqlite] Memory usage of sqlite3

On Tue, 16 Jul 2013 18:17:41 +0200 Paolo Bolzoni wrote: > I tried the experiment again with -g3 -O0, I got less > information than expected (there are still many unknown > symbols in libsqlite3.so), but the function requiring all > this memory is sqlite3_step. Can you show us the query and/or sc

Re: [sqlite] Memory usage of sqlite3

On 16 Jul 2013, at 5:17pm, Paolo Bolzoni wrote: > the function requiring all > this memory is sqlite3_step. > > So maybe it is one complex query? Possibly a query for which no good index exists, so SQLite decides to make up its own temporary index. If you consider this amount of memory usage

Re: [sqlite] Memory usage of sqlite3

I tried the experiment again with -g3 -O0, I got less information than expected (there are still many unknown symbols in libsqlite3.so), but the function requiring all this memory is sqlite3_step. So maybe it is one complex query? I would like to avoid excessive swapping on the production server,

Re: [sqlite] Memory usage of sqlite3

On Tue, Jul 16, 2013 at 1:00 PM, Dan Kennedy wrote: > On 07/16/2013 01:49 AM, Paolo Bolzoni wrote: > A very large blob or string result? I would exclude this, I do use blobs... but they are at most few dozen of bytes... > Code allocates (or leaks) > tremendous numbers of sqlite3_stmt* handles? T

Re: [sqlite] Memory usage of sqlite3

On 07/16/2013 01:49 AM, Paolo Bolzoni wrote: From 35-40MB to 940MB; I would put massif result but I think the list deletes attachments. A very large blob or string result? Code allocates (or leaks) tremendous numbers of sqlite3_stmt* handles? SQLite has various APIs for querying memory usage:

Re: [sqlite] Memory usage of sqlite3

The test ended sometime during the night and setting temp_store to 0 the result is exactly the same. I suspect it was the default anyway. On Mon, Jul 15, 2013 at 9:20 PM, Paolo Bolzoni wrote: > On Mon, Jul 15, 2013 at 9:08 PM, Eduardo Morras wrote: >> On Mon, 15 Jul 2013 20:49:52 +0200 >> Paol

Re: [sqlite] Memory usage of sqlite3

On Mon, Jul 15, 2013 at 9:08 PM, Eduardo Morras wrote: > On Mon, 15 Jul 2013 20:49:52 +0200 > Paolo Bolzoni wrote: > >> From 35-40MB to 940MB; I would put massif result but I think the >> list deletes attachments. > > What does PRAGMA temp_store show? Set it to 0 and recheck. Did you compile > w

Re: [sqlite] Memory usage of sqlite3

On Mon, 15 Jul 2013 20:49:52 +0200 Paolo Bolzoni wrote: > From 35-40MB to 940MB; I would put massif result but I think the > list deletes attachments. What does PRAGMA temp_store show? Set it to 0 and recheck. Did you compile with SQLITE_TEMP_STORE set to 3? --- --- Eduardo Morras

Re: [sqlite] Memory usage of sqlite3

On Mon, Jul 15, 2013 at 8:59 PM, Simon Slavin wrote: > > On 15 Jul 2013, at 7:49pm, Paolo Bolzoni > wrote: > >> From 35-40MB to 940MB; I would put massif result but I think the >> list deletes attachments. > > Do you have in-memory tables ? No. > Do you use sqlite3_exec() ? Twice, to activate t

Re: [sqlite] Memory usage of sqlite3

On Mon, Jul 15, 2013 at 08:49:52PM +0200, Paolo Bolzoni scratched on the wall: > >From 35-40MB to 940MB; I would put massif result but I think the > list deletes attachments. By default, the page-cache is 2000. Pages are typically 1KB, but have some minor overhead in the cache. Assuming you

Re: [sqlite] Memory usage of sqlite3

On 15 Jul 2013, at 7:49pm, Paolo Bolzoni wrote: > From 35-40MB to 940MB; I would put massif result but I think the > list deletes attachments. Do you have in-memory tables ? Do you use sqlite3_exec() ? Do you have SELECTs for which there is no good index, forcing sqlite3 to make up its own ?

Re: [sqlite] Memory usage of sqlite3

>From 35-40MB to 940MB; I would put massif result but I think the list deletes attachments. On Mon, Jul 15, 2013 at 8:41 PM, Stephan Beal wrote: > On Mon, Jul 15, 2013 at 8:39 PM, Paolo Bolzoni < > paolo.bolzoni.br...@gmail.com> wrote: > >> So, sorry if the question sounds very vague. But what ca

Re: [sqlite] Memory usage of sqlite3

On Mon, Jul 15, 2013 at 8:39 PM, Paolo Bolzoni < paolo.bolzoni.br...@gmail.com> wrote: > So, sorry if the question sounds very vague. But what can > cause high memory usage in sqlite? A large transaction > maybe? > What is "high"? In my apps sqlite tends to use 200-400kb or so, which i don't con

[sqlite] Memory usage of sqlite3

I wrote an C++ application that uses sqlite3 to save part of the data when it become larger than a known threshold. The idea is to use at most a known quantity of memory; to check if it is working I executed a relevant test using valgrind's massif. It worked fairly well most of the time, but in t

[sqlite] Track SQLite memory usage from inside Java code

SQL's memory usage. I found a solution for C++ but I use Java. Any help would be greatly appreciated! Mira -- View this message in context: http://sqlite.1065341.n5.nabble.com/Track-SQLite-memory-usage-from-inside-Java-code-tp62756.html Sent from the SQLite mailing list archive at Nabbl

Re: [sqlite] Memory Usage/ Drawbacks of Statements

Hi Eric, On 24/4/2012 10:03 PM, Eric Minbiole wrote: Similar to Pavel's suggestion, our implementation maintains a simple cache of prepared statements, keyed by the SQL query that created them. For example: pStatement = Cache.GetQuery("SELECT * FROM xyz"); would return the cached statement

Re: [sqlite] Memory Usage/ Drawbacks of Statements

Similar to Pavel's suggestion, our implementation maintains a simple cache of prepared statements, keyed by the SQL query that created them. For example: pStatement = Cache.GetQuery("SELECT * FROM xyz"); would return the cached statement if the query had been seen before, or would auto-create

Re: [sqlite] Memory Usage/ Drawbacks of Statements

Thanks Pavel, That gives me something new to do with SQLite over the next few weeks. On 23/4/2012 8:47 PM, Pavel Ivanov wrote: 1. Do statements do any thing that would require a lot of memory to be maintained? No, they don't need a lot of memory, but still some memory is used. So if you have l

Re: [sqlite] Memory Usage/ Drawbacks of Statements

> 1. Do statements do any thing that would require a lot of memory to be > maintained? No, they don't need a lot of memory, but still some memory is used. So if you have like thousands of statements you should worry about this. If you have 20 or 30 statements your database cache will likely consum

[sqlite] Memory Usage/ Drawbacks of Statements

Hi, our system does fairly predictable queries when it runs. A number of modules all access data using a handful of queries of each. We open the database at the start and close it at the end of the program. Each query follows the usual pattern of prepare - bind - step - reset - (eventually)

Re: [sqlite] memory usage after VACUUM

On 9 Mar 2011, at 15:23, Nick Hodapp wrote: > I'm using sqlite in an iOS app, via the popular FMDB wrapper. > > My profiling tool is showing me that the app is using 2.5 MB of memory > before a VACUUM, and nearly 6MB after. The tool shows that the extra memory > was allocated by sqlite3MemMallo

Re: [sqlite] memory usage after VACUUM

> Is there any sqlite function I can call, or some other technique, to reduce > the memory allocated and hung-onto by sqlite, particularly during a VACUUM? Yes, execute "pragma cache_size = 100" for example, or put other number of your liking into there. If closing and re-opening of the database

Re: [sqlite] memory usage after VACUUM

On 09.03.2011, at 16:23, Nick Hodapp wrote: > I'm using sqlite in an iOS app, via the popular FMDB wrapper. > > My profiling tool is showing me that the app is using 2.5 MB of memory > before a VACUUM, and nearly 6MB after. The tool shows that the extra memory > was allocated by sqlite3MemMallo

[sqlite] memory usage after VACUUM

I'm using sqlite in an iOS app, via the popular FMDB wrapper. My profiling tool is showing me that the app is using 2.5 MB of memory before a VACUUM, and nearly 6MB after. The tool shows that the extra memory was allocated by sqlite3MemMalloc(). If I close and re-open the database then the extra

Re: [sqlite] SQLite Memory Usage

Information Systems From: sqlite-users-boun...@sqlite.org on behalf of Sachin.2.Gupta Sent: Mon 11/8/2010 4:47 AM To: sqlite-users@sqlite.org Subject: EXTERNAL:[sqlite] SQLite Memory Usage Hi, We are trying to Integrate SQLite in our Application and are trying to populate as a Ca

[sqlite] SQLite Memory Usage

Hi, We are trying to Integrate SQLite in our Application and are trying to populate as a Cache. We are planning to use it as a In Memory Database. Using it for the first time. Our Application is C++ based. Our Application interacts with the Master Database to fetch data and performs numerous o

Re: [sqlite] Memory usage ??? one data base versus two smaller ones

On Fri, Feb 19, 2010 at 09:39:08AM -0300, Israel Lins Albuquerque scratched on the wall: > Samuel, > > Each one attached database has its own page cache with 2000 > (default number of pages in cache) * 1024 (default size in > bytes of a page), On many Windows systems it will default to 4096.

Re: [sqlite] Memory usage – one data base versus tw o smaller ones

Quarta-feira, 17 de Fevereiro de 2010 16:38:27 (GMT-0300) Auto-Detected Assunto: [sqlite] Memory usage – one data base versus two smaller ones For some reasons it is more convenient for the project to have a few smaller databases with unrelated data than one containing everything. My only c

[sqlite] Memory usage – one data base versus tw o smaller ones

For some reasons it is more convenient for the project to have a few smaller databases with unrelated data than one containing everything. My only concern is RAM memory. How much burden/memory overhead an additional database would introduce? Thank you for your input, Samuel __

Re: [sqlite] memory usage and queries

I fixed this by reducing the cache size from 2000 to 200. I may want to adjust some more but will continue to test. Chris ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] memory usage and queries

I am running into issues where I am running out of memory on my embedded app. I have stored 10K records in a table. There is an index on a key field and a sort field. An external program needs to reconcile its data with the data in this table. It does so 200 records at a time. The code I am us

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 7:12 PM, Marco Bambini wrote: > Hello Dr. Hipp, > > I was finally able to track down the issue ... the problem is due to > the fact that after each write operation the client executes a query > like: > "SELECT 123 AS changes;" (the number 123 changes all the time) > this

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 10:54 AM, Marco Bambini wrote: > and the read lock is released when the virtual machine is finalized > right? ... even if the writer is inside a BEGIN IMMEDIATE transaction? > Correct. D. Richard Hipp d...@hwaci.com ___ sqlite

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 9:25 AM, Marco Bambini wrote: > cache_size is set to default 2000, page size is 1K... > > here you go the output of sqlite3_status: > 2009-04-21 15:24:25 SQLITE_STATUS_MEMORY_USED current: 106704136 high: > 109873952 > 2009-04-21 15:24:25 SQLITE_STATUS_PAGECACHE_USED curre

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 9:25 AM, Marco Bambini wrote: > cache_size is set to default 2000, page size is 1K... > > here you go the output of sqlite3_status: > 2009-04-21 15:24:25 SQLITE_STATUS_MEMORY_USED current: 106704136 high: > 109873952 > 2009-04-21 15:24:25 SQLITE_STATUS_PAGECACHE_USED curre

Re: [sqlite] Memory Usage

cache_size is set to default 2000, page size is 1K... here you go the output of sqlite3_status: 2009-04-21 15:24:25 SQLITE_STATUS_MEMORY_USED current: 106704136 high: 109873952 2009-04-21 15:24:25 SQLITE_STATUS_PAGECACHE_USED current: 0 high: 0 2009-04-21 15:24:25 SQLITE_STATUS_PAGEC

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 9:07 AM, Marco Bambini wrote: > The database is on-disk ... does huge not committed transactions uses > memory? > It should do writes to disk periodically to free up memory, once you hit your cache_size limit. What is cache_size set to. What are the output from sqlite3_st

Re: [sqlite] Memory Usage

The database is on-disk ... does huge not committed transactions uses memory? -- Marco Bambini http://www.sqlabs.com http://www.creolabs.com/payshield/ On Apr 21, 2009, at 2:27 PM, D. Richard Hipp wrote: > > On Apr 21, 2009, at 8:22 AM, Marco Bambini wrote: > >> Yes, executing sqlite3_mem

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 8:22 AM, Marco Bambini wrote: > Yes, executing sqlite3_memory_used () after 183,000 INSERT statement > returns: 106,766,848. > Database is never closed during application lifetime. > Each statement is prepared, stepped and properly finalized. > We do that kind of test all the

Re: [sqlite] Memory Usage

http://www.creolabs.com/payshield/ On Apr 21, 2009, at 2:11 PM, D. Richard Hipp wrote: > > On Apr 21, 2009, at 8:08 AM, Marco Bambini wrote: > >> Hello guys, >> >> I am trying to write you again about a simple question... how can I >> limit sqlite memory usage

Re: [sqlite] Memory Usage

On Apr 21, 2009, at 8:08 AM, Marco Bambini wrote: > Hello guys, > > I am trying to write you again about a simple question... how can I > limit sqlite memory usage during insert commands? It seems that the > amount of memory usage increases when the number of objects inserted >

[sqlite] Memory Usage

Hello guys, I am trying to write you again about a simple question... how can I limit sqlite memory usage during insert commands? It seems that the amount of memory usage increases when the number of objects inserted into the database is increased and memory is never freed. I tried to set

[sqlite] Memory usage

Hello guys, I need your help in order to solve a very annoying issue with sqlite 3.6.11. I have two opened db inside my application and when I insert 180,000 rows inside a transaction I can see heap memory usage that exceeds 100MB (data is written twice so I have 2 transactions inside two

[sqlite] memory usage

Hello, My multi-threaded application has various sqlite db's open simultaneously, in memory using the :memory: keyword, disk based db's and at times, tmpfs (ram) db's. Is there a way to view each individual database's memory usage? I found the functions sqlite3_memory_used() and sqlite3_status(SQL

[sqlite] memory usage

Hello, My multi-threaded application has various sqlite db's open simultaneously, in memory using the :memory: keyword, disk based db's and at times, tmpfs (ram) db's. Is there a way to view each individual database's memory usage? I found the functions sqlite3_memory_used() and sqlite3_status(SQL

[sqlite] Fwd: sqlite memory usage

Hello, My multi-threaded application has various sqlite db's open simultaneously, in memory using the :memory: keyword, disk based db's and at times, tmpfs (ram) db's. Is there a way to view each individual database's memory usage? I found the functions sqlite3_memory_used() and sqlite3_status(SQL

[sqlite] memory usage

Hello,My application has several sqlite db's open simultaneously, in memory using the :memory: keyword, disk based db's and at times, tmpfs based db's. Is there a way to view each individual database's memory usage? I found the functions sqlite3_memory_used() and sqlite3_status(SQLITE_STATUS_MEMOR

[sqlite] sqlite memory usage

Hello,My first mail attempt appears not to have gotten through yesterday due to the spam storm. My multi-threaded application has various sqlite db's open simultaneously, in memory using the :memory: keyword, disk based db's and at times, tmpfs based db's. Is there a way to view each individual d

Re: [sqlite] Memory Usage

Once again you're missing the point. Of course you can get a malloc/free implementation that performs garbage collection, such as Boehm's conservative GC. But C garbage collection and malloc/free memory fragmentation are quite different things. You can still get heavily fragmented memory with a

Re: [sqlite] Memory Usage

On 11/19/07, John Stanton <[EMAIL PROTECTED]> wrote: > Joe Wilson wrote: > > If a C program employs perfect 1:1 malloc'ing to free'ing, i.e., has no > > memory leaks, then garbage collection is irrelevant to the topic of > > memory fragmentation. It's not like C can employ a copying garbage > > co

Re: [sqlite] Memory Usage

You confused my point which is that your usual malloc/free definitely does no garbage collection. That does not mean that a C language program cannot perform garbage collection, just look at a Java run time package for an example. If you never execute a free your dynamic memory is essentially

Re: [sqlite] Memory Usage

On 11/19/07, John Stanton <[EMAIL PROTECTED]> wrote: > Malloc is a concept implemented in various ways, some more successful > than others but all of them hidden from the programmer. Free tries to > give back memory but as you can appreciate unless you use some garbage > collection scheme with ba

Re: [sqlite] Memory Usage

Well said. While it may be true that some memory allocators are lacking, the ones I use are quite good. I view with great suspicion developers who thinks they can outsmart the pool allocator. These folks usually add great complexity while having at best a neutral impact on performance and r

Re: [sqlite] Memory Usage

--- John Stanton <[EMAIL PROTECTED]> wrote: > Malloc is a concept implemented in various ways, some more successful > than others but all of them hidden from the programmer. Free tries to > give back memory but as you can appreciate unless you use some garbage > collection scheme with backwards

Re: [sqlite] Memory Usage

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote: > Our studies to date indicate that SQLite neither leaks nor fragments > memory. Preventing leaks is relatively easy. Preventing memory > fragmentation less so. Yet we are not seeing memory fragmentation > as a problem for the workloads we have te

Re: [sqlite] Memory Usage

James Dennett wrote: -Original Message- From: John Stanton [mailto:[EMAIL PROTECTED] Sent: Monday, November 19, 2007 12:14 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Memory Usage Dynamic allocation is not the problem, it is malloc and free. there is a difference between

Re: [sqlite] Memory Usage

D. Richard Hipp wrote: On Nov 19, 2007, at 12:36 PM, James Dennett wrote: -Original Message- From: John Stanton [mailto:[EMAIL PROTECTED] Sent: Monday, November 19, 2007 7:36 AM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Memory Usage Not only applicable to real time systems

Re: [sqlite] Memory Usage

2007 7:36 AM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Memory Usage Not only applicable to real time systems. If you want a program to run with stability over a long time the first step it to eliminate frees and if malloc is used confine it to the intialization. I have to challenge

RE: [sqlite] Memory Usage

> -Original Message- > From: John Stanton [mailto:[EMAIL PROTECTED] > Sent: Monday, November 19, 2007 12:14 PM > To: sqlite-users@sqlite.org > Subject: Re: [sqlite] Memory Usage > > Dynamic allocation is not the problem, it is malloc and free. there is > a

Re: [sqlite] Memory Usage

: Re: [sqlite] Memory Usage Not only applicable to real time systems. If you want a program to run with stability over a long time the first step it to eliminate frees and if malloc is used confine it to the intialization. I have to challenge this, not because it's entirely wrong (it&

Re: [sqlite] Memory Usage

On Nov 19, 2007, at 12:36 PM, James Dennett wrote: -Original Message- From: John Stanton [mailto:[EMAIL PROTECTED] Sent: Monday, November 19, 2007 7:36 AM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Memory Usage Not only applicable to real time systems. If you want a program to

RE: [sqlite] Memory Usage

> -Original Message- > From: John Stanton [mailto:[EMAIL PROTECTED] > Sent: Monday, November 19, 2007 7:36 AM > To: sqlite-users@sqlite.org > Subject: Re: [sqlite] Memory Usage > > Not only applicable to real time systems. If you want a program to run > with st

Re: [sqlite] Memory Usage

Not only applicable to real time systems. If you want a program to run with stability over a long time the first step it to eliminate frees and if malloc is used confine it to the intialization. Jim Dodgen wrote: One other note, just about all real-time systems limit the dynamic allocation of

Re: [sqlite] Memory Usage

One other note, just about all real-time systems limit the dynamic allocation of memory because you lose the deterministic behavior, typically all memory is allocated when the task starts, memory is usually managed internally in standard sized chunks. Also for long running tasks (months/years)

Re: [sqlite] Memory Usage

On Nov 18, 2007, at 8:12 AM, Russell Leighton wrote: On Nov 17, 2007, at 4:56 PM, [EMAIL PROTECTED] wrote: If you compile with -DSQLITE_MEMORY_SIZE= then SQLite will *never* call malloc(). Instead, it uses a static array that is bytes in size for all of its memory needs. You can

Re: [sqlite] Memory Usage

On Nov 17, 2007, at 4:56 PM, [EMAIL PROTECTED] wrote: If you compile with -DSQLITE_MEMORY_SIZE= then SQLite will *never* call malloc(). Instead, it uses a static array that is bytes in size for all of its memory needs. You can get by with as little as 100K or so of memory, though th

Re: [sqlite] Memory Usage

Dr. Hipp, thanks, I'm sure I can use one or more of your solutions below to solve my problem... Scott SQLite does maintain a cache of the database file. It will hold up to 2000 pages by default. You can change the cache size by using the "PRAGMA cache_size=N" pragma. You can set N as small a

Re: [sqlite] Memory Usage

Raymond Hurst <[EMAIL PROTECTED]> wrote: > Hi Scott, > > Ooops..meant to say the following. > > My initial evaluation of this database was that it allocates memory for > each operation on the database. It returns the memory only when the > database is CLOSED. So the behavior you see is normal.

Re: [sqlite] Memory Usage

There must be a way to flush what ever is being cached. Help It's hard to believe I'm the only guy that wants to keep the database open and just do inserts, to save CPU time. Scott rhurst2 wrote: > > Hi Scott, > > Ooops..meant to say the following. > > My initial evaluation of t

Re: [sqlite] Memory Usage

Hi Scott, Ooops..meant to say the following. My initial evaluation of this database was that it allocates memory for each operation on the database. It returns the memory only when the database is CLOSED. So the behavior you see is normal. Ray Hurst ScottDerrick wrote: I am using sqlite3 in

Re: [sqlite] Memory Usage

Hi Scott, My initial evaluation of this database was that it allocates memory for each operation on the database. It returns the memory only when the database is committed. So the behavior you see is normal. Ray Hurst ScottDerrick wrote: I am using sqlite3 in a DAQ device. Data can be viewed

[sqlite] Memory Usage

I am using sqlite3 in a DAQ device. Data can be viewed on the unit using a Rails enabled web server. The data is being stored to the database every 1 to 5 seconds. I wanted to leave the the database open for as long as teh application is running and then use a IMMEDIATE, PREPARE-INSERT(x), FINA

Re: [sqlite] Memory Usage

At 14:40 30/10/2006, you wrote: I am suffering a 4GB memory 64-bit Zeon Linux box, which keeps crashing with 'No available memory'. I'm finding it quite hard to break down the memory into what processes are paged-in and using what's available. Sqlite seemed to be the smoking gun, so although

Re: [sqlite] Memory Usage

If you are seeing different memory usage patterns for identical code based on if it is run from xinetd or on your command line, then I would check the process environment that xinetd creates. Maybe some component that sqlite uses is acting differently based on environment variables? I d

Re: [sqlite] Memory Usage

What happens when you write a simple test program to open the DB? You can certainly run Valgrind on that or even put in your own debug statements to examine the heap usage. This would be the first thing to do in indentifying the problem. It will tell you whether to look at Sqlite or your app

Re: [sqlite] Memory Usage

Martin, Excellent, thanks for the reference. I can now very clearly see that sqlite3 is in fact one of the smaller memory users. In fact 7.5 times less than MySQL client I am also linked to, and that's just a client not an entire database... Thanks, Martin Jenkins wrote: Ben Clewett wrote

Re: [sqlite] Memory Usage

Ben Clewett wrote: If you know a good URL on Linux virtual memory and allocation, I would be extremely interested. You could try: http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html The next two link to pages with links to a PDF of the "gorman" book "Understan

Re: [sqlite] Memory Usage

Hi Nuno, Sqlite is one mailing list I have consistently found absolutely excelent knowledge, thanks again for your information. I don't know whether this should be off-thread now, but I don't have your email address. I'll have to research memory allocation further. But I'm glad to know tha

Re: [sqlite] Memory Usage

I don't know whether I am right in this perspective. Just to know whether sqlite is causing the high memory usage, comment the commands (statements) related to sqlite and check the memory status. Thanks, Lloyd. On Mon, 2006-10-30 at 12:45 +, Nuno Lucas wrote: > On 10/30/06, Ben Clewett <

Re: [sqlite] Memory Usage

On 10/30/06, Ben Clewett <[EMAIL PROTECTED]> wrote: Nuno, Thanks for the excelent description of my error. I have learnt a little more about Linux virtual memory model. Very glad to hear Sqlite is as perfect as ever :) My problem, which is definitely my problem, is that 90 x 16MB of reserved

Re: [sqlite] Memory Usage 2

Ben Clewett <[EMAIL PROTECTED]> wrote: > > Is the allocation of 4108 KB normal for Sqlite? No. Not for me. If you open the same database using the command-line client, how much memory does it use? > Why, then run from xined, is the memory allocation four times as much? > Why is just Sqlite eff

Re: [sqlite] Memory Usage

Nuno, Thanks for the excelent description of my error. I have learnt a little more about Linux virtual memory model. Very glad to hear Sqlite is as perfect as ever :) My problem, which is definitely my problem, is that 90 x 16MB of reserved memory is still a loss of 1.4G. Especially as I

Re: [sqlite] Memory Usage

On 10/30/06, Ben Clewett <[EMAIL PROTECTED]> wrote: Hi Numo and others, I am very glad to hear the consensus is that there is nothing wrong with libsqlite3.so.0.8.6. However the fact is that the 'open' still acquires 16MB of memory. Immediately Before: VmSize: 8572 kB VmLck:

Re: [sqlite] Memory Usage 2

Hi again Numo and others, I am looking further into this problem. I hope I am not asking stupid questions. I can confirm definitely that my program, when started by 'xinetd' still give the following result: (As previous email) sqlite3_open(sDatabaseFile, &hSqlite) (= SQLITE_OK) Immedia

Re: [sqlite] Memory Usage

Hi Numo and others, I am very glad to hear the consensus is that there is nothing wrong with libsqlite3.so.0.8.6. However the fact is that the 'open' still acquires 16MB of memory. Immediately Before: VmSize: 8572 kB VmLck: 0 kB VmRSS:2252 kB VmDa

Re: [sqlite] Memory Usage

Most probably it will be a memory leak in your program. We must release the dynamically allocated memory ourselves. So check whether you are forgetting to do that. Most probably that leak will be happening inside some loops or repeatedly calling functions. On Fri, 2006-10-27 at 17:00 +0100, Ben C

Re: [sqlite] Memory Usage

At 18:00 27/10/2006, you wrote: Dear Sqlite, I very much enjoy using Sqlite, it is extremely useful. I have a memory usage query. I am linking to libsqlite3.so.0.8.6. After calling sqlite3_open(...) I find my programs data memory jumps by 16392 Kb. This seems a lot. The database I am open

Re: [sqlite] Memory Usage

On 10/27/06, Ben Clewett <[EMAIL PROTECTED]> wrote: I am linking to libsqlite3.so.0.8.6. After calling sqlite3_open(...) I find my programs data memory jumps by 16392 Kb. This seems a lot. The database I am opening is only 26K in size. There are many different ways of memory "jump" (like lin

  1   2   >