[sqlite] Totalview Debugger MemoryScape showing leak in my SQLite code?

Hi there, I am a huge fan of SQLIte and have recently done some code that I am debugging with TotalView and MemoryScape (http://www.roguewave.com) - VERY VERY nice debugger, I have used GDB as well alot but Totalview is simply awesome NOTE: C coding is NOT my day job ;-) I am using

[sqlite] Totalview Debugger MemoryScape showing leak in my SQLite code?

Hi there, I am a huge fan of SQLIte and have recently done some code that I am debugging with TotalView and MemoryScape (http://www.roguewave.com) - VERY VERY nice debugger, I have used GDB as well alot but Totalview is simply awesome NOTE: C coding is NOT my day job ;-) I am using

[sqlite] Totalview Debugger MemoryScape showing leak in my SQLite code?

Hi there, I am a huge fan of SQLIte and have recently done some code that I am debugging with TotalView and MemoryScape (http://www.roguewave.com) - VERY VERY nice debugger, I have used GDB as well alot but Totalview is simply awesome NOTE: C coding is NOT my day job ;-) I am using

[sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hi there, SQLite is a perfect fit for a logger, the only question I have is once it is in production my database will grow rapidly, how can I implement / mimic a type of rotating log? So in my mind I am thinking that perhaps I can LIMIT the size of the SQLIte DB to say 5 MB? And once the DB

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

there is a certain amount of write overhead. Why do you think you would want to do this? /eno On 10.05.2011, at 10:09, Lynton Grice wrote: Hi there, SQLite is a perfect fit for a logger, the only question I have is once it is in production my database will grow rapidly, how can I implement / mimic

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hi Stephan, Thanks for this, much appreciated. My application is written in pure C, so I guess I will not be able to use your C++ code? Chat later Lynton On 10/05/2011 12:06, Stephan Beal wrote: On Tue, May 10, 2011 at 11:52 AM, Enrico Thierbache...@open-lab.org wrote: I don't think

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hi all, Thanks for your comments...much appreciated.. BTW: if I am using SQLIte as a logger, what PRAGMA statement can I use to say FIX the sqlite database size to say 5 MB? Also, lets say I have a AUTOINCREMENT INTEGER PRIMARY KEY, what will happen when it reaches 5 MB? Will it just

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hi Simon, Thanks for the feedback below, I will write some delete logic to run from time to time...;-) Lynton On 10/05/2011 13:34, Simon Slavin wrote: On 10 May 2011, at 11:42am, Lynton Grice wrote: BTW: if I am using SQLIte as a logger, what PRAGMA statement can I use to say FIX

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hi Enrico, I like the ON INSERT trigger.good idea. So perhaps you have a setLogMaxSize type function in C that allows the client program to say hey, I only want the log to hold a max of 10 000 records.and then I do a select count(*) inside the ON INSERT type trigger and delete

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hey NIco, Now this is great.in fact I was playing with an update hook the other dayand was going to put the deletion logic under the SQLITE_INSERT below But your code looks better ;-) Thanks ! void update_callback( void* udp, int type, const char* db_name, const char* tbl_name,

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

to get the current TIME, then if the time is = log_check_interval then it is time to DELETE all records older than the history_retain_time. I think that would be better. Chat later Lynton On 10/05/2011 16:38, Stephan Beal wrote: On Tue, May 10, 2011 at 4:32 PM, Lynton Grice lynton.gr

Re: [sqlite] SQLite as a Logger: How to mimic rotation of logs?

Hi Jay, Wow, thanks for your detailed message below.much appreciated ;-) I will try the PRAGMA and also the msg_seq.great. Lynton On 10/05/2011 19:00, Jay A. Kreibich wrote: On Tue, May 10, 2011 at 12:42:14PM +0200, Lynton Grice scratched on the wall: Hi all, Thanks

[sqlite] sqlite3_bind_blob CHOPS off at first NULL char

Hi there, I have a 15MB file I need to read and store in an SQLite database. There are NULL characters in the first couple header fields and the rest is pure BINARY data. You can imagine the file looking like: ppphNULNUL3STR.and then all the BINARY data.. I am no C expert but I

[sqlite] sqlite3_bind_blob CHOPS off at first NULL char

Hi there, I have a 15MB file I need to read and store in an SQLite database. I have attached a file to show you that there are NULL characters in the first couple header fields and the rest is pure BINARY data. But just incase the image gets stipped off while posting you can imagine the file

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

Hi Paul, Yes, I am using fsize as the msg.num_bytes_in.. When I run SELECT Length(raw_stream_in) FROM test I get the FULL 13035138 bytes But it I read the raw_stream_in and sent it to a file (using .output result.txt) I only see chars up until the FIRST NUL. Any ideas? Thanks

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

Hi Richard, My apologies on the subject line, I did not mean it to say the function is incorrect, but merely to understand why it is chopping it off I am responding to the other now Lynton On 03/04/2011 13:16, Richard Hipp wrote: Others have responded with requests for information

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Lynton Grice [lynton.gr...@logosworld.com] Sent: Sunday, April 03, 2011 5:37 AM To: sqlite-users@sqlite.org Subject: EXT :[sqlite] sqlite3_bind_blob CHOPS off at first NULL char Hi there, I have a 15MB file I need to read

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char - FULL CODE

Hi Drake, Hi Richard, Hi Paul, Thanks for all your comments, I really appreciate the help. Please see FULL code below. You can literally copy and paste this and compile it with something like: gcc -m64 -Wall -g -I./ -c ./main.c -o ./bigbyte.o gcc -o ./bigbyte ./bigbyte.o -L./ -Wl,-rpath,./

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

code. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Lynton Grice [lynton.gr...@logosworld.com] Sent: Sunday, April 03

Re: [sqlite] :Re: sqlite3_bind_blob CHOPS off at first NULL char

[sqlite-users-boun...@sqlite.org] on behalf of Lynton Grice [lynton.gr...@logosworld.com] Sent: Sunday, April 03, 2011 7:05 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char Hi Michael, When I mean't the .output I was meaning

Re: [sqlite] :Re: sqlite3_bind_blob CHOPS off at first NULL char

Thanks, issue solved with the following: len = sqlite3_column_bytes(stmt,2); memcpy(msg-raw_stream_in, sqlite3_column_text(stmt, 2), len); Thanks to everyone for your help ;-) Lynton On 03/04/2011 14:52, Paul van Helden wrote: On Sun, Apr 3, 2011 at 2:46 PM, Lynton

[sqlite] Runtime memory growth concerns

Hi there, I am a huge SQLite fan and am very curious as to why the following would happen. Essentially I have a queue implementation using SQLIte in WAL mode and it is working really well. However, at runtime if I watch the Task Manager in windows as thousands of messages are sent to

Re: [sqlite] Runtime memory growth concerns

out that sqlite3_step(stmt); line then all is normal. Any ideas? Do I need to set some PRAGMA stuff to get the memory better? Thanks Lynton From: Lynton Grice [mailto:lynton.gr...@logosworld.com] Sent: 07 December 2010 08:27 PM To: 'sqlite-users@sqlite.org' Subject: Runtime

Re: [sqlite] Runtime memory growth concerns

? From: Lynton Grice [mailto:lynton.gr...@logosworld.com] Sent: 07 December 2010 10:26 PM To: 'sqlite-users@sqlite.org' Subject: RE: Runtime memory growth concerns Hi there, Regarding the message I sent earlier...Just a quick update from my side, it seems when I run the following

Re: [sqlite] Runtime memory growth concerns

Hi there, I am playing with the PRAGMA cache_size and it seems to help lots. I will keep you posted ;-) Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Lynton Grice Sent: 07 December 2010 10:31 PM To: sqlite-users

Re: [sqlite] Runtime memory growth concerns

that needs to be cleaned up correctly... But anyway, the PRAGMA cache_size did help a lot Chat later Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Lynton Grice Sent: 08 December 2010 03:03 PM To: 'General Discussion

[sqlite] Concern over runtime memory growth?

Hi there, I have implemented a queue using SQLIte in WAL mode and it seems to be working well.. Now that I am testing and send thousands of messages to the queue I am watching the memory growth of the application grow and grow. I have make a queue shared lib / dll and have seen

Re: [sqlite] Concern over runtime memory growth?

To: General Discussion of SQLite Database Subject: Re: [sqlite] Concern over runtime memory growth? On Wed, Dec 8, 2010 at 8:58 AM, Lynton Grice lynton.gr...@logosworld.comwrote: Hi there, I have implemented a queue using SQLIte in WAL mode and it seems to be working well.. Now that I am testing

Re: [sqlite] Compile 64bit version of SQLIte on Solaris?

information available How can I get the ELF class to reflect ELF 64-bit? Any ideas? Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Lynton Grice Sent: 22 November 2010 09:33 AM To: 'General Discussion of SQLite

Re: [sqlite] Compile 64bit version of SQLIte on Solaris?

November 2010 11:05 AM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Compile 64bit version of SQLIte on Solaris? On 11/22/2010 03:25 PM, Lynton Grice wrote: Hi there, That does not work, I tried that already. # /usr/local/bin/gcc -m64 -D_FILE_OFFSET_BITS=64 -c -fPIC -DHAVE_USLEEP sqlite3.c

Re: [sqlite] SQLite really SLOW on Solaris?

it was on Debian. Any ideas? Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Eric Smith Sent: 20 November 2010 09:13 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] SQLite really SLOW on Solaris? Lynton

Re: [sqlite] SQLite really SLOW on Solaris?

...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Slavin Sent: 21 November 2010 12:23 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] SQLite really SLOW on Solaris? On 21 Nov 2010, at 10:10am, Lynton Grice wrote: in my queue implementation I add

Re: [sqlite] SQLite really SLOW on Solaris?

13:08:31 +0200, Lynton Grice lynton.gr...@logosworld.com wrote: Hi Simon, Yup, on Solaris I ran this and it was 1000 milliseconds for the following: int theReturn = sqlite3_sleep(1); Any way you know I can make it sleep on Solaris for 1 millisecond? Any other C tricks you know of? Standard C

[sqlite] Compile 64bit version of SQLIte on Solaris?

Hi there, I am getting the following error when trying to compile SQLite on a Solaris 64 bit machine: /usr/local/bin/gcc -m64 -R/usr/sfw/lib/64 -c -fPIC -DHAVE_USLEEP sqlite3.c sqlite3.c:1: sorry, unimplemented: 64-bit mode not compiled in Can someone please let me know how I can do

Re: [sqlite] Compile 64bit version of SQLIte on Solaris?

shared lib in ELF class 64.. This possible? Thanks Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Lynton Grice Sent: 22 November 2010 09:28 AM To: sqlite-users@sqlite.org Subject: [sqlite] Compile 64bit version of SQLIte

Re: [sqlite] Compile 64bit version of SQLIte on Solaris?

, not stripped, no debugging information available How can I get the ELF class to reflect ELF 64-bit? Any ideas? Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Lynton Grice Sent: 22 November 2010 09:33 AM To: 'General

[sqlite] SQLite really SLOW on Solaris?

Hi there, I compiled SQLite on Debain (VM) first using the following: /usr/local/bin/gcc -c -fPIC sqlite3.c /usr/local/bin/gcc -shared -o libsqlite3.so sqlite3.o And then I created my own shared lib using SQLite and when I use the lib on Debian it is SUPER FAST. Then I installed

Re: [sqlite] SQLite really SLOW on Solaris?

-users-boun...@sqlite.org] On Behalf Of Roger Binns Sent: 20 November 2010 07:31 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] SQLite really SLOW on Solaris? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2010 07:12 AM, Lynton Grice wrote: On Debian it processes a couple

Re: [sqlite] SQLite really SLOW on Solaris?

Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Eric Smith Sent: 20 November 2010 09:13 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] SQLite really SLOW on Solaris? Lynton Grice wrote: Many thanks for your feedback, much

Re: [sqlite] WAL and multiple writers?

Hi there, Well I'm sure I was, all I did was add a busy handler and it seems to work 100% now ;-) Lynton -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Sylvain Pointeau Sent: 16 November 2010 06:09 PM To: General Discussion

Re: [sqlite] WAL and multiple writers?

: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Lynton Grice Sent: 14 November 2010 05:12 PM To: 'General Discussion of SQLite Database' Subject: [sqlite] WAL and multiple writers? Importance: High Hi there, I have a queue implemented in C using SQLIte

Re: [sqlite] WAL and multiple writers?

:05 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] WAL and multiple writers? On Mon, Nov 15, 2010 at 5:40 AM, Simon Slavin slav...@bigfraud.org wrote: On 15 Nov 2010, at 10:13am, Lynton Grice wrote: Any ideas on having 2 separate applications accessing the same WAL

[sqlite] WAL and multiple writers?

Hi there, I have a queue implemented in C using SQLIte in WAL mode. When I send messages to the queueit is fine, and I can receive messages from the queue no problem. BUT when I try run BOTH applications at the same time the sender works 100% and the receiver seems to be able to

Re: [sqlite] Compiling SQLite on Solaris 64bit and SPARC?

Of Lynton Grice Sent: 07 November 2010 07:01 PM To: 'General Discussion of SQLite Database' Subject: [sqlite] Compiling SQLite on Solaris 64bit and SPARC? Importance: High Hi there, I have just downloaded the latest version of SQLite (sqlite-amalgamation-3.7.3.tar.gz) and am trying to compile

Re: [sqlite] Compiling SQLite on Solaris 64bit and SPARC?

] On Behalf Of David Kirkby Sent: 08 November 2010 09:35 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Compiling SQLite on Solaris 64bit and SPARC? On 8 November 2010 16:09, Lynton Grice lynton.gr...@logosworld.com wrote: Hi all, Anybody out there got the latest version

[sqlite] Compiling SQLite on Solaris 64bit and SPARC?

Hi there, I have just downloaded the latest version of SQLite (sqlite-amalgamation-3.7.3.tar.gz) and am trying to compile it on my Solaris 10 64bit machine, but get an error for some reason (see below). I ran a ./configure (with no parameters) and it seemed to be fine, but then the moment

[sqlite] SQLite is perfect for FILE based MESSAGE QUEUE?

Hi there, I am a HUGE SQLite fan and have an interesting question I have been scratching my head about for a couple days now. First off my daily job is very much around messaging and I am very familiar with message queue products like Websphere MQ and Fiorano MQ. When you install

Re: [sqlite] SQLite is perfect for FILE based MESSAGE QUEUE?

...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Drake Wilson Sent: 30 September 2010 11:51 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] SQLite is perfect for FILE based MESSAGE QUEUE? Quoth Lynton Grice lynton.gr...@logosworld.com, on 2010-09-30 10:11:54

Re: [sqlite] SQLite is perfect for FILE based MESSAGE QUEUE?

Database Subject: Re: [sqlite] SQLite is perfect for FILE based MESSAGE QUEUE? Quoth Lynton Grice lynton.gr...@logosworld.com, on 2010-09-30 12:05:18 +0200: BTW: What is WAL WAL mode uses a write-ahead log instead of a rollback journal, which can help reduce write activity but requires the use

Re: [sqlite] SQLite is perfect for FILE based MESSAGE QUEUE?

. I ended up using just one table in one DB as indexing makes things simple and fast and I support arbitrary queues with multiple readers. WAL i've been trying these last few days and seems trouble-free. On 30/09/2010 6:11 PM, Lynton Grice wrote: Hi there, I am a HUGE SQLite fan and have