Windows XP, visual studio .NET.
The malloc call ends up calling RtlAllocateHeap().
I hacked the code to allocate the aType structure from a circular ring
buffer. This small change made my app go from spending 60% of its time in
malloc to about 3%.
Here's the allocator I hacked in. No freeing is needed, the buffer just
wraps. This assumes that by the time a wrap occurs, memory perviously
allocated at the top is no longer needed. It seems to work ok from my
limited testing. :-)
#define SQLITE_WORK_BUFF_SIZE (128*1024) // make power of 2
#define SQLITE_WORK_BUFF_MASK (SQLITE_WORK_BUFF_SIZE-1)
char sqlite_workBuff[SQLITE_WORK_BUFF_SIZE];
int sqlite_writeIdx=0;
void *SQLITE_ALLOC(int nBytes)
{
void *ret;
sqlite_writeIdx = (sqlite_writeIdx + nBytes) & SQLITE_WORK_BUFF_MASK;
ret = sqlite_workBuff + sqlite_writeIdx;
sqlite_writeIdx+=nBytes;
return ret;
}
Matthew Arrington
Sony Computer Entertainment America
"D. Richard Hipp"
<[EMAIL PROTECTED]>
To
01/10/2005 11:18 sqlite-users@sqlite.org
AM cc
Subject
Please respond to Re: [sqlite] excessive malloc()
[EMAIL PROTECTED] calls
te.org
[EMAIL PROTECTED] wrote:
> I switched my application over to SQLite3 and did some performance
> profiling and found that the majority of the processing time spent is
> making calls to malloc().
It sounds like you have a bad malloc() implementation. What
OS and compiler are you using. Malloc() on linux is normally
very good. Malloc() on win95/98/ME is often quite bad.
Malloc(0 on winNT/2K/XP is in between. Or at least that has
been my experience. If your system gas a slow malloc
implementation, you would do well to use one of the many
fine open-source implementations as a substitute.
>
> I create each table using SQL command:
> CREATE TABLE 'SPY' (key INTEGER PRIMARYKEY UNIQUE, open FLOAT, low FLOAT,
> high FLOAT, close FLOAT, volume FLOAT)
>
> I need each row sorted by date, which I setup as my primary key. Ideally
> each table would be stored sorted on disk since I do a very small amount
of
> updating of only one record per day per table. I could not figure out a
way
> to make do that.
If you put a space between PRIMARY and KEY, it will order the
entrys by the spy.key field. Omit the UNIQUE keyword as that
is redundant. PRIMARY KEYs are always UNIQUE.
>
> The actual code making the call is:
>
> /* Read and parse the table header. Store the results of the parse
> ** into the record header cache fields of the cursor.
> */
> if( pC && pC->cacheValid ){
> aType = pC->aType;
> aOffset = pC->aOffset;
> }else{
> int avail; /* Number of bytes of available data */
> if( pC && pC->aType ){
> aType = pC->aType;
> }else{
> aType = sqliteMallocRaw( 2*nField*sizeof(aType) );
> <<<<<<<<<<<<<<<<<<<<<<< HERE
> }
> aOffset = &aType[nField];
> if( aType==0 ){
> goto no_mem;
> }
>
The line of code marked should only execute once per
SELECT statement, unless you are running with triggers
in which case it will execute once per row. Either way,
it won't be executing 5.5 million times on a 3000 row
table.
--
D. Richard Hipp -- [EMAIL PROTECTED] -- 704.948.4565
