On 3/22/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
"P Kishor" <[EMAIL PROTECTED]> wrote:
> Richard,
>
> On 3/22/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> ...
> > The problem is that your working set is bigger than your cache
> > which is causing thrashing. I suggest a solution like this:
> >
> > Add entries to table ONE until the table and its unique index get
> > so big that they no longer fit in cache all at once.
>
> What is this "cache" you talk about? Is this the hard disk cache, or
> some kind of OS-level cache, or a SQLite level cache?
All of the above.
> How can I find
> the size of this, and is this cache user-adjustable? If yes, how?
>
On Linux, the "top" command will show you how much OS disk
cache you are currently using. The hard disk cache is usually
small enough that it can be neglected. The OS cache is also
usually much larger than SQLite's own internal cache.
You might run an experiment where you start inserting and
timing each insert, then switch to a different table when
the insert speed drops below a threshold.
ok. I was hoping for something magical like 'showme_cache_size'. This
sounds more voodoo-ish, but do-able. In any case, it doesn't seem that
this OS cache is user-adjustable, at least not without pipe wrenches
and WD-40.
On a note more related to the OP, even if one could adjust the cache
to the working set (and that would be a moving target -- either make
the cache gigantic, or keep on querying the relative sizes of the two
and somehow keep on adjusting the cache, the time taken to determine
uniqueness in a bigger working set will keep on increasing as the
working set itself keeps on increasing... the curve won't be flat.
--
Puneet Kishor http://punkish.eidesis.org/
Nelson Inst. for Env. Studies, UW-Madison http://www.nelson.wisc.edu/
Open Source Geospatial Foundation http://www.osgeo.org/education/
---------------------------------------------------------------------
collaborate, communicate, compete
=====================================================================
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
