Dear Tim,
As you had rightly pointed out we have data which is not volatile. This data
gets updated once an hour by another process (cron job). Concurrency is not
really an issue, because we are not updating the data.
We're now continuing our benchmark on some scaling issues.... basically when
does dbm degenerate. We are increasing number of entries in the dbm file to
see when it will break.
Murali
Differentiated Software Solutions Pvt. Ltd.
176, Ground Floor, 6th Main,
2nd Block, RT Nagar
Bangalore - 560032
Phone : 91 80 3431470
www.diffs-india.com
----- Original Message -----
From: Tim Sweetman <[EMAIL PROTECTED]>
To: Differentiated Software Solutions Pvt. Ltd <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>
Sent: Thursday, November 09, 2000 8:59 PM
Subject: Re: Fast DB access
> Hi,
>
> Firstly, thanks for bringing these results back to the mailing list...
> having seen this sort of problem previously, but without (IIRC) having
> done side-by-side comparisons between these various techniques, I'm keen
> to see what you find.
>
> "Differentiated Software Solutions Pvt. Ltd" wrote:
> > 2. Building the entire DB into a hash variable inside the mod_perl
program
> > is the fastest.... we found it to be 25 times faster than querying a
> > postgres database !!
> > 3. We have a problem rebuilding this database in the ram.... even say
every
> > 1000 requests. We tried using dbm and found it a good compromise
solution.
> > We found that it is about 8 times faster than postgres querying.
>
> I assume from this that your data changes, but slowly, and you're
> getting better performance by accepting that your data be slightly out
> of date.
>
> > 4. Another surprising finding was.... we built a denormalised db on the
> > Linux file system itself, by using the directory and file name as the
key on
> > which we wanted to search. We found that dbm was faster than this.
>
> I'm curious about how you're dealing with the concurrency aspect with
> solutions 2-3. My guess is that, for 2, you're simply storing a hash in
> the memory, which means that each Apache child has its own copy. There
> will, every 1000 requests in that child, be the overhead of querying the
> DB & rebuilding the hash.
>
> 3 presumably means having only _one_ DBMfile. Do the CGI/mod-Perl
> processes rebuild this periodically, or is this done offline by another
> process? Do the CGI/mod-Perl processes have to wait while writes are
> going on?
>
> Cheers
>
> --
> Tim Sweetman
> A L Digital
> ---- moving sideways --->
