Hi,
This is a follow up of mails sent to this mailing
list last month.
We were benchmarking several db access methods and
comparing them with postgres.
Lots of people advised us to try pg 7.1 instead of
pg 6.5.3
This turns out to be good advice as regards
performance. (We would like to
, April 19, 2001 4:13 AM
Subject: Re: Fast DB access
"Chutzpah" is an interesting way of putting it. I've been thinking
of them as "slimeballs in the busy of conning webkids into
thinking they have a real RDBM product".
(It isn't a moot point, because it's the
17, 2001 4:41
PM
Subject: Fast DB access
Hi,
A few months back we asked modperl mailing list
on alternate methods of DB access to postgres (with the same subject). We got
some decent alternatives. We are putting back some of the work we have done on
this issue.
We had
01 8:08 PM
Subject: [OT] Re: Fast DB access
On 18 Apr 2001, Clayton Cottingham aka drfrog wrote:
[drfrog]$ perl fast_db.pl
postgres
16 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) @ 400.00/s (n=20)
mysql
3 wallclock secs ( 0.07 usr + 0.00 sys = 0.07 CPU) @ 285.71/s (n=20)
pos
Thanks for pointing out the mistake in postgres.
Your Advice makes lots of sense.
V Murali
- Original Message -
From: Cees Hek [EMAIL PROTECTED]
To: Murali V [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, April 20, 2001 1:45 AM
Subject: [OT] Re: Fast DB access
On Thu, 19 Apr
at www.diffsoft.com
- Original Message -
From: Cees Hek [EMAIL PROTECTED]
To: Murali V [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, April 20, 2001 1:45 AM
Subject: [OT] Re: Fast DB access
On Thu, 19 Apr 2001, Murali V wrote:
Hi,
If you read the code more deeply, you'll find
On Thu, 19 Apr 2001, Murali V wrote:
Hi,
If you read the code more deeply, you'll find that the timeit is only
wrapped around select and not around insert.
We've written the insert code so that in the first round you can populate
the database.
You comment out the insert code after the
3631445, 3431470
Visit us at www.diffsoft.com
- Original Message -
From: Perrin Harkins [EMAIL PROTECTED]
To: [EMAIL PROTECTED]; Joe Brenner [EMAIL PROTECTED]
Sent: Thursday, April 19, 2001 4:13 AM
Subject: Re: Fast DB access
"Chutzpah" is an interesting way of putting
On Thu, 19 Apr 2001, Differentiated Software Solutions Pvt. Ltd., wrote:
I get a feeling that the point we were trying to make is going to be
missed. MLDBM is not a bad alternative to databases under specific
conditions !!
That point was definately not missed by me, and I have learned
please be advised i also posted this benchmark to
[EMAIL PROTECTED]
some interesting thoughts etc on this there too
thread is:
[SQL] any proper benchmark scripts?
if anyone is on the mysql lists please post to there
at www.diffsoft.com
- Original Message -
From: Cees Hek [EMAIL PROTECTED]
To: Murali V [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Friday, April 20, 2001 1:45 AM
Subject: [OT] Re: Fast DB access
On Thu, 19 Apr 2001, Murali V wrote:
Hi,
If you read the code more deeply, you'll
:
Differentiated Software
Solutions Pvt. Ltd.,
To: [EMAIL PROTECTED]
Sent: Tuesday, April 17, 2001 4:41
PM
Subject: Fast DB access
Hi,
A few months back we asked modperl mailing list
on alternate methods of DB access to postgres (with the same subject). We got
some
On Wed, 18 Apr 2001, Differentiated Software Solutions Pvt. Ltd., wrote:
Hi,
There are 4 responses to our results. We will answer them to the best of our ability.
MATT This is a very very old version of postgresql. Try it again with 7.1 for
MATT more respectable results.
Accepted. We
On 17 Apr 2001 18:24:43 -0700, clayton wrote:
i wanted a good benchmark for postgres and mysql
{i hope to transpose the sql properly!}
This is a good comparison of MySQL and PostgreSQL 7.0:
"Open Source Databases: As The Tables Turn" --
http://www.phpbuilder.com/columns/tim20001112.php3
On 18 Apr 2001 12:00:57 +0530, Differentiated Software Solutions Pvt. Ltd., wrote:
Hi,
There are 4 responses to our results. We will answer them to the best of our ability.
MATT This is a very very old version of postgresql. Try it again with 7.1 for
MATT more respectable results.
Matthew Kennedy wrote:
On 17 Apr 2001 18:24:43 -0700, clayton wrote:
i wanted a good benchmark for postgres and mysql
{i hope to transpose the sql properly!}
This is a good comparison of MySQL and PostgreSQL 7.0:
"Open Source Databases: As The Tables Turn" --
Matthew Kennedy wrote:
I'm on several postgresql mailing lists and couldn't find a recent post
from you complaining about 6.5.3 performance problems (not even by an
archive search). Your benchmark is worthless until you try postgresql
7.1. There have been two major releases of postgresql
clayton cottingham [EMAIL PROTECTED] writes:
Matthew Kennedy wrote:
On 17 Apr 2001 18:24:43 -0700, clayton wrote:
i wanted a good benchmark for postgres and mysql
{i hope to transpose the sql properly!}
This is a good comparison of MySQL and PostgreSQL 7.0:
"Open
On 18 Apr 2001 08:49:38 -0700, clayton cottingham wrote:
Matthew Kennedy wrote:
On 17 Apr 2001 18:24:43 -0700, clayton wrote:
i wanted a good benchmark for postgres and mysql
{i hope to transpose the sql properly!}
This is a good comparison of MySQL and PostgreSQL 7.0:
Matthew Kennedy wrote:
This might help too:
http://www.angelfire.com/nv/aldev/pgsql/GreatBridge.html
Of course benchmarks are so debatable anyway..
Matt
i saw those they are pretty good
but greatbridge is tied into postgres
somehow
im looking for impartial benchmarks
nonetheless i
On Wed, 18 Apr 2001, ed phillips wrote:
You can scale any of these databases; Oracle, MySQL or PostgreSQL, but
please research each one thoroughly and tune it properly before you do
your benchmarking. And, again, MySQL does support transactions now.
Such chutzpah for them to have promoted
[EMAIL PROTECTED] wrote:
Matthew Kennedy wrote:
I'm on several postgresql mailing lists and couldn't find a recent post
from you complaining about 6.5.3 performance problems (not even by an
archive search). Your benchmark is worthless until you try postgresql
7.1. There have been two
"Chutzpah" is an interesting way of putting it. I've been thinking
of them as "slimeballs in the busy of conning webkids into
thinking they have a real RDBM product".
(It isn't a moot point, because it's the same people working on
it: human character issues are actually relevant when
At 3:43 PM -0700 4/18/01, Perrin Harkins wrote:
"Chutzpah" is an interesting way of putting it. I've been thinking
of them as "slimeballs in the busy of conning webkids into
thinking they have a real RDBM product".
(It isn't a moot point, because it's the same people working on
it: human
[drfrog]$ perl fast_db.pl
postgres
16 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) @ 400.00/s (n=20)
mysql
3 wallclock secs ( 0.07 usr + 0.00 sys = 0.07 CPU) @ 285.71/s (n=20)
postgres
17 wallclock secs ( 0.06 usr + 0.00 sys = 0.06 CPU) @ 333.33/s (n=20)
mysql
3 wallclock secs ( 0.01
Clayton Cottingham aka drfrog wrote:
[drfrog]$ perl fast_db.pl
postgres
16 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) @ 400.00/s (n=20)
mysql
3 wallclock secs ( 0.07 usr + 0.00 sys = 0.07 CPU) @ 285.71/s (n=20)
postgres
17 wallclock secs ( 0.06 usr + 0.00 sys = 0.06 CPU) @
On 18 Apr 2001, Clayton Cottingham aka drfrog wrote:
[drfrog]$ perl fast_db.pl
postgres
16 wallclock secs ( 0.05 usr +0.00 sys = 0.05 CPU) @ 400.00/s (n=20)
mysql
3 wallclock secs ( 0.07 usr +0.00 sys = 0.07 CPU) @ 285.71/s (n=20)
postgres
17 wallclock secs ( 0.06 usr +
17, 2001 4:41
PM
Subject: Fast DB access
Hi,
A few months back we asked modperl mailing list
on alternate methods of DB access to postgres (with the same subject). We got
some decent alternatives. We are putting back some of the work we have done on
this issue.
We had
l 19, 2001 8:08 PM
Subject: [OT] Re: Fast DB access
On 18 Apr 2001, Clayton Cottingham aka drfrog wrote:
[drfrog]$ perl fast_db.pl
postgres
16 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) @ 400.00/s (n=20)
mysql
3 wallclock secs ( 0.07 usr + 0.00 sys = 0.07 CPU) @ 285.
Hi,
A few months back we asked modperl mailing list on
alternate methods of DB access to postgres (with the same subject). We got some
decent alternatives. We are putting back some of the work we have done on this
issue.
We had a project to program an ad server. This is
not really an OLTP
On Tue, 17 Apr 2001, Differentiated Software Solutions Pvt. Ltd., wrote:
H/W : Celeron 433 with 64 MB RAM, IDE HDD using RH 6.1, perl 5.005,
Postgres 6.5.3
This is a very very old version of postgresql. Try it again with 7.1 for
more respectable results.
--
Matt/
/||** Founder and
Matt Sergeant writes:
On Tue, 17 Apr 2001, Differentiated Software Solutions Pvt. Ltd., wrote:
H/W : Celeron 433 with 64 MB RAM, IDE HDD using RH 6.1, perl 5.005,
Postgres 6.5.3
This is a very very old version of postgresql. Try it again with 7.1 for
more respectable results.
b) Flat file : Create a Linux directory structure with the same hierarchy
as
the attributesi.e., directory structure has
publishers/sizes/types/ip numbers. ip numbers is the file name
which
contains a list of ads. Objective is to pick the right file, open this
file
and create a hash with
Matt Sergeant wrote:
On Tue, 17 Apr 2001, Differentiated Software Solutions Pvt. Ltd., wrote:
H/W : Celeron 433 with 64 MB RAM, IDE HDD using RH 6.1, perl 5.005,
Postgres 6.5.3
This is a very very old version of postgresql. Try it again with 7.1 for
more respectable results.
im
Matt Sergeant sent the following bits through the ether:
Note that Theo Schlossnagel was saying over lunch at ApacheCon that if
your filename has more than 8 characters on Linux (ext2fs) it skips from a
hashed algorithm to a linear algorithm (or something to that affect). So
go careful
At 09:20 PM 11/09/00 +, Tim Bunce wrote:
On Thu, Nov 09, 2000 at 08:27:29PM +, Matt Sergeant wrote:
On Thu, 9 Nov 2000, Ask Bjoern Hansen wrote:
If you're always looking stuff up on simple ID numbers and
"stuff" is a very simple data structure, then I doubt any DBMS can
beat
Message -
From: "Gunther Birznieks" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Friday, November 10, 2000 12:58 AM
Subject: Re: Fast DB access
Isn't that a beta-level filesystem?
At 12:47 AM 11/10/2000 -0600, Les Mikesell wrote:
- Original Message -
From: "Ti
On Fri, 10 Nov 2000, Les Mikesell wrote:
[ReiserFS]
production just to avoid the possibility of a slow fsck after a crash,
but it is enormously faster at creating and deleting files too because
everything is indexed so it would be an ideal stash for fast changing
session data. If you don't
Matthew Byng-Maddick wrote:
On Fri, 10 Nov 2000, Les Mikesell wrote:
[ReiserFS]
production just to avoid the possibility of a slow fsck after a crash,
but it is enormously faster at creating and deleting files too because
everything is indexed so it would be an ideal stash for fast
Hi,
Firstly, thanks for bringing these results back to the mailing list...
having seen this sort of problem previously, but without (IIRC) having
done side-by-side comparisons between these various techniques, I'm keen
to see what you find.
"Differentiated Software Solutions Pvt. Ltd" wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Most modern DBMS software should be able to handle 50 queries per second
on decent hardware, provided the conditions are right. You're not going to
get anything better with flat files.
Hmm... I guess it all depends on what your queries look
On Thu, 9 Nov 2000, Ask Bjoern Hansen wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Most modern DBMS software should be able to handle 50 queries per second
on decent hardware, provided the conditions are right. You're not going to
get anything better with flat files.
Sweetman [EMAIL PROTECTED]
To: Differentiated Software Solutions Pvt. Ltd [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Thursday, November 09, 2000 8:59 PM
Subject: Re: Fast DB access
Hi,
Firstly, thanks for bringing these results back to the mailing list...
having seen this sort of problem
Isn't that a beta-level filesystem?
At 12:47 AM 11/10/2000 -0600, Les Mikesell wrote:
- Original Message -
From: "Tim Bunce" [EMAIL PROTECTED]
If you're always looking stuff up on simple ID numbers and
"stuff" is a very simple data structure, then I doubt any DBMS can
beat
On Thu, Nov 09, 2000 at 11:54:42AM -0800, Ask Bjoern Hansen wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
If you're always looking stuff up on simple ID numbers and
"stuff" is a very simple data structure, then I doubt any DBMS can
beat
open D, "/data/1/12/123456" or ...
from a
- Original Message -
From: "Tim Bunce" [EMAIL PROTECTED]
If you're always looking stuff up on simple ID numbers and
"stuff" is a very simple data structure, then I doubt any DBMS can
beat
open D, "/data/1/12/123456" or ...
from a fast local filesystem.
Note
On Thu, Nov 09, 2000 at 08:27:29PM +, Matt Sergeant wrote:
On Thu, 9 Nov 2000, Ask Bjoern Hansen wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Most modern DBMS software should be able to handle 50 queries per second
on decent hardware, provided the conditions are right.
: Fast DB access
On Wed, 11 Oct 2000, Matt Sergeant wrote:
I really think that sometimes going for a flat file layout *can* be
much
more reliable and scalable then RDBMS software. It all really depends
on
what you plan to do with the data and what you would like to get out
of
it.
I
: [EMAIL PROTECTED]
Sent: Wednesday, October 11, 2000 1:56 PM
Subject: Re: Fast DB access
"Differentiated Software Solutions Pvt. Ltd" wrote:
Hi,
We have an application where we will have to service as high as 50
queries a second.
We've discovered that most database just c
Hi there,
On Wed, 8 Nov 2000, Differentiated Software Solutions Pvt. Ltd wrote:
We are returning after extensive tests of various options suggested.
Did you try different indexing mechanisms in your tests?
73,
Ged.
"Differentiated Software Solutions Pvt. Ltd" wrote:
3. We have a problem rebuilding this database in the ram even say every
1000 requests.
What problem are you having with it?
We tried using dbm and found it a good compromise solution.
We found that it is about 8 times faster than
On Wed, Nov 08, 2000 at 10:49:00AM -0800, Perrin Harkins wrote:
Also, you could try using mmap for reading the files, or possibly the
Cache::Mmap module.
If you do play with mmap, note that it can lose some or all of it's
effeciency in SMP environments, or so I've read.
- Barrie
PM
Subject: Re: Fast DB access
Hi there,
On Wed, 8 Nov 2000, Differentiated Software Solutions Pvt. Ltd wrote:
We are returning after extensive tests of various options suggested.
Did you try different indexing mechanisms in your tests?
73,
Ged.
-
From: Perrin Harkins [EMAIL PROTECTED]
To: Differentiated Software Solutions Pvt. Ltd [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Thursday, November 09, 2000 12:19 AM
Subject: Re: Fast DB access
"Differentiated Software Solutions Pvt. Ltd" wrote:
3. We have a problem
On Thu, 9 Nov 2000, Differentiated Software Solutions Pvt. Ltd wrote:
When we rebuild the hash in the RAM it takes too much time.
Did you try using Storable as the data format? It has a function to load
from files which is very fast.
- Perrin
Hi,
We have an application where we will have to
service as high as 50 queries a second.
We've discovered that most database just cannot
keep pace.
The only option we know is to service queries out
of flat files.
Can somebody give us pointers o n what modules are
available to create flat
On Wed, 11 Oct 2000, Differentiated Software Solutions Pvt. Ltd wrote:
Hi,
We have an application where we will have to service as high as 50
queries a second. We've discovered that most database just cannot keep
pace.
The only option we know is to service queries out of flat files. Can
"Differentiated Software Solutions Pvt. Ltd" wrote:
Hi,
We have an application where we will have to service as high as 50
queries a second.
We've discovered that most database just cannot keep pace.
The only option we know is to service queries out of flat files.
There is a DBD
On Wed, 11 Oct 2000, Differentiated Software Solutions Pvt. Ltd wrote:
Hi,
We have an application where we will have to service as high as 50 queries a second.
We've discovered that most database just cannot keep pace.
The only option we know is to service queries out of flat files.
"Differentiated Software Solutions Pvt. Ltd" [EMAIL PROTECTED] writes:
We have an application where we will have to service as high as 50 =
queries a second.
We've discovered that most database just cannot keep pace.
The only option we know is to service queries out of flat files.
Can
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Most modern DBMS software should be able to handle 50 queries per second
on decent hardware, provided the conditions are right. You're not going to
get anything better with flat files.
Hmm... I guess it all depends on what your queries look like, but
On Wed, 11 Oct 2000, Sander van Zoest wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Most modern DBMS software should be able to handle 50 queries per second
on decent hardware, provided the conditions are right. You're not going to
get anything better with flat files.
Hmm... I
On Wed, 11 Oct 2000, Matt Sergeant wrote:
I really think that sometimes going for a flat file layout *can* be much
more reliable and scalable then RDBMS software. It all really depends on
what you plan to do with the data and what you would like to get out of
it.
I think you chose the
On Wed, 11 Oct 2000, Sander van Zoest wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Lots of places use databases for read-only queries. Having a database
that gets lots of similar queries that are read-only makes it an
unnecessary single point of failure. Why not use the local disk and
On Wed, 11 Oct 2000, Matt Sergeant wrote:
On Wed, 11 Oct 2000, Sander van Zoest wrote:
On Wed, 11 Oct 2000, Matt Sergeant wrote:
Lots of places use databases for read-only queries. Having a database
that gets lots of similar queries that are read-only makes it an
unnecessary single
65 matches
Mail list logo
