At 04:49 PM 8/25/2005, Chris Browne wrote:
[EMAIL PROTECTED] (Ron) writes:
> At 03:45 PM 8/25/2005, Josh Berkus wrote:
>> > Ask me sometime about my replacement for GNU sort. Â It uses the
>> > same sorting algorithm, but it's an order of magnitude faster due
>> > to better I/O strategy. Â Someda
[EMAIL PROTECTED] (Ron) writes:
> At 03:45 PM 8/25/2005, Josh Berkus wrote:
>> > Ask me sometime about my replacement for GNU sort. Â It uses the
>> > same sorting algorithm, but it's an order of magnitude faster due
>> > to better I/O strategy. Â Someday, in my infinite spare time, I
>> > hope to
At 03:45 PM 8/25/2005, Josh Berkus wrote:
Jeff,
> Ask me sometime about my replacement for GNU sort. Â It uses the same
> sorting algorithm, but it's an order of magnitude faster due to better
> I/O strategy. Â Someday, in my infinite spare time, I hope to demonstrate
> that kind of improvement
Jeff,
> Ask me sometime about my replacement for GNU sort. It uses the same
> sorting algorithm, but it's an order of magnitude faster due to better
> I/O strategy. Someday, in my infinite spare time, I hope to demonstrate
> that kind of improvement with a patch to pg.
Since we desperately need
Since Bruce referred to the "corporate software world" I'll chime in...
It has been a while since adding knobs and dials has been considered a good
idea. Customers are almost always bad at tuning their systems, which decreases
customer satisfaction. While many people assume the corporate types
On Wed, Aug 24, 2005 at 12:12:22PM -0400, Chris Browne wrote:
> Everyone involved in development seems to me to have a reasonably keen
> understanding as to what the potential benefits of threading are; the
> value is that there fall out plenty of opportunities to parallelize
> the evaluation of po
Agreed!!!
But the knowledge to Auto-tune your application comes from years of
understanding of how users are using the so-called "knobs".. But if the "knobs"
are not there in the first place.. how do you know what people are using?
The "so-called" big boys are also using their knowledge base of
Tom, Gavin,
> > To get decent I/O you need 1MB fundamental units all the way down the
> > stack.
>
> It would also be a good idea to have an application that isn't likely
> to change a single bit in a 1MB range and then expect you to record
> that change. This pretty much lets Postgres out of th
[EMAIL PROTECTED] (Steve Poe) writes:
> Chris,
>
> Unless I am wrong, you're making the assumpting the amount of time spent
> and ROI is known. Maybe those who've been down this path know how to get
> that additional 2-4% in 30 minutes or less?
>
> While each person and business' performance gains
This thread covers several performance ideas. First is the idea that
more parameters should be configurable. While this seems like a noble
goal, we try to make parameters auto-tuning, or if users have to
configure it, the parameter should be useful for a significant number of
users.
In the com
of effort reinventing the wheel ... but our time will be repaid much
more if we work at levels that the OS cannot have knowledge of, such as
join planning and data statistics.
Considering a global budget of man-hours which is the best ?
1- Spend it on reimplementing half of VFS in pos
On Wed, 2005-08-24 at 01:56 -0400, Tom Lane wrote:
> "Jeffrey W. Baker" <[EMAIL PROTECTED]> writes:
> > On Wed, 2005-08-24 at 17:20 +1200, Guy Thornley wrote:
> >> Dont forget that already in postgres, you have a process per connection,
> >> and
> >> all the processes take care of their own I/O.
>
"Jeffrey W. Baker" <[EMAIL PROTECTED]> writes:
> On Wed, 2005-08-24 at 17:20 +1200, Guy Thornley wrote:
>> Dont forget that already in postgres, you have a process per connection, and
>> all the processes take care of their own I/O.
> That's the problem. Instead you want 1 or 4 or 10 i/o slaves
>
On Wed, 2005-08-24 at 17:20 +1200, Guy Thornley wrote:
> As for the async IO, sure you might think 'oh async IO would be so cool!!'
> and I did, once, too. But then I sat down and _thought_ about it, and
> decided well, no, actually, theres _very_ few areas it could actually help,
> and in most cas
> Unfortunately I'm really afraid that this conversation is about trees
> when the forest is the problem. PostgreSQL doesn't even have an async
> reader, which is the sort of thing that could double or triple its
> performance. You're talking about block sizes and such, but the kinds
> of improve
"Jeffrey W. Baker" <[EMAIL PROTECTED]> writes:
> To get decent I/O you need 1MB fundamental units all the way down the
> stack.
It would also be a good idea to have an application that isn't likely
to change a single bit in a 1MB range and then expect you to record
that change. This pretty much l
On Tue, 2005-08-23 at 19:31 -0700, Josh Berkus wrote:
> Steve,
>
> > I would assume that dbt2 with STP helps minimize the amount of hours
> > someone has to invest to determine performance gains with configurable
> > options?
>
> Actually, these I/O operation issues show up mainly with DW workloa
Josh Berkus wrote:
Steve,
I would assume that dbt2 with STP helps minimize the amount of hours
someone has to invest to determine performance gains with configurable
options?
Actually, these I/O operation issues show up mainly with DW workloads, so the
STP isn't much use there. If
Steve,
> I would assume that dbt2 with STP helps minimize the amount of hours
> someone has to invest to determine performance gains with configurable
> options?
Actually, these I/O operation issues show up mainly with DW workloads, so the
STP isn't much use there. If I can ever get some of th
Hi Jim,
| How many of these things are currently easy to change with a recompile?
| I should be able to start testing some of these ideas in the near
| future, if they only require minor code or configure changes.
The following
* Data File Size 1GB
* WAL File Size of 16MB
* Block Size of 8K
On Tue, 2005-08-23 at 19:12 -0400, Michael Stone wrote:
> On Tue, Aug 23, 2005 at 05:29:01PM -0400, Jignesh Shah wrote:
> >Actually some of that readaheads,etc the OS does already if it does
> >some sort of throttling/clubbing of reads/writes.
>
> Note that I specified the fully cached case--eve
Chris,
Unless I am wrong, you're making the assumpting the amount of time spent
and ROI is known. Maybe those who've been down this path know how to get
that additional 2-4% in 30 minutes or less?
While each person and business' performance gains (or not) could vary,
someone spending the 50-100h
On Tue, Aug 23, 2005 at 06:09:09PM -0400, Chris Browne wrote:
> [EMAIL PROTECTED] (Jignesh Shah) writes:
> >> Does that include increasing the size of read/write blocks? I've
> >> noticedthat with a large enough table it takes a while to do a
> >> sequential scan, even if it's cached; I wonder if t
On Tue, Aug 23, 2005 at 06:09:09PM -0400, Chris Browne wrote:
What we have been finding, as RAID controllers get smarter, is that it
is getting increasingly futile to try to attach knobs to 'disk stuff;'
it is *way* more effective to add a few more spindles to an array than
it is to fiddle with w
On Tue, Aug 23, 2005 at 05:29:01PM -0400, Jignesh Shah wrote:
Actually some of that readaheads,etc the OS does already if it does
some sort of throttling/clubbing of reads/writes.
Note that I specified the fully cached case--even with the workload in
RAM the system still has to process a heck
[EMAIL PROTECTED] (Jignesh Shah) writes:
>> Does that include increasing the size of read/write blocks? I've
>> noticedthat with a large enough table it takes a while to do a
>> sequential scan, even if it's cached; I wonder if the fact that it
>> takes a million read(2) calls to get through an 8G
> Does that include increasing the size of read/write blocks? I've
> noticedthat with a large enough table it takes a while to do a
> sequential scan,
> even if it's cached; I wonder if the fact that it takes a million
> read(2) calls to get through an 8G table is part of that.
>
Actually some
27 matches
Mail list logo