On Sun, Sep 2, 2012 at 5:39 PM, Jeff Janes wrote:
> On Thu, Aug 30, 2012 at 8:17 PM, Tom Lane wrote:
>> Robert Haas writes:
>>> On Thu, Aug 30, 2012 at 4:51 PM, Tom Lane wrote:
Bruce Momjian writes:
> On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
>> Ok, I modified
On Thu, Aug 30, 2012 at 8:17 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Thu, Aug 30, 2012 at 4:51 PM, Tom Lane wrote:
>>> Bruce Momjian writes:
On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
> Ok, I modified the part of pg_dump where tremendous number of LOCK
>
Robert Haas writes:
> On Thu, Aug 30, 2012 at 4:51 PM, Tom Lane wrote:
>> Bruce Momjian writes:
>>> On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
Ok, I modified the part of pg_dump where tremendous number of LOCK
TABLE are issued. I replace them with single LOCK TABLE w
On Thu, Aug 30, 2012 at 4:51 PM, Tom Lane wrote:
> Bruce Momjian writes:
>> On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
Ok, I modified the part of pg_dump where tremendous number of LOCK
TABLE are issued. I replace them with single LOCK TABLE with multiple
tables.
Bruce Momjian writes:
> On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
>>> Ok, I modified the part of pg_dump where tremendous number of LOCK
>>> TABLE are issued. I replace them with single LOCK TABLE with multiple
>>> tables. With 100k tables LOCK statements took 13 minutes in tot
On Thu, Aug 30, 2012 at 04:51:56PM -0400, Tom Lane wrote:
> Bruce Momjian writes:
> > On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
> >>> Ok, I modified the part of pg_dump where tremendous number of LOCK
> >>> TABLE are issued. I replace them with single LOCK TABLE with multiple
>
On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote:
> >> Yeah, Jeff's experiments indicated that the remaining bottleneck is lock
> >> management in the server. What I fixed so far on the pg_dump side
> >> should be enough to let partial dumps run at reasonable speed even if
> >> the who
On Thu, May 31, 2012 at 12:25 PM, Tom Lane wrote:
>> No, Tatsuo's patch attacks a phase dominated by latency in some
>> setups.
>
> No, it does not. The reason it's a win is that it avoids the O(N^2)
> behavior in the server. Whether the bandwidth savings is worth worrying
> about cannot be prov
Claudio Freire writes:
> On Thu, May 31, 2012 at 11:50 AM, Tom Lane wrote:
>> The performance patches we applied to pg_dump over the past couple weeks
>> were meant to relieve pain in situations where the big server-side
>> lossage wasn't the dominant factor in runtime (ie, partial dumps).
>> But
On Thu, May 31, 2012 at 11:04:12AM -0400, Robert Haas wrote:
> On Thu, May 31, 2012 at 10:50 AM, Tom Lane wrote:
> > Robert Haas writes:
> >> On Thu, May 31, 2012 at 10:31 AM, Tom Lane wrote:
> >>> I'm not; Jeff Janes is. But you shouldn't be holding your breath
> >>> anyway, since it's 9.3 mat
On Thu, May 31, 2012 at 10:50 AM, Tom Lane wrote:
> Robert Haas writes:
>> On Thu, May 31, 2012 at 10:31 AM, Tom Lane wrote:
>>> I'm not; Jeff Janes is. But you shouldn't be holding your breath
>>> anyway, since it's 9.3 material at this point.
>
>> I agree we can't back-patch that change, but
On Thu, May 31, 2012 at 11:50 AM, Tom Lane wrote:
> The performance patches we applied to pg_dump over the past couple weeks
> were meant to relieve pain in situations where the big server-side
> lossage wasn't the dominant factor in runtime (ie, partial dumps).
> But this one is targeting exactly
On Thu, May 31, 2012 at 10:50:51AM -0400, Tom Lane wrote:
> Robert Haas writes:
> > On Thu, May 31, 2012 at 10:31 AM, Tom Lane wrote:
> >> I'm not; Jeff Janes is. �But you shouldn't be holding your breath
> >> anyway, since it's 9.3 material at this point.
>
> > I agree we can't back-patch that
Robert Haas writes:
> On Thu, May 31, 2012 at 10:31 AM, Tom Lane wrote:
>> I'm not; Jeff Janes is. But you shouldn't be holding your breath
>> anyway, since it's 9.3 material at this point.
> I agree we can't back-patch that change, but then I think we ought to
> consider back-patching some var
On Thu, May 31, 2012 at 10:31 AM, Tom Lane wrote:
> Claudio Freire writes:
>> It's not clear whether Tom is already working on that O(N^2) fix in locking.
>
> I'm not; Jeff Janes is. But you shouldn't be holding your breath
> anyway, since it's 9.3 material at this point.
I agree we can't back-
Claudio Freire writes:
> It's not clear whether Tom is already working on that O(N^2) fix in locking.
I'm not; Jeff Janes is. But you shouldn't be holding your breath
anyway, since it's 9.3 material at this point.
regards, tom lane
--
Sent via pgsql-hackers mailing lis
On Thu, May 31, 2012 at 11:17 AM, Robert Klemme
wrote:
>
> OK, my fault was to assume you wanted to measure only your part, while
> apparently you meant overall savings. But Tom had asked for separate
> measurements if I understood him correctly. Also, that measurement of
> your change would go
On Thu, May 31, 2012 at 4:07 PM, Tatsuo Ishii wrote:
>> On Thu, May 31, 2012 at 10:45 AM, Tatsuo Ishii wrote:
>>> Just for record, I rerun the test again with my single-LOCK patch, and
>>> now total runtime of pg_dump is 113 minutes.
>>> 188 minutes(9.0)->125 minutes(git master)->113 minutes(with
> On Thu, May 31, 2012 at 10:45 AM, Tatsuo Ishii wrote:
>> Just for record, I rerun the test again with my single-LOCK patch, and
>> now total runtime of pg_dump is 113 minutes.
>> 188 minutes(9.0)->125 minutes(git master)->113 minutes(with my patch).
>>
>> So far, I'm glad to see 40% time savings
On Thu, May 31, 2012 at 10:45 AM, Tatsuo Ishii wrote:
> Just for record, I rerun the test again with my single-LOCK patch, and
> now total runtime of pg_dump is 113 minutes.
> 188 minutes(9.0)->125 minutes(git master)->113 minutes(with my patch).
>
> So far, I'm glad to see 40% time savings at thi
>>> We recently fixed a couple of O(N^2) loops in pg_dump, but those covered
>>> extremely specific cases that might or might not have anything to do
>>> with what you're seeing. The complainant was extremely helpful about
>>> tracking down the problems:
>>> http://archives.postgresql.org/pgsql-ge
> I'm not excited by this patch. It dodges the O(N^2) lock behavior for
> the initial phase of acquiring the locks, but it does nothing for the
> lock-related slowdown occurring in all pg_dump's subsequent commands.
> I think we really need to get in the server-side fix that Jeff Janes is
> workin
Stephen Frost writes:
> The current situation where the client-to-server latency accounts for
> multiple minutes of time is just ridiculous, however, so I feel we need
> some form of this patch, even if the server side is magically made much
> faster. The constant back-and-forth isn't cheap.
No,
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Tatsuo Ishii writes:
> > Shall I commit to master and all supported branches?
>
> I'm not excited by this patch. It dodges the O(N^2) lock behavior for
> the initial phase of acquiring the locks, but it does nothing for the
> lock-related slowdown occurri
Tatsuo Ishii writes:
>> Ok, I modified the part of pg_dump where tremendous number of LOCK
>> TABLE are issued. I replace them with single LOCK TABLE with multiple
>> tables. With 100k tables LOCK statements took 13 minutes in total, now
>> it only takes 3 seconds. Comments?
> Shall I commit to m
>> Yeah, Jeff's experiments indicated that the remaining bottleneck is lock
>> management in the server. What I fixed so far on the pg_dump side
>> should be enough to let partial dumps run at reasonable speed even if
>> the whole database contains many tables. But if psql is taking
>> AccessShar
26 matches
Mail list logo
