> On Thursday, September 27, 2012 9:12 AM Noah Misch wrote:
> On Mon, Sep 24, 2012 at 10:57:02AM +, Amit kapila wrote:
> > Rebased version of patch based on latest code.
>
> I like the direction you're taking with this patch; the gains are
> striking,
> especially considering the isolation of
On Thursday, August 30, 2012 11:23 PM Robert Haas
[mailto:robertmh...@gmail.com] wrote:
On Fri, Aug 10, 2012 at 1:25 AM, Amit Kapila wrote:
>>> I think the property that recovery only needs to worry about each
>>> block individually is one that we want to preserve. Supporting this
>>> optimizatin
On Fri, Aug 10, 2012 at 1:25 AM, Amit Kapila wrote:
>> I think the property that recovery only needs to worry about each
>> block individually is one that we want to preserve. Supporting this
>> optimizating only when full_page_writes=off seems ugly,
>
> I think recovery needs to worry about mult
On August 27, 2012 7:00 PM Amit Kapila wrote:
On August 27, 2012 5:58 PM Heikki Linnakangas wrote:
On 27.08.2012 15:18, Amit kapila wrote:
>>> I have implemented the WAL Reduction Patch for the case of HOT Update as
>> Let's do it for HOT updates only. Simon & Robert made good arguments on
>> wh
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Monday, August 27, 2012 5:58 PM
To: Amit kapila
On 27.08.2012 15:18, Amit kapila wrote:
>> I have implemented the WAL Reduction Patch for the case of HOT Update as
pointed out by Simon and Robert. In this patch it only goe
On 27.08.2012 15:18, Amit kapila wrote:
I have implemented the WAL Reduction Patch for the case of HOT Update as
pointed out by Simon and Robert. In this patch it only goes for Optimized WAL
in case of HOT Update with other restrictions same as in previous patch.
The performance numbers for th
From: Bruce Momjian [mailto:br...@momjian.us]
Sent: Friday, August 24, 2012 2:12 AM
On Wed, Aug 22, 2012 at 07:38:33PM +0530, Amit Kapila wrote:
>> I had made sure no full_page_write happens by making checkpoint interval
and
>> checkpoints segments longer.
>>
>
>
>> Original code - 1.8GMo
On Wed, Aug 22, 2012 at 07:38:33PM +0530, Amit Kapila wrote:
> I had made sure no full_page_write happens by making checkpoint interval and
> checkpoints segments longer.
>
>
>
> Original code - 1.8GModified code - 1.1G Diff - 63% reduction, incase of
> fill factor 100.
> Original code -
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila
Sent: Wednesday, August 22, 2012 8:34 AM
From: Jesper Krogh [mailto:jes...@krogh.cc]
Sent: Wednesday, August 22, 2012 1:13 AM
On 21/08/12 16:57, Amit kapila wrote:
>>Test results:
>>>1
From: Jesper Krogh [mailto:jes...@krogh.cc]
Sent: Wednesday, August 22, 2012 1:13 AM
On 21/08/12 16:57, Amit kapila wrote:
>>Test results:
>>1. The pgbench test run for 10min.
>> 2. The test reult is for modified pgbench (such that total row size is
1800 and updated columns are of length 300)
From: Robert Haas [mailto:robertmh...@gmail.com]
Sent: Thursday, August 09, 2012 11:18 PM
On Thu, Aug 9, 2012 at 12:43 PM, Heikki Linnakangas
wrote:
>>> So suppose that the following sequence of events occurs:
>>
>>> 1. Tuple A on page 1 is updated. The new version, tuple B, is placed on
>>> pag
On Thu, Aug 9, 2012 at 12:43 PM, Heikki Linnakangas
wrote:
>> So suppose that the following sequence of events occurs:
>>
>> 1. Tuple A on page 1 is updated. The new version, tuple B, is placed on
>> page 2.
>> 2. The table is vacuumed, removing tuple A.
>> 3. Page 1 is written durably to disk.
>
On 09.08.2012 19:39, Robert Haas wrote:
On Thu, Aug 9, 2012 at 9:09 AM, Heikki Linnakangas
wrote:
I meant corruption caused by anything, like disk failure, bugs, cosmic rays,
etc. The point is that currently the WAL record contains all the information
required to reconstruct the old tuple. Wit
On Thu, Aug 9, 2012 at 9:09 AM, Heikki Linnakangas
wrote:
> I meant corruption caused by anything, like disk failure, bugs, cosmic rays,
> etc. The point is that currently the WAL record contains all the information
> required to reconstruct the old tuple. With a diff method, that's no longer
> th
From: Simon Riggs [mailto:si...@2ndquadrant.com]
Sent: Thursday, August 09, 2012 5:29 PM
On 9 August 2012 12:17, Amit Kapila wrote:
>> This optimization is to reduce the amount of WAL and definitely adding
>> anything extra will have some impact.
> Of course. The question is "How much impact?".
On 09.08.2012 15:56, Amit Kapila wrote:
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Thursday, August 09, 2012 4:59 PM
On 09.08.2012 14:11, Simon Riggs wrote:
But then again, full-page writes cover that too. There
will be a full-page image of the old block in the
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Thursday, August 09, 2012 4:59 PM
On 09.08.2012 14:11, Simon Riggs wrote:
>>> But then again, full-page writes cover that too. There
>>> will be a full-page image of the old block in the WAL anyway.
>
>> Right, but we're
On 9 August 2012 12:17, Amit Kapila wrote:
> This optimization is to reduce the amount of WAL and definitely adding
> anything extra will have some impact.
Of course. The question is "How much impact?". Each tweak has
progressively less and less gain. This isn't a binary choice.
Squeezing the l
On 09.08.2012 14:11, Simon Riggs wrote:
Given the marginal gain because of a low percentage of cross-block
updates, I'm not keen. Low percentage because HOT tries hard to keep
things on same block - even for non-HOT updates (which is the case,
even though it sounds weird).
That depends entirely
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Simon Riggs
Sent: Thursday, August 09, 2012 2:49 PM
On 9 August 2012 09:49, Amit Kapila wrote:
>>> I'd suggest we do this only when the saving is large enough for
>>> benefit, rather than do this eve
On 9 August 2012 11:30, Heikki Linnakangas
wrote:
> On 09.08.2012 12:18, Simon Riggs wrote:
>>
>> On 9 August 2012 09:49, Amit Kapila wrote:
>>
>>>WAL reduction is done for the case even when old and new are on
>>> different
>>> data blocks as well.
>>
>>
>> That makes me feel nervous. I doub
On 09.08.2012 12:18, Simon Riggs wrote:
On 9 August 2012 09:49, Amit Kapila wrote:
WAL reduction is done for the case even when old and new are on different
data blocks as well.
That makes me feel nervous. I doubt the marginal gain is worth it.
Most updates don't cross blocks.
That was
On 9 August 2012 09:49, Amit Kapila wrote:
>> I'd suggest we do this only when the saving is large enough for
>> benefit, rather than do this every time.
> Do you mean to say that when length of updated values of tuple is less
> than some threshold(1/3 or 2/3, etc..) value of
> total length?
From: Simon Riggs [mailto:si...@2ndquadrant.com]
Sent: Thursday, August 09, 2012 12:36 PM
On 3 August 2012 12:46, Amit kapila wrote:
>> Frame the new tuple from old tuple and WAL record:
> Sounds good.
Thanks.
> I'd suggest we do this only when the saving is large enough for
> benefit, rathe
On 3 August 2012 12:46, Amit kapila wrote:
> Frame the new tuple from old tuple and WAL record:
Sounds good.
I'd suggest we do this only when the saving is large enough for
benefit, rather than do this every time.
You don't mention whether or not the old and the new tuple are on the
same data
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Monday, August 06, 2012 2:32 PM
To: Amit Kapila
Cc: 'Bruce Momjian'; pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] [WIP] Performance Improvement by reducing WAL for
Update Operation
On 06.08.2012 06:10, A
On 06.08.2012 06:10, Amit Kapila wrote:
Currently the solution for fixed length columns cannot handle the case of
variable length columns and NULLS. The reason is for fixed length columns
there is no need of diff technology between old and new tuple, however for
other cases it will be required.
F
From: Bruce Momjian [mailto:br...@momjian.us]
Sent: Saturday, August 04, 2012 8:06 PM
On Sat, Aug 4, 2012 at 05:21:06PM +0300, Heikki Linnakangas wrote:
On 04.08.2012 11:01, Amit Kapila wrote:
>>Missed one point which needs to be handled is pg_upgrade
>
> I don't think there's anything to do fo
On Sat, Aug 4, 2012 at 05:21:06PM +0300, Heikki Linnakangas wrote:
> On 04.08.2012 11:01, Amit Kapila wrote:
> >Missed one point which needs to be handled is pg_upgrade
>
> I don't think there's anything to do for pg_upgrade. This doesn't
> change the on-disk data format, just the WAL format, and
On 04.08.2012 11:01, Amit Kapila wrote:
Missed one point which needs to be handled is pg_upgrade
I don't think there's anything to do for pg_upgrade. This doesn't change
the on-disk data format, just the WAL format, and pg_upgrade isn't
sensitive to WAL format changes.
--
Heikki Linnakang
gresql.org
Subject: Re: [HACKERS] [WIP] Performance Improvement by reducing WAL for
Update Operation
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Saturday, August 04, 2012 1:33 AM
On 03.08.2012 14:46, Amit kapila wrote:
>> Currently the change is done only for fixe
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Saturday, August 04, 2012 1:33 AM
On 03.08.2012 14:46, Amit kapila wrote:
>> Currently the change is done only for fixed length columns for simple
tables and the tuple should not contain NULLS.
>
>> This is a Proof of conc
On 03.08.2012 14:46, Amit kapila wrote:
Currently the change is done only for fixed length columns for simple tables
and the tuple should not contain NULLS.
This is a Proof of concept, the design and implementation needs to be changed
based on final design required for handling other scenario'
, August 03, 2012 5:17 PM
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] [WIP] Performance Improvement by reducing WAL for Update
Operation
Problem statement:
---
Reducing wal size for an update operation for performance improvement.
Advantages
34 matches
Mail list logo
