Hi all,
I examined the effect of block-hole compression patch.
I compared the amounts of writing of the WAL data of CVS(7/11) and
8.0.3. (The amount of the WAL data writing was measured by the number of
executions of the write() function in XLogWrite().)
And, I measured the size of the hole.
Env
> > I'd say that's an improvement worth having, especially considering that
> > it requires no net expenditure of CPU time. But the table is certainly
> > still open to discuss more complicated approaches.
>
> If it's not hard to hack in as a test, it'd be interesting to see what
> additional gai
On Sat, Jun 04, 2005 at 11:46:07AM -0400, Tom Lane wrote:
> I've completed a test run for this (it's essentially MySQL's sql-bench
> done immediately after initdb). What I get is:
>
> CVS tip of 6/1: ending WAL offset = 0/A364A780 = 2741282688 bytes written
>
> CVS tip of 6/2: ending WAL offset
Tom Lane wrote:
In the XLogInsert(), it makes two kinds of logs, "whole buffer(page)
log" and "partial buffer log", isn't it? Is it only "who buffer
log"
to generate a log with "hole"?
>
>>
>> Right.
I see.
I think, it is important to reduce the n
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
> On Mon, 6 Jun 2005, Tom Lane wrote:
>> Heikki Linnakangas <[EMAIL PROTECTED]> writes:
>>> Vacuum doesn't zero out the free space between lower and upper,
>>
>> It does now ;-)
> Oh :). Does it affect vacuum performance?
I haven't tried to measure
On Mon, 6 Jun 2005, Tom Lane wrote:
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
Vacuum doesn't zero out the free space between lower and upper,
It does now ;-)
Oh :). Does it affect vacuum performance?
How about adding a flag to XLogRecData to indicate if the space between
pd_lower and
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
> Vacuum doesn't zero out the free space between lower and upper,
It does now ;-)
> How about adding a flag to XLogRecData to indicate if the space between
> pd_lower and pd_upper is meaningful or not? The XLogInsert caller probably
> knows that. T
On Mon, 6 Jun 2005, Tom Lane wrote:
Junji TERAMOTO <[EMAIL PROTECTED]> writes:
In "whole buffer log", there is a page header that includes offset of
"hole" (lower and upper). If we use that information, we don't need
any overhead, do we?
No, because the WAL code cannot assume that all pages
Junji TERAMOTO <[EMAIL PROTECTED]> writes:
> In the XLogInsert(), it makes two kinds of logs, "whole buffer(page)
> log" and "partial buffer log", isn't it? Is it only "who buffer log"
> to generate a log with "hole"?
Right.
> Tom Lane wrote:
>> The overhead needed is only 2 bytes to show the nu
> -Original Message-
> From: Tom Lane [mailto:[EMAIL PROTECTED]
> Sent: 04 June 2005 16:46
> To: Mark Cave-Ayland (External)
> Cc: pgsql-hackers@postgresql.org
> Subject: Re: Quick-and-dirty compression for WAL backup blocks
(cut)
> I've completed a test run for this (it's essentially M
Hello all,
I am interested in how to "Compress WAL entries".
Then, I study the source now, and read this discussion.
There are some questions.
1.
In the XLogInsert(), it makes two kinds of logs, "whole buffer(page)
log" and "partial buffer log", isn't it? Is it only "
"Mark Cave-Ayland" <[EMAIL PROTECTED]> writes:
>> A run-length compressor would be reasonably quick but I think that the
>> omit-the-middle-hole approach gets most of the possible win with even
>> less work.
> I can't think that a RLE scheme would be much more expensive than a 'count
> the hole' a
On Wed, Jun 01, 2005 at 10:12:34AM -0400, Tom Lane wrote:
> "Mark Cave-Ayland" <[EMAIL PROTECTED]> writes:
> > I also noticed your comment above that mentioned that compression would be
> > less effective as the pages became more full. Would changing the loading
> > factor of database pages have an
"Mark Cave-Ayland" <[EMAIL PROTECTED]> writes:
> I also noticed your comment above that mentioned that compression would be
> less effective as the pages became more full. Would changing the loading
> factor of database pages have an effect here, as I would have thought that
> the WAL would be fsyn
> Date: Tue, 31 May 2005 16:26:20 -0400
> From: Tom Lane <[EMAIL PROTECTED]>
> To: pgsql-hackers@postgreSQL.org
> Subject: Quick-and-dirty compression for WAL backup blocks
> Message-ID: <[EMAIL PROTECTED]>
(cut)
> ... make the WAL writing logic aware of the layout
> of buffer pages --- specific
Simon Riggs <[EMAIL PROTECTED]> writes:
> Is this a change that would be backpatched as you suggested previously?
I don't think we can backpatch any of these items, since they involve
changes in the on-disk file format. I was thinking of them as CVS HEAD
changes only.
reg
On Tue, 2005-05-31 at 16:26 -0400, Tom Lane wrote:
> The TODO item that comes to mind immediately is "Compress WAL entries".
> A more concrete version of this is: examine the page to see if the
> pd_lower field is between SizeOfPageHeaderData and BLCKSZ, and if so
> whether there is a run of consec
It seems we are more or less agreed that 32-bit CRC ought to be enough
for WAL; and we also need to make a change to ensure that backup blocks
are positively linked to their parent WAL record, as I noted earlier
today. So as long as we have to mess with the WAL record format, I was
wondering what
18 matches
Mail list logo
