On 07/17/17 11:29, Michael Paquier wrote:
> FWIW, I would rather see any optimization done in
> AdvanceXLInsertBuffer() instead of seeing a second memset re-zeroing
> the WAL page header after its data has been initialized by
> AdvanceXLInsertBuffer() once.
Is that an aesthetic 'rather', or is th
On Thu, Jul 6, 2017 at 3:48 PM, Heikki Linnakangas wrote:
> On 07/03/2017 06:30 PM, Chapman Flack wrote:
>> Although it's moot in the straightforward approach of re-zeroing in
>> the loop, it would still help my understanding of the system to know
>> if there is some subtlety that would have broke
On Fri, Jun 23, 2017 at 6:08 AM, Chapman Flack wrote:
> Well, gzip was doing pretty well; it could get a 16 MB segment file down
> to under 27 kB, or less than 14 bytes for each of 2000 pages, when a page
> header is what, 20 bytes, it looks like? I'm not sure how much better
> I'd expect a (non-c
On 07/03/2017 06:30 PM, Chapman Flack wrote:
On 07/03/2017 09:39 AM, Heikki Linnakangas wrote:
Hmm. That's not the problem, though. Imagine that instead of the loop
above, you do just:
WALInsertLockUpdateInsertingAt(CurrPos);
AdvanceXLInsertBuffer(EndPos, false);
AdvanceXLInsertBuffer() will
On 07/03/2017 09:39 AM, Heikki Linnakangas wrote:
> Hmm. That's not the problem, though. Imagine that instead of the loop
> above, you do just:
>
> WALInsertLockUpdateInsertingAt(CurrPos);
> AdvanceXLInsertBuffer(EndPos, false);
>
> AdvanceXLInsertBuffer() will call XLogWrite(), to flush out any
On 07/03/2017 09:39 AM, Heikki Linnakangas wrote:
> The most straightforward solution would be to just clear each page with
> memset() in the loop. It's a bit wasteful to clear the page again, just
> after AdvanceXLInsertBuffer() has initialized it, but this isn't
> performance-critical.
An in th
On 06/26/2017 04:20 AM, Chapman Flack wrote:
I notice CopyXLogRecordToWAL contains this loop (in the case where
the record being copied is a switch):
while (CurrPos < EndPos)
{
/* initialize the next page (if not initialized already) */
WALInsertLockUpdateInsertingAt(CurrPos);
Advanc
On 06/25/17 21:20, Chapman Flack wrote:
> I want to make sure I understand what the deadlock potential is
> in this case. AdvanceXLInsertBuffer will call WaitXLogInsertionsToFinish
> ...
> Does not the fact we hold all the insertion slots exclude the possibility
> that any dirty buffer (preceding t
I notice CopyXLogRecordToWAL contains this loop (in the case where
the record being copied is a switch):
while (CurrPos < EndPos)
{
/* initialize the next page (if not initialized already) */
WALInsertLockUpdateInsertingAt(CurrPos);
AdvanceXLInsertBuffer(CurrPos, false);
CurrPos +=
On 06/21/17 04:51, Heikki Linnakangas wrote:
> (I'm cleaning up my inbox, hence the delayed reply)
I had almost completely forgotten ever bringing it up. :)
> When I wrote that code, I don't remember if I realized that we're
> initializing the page headers, or if I thought that it's good enough e
(I'm cleaning up my inbox, hence the delayed reply)
On 08/02/2016 10:51 PM, Robert Haas wrote:
On Tue, Aug 2, 2016 at 2:33 PM, Bruce Momjian wrote:
On Tue, Jul 26, 2016 at 05:42:43PM -0400, Chapman Flack wrote:
Even so, I'd be curious whether it would break anything to have
xlp_pageaddr simpl
On Tue, Aug 2, 2016 at 2:33 PM, Bruce Momjian wrote:
> On Tue, Jul 26, 2016 at 05:42:43PM -0400, Chapman Flack wrote:
>> Even so, I'd be curious whether it would break anything to have
>> xlp_pageaddr simply set to InvalidXLogRecPtr in the dummy zero
>> pages written to fill out a segment. At leas
On 08/02/2016 02:33 PM, Bruce Momjian wrote:
> My guess is that the bytes are there to detect problems where
> a 512-byte disk sector is zeroed by a disk failure.
Does that seem plausible? (a) there is only one such header for
every 16 512-byte disk sectors, so it only affords a 6% chance of
dete
On Tue, Jul 26, 2016 at 05:42:43PM -0400, Chapman Flack wrote:
> Even so, I'd be curious whether it would break anything to have
> xlp_pageaddr simply set to InvalidXLogRecPtr in the dummy zero
> pages written to fill out a segment. At least until it's felt
> that archive_timeout has been so decide
On 07/26/16 20:01, Michael Paquier wrote:
> On Tue, Jul 26, 2016 at 9:48 PM, Amit Kapila wrote:
>> Does any body else see the use case
>> reported by Chapman important enough that we try to have some solution
>> for it in-core?
>
> The lack of updates in the pg_lesslog project is a sign that it i
On Tue, Jul 26, 2016 at 9:48 PM, Amit Kapila wrote:
> Does any body else see the use case
> reported by Chapman important enough that we try to have some solution
> for it in-core?
The lack of updates in the pg_lesslog project is a sign that it is not
that much used. I does not seem a good idea t
On 07/26/2016 04:21 PM, Robert Haas wrote:
> I'm kind of curious WHY you are using archiving and forcing regular
> segment switches rather than just using streaming replication.
> ... AFAIK, streaming replication
> essentially obsoleted that use case. You can just dribble the
> individual bytes o
On Fri, Jul 22, 2016 at 6:02 PM, Chapman Flack wrote:
> At $work, we have a usually-low-activity PG database, so that almost
> always the used fraction of each 16 MB WAL segment is far smaller
> than 16 MB, and so it's a big win for archived-WAL storage space
> if an archive-command can be written
On 07/26/2016 08:48 AM, Amit Kapila wrote:
> general, if you have a very low WAL activity, then the final size of
> compressed WAL shouldn't be much even if you use gzip. It seems your
9.5 pg_xlog, low activity test cluster (segment switches forced
only by checkpoint timeouts), compressed with g
On Tue, Jul 26, 2016 at 9:02 AM, Chapman Flack wrote:
> On 07/25/16 22:09, Michael Paquier wrote:
>
>> This is over-complicating things for little gain. The new behavior of
>> filling in with zeros the tail of a segment makes things far better
>> when using gzip in archive_command.
>
> Then how ab
On 07/25/16 22:09, Michael Paquier wrote:
> This is over-complicating things for little gain. The new behavior of
> filling in with zeros the tail of a segment makes things far better
> when using gzip in archive_command.
Then how about filling with actual zeros, instead of with mostly-zeros
as i
On Mon, Jul 25, 2016 at 11:21 PM, Chapman Flack wrote:
> The impression that leaves is of tools that relied too heavily
> on internal format knowledge to be viable outside of core, which
> have had at least periods of incompatibility with newer PG versions,
> and whose current status, if indeed an
On 07/23/2016 08:25 AM, Amit Kapila wrote:
> On Sat, Jul 23, 2016 at 3:32 AM, Chapman Flack wrote:
>>
>> Would it then be possible to go back to the old behavior (or make
>> it selectable) of not overwriting the full 16 MB every time?
>>
>
> I don't see going back to old behaviour is an improveme
On Sat, Jul 23, 2016 at 3:32 AM, Chapman Flack wrote:
>
> Would it then be possible to go back to the old behavior (or make
> it selectable) of not overwriting the full 16 MB every time?
>
I don't see going back to old behaviour is an improvement, because as
as you pointed out above that it helps
Teaser: change made in 9.4 to simplify WAL segment compression
made it easier to compress a low-activity-period WAL segment
from 16 MB to about 27 kB ... but much harder to do better than
that, as I was previously doing (about two orders of magnitude
better).
At $work, we have a usually-low-activi
25 matches
Mail list logo
