Hi,
I noticed -k/--data-checksums is currently in the less commonly used
options part of the initdb --help output:
|Less commonly used options:
| -d, --debug generate lots of debugging output
| -k, --data-checksums use data page checksums
I think enough people use data check
On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
> I think enough people use data checksums these days that it warrants to
> be moved into the "normal part", like in the attached.
+1. Let's see first what others think about this change.
--
Michael
signature.asc
Description: PGP si
Greetings,
* Michael Paquier (mich...@paquier.xyz) wrote:
> On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
> > I think enough people use data checksums these days that it warrants to
> > be moved into the "normal part", like in the attached.
>
> +1. Let's see first what others th
Heya,
(changing the subject as we're moving the goalposts)
Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
> * Michael Paquier (mich...@paquier.xyz) wrote:
> > On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
> > > I think enough people use data checksums these days t
On Thu, 2021-01-07 at 16:14 -0500, Stephen Frost wrote:
> I expected there'd be some disagreement on this, but I do continue to
> feel that it's sensible to enable checksums by default.
+1
I think the problem here (apart from the original line of argumentation)
is that there are two kinds of Post
Hi,
On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote:
> On Thu, 2021-01-07 at 16:14 -0500, Stephen Frost wrote:
> > I expected there'd be some disagreement on this, but I do continue to
> > feel that it's sensible to enable checksums by default.
>
> +1
I don't disagree with this in principle, b
On 1/8/21 5:03 AM, Andres Freund wrote:
On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote:
The serious crowd are more likely to choose a non-default setting
to avoid paying the price for a feature that they don't need.
I don't really buy this argument. That way we're going to have an ever gro
On Fri, Jan 8, 2021 at 10:53:35AM +0100, Laurenz Albe wrote:
> On Thu, 2021-01-07 at 16:14 -0500, Stephen Frost wrote:
> > I expected there'd be some disagreement on this, but I do continue to
> > feel that it's sensible to enable checksums by default.
>
> +1
>
> I think the problem here (apart
On Fri, Jan 8, 2021 at 09:46:58AM -0500, David Steele wrote:
> On 1/8/21 5:03 AM, Andres Freund wrote:
> > On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote:
> > >
> > > The serious crowd are more likely to choose a non-default setting
> > > to avoid paying the price for a feature that they don't
On Fri, Jan 8, 2021 at 4:57 PM Bruce Momjian wrote:
>
> On Fri, Jan 8, 2021 at 09:46:58AM -0500, David Steele wrote:
> > On 1/8/21 5:03 AM, Andres Freund wrote:
> > > On Fri, Jan 8, 2021, at 01:53, Laurenz Albe wrote:
> > > >
> > > > The serious crowd are more likely to choose a non-default setti
On Fri, Jan 8, 2021 at 05:06:24PM +0100, Magnus Hagander wrote:
> On Fri, Jan 8, 2021 at 4:57 PM Bruce Momjian wrote:
> > I think once we have better online enabling of checksums people can more
> > easily test the overhead on their workloads.
>
> Yeah, definitely.
>
> If they have equivalent h
On Mon, Jan 04, 2021 at 07:11:43PM +0100, Michael Banck wrote:
> Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
>> * Michael Paquier (mich...@paquier.xyz) wrote:
>>> On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
>>> > I think enough people use data checksums these d
Hi,
On 2021-01-04 19:11:43 +0100, Michael Banck wrote:
> Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
> > * Michael Paquier (mich...@paquier.xyz) wrote:
> > > On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
> > > > I think enough people use data checksums these day
Hi,
Am Mittwoch, den 06.01.2021, 10:52 +0900 schrieb Michael Paquier:
> On Mon, Jan 04, 2021 at 07:11:43PM +0100, Michael Banck wrote:
> > So maybe we should switch on wal_compression if we enable data checksums
> > by default.
>
> I don't agree with this assumption. In some CPU-bounded workload
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2021-01-04 19:11:43 +0100, Michael Banck wrote:
> > Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
> > > * Michael Paquier (mich...@paquier.xyz) wrote:
> > > > On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
On Wed, Jan 6, 2021 at 12:02:40PM -0500, Stephen Frost wrote:
> > It unfortunately also hurts other workloads. If we moved towards a saner
> > compression algorithm that'd perhaps not be an issue anymore...
>
> I agree that improving compression performance would be good but I don't
> see that as
Greetings,
* Bruce Momjian (br...@momjian.us) wrote:
> On Wed, Jan 6, 2021 at 12:02:40PM -0500, Stephen Frost wrote:
> > > It unfortunately also hurts other workloads. If we moved towards a saner
> > > compression algorithm that'd perhaps not be an issue anymore...
> >
> > I agree that improving
On Wed, Jan 6, 2021 at 8:31 AM Michael Banck wrote:
>
> Hi,
>
> Am Mittwoch, den 06.01.2021, 10:52 +0900 schrieb Michael Paquier:
> > On Mon, Jan 04, 2021 at 07:11:43PM +0100, Michael Banck wrote:
> > > So maybe we should switch on wal_compression if we enable data checksums
> > > by default.
> >
Hi,
On 2021-01-06 12:02:40 -0500, Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2021-01-04 19:11:43 +0100, Michael Banck wrote:
> > > Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
> > > > I agree with this, but I'd also like to propose, again, as has be
Hi,
On 2021-01-06 18:27:48 +0100, Magnus Hagander wrote:
> The other argument is that admins can cheaply and quickly turn off
> checksums if they don't want them.
>
> The same cannot be said for turning them *on* again, that's a very
> slow offline operation at this time.
>
> Turning off checksu
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2021-01-06 12:02:40 -0500, Stephen Frost wrote:
> > * Andres Freund (and...@anarazel.de) wrote:
> > > On 2021-01-04 19:11:43 +0100, Michael Banck wrote:
> > > > Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
> > > > > I ag
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2021-01-06 18:27:48 +0100, Magnus Hagander wrote:
> > The other argument is that admins can cheaply and quickly turn off
> > checksums if they don't want them.
> >
> > The same cannot be said for turning them *on* again, that's a very
>
On Wed, Jan 6, 2021 at 6:58 PM Andres Freund wrote:
>
> Hi,
>
> On 2021-01-06 18:27:48 +0100, Magnus Hagander wrote:
> > The other argument is that admins can cheaply and quickly turn off
> > checksums if they don't want them.
> >
> > The same cannot be said for turning them *on* again, that's a v
Hi,
On Wed, Jan 06, 2021 at 09:55:08AM -0800, Andres Freund wrote:
> On 2021-01-06 12:02:40 -0500, Stephen Frost wrote:
> > * Andres Freund (and...@anarazel.de) wrote:
> > > On 2021-01-04 19:11:43 +0100, Michael Banck wrote:
> > > > This looks much better from the WAL size perspective, there's now
Am Mittwoch, den 06.01.2021, 09:58 -0800 schrieb Andres Freund:
> It still requires running a binary locally on the DB server, no? Which
> means it'll not be an option on most cloud providers...
At least Azure and RDS seem to have data_checksums on anyway, I don't
have a GCP test instance around
Am Mittwoch, den 06.01.2021, 19:07 +0100 schrieb Michael Banck:
> Am Mittwoch, den 06.01.2021, 09:58 -0800 schrieb Andres Freund:
> > It still requires running a binary locally on the DB server, no? Which
> > means it'll not be an option on most cloud providers...
>
> At least Azure and RDS seem
Hi,
On 2021-01-06 13:01:59 -0500, Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > > imv, enabling page checksums is akin to having fsync enabled by default.
> > > Does it impact performance? Yes, surely quite a lot, but it's also the
> > > safe and sane choice when it comes
On Wed, Jan 6, 2021 at 9:55 AM Andres Freund wrote:
> Vacuum performance is one of *THE* major complaints about
> postgres. Making it run slower by a lot obviously exascerbates that
> problem significantly. I think it'd be prohibitively expensive if it
> were 1.5x, not to even speak of 15x.
+1. I
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2021-01-06 13:01:59 -0500, Stephen Frost wrote:
> > * Andres Freund (and...@anarazel.de) wrote:
> > > > imv, enabling page checksums is akin to having fsync enabled by default.
> > > > Does it impact performance? Yes, surely quite a lot,
On Wed, Jan 6, 2021 at 11:44 AM Stephen Frost wrote:
> Having fsync off won't actually cause corruption unless you have an OS
> crash or don't sync the disks when you reboot the system though- so it's
> a hedge against certain failure conditions, as is checksums.
I find this argument baffling. Do
Greetings,
* Peter Geoghegan (p...@bowt.ie) wrote:
> On Wed, Jan 6, 2021 at 11:44 AM Stephen Frost wrote:
> > Having fsync off won't actually cause corruption unless you have an OS
> > crash or don't sync the disks when you reboot the system though- so it's
> > a hedge against certain failure con
On Wed, Jan 6, 2021 at 12:03 PM Stephen Frost wrote:
> Do you really believe it to be wrong? Do we stop performing the correct
> write calls in the correct order to the kernel with fsync being off? If
> the kernel actually handles all of our write calls correctly and we
> cleanly shut down and t
Greetings,
* Peter Geoghegan (p...@bowt.ie) wrote:
> On Wed, Jan 6, 2021 at 12:03 PM Stephen Frost wrote:
> > Do you really believe it to be wrong? Do we stop performing the correct
> > write calls in the correct order to the kernel with fsync being off? If
> > the kernel actually handles all o
On Wed, Jan 6, 2021 at 12:30 PM Stephen Frost wrote:
> As already mentioned, it's also, at least today, far
> simpler to disable checksums than to enable them, which is something
> else to consider when thinking about what the default should be.
That is a valid concern. I just don't think that it
Hi,
Am Mittwoch, den 06.01.2021, 12:56 -0800 schrieb Peter Geoghegan:
> On Wed, Jan 6, 2021 at 12:30 PM Stephen Frost wrote:
> > As already mentioned, it's also, at least today, far
> > simpler to disable checksums than to enable them, which is something
> > else to consider when thinking about w
On Wed, Jan 6, 2021 at 1:04 PM Michael Banck wrote:
> At least data_checksums=on for Azure Managed Postgres, Amazon RDS and
> Google Cloud SQL Postgres. It might not be on for all types (I just
> checked the basic one each earlier today), but I guess it is.
So you tested this using "show data_che
Am Mittwoch, den 06.01.2021, 13:08 -0800 schrieb Peter Geoghegan:
> On Wed, Jan 6, 2021 at 1:04 PM Michael Banck
> wrote:
> > At least data_checksums=on for Azure Managed Postgres, Amazon RDS and
> > Google Cloud SQL Postgres. It might not be on for all types (I just
> > checked the basic one eac
On Wed, Jan 6, 2021 at 1:08 PM Peter Geoghegan wrote:
> So you tested this using "show data_checksums;" in psql in each case?
>
> What does "show full_page_writes;" show you?
Another consideration is checkpoint_timeout and max_wal_size.
--
Peter Geoghegan
Hi,
Am Mittwoch, den 06.01.2021, 13:19 -0800 schrieb Peter Geoghegan:
> On Wed, Jan 6, 2021 at 1:08 PM Peter Geoghegan wrote:
> > So you tested this using "show data_checksums;" in psql in each case?
> >
> > What does "show full_page_writes;" show you?
>
> Another consideration is checkpoint_ti
On Wed, Jan 6, 2021 at 1:29 PM Michael Banck wrote:
> That one seems to be 5min everywhere, and one can change it except on
> Azure.
Okay, thanks for clearing that up. Looks like all of the big 3 cloud
providers use Postgres checksums in a straightforward way.
I don't have much more to say on th
On Thu, Jan 7, 2021 at 3:32 AM Peter Geoghegan wrote:
>
> On Wed, Jan 6, 2021 at 1:29 PM Michael Banck
> wrote:
> > That one seems to be 5min everywhere, and one can change it except on
> > Azure.
>
> Okay, thanks for clearing that up. Looks like all of the big 3 cloud
> providers use Postgres c
Greetings,
* Peter Geoghegan (p...@bowt.ie) wrote:
> On Wed, Jan 6, 2021 at 12:30 PM Stephen Frost wrote:
> > As already mentioned, it's also, at least today, far
> > simpler to disable checksums than to enable them, which is something
> > else to consider when thinking about what the default sho
Greetings,
* Michael Banck (michael.ba...@credativ.de) wrote:
> Am Mittwoch, den 06.01.2021, 13:08 -0800 schrieb Peter Geoghegan:
> > On Wed, Jan 6, 2021 at 1:04 PM Michael Banck
> > wrote:
> > > At least data_checksums=on for Azure Managed Postgres, Amazon RDS and
> > > Google Cloud SQL Postgre
On Thu, Jan 7, 2021 at 1:14 PM Stephen Frost wrote:
> I expected there'd be some disagreement on this, but I do continue to
> feel that it's sensible to enable checksums by default. I also don't
> think there's anything particularly wrong with such a difference of
> opinion, though it likely mean
Greetings,
* Peter Geoghegan (p...@bowt.ie) wrote:
> On Thu, Jan 7, 2021 at 1:14 PM Stephen Frost wrote:
> > Much of this line of discussion seems to be, incorrectly, focused on my
> > mere mention of viewing the use of fsync and checksums as mechanism for
> > addressing certain risks, but that d
From: Stephen Frost
> I hadn't intended to make an argument that enabling checksums was
> equivilant to enabling or disabling fsync- I said it was 'akin', by which I
> meant it
> was similar in character, as in, as I said previously, a way for PG to hedge
> against certain external-to-PG risks (t
46 matches
Mail list logo
