Re: [GENERAL] Initdb --data-checksums by default

> On Apr 22, 2016, at 3:21 AM, Stuart Bishop wrote: > > On 20 April 2016 at 14:43, Alex Ignatov wrote: >> Hello everyone! >> Today in Big Data epoch silent data corruption becoming more and more issue >> to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte >> disk bit ro

Re: [GENERAL] Initdb --data-checksums by default

On 21.04.2016 20:26, Vick Khera wrote: On Thu, Apr 21, 2016 at 9:00 AM, Alex Ignatov mailto:a.igna...@postgrespro.ru>> wrote: Ms Windows doesnt have ZFS support. AIX also doesnt. Z/OS also. Any other commercial Linux distros don't have ZFS support. Yes you can compile it and u

Re: [GENERAL] Initdb --data-checksums by default

On 20 April 2016 at 14:43, Alex Ignatov wrote: > Hello everyone! > Today in Big Data epoch silent data corruption becoming more and more issue > to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte > disk bit rot is the real issue. > I think that today checksumming data mu

Re: [GENERAL] Initdb --data-checksums by default

On Thu, Apr 21, 2016 at 9:00 AM, Alex Ignatov wrote: > Ms Windows doesnt have ZFS support. AIX also doesnt. Z/OS also. Any other > commercial Linux distros don't have ZFS support. Yes you can compile it and > use on production but... > But PG runs on the above OS, but have check sum off by defau

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 23:28, Vick Khera wrote: On Wed, Apr 20, 2016 at 3:43 AM, Alex Ignatov mailto:a.igna...@postgrespro.ru>> wrote: What do you think about defaulting --data-checksums in initdb? I think that ZFS storing my database files already does this and can correct for it using repli

Re: [GENERAL] Initdb --data-checksums by default

On Wed, Apr 20, 2016 at 3:43 AM, Alex Ignatov wrote: > What do you think about defaulting --data-checksums in initdb? > I think that ZFS storing my database files already does this and can correct for it using replicated copies, so why do I need a second layer of checksums?

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 16:58, Alvaro Herrera wrote: Alex Ignatov wrote: Hello everyone! Today in Big Data epoch silent data corruption becoming more and more issue to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte disk bit rot is the real issue. I think that today checksumming

Re: [GENERAL] Initdb --data-checksums by default

Alex Ignatov wrote: > Hello everyone! > Today in Big Data epoch silent data corruption becoming more and more issue > to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte > disk bit rot is the real issue. > I think that today checksumming data must be mandatory set by defau

Re: [GENERAL] Initdb --data-checksums by default

On 04/20/2016 02:22 AM, Alex Ignatov wrote: Why do you think that common pg-users doesn't care about their data? Also why do we have wal_level=minimal fsync=on and other stuff? Because Postgres will not run without WAL files and the setting is the least you can have. Doing so you lose the

Re: [GENERAL] Initdb --data-checksums by default

On 4/20/2016 1:00 AM, Alex Ignatov wrote: Uncorrectable read error rate ~ 10^-15- 10^-14. This error stays undetected and uncorrectable. what are your units here? disk IO is done by blocks, and the CRCs and trellis codes used by the disk drives for error detection and correction are applied

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 12:27, Andreas Joseph Krogh wrote: På onsdag 20. april 2016 kl. 11:22:33, skrev Alex Ignatov mailto:a.igna...@postgrespro.ru>>: [snip] Why do you think that common pg-users doesn't care about their data? Did I say that? Also why do we have wal_level=minimal fsync=

Re: [GENERAL] Initdb --data-checksums by default

På onsdag 20. april 2016 kl. 11:22:33, skrev Alex Ignatov < a.igna...@postgrespro.ru >: [snip] Why do you think that common pg-users doesn't care about their data?   Did I say that?   Also why do we have wal_level=minimal fsync=on and other stuff?   To make certain

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 12:10, Andreas Joseph Krogh wrote: På onsdag 20. april 2016 kl. 11:02:31, skrev Alex Ignatov mailto:a.igna...@postgrespro.ru>>: On 20.04.2016 11:40, Andreas Joseph Krogh wrote: På onsdag 20. april 2016 kl. 10:33:14, skrev Alex Ignatov mailto:a.igna...@postgrespro.ru

Re: [GENERAL] Initdb --data-checksums by default

På onsdag 20. april 2016 kl. 11:02:31, skrev Alex Ignatov < a.igna...@postgrespro.ru >:   On 20.04.2016 11:40, Andreas Joseph Krogh wrote: På onsdag 20. april 2016 kl. 10:33:14, skrev Alex Ignatov < a.igna...@postgrespro.ru >: On

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 11:40, Andreas Joseph Krogh wrote: På onsdag 20. april 2016 kl. 10:33:14, skrev Alex Ignatov mailto:a.igna...@postgrespro.ru>>: On 20.04.2016 11:29, Devrim Gündüz wrote: > Hi, > > On Wed, 2016-04-20 at 10:43 +0300, Alex Ignatov wrote: >> Today in Big Data e

Re: [GENERAL] Initdb --data-checksums by default

På onsdag 20. april 2016 kl. 10:33:14, skrev Alex Ignatov < a.igna...@postgrespro.ru >: On 20.04.2016 11:29, Devrim Gündüz wrote: > Hi, > > On Wed, 2016-04-20 at 10:43 +0300, Alex Ignatov wrote: >> Today in Big Data epoch silent data corruption becoming more

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 11:29, Devrim Gündüz wrote: Hi, On Wed, 2016-04-20 at 10:43 +0300, Alex Ignatov wrote: Today in Big Data epoch silent data corruption becoming more and more issue to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte disk bit rot is the real issue. I think

Re: [GENERAL] Initdb --data-checksums by default

Hi, On Wed, 2016-04-20 at 10:43 +0300, Alex Ignatov wrote: > Today in Big Data epoch silent data corruption becoming more and more  > issue to afraid of. With uncorrectable read error rate ~ 10^-15  on  > multiterabyte disk bit rot is the real issue. > I think that today checksumming data  must b

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 10:58, Michael Paquier wrote: On Wed, Apr 20, 2016 at 4:43 PM, Alex Ignatov wrote: Hello everyone! Today in Big Data epoch silent data corruption becoming more and more issue to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte disk bit rot is the real is

Re: [GENERAL] Initdb --data-checksums by default

On 20.04.2016 10:47, John R Pierce wrote: On 4/20/2016 12:43 AM, Alex Ignatov wrote: Today in Big Data epoch silent data corruption becoming more and more issue to afraid of. With uncorrectable read error rate ~ 10^-15on multiterabyte disk bit rot is the real issue. are not those uncorrect

Re: [GENERAL] Initdb --data-checksums by default

On Wed, Apr 20, 2016 at 4:43 PM, Alex Ignatov wrote: > Hello everyone! > Today in Big Data epoch silent data corruption becoming more and more issue > to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte > disk bit rot is the real issue. > I think that today checksumming da

Re: [GENERAL] Initdb --data-checksums by default

On 4/20/2016 12:43 AM, Alex Ignatov wrote: Today in Big Data epoch silent data corruption becoming more and more issue to afraid of. With uncorrectable read error rate ~ 10^-15on multiterabyte disk bit rot is the real issue. are not those uncorrectable errors detected by the disk hardware ? t

[GENERAL] Initdb --data-checksums by default

Hello everyone! Today in Big Data epoch silent data corruption becoming more and more issue to afraid of. With uncorrectable read error rate ~ 10^-15 on multiterabyte disk bit rot is the real issue. I think that today checksumming data must be mandatory set by default. Only if someone doesn'