Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Achilleas Mantzios]

On 10/10/19 11:31 π.μ., Wim Bertels wrote:



3) Were there ever any problems with BSD?

as far as i understand BSD and variants are very solid,
so good for server use, not for desktop


Desktop software might be a little old, but that does not mean that this 
is not usable.


I run FreeBSD desktop for ages.

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Stephen Frost]

Greetings,

* Pól Ua Laoínecháin (lineh...@tcd.ie) wrote:
> > > 1) Is my lecturer full of it or does he really have a point?
> 
> > He's full of it, as far as I can tell anyway, based on what you've
> > shared with us.  Just look at the committers and the commit history to
> > PostgreSQL, and look at who the largest contributors are and who they
> > work for.  That alone might be enough to surprise your lecturer with.
> 
> The only non-PostgreSQL company that I could find was Fujitisu - where
> can I find a (list of) the others?

Not sure where you were looking...  The contributors list is here:

https://www.postgresql.org/community/contributors/

The committers list is here:

https://wiki.postgresql.org/wiki/Committers

The git tree is here:

https://git.postgresql.org/gitweb/?p=postgresql.git;a=summary

Perhaps not the best stat, but you can view the contributions by
committer pretty easily, for 2018, here:

https://github.com/postgres/postgres/graphs/contributors?from=2018-01-01&to=2018-12-31&type=c

Note that this isn't very representative of the actual authors though-
we don't track those in the way git would prefer, instead we note who
the author of a given patch was in the commit message itself.

> > Databases that do direct I/O don't depend on fsync.  That said, I do
> > think this could have been an issue for Oracle if you ran it without
> > direct i/o.
> 
> I think that Oracle are big into asyncio? I know that you have to sudo
> dnf install some_library with a name like asio/asyncio or something
> like that?

Oracle supports both, but running with direct i/o is pretty popular,
yes.

> Anyway, why doesn't PostgreSQL use Direct I/O?

There's an awful lot that the kernel provides when it comes to things
like good read-ahead and dealing with disks and SSDs and such that we
(currently, at least) prefer to leverage instead of writing lots of new
code to deal with that ourselves, which would be required to use Direct
I/O (and not have it be completely terrible performance wise, anyway).

The whole issue behind fsync was because our expectation (and POSIX's,
if you ask me anyway) was different from what the Linux kernel was
providing (specifically, you could end up in a situation where an
fsync() call "worked" and didn't return an error, even though there
remained pages that were dirty and not written out).  Now, this is under
other error conditions typically and you'll get messages in the kernel
log about such failures usually, so if you're properly monitoring and
managing your systems there's a good chance you would have realized
there was a problem even though the Linux kernel was telling PG that
everything was fine (have backups!!).

Thanks,

Stephen


signature.asc
Description: PGP signature

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Simon Riggs]

On Wed, 9 Oct 2019 at 20:06, Pól Ua Laoínecháin  wrote:


> One of my courses is "Advanced Databases" - yummy I thought - it's not
> even compulsory for me but I just *_had_* to take this module. The
> lecturer is a bit of an Oracle fan-boy (ACE director no less...
> hmmm...) and I want(ed) - becoming less enthusiasic by the minute - to
> do my dissertation with him.


It's a good thing that you have the opportunity to do that course and to
have an lecturer with strong real-world experience.


> So, we're having a chat and I make plain
> my love of good 'ol PostgreSQL as my RDBMS of choice and he tells me
> that there are problems with random block corruption with PostgreSQL.
> I said "really" and before that conversation could go any further,
> another student came over and asked a question.
>

I think its true that there have been reported problems with block
corruption with both Oracle and PostgreSQL. The main difference is that the
PostgreSQL project is open enough for people to see much of that on public
record.

Given the efforts made on resilience and recovery, such as PITR, block
checksums, those problems are pretty much solved, based upon a
statistically sufficient sample of the real world: 2ndQuadrant customers.
Some block-level problems do still recur - the recent fsync() problems were
reported by us and have been handled (not resolved), but they were and are
rare. There are still many issues of other kinds.

Many Oracle customers I have worked with years ago experienced block
corruptions and it was very common to hear the reply "talk to your disk
vendor". Those happened probably 20-30 years earlier, so in many cases have
now been forgotten. There isn't an easy way to go back and check for trends
on that.

Having said all of that, its easy to point at some of these things and use
them as FUD - fear, uncertainty and doubt. No doubt unintentionally.

I'd go and learn more from your lecturer. Your disagreement has made you
think, so he helped you. Learning from your own mistakes takes longer.

-- 
Simon Riggshttp://www.2ndQuadrant.com/

PostgreSQL Solutions for the Enterprise

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Geoff Winkless]

On Thu, 10 Oct 2019 at 09:31, Wim Bertels  wrote:
> sometimes people are really a fan of certain product,
> sometimes in combination with the thought that all other products are
> bad; i don't know if this is the case, you could compare it with
> soccer, a barcalona fan will never become a real madrid fan and vice
> versa; so "rational" decisions (at first, based on some reasoning) tend
> to get loaded emotional feelings.

Yeah, this. Bear in mind it's possible that having made a sweeping
statement that he cannot back up and that he secretly knows was
unfounded, your lecturer will be defensive and uncomfortable. Chances
are after your conversation he will have gone away and done the same
research you did and may well have modified his opinion but will be
too embarrassed to admit that to you.

Geoff

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Wim Bertels]

> 
> Now, I have four questions:
> 
> 1) Is my lecturer full of it or does he really have a point?

Hallo Pol,

i don't know, a also teaching a databases,
personally i never experienced this

sometimes people are really a fan of certain product,
sometimes in combination with the thought that all other products are
bad; i don't know if this is the case, you could compare it with
soccer, a barcalona fan will never become a real madrid fan and vice
versa; so "rational" decisions (at first, based on some reasoning) tend
to get loaded emotional feelings.

in these kind of discussions both parties should have there say,
not just one (in this case you, we haven't heard the teacher) 

there is no such thing as a perfect dbms,
nor postgres, nor oracle, nor ..,

> 
> 2) The actual concrete acknowledged problem with fsync that affected
> PostgreSQL - why didn't it affect Oracle? Or MySQL? Or did it but it
> was so rare that it never became apparent - it wasn't that obvious
> with PostgreSQL either - one of those rare and intermittent problems?

you can set fsync to off (not default), for more performance,
but it comes with the cost of D in ACID, you no longer have it 

> 
> 3) Were there ever any problems with BSD?

as far as i understand BSD and variants are very solid,
so good for server use, not for desktop 

> 
> 4) What is the OS of choice for *_serious_* PostgreSQL installations?

it depends,

if transparancy is important to you, choose an opensource os
if time is important to you, choose what you already know
if you are willing to spent time on it, i would personally choose a
linux, bsd or solaris based os

if it helps:
i my case, i had to make a decision about the dbms for the classes as
well, the reasons i have choosen postgres are in a nutshell:
* free
* open
* runs good on servers that are comparable with an average desktop pc
or better
* close to ISO sql standard (the reason why i didn't choose mysql/now
mariadb)
* seems to have a future
-* within all these, postgres seems to have implemented most features
* after using it for a while (18 years now), i should now add: a great
community

some links:
https://www.top500.org/
https://en.wikipedia.org/wiki/DB-Engines_ranking
https://db-engines.com/en/ranking

> 
> I hope that I have been clear, but should anyone require any
> clarification, please don't hesitate to ask me.
> 
> Tia and rgs,
> 
> Pól...
> 
> 
-- 
mvg,
Wim Bertels
--
Lector
UC Leuven-Limburg
--
The human race has one really effective weapon, and that is laughter.
-- Mark Twain

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Pól Ua Laoínecháin]

Hi, and thanks for responding,

> First off- please try to craft a new email in the future...

My apologies to you and the group - I'll do that in future.

> > 1) Is my lecturer full of it or does he really have a point?

> He's full of it, as far as I can tell anyway, based on what you've
> shared with us.  Just look at the committers and the commit history to
> PostgreSQL, and look at who the largest contributors are and who they
> work for.  That alone might be enough to surprise your lecturer with.

The only non-PostgreSQL company that I could find was Fujitisu - where
can I find a (list of) the others?

> Databases that do direct I/O don't depend on fsync.  That said, I do
> think this could have been an issue for Oracle if you ran it without
> direct i/o.

I think that Oracle are big into asyncio? I know that you have to sudo
dnf install some_library with a name like asio/asyncio or something
like that?

Anyway, why doesn't PostgreSQL use Direct I/O?

Thanks again and rgs,

Pól...

> Stephen

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Tim Clarke]

On 09/10/2019 20:45, Alan Hodgson wrote:
> Assuming you're not a troll ...
>
> On Wed, 2019-10-09 at 20:06 +0100, Pól Ua Laoínecháin wrote:
>> 1) Is my lecturer full of it or does he really have a point?
> He's more than full of it. PostgreSQL has had a few bugs over the year
> that could have resulted in data corruption, but they're pretty rare
> and fixed as soon as they're found. PostgreSQL is the most reliable
> software I run, and virtually the only major piece I don't hesitate to
> upgrade without waiting to see what bugs other people find first.
>> 4) What is the OS of choice for *_serious_* PostgreSQL installations?
> That's a religious question, not a technical question. I think even
> Microsoft makes a decent server OS nowadays. But I expect a large
> majority of PostgreSQL installations are running on Linux, as are the
> vast majority of all server apps nowadays. Having said that, I don't
> run a "serious" PostgreSQL installation; some of the people here run
> databases that do tens of thousands of TPS and hold many TiB of data.
> You'd have to ask them I guess.


Pretty sure if you run through Oracle's back catalogue you'll find a
similar data corruption for an entirely unrelated reason. Just guessing
but chances are


Tim Clarke



Telephone: Witham: +44(0)1376 503500 | London: +44 (0)20 3009 0853 | Frankfurt: 
+49 (0)69 7191 6000 | Hong Kong: +852 5803 1687 | Toronto: +1 647 503 2848
Web: https://www.manifest.co.uk/



Minerva Analytics Ltd - A Solactive Company
9 Freebournes Court | Newland Street | Witham | Essex | CM8 2BL | United Kingdom



Copyright: This e-mail may contain confidential or legally privileged 
information. If you are not the named addressee you must not use or disclose 
such information, instead please report it to 
ad...@minerva.info
Legal: Minerva Analytics is the trading name of: Minerva Analytics Ltd: 
Registered in England Number 11260966 & The Manifest Voting Agency Ltd: 
Registered in England Number 2920820 Registered Office at above address. Please 
Click Here https://www.manifest.co.uk/legal/ for further information.

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Alan Hodgson]

Assuming you're not a troll ...

On Wed, 2019-10-09 at 20:06 +0100, Pól Ua Laoínecháin wrote:
> 1) Is my lecturer full of it or does he really have a point?
> 

He's more than full of it. PostgreSQL has had a few bugs over the year
that could have resulted in data corruption, but they're pretty rare and
fixed as soon as they're found. PostgreSQL is the most reliable software
I run, and virtually the only major piece I don't hesitate to upgrade
without waiting to see what bugs other people find first.

> 
> 4) What is the OS of choice for *_serious_* PostgreSQL installations?

That's a religious question, not a technical question. I think even
Microsoft makes a decent server OS nowadays. But I expect a large
majority of PostgreSQL installations are running on Linux, as are the
vast majority of all server apps nowadays. Having said that, I don't run
a "serious" PostgreSQL installation; some of the people here run
databases that do tens of thousands of TPS and hold many TiB of data.
You'd have to ask them I guess.

Re: Is my lecturer wrong about PostgreSQL? I think he is!

[Stephen Frost]

Greetings,

First off- please try to craft a new email in the future rather than
respond to an existing one.  You may not realize this but there's some
headers that get copied when you do a reply that cause the email to show
up as being a reply, even if you remove all the "obvious" bits from it.

* Pól Ua Laoínecháin (lineh...@tcd.ie) wrote:
> 1) Is my lecturer full of it or does he really have a point?

He's full of it, as far as I can tell anyway, based on what you've
shared with us.  Just look at the committers and the commit history to
PostgreSQL, and look at who the largest contributors are and who they
work for.  That alone might be enough to surprise your lecturer with.

> 2) The actual concrete acknowledged problem with fsync that affected
> PostgreSQL - why didn't it affect Oracle? Or MySQL? Or did it but it
> was so rare that it never became apparent - it wasn't that obvious
> with PostgreSQL either - one of those rare and intermittent problems?

Databases that do direct I/O don't depend on fsync.  That said, I do
think this could have been an issue for Oracle if you ran it without
direct i/o.

> 3) Were there ever any problems with BSD?

As I understand it, no.

> 4) What is the OS of choice for *_serious_* PostgreSQL installations?

BSD and Linux are both quite popular platforms for running PG, and
people run very serious workloads on both.

Thanks,

Stephen


signature.asc
Description: PGP signature