On Fri, May 29, 2015 at 12:55 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Abhijit Menon-Sen a...@2ndquadrant.com writes:
I have to leave shortly, so I'll look at the initdb cleanup tomorrow.
Here's a revision of that patch that's more along the lines of what you
committed.
Will look at that
On Fri, May 29, 2015 at 9:25 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Speaking of which, could somebody test that the committed patch does
what it's supposed to on Windows? You were worried upthread about
whether the tests for symlinks (aka junction points) behaved correctly,
and I have no
On Fri, May 29, 2015 at 5:01 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Fri, May 29, 2015 at 9:25 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Speaking of which, could somebody test that the committed patch does
what it's supposed to on Windows? You were worried upthread about
whether the
Re: Tom Lane 2015-05-28 5740.1432849...@sss.pgh.pa.us
Abhijit Menon-Sen a...@2ndquadrant.com writes:
Here's an updated patch for the fsync problem(s).
I've committed this after some mostly-cosmetic rearrangements.
Fwiw, I can confirm that the problem is fixed for 9.5. The regression
tests
On Fri, May 29, 2015 at 2:28 PM, Michael Paquier michael.paqu...@gmail.com
wrote:
On Fri, May 29, 2015 at 5:01 PM, Amit Kapila amit.kapil...@gmail.com
wrote:
Test-3 - Symlinks in pg_tblspc.
1. Create couple of tablespaces which creates symlinks
in pg_tblspc
2. Crash the server
3.
On Fri, May 29, 2015 at 6:49 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Fri, May 29, 2015 at 2:28 PM, Michael Paquier michael.paqu...@gmail.com
wrote:
On Fri, May 29, 2015 at 5:01 PM, Amit Kapila amit.kapil...@gmail.com
wrote:
Test-3 - Symlinks in pg_tblspc.
1. Create couple of
At 2015-05-29 13:14:18 -0400, t...@sss.pgh.pa.us wrote:
Pushed with minor revisions.
Thanks, looks good.
Since we're only logging the failures anyway, I think it is reasonable
to log a complaint for any unwritable file in the data directory.
Sounds reasonable, patch attached. ETXTBSY has
On 2015-05-29 13:49:16 -0400, Tom Lane wrote:
Andres Freund and...@anarazel.de writes:
On 2015-05-29 13:14:18 -0400, Tom Lane wrote:
Abhijit Menon-Sen a...@2ndquadrant.com writes:
As I mentioned yesterday, I'm not really on board with ignoring EACCES,
except for the directories-on-Windows
* Andres Freund (and...@anarazel.de) wrote:
On 2015-05-29 13:49:16 -0400, Tom Lane wrote:
That sounds like a potentially nontrivial amount of repetitive log bleat
after every crash start? One which the user can't really stop?
Why can't the user stop it?
Because it makes a good
Abhijit Menon-Sen a...@2ndquadrant.com writes:
At 2015-05-28 17:37:16 -0400, t...@sss.pgh.pa.us wrote:
I have to leave shortly, so I'll look at the initdb cleanup tomorrow.
Here's a revision of that patch that's more along the lines of what you
committed.
Pushed with minor revisions.
It
On 2015-05-29 13:14:18 -0400, Tom Lane wrote:
Abhijit Menon-Sen a...@2ndquadrant.com writes:
As I mentioned yesterday, I'm not really on board with ignoring EACCES,
except for the directories-on-Windows case. Since we're only logging
the failures anyway, I think it is reasonable to log a
On 2015-05-29 14:15:48 -0400, Tom Lane wrote:
Andres Freund and...@anarazel.de writes:
On 2015-05-29 13:49:16 -0400, Tom Lane wrote:
Why can't the user stop it?
Because it makes a good amount of sense to have e.g. certificates not
owned by postgres and not writeable? You don't
Andres Freund and...@anarazel.de writes:
On 2015-05-29 13:14:18 -0400, Tom Lane wrote:
Abhijit Menon-Sen a...@2ndquadrant.com writes:
As I mentioned yesterday, I'm not really on board with ignoring EACCES,
except for the directories-on-Windows case. Since we're only logging
the failures
Andres Freund and...@anarazel.de writes:
On 2015-05-29 13:49:16 -0400, Tom Lane wrote:
Why can't the user stop it?
Because it makes a good amount of sense to have e.g. certificates not
owned by postgres and not writeable? You don't necessarily want to
symlink them somewhere else, because
Re: Tom Lane 2015-05-29 13871.1432921...@sss.pgh.pa.us
Why can't the user stop it? We won't be bleating about the case of a
symlink to a non-writable file someplace else, which is the Debian use
case. I don't see a very good excuse to have a non-writable file right
in the data directory.
Michael Paquier michael.paqu...@gmail.com writes:
On Fri, May 29, 2015 at 6:49 PM, Amit Kapila amit.kapil...@gmail.com wrote:
Test - 2 - Directory Symlink for pg_xlog
First 4 steps are same as Test-1
5. mklink /D E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc
6. Restart Server - Error
-
Abhijit Menon-Sen a...@2ndquadrant.com writes:
Here's an updated patch for the fsync problem(s).
I've committed this after some mostly-cosmetic rearrangements.
4. As a partial aside, why does fsync_fname use OpenTransientFile? It
looks like it should use BasicOpenFile as pre_sync_fname
At 2015-05-28 19:56:15 +0530, a...@2ndquadrant.com wrote:
I have a separate patch to initdb with the corresponding changes, which
I will post after dinner and a bit more testing.
Here's that patch too.
I was a bit unsure how far to go with this. It fixes the problem of not
following pg_xlog
On Thu, May 28, 2015 at 1:04 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Thu, May 28, 2015 at 10:26 AM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
2. Robert, are you comfortable with what fsync_pgdata() does in xlog.c?
I wasn't sure if I should move
Abhijit Menon-Sen a...@2ndquadrant.com writes:
I have to leave shortly, so I'll look at the initdb cleanup tomorrow.
Here's a revision of that patch that's more along the lines of what you
committed.
Will look at that tomorrow ...
It occurred to me that we should probably also silently skip
At 2015-05-28 17:37:16 -0400, t...@sss.pgh.pa.us wrote:
I've committed this after some mostly-cosmetic rearrangements.
Thank you.
I have to leave shortly, so I'll look at the initdb cleanup tomorrow.
Here's a revision of that patch that's more along the lines of what you
committed.
It
Hi.
Here's an updated patch for the fsync problem(s).
A few points that may be worth considering:
1. I've made the ReadDir → ReadDirExtended change, but in retrospect I
don't think it's really worth it. Using ReadDir and letting it die
is probably fine. (Aside: I left ReadDirExtended
On Thu, May 28, 2015 at 10:26 AM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
2. Robert, are you comfortable with what fsync_pgdata() does in xlog.c?
I wasn't sure if I should move that to fd.c as well. I think it's
borderline OK for now.
I think if the function is specific as
Robert Haas robertmh...@gmail.com writes:
On Thu, May 28, 2015 at 10:26 AM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
2. Robert, are you comfortable with what fsync_pgdata() does in xlog.c?
I wasn't sure if I should move that to fd.c as well. I think it's
borderline OK for now.
I think
At 2015-05-27 11:46:39 +0530, a...@2ndquadrant.com wrote:
I'm trying a couple of approaches to that (e.g. using readdir directly
instead of ReadDir), but other suggestions are welcome.
Here's what that looks like, but not yet fully tested.
-- Abhijit
diff --git
Abhijit Menon-Sen a...@2ndquadrant.com writes:
At 2015-05-27 20:22:18 -0400, t...@sss.pgh.pa.us wrote:
I doubt that that (not using AllocateDir) [â¦]
(Disregarding per your followup.)
What I think is a reasonable compromise is to treat AllocateDir
failure as nonfatal, but to continue using
Abhijit Menon-Sen a...@2ndquadrant.com writes:
At 2015-05-27 11:46:39 +0530, a...@2ndquadrant.com wrote:
I'm trying a couple of approaches to that (e.g. using readdir directly
instead of ReadDir), but other suggestions are welcome.
Here's what that looks like, but not yet fully tested.
I
I wrote:
Abhijit Menon-Sen a...@2ndquadrant.com writes:
At 2015-05-27 11:46:39 +0530, a...@2ndquadrant.com wrote:
I'm trying a couple of approaches to that (e.g. using readdir directly
instead of ReadDir), but other suggestions are welcome.
Here's what that looks like, but not yet fully
At 2015-05-27 20:22:18 -0400, t...@sss.pgh.pa.us wrote:
I doubt that that (not using AllocateDir) […]
(Disregarding per your followup.)
What I think is a reasonable compromise is to treat AllocateDir
failure as nonfatal, but to continue using ReadDir etc which means
that any post-open
At 2015-05-27 23:41:29 -0400, t...@sss.pgh.pa.us wrote:
What about turning ReadDir into a wrapper around a ReadDirExtended
function that adds an elevel argument?
Sounds reasonable, doing.
-- Abhijit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to
At 2015-05-26 19:07:20 +0200, and...@anarazel.de wrote:
Abhijit, do you recall why the code was changed to follow all symlinks
in contrast to explicitly going through the tablespaces as initdb -S
does? I'm pretty sure early versions of the patch pretty much had a
verbatim copy of the initdb
At 2015-05-26 22:44:03 +0200, and...@anarazel.de wrote:
So, this was discussed in the following thread, starting at:
http://archives.postgresql.org/message-id/20150403163232.GA28444%40eldon.alvh.no-ip.org
Sorry, I didn't see this before replying.
There are no other places we it's allowed to
* Abhijit Menon-Sen (a...@2ndquadrant.com) wrote:
At 2015-05-26 03:54:51 +0200, and...@anarazel.de wrote:
Another thing is whether we should handle a recursive symlink in
pgdata? I personally think not, but...
I think not too.
Yikes.. That's definitely the kind of thing that's why I
On Mon, May 25, 2015 at 9:54 PM, Andres Freund and...@anarazel.de wrote:
On 2015-05-25 21:33:03 -0400, Robert Haas wrote:
On Mon, May 25, 2015 at 2:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Perhaps, but if we didn't have permission to write the file, it's hard to
argue that it's our
On Mon, May 25, 2015 at 9:54 PM, Stephen Frost sfr...@snowman.net wrote:
I certainly see your point, but Tom also pointed out that it's not great
to ignore failures during this phase:
* Tom Lane (t...@sss.pgh.pa.us) wrote:
Greg Stark st...@mit.edu writes:
What exactly is failing?
Is it
On 05/26/2015 08:05 AM, Robert Haas wrote:
On Mon, May 25, 2015 at 9:54 PM, Stephen Frost sfr...@snowman.net wrote:
I certainly see your point, but Tom also pointed out that it's not great
to ignore failures during this phase:
* Tom Lane (t...@sss.pgh.pa.us) wrote:
Greg Stark st...@mit.edu
On 05/26/2015 11:58 AM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
OK, I'm late to the party. But why exactly are we syncing absolutely
everything? That seems over-broad.
If we try to be selective, we risk errors of omission, which no one would
ever notice until someone's data
Andrew Dunstan and...@dunslane.net writes:
On 05/26/2015 11:58 AM, Tom Lane wrote:
One thing perhaps we *should* be selective about, though, is which
symlinks we try to follow. I think that a good case could be made
for ignoring symlinks everywhere except in the pg_tablespace directory.
If
Andrew Dunstan and...@dunslane.net writes:
OK, I'm late to the party. But why exactly are we syncing absolutely
everything? That seems over-broad.
If we try to be selective, we risk errors of omission, which no one would
ever notice until someone's data got eaten in a low-probability crash
Robert Haas robertmh...@gmail.com writes:
Anything we do short of making all errors in this area non-fatal is
going to leave behind startup-failure cases that exist today, and we
have no evidence at this time that such startup failures would be
justified by any actual data loss risk.
Yeah.
On 2015-05-26 19:07:20 +0200, Andres Freund wrote:
It is somewhat interesting that similar code has been used in
pg_upgrade, via initdb -S, for a while now, without, to my knowledge, it
causing reported problem. I think the relevant difference is that that
code doesn't follow symlinks. It's
On 2015-05-26 10:41:12 -0400, Tom Lane wrote:
Yeah. Perhaps I missed it, but was the original patch motivated by
actual failures that had been seen in the field, or was it just a
hypothetical concern?
I'd mentioned that it might be a good idea to do this while
investingating a bug with
On Tue, May 26, 2015 at 6:16 PM, Andrew Dunstan and...@dunslane.net wrote:
On 05/26/2015 11:58 AM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
OK, I'm late to the party. But why exactly are we syncing absolutely
everything? That seems over-broad.
If we try to be selective,
On 2015-05-26 19:07:20 +0200, Andres Freund wrote:
Abhijit, do you recall why the code was changed to follow all symlinks
in contrast to explicitly going through the tablespaces as initdb -S
does? I'm pretty sure early versions of the patch pretty much had a
verbatim copy of the initdb logic?
On May 24, 2015 7:52:53 AM PDT, Tom Lane t...@sss.pgh.pa.us wrote:
Christoph Berg m...@debian.org writes:
Re: To Andres Freund 2015-05-24 20150524075244.gb27...@msg.df7cb.de
Re: Andres Freund 2015-05-24
20150524005245.gd32...@alap3.anarazel.de
How about, to avoid masking actual problems, we
* Andres Freund (and...@anarazel.de) wrote:
On 2015-05-25 13:38:01 -0400, Tom Lane wrote:
Andres Freund and...@anarazel.de writes:
On May 24, 2015 7:52:53 AM PDT, Tom Lane t...@sss.pgh.pa.us wrote:
If we'd merge it with initdb's list I think I'd not be that bad. I'm
thinking of some
On 2015-05-25 13:38:01 -0400, Tom Lane wrote:
Andres Freund and...@anarazel.de writes:
On May 24, 2015 7:52:53 AM PDT, Tom Lane t...@sss.pgh.pa.us wrote:
If we'd merge it with initdb's list I think I'd not be that bad. I'm
thinking of some header declaring it, roughly like the rmgr list.
Alvaro Herrera alvhe...@2ndquadrant.com writes:
Tom Lane wrote:
Well, that opens us to errors of omission, ie failing to fsync things we
should have. Maybe that's an okay risk, but personally I'd judge that
fsync everything and ignore (some?) errors is probably a more robust
approach over
* Andres Freund (and...@anarazel.de) wrote:
On 2015-05-25 14:02:28 -0400, Tom Lane wrote:
Stephen Frost sfr...@snowman.net writes:
I've not followed this thread all that closely, but I do tend to agree
with the idea of only try to mess with files that are *clearly* ours to
mess with.
Andres Freund and...@anarazel.de writes:
On May 24, 2015 7:52:53 AM PDT, Tom Lane t...@sss.pgh.pa.us wrote:
Christoph Berg m...@debian.org writes:
pg_log/ is also admin domain. What about only recursing into
well-known directories + postgresql.auto.conf?
The idea that this code would know
On 2015-05-25 14:02:28 -0400, Tom Lane wrote:
Stephen Frost sfr...@snowman.net writes:
I've not followed this thread all that closely, but I do tend to agree
with the idea of only try to mess with files that are *clearly* ours to
mess with.
Well, that opens us to errors of omission, ie
Tom Lane wrote:
Stephen Frost sfr...@snowman.net writes:
I've not followed this thread all that closely, but I do tend to agree
with the idea of only try to mess with files that are *clearly* ours to
mess with.
Well, that opens us to errors of omission, ie failing to fsync things we
* Andres Freund (and...@anarazel.de) wrote:
On 2015-05-25 14:14:10 -0400, Stephen Frost wrote:
That seems overly complicated, for my 2c at least. I don't particularly
like trying to mess with files that might be rightfully considered not
ours either.
I'd not consider an fsync to be
On 05/25/2015 01:21 PM, Tom Lane wrote:
And from the other direction, where exactly is it written that
distros/users will only create problematic files at the top level of
$PGDATA? I'd have zero confidence in such an assertion applied to
tablespace directories, for sure.
Yes, absolutely.
Stephen Frost sfr...@snowman.net writes:
I've not followed this thread all that closely, but I do tend to agree
with the idea of only try to mess with files that are *clearly* ours to
mess with.
Well, that opens us to errors of omission, ie failing to fsync things we
should have. Maybe that's
On 2015-05-25 14:14:10 -0400, Stephen Frost wrote:
That seems overly complicated, for my 2c at least. I don't particularly
like trying to mess with files that might be rightfully considered not
ours either.
I'd not consider an fsync to be messing with files, especially if
they're in PGDATA.
Andres Freund and...@anarazel.de writes:
On 2015-05-25 14:14:10 -0400, Stephen Frost wrote:
Additionally we could attempt to fsync with a readonly fd before trying
the read-write fd...
Not really sure I see that as helping.
On most OSs, except windows and some obscure unixes, a readonly fd
On Mon, May 25, 2015 at 2:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Andres Freund and...@anarazel.de writes:
On 2015-05-25 14:14:10 -0400, Stephen Frost wrote:
Additionally we could attempt to fsync with a readonly fd before trying
the read-write fd...
Not really sure I see that as helping.
Robert,
* Robert Haas (robertmh...@gmail.com) wrote:
On Mon, May 25, 2015 at 2:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Andres Freund and...@anarazel.de writes:
On 2015-05-25 14:14:10 -0400, Stephen Frost wrote:
Not really sure I see that as helping.
On most OSs, except windows and some
On 2015-05-25 21:33:03 -0400, Robert Haas wrote:
On Mon, May 25, 2015 at 2:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Perhaps, but if we didn't have permission to write the file, it's hard to
argue that it's our responsibility to fsync it. So this seems like it's
adding complexity without
At 2015-05-26 03:54:51 +0200, and...@anarazel.de wrote:
Say a symlink goes to a binary, which is currently being executed:
ETXTBSY. Or the file is in a readonly filesystem: EROFS. So we'd
need to ignore a lot of errors, possibly ignoring valid ones.
Right. That's why I started out by being
Christoph Berg m...@debian.org writes:
Re: To Andres Freund 2015-05-24 20150524075244.gb27...@msg.df7cb.de
Re: Andres Freund 2015-05-24 20150524005245.gd32...@alap3.anarazel.de
How about, to avoid masking actual problems, we have a more
differentiated logic for the toplevel data directory?
Re: To Andres Freund 2015-05-24 20150524075244.gb27...@msg.df7cb.de
Re: Andres Freund 2015-05-24 20150524005245.gd32...@alap3.anarazel.de
How about, to avoid masking actual problems, we have a more
differentiated logic for the toplevel data directory? I think we could
just skip all
Re: Andres Freund 2015-05-24 20150524005245.gd32...@alap3.anarazel.de
How about, to avoid masking actual problems, we have a more
differentiated logic for the toplevel data directory? I think we could
just skip all non-directory files in there data_directory itself. None
of the files in the
Christoph Berg m...@debian.org writes:
the new fsync-pgdata-on-recovery code tries to open all files using
O_RDWR. At least on 9.1, this can make recovery fail:
Hm. I wonder whether it would be all right to just skip files for which
we get EPERM on open(). The argument being that if we can't
Re: Tom Lane 2015-05-23 2284.1432413...@sss.pgh.pa.us
Christoph Berg m...@debian.org writes:
the new fsync-pgdata-on-recovery code tries to open all files using
O_RDWR. At least on 9.1, this can make recovery fail:
Hm. I wonder whether it would be all right to just skip files for which
On 2015-05-23 16:33:29 -0400, Tom Lane wrote:
Christoph Berg m...@debian.org writes:
the new fsync-pgdata-on-recovery code tries to open all files using
O_RDWR. At least on 9.1, this can make recovery fail:
Hm. I wonder whether it would be all right to just skip files for which
we get
67 matches
Mail list logo
