Re: [HACKERS] pg_xlog -> pg_xjournal?
On 05/03/2016 06:38 PM, Michael Paquier wrote: On Mon, May 2, 2016 at 10:29 PM, Robert Haas wrote: On Thu, Apr 28, 2016 at 11:46 PM, Craig Ringer wrote: I just helped another person yesterday who deleted their pg_xlog. The biggest reason I've seen pg_xlog get deleted is not because it's called pg_xlog, but because it's located someplace other than under the data directory and referenced via a symlink. Renaming it might still make it less likely for people to get trigger happy, though. FWIW, I have seen a couple of times the same pattern as Craig: "because it contains log in its name implies that removing it causes no harm". "You can't fix stupid.", Ron White. That said, I too have run into the "Oh it said, log... we thought it was o.k. to delete." Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Mon, May 2, 2016 at 10:29 PM, Robert Haas wrote: > On Thu, Apr 28, 2016 at 11:46 PM, Craig Ringer wrote: >> I just helped another person yesterday who deleted their pg_xlog. > > The biggest reason I've seen pg_xlog get deleted is not because it's > called pg_xlog, but because it's located someplace other than under > the data directory and referenced via a symlink. Renaming it might > still make it less likely for people to get trigger happy, though. FWIW, I have seen a couple of times the same pattern as Craig: "because it contains log in its name implies that removing it causes no harm". -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Thu, Apr 28, 2016 at 11:46 PM, Craig Ringer wrote: > On 29 April 2016 at 10:12, Bruce Momjian wrote: >> My larger question was, was 9.6 an ideal time to do this, and if so, why >> did this issue not get done. If 9.6 wasn't in some way ideal, we can do >> it in 9.7. > > Doing it at the very beginning of the release cycle seems like a good idea. Yeah. If we do this, it's is going to affect quite a few bits and pieces that know about pg_xlog, not to mention (I think) lots of third-party tools. So any patch doing this needs to be very carefully reviewed before it goes into core, and on a timeline that gives outside-of-core stuff a chance to react to it. > I just helped another person yesterday who deleted their pg_xlog. The biggest reason I've seen pg_xlog get deleted is not because it's called pg_xlog, but because it's located someplace other than under the data directory and referenced via a symlink. Renaming it might still make it less likely for people to get trigger happy, though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 29 April 2016 at 10:12, Bruce Momjian wrote: > > My larger question was, was 9.6 an ideal time to do this, and if so, why > did this issue not get done. If 9.6 wasn't in some way ideal, we can do > it in 9.7. > > Doing it at the very beginning of the release cycle seems like a good idea. I just helped another person yesterday who deleted their pg_xlog. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Thu, Apr 28, 2016 at 10:07:40PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > Are we going to rename pg_xlog or pg_clog for 9.6? > > NO. We don't even have a patch for this, much less one that's been > through any review. This suggestion is at least two months too late. My larger question was, was 9.6 an ideal time to do this, and if so, why did this issue not get done. If 9.6 wasn't in some way ideal, we can do it in 9.7. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
Bruce Momjian writes: > Are we going to rename pg_xlog or pg_clog for 9.6? NO. We don't even have a patch for this, much less one that's been through any review. This suggestion is at least two months too late. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Thu, Apr 28, 2016 at 04:30:39PM -0700, Andres Freund wrote: > On 2016-04-28 19:23:26 -0400, Bruce Momjian wrote: > > > > Are we going to rename pg_xlog or pg_clog for 9.6? > > If we do so, we should do it at a good bit earlier in the cycle imo. Well, we talked about it in May of 2015, but didn't do anything. Was there some big WAL user API change in this release that encouraged the idea of the change? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 2016-04-28 19:23:26 -0400, Bruce Momjian wrote: > > Are we going to rename pg_xlog or pg_clog for 9.6? If we do so, we should do it at a good bit earlier in the cycle imo. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
Are we going to rename pg_xlog or pg_clog for 9.6? --- On Sun, May 31, 2015 at 10:44:54PM +0200, Joel Jacobson wrote: > On Sun, May 31, 2015 at 7:46 PM, Tom Lane wrote: > > Hm. I think the impact on third-party backup tools would be rather bad, > > but there's a simple modification of the idea that might fix that: > > just always create pg_xlog as a symlink to pg_xjournal during initdb. > > Anybody who blindly removes pg_xlog won't have done anything > > irreversible. We could deprecate pg_xlog and stop creating the symlink > > after a few releases, once third-party tools have had a reasonable > > amount of time to adjust. > > I like the solution. Simple and effective. > +1 > > > In the end though, this is a lot of thrashing for a problem that > > only comes up rarely ... > > It happens often enough for the problem to be the first mentioned > use-case of pg_resetxlog at Stack Overflow: > > "pg_resetxlog is a tool of last resort for getting your database > running again after: > 1. You deleted files you shouldn't have from pg_xlog;" > > (http://stackoverflow.com/questions/12897429/what-does-pg-resetxlog-do-and-how-does-it-work) > > Preventing failure in the case of faults is of course one of the > primary objectives of any database. > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Tue, Jun 2, 2015 at 2:45 AM, Mark Kirkwood wrote: > On 01/06/15 05:29, Joel Jacobson wrote: > >> While anyone who is familiar with postgres would never do something as >> stupid as to delete pg_xlog, >> according to Google, there appears to be a fair amount of end-users out >> there having made the irrevocable mistake of deleting the precious >> directory, >> a decision made on the assumption that since "it has *log* in the name >> so it must be unimportant" >> ( >> http://stackoverflow.com/questions/12897429/what-does-pg-resetxlog-do-and-how-does-it-work >> ). >> >> If we could turn back time, would we have picked "pg_xlog" as the most >> optimal name for this important directory, or would we have come up with >> a more user-friendly name? >> >> Personally, I have never had any problems with pg_xlog, but I realize >> there are quite a few unlucky new users who end up in trouble. >> >> My suggestion is to use "pg_xjournal" instead of "pg_xlog" when new >> users create a new data directory using initdb, and allow for both >> directories to exist (exclusive or, i.e. either one or the other, but >> not both). That way we don't complicate the life for any existing users, >> all their tools will continue to work who rely on pg_xlog to be named >> pg_xlog, but only force new users to do a bit of googling when they >> can't use whatever tool that can't find pg_xlog. When they find out it's >> an important directory, they can simply create a symlink and their old >> not yet updated tool will work again. >> >> Thoughts? >> >> > +1 > > Strongly agree - I have had people on my dba course ask about deleting > these pesky 'log' directories (obvious confusion/conflation with pg_log > ...)! A change of name would help reduce the temptation! > > Why is it named pg_log by default anyway? While base and global are not named pg_base (or pg_default) and pg_global ? If we are going to break things in some release, maybe we should rename them all to have a little more rhyme or reason to them. Or is there already a rhyme or reason I am overlooking? I would think all the config and human-readable log files/directories should have one prefix (or absence of prefix), and all the internal files/directories with no user serviceable parts should have a different one. Cheers, Jeff
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 6/2/15 4:58 PM, David Steele wrote: On 5/31/15 1:46 PM, Tom Lane wrote: Hm. I think the impact on third-party backup tools would be rather bad, but there's a simple modification of the idea that might fix that: just always create pg_xlog as a symlink to pg_xjournal during initdb. Anybody who blindly removes pg_xlog won't have done anything irreversible. We could deprecate pg_xlog and stop creating the symlink after a few releases, once third-party tools have had a reasonable amount of time to adjust. As the author of a third-party backup tool I'd prefer to make a clean break and just rename the directories in a single release. 9.5 has similar backup/restore related changes with no nod to backwards compatibility. And that's fine. Applications can iterate faster than databases and they should. +1. I think we're making a mountain out of a mole-hill and putting any possibility of improvement here at risk. (And I definitely think this needs improvement). Two options to make lives easier: 1) An initdb option to create the necessary symlinks as Tom suggests, just not by default. 2) Instructions in the release notes for users who did not see the initdb option in the first place. #2 seems reasonable. #1 seems like it's partway up the molemountain. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 5/31/15 1:46 PM, Tom Lane wrote: > Hm. I think the impact on third-party backup tools would be rather bad, > but there's a simple modification of the idea that might fix that: > just always create pg_xlog as a symlink to pg_xjournal during initdb. > Anybody who blindly removes pg_xlog won't have done anything > irreversible. We could deprecate pg_xlog and stop creating the symlink > after a few releases, once third-party tools have had a reasonable > amount of time to adjust. As the author of a third-party backup tool I'd prefer to make a clean break and just rename the directories in a single release. 9.5 has similar backup/restore related changes with no nod to backwards compatibility. And that's fine. Applications can iterate faster than databases and they should. Two options to make lives easier: 1) An initdb option to create the necessary symlinks as Tom suggests, just not by default. 2) Instructions in the release notes for users who did not see the initdb option in the first place. -- - David Steele da...@pgmasters.net signature.asc Description: OpenPGP digital signature
Re: [HACKERS] pg_xlog -> pg_xjournal?
* Michael Nolan wrote: > Why not take a simpler approach and create a zero length file in directories > that should not be fiddled with by non-experts using a file name something > like "DO.NOT.DELETE.THESE.FILES"? Move the critical things into a new subdirectory? $PGDATA/pg_critical/pg_xlog? Perhaps "pg_internal" instead, or "pg_indispensable", or "pg_here_be_dragons", or "pg_delete_this_or_anything_in_here_and_you_wil l_have_utterly_destroyed_your_database"? That last one should be clear enough, if a tad long for deeply nested PGDATA locations. Symlinking the old path to the new one would be impossible, of course, if the intent is to protect against "rm -rf *log*/*" fiends. -- Christian -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Tue, Jun 2, 2015 at 11:03 PM, Abhijit Menon-Sen wrote: > At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: >> >> No, it won't prevent the incredibly stupid from doing incredibly >> stupid things, nothing will. > > I hate to speechify, but I think we should try hard to avoid framing > such questions in terms of "incredibly stupid" people and the things > they might do. > > We have anecdotal and circumstantial evidence that the names pg_xlog and > pg_clog have given some people the impression that they can delete files > therein. Sometimes do this when their server is in imminent danger of > running out of space, sometimes not. But our documentation makes it > clear that these files are important. > > I think naming these directories to convey the right impression is a > straightforward interface design problem, and we also know that big > flashing red warnings are less effective than one might hope for. I > do not think a bigger, stripier warning is worth doing in isolation. > I do think it's worth choosing better names. > +1. Regards, Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Tue, Jun 2, 2015 at 07:33:19PM +0530, Abhijit Menon-Sen wrote: > At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: > > > > No, it won't prevent the incredibly stupid from doing incredibly > > stupid things, nothing will. > > I hate to speechify, but I think we should try hard to avoid framing > such questions in terms of "incredibly stupid" people and the things > they might do. > > We have anecdotal and circumstantial evidence that the names pg_xlog and > pg_clog have given some people the impression that they can delete files > therein. Sometimes do this when their server is in imminent danger of > running out of space, sometimes not. But our documentation makes it > clear that these files are important. > > I think naming these directories to convey the right impression is a > straightforward interface design problem, and we also know that big > flashing red warnings are less effective than one might hope for. I > do not think a bigger, stripier warning is worth doing in isolation. > I do think it's worth choosing better names. I think we also have to consider that these admins are in crisis mode when the run out of disk space, so they are not in a calm environment where decisions are considered carefully. You can say decisions should always be considered carefully, but I am not sure how realistic that is. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 06/01/2015 09:35 PM, Michael Nolan wrote: Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? +1 No, it won't prevent the incredibly stupid from doing incredibly stupid things, nothing will. Darwin? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own emotions, so everyone else should do it for you. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: > > No, it won't prevent the incredibly stupid from doing incredibly > stupid things, nothing will. I hate to speechify, but I think we should try hard to avoid framing such questions in terms of "incredibly stupid" people and the things they might do. We have anecdotal and circumstantial evidence that the names pg_xlog and pg_clog have given some people the impression that they can delete files therein. Sometimes do this when their server is in imminent danger of running out of space, sometimes not. But our documentation makes it clear that these files are important. I think naming these directories to convey the right impression is a straightforward interface design problem, and we also know that big flashing red warnings are less effective than one might hope for. I do not think a bigger, stripier warning is worth doing in isolation. I do think it's worth choosing better names. -- Abhijit P.S. Unrelated to Michael's mail, but I also don't think it's worth debating whether people will run "rm -rf *log" or "rm -rf log/*" or whatever other variant you can think of. I'm arguing for correcting a mis-perception, not try to dodge specific harmful commands. Tom's proposal of using a symlink but dropping it after third-party tools have had time to catch up seems like the best approach to me. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 06/02/2015 03:06 AM, Joel Jacobson wrote: On Tue, Jun 2, 2015 at 6:35 AM, Michael Nolan wrote: Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? Then the smart sysadmin will say "but I didn't delete any *files*, I just deleted a log *directory*" :-) Fire that sysadmin, not because they are incompetent but because either they are an idiot or they think they can treat you as one. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 01/06/15 05:29, Joel Jacobson wrote: While anyone who is familiar with postgres would never do something as stupid as to delete pg_xlog, according to Google, there appears to be a fair amount of end-users out there having made the irrevocable mistake of deleting the precious directory, a decision made on the assumption that since "it has *log* in the name so it must be unimportant" (http://stackoverflow.com/questions/12897429/what-does-pg-resetxlog-do-and-how-does-it-work). If we could turn back time, would we have picked "pg_xlog" as the most optimal name for this important directory, or would we have come up with a more user-friendly name? Personally, I have never had any problems with pg_xlog, but I realize there are quite a few unlucky new users who end up in trouble. My suggestion is to use "pg_xjournal" instead of "pg_xlog" when new users create a new data directory using initdb, and allow for both directories to exist (exclusive or, i.e. either one or the other, but not both). That way we don't complicate the life for any existing users, all their tools will continue to work who rely on pg_xlog to be named pg_xlog, but only force new users to do a bit of googling when they can't use whatever tool that can't find pg_xlog. When they find out it's an important directory, they can simply create a symlink and their old not yet updated tool will work again. Thoughts? +1 Strongly agree - I have had people on my dba course ask about deleting these pesky 'log' directories (obvious confusion/conflation with pg_log ...)! A change of name would help reduce the temptation! regards Mark -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Tue, Jun 2, 2015 at 6:35 AM, Michael Nolan wrote: > Why not take a simpler approach and create a zero length file in directories > that should not be fiddled with by non-experts using a file name something > like "DO.NOT.DELETE.THESE.FILES"? Then the smart sysadmin will say "but I didn't delete any *files*, I just deleted a log *directory*" :-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
Le 2 juin 2015 6:37 AM, "Michael Nolan" a écrit : > > Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? > > No, it won't prevent the incredibly stupid from doing incredibly stupid things, nothing will. Sounds much simpler and smarter to me than the other solutions.
Re: [HACKERS] pg_xlog -> pg_xjournal?
Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like "DO.NOT.DELETE.THESE.FILES"? No, it won't prevent the incredibly stupid from doing incredibly stupid things, nothing will. -- Mike Nolan
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Tue, Jun 2, 2015 at 8:48 AM, Josh Berkus wrote: > On 06/01/2015 04:22 PM, Thomas Munro wrote: >> On Tue, Jun 2, 2015 at 8:17 AM, Josh Berkus wrote: >>> Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". >>> Please let's not add yet another term for the WAL. >> >> +1 for pg_wal if it has to be renamed. >> >> If pg_clog also has to be renamed, how about using your other >> suggestion "pg_xact" for that? It fits alongside pg_multixact. > > Yes, or pg_commit would work too (it's the "commit log"). > > However, choosing a new name is the easy part. If we are talking about deletion of paths named as *log* here... pg_commit sounds like a recipe for a user thinking something like that: I have a long-running transaction and it *won't* commit, so deleting pg_commit will enforce its rollback. Then let's delete it. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 06/01/2015 04:22 PM, Thomas Munro wrote: > On Tue, Jun 2, 2015 at 8:17 AM, Josh Berkus wrote: >> Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". >> Please let's not add yet another term for the WAL. > > +1 for pg_wal if it has to be renamed. > > If pg_clog also has to be renamed, how about using your other > suggestion "pg_xact" for that? It fits alongside pg_multixact. Yes, or pg_commit would work too (it's the "commit log"). However, choosing a new name is the easy part. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Tue, Jun 2, 2015 at 8:17 AM, Josh Berkus wrote: > Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". > Please let's not add yet another term for the WAL. +1 for pg_wal if it has to be renamed. If pg_clog also has to be renamed, how about using your other suggestion "pg_xact" for that? It fits alongside pg_multixact. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Mon, Jun 1, 2015 at 05:57:27PM -0400, David Steele wrote: > On 6/1/15 4:42 PM, Joel Jacobson wrote: > >> Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". > >> Please let's not add yet another term for the WAL. > > I like pg_wal. It's correct and suitably mysterious. +1 -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 6/1/15 4:42 PM, Joel Jacobson wrote: >> Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". >> Please let's not add yet another term for the WAL. I like pg_wal. It's correct and suitably mysterious. -- - David Steele da...@pgmasters.net signature.asc Description: OpenPGP digital signature
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Mon, Jun 1, 2015 at 10:17 PM, Josh Berkus wrote: > If we symlink pg_xlog, then it will still trip up anyone who does "rm > -rf *log*/*" or deletes files directly from inside the directory, both > of which I've seen. Deleting the directory itself is comparatively rare > in my experience. So for this do to any good, we'd have to plan to > (eventally, at least) get rid of the symlinks. Do links work the same > on Windows, btw? Hm, I don't agree the symlink version wouldn't do *any* good. I think it's a good step since it solves the rm -rf pg_xlog problem, and it solves the problem if a sysadmin uses /usr/bin/du to find large directories suitable for deletion, as it won't show symlinks. > Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". > Please let's not add yet another term for the WAL. +1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 05/31/2015 10:46 AM, Tom Lane wrote: > Joel Jacobson writes: >> If we could turn back time, would we have picked "pg_xlog" as the most >> optimal name for this important directory, or would we have come up with a >> more user-friendly name? > > Yeah... > >> My suggestion is to use "pg_xjournal" instead of "pg_xlog" when new users >> create a new data directory using initdb, and allow for both directories to >> exist (exclusive or, i.e. either one or the other, but not both). That way >> we don't complicate the life for any existing users, all their tools will >> continue to work who rely on pg_xlog to be named pg_xlog, but only force >> new users to do a bit of googling when they can't use whatever tool that >> can't find pg_xlog. When they find out it's an important directory, they >> can simply create a symlink and their old not yet updated tool will work >> again. > > Hm. I think the impact on third-party backup tools would be rather bad, > but there's a simple modification of the idea that might fix that: > just always create pg_xlog as a symlink to pg_xjournal during initdb. > Anybody who blindly removes pg_xlog won't have done anything > irreversible. We could deprecate pg_xlog and stop creating the symlink > after a few releases, once third-party tools have had a reasonable > amount of time to adjust. > > Note that we'd really also have to rename pg_clog etc if you want to > protect against people who "rm -rf *log" without reading documentation. > But I don't see why the same trick wouldn't work for all of them. Allow me to be the one to point out other practical difficulties. If we symlink pg_xlog, then it will still trip up anyone who does "rm -rf *log*/*" or deletes files directly from inside the directory, both of which I've seen. Deleting the directory itself is comparatively rare in my experience. So for this do to any good, we'd have to plan to (eventally, at least) get rid of the symlinks. Do links work the same on Windows, btw? Plus, updating oodles and oodles of documentation. This would have to be done as part of the 10.0 release, I think, and we'd have to warn the heck out of it. Also ... if we were to rename it, it should be "pg_wal" or "pg_xact". Please let's not add yet another term for the WAL. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On 5/31/15 1:29 PM, Joel Jacobson wrote: > My suggestion is to use "pg_xjournal" instead of "pg_xlog" If we're going to make any changes in this area, I would like to see a more comprehensive solution for separating user-editable files from internal files. (E.g., make subdirectories "etc" and "var"; just a thought.) If we're just going to move or symlink one file every release or two, we'll have a never-settling mess. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
At 2015-05-31 13:46:33 -0400, t...@sss.pgh.pa.us wrote: > > just always create pg_xlog as a symlink to pg_xjournal during initdb. At first glance, the Subject: of this thread made me think that *was* Joel's proposal. :-) But I think it's a great idea, and worth doing. I think "pg_journal" (no "x") is sufficient. The journal is an idea that people are familiar with from filesystems anyway. > Note that we'd really also have to rename pg_clog etc pg_clog could become pg_commits or pg_xactstatus or pg_commit_status or something. What else is there? I'd hope pg_logical can be left alone. > A more difficult question is whether we'd also rename pg_resetxlog, > pg_receivexlog, etc. I don't think it's necessary. (Of course, people have wanted to rename pg_resetxlog to make it sound more scary anyway, but that's a different matter.) > In the end though, this is a lot of thrashing for a problem that > only comes up rarely ... I'll agree with Joel that it comes up far too often for comfort anyway. I've known a number of people who were on the verge of deleting stuff from pg_xlog, but just happened to check with me first. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
On Sun, May 31, 2015 at 7:46 PM, Tom Lane wrote: > Hm. I think the impact on third-party backup tools would be rather bad, > but there's a simple modification of the idea that might fix that: > just always create pg_xlog as a symlink to pg_xjournal during initdb. > Anybody who blindly removes pg_xlog won't have done anything > irreversible. We could deprecate pg_xlog and stop creating the symlink > after a few releases, once third-party tools have had a reasonable > amount of time to adjust. I like the solution. Simple and effective. +1 > In the end though, this is a lot of thrashing for a problem that > only comes up rarely ... It happens often enough for the problem to be the first mentioned use-case of pg_resetxlog at Stack Overflow: "pg_resetxlog is a tool of last resort for getting your database running again after: 1. You deleted files you shouldn't have from pg_xlog;" (http://stackoverflow.com/questions/12897429/what-does-pg-resetxlog-do-and-how-does-it-work) Preventing failure in the case of faults is of course one of the primary objectives of any database. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_xlog -> pg_xjournal?
Joel Jacobson writes: > If we could turn back time, would we have picked "pg_xlog" as the most > optimal name for this important directory, or would we have come up with a > more user-friendly name? Yeah... > My suggestion is to use "pg_xjournal" instead of "pg_xlog" when new users > create a new data directory using initdb, and allow for both directories to > exist (exclusive or, i.e. either one or the other, but not both). That way > we don't complicate the life for any existing users, all their tools will > continue to work who rely on pg_xlog to be named pg_xlog, but only force > new users to do a bit of googling when they can't use whatever tool that > can't find pg_xlog. When they find out it's an important directory, they > can simply create a symlink and their old not yet updated tool will work > again. Hm. I think the impact on third-party backup tools would be rather bad, but there's a simple modification of the idea that might fix that: just always create pg_xlog as a symlink to pg_xjournal during initdb. Anybody who blindly removes pg_xlog won't have done anything irreversible. We could deprecate pg_xlog and stop creating the symlink after a few releases, once third-party tools have had a reasonable amount of time to adjust. Note that we'd really also have to rename pg_clog etc if you want to protect against people who "rm -rf *log" without reading documentation. But I don't see why the same trick wouldn't work for all of them. A more difficult question is whether we'd also rename pg_resetxlog, pg_receivexlog, etc. It would be hard to make those changes similarly transparent. In the end though, this is a lot of thrashing for a problem that only comes up rarely ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pg_xlog -> pg_xjournal?
While anyone who is familiar with postgres would never do something as stupid as to delete pg_xlog, according to Google, there appears to be a fair amount of end-users out there having made the irrevocable mistake of deleting the precious directory, a decision made on the assumption that since "it has *log* in the name so it must be unimportant" (http://stackoverflow .com/questions/12897429/what-does-pg-resetxlog-do-and-how-does-it-work). If we could turn back time, would we have picked "pg_xlog" as the most optimal name for this important directory, or would we have come up with a more user-friendly name? Personally, I have never had any problems with pg_xlog, but I realize there are quite a few unlucky new users who end up in trouble. My suggestion is to use "pg_xjournal" instead of "pg_xlog" when new users create a new data directory using initdb, and allow for both directories to exist (exclusive or, i.e. either one or the other, but not both). That way we don't complicate the life for any existing users, all their tools will continue to work who rely on pg_xlog to be named pg_xlog, but only force new users to do a bit of googling when they can't use whatever tool that can't find pg_xlog. When they find out it's an important directory, they can simply create a symlink and their old not yet updated tool will work again. Thoughts?
