Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 8:54 GMT+02:00 Michael Paquier : > On Fri, May 29, 2015 at 3:20 PM, Guillaume Lelarge > wrote: > > Le 29 mai 2015 8:10 AM, "Pavel Stehule" a > écrit : > >> > >> Hi > >> > >> I am not sure if PGXN can substitute contrib - mainly due deployment - > It > >> doesn't helps with MS Windows. Installing necessary software for > compilation > >> there is terrible. > >> > > > > I agree it's hard to compile an extension on Windows, but that's already > > what we have. And I'm sure EDB will put all interesting contrib modules > in > > their windows installer to help users. They already go way further than > any > > Linux packages. > > Speaking with my Windows packager hat on, compiling with MinGW would > not be that terrible I think for extensions, as all the existing > Makefile machinery could be used for this purpose. MSVC stuff is more > complicated though with what we have in-core, but still I think that > we could do something with them if we refactor a bit the code and make > easier the declaration of Project objects and have some proper > documentation in the extension chapter, the idea being that users > should not need to build more than a simple build.pl file linking to > some of .pm files of the in-core perl module scripts, basically with a > switch to src/tools/msvc. You would need to have those modules as well > as the compiled deliverables to compile the extensions, but that's the > same deal as any devel-* package on Linux. > I had a lot of problems with MSVC installation Issues: 1. VS requires relatively new MS Windows - problem for people with Ms Win 7 and older 2. After installation you have to find and apply some critical fixes - some is bad documented. I have a few customers on MS Win - nobody was able to compile extension there - and I had lot of trables - (but I worked with MSVC ten years ago) Regards Pavel > -- > Michael >
Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested I agree with Peter that "We don't tab-complete everything we possibly could", but using tabs after "SET ROLE TO " provides "DEFAULT" as an option which seems wrong. This patch adds list of roles over there, which I guess good to have than giving something unusual. I reviewed this straight forward patch and looks good to me. Since we might not want this, review is done and thus passing it to committer to decide. The new status of this patch is: Ready for Committer -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx
2015-05-29 9:28 GMT+02:00 Jeevan Chalke : > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: tested, passed > Spec compliant: tested, passed > Documentation:not tested > > I agree with Peter that "We don't tab-complete everything we possibly > could", but using tabs after "SET ROLE TO " provides "DEFAULT" as an option > which seems wrong. > This patch adds list of roles over there, which I guess good to have than > giving something unusual. > Surely, we cannot to support tab-complete everywhere. But if we can do it simply, we should to do it. Why: 1. It is good help for beginners 2. I am PostgreSQL lecture and evangelist in Czech Republic and Slovak Republic. The tabcomplete possibility is surprisingly good factor for accepting PostgreSQL concept, architecture, psql But back to this topic. I am thinking so it is little bit different due fact so we support two very syntax for one feature. And looks little bit strange, so one way is supported by autocomplete and second not. > > I reviewed this straight forward patch and looks good to me. > > Since we might not want this, review is done and thus passing it to > committer to decide. ok > > The new status of this patch is: Ready for Committer > Thank you very much Regards Pavel > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote: > 1. VS requires relatively new MS Windows - problem for people with Ms Win 7 > and older Really, I use Win 2k8 stuff and Win7 quite a lot. > 2. After installation you have to find and apply some critical fixes - some > is bad documented. Example? Perhaps we could improve the docs then. > I have a few customers on MS Win - nobody was able to compile extension > there - and I had lot of trables - (but I worked with MSVC ten years ago) Well, FWIW, I have some extensions compiled on Windows using MSVC modules with stuff like perl -I src/tools/msvc build.pl that are out of the vanilla scripts and it works... The script is rather simple even if a bit grotty. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 9:42 GMT+02:00 Michael Paquier : > On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote: > > 1. VS requires relatively new MS Windows - problem for people with Ms > Win 7 > > and older > > Really, I use Win 2k8 stuff and Win7 quite a lot. > On Win 7 you have to search and install now unsupported VS EE 2010. VS Express 2013 requires Win 8. And finding VS 2010 is not simply - I had to take install files from some private sources. > > 2. After installation you have to find and apply some critical fixes - > some > > is bad documented. > > Example? Perhaps we could improve the docs then. > It was a VS 2010 related issues - not PostgreSQL issues > > > I have a few customers on MS Win - nobody was able to compile extension > > there - and I had lot of trables - (but I worked with MSVC ten years ago) > > Well, FWIW, I have some extensions compiled on Windows using MSVC > modules with stuff like perl -I src/tools/msvc build.pl that are out > of the vanilla scripts and it works... The script is rather simple > even if a bit grotty. > see steps from http://dba.stackexchange.com/questions/68644/problems-while-building-plpgsql-check-on-windows It is not hard for Windows C++ developer, It is little bit messy for Linux developer, and it is out of possibility usual Windows Visual Basic (and similar level) developers. > -- > Michael >
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On Fri, May 29, 2015 at 12:55 PM, Tom Lane wrote: > Abhijit Menon-Sen writes: >>> I have to leave shortly, so I'll look at the initdb cleanup tomorrow. > >> Here's a revision of that patch that's more along the lines of what you >> committed. > > Will look at that tomorrow ... > >> It occurred to me that we should probably also silently skip EACCES on >> opendir and stat/lstat in walkdir. That's what the attached initdb patch >> does. If you think it's a good idea, there's also a small fixup attached >> for the backend version too. > > Actually, I was seriously considering removing the don't-log-EACCES > special case (at least for files, perhaps not for directories). > The reason being that it's darn hard to verify that the code is doing > what it's supposed to when there is no simple way to get it to log any > complaints. I left it as you wrote it in the committed version, but > I think both that and the ETXTBSY special case do not really have value > as long as their only effect is to suppress a LOG entry. > > Speaking of which, could somebody test that the committed patch does > what it's supposed to on Windows? You were worried upthread about > whether the tests for symlinks (aka junction points) behaved correctly, > and I have no way to check that either. The error can be reproduced by using junction (https://technet.microsoft.com/en-us/sysinternals/bb896768.aspx) to define a junction point within PGDATA and then for example mark as read-only a configuration file included in postgresql.conf through the junction point. Using this method, I have checked that 9.4.1 works, reproduced the permission error with 9.4.2, and checked as well that a3ae3db4 fixed the problem. So things look to work fine. Regards, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bugfix: incomplete implementation of errhidecontext
Pavel, will it be good if you separately submit the "bugfix: incomplete implementation of errhidecontext" patch in this commitfest? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bugfix: incomplete implementation of errhidecontext
2015-05-29 9:53 GMT+02:00 Jeevan Chalke : > Pavel, will it be good if you separately submit the > "bugfix: incomplete implementation of errhidecontext" > patch in this commitfest? > > ok, I'll do it Pavel > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On Fri, May 29, 2015 at 9:25 AM, Tom Lane wrote: > > > Speaking of which, could somebody test that the committed patch does > what it's supposed to on Windows? You were worried upthread about > whether the tests for symlinks (aka junction points) behaved correctly, > and I have no way to check that either. > I have done some tests for the committed patch for this issue (especially to check the behaviour of symlink tests in the new code) on Windows - 7 and below are results: Test symlink for pg_xlog - Test -1 - Junction point for pg_xlog 1. Create a database (initdb) 2. Start server and forcefully crash it 3. cp E:\PGData\pg_xlog E:\TempLog\xlog_symlink_loc 4. mv E:\PGData\pg_xlog E:\bak_pg_xlog 5. Created a directory junction (equivalent symlink) mklink /J E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc 6. Restart Server - operation is successful. I have debugged this operation, it does what it suppose to do, detects the junction point and does fsync. Test - 2 - Directory Symlink for pg_xlog First 4 steps are same as Test-1 5. mklink /D E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc 6. Restart Server - Error - FATAL: required WAL directory "pg_xlog" does not exist This error occurs in ValidateXLOGDirectoryStructure()->stat(XLOGDIR, &stat_buf) 7. If I manually step-through that line, it proceeds and in SyncDataDirectory(), it detects the symlink; 8. After that in SyncDataDirectory(), when it does walkdir for datadir, for ./pg_xlog it reports the log message and then proceeds normal and the server is started. Test-3 - Symlinks in pg_tblspc. 1. Create couple of tablespaces which creates symlinks in pg_tblspc 2. Crash the server 3. Restart Server - It works fine. I am not sure Test-2 is a valid thing and we should support it or not, but the current patch is sane w.r.t that form of symlinks as well. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] WIP: Enhanced ALTER OPERATOR
On Thu, May 28, 2015 at 6:43 PM, Tom Lane wrote: > Alexander Korotkov writes: > > Could we address both this problems by denying changing existing > > commutators and negator? ISTM that setting absent commutator and negator > is > > quite enough for ALTER OPERATOR. User extensions could need setting of > > commutator and negator because they could add new operators which don't > > exist before. But it's rather uncommon to unset or change commutator or > > negator. > > Note that this functionality is already covered, in that you can specify > the commutator/negator linkage when you create the second operator. > I'm not particularly convinced that we need to have it in ALTER OPERATOR. > Yeah, I didn't notice that CREATE OPERATOR sets commutator and negator on opposite side as well. Then we probably can leave ALTER OPERATOR without altering commutator and negator. BTW, does DROP OPERATOR works correctly? # create operator == (procedure = int8eq, leftarg = bigint, rightarg = bigint); CREATE OPERATOR # create operator !== (procedure = int8ne, leftarg = bigint, rightarg = bigint, negator = ==); CREATE OPERATOR # select oid, * from pg_operator where oprnamespace = (select oid from pg_namespace where nspname = 'public'); oid | oprname | oprnamespace | oprowner | oprkind | oprcanmerge | oprcanhash | oprleft | oprright | oprresult | oprcom | oprnegate | oprcode | oprrest | oprjoin ---+-+--+--+-+-++-+--+---++---+-+-+- 46355 | !== | 2200 | 10 | b | f | f | 20 | 20 |16 | 0 | 46354 | int8ne | - | - 46354 | == | 2200 | 10 | b | f | f | 20 | 20 |16 | 0 | 46355 | int8eq | - | - (2 rows) # create table test (id bigint); CREATE TABLE # explain verbose select * from test where not id == 10::bigint; QUERY PLAN --- Seq Scan on public.test (cost=0.00..38.25 rows=1130 width=8) Output: id Filter: (test.id !== '10'::bigint) (3 rows) # drop operator !== (int8, int8); DROP OPERATOR # select oid, * from pg_operator where oprnamespace = (select oid from pg_namespace where nspname = 'public'); oid | oprname | oprnamespace | oprowner | oprkind | oprcanmerge | oprcanhash | oprleft | oprright | oprresult | oprcom | oprnegate | oprcode | oprrest | oprjoin ---+-+--+--+-+-++-+--+---++---+-+-+- 46354 | == | 2200 | 10 | b | f | f | 20 | 20 |16 | 0 | 46355 | int8eq | - | - (1 row) # explain verbose select * from test where not id == 10::bigint; ERROR: cache lookup failed for function 0 DROP OPERATOR leaves broken reference in oprnegate. Should we fix it? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 7:20 AM, Guillaume Lelarge wrote: > Le 29 mai 2015 8:10 AM, "Pavel Stehule" a écrit : >> >> Hi >> >> I am not sure if PGXN can substitute contrib - mainly due deployment - It >> doesn't helps with MS Windows. Installing necessary software for compilation >> there is terrible. >> > > I agree it's hard to compile an extension on Windows, but that's already > what we have. And I'm sure EDB will put all interesting contrib modules in > their windows installer to help users. They already go way further than any > Linux packages. The only extra extension we ship is pl/debugger. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule wrote: > > > 2015-05-29 8:20 GMT+02:00 Guillaume Lelarge : >> >> Le 29 mai 2015 8:10 AM, "Pavel Stehule" a écrit >> : >> > >> > Hi >> > >> > I am not sure if PGXN can substitute contrib - mainly due deployment - >> > It doesn't helps with MS Windows. Installing necessary software for >> > compilation there is terrible. >> > >> >> I agree it's hard to compile an extension on Windows, but that's already >> what we have. And I'm sure EDB will put all interesting contrib modules in >> their windows installer to help users. They already go way further than any >> Linux packages. > > I afraid so dependency on EDB in this case is wrong - I have respect to EDB > due work, but installation other extension from EDB stack is difficult, > unclean, and nothing what I would to use as new base. The five or six mouse clicks required to install something like Slony or PostGIS (or both at once) is difficult and unclean? -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 9:35 AM, Dave Page wrote: > On Fri, May 29, 2015 at 7:20 AM, Guillaume Lelarge > wrote: >> Le 29 mai 2015 8:10 AM, "Pavel Stehule" a écrit : >>> >>> Hi >>> >>> I am not sure if PGXN can substitute contrib - mainly due deployment - It >>> doesn't helps with MS Windows. Installing necessary software for compilation >>> there is terrible. >>> >> >> I agree it's hard to compile an extension on Windows, but that's already >> what we have. And I'm sure EDB will put all interesting contrib modules in >> their windows installer to help users. They already go way further than any >> Linux packages. > > The only extra extension we ship is pl/debugger. To clarify - that's the only one we ship in the PostgreSQL installer. There are many more that we ship (both from EDB and other sources), but users use StackBuilder to choose, download and install them. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule wrote: > > > 2015-05-29 9:42 GMT+02:00 Michael Paquier : >> >> On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote: >> > 1. VS requires relatively new MS Windows - problem for people with Ms >> > Win 7 >> > and older >> >> Really, I use Win 2k8 stuff and Win7 quite a lot. > > > On Win 7 you have to search and install now unsupported VS EE 2010. I've been running 2013 on Windows 7 since it came out. Works perfectly well, and didn't require any unusual installation. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 10:40 GMT+02:00 Dave Page : > On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule > wrote: > > > > > > 2015-05-29 9:42 GMT+02:00 Michael Paquier : > >> > >> On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote: > >> > 1. VS requires relatively new MS Windows - problem for people with Ms > >> > Win 7 > >> > and older > >> > >> Really, I use Win 2k8 stuff and Win7 quite a lot. > > > > > > On Win 7 you have to search and install now unsupported VS EE 2010. > > I've been running 2013 on Windows 7 since it came out. Works perfectly > well, and didn't require any unusual installation. > I had different experience - When I did it year ago, I had to searching and install VS 2010. Pavel > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company >
Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 10:37 GMT+02:00 Dave Page : > On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule > wrote: > > > > > > 2015-05-29 8:20 GMT+02:00 Guillaume Lelarge : > >> > >> Le 29 mai 2015 8:10 AM, "Pavel Stehule" a > écrit > >> : > >> > > >> > Hi > >> > > >> > I am not sure if PGXN can substitute contrib - mainly due deployment - > >> > It doesn't helps with MS Windows. Installing necessary software for > >> > compilation there is terrible. > >> > > >> > >> I agree it's hard to compile an extension on Windows, but that's already > >> what we have. And I'm sure EDB will put all interesting contrib modules > in > >> their windows installer to help users. They already go way further than > any > >> Linux packages. > > > > I afraid so dependency on EDB in this case is wrong - I have respect to > EDB > > due work, but installation other extension from EDB stack is difficult, > > unclean, and nothing what I would to use as new base. > > The five or six mouse clicks required to install something like Slony > or PostGIS (or both at once) is difficult and unclean? > I had a problem with downloading isolated packages with stackbuilder - but it was specific, because comp where some packages was installed was not access to internet. > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company >
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On Fri, May 29, 2015 at 5:01 PM, Amit Kapila wrote: > On Fri, May 29, 2015 at 9:25 AM, Tom Lane wrote: >> >> >> Speaking of which, could somebody test that the committed patch does >> what it's supposed to on Windows? You were worried upthread about >> whether the tests for symlinks (aka junction points) behaved correctly, >> and I have no way to check that either. >> > > I have done some tests for the committed patch for this issue > (especially to check the behaviour of symlink tests in the > new code) on Windows - 7 and below are results: > > Test symlink for pg_xlog > - > Test -1 - Junction point for pg_xlog > 1. Create a database (initdb) > 2. Start server and forcefully crash it > 3. cp E:\PGData\pg_xlog E:\TempLog\xlog_symlink_loc > 4. mv E:\PGData\pg_xlog E:\bak_pg_xlog > 5. Created a directory junction (equivalent symlink) > mklink /J E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc > 6. Restart Server - operation is successful. > I have debugged this operation, it does what it suppose to do, > detects the junction point and does fsync. > > Test - 2 - Directory Symlink for pg_xlog > First 4 steps are same as Test-1 > 5. mklink /D E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc > 6. Restart Server - Error > - FATAL: required WAL directory "pg_xlog" does not exist > This error occurs in > ValidateXLOGDirectoryStructure()->stat(XLOGDIR, &stat_buf) > 7. If I manually step-through that line, it proceeds and in > SyncDataDirectory(), it detects the symlink; > 8. After that in SyncDataDirectory(), when it does walkdir for > datadir, for ./pg_xlog it reports the log message and then > proceeds normal and the server is started. > > Test-3 - Symlinks in pg_tblspc. > 1. Create couple of tablespaces which creates symlinks > in pg_tblspc > 2. Crash the server > 3. Restart Server - It works fine. > > I am not sure Test-2 is a valid thing and we should support it or > not, but the current patch is sane w.r.t that form of symlinks > as well. It is always good to check, but is that relevant to the bug fixed? I mean, you need to symlink a file in PGDATA that server has no write permission on... For example tablespaces can be written to in test 3, so that would work even with 9.4.2 after crashing the server with -m immediate. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 5:52 PM, Pavel Stehule wrote: > > > 2015-05-29 10:40 GMT+02:00 Dave Page : >> >> On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule >> wrote: >> > >> > >> > 2015-05-29 9:42 GMT+02:00 Michael Paquier : >> >> >> >> On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote: >> >> > 1. VS requires relatively new MS Windows - problem for people with Ms >> >> > Win 7 >> >> > and older >> >> >> >> Really, I use Win 2k8 stuff and Win7 quite a lot. >> > >> > >> > On Win 7 you have to search and install now unsupported VS EE 2010. >> >> I've been running 2013 on Windows 7 since it came out. Works perfectly >> well, and didn't require any unusual installation. > > > I had different experience - When I did it year ago, I had to searching and > install VS 2010. And it is not mandatory to use or install visual basic, a simple Windows SDK with perl installed is enough to build and install PG from a command prompt. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Fri, May 29, 2015 at 9:55 AM, Pavel Stehule wrote: > > > 2015-05-29 10:37 GMT+02:00 Dave Page : >> >> On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule >> wrote: >> > >> > >> > 2015-05-29 8:20 GMT+02:00 Guillaume Lelarge : >> >> >> >> Le 29 mai 2015 8:10 AM, "Pavel Stehule" a >> >> écrit >> >> : >> >> > >> >> > Hi >> >> > >> >> > I am not sure if PGXN can substitute contrib - mainly due deployment >> >> > - >> >> > It doesn't helps with MS Windows. Installing necessary software for >> >> > compilation there is terrible. >> >> > >> >> >> >> I agree it's hard to compile an extension on Windows, but that's >> >> already >> >> what we have. And I'm sure EDB will put all interesting contrib modules >> >> in >> >> their windows installer to help users. They already go way further than >> >> any >> >> Linux packages. >> > >> > I afraid so dependency on EDB in this case is wrong - I have respect to >> > EDB >> > due work, but installation other extension from EDB stack is difficult, >> > unclean, and nothing what I would to use as new base. >> >> The five or six mouse clicks required to install something like Slony >> or PostGIS (or both at once) is difficult and unclean? > > > I had a problem with downloading isolated packages with stackbuilder - but > it was specific, because comp where some packages was installed was not > access to internet. Run SB on a machine that is connected and: - Select "remote server" on the first step. This will disable any filtering based on database server version, which is normally applied if you're installing to the local machine (it will only offer packages compatible with the database servers you have installed). This is not required if you have the appropriate DB server installed on the internet connected machine as well. - Make a note of the Download Directory on step 3 - Check "Skip installation" on step 4. Copy the downloaded installed from the Download Directory to your server machine, and run them. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 11:02 GMT+02:00 Dave Page : > On Fri, May 29, 2015 at 9:55 AM, Pavel Stehule > wrote: > > > > > > 2015-05-29 10:37 GMT+02:00 Dave Page : > >> > >> On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule > > >> wrote: > >> > > >> > > >> > 2015-05-29 8:20 GMT+02:00 Guillaume Lelarge : > >> >> > >> >> Le 29 mai 2015 8:10 AM, "Pavel Stehule" a > >> >> écrit > >> >> : > >> >> > > >> >> > Hi > >> >> > > >> >> > I am not sure if PGXN can substitute contrib - mainly due > deployment > >> >> > - > >> >> > It doesn't helps with MS Windows. Installing necessary software for > >> >> > compilation there is terrible. > >> >> > > >> >> > >> >> I agree it's hard to compile an extension on Windows, but that's > >> >> already > >> >> what we have. And I'm sure EDB will put all interesting contrib > modules > >> >> in > >> >> their windows installer to help users. They already go way further > than > >> >> any > >> >> Linux packages. > >> > > >> > I afraid so dependency on EDB in this case is wrong - I have respect > to > >> > EDB > >> > due work, but installation other extension from EDB stack is > difficult, > >> > unclean, and nothing what I would to use as new base. > >> > >> The five or six mouse clicks required to install something like Slony > >> or PostGIS (or both at once) is difficult and unclean? > > > > > > I had a problem with downloading isolated packages with stackbuilder - > but > > it was specific, because comp where some packages was installed was not > > access to internet. > > Run SB on a machine that is connected and: > > - Select "remote server" on the first step. This will disable any > filtering based on database server version, which is normally applied > if you're installing to the local machine (it will only offer packages > compatible with the database servers you have installed). This is not > required if you have the appropriate DB server installed on the > internet connected machine as well. > > - Make a note of the Download Directory on step 3 > > - Check "Skip installation" on step 4. > > Copy the downloaded installed from the Download Directory to your > server machine, and run them. > Dave, I didn't say so it is difficult task - I solved it relatively simply - but my customer - long year Oracle DBA, Postgres beginner was messy and needed help and access to EDB support. He expected and searched a web page with download, what is much more usual. Regards Pavel > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company >
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On Fri, May 29, 2015 at 2:28 PM, Michael Paquier wrote: > > On Fri, May 29, 2015 at 5:01 PM, Amit Kapila wrote: > > > > Test-3 - Symlinks in pg_tblspc. > > 1. Create couple of tablespaces which creates symlinks > > in pg_tblspc > > 2. Crash the server > > 3. Restart Server - It works fine. > > > > I am not sure Test-2 is a valid thing and we should support it or > > not, but the current patch is sane w.r.t that form of symlinks > > as well. > > It is always good to check, but is that relevant to the bug fixed? I > mean, you need to symlink a file in PGDATA that server has no write > permission on... For example tablespaces can be written to in test 3, > so that would work even with 9.4.2 after crashing the server with -m > immediate. I have just tried to cover the paths which has symlink related code in the new commit (in functions SyncDataDirectory() and walkdir()), because thats what I understood from Tom's mail. Without test-3, it won't cover walkdir symlink test path, I didn't knew that there was any confusion about verification of permissions problem on Windows and I haven't looked to verify the whole patch. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Free indexed_tlist memory explicitly within set_plan_refs()
On Thu, May 28, 2015 at 6:31 PM, Peter Geoghegan wrote:
> This concerns a thinko in unique index inference. See the commit
> message for full details.
It seems I missed a required defensive measure here. Attached patch
adds it, too.
--
Peter Geoghegan
From c5aee669bbdf58f38f1063934a1d93286506de7b Mon Sep 17 00:00:00 2001
From: Peter Geoghegan
Date: Fri, 29 May 2015 02:53:41 -0700
Subject: [PATCH 4/4] Additional defensive measure
---
src/backend/optimizer/util/plancat.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 1b81f4c..4f38972 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -758,7 +758,7 @@ infer_collation_opclass_match(InferenceElem *elem, Relation idxRel,
if (((Var *) elem->expr)->varattno == attno)
return true;
}
- else
+ else if (attno == 0)
{
Node *nattExpr = list_nth(idxExprs, (natt - 1) - nplain);
--
1.9.1
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bugfix: incomplete implementation of errhidecontext
Done https://commitfest.postgresql.org/5/257/ 2015-05-29 9:56 GMT+02:00 Pavel Stehule : > > > 2015-05-29 9:53 GMT+02:00 Jeevan Chalke : > >> Pavel, will it be good if you separately submit the >> "bugfix: incomplete implementation of errhidecontext" >> patch in this commitfest? >> >> > ok, I'll do it > > Pavel > > >> >> -- >> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgsql-hackers >> > >
Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1
On Fri, May 29, 2015 at 11:24 AM, Robert Haas wrote: > A. Most obviously, we should fix pg_upgrade so that it installs > chkpnt_oldstMulti instead of chkpnt_nxtmulti into datfrozenxid, so > that we stop creating new instances of this problem. That won't get > us out of the hole we've dug for ourselves, but we can at least try to > stop digging. (This is assuming I'm right that chkpnt_nxtmulti is the > wrong thing - anyone want to double-check me on that one?) Yes, it seems like this could lead to truncation of multixacts still referenced by tuples, leading to errors when updating, locking, vacuuming. Why don't we have reports of that? > B. We need to change find_multixact_start() to fail softly. This is > important because it's legitimate for it to fail in recovery, as > discussed upthread, and also because we probably want to eliminate the > fail-to-start hazard introduced in 9.4.2 and 9.3.7. > find_multixact_start() is used in three places, and they each require > separate handling: Here is an experimental WIP patch that changes StartupMultiXact and SetMultiXactIdLimit to find the oldest multixact that exists on disk (by scanning the directory), and uses that if it is more recent than the oldestMultiXactId from shmem, when calling DetermineSafeOldestOffset. I'm not all that happy with it, see below, but let me know what you think. Using unpatched master, I reproduced the startup error with a bit of a short cut: 1. initdb, generate enough multixacts to get more than one offsets file 2. ALTER DATABASE template0 ALLOW_CONNECTION = true;, vacuumdb --freeze --all, CHECKPOINT 3. verify that pg_control now holds a large oldestMultiXactId, and note NextMultiXactId 4. shutdown, pg_resetxlog -m (NextMultiXactId from 3),1 pg_data 5. start up: fails Apply this patch, and it starts up successfully. What are the repro steps for the replay problem? Is a basebackup of a large database undergoing truncation and some good timing needed? > - In SetMultiXactIdLimit, find_multixact_start() is used to set > MultiXactState->oldestOffset, which is used to determine how > aggressively to vacuum. If find_multixact_start() fails, we don't > know how aggressively we need to vacuum to prevent members wraparound; > it's probably best to decide to vacuum as aggressively as possible. > Of course, if we're in recovery, we won't vacuum either way; the fact > that it fails softly is good enough. Isn't it enough to use the start offset for the most recent of the oldest multixact ID and the oldest multixact found by scanning pg_multixact/offsets? In this patch, it does that, but I'm not happy with the time the work is done, it just doesn't seem right for SetMultiXactIdLimit to be scanning that directory. The result of that operation should only change when files have been truncated anyway, and the truncate code was already doing a filesystem scan. Maybe the truncate code should store the earliest multixact ID found on disk in shared memory, so that SetMultiXactIdLimit can use it for free. I tried to get that working but couldn't figure out where it should be initialised -- StartupMultiXact is too late (StartupXLOG calls SetMultiXactIdLimit before that), but BootstrapMultiXact and MultiXactShmemInit didn't seem like the right places either. > - In DetermineSafeOldestOffset, find_multixact_start() is used to set > MultiXactState->offsetStopLimit. If it fails here, we don't know when > to refuse multixact creation to prevent wraparound. Again, in > recovery, that's fine. If it happens in normal running, it's not > clear what to do. Refusing multixact creation is an awfully blunt > instrument. Maybe we can scan pg_multixact/offsets to determine a > workable stop limit: the first file greater than the current file that > exists, minus two segments, is a good stop point. Perhaps we ought to > use this mechanism here categorically, not just when > find_multixact_start() fails. It might be more robust than what we > have now. Done in this patch -- the truncate code calls DetermineSafeOldestOffset with the earliest SLRU found by scanning if that's more recent than the shmem value, and then DetermineSafeOldestOffset applies the step-back-one-whole-segment logic to that as before. > - In TruncateMultiXact, find_multixact_start() is used to set the > truncation point for the members SLRU. If it fails here, I'm guessing > the right solution is not to truncate anything - instead, rely on > intense vacuuming to eventually advance oldestMXact to a value whose > member data still exists; truncate then. TruncateMultiXact already contained logic to do nothing at all if oldestMXact is older than the earliest it can find on disk. I moved that code into find_earliest_multixact_on_disk() to be able to use it elsewhere too, in this patch. > C. I think we should also change TruncateMultiXact() to truncate > offsets first, and then members. As things stand, if we truncate > members first, we increase the risk of seeing an off
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
Re: Tom Lane 2015-05-28 <5740.1432849...@sss.pgh.pa.us> > Abhijit Menon-Sen writes: > > Here's an updated patch for the fsync problem(s). > > I've committed this after some mostly-cosmetic rearrangements. Fwiw, I can confirm that the problem is fixed for 9.5. The regression tests I've added to postgresql-common pass. Thanks! Christoph -- c...@df7cb.de | http://www.df7cb.de/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1
Re: Robert Haas 2015-05-29 > > FTR: Robert, you have been a Samurai on this issue. Our many thanks. > > Thanks! I really appreciate the kind words. I'm still watching with admiration. This list of steps-to-reproduce is the longest and at the same time best I've ever seen. If anyone ever asks me why debugging in open source projects is better than closed source, I'll point them to this thread. Christoph -- c...@df7cb.de | http://www.df7cb.de/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On Fri, May 29, 2015 at 6:49 PM, Amit Kapila wrote: > On Fri, May 29, 2015 at 2:28 PM, Michael Paquier > wrote: >> >> On Fri, May 29, 2015 at 5:01 PM, Amit Kapila >> wrote: >> > >> > Test-3 - Symlinks in pg_tblspc. >> > 1. Create couple of tablespaces which creates symlinks >> > in pg_tblspc >> > 2. Crash the server >> > 3. Restart Server - It works fine. >> > >> > I am not sure Test-2 is a valid thing and we should support it or >> > not, but the current patch is sane w.r.t that form of symlinks >> > as well. >> >> It is always good to check, but is that relevant to the bug fixed? I >> mean, you need to symlink a file in PGDATA that server has no write >> permission on... For example tablespaces can be written to in test 3, >> so that would work even with 9.4.2 after crashing the server with -m >> immediate. > > I have just tried to cover the paths which has symlink related code > in the new commit (in functions SyncDataDirectory() and walkdir()), > because thats what I understood from Tom's mail. Without test-3, it > won't cover walkdir symlink test path, I didn't knew that there was any > confusion about verification of permissions problem on Windows > and I haven't looked to verify the whole patch. Reading once again this thread (after some good red wine + alpha), it looks that you are right: not many checks with Windows junctions have actually been with what has been committed... > Test - 2 - Directory Symlink for pg_xlog > First 4 steps are same as Test-1 > 5. mklink /D E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc > 6. Restart Server - Error > - FATAL: required WAL directory "pg_xlog" does not exist > This error occurs in > ValidateXLOGDirectoryStructure()->stat(XLOGDIR, &stat_buf) > 7. If I manually step-through that line, it proceeds and in > SyncDataDirectory(), it detects the symlink; > 8. After that in SyncDataDirectory(), when it does walkdir for > datadir, for ./pg_xlog it reports the log message and then > proceeds normal and the server is started. Regarding this test, I just tested initdb -X and it works correctly after crashing the server, so there is at least a workaround for the error you are triggering here. Now, shouldn't we improve things as well with mklink? That's a rather narrow case and we have a workaround for it at initialization with initdb. I haven't looked much at the code so I don't have a patch at hand but thoughts on this matter are welcome. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proleakproof vs opr_sanity test
On Fri, May 29, 2015 at 12:09 AM, Tom Lane wrote: > So I think we ought to fix xideqint4 to be marked leakproof and then > add this test. That would only be in HEAD though since it'd require > an initdb. Any objections? FWIW, this makes sense. > Is there a reason to believe that a > built-in function might be leakproof when invoked from one function > OID but not another? I can't think about one after pondering about that a bit... -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On 05/28/2015 11:01 PM, Fabien COELHO wrote: Also, removing a feature is a regression, and someone is always bound to complain... We aren't removing any features. These are all items that are NOT installed or functional by default. Sincerely, JD -- The most kicking donkey PostgreSQL Infrastructure company in existence. The oldest, the most experienced, the consulting company to the stars. Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 - 24x7 - 365 - Proactive and Managed Professional Services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On 05/28/2015 11:08 PM, Pavel Stehule wrote: Hi I am not sure if PGXN can substitute contrib - mainly due deployment - It doesn't helps with MS Windows. Installing necessary software for compilation there is terrible. Anyone who is building for Windows won't have that problem. They already have the environment setup. Windows users for the most part are going to download our stackbuilder distro and go from there. They are not going to compile modules. What I am suggesting doesn't stop that. JD -- The most kicking donkey PostgreSQL Infrastructure company in existence. The oldest, the most experienced, the consulting company to the stars. Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 - 24x7 - 365 - Proactive and Managed Professional Services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1
Thomas Munro writes: > On Fri, May 29, 2015 at 11:24 AM, Robert Haas wrote: >> B. We need to change find_multixact_start() to fail softly. > Here is an experimental WIP patch that changes StartupMultiXact and > SetMultiXactIdLimit to find the oldest multixact that exists on disk > (by scanning the directory), and uses that if it is more recent than > the oldestMultiXactId from shmem, Not sure about the details of this patch, but I was planning to propose what I think is the same thing: the way to make find_multixact_start() fail softly is to have it find the oldest actually existing file if the one that should be there isn't. This would preserve the idea that we aren't trusting the multixact tracking data to be completely right, which was the point of 78db307bb and which evidently is still essential. It would also obviate the need for other places to contain similar logic. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
Michael Paquier writes: > On Fri, May 29, 2015 at 6:49 PM, Amit Kapila wrote: >> Test - 2 - Directory Symlink for pg_xlog >> First 4 steps are same as Test-1 >> 5. mklink /D E:\ PGData\pg_xlog E:\TempLog\xlog_symlink_loc >> 6. Restart Server - Error >> - FATAL: required WAL directory "pg_xlog" does not exist >> This error occurs in >> ValidateXLOGDirectoryStructure()->stat(XLOGDIR, &stat_buf) >> 7. If I manually step-through that line, it proceeds and in >> SyncDataDirectory(), it detects the symlink; >> 8. After that in SyncDataDirectory(), when it does walkdir for >> datadir, for ./pg_xlog it reports the log message and then >> proceeds normal and the server is started. > Regarding this test, I just tested initdb -X and it works correctly > after crashing the server, so there is at least a workaround for the > error you are triggering here. Now, shouldn't we improve things as > well with mklink? That's a rather narrow case and we have a workaround > for it at initialization with initdb. I haven't looked much at the > code so I don't have a patch at hand but thoughts on this matter are > welcome. My (vague) recollection is that what mklink creates is not the same as a junction point and we intentionally only support the latter as pseudo-symlinks. You'd need to trawl the archives for more detail, unless someone else remembers. So I think we're good here. Thanks for all the testing! regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
* Joshua D. Drake (j...@commandprompt.com) wrote: > On 05/28/2015 11:01 PM, Fabien COELHO wrote: > >Also, removing a feature is a regression, and someone is always bound to > >complain... > > We aren't removing any features. These are all items that are NOT > installed or functional by default. Uh, we've already had reports by Debian users about problems upgrading when they forgot to install the contrib package for the version they're moving to, so I'm not sure it's quite that simple. Or, at least, it won't be for the packagers, who I do believe this would be creating a fair bit of work for. That work will be much less if we simply split what's in contrib now into extension and contrib directories, as it's still all one source repo to the packagers. If we punt things out (unless they're being formally deprecated/removed) to another repo somewhere, then the packagers are going to have to deal with new source repos and related complexity. I'm not saying that's a horrible thing and it might make sense in some cases, but generally it's a lot easier to go from one soruce package to a bunch of binary ones than from lots of tiny source packages to lots of tiny packages. The versioning aspect of this does come into play though, as having everything with one relatively slow versioning cycle (on the order of months) is actually keeping the load on the packagers down. Lots of little releases, all at different times, from lots of different source packages, would increase the workload. I'm not sure where I ultimately come down on the question about in-repo vs. out-of-repo. My gut feeling is that if the community is willing to continue maintaining contrib modules, then that's ultimately a good thing and many of them are relatively low-maintenance anyway. Having a high barrier to entry for new modules looks a bit odd, given the definition of contrib, but would be more understandable with a proper 'extensions' directory. Of course, we'd have to collectivly agree that we feel comfortable with a lower barrier for contrib that what is being done now, if the distinction is going to have any meaning. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] [PATCH] Document that directly callable functions may use fn_extra
On 5/28/15 10:15 PM, Craig Ringer wrote: > I was a puzzled by src/backend/utils/fmgr/README and fmgr.h's > descriptions of fcinfo->flinfo->fn_extra (FmgrInfo.fn_extra) as they > seem to conflict with actual usage. > > The docs suggest that fl_extra is for the use of function call handlers, > but in practice it's also used heavily by function implementations as a > cache space. The documentation could also be updated about this. It mentions using fn_extra for PL handlers, gist functions, and set-returning functions, but it doesn't say that you can use it any old function for anything you want. I'm not sure how up to date that README actually is. It looks more like a historical document describing the original proposal. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1
On Fri, May 29, 2015 at 10:17 AM, Tom Lane wrote: > Thomas Munro writes: >> On Fri, May 29, 2015 at 11:24 AM, Robert Haas wrote: >>> B. We need to change find_multixact_start() to fail softly. > >> Here is an experimental WIP patch that changes StartupMultiXact and >> SetMultiXactIdLimit to find the oldest multixact that exists on disk >> (by scanning the directory), and uses that if it is more recent than >> the oldestMultiXactId from shmem, > > Not sure about the details of this patch, but I was planning to propose > what I think is the same thing: the way to make find_multixact_start() > fail softly is to have it find the oldest actually existing file if the > one that should be there isn't. Working on that now. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Proposal] More Vacuum Statistics
On Thu, May 28, 2015 at 4:08 AM, Naoya Anzai wrote: > > 2. Page visibility rate of each table > There is no way to know how many page-bits are them of each tables stored > in their visibility maps. If we can show this information, then we will be > able to guess vacuum overhead for the table. For example, if this table is > a > very big table but page visibility rate is high, then we can advise > pg-users > that vacuum for this table will execute faster than they think by low I/O > overhead. > Furthermore, this information can also be used in order to inform pg-users > about "real" index-only-scan usability. > Isn't this already pg_class.relallvisible? Cheers, Jeff
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
Abhijit Menon-Sen writes: > At 2015-05-28 17:37:16 -0400, t...@sss.pgh.pa.us wrote: >> I have to leave shortly, so I'll look at the initdb cleanup tomorrow. > Here's a revision of that patch that's more along the lines of what you > committed. Pushed with minor revisions. > It occurred to me that we should probably also silently skip EACCES on > opendir and stat/lstat in walkdir. That's what the attached initdb patch > does. If you think it's a good idea, there's also a small fixup attached > for the backend version too. As I mentioned yesterday, I'm not really on board with ignoring EACCES, except for the directories-on-Windows case. Since we're only logging the failures anyway, I think it is reasonable to log a complaint for any unwritable file in the data directory. I've not done it yet, but what I think we oughta do is revert if (errno == EACCES || (isdir && errno == EISDIR)) back to the former logic if (isdir && (errno == EISDIR || errno == EACCES)) and even then, maybe the EACCES exclusion ought to be Windows only? Also I want to get rid of the ETXTBSY special cases. That one doesn't seem like something that we should silently ignore: what the heck are executables doing in the data directory? Or is there some other meaning on Windows? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Need Force flag for pg_drop_replication_slot()
Hackers,
pg_drop_replication_slot() can be a time-critical function when the
master is running out of disk space because the replica is falling
behind. So I was a little startled by this:
cio=# select
pg_drop_replication_slot('bdr_24577_6147720645156311471_1_25383__');
ERROR: replication slot "bdr_24577_6147720645156311471_1_25383__" is
already active
You have to first terminate the replication connection before you can
delete the slot ... and do it fast enough that the replica doesn't
reconnect before you drop the slot.
While I'm just doing this during testing, it could be a critical fail in
production. I think the simplest way to resolve this would be to add a
boolean flag to pg_drop_replication_slot(), which would terminate the
replication connection and delete the slot as a single operation.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pgindent vs emacs
One of the annoying inconsistencies between emacs and pgindent is that emacs refuses to offset a block following a case label, while pgindent does. Is there anything we can do to induce emacs to do what pgindent does? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
Hi, On 2015-05-29 10:15:56 -0700, Josh Berkus wrote: > pg_drop_replication_slot() can be a time-critical function when the > master is running out of disk space because the replica is falling > behind. I don't buy this argument. The same is true for DROP TABLE, TRUNCATE, DROP DATABASE etc. I mean, I agree it'd be convenient, but I can't see it as "critical". > While I'm just doing this during testing, it could be a critical fail in > production. I think the simplest way to resolve this would be to add a > boolean flag to pg_drop_replication_slot(), which would terminate the > replication connection and delete the slot as a single operation. There's no "single operation" for terminating a backend *and* doing something... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
At 2015-05-29 13:14:18 -0400, t...@sss.pgh.pa.us wrote:
>
> Pushed with minor revisions.
Thanks, looks good.
> Since we're only logging the failures anyway, I think it is reasonable
> to log a complaint for any unwritable file in the data directory.
Sounds reasonable, patch attached. ETXTBSY has no special meaning that I
can find under Windows, so I included that change as well.
-- Abhijit
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index b4f6590..70e2347 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -2646,11 +2646,12 @@ pre_sync_fname(const char *fname, bool isdir, int elevel)
if (fd < 0)
{
- if (errno == EACCES || (isdir && errno == EISDIR))
+ /* Some platforms don't support opening directories at all. */
+ if (isdir && errno == EISDIR)
return;
-#ifdef ETXTBSY
- if (errno == ETXTBSY)
+#ifdef WIN32
+ if (isdir && errno == EACCES)
return;
#endif
@@ -2701,11 +2702,12 @@ fsync_fname_ext(const char *fname, bool isdir, int elevel)
fd = OpenTransientFile((char *) fname, flags, 0);
if (fd < 0)
{
- if (errno == EACCES || (isdir && errno == EISDIR))
+ /* Some platforms don't support opening directories at all. */
+ if (isdir && errno == EISDIR)
return;
-#ifdef ETXTBSY
- if (errno == ETXTBSY)
+#ifdef WIN32
+ if (isdir && errno == EACCES)
return;
#endif
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index f0d66fa..f343168 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -617,11 +617,12 @@ pre_sync_fname(const char *fname, bool isdir)
if (fd < 0)
{
- if (errno == EACCES || (isdir && errno == EISDIR))
+ /* Some platforms don't support opening directories at all. */
+ if (isdir && errno == EISDIR)
return;
-#ifdef ETXTBSY
- if (errno == ETXTBSY)
+#ifdef WIN32
+ if (isdir && errno == EACCES)
return;
#endif
@@ -682,11 +683,12 @@ fsync_fname_ext(const char *fname, bool isdir)
fd = open(fname, flags);
if (fd < 0)
{
- if (errno == EACCES || (isdir && errno == EISDIR))
+ /* Some platforms don't support opening directories at all. */
+ if (isdir && errno == EISDIR)
return;
-#ifdef ETXTBSY
- if (errno == ETXTBSY)
+#ifdef WIN32
+ if (isdir && errno == EACCES)
return;
#endif
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On 2015-05-29 13:14:18 -0400, Tom Lane wrote: > Abhijit Menon-Sen writes: > As I mentioned yesterday, I'm not really on board with ignoring EACCES, > except for the directories-on-Windows case. Since we're only logging > the failures anyway, I think it is reasonable to log a complaint for any > unwritable file in the data directory. That sounds like a potentially nontrivial amount of repetitive log bleat after every crash start? One which the user can't really stop? > Also I want to get rid of the ETXTBSY special cases. That one doesn't > seem like something that we should silently ignore: what the heck are > executables doing in the data directory? Or is there some other meaning > on Windows? I've seen a bunch of binaries placed in the data directory as archive/restore commands. Those will be busy a good amount of the time. While it'd not be my choice to do that, it's not entirely unreasonable. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2015-05-29 10:15:56 -0700, Josh Berkus wrote: > > pg_drop_replication_slot() can be a time-critical function when the > > master is running out of disk space because the replica is falling > > behind. > > I don't buy this argument. The same is true for DROP TABLE, TRUNCATE, > DROP DATABASE etc. I disagree about that being the same. > I mean, I agree it'd be convenient, but I can't see it as "critical". Just a random thought- do we check the LOGIN attribute for replication connections? If so, you could tweak that, but that may be an issue if you have multiple replicas using the same role. I'm not sure that it's *critical*, but I could see an argument for adding this post-feature-freeze, which I'm guessing is what Josh was getting at. > > While I'm just doing this during testing, it could be a critical fail in > > production. I think the simplest way to resolve this would be to add a > > boolean flag to pg_drop_replication_slot(), which would terminate the > > replication connection and delete the slot as a single operation. > > There's no "single operation" for terminating a backend *and* doing > something... That's a good point, we'd need to figure out how to make this actually work reliably in the face of a very fast reconnecting process, if we're going to do it. Thanks! Stephen signature.asc Description: Digital signature
[HACKERS] [PATCH, TRIVIAL] don't specify S_IRUSR|S_IWUSR without O_CREAT
Just for the record: a minor nit I noticed yesterday.
-- Abhijit
>From 07353c86483f7e26d44a9bbe94b32315537cee73 Mon Sep 17 00:00:00 2001
From: Abhijit Menon-Sen
Date: Fri, 29 May 2015 23:15:15 +0530
Subject: The file mode is ignored without O_CREAT, so set it to 0
---
src/backend/storage/file/fd.c | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index b4f6590..2f313ff 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -416,6 +416,7 @@ void
fsync_fname(char *fname, bool isdir)
{
int fd;
+ int flags;
int returncode;
/*
@@ -423,14 +424,13 @@ fsync_fname(char *fname, bool isdir)
* systems don't allow us to fsync files opened read-only; so we need both
* cases here
*/
+ flags = PG_BINARY;
if (!isdir)
- fd = OpenTransientFile(fname,
- O_RDWR | PG_BINARY,
- S_IRUSR | S_IWUSR);
+ flags |= O_RDWR;
else
- fd = OpenTransientFile(fname,
- O_RDONLY | PG_BINARY,
- S_IRUSR | S_IWUSR);
+ flags |= O_RDONLY;
+
+ fd = OpenTransientFile(fname, flags, 0);
/*
* Some OSs don't allow us to open directories at all (Windows returns
--
1.9.1
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgindent vs emacs
On 2015-05-29 13:37:40 -0400, Andrew Dunstan wrote: > One of the annoying inconsistencies between emacs and pgindent is that emacs > refuses to offset a block following a case label, while pgindent does. Is > there anything we can do to induce emacs to do what pgindent does? Are you using the logic from src/tools/editors/emacs.samples I don't see that problem here. I've further tuned my emacs for pPG, but afaics nothing but relevant for this but the above. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
Andres Freund writes: > On 2015-05-29 13:14:18 -0400, Tom Lane wrote: >> Abhijit Menon-Sen writes: >> As I mentioned yesterday, I'm not really on board with ignoring EACCES, >> except for the directories-on-Windows case. Since we're only logging >> the failures anyway, I think it is reasonable to log a complaint for any >> unwritable file in the data directory. > That sounds like a potentially nontrivial amount of repetitive log bleat > after every crash start? One which the user can't really stop? Why can't the user stop it? We won't be bleating about the case of a symlink to a non-writable file someplace else, which is the Debian use case. I don't see a very good excuse to have a non-writable file right in the data directory. >> Also I want to get rid of the ETXTBSY special cases. That one doesn't >> seem like something that we should silently ignore: what the heck are >> executables doing in the data directory? Or is there some other meaning >> on Windows? > I've seen a bunch of binaries placed in the data directory as > archive/restore commands. Those will be busy a good amount of the > time. While it'd not be my choice to do that, it's not entirely > unreasonable. I'd say it's a pretty damn-fool arrangement: for starters, it's an unnecessary security hazard. In any case, if the cost of such a file is one more line of log output during a crash restart, most people would have no problem at all in ignoring that log output. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 05/29/2015 10:45 AM, Stephen Frost wrote: > Andres, > > * Andres Freund (and...@anarazel.de) wrote: >> On 2015-05-29 10:15:56 -0700, Josh Berkus wrote: >>> pg_drop_replication_slot() can be a time-critical function when the >>> master is running out of disk space because the replica is falling >>> behind. >> >> I don't buy this argument. The same is true for DROP TABLE, TRUNCATE, >> DROP DATABASE etc. > > I disagree about that being the same. > >> I mean, I agree it'd be convenient, but I can't see it as "critical". So, here's they scenario: 1. you're almost out of disk space due to a replica falling behind, like down to 16mb left. Or maybe you are out of disk space. 2. You need to drop the laggy replication slots in a hurry to get your master working again. 3. Now you have to do this timing-sensitive two-stage drop to make it work. When our users are having production emergencies, I don't think that it's helpful for us to make the process of getting out of those situations more complicated than it absolutely has to be. > Just a random thought- do we check the LOGIN attribute for replication > connections? If so, you could tweak that, but that may be an issue if > you have multiple replicas using the same role. > > I'm not sure that it's *critical*, but I could see an argument for > adding this post-feature-freeze, which I'm guessing is what Josh was > getting at. Well, I'll let others decide that. If we could come up with a script which would reliably do the terminate-then-drop, it would be fine for 9.5. I'm not sure that's possible though, because I don't see any way to infallibly relate the pg_stat_replication entry with the pg_replication_slot entry. Imagine having 3 slots and 6 replicas, and only one slot is behind; how do you figure out what to terminate? > >>> While I'm just doing this during testing, it could be a critical fail in >>> production. I think the simplest way to resolve this would be to add a >>> boolean flag to pg_drop_replication_slot(), which would terminate the >>> replication connection and delete the slot as a single operation. >> >> There's no "single operation" for terminating a backend *and* doing >> something... > > That's a good point, we'd need to figure out how to make this actually > work reliably in the face of a very fast reconnecting process, if we're > going to do it. Yeah, which means that this is probably something for 9.6. Although if we can at least come up with something for the documentation for 9.5, it would be really helpful. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On 2015-05-29 13:49:16 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2015-05-29 13:14:18 -0400, Tom Lane wrote: > >> Abhijit Menon-Sen writes: > >> As I mentioned yesterday, I'm not really on board with ignoring EACCES, > >> except for the directories-on-Windows case. Since we're only logging > >> the failures anyway, I think it is reasonable to log a complaint for any > >> unwritable file in the data directory. > > > That sounds like a potentially nontrivial amount of repetitive log bleat > > after every crash start? One which the user can't really stop? > > Why can't the user stop it? Because it makes a good amount of sense to have e.g. certificates not owned by postgres and not writeable? You don't necessarily want to symlink them somewhere else, because that makes moving clusters around harder than when they're self contained. > I'd say it's a pretty damn-fool arrangement: for starters, it's an > unnecessary security hazard. I don't buy the security argument at all. You likely have postgresql.conf in the data directoy. You can write to at least .auto, which will definitely reside the data directory. That contains archive_command. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
* Josh Berkus (j...@agliodbs.com) wrote: > 1. you're almost out of disk space due to a replica falling behind, like > down to 16mb left. Or maybe you are out of disk space. This right here is a real issue. What I'd personally like to see is an option which says "you have X GB of disk space. Once it's gone, forget about all replicas or failing archive commands or whatever, and just stop holding on to ancient WAL that you no longer need to operate." Perhaps there would be a warning threshold there too, where you start getting complaints in the log if things are falling too far behind. Ideally, you'd have a monitoring system which is checking for that, but it'd be trivial to include and could be useful for environments that don't have proper monitoring yet. Having this work on the replicas would be nice too. I realize we have time-based constraints there which say "kill off queries which are blocking us from moving forward after X time", but it'd be awful nice to have a size-based way too, to avoid having PG crash when it runs out of space. I have to admit that I'm getting quite tired of the ways in which PG can crash due to out of memory (yes, I know, it's the OOM killer because of a misconfigured Linux box, but still), out of disk space on the master, out of space on the replica, etc, etc. Thanks! Stephen signature.asc Description: Digital signature
[HACKERS] postpone next week's release
Hi, I think we should postpone next week's release. I have been hard at work on the multixact-related bugs that were reported in 9.4.2 and 9.3.7, and the subsequent bugs found by code-reading, but getting them all fixed by Monday doesn't seem realistic. Such fixes should have careful review, and not be dashed into the tree under time pressure. We could do the release anyway to relieve the pain caused by the fsync-pgdata hard-failure problem, but it seems to me that if we do that, we're just going to end up having to do yet another release almost right away. I think it would be better to wait and do one release that fixes both sets of issues. Thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 05/29/2015 11:01 AM, Stephen Frost wrote: > * Josh Berkus (j...@agliodbs.com) wrote: >> > 1. you're almost out of disk space due to a replica falling behind, like >> > down to 16mb left. Or maybe you are out of disk space. > This right here is a real issue. What I'd personally like to see is an > option which says "you have X GB of disk space. Once it's gone, forget > about all replicas or failing archive commands or whatever, and just > stop holding on to ancient WAL that you no longer need to operate." The substantial challenge here is how do we determine that you're "almost out of disk space"? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Thu, May 28, 2015 at 11:01 PM, Fabien COELHO wrote: > > FWIW, I don't mind which one we put in core and which one we put out of >> core. But I like Joshua's idea of getting rid of contribs and pushing them >> out as any other extensions. >> > > Hmmm. > > I like the contrib directory as a living example of "how to do an > extension" directly available in the source tree. It also allows to test > in-tree that the extension mechanism works. So I think it should be kept at > least with a minimum set of dummy examples for this purpose, even if all > current extensions are moved out. > It is mostly an example of "How to do an contrib module" rather than "how to do an extension". There are differences between those things regarding the the USE_PGXS and some other things. If we want to keep it as an example of what we want people to do in the future, it needs be a really good example. And if we want to step new things from going into contrib, we wouldn't want to provide an example of how to put new things into it. > > Also, removing a feature is a regression, and someone is always bound to > complain... What is the real benefit? ISTM that it is a solution that fixes > no important problem. Reaching a consensus about what to move here or there > will consume valuable time that could be spent on more important tasks... > Is it worth it? > Yeah, I don't really see the benefit either. Some could be moved to core, and some could just be removed, but many of them it just seems like we would end up inventing a new 'contrib' to which is the same as the old, but with a different name. Cheers, Jeff
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 2015-05-29 10:53:30 -0700, Josh Berkus wrote: > On 05/29/2015 10:45 AM, Stephen Frost wrote: > So, here's they scenario: > > 1. you're almost out of disk space due to a replica falling behind, like > down to 16mb left. Or maybe you are out of disk space. > > 2. You need to drop the laggy replication slots in a hurry to get your > master working again. > > 3. Now you have to do this timing-sensitive two-stage drop to make it work. How is this measurably worse than trying to truncate a log table that has grown too large? That's often harder to fight actually, because there's dozens of other processes that might be using the relation? In one case you don't have wait ordering, but only one locker, in the other case you have multiple waiters, and to benefit from wait ordering you need multiple sessions. Again, I'm not against improving either situation, it's just that the urgency argument doesn't seem worth its weight. Note that all of this is 9.4 code, not 9.5. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
Andres Freund writes: > On 2015-05-29 13:49:16 -0400, Tom Lane wrote: >> Why can't the user stop it? > Because it makes a good amount of sense to have e.g. certificates not > owned by postgres and not writeable? You don't necessarily want to > symlink them somewhere else, because that makes moving clusters around > harder than when they're self contained. Meh. Well, I'm willing to yield on the EACCES point, but I still find the exclusion for ETXTBSY to be ugly and inappropriate. >> I'd say it's a pretty damn-fool arrangement: for starters, it's an >> unnecessary security hazard. > I don't buy the security argument at all. You likely have > postgresql.conf in the data directoy. You can write to at least .auto, > which will definitely reside the data directory. That contains > archive_command. The fact that a superuser might have multiple ways to subvert things doesn't make it a good idea to add another one: the attack surface could be larger, or at least different. But even if you don't buy that it's a security hazard, why would it be a good idea to have executables inside $PGDATA? That would for example lead to them getting copied by pg_basebackup, which seems unlikely to be a good thing. And if you did have such executables there, why would they be active during a postmaster restart? I really seriously doubt that this is either common enough or useful enough to justify suppressing warning messages about it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
On 2015-05-29 14:15:48 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2015-05-29 13:49:16 -0400, Tom Lane wrote: > >> Why can't the user stop it? > > > Because it makes a good amount of sense to have e.g. certificates not > > owned by postgres and not writeable? You don't necessarily want to > > symlink them somewhere else, because that makes moving clusters around > > harder than when they're self contained. > > Meh. Well, I'm willing to yield on the EACCES point, but I still find > the exclusion for ETXTBSY to be ugly and inappropriate. Ok. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgindent vs emacs
On 05/29/2015 01:49 PM, Andres Freund wrote: On 2015-05-29 13:37:40 -0400, Andrew Dunstan wrote: One of the annoying inconsistencies between emacs and pgindent is that emacs refuses to offset a block following a case label, while pgindent does. Is there anything we can do to induce emacs to do what pgindent does? Are you using the logic from src/tools/editors/emacs.samples I don't see that problem here. I've further tuned my emacs for pPG, but afaics nothing but relevant for this but the above. Hmm, yes, you're right, I was missing something. It also turns out it depends on stuff we can't put in .dir-locals.el. Sorry for the noise. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On Thu, May 28, 2015 at 11:26 PM, Guillaume Lelarge wrote: > Le 29 mai 2015 8:01 AM, "Fabien COELHO" a écrit : > > > > > >> FWIW, I don't mind which one we put in core and which one we put out of > >> core. But I like Joshua's idea of getting rid of contribs and pushing > them > >> out as any other extensions. > > > > > > Hmmm. > > > > I like the contrib directory as a living example of "how to do an > extension" directly available in the source tree. It also allows to test > in-tree that the extension mechanism works. So I think it should be kept at > least with a minimum set of dummy examples for this purpose, even if all > current extensions are moved out. > > > > Agreed. > > > Also, removing a feature is a regression, and someone is always bound to > complain... What is the real benefit? ISTM that it is a solution that fixes > no important problem. Reaching a consensus about what to move here or there > will consume valuable time that could be spent on more important tasks... > Is it worth it? > > > > It would be less confusing for users. Contrib modules seem to be first > class extensions, leaving doubt on other extensions. > Presumably there are still going to be some extensions maintained by -hackers, and some not. I don't think we are going to change that, so the difference will still need to be explained, regardless of what words are used. And people *should* have doubts about other extensions. Couldn't any talented programmer write an extension which gives them a backdoor into the deployer's system, and then upload it to github? I would certainly be cautious about installing any old extension I found some some place on the internet. > But the fact they aren't in core make them not fully trusted by some > users. > Would it help if we called it "base" or "minimal" rather than "core" in the docs? (And called 'contrib' something different as well? The docs already do call it "Additional Supplied Modules" and use "contrib" only when referring the the directory, not the concept.) > Trying to explain all that in a training is a PITA. It would be much less > confusing if they were either in core or in their own repository. > Several of the contrib modules should be kept in tight sync with src or else testing and debugging would be a disaster. Putting them in different git repositories wouldn't work well. Unless those would among the ones moved to "core". Cheers, Jeff
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
* Andres Freund (and...@anarazel.de) wrote: > On 2015-05-29 13:49:16 -0400, Tom Lane wrote: > > > That sounds like a potentially nontrivial amount of repetitive log bleat > > > after every crash start? One which the user can't really stop? > > > > Why can't the user stop it? > > Because it makes a good amount of sense to have e.g. certificates not > owned by postgres and not writeable? You don't necessarily want to > symlink them somewhere else, because that makes moving clusters around > harder than when they're self contained. A certain other file might be non-writable by PG too... (*cough* .auto *cough*). > > I'd say it's a pretty damn-fool arrangement: for starters, it's an > > unnecessary security hazard. > > I don't buy the security argument at all. You likely have > postgresql.conf in the data directoy. You can write to at least .auto, > which will definitely reside the data directory. That contains > archive_command. I'm not sure that I see the security issue here either.. We're not talking about setuid shell scripts or anything that isn't running as the PG user, which a superuser could take over anyway.. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
* Josh Berkus (j...@agliodbs.com) wrote: > On 05/29/2015 11:01 AM, Stephen Frost wrote: > > * Josh Berkus (j...@agliodbs.com) wrote: > >> > 1. you're almost out of disk space due to a replica falling behind, like > >> > down to 16mb left. Or maybe you are out of disk space. > > This right here is a real issue. What I'd personally like to see is an > > option which says "you have X GB of disk space. Once it's gone, forget > > about all replicas or failing archive commands or whatever, and just > > stop holding on to ancient WAL that you no longer need to operate." > > The substantial challenge here is how do we determine that you're > "almost out of disk space"? Eh? That "X GB" above was intended to be the value of a GUC. I know how big my WAL partition is. Let me tell PG how big it is and to not do anything that'll end up going over that amount, and we'll never see a crash due to out of disk space for WAL again. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 05/29/2015 11:07 AM, Andres Freund wrote: > On 2015-05-29 10:53:30 -0700, Josh Berkus wrote: >> On 05/29/2015 10:45 AM, Stephen Frost wrote: >> So, here's they scenario: >> >> 1. you're almost out of disk space due to a replica falling behind, like >> down to 16mb left. Or maybe you are out of disk space. >> >> 2. You need to drop the laggy replication slots in a hurry to get your >> master working again. >> >> 3. Now you have to do this timing-sensitive two-stage drop to make it work. > > How is this measurably worse than trying to truncate a log table that > has grown too large? That's often harder to fight actually, because > there's dozens of other processes that might be using the relation? In > one case you don't have wait ordering, but only one locker, in the other > case you have multiple waiters, and to benefit from wait ordering you > need multiple sessions. > > Again, I'm not against improving either situation, it's just that the > urgency argument doesn't seem worth its weight. Well, I wouldn't mind a solution for drop table and drop database, either. I'm pretty sure that's on our TODO list. Oh, I see the confusion. When I say "time-critical", I was referring to the situation where someone is running out of disk space. Not coming up with a patch. AFAIK, hardly anyone is using replication slots, still. > > Note that all of this is 9.4 code, not 9.5. Yes, but I'm not suggesting backporting it, just maybe a backported doc patch. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
* Andres Freund (and...@anarazel.de) wrote: > How is this measurably worse than trying to truncate a log table that > has grown too large? That's often harder to fight actually, because > there's dozens of other processes that might be using the relation? In > one case you don't have wait ordering, but only one locker, in the other > case you have multiple waiters, and to benefit from wait ordering you > need multiple sessions. Because we don't fall over if we can't extend a relation. We do fall over if we can't write WAL. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] RFC: Remove contrib entirely
All, So there are currently three kinds of things in contrib: A. Extra commands and tools which aren't considered general enough, or reliable enough, to be included by default, e.g. pg_standby, pgbench and vacuumlo. B. Developer tools, like spi, start-scripts, and oid2name. C. "Core Extensions", which fall into three further groups: C1: encryption extensions we can't include in core for legal reasons (pg_crypto) C2: example extensions which show useful things about how to build an extension C3: Admin extensions which are not core because they carry risks (e.g. pgstattuple, auto_explain) C4: Extensions which are generally useful, used, and maintained with Postgres (e.g. hstore, citext) So A and B need to stay somewhere in the source distribution. I would like to see them go into /admin-tools and /developer-tools directories; I believe that Greg Smith came up with a patch to do just this at sometime in the past. C2 could also go into /developer-tools, and C3 are really just more admin-tools. C1 would need to stick around as a special case. To move C4, you'd have to solve the problem of "how do we turn a former external extension into a core feature", which nobody yet has solved. All of this ignores the critical part of this, which is packaging. Right now packagers ship a "contrib" package which includes everything in /contrib. Shifting to having any other arrangement is going to involve working with them and convincing them that a change to packaging is worthwhile. And then getting the news to our users. Given that, there needs to be significant benefit to our users in the change. So, what's the benefit? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 02:02:43PM -0400, Robert Haas wrote: > Hi, > > I think we should postpone next week's release. I have been hard at > work on the multixact-related bugs that were reported in 9.4.2 and > 9.3.7, and the subsequent bugs found by code-reading, but getting them > all fixed by Monday doesn't seem realistic. Such fixes should have > careful review, and not be dashed into the tree under time pressure. > > We could do the release anyway to relieve the pain caused by the > fsync-pgdata hard-failure problem, but it seems to me that if we do > that, we're just going to end up having to do yet another release > almost right away. I think it would be better to wait and do one > release that fixes both sets of issues. It does seem wise to make sure we have all these items fixed. We have PR'ed the recovery failure issue so I think we are good at this point. I see having to put out another multi-xact-only fix release the week after as being a bigger negative. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postpone next week's release
* Robert Haas (robertmh...@gmail.com) wrote: > I think we should postpone next week's release. I have been hard at > work on the multixact-related bugs that were reported in 9.4.2 and > 9.3.7, and the subsequent bugs found by code-reading, but getting them > all fixed by Monday doesn't seem realistic. Such fixes should have > careful review, and not be dashed into the tree under time pressure. > > We could do the release anyway to relieve the pain caused by the > fsync-pgdata hard-failure problem, but it seems to me that if we do > that, we're just going to end up having to do yet another release > almost right away. I think it would be better to wait and do one > release that fixes both sets of issues. Agreed. I just caution that we appreciate PGCon coming up and that we do our best to avoid running into a case where we have to push it further due to everyone being at the conference. Thanks! Stephen signature.asc Description: Digital signature
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 02:54:31PM -0400, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > I think we should postpone next week's release. I have been hard at > > work on the multixact-related bugs that were reported in 9.4.2 and > > 9.3.7, and the subsequent bugs found by code-reading, but getting them > > all fixed by Monday doesn't seem realistic. Such fixes should have > > careful review, and not be dashed into the tree under time pressure. > > > > We could do the release anyway to relieve the pain caused by the > > fsync-pgdata hard-failure problem, but it seems to me that if we do > > that, we're just going to end up having to do yet another release > > almost right away. I think it would be better to wait and do one > > release that fixes both sets of issues. > > Agreed. > > I just caution that we appreciate PGCon coming up and that we do our > best to avoid running into a case where we have to push it further due > to everyone being at the conference. This brings up the issue of when we want to do 9.5 beta. Ideas? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously
Re: Tom Lane 2015-05-29 <13871.1432921...@sss.pgh.pa.us> > Why can't the user stop it? We won't be bleating about the case of a > symlink to a non-writable file someplace else, which is the Debian use > case. I don't see a very good excuse to have a non-writable file right > in the data directory. I've repeatedly seen PGDATA or pg_xlog been put directly on a mountpoint, which means there well be a non-writable lost+found directory there. (A case with pg_xlog was also reported as a support case at credativ.) I'm usually advising against using the top level directory directly, but it's not uncommon to encounter it. > In any case, if the cost of such a file is one more line of log output > during a crash restart, most people would have no problem at all in > ignoring that log output. Nod. Christoph -- c...@df7cb.de | http://www.df7cb.de/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
So, here's an example of why it's hard to give our users a workaround.
cio=# select * from pg_replication_slots;
slot_name| plugin | slot_type | datoid |
database | active | xmin | catalog_xmin | restart_lsn
-++---++--++--+--+-
bdr_24577_6147720645156311471_1_26507__ | bdr| logical | 24577 |
cio | t | | 906 | 0/1C4F410
(1 row)
cio=# select
pg_drop_replication_slot('bdr_24577_6147720645156311471_1_26507__');
ERROR: replication slot "bdr_24577_6147720645156311471_1_26507__" is
already active
cio=# select * from pg_stat_replication;
-[ RECORD 1 ]+---
pid | 28481
usesysid | 10
usename | postgres
application_name | bdr (6147720645156311471,1,26507,):receive
client_addr | 172.17.0.11
client_hostname |
client_port | 44583
backend_start| 2015-05-29 18:10:34.601796+00
backend_xmin |
state| streaming
sent_location| 0/1C4F448
write_location | 0/1C4F448
flush_location | 0/1C4F448
replay_location | 0/1C4F448
sync_priority| 0
sync_state | async
Now, BDR is good because it sets an application_name which lets me
figure out what's using the replication slot. But that's by no means
required; other LC plug-ins, I expect, do not do so. So there's no way
for the user to figure out which replication connection relates to which
slots, as far as I can tell.
In this test, it's easy because there's only one replication connection
and one slot. But imagine the case of 14 replication connections with
their own slots. How could you possibly figure out which one was the
laggy one?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1
On Fri, May 29, 2015 at 12:43 PM, Robert Haas wrote: > Working on that now. OK, here's a patch. Actually two patches, differing only in whitespace, for 9.3 and for master (ha!). I now think that the root of the problem here is that DetermineSafeOldestOffset() and SetMultiXactIdLimit() were largely ignorant of the possibility that they might be called at points in time when the cluster was inconsistent. SetMultiXactIdLimit() bracketed certain parts of its logic with if (!InRecovery), but those guards were ineffective because it gets called before InRecovery is set in the first place. It seems pretty clear that we can't effectively determine anything about member wraparound until the cluster is consistent. Before then, there might be files missing from the offsets or members SLRUs which get put back during replay. There could even be gaps in the sequence of files, with some things having made it to disk before the crash (or having made it into the backup) and others not. So all the work of determining what the safe stop points and vacuum thresholds for members are needs to be postponed until TrimMultiXact() time. And that's fine, because we don't need this information in recovery anyway - it only affects behavior in normal running. So this patch does the following: 1. Moves the call to DetermineSafeOldestOffset() that appears in StartupMultiXact() to TrimMultiXact(), so that we don't try to do this until we're consistent. Also, instead of passing MultiXactState->oldestMultiXactId, pass the newer of that value and the earliest offset that exists on disk. That way, it won't try to read data that's not there. Note that the second call to DetermineSafeOldestOffset() in TruncateMultiXact() doesn't need a similar guard, because we already bail out of that function early if the multixacts we're going to truncate away don't exist. 2. Adds a new flag MultiXactState->didTrimMultiXact indicate whether we've finished TrimMultiXact(), and arranges for SetMultiXactIdLimit() to use that rather than InRecovery to test whether it's safe to do complicated things that might require that the cluster is consistent. This is a slight behavior change, since formerly we would have tried to do that stuff very early in the startup process, and now it won't happen until somebody completes a vacuum operation. If that's a problem, we could consider doing it in TrimMultiXact(), but I don't think it's safe the way it was. The new flag also prevents oldestOffset from being set while in recovery; I think it would be safe to do that in recovery once we've reached consistency, but I don't believe it's necessary. 3. Arranges for TrimMultiXact() to set oldestOffset. This is necessary because, previously, we relied on SetMultiXactIdLimit doing that during early startup or during recovery, and that's no longer true. Here too we set oldestOffset keeping in mind that our notion of the oldest multixact may point to something that doesn't exist; if so, we use the oldest MXID that does. 4. Modifies TruncateMultiXact() so that it doesn't re-scan the SLRU directory on every call to find the oldest file that exists. Instead, it arranges to remember the value from the first scan and then updates it thereafter to reflect its own truncation activity. This isn't absolutely necessary, but because this oldest-file logic is used in multiple places (TrimMultiXact, SetMultiXactIdLimit, and TruncateMultiXact all need it directly or indirectly) caching the value seems like a better idea than recomputing it frequently. I have tested that this patch fixes Steve Kehlet's problem, or at least what I believe to be Steve Kehlet's problem based on the reproduction scenario I described upthread. I believe it will also fix the problems with starting up from a base backup with Alvaro mentioned upthread. It won't fix the fact that pg_upgrade is putting a wrong value into everybody's datminmxid field, which should really be addressed too, but I've been working on this for about three days virtually non-stop and I don't have the energy to tackle it right now. If anyone feels the urge to step into that breech, I think what it needs to do is: when upgrading from a 9.3-or-later instance, copy over each database's datminmxid into the corresponding database in the new cluster. Aside from that, it's very possible that despite my best efforts this has serious bugs. Review and testing would be very much appreciated. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index 699497c..8d28a5c 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -197,8 +197,9 @@ typedef struct MultiXactStateData MultiXactOffset nextOffset; /* - * Oldest multixact that is still on disk. Anything older than this - * should not be consulted. These values are updated by vacuum. + * Old
Re: [HACKERS] [CORE] postpone next week's release
On Fri, May 29, 2015 at 8:02 PM, Robert Haas wrote: > Hi, > > I think we should postpone next week's release. I have been hard at > work on the multixact-related bugs that were reported in 9.4.2 and > 9.3.7, and the subsequent bugs found by code-reading, but getting them > all fixed by Monday doesn't seem realistic. Such fixes should have > careful review, and not be dashed into the tree under time pressure. > > We could do the release anyway to relieve the pain caused by the > fsync-pgdata hard-failure problem, but it seems to me that if we do > that, we're just going to end up having to do yet another release > almost right away. I think it would be better to wait and do one > release that fixes both sets of issues. > > Thoughts? > I'm a bit split on this. We *definitely* don't want to release the multixact fix without it being carefully reviewed, that's the part I'm not split about :) And I fully appreciate we can't have that done by monday. However, the file-permission thing seems to hit quite a few people (have we ever had this many bug reports after a minor release), which means wed really want to get that out quickly. Do you have any feeling of how likely people are to actually hit the multixact one? I've followed some of that impressive debugging you guys did, and I know it's a pretty critical bug if you hit it, but how wide-spread will it be? I guess one option we could do is encourage packagers to push updated packages (-2 versions) basically. But if we do that, perhaps we might as well release anyway? AIUI, the permission thing won't actually be very likely to affect Windows users. And Windows packages are the ones that take by far the most work to make. Perhaps we should consider skipping making packages of that version on Windows, and then plan to push yet another minor one or two weeks later, that goes out on all platforms? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 8:54 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > I think we should postpone next week's release. I have been hard at > > work on the multixact-related bugs that were reported in 9.4.2 and > > 9.3.7, and the subsequent bugs found by code-reading, but getting them > > all fixed by Monday doesn't seem realistic. Such fixes should have > > careful review, and not be dashed into the tree under time pressure. > > > > We could do the release anyway to relieve the pain caused by the > > fsync-pgdata hard-failure problem, but it seems to me that if we do > > that, we're just going to end up having to do yet another release > > almost right away. I think it would be better to wait and do one > > release that fixes both sets of issues. > > Agreed. > > I just caution that we appreciate PGCon coming up and that we do our > best to avoid running into a case where we have to push it further due > to everyone being at the conference. > If we plan it, we certainly *can* make a release during pgcon. If that's what the reasonable timing comes down to, I think getting these fixes out definitely has to be considered more important than the conference, so a few of us will just have to take a break... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [HACKERS] RFC: Remove contrib entirely
On 05/29/2015 11:02 AM, Jeff Janes wrote: Also, removing a feature is a regression, and someone is always bound to complain... What is the real benefit? ISTM that it is a solution that fixes no important problem. Reaching a consensus about what to move here or there will consume valuable time that could be spent on more important tasks... Is it worth it? Yeah, I don't really see the benefit either. Some could be moved to core, and some could just be removed, but many of them it just seems like we would end up inventing a new 'contrib' to which is the same as the old, but with a different name. Name one. Sincerely, JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own emotions, so everyone else should do it for you. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [CORE] postpone next week's release
On Fri, May 29, 2015 at 3:09 PM, Magnus Hagander wrote: > Do you have any feeling of how likely people are to actually hit the > multixact one? I've followed some of that impressive debugging you guys did, > and I know it's a pretty critical bug if you hit it, but how wide-spread > will it be? That precise problem has been reported a few times, but it may not be widespread. I don't know. My bigger concern is that, at present, taking a base backup is broken. I haven't figured out the exact reproduction scenario, but I think it's something like this: - begin base backup - checkpoint happens, truncating pg_multixact - at this point pg_multixact gets copied - end base backup I think what will happen on replay is that replaying the checkpoint, it will try to reference pg_multixact files that don't exist any more and die with a fatal error. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
On 05/29/2015 11:27 AM, Jeff Janes wrote: It would be less confusing for users. Contrib modules seem to be first class extensions, leaving doubt on other extensions. Presumably there are still going to be some extensions maintained by -hackers, and some not. I don't think we are going to change that, so the difference will still need to be explained, regardless of what words are used. And people *should* have doubts about other extensions. Couldn't any talented programmer write an extension which gives them a backdoor into the deployer's system, and then upload it to github? Yes, it is called Open Source development. I would certainly be cautious about installing any old extension I found some some place on the internet. But the fact they aren't in core make them not fully trusted by some users. No. This is completely wrong thinking. If you are compiling this stuff from source you are taking that risk on yourself. Most people are not compiling from source, they are installing from a distribution (or downloading a distribution package). Trying to explain all that in a training is a PITA. It would be much less confusing if they were either in core or in their own repository. Several of the contrib modules should be kept in tight sync with src or else testing and debugging would be a disaster. Putting them in different git repositories wouldn't work well. Unless those would among the ones moved to "core". Note: I actually don't care if the current contrib gets pushed into core proper and is default installed. I care about this idea that contrib exists. It isn't needed and leads to a discussion like this one (or the pg_audit), almost every release. Contrib made sense years ago. It does not any longer. Let's put the old horse down and raise a new herd of ponies on a new pasture. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own emotions, so everyone else should do it for you. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 05/29/2015 12:08 PM, Josh Berkus wrote: Now, BDR is good because it sets an application_name which lets me figure out what's using the replication slot. But that's by no means required; other LC plug-ins, I expect, do not do so. So there's no way for the user to figure out which replication connection relates to which slots, as far as I can tell. In this test, it's easy because there's only one replication connection and one slot. But imagine the case of 14 replication connections with their own slots. How could you possibly figure out which one was the laggy one? The client_addr? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own emotions, so everyone else should do it for you. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 2015-05-29 14:39:02 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > How is this measurably worse than trying to truncate a log table that > > has grown too large? That's often harder to fight actually, because > > there's dozens of other processes that might be using the relation? In > > one case you don't have wait ordering, but only one locker, in the other > > case you have multiple waiters, and to benefit from wait ordering you > > need multiple sessions. > > Because we don't fall over if we can't extend a relation. > > We do fall over if we can't write WAL. As nearly everybody uses the same filesystem for pg_xlog and the actual databases, that distinction isn't worth much. You'll still fail when writing the WAL, even if the disk space has been used by a relation instead of WAL. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 2015-05-29 12:08:24 -0700, Josh Berkus wrote: > Now, BDR is good because it sets an application_name which lets me > figure out what's using the replication slot. But that's by no means > required; other LC plug-ins, I expect, do not do so. So there's no way > for the user to figure out which replication connection relates to which > slots, as far as I can tell. > > In this test, it's easy because there's only one replication connection > and one slot. But imagine the case of 14 replication connections with > their own slots. How could you possibly figure out which one was the > laggy one? 9.5 shows the pid. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
* Magnus Hagander (mag...@hagander.net) wrote: > On Fri, May 29, 2015 at 8:54 PM, Stephen Frost wrote: > > > * Robert Haas (robertmh...@gmail.com) wrote: > > > I think we should postpone next week's release. I have been hard at > > > work on the multixact-related bugs that were reported in 9.4.2 and > > > 9.3.7, and the subsequent bugs found by code-reading, but getting them > > > all fixed by Monday doesn't seem realistic. Such fixes should have > > > careful review, and not be dashed into the tree under time pressure. > > > > > > We could do the release anyway to relieve the pain caused by the > > > fsync-pgdata hard-failure problem, but it seems to me that if we do > > > that, we're just going to end up having to do yet another release > > > almost right away. I think it would be better to wait and do one > > > release that fixes both sets of issues. > > > > Agreed. > > > > I just caution that we appreciate PGCon coming up and that we do our > > best to avoid running into a case where we have to push it further due > > to everyone being at the conference. > > If we plan it, we certainly *can* make a release during pgcon. If that's > what the reasonable timing comes down to, I think getting these fixes out > definitely has to be considered more important than the conference, so a > few of us will just have to take a break... I don't disagree with you about any of that, just wanted to make mention of the timing. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 21:20 GMT+02:00 Joshua D. Drake : > > On 05/29/2015 11:27 AM, Jeff Janes wrote: > > It would be less confusing for users. Contrib modules seem to be >> first class extensions, leaving doubt on other extensions. >> >> >> Presumably there are still going to be some extensions maintained by >> -hackers, and some not. I don't think we are going to change that, so >> the difference will still need to be explained, regardless of what words >> are used. And people *should* have doubts about other extensions. >> Couldn't any talented programmer write an extension which gives them a >> backdoor into the deployer's system, and then upload it to github? >> > > Yes, it is called Open Source development. > > >> I would certainly be cautious about installing any old extension I found >> some some place on the internet. >> >> But the fact they aren't in core make them not fully trusted by some >> users. >> > > No. This is completely wrong thinking. If you are compiling this stuff > from source you are taking that risk on yourself. > > Most people are not compiling from source, they are installing from a > distribution (or downloading a distribution package). > > >> Trying to explain all that in a training is a PITA. It would be much >> less confusing if they were either in core or in their own repository. >> >> Several of the contrib modules should be kept in tight sync with src or >> else testing and debugging would be a disaster. Putting them in >> different git repositories wouldn't work well. Unless those would among >> the ones moved to "core". >> >> > Note: I actually don't care if the current contrib gets pushed into core > proper and is default installed. > > I care about this idea that contrib exists. It isn't needed and leads to a > discussion like this one (or the pg_audit), almost every release. > > Contrib made sense years ago. It does not any longer. Let's put the old > horse down and raise a new herd of ponies on a new pasture. > Still there is strong sense - it is a referential implementation of our extension API. We need it to find regressions, changes. I don't believe so external extensions can do it. Only PostGIS is massively accepted and developed by more than few people. Personally I am thinking so removing contrib is not good idea. Pavel > > JD > > -- > Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 > PostgreSQL Centered full stack support, consulting and development. > Announcing "I'm offended" is basically telling the world you can't > control your own emotions, so everyone else should do it for you. > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] [CORE] postpone next week's release
On 05/29/2015 12:18 PM, Robert Haas wrote: On Fri, May 29, 2015 at 3:09 PM, Magnus Hagander wrote: Do you have any feeling of how likely people are to actually hit the multixact one? I've followed some of that impressive debugging you guys did, and I know it's a pretty critical bug if you hit it, but how wide-spread will it be? That precise problem has been reported a few times, but it may not be widespread. I don't know. My bigger concern is that, at present, taking a base backup is broken. This I think is the bigger issue. They both are horrible but basebackup being broken is rather... egregious. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own emotions, so everyone else should do it for you. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
Magnus Hagander writes: > On Fri, May 29, 2015 at 8:54 PM, Stephen Frost wrote: >> I just caution that we appreciate PGCon coming up and that we do our >> best to avoid running into a case where we have to push it further due >> to everyone being at the conference. > If we plan it, we certainly *can* make a release during pgcon. If that's > what the reasonable timing comes down to, I think getting these fixes out > definitely has to be considered more important than the conference, so a > few of us will just have to take a break... I think there's no way that we wait more than one additional week to push the fsync fix. So the problem is not with scheduling the update releases, it's with whether we can also fit in a 9.5 beta release before PGCon. (I can't see doing a beta *during* PGCon week. I for one am going to be on an airplane at the time I'd normally have to be Doing Release Stuff.) I know Josh doesn't like to do beta1 releases concurrently with back branches because it confuses the PR messaging. But we could make an exception perhaps; or do all those releases the same week but announce the beta the day after the bugfix releases. Or we just let the beta slide till after PGCon, but then I think we're missing some excitement factor. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 9:32 PM, Tom Lane wrote: > Magnus Hagander writes: > > On Fri, May 29, 2015 at 8:54 PM, Stephen Frost > wrote: > >> I just caution that we appreciate PGCon coming up and that we do our > >> best to avoid running into a case where we have to push it further due > >> to everyone being at the conference. > > > If we plan it, we certainly *can* make a release during pgcon. If that's > > what the reasonable timing comes down to, I think getting these fixes out > > definitely has to be considered more important than the conference, so a > > few of us will just have to take a break... > > I think there's no way that we wait more than one additional week to push > the fsync fix. So the problem is not with scheduling the update releases, > it's with whether we can also fit in a 9.5 beta release before PGCon. > I think 9.5 beta has to stand back. The question is what we do with the potentially two minor releases. Then we can slot in the beta whenever. If we do the minor as currently planned, can we do another one the week after to deal with the multixact issues? (scheduling wise we're going to have to do one the week after *regardless*, the question is if we can make two different ones, or if we need to fold them into one) (I can't see doing a beta *during* PGCon week. I for one am going to be > on an airplane at the time I'd normally have to be Doing Release Stuff.) > Agreed. We can push a *minor* during pgcon, but not beta. I know Josh doesn't like to do beta1 releases concurrently with back > branches because it confuses the PR messaging. But we could make an > exception perhaps; or do all those releases the same week but announce > the beta the day after the bugfix releases. > I can't comment on the PR parts, I'll leave that to Josh. > > Or we just let the beta slide till after PGCon, but then I think we're > missing some excitement factor. > Well, most of the people going to pgcon know it already. And most of the excitement affects people who are not at pgcon (simply based on that most of our users are not at pgcon). If doing it the week after pgcon is what ends up making sense once weve figured out what to do with the minors, then so be it, IMNSHO. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [CORE] [HACKERS] postpone next week's release
* Tom Lane (t...@sss.pgh.pa.us) wrote: > (I can't see doing a beta *during* PGCon week. I for one am going to be > on an airplane at the time I'd normally have to be Doing Release Stuff.) [...] > Or we just let the beta slide till after PGCon, but then I think we're > missing some excitement factor. Personally, I'd be all for a "watch Tom do the 9.5 beta release!" Unconference slot... :) (mostly kidding, but I'm 100% sure it'd draw a huge crowd..) Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 05/29/2015 12:27 PM, Andres Freund wrote: > On 2015-05-29 12:08:24 -0700, Josh Berkus wrote: >> Now, BDR is good because it sets an application_name which lets me >> figure out what's using the replication slot. But that's by no means >> required; other LC plug-ins, I expect, do not do so. So there's no way >> for the user to figure out which replication connection relates to which >> slots, as far as I can tell. >> >> In this test, it's easy because there's only one replication connection >> and one slot. But imagine the case of 14 replication connections with >> their own slots. How could you possibly figure out which one was the >> laggy one? > > 9.5 shows the pid. OK, will test, thanks. --Josh Berkus -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
Magnus Hagander writes: > On Fri, May 29, 2015 at 9:32 PM, Tom Lane wrote: >> I think there's no way that we wait more than one additional week to push >> the fsync fix. So the problem is not with scheduling the update releases, >> it's with whether we can also fit in a 9.5 beta release before PGCon. > I think 9.5 beta has to stand back. The question is what we do with the > potentially two minor releases. Then we can slot in the beta whenever. > If we do the minor as currently planned, can we do another one the week > after to deal with the multixact issues? (scheduling wise we're going to > have to do one the week after *regardless*, the question is if we can make > two different ones, or if we need to fold them into one) I suppose we could, but it doubles the amount of release gruntwork involved, and it doesn't exactly make us look good to our users either. I believe Christoph indicated that he was going to cherry-pick the fsync patch and push out an intermediate Debian package with that fix, so at least for that community there is not an urgent reason to get out a set of releases with only the fsync fixes and not the multixact fixes. I'm not clear though on how many of the other reports we heard came from Debian users. (Some of them did, but maybe not all.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Magnus Hagander writes: > > On Fri, May 29, 2015 at 9:32 PM, Tom Lane wrote: > >> I think there's no way that we wait more than one additional week to push > >> the fsync fix. So the problem is not with scheduling the update releases, > >> it's with whether we can also fit in a 9.5 beta release before PGCon. > > > I think 9.5 beta has to stand back. The question is what we do with the > > potentially two minor releases. Then we can slot in the beta whenever. > > > If we do the minor as currently planned, can we do another one the week > > after to deal with the multixact issues? (scheduling wise we're going to > > have to do one the week after *regardless*, the question is if we can make > > two different ones, or if we need to fold them into one) > > I suppose we could, but it doubles the amount of release gruntwork > involved, and it doesn't exactly make us look good to our users either. Agreed. Makes it look like we can't manage to figure out our bugs and put fixes for them together in sensible releases.. Thanks! Stephen signature.asc Description: Digital signature
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 9:46 PM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Magnus Hagander writes: > > > On Fri, May 29, 2015 at 9:32 PM, Tom Lane wrote: > > >> I think there's no way that we wait more than one additional week to > push > > >> the fsync fix. So the problem is not with scheduling the update > releases, > > >> it's with whether we can also fit in a 9.5 beta release before PGCon. > > > > > I think 9.5 beta has to stand back. The question is what we do with the > > > potentially two minor releases. Then we can slot in the beta whenever. > > > > > If we do the minor as currently planned, can we do another one the week > > > after to deal with the multixact issues? (scheduling wise we're going > to > > > have to do one the week after *regardless*, the question is if we can > make > > > two different ones, or if we need to fold them into one) > > > > I suppose we could, but it doubles the amount of release gruntwork > > involved, and it doesn't exactly make us look good to our users either. > > Agreed. Makes it look like we can't manage to figure out our bugs and > put fixes for them together in sensible releases.. > The flipside of that is that we have a bug fix that's preventing peoples databases from starting, and we're the intentionally delaying the shipment of it. Though i guess a mitigating fact there is that it is very easy to manually recover from that. But it's painful if your db server restarts awhen you're not around... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1
On Thu, May 28, 2015 at 07:24:26PM -0400, Robert Haas wrote: > On Thu, May 28, 2015 at 4:06 PM, Joshua D. Drake > wrote: > > FTR: Robert, you have been a Samurai on this issue. Our many thanks. > > Thanks! I really appreciate the kind words. > > So, in thinking through this situation further, it seems to me that > the situation is pretty dire: > > 1. If you pg_upgrade to 9.3 before 9.3.5, then you may have relminmxid > or datminmxid values which are 1 instead of the correct value. > Setting the value to 1 was too far in the past if your MXID counter is > < 2B, and too far in the future if your MXID counter is > 2B. > > 2. If you pg_upgrade to 9.3.7 or 9.4.2, then you may have datminmxid > values which are equal to the next-mxid counter instead of the correct > value; in other words, they are two new. > > 3. If you pg_upgrade to 9.3.5, 9.3.6, 9.4.0, or 9.4.1, then you will > have the first problem for tables in template databases, and the > second one for the rest. (See 866f3017a.) I think we need to step back and look at the brain power required to unravel the mess we have made regarding multi-xact and fixes. (I bet few people can even remember which multi-xact fixes went into which releases --- I can't.) Instead of working on actual features, we are having to do this complex diagnosis because we didn't do a thorough analysis at the time a pattern of multi-xact bugs started to appear. Many projects deal with this compound bug debt regularly, but we have mostly avoided this fate. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
* Magnus Hagander (mag...@hagander.net) wrote: > On Fri, May 29, 2015 at 9:46 PM, Stephen Frost wrote: > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > Magnus Hagander writes: > > > > On Fri, May 29, 2015 at 9:32 PM, Tom Lane wrote: > > > >> I think there's no way that we wait more than one additional week to > > push > > > >> the fsync fix. So the problem is not with scheduling the update > > releases, > > > >> it's with whether we can also fit in a 9.5 beta release before PGCon. > > > > > > > I think 9.5 beta has to stand back. The question is what we do with the > > > > potentially two minor releases. Then we can slot in the beta whenever. > > > > > > > If we do the minor as currently planned, can we do another one the week > > > > after to deal with the multixact issues? (scheduling wise we're going > > to > > > > have to do one the week after *regardless*, the question is if we can > > make > > > > two different ones, or if we need to fold them into one) > > > > > > I suppose we could, but it doubles the amount of release gruntwork > > > involved, and it doesn't exactly make us look good to our users either. > > > > Agreed. Makes it look like we can't manage to figure out our bugs and > > put fixes for them together in sensible releases.. > > > > The flipside of that is that we have a bug fix that's preventing peoples > databases from starting, and we're the intentionally delaying the shipment > of it. Though i guess a mitigating fact there is that it is very easy to > manually recover from that. But it's painful if your db server restarts > awhen you're not around... And we have *another* fix for a *data corruption* bug which is coming in the following *week*. Yes, I think delaying a week to get both in is better than putting out a fix for one bug when we *know* there's a data corruption bug sitting in that code, and we're putting out a fix for it the following week. If we were talking about a month-long delay, that'd be one thing, but that isn't the impression I've got about what we're talking about. Thanks! Stephen signature.asc Description: Digital signature
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 03:32:57PM -0400, Tom Lane wrote: > I know Josh doesn't like to do beta1 releases concurrently with back > branches because it confuses the PR messaging. But we could make an > exception perhaps; or do all those releases the same week but announce > the beta the day after the bugfix releases. > > Or we just let the beta slide till after PGCon, but then I think we're > missing some excitement factor. I am unclear if we are anywhere near ready for beta1 even in June. Are we? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
* Bruce Momjian (br...@momjian.us) wrote: > On Fri, May 29, 2015 at 03:32:57PM -0400, Tom Lane wrote: > > I know Josh doesn't like to do beta1 releases concurrently with back > > branches because it confuses the PR messaging. But we could make an > > exception perhaps; or do all those releases the same week but announce > > the beta the day after the bugfix releases. > > > > Or we just let the beta slide till after PGCon, but then I think we're > > missing some excitement factor. > > I am unclear if we are anywhere near ready for beta1 even in June. Are > we? I'm all about having that discussion... but can we do it on another thread or at least wait til we've decided about the back-branch releases? They are clearly the more important issue to consider. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] RFC: Remove contrib entirely
On 05/29/2015 12:30 PM, Pavel Stehule wrote: Contrib made sense years ago. It does not any longer. Let's put the old horse down and raise a new herd of ponies on a new pasture. Still there is strong sense - it is a referential implementation of our extension API. We need it to find regressions, changes. I don't believe No, then we need a proper test suite for the extension API. so external extensions can do it. Only PostGIS is massively accepted and developed by more than few people. Personally I am thinking so removing contrib is not good idea. Is there an extension/contrib module in the last decade that more than once has shown to help us with that? Sincerely, JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Centered full stack support, consulting and development. Announcing "I'm offended" is basically telling the world you can't control your own emotions, so everyone else should do it for you. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
Stephen Frost writes: > * Bruce Momjian (br...@momjian.us) wrote: >> I am unclear if we are anywhere near ready for beta1 even in June. Are >> we? > I'm all about having that discussion... but can we do it on another > thread or at least wait til we've decided about the back-branch > releases? They are clearly the more important issue to consider. It's the same discussion though, ie what releases are we expecting to get out in the next couple of weeks. It's possible that we ought to give up on a pre-conference beta. Certainly a whole lot of time that I'd hoped would go into reviewing 9.5 feature commits has instead gone into back-branch bug chasing this week. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
* Tom Lane (t...@sss.pgh.pa.us) wrote: > It's possible that we ought to give up on a pre-conference beta. > Certainly a whole lot of time that I'd hoped would go into reviewing > 9.5 feature commits has instead gone into back-branch bug chasing this > week. I guess that's what I'm getting at. We need to take care of the back-branches and that means pushing beta back. I fully expect a good discussion on when to release beta when we get closer on that, but we're not going to be close while we have outstanding big back-branch bugs. Thanks! Stephen signature.asc Description: Digital signature
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 04:01:00PM -0400, Tom Lane wrote: > Stephen Frost writes: > > * Bruce Momjian (br...@momjian.us) wrote: > >> I am unclear if we are anywhere near ready for beta1 even in June. Are > >> we? > > > I'm all about having that discussion... but can we do it on another > > thread or at least wait til we've decided about the back-branch > > releases? They are clearly the more important issue to consider. > > It's the same discussion though, ie what releases are we expecting to > get out in the next couple of weeks. Agreed. If we want to put out beta1 before PGCon, I need to start on the release notes on Monday. > It's possible that we ought to give up on a pre-conference beta. > Certainly a whole lot of time that I'd hoped would go into reviewing > 9.5 feature commits has instead gone into back-branch bug chasing this > week. Based on what has transpired in the past two weeks, I am thinking we need to move _slower_, not faster. I am concerned we have focused so much on new features that we have taken our eye off of reliability. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
On 05/29/2015 01:03 PM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: It's possible that we ought to give up on a pre-conference beta. Certainly a whole lot of time that I'd hoped would go into reviewing 9.5 feature commits has instead gone into back-branch bug chasing this week. I guess that's what I'm getting at. We need to take care of the back-branches and that means pushing beta back. +1 JD -- The most kicking donkey PostgreSQL Infrastructure company in existence. The oldest, the most experienced, the consulting company to the stars. Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 - 24x7 - 365 - Proactive and Managed Professional Services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
On 05/29/2015 11:30 AM, Stephen Frost wrote: > I know how big my WAL partition is. Let me tell PG how big it is and to > not do anything that'll end up going over that amount, and we'll never > see a crash due to out of disk space for WAL again. H. Do we have a clear idea anywhere in server memory how many WAL segments there are? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [CORE] [HACKERS] postpone next week's release
On Fri, May 29, 2015 at 4:01 PM, Tom Lane wrote: > It's possible that we ought to give up on a pre-conference beta. > Certainly a whole lot of time that I'd hoped would go into reviewing > 9.5 feature commits has instead gone into back-branch bug chasing this > week. I'm personally kind of astonished that we're even thinking about beta so soon. I mean, we at least need to go through the stuff listed here, I think: https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items The bigger issue is: what's NOT on that list that should be? I think we need to devote some cycles to figuring that out, and I sure haven't had any this week. In any case, I think the negative PR that we're going to get from not getting this multixact stuff taken care of is going to far outweigh any positive PR from getting 9.5beta1 out a little sooner, especially if 9.5beta1 is bug-ridden because we gave it no time to settle. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need Force flag for pg_drop_replication_slot()
* Josh Berkus (j...@agliodbs.com) wrote: > On 05/29/2015 11:30 AM, Stephen Frost wrote: > > I know how big my WAL partition is. Let me tell PG how big it is and to > > not do anything that'll end up going over that amount, and we'll never > > see a crash due to out of disk space for WAL again. > > H. Do we have a clear idea anywhere in server memory how many WAL > segments there are? Why does it need to be in shared memory..? Clearly, when we're looking at cleaning up the WAL files, we know if the archive command is failing and what file we're trying to archive, or if we're not able to recycle a given file because we have logical replication slots that want it, etc. We certainly know where we're currently at in the WAL stream and we know how big each WAL file is.. We just need a knob to be able to say "alright, this WAL file might still be desired by something, but we're running out of room for *new* WAL and, therefore, that's just too bad for those process that want it" and recycle it anyway. There are probably error conditions we have to consider for replication slots when that happens, etc, but I don't think we lack the info to make the decision, except for what value to set the knob to, which is clearly system-dependent. Thanks! Stephen signature.asc Description: Digital signature
Re: [CORE] [HACKERS] postpone next week's release
Robert Haas writes: > I'm personally kind of astonished that we're even thinking about beta > so soon. I mean, we at least need to go through the stuff listed > here, I think: > https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items Well, maybe we ought to call it an alpha not a beta, but I think we ought to put out some kind of release that we can encourage people to test. What you are suggesting is that we serialize resolution of the known issues with discovery of new issues, and that's not an efficient use of time. Especially seeing that we're approaching the summer season where we won't get much input at all. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: Remove contrib entirely
2015-05-29 21:59 GMT+02:00 Joshua D. Drake : > > On 05/29/2015 12:30 PM, Pavel Stehule wrote: > > Contrib made sense years ago. It does not any longer. Let's put the >> old horse down and raise a new herd of ponies on a new pasture. >> >> >> Still there is strong sense - it is a referential implementation of our >> extension API. We need it to find regressions, changes. I don't believe >> > > No, then we need a proper test suite for the extension API. > maybe partially, but it is. > > so external extensions can do it. Only PostGIS is massively accepted and >> developed by more than few people. Personally I am thinking so removing >> contrib is not good idea. >> > > Is there an extension/contrib module in the last decade that more than > once has shown to help us with that? > What I know - 9.5 transformations for testing on more platforms. It is hard to calculate how often the code from contrib helps - but any feature last four years has not to break contrib test too, so I believe it enforce better API stability. It is hard to imagine to design and maintaining any extension API without platform like contrib. It can be renamed, divided, but some like contrib must exists in core code base if PostgreSQL should be extensible database. > > > Sincerely, > > > JD > > > > -- > Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 > PostgreSQL Centered full stack support, consulting and development. > Announcing "I'm offended" is basically telling the world you can't > control your own emotions, so everyone else should do it for you. >
