Re: [HACKERS] FDW-based dblink (WIP)
Itagaki Takahiro wrote: > However, automatic transaction management needs help by core. Is it > acceptable to have two-phase callbacks? Probably, but it's far too early to decide what the interface should look like. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hot standby - merged up to CVS HEAD
Robert Haas wrote: > On Mon, Aug 17, 2009 at 6:50 AM, Robert Haas wrote: >>> I think there's a race condition in the way LogCurrentRunningXacts() is >>> called at the end of checkpoint. This can happen in the master: >>> >>> 1. Checkpoint starts >>> 2. Transaction 123 begins, and does some updates >>> 3. Checkpoint ends. LogCurrentRunningXacts() is called. >>> 4. LogCurrentRunningXacts() gets the list of currently running >>> transactions by calling GetCurrentTransactionData(). >>> 5. Transaction 123 ends, writing commit record to WAL >>> 6. LogCurrentRunningXacts() writes the list of running XIDs to WAL. This >>> includes XID 123, since that was still running at step 4. >>> >>> When that is replayed, ProcArrayUpdateTransactions() will zap the >>> unobserved xids array with the list that includes XID 123, even though >>> we already saw a commit record for it. >> Sounds like we need some locking there, then. This exceeds my current >> depth of understanding of the patch, but I'll see if I can figure it >> out. > > I looked at this a little more. I'm wondering if we can fix this by > making ProcArrayUpdateRecoveryTransactions() smarter. Can we just > refuse to add an Xid to the UnobservedXids() array if in fact we've > already observed it? (Not sure how to check that.) There's also the opposite problem: If a transaction starts (and writes a WAL record) between LogCurrentRunningXacts() and XLogInstrt(), it is not present in the RunningXacts record. When the standby replays the RunningXacts record, it removes the XID of that transaction from the array, even though it's still running. > Fixing this on the > master would seem to require acquiring the WALInsertLock before > calling GetRunningTransactionData() and holding it until we finish > writing that data to WAL, which I suspect someone's going to complain > about... Yeah, it's hard to get that locking right without risking deadlock. As the patch stands, we only write a RunningXacts record once per checkpoint, so it's not performance critical, but we must avoid deadlocks. If there's a way, I would prefer a solution where the RunningXacts snapshot represents the situation the moment it appears in WAL, not some moment before it. It makes the logic easier to understand. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] auto_explain log_verbose causes regression failure
Robert Haas wrote: > It looks like this is enough to reproduce the cache lookup failure: The "cache loopup failure" part could be fixed by the attached patch. It forbids explaining if the transaction is in error state. I cannot reproduce "unexpected refassgnexpr" and "unexpected FieldStore" errors yet. We might need another fix for them. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center auto_explain-20090820.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW-based dblink (WIP)
Tom Lane wrote:
> I don't believe there is any consensus for integrating dblink into core,
> and I for one will resist that strongly. Keep it in contrib.
OK, our consensus is that dblink should be replaced with SQL/MED interface
and then we'll start to consider integrating into core.
However, automatic transaction management needs help by core. Is it
acceptable to have two-phase callbacks? Registered callbacks are
called with TWOPHASE_EVENT_PRE_COMMIT when a transaction is about
to be committed or prepared. The argument gxact is NULL if the
transaction is committed without 2PC.
typedef enum
{
TWOPHASE_EVENT_PRE_COMMIT,
TWOPHASE_EVENT_POST_COMMIT,
TWOPHASE_EVENT_POST_ABORT,
TWOPHASE_EVENT_RECOVER,
} TwoPhaseEvent;
typedef void (*TwoPhaseEventCallback) (
TwoPhaseEvent event, GlobalTransaction gxact, void *arg);
Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center
twophase_callbacks-20090820.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] auto_explain log_verbose causes regression failure
On Wed, Aug 19, 2009 at 10:07 PM, Robert Haas wrote: > On Wed, Aug 19, 2009 at 9:39 PM, Andrew Dunstan wrote: >> Robert Haas wrote: >>> >>> On Wed, Aug 19, 2009 at 7:57 PM, Andrew >>> Dunstan wrot> >>> I am getting a repeatable failure on the HEAD regression tests when auto_explain's log_verbose is set. If auto_explain.log_verbose is turned off the failure disappears. Data below. cheers andrew config settings: custom_variable_classes = 'auto_explain' auto_explain.log_min_duration = 0 auto_explain.log_format = 'xml' auto_explain.log_analyze = on auto_explain.log_verbose = on shared_preload_libraries = 'auto_explain' >>> >>> I can't figure out how to make this config work. I dumped these >>> settings into a file called "t" and then did, from src/test/regress, >>> TEMP_CONFIG=t make check. >>> >> >> >> I did make install, initdb, put settings in postgresql.conf, then make >> installcheck > > It looks like this is enough to reproduce the cache lookup failure: > > begin; > create table abc (a int); > insert into abc values (5); > insert into abc values (10); > insert into abc values (15); > declare foo cursor for select * from abc; > fetch from foo; > abort; ...and it also looks like the same 3 tests fail in REL8_4_STABLE, which is the oldest branch that contains auto_explain. So I suspect this has been broken all along. I still don't know how to fix it, but at least now I don't feel guilty for breaking it... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] auto_explain log_verbose causes regression failure
On Wed, Aug 19, 2009 at 9:39 PM, Andrew Dunstan wrote: > Robert Haas wrote: >> >> On Wed, Aug 19, 2009 at 7:57 PM, Andrew >> Dunstan wrot> >> >>> >>> I am getting a repeatable failure on the HEAD regression tests when >>> auto_explain's log_verbose is set. If auto_explain.log_verbose is turned >>> off >>> the failure disappears. Data below. >>> >>> cheers >>> >>> andrew >>> >>> config settings: >>> >>> custom_variable_classes = 'auto_explain' >>> auto_explain.log_min_duration = 0 >>> auto_explain.log_format = 'xml' >>> auto_explain.log_analyze = on >>> auto_explain.log_verbose = on >>> shared_preload_libraries = 'auto_explain' >>> >> >> I can't figure out how to make this config work. I dumped these >> settings into a file called "t" and then did, from src/test/regress, >> TEMP_CONFIG=t make check. >> > > > I did make install, initdb, put settings in postgresql.conf, then make > installcheck It looks like this is enough to reproduce the cache lookup failure: begin; create table abc (a int); insert into abc values (5); insert into abc values (10); insert into abc values (15); declare foo cursor for select * from abc; fetch from foo; abort; And here's the backtrace: #0 elog_start (filename=0x83c251 "lsyscache.c", lineno=805, funcname=0x83c680 "get_relid_attribute_name") at elog.c:1085 #1 0x006ce1c4 in get_relid_attribute_name (relid=63508, attnum=) at lsyscache.c:805 #2 0x006789bb in get_variable (var=0x0, levelsup=33399048, showstar=1 '\001', context=0x7fff21f99ad0) at ruleutils.c:3515 #3 0x006784e3 in deparse_expression_pretty (expr=0x20033c0, dpcontext=0x1fda130, forceprefix=0 '\0', showimplicit=-128 '\200', prettyFlags=0, startIndent=0) at ruleutils.c:1915 #4 0x0051bb89 in ExplainNode (plan=0x2003270, planstate=0x202e9f8, outer_plan=0x0, relationship=0x0, plan_name=, es=0x7fff21f99c50) at explain.c:1145 #5 0x7f3aaf26bfd2 in explain_ExecutorEnd (queryDesc=0x20038b8) at auto_explain.c:225 #6 0x0052aaa7 in PortalCleanup (portal=0x20267b0) at portalcmds.c:268 #7 0x006f1ca0 in AtAbort_Portals () at portalmem.c:678 #8 0x004892de in AbortTransaction () at xact.c:2036 #9 0x0048c8d5 in CommitTransactionCommand () at xact.c:2291 #10 0x00619897 in finish_xact_command () at postgres.c:2364 #11 0x0061a80d in exec_simple_query (query_string=0x1fe5828 "abort;") at postgres.c:1023 #12 0x0061bc77 in PostgresMain (argc=4, argv=, username=0x1f46ec0 "rhaas") at postgres.c:3615 #13 0x005e8960 in ServerLoop () at postmaster.c:3392 #14 0x005e972a in PostmasterMain (argc=3, argv=0x1f442a0) at postmaster.c:1038 #15 0x0058c808 in main (argc=3, argv=0x1f442a0) at main.c:188 It looks to me like the problem might be something along the lines of "by the time ExplainNode() gets called, the table is not visible any more, so the syscache lookup fails". But that's just a guess... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] auto_explain log_verbose causes regression failure
Robert Haas wrote: On Wed, Aug 19, 2009 at 7:57 PM, Andrew Dunstan wrot> I am getting a repeatable failure on the HEAD regression tests when auto_explain's log_verbose is set. If auto_explain.log_verbose is turned off the failure disappears. Data below. cheers andrew config settings: custom_variable_classes = 'auto_explain' auto_explain.log_min_duration = 0 auto_explain.log_format = 'xml' auto_explain.log_analyze = on auto_explain.log_verbose = on shared_preload_libraries = 'auto_explain' I can't figure out how to make this config work. I dumped these settings into a file called "t" and then did, from src/test/regress, TEMP_CONFIG=t make check. I did make install, initdb, put settings in postgresql.conf, then make installcheck cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] auto_explain log_verbose causes regression failure
On Wed, Aug 19, 2009 at 7:57 PM, Andrew Dunstan wrot> > I am getting a repeatable failure on the HEAD regression tests when > auto_explain's log_verbose is set. If auto_explain.log_verbose is turned off > the failure disappears. Data below. > > cheers > > andrew > > config settings: > > custom_variable_classes = 'auto_explain' > auto_explain.log_min_duration = 0 > auto_explain.log_format = 'xml' > auto_explain.log_analyze = on > auto_explain.log_verbose = on > shared_preload_libraries = 'auto_explain' I can't figure out how to make this config work. I dumped these settings into a file called "t" and then did, from src/test/regress, TEMP_CONFIG=t make check. This results in: make -C ../../../src/port all make[1]: Entering directory `/home/rhaas/pgsql-git/src/port' make[1]: Nothing to be done for `all'. make[1]: Leaving directory `/home/rhaas/pgsql-git/src/port' rm -rf ./testtablespace mkdir ./testtablespace ./pg_regress --inputdir=. --dlpath=. --multibyte=SQL_ASCII --load-language=plpgsql --temp-install=./tmp_check --top-builddir=../../.. --schedule=./parallel_schedule --temp-config=t == removing existing temp installation== == creating temporary installation== == initializing database system == == starting postmaster== pg_regress: postmaster did not respond within 60 seconds Examine /home/rhaas/pgsql-git/src/test/regress/log/postmaster.log for the reason make: *** [check] Error 2 That file contains: FATAL: could not access file "auto_explain": No such file or directory I tried copying auto_explain.so into $PWD, but that did not help. Can you describe how you got this to run the tests at all? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] fillfactor hides autovacuum parameters in 8.4.0
Here is a patch to fix a bug in handling default values in reloptions. This fix should be applied to HEAD and 8.4.0. I used 'magic number -1' to propagate "not-specified" information to autovacuum process. It might look strange because the default value is out of range of the reloption, but I think it has less impact to the codes comapred with other solutions (dynamic default values etc.). > To fix the bug, each field in StdRdOptions should have "not-specified" flag. > If not specified, autovacuum should use current GUC settings instead of > reloptions. Is it possible to change the default values of reloptions > to some magic number (-1 or so) ? Regards, --- ITAGAKI Takahiro NTT Open Source Software Center fix-relopts-20090820.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] auto_explain log_verbose causes regression failure
I am getting a repeatable failure on the HEAD regression tests when
auto_explain's log_verbose is set. If auto_explain.log_verbose is turned
off the failure disappears. Data below.
cheers
andrew
config settings:
custom_variable_classes = 'auto_explain'
auto_explain.log_min_duration = 0
auto_explain.log_format = 'xml'
auto_explain.log_analyze = on
auto_explain.log_verbose = on
shared_preload_libraries = 'auto_explain'
regression diffs:
***
/home/andrew/pgl/pgsql.hetry/src/test/regress/expected/transactions.out
2009-08-09 19:29:31.0 -0400
---
/home/andrew/pgl/pgsql.hetry/src/test/regress/results/transactions.out
2009-08-19 19:48:20.0 -0400
***
*** 496,504
(1 row)
rollback to x;
-- should fail
fetch from foo;
! ERROR: cursor "foo" does not exist
commit;
begin;
create table abc (a int);
--- 496,506
(1 row)
rollback to x;
+ WARNING: AbortSubTransaction while in ABORT state
+ ERROR: cache lookup failed for attribute 1 of relation 28000
-- should fail
fetch from foo;
! ERROR: current transaction is aborted, commands ignored until end of
transaction block
commit;
begin;
create table abc (a int);
***
*** 527,532
--- 529,536
(1 row)
abort;
+ WARNING: AbortTransaction while in ABORT state
+ ERROR: cache lookup failed for attribute 1 of relation 28003
-- tests for the "tid" type
SELECT '(3, 3)'::tid = '(3, 4)'::tid;
?column?
==
*** /home/andrew/pgl/pgsql.hetry/src/test/regress/expected/arrays.out
2009-08-09 19:29:31.0 -0400
--- /home/andrew/pgl/pgsql.hetry/src/test/regress/results/arrays.out
2009-08-19 19:48:21.0 -0400
***
*** 15,22
--- 15,25
--
INSERT INTO arrtest (a[1:5], b[1:1][1:2][1:2], c, d, f, g)
VALUES ('{1,2,3,4,5}', '{{{0,0},{1,2}}}', '{}', '{}', '{}', '{}');
+ ERROR: unexpected refassgnexpr
UPDATE arrtest SET e[0] = '1.1';
+ ERROR: unexpected refassgnexpr
UPDATE arrtest SET e[1] = '2.2';
+ ERROR: unexpected refassgnexpr
INSERT INTO arrtest (f)
VALUES ('{"too long"}');
ERROR: value too long for type character(5)
***
*** 24,38
VALUES ('{11,12,23}', '{{3,4},{4,5}}', '{"foobar"}',
'{{"elt1", "elt2"}}', '{"3.4", "6.7"}',
'{"abc","abcde"}', '{"abc","abcde"}');
INSERT INTO arrtest (a, b[1:2], c, d[1:2])
VALUES ('{}', '{3,4}', '{foo,bar}', '{bar,foo}');
SELECT * FROM arrtest;
a |b| c | d |
e|f| g
!
-+-+---+---+-+-+-
! {1,2,3,4,5} | {{{0,0},{1,2}}} | {}| {}|
[0:1]={1.1,2.2} | {} | {}
! {11,12,23} | {{3,4},{4,5}} | {foobar} | {{elt1,elt2}} |
{3.4,6.7} | {"abc ",abcde} | {abc,abcde}
! {} | {3,4} | {foo,bar} | {bar,foo}
| | |
! (3 rows)
SELECT arrtest.a[1],
arrtest.b[1][1][1],
--- 27,40
VALUES ('{11,12,23}', '{{3,4},{4,5}}', '{"foobar"}',
'{{"elt1", "elt2"}}', '{"3.4", "6.7"}',
'{"abc","abcde"}', '{"abc","abcde"}');
+ ERROR: unexpected refassgnexpr
INSERT INTO arrtest (a, b[1:2], c, d[1:2])
VALUES ('{}', '{3,4}', '{foo,bar}', '{bar,foo}');
+ ERROR: unexpected refassgnexpr
SELECT * FROM arrtest;
a | b | c | d | e | f | g
! ---+---+---+---+---+---+---
! (0 rows)
SELECT arrtest.a[1],
arrtest.b[1][1][1],
***
*** 41,60
arrtest.e[0]
FROM arrtest;
a | b | c| d | e
! +---++--+-
! 1 | 0 || | 1.1
! 11 | | foobar | elt1 |
! | | foo| |
! (3 rows)
SELECT a[1], b[1][1][1], c[1], d[1][1], e[0]
FROM arrtest;
a | b | c| d | e
! +---++--+-
! 1 | 0 || | 1.1
! 11 | | foobar | elt1 |
! | | foo| |
! (3 rows)
SELECT a[1:3],
b[1:1][1:2][1:2],
--- 43,56
arrtest.e[0]
FROM arrtest;
a | b | c | d | e
! ---+---+---+---+---
! (0 rows)
SELECT a[1], b[1][1][1], c[1], d[1][1], e[0]
FROM arrtest;
a | b | c | d | e
! ---+---+---+---+---
! (0 rows)
SELECT a[1:3],
b[1:1][1:2][1:2],
***
*** 62,90
d[1:1][1:2]
FROM arrtest;
a |b| c | d
! +-+---+---
! {1,2,3}| {{{0,0},{1,2}}} | {}| {}
! {11,12,23} | {} | {foobar} | {{elt1,elt2}}
! {} | {} | {foo,bar} | {}
! (3 rows)
SELECT array_ndims(a) AS a,array_ndims(b) AS b,array_ndims(c) AS c
FROM arrtest;
a | b | c
---+---+---
! 1 | 3 |
! 1 | 2 | 1
!| 1 | 1
! (3 rows)
SELECT array_dims(a) AS a,array_dims(b) AS b,array_dims(c) AS c
FRO
Re: [HACKERS] Why ACL_EXECUTE is checked on FindConversion()?
Tom Lane wrote: > KaiGai Kohei writes: >> When FindConversion() is called, it also checks current user's ACL_EXECUTE >> privilege on the conproc of the fetched conversion. > >> Why this check is applied on FindConversion(), instead of >> FindDefaultConversion()? > > Does seem pretty inconsistent, doesn't it? Anyway, I could not understand the intention of the checks, rather than inconsistency. So, I doubted it might be a bug, if it intended to provide a permission something like ACL_USAGE on conversion. > The original idea may have been to provide a substitute for a USAGE > ACL check on conversions, in which case it's not totally insane: if > you make a conversion default then you're implicitly granting it to > public. But there's no documentation about this. If ACL_EXECUTE checks is deployed on FindDefaultConversion(), it performs as if something like ACL_USAGE permission. However, FindConversion() is called from ALTER or DROP CONVERSION, so it controls DDL statements in addition to its ownership. Please note that this check (ACL_EXECUTE) is not applied when user choose a certain conversion. > Offhand I see no really good reason to have a usage check on > conversions, and would be happy with removing this one. The function > permission check at CREATE CONVERSION time ought to be sufficient. It seems to me reasonable. If user does not have ACL_EXECUTE privilege on the function used in the conversion, he does not alter or drop the conversion, even if he has ownership of the conversion. This behavior is hard to understand. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
I wrote: > Oh, right -- it does let PostgreSQL automatically deal with the file > left by a different instance, but could still fail on it's own file. Er, it does let PostgreSQL automatically deal with a different instance using the PID matching what this instance left in its file, but could be a problem if something else under this same user ID, like a higher level script above pg_ctl, is now using the PID. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Tom Lane wrote: > "Kevin Grittner" writes: > Well, using a different user per instance is a good idea because > then the safety analysis I gave holds rigorously for each instance. > It doesn't get you out of the problem by itself, because the problem > as described can happen with just one instance. Oh, right -- it does let PostgreSQL automatically deal with the file left by a different instance, but could still fail on it's own file. >> It must buy something in our environment, because our attempts to >> use the sample script with minimal modification were problematic. >> Unfortunately I forget the details, but our problems vanished when >> we switched to pg_ctl. (Well, except for that one unfortunate >> episode mentioned above.) > > Hmm. As stated, I would expect pg_ctl to make it worse. It would > be interesting to have a closer look at your before-and-after > scripts. I don't remember whether the problem had anything to do with the lock files; it could have been that pg_ctl was doing something else which worked better for us than the direct invocation of postmaster, at lease with the options we were using. I'll experiment and see what happens in a test environment. I don't think we saved our failed attempts from years back. Also, I was quite green with Linux at the time, and it was my first initscript tinkering -- it's not outside the range of possibility that I did something dumb with the direct attempt. That said, knowing so little about what I was doing with it had me starting from the point of changing as little as I could manage from the sample to try to get it going. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Greg Stark writes: > On Wed, Aug 19, 2009 at 10:03 PM, Tom Lane wrote: >> What this all leads to is that it's safe to launch a postmaster from >> an init script via something like >>su - postgres sh -c "postmaster ..." > Surely you don't want "-"? If you run postgres's .profile etc. then > random user customization for the postgres user could interfere with > your startup process. Well, people who put random things into the postgres user's .profile deserve what they get. But it is useful to be able to customize the postmaster's PATH and other variables. As far as I know, all the Linux distros do use "su -" or equivalent, and so do the contrib start-scripts. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
"Kevin Grittner" writes: > Right -- we did run into this in spades when our backup server, > running dozens of instances of PostgreSQL in "warm standby" to confirm > the integrity of the files received, crashed hard. I wasn't sure if > this was the problem being addressed. One obvious solution, which we > now rigorously observe, is to use a different OS user for each > PostgreSQL instance. I assume that pg_ctl is safe in such an > environment? Well, using a different user per instance is a good idea because then the safety analysis I gave holds rigorously for each instance. It doesn't get you out of the problem by itself, because the problem as described can happen with just one instance. > It must buy something in our environment, because our attempts to use > the sample script with minimal modification were problematic. > Unfortunately I forget the details, but our problems vanished when we > switched to pg_ctl. (Well, except for that one unfortunate episode > mentioned above.) Hmm. As stated, I would expect pg_ctl to make it worse. It would be interesting to have a closer look at your before-and-after scripts. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Tom, > (Personally, I use scripts based on start-scripts/osx/ for a number of > services on my own machines, so if there's something wrong with them > I'd definitely like to know what it is.) I quote: "# What to use to start up the postmaster (we do NOT use pg_ctl for this, # as it adds no value and can cause the postmaster to misrecognize a stale # lock file) DAEMON="$prefix/bin/postmaster" -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
On Wed, Aug 19, 2009 at 10:03 PM, Tom Lane wrote: > What this all leads to is that it's safe to launch a postmaster from > an init script via something like > su - postgres sh -c "postmaster ..." Surely you don't want "-"? If you run postgres's .profile etc. then random user customization for the postgres user could interfere with your startup process. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Geometric bewilderment
On Wed, Aug 19, 2009 at 07:29:43PM +1000, Paul Matthews wrote: > Suspect that attaching large amounts of code is a breach of > etiquette. Code attachments aren't, but HTML messages are, so in future, please send only text :) Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Tom Lane wrote: > The problem is that after a system crash and reboot, an old > postmaster.pid file might be left behind. The postmaster can only > safely remove this lock file if it is *certain* that it doesn't > represent another live postmaster process. Otherwise it is honor- > bound to commit hara-kiri instead of starting up. It can tell > whether or not the PID in the file belongs to a live process and > whether that process belongs to the postgres userid (by attempting > kill(PID, 0) and seeing what it gets). If not, it can remove the > file with a clear conscience. Right -- we did run into this in spades when our backup server, running dozens of instances of PostgreSQL in "warm standby" to confirm the integrity of the files received, crashed hard. I wasn't sure if this was the problem being addressed. One obvious solution, which we now rigorously observe, is to use a different OS user for each PostgreSQL instance. I assume that pg_ctl is safe in such an environment? > The long and the short of it is that it's best to not use pg_ctl. > As mentioned, it doesn't buy much of anything for an initscript > anyway. It must buy something in our environment, because our attempts to use the sample script with minimal modification were problematic. Unfortunately I forget the details, but our problems vanished when we switched to pg_ctl. (Well, except for that one unfortunate episode mentioned above.) > The whole thing is really only a problem for initscript authors (who > all know about it by now ;-)) Well, one of them (at least) didn't quite understand the whole issue until receiving your email. Thanks for the clear description. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
"David E. Wheeler" writes: > Nice summary, Tom. Do the distro packagers know this, though? All the active ones I know of learned it the hard way, or were paying attention when someone else did. Still, it wouldn't be a bad thing for us to document it somewhere. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Should we add a comment to the startup scripts linking this email? http://archives.postgresql.org/message-id/28922.1250715...@sss.pgh.pa.us --- Tom Lane wrote: > "Kevin Grittner" writes: > > Tom Lane wrote: > >>> we do NOT use pg_ctl for [postmaster start], as it adds no value > >>> and can cause the postmaster to misrecognize a stale lock file > > >> And? That statement was and remains perfectly correct. > > > Is this mentioned in the documentation somewhere that I've missed? > > I'm curious what the issues are, and why we can solve it in a bash > > script but not pg_ctl. > > It's been covered repeatedly in the archives, but I'm not sure if it's > in the docs anywhere. The problem is that after a system crash and > reboot, an old postmaster.pid file might be left behind. The postmaster > can only safely remove this lock file if it is *certain* that it doesn't > represent another live postmaster process. Otherwise it is honor-bound > to commit hara-kiri instead of starting up. It can tell whether or not > the PID in the file belongs to a live process and whether that process > belongs to the postgres userid (by attempting kill(PID, 0) and seeing > what it gets). If not, it can remove the file with a clear conscience. > However, because of the way that Unix startup works, it is very likely > that successive system boots will assign nearly (but not necessarily > exactly) the same PID that the postmaster had on the previous cycle. > So there's a high probability of a false positive from this test. > If the PID matches our own exactly, we can discount it as a false > positive. If it matches our parent's exactly, we can also discount it > (knowing that a postmaster would never launch another postmaster > directly, and being able to get the parent's PID via getppid()). > But further up the chain, we're out of luck, because there is no > "get grandparent pid" operation in Unix. > > What this all leads to is that it's safe to launch a postmaster from > an init script via something like > su - postgres sh -c "postmaster ..." > The postmaster's parent process is a shell belonging to postgres, > which it can discount via getppid(), and all further-up ancestors > belong to root, so we can discount them via the kill test. So a > false PID match cannot lead to failing to start. (You still have to > be a bit careful about the form of the shell command, or there might > be an intermediate postgres-owned shell process.) > > On the other hand, if you do > su - postgres sh -c "pg_ctl ..." > then the postmaster's parent process is pg_ctl, and its grandparent > is a postgres-owned shell process, and it cannot tell that > postgres-owned shell process apart from a genuine conflicting > postmaster. So a chance match of the shell process's PID to what is in > the leftover postmaster.pid file will force it to refuse to start. > And that chance match is not a low probability --- in my experience > it's one in ten or worse, in a reasonably stable system environment. > > You can imagine various workarounds involving having pg_ctl pass down > its parent's PID, but you'll still get screwed if the initscript author > is careless about how many levels of postgres-owned shell process there > are. The long and the short of it is that it's best to not use pg_ctl. > As mentioned, it doesn't buy much of anything for an initscript anyway. > > These considerations don't apply to ordinary hand launching of the > postmaster, for the primary reason that the chance of a false PID match > is several orders of magnitude smaller when you're talking about a > manual restart --- the likely postmaster PID now ranges over the whole > PID space instead of being within a few counts of the same thing. So we > don't need to discourage people from using pg_ctl for ordinary restarts. > The whole thing is really only a problem for initscript authors (who all > know about it by now ;-)) > > regards, tom lane > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
On Aug 19, 2009, at 2:03 PM, Tom Lane wrote: These considerations don't apply to ordinary hand launching of the postmaster, for the primary reason that the chance of a false PID match is several orders of magnitude smaller when you're talking about a manual restart --- the likely postmaster PID now ranges over the whole PID space instead of being within a few counts of the same thing. So we don't need to discourage people from using pg_ctl for ordinary restarts. The whole thing is really only a problem for initscript authors (who all know about it by now ;-)) Nice summary, Tom. Do the distro packagers know this, though? Do we have notes for distro packagers somewhere? Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
"Kevin Grittner" writes: > Tom Lane wrote: >>> we do NOT use pg_ctl for [postmaster start], as it adds no value >>> and can cause the postmaster to misrecognize a stale lock file >> And? That statement was and remains perfectly correct. > Is this mentioned in the documentation somewhere that I've missed? > I'm curious what the issues are, and why we can solve it in a bash > script but not pg_ctl. It's been covered repeatedly in the archives, but I'm not sure if it's in the docs anywhere. The problem is that after a system crash and reboot, an old postmaster.pid file might be left behind. The postmaster can only safely remove this lock file if it is *certain* that it doesn't represent another live postmaster process. Otherwise it is honor-bound to commit hara-kiri instead of starting up. It can tell whether or not the PID in the file belongs to a live process and whether that process belongs to the postgres userid (by attempting kill(PID, 0) and seeing what it gets). If not, it can remove the file with a clear conscience. However, because of the way that Unix startup works, it is very likely that successive system boots will assign nearly (but not necessarily exactly) the same PID that the postmaster had on the previous cycle. So there's a high probability of a false positive from this test. If the PID matches our own exactly, we can discount it as a false positive. If it matches our parent's exactly, we can also discount it (knowing that a postmaster would never launch another postmaster directly, and being able to get the parent's PID via getppid()). But further up the chain, we're out of luck, because there is no "get grandparent pid" operation in Unix. What this all leads to is that it's safe to launch a postmaster from an init script via something like su - postgres sh -c "postmaster ..." The postmaster's parent process is a shell belonging to postgres, which it can discount via getppid(), and all further-up ancestors belong to root, so we can discount them via the kill test. So a false PID match cannot lead to failing to start. (You still have to be a bit careful about the form of the shell command, or there might be an intermediate postgres-owned shell process.) On the other hand, if you do su - postgres sh -c "pg_ctl ..." then the postmaster's parent process is pg_ctl, and its grandparent is a postgres-owned shell process, and it cannot tell that postgres-owned shell process apart from a genuine conflicting postmaster. So a chance match of the shell process's PID to what is in the leftover postmaster.pid file will force it to refuse to start. And that chance match is not a low probability --- in my experience it's one in ten or worse, in a reasonably stable system environment. You can imagine various workarounds involving having pg_ctl pass down its parent's PID, but you'll still get screwed if the initscript author is careless about how many levels of postgres-owned shell process there are. The long and the short of it is that it's best to not use pg_ctl. As mentioned, it doesn't buy much of anything for an initscript anyway. These considerations don't apply to ordinary hand launching of the postmaster, for the primary reason that the chance of a false PID match is several orders of magnitude smaller when you're talking about a manual restart --- the likely postmaster PID now ranges over the whole PID space instead of being within a few counts of the same thing. So we don't need to discourage people from using pg_ctl for ordinary restarts. The whole thing is really only a problem for initscript authors (who all know about it by now ;-)) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Josh Berkus wrote: > Tom, > > >> "# What to use to start up the postmaster (we do NOT use pg_ctl for this, > >> # as it adds no value and can cause the postmaster to misrecognize a stale > >> # lock file) > >> DAEMON="$prefix/bin/postmaster" > > > > And? That statement was and remains perfectly correct. We don't use > > pg_ctl to start the postmaster in the Linux initscripts, either. > > Then WTF do we have pg_ctl, and why do our docs tell people to use it? I assume on boot we _know_ there can't be another postmaster running, while normal pg_ctl does not know that. Perhaps we just need to state that in the comment. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Tom, >> "# What to use to start up the postmaster (we do NOT use pg_ctl for this, >> # as it adds no value and can cause the postmaster to misrecognize a stale >> # lock file) >> DAEMON="$prefix/bin/postmaster" > > And? That statement was and remains perfectly correct. We don't use > pg_ctl to start the postmaster in the Linux initscripts, either. Then WTF do we have pg_ctl, and why do our docs tell people to use it? -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Tom Lane wrote: > Josh Berkus writes: >> we do NOT use pg_ctl for [postmaster start], as it adds no value >> and can cause the postmaster to misrecognize a stale lock file > And? That statement was and remains perfectly correct. Is this mentioned in the documentation somewhere that I've missed? I'm curious what the issues are, and why we can solve it in a bash script but not pg_ctl. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4961: pg_standby.exe crashes with no args
On Wed, Aug 19, 2009 at 11:45, Heikki Linnakangas wrote: > Magnus Hagander wrote: >> This would amount to fairly major surgery for pg_standby on Win32. Is >> that something we'd want to backpatch, or do we want to backpatch just >> the removal of the signal() calls which would amount to not supporting >> signals in pg_standby on win32? > > I think we should just remove the signals support for win32. The trigger > file method still works, and the signal method has always been a bit > iffy (it doesn't work when pg_standby isn't running, for example, which > happens between processing of each WAL file, because there's no process > to signal). Fair enough. > Is pg_standby killed correctly when postmaster dies? No idea. I can set up and env to test, but I don't have one ready. Anybody else have tried this? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Alvaro Herrera writes: > Tom Lane wrote: >> (Personally, I use scripts based on start-scripts/osx/ for a number of >> services on my own machines, so if there's something wrong with them >> I'd definitely like to know what it is.) > What kind of "based on"? I mean, are there some changes of yours that > could be applied to the contrib script? No, I've just modified them to start other services (bind, sendmail, ...). I'm sure OSX startup scripts are documented elsewhere, but this is a resource right under my nose, so I used it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
On Wed, Aug 19, 2009 at 3:00 PM, Josh Berkus wrote: > Tom, Greg, Robert, > > Here's my suggestion: > > 1. First, estimate the cost of the node with a very pessimistic (50%?) > selectivity for the calculation. There is no such thing as a pessimistic selectivity estimation. Right now a lot of things use nested loops when they should hash, because we use 0.5%. If we changed it to 50%, then we'd have the opposite problem (and maybe some merge joins where we should have hashing). Unfortunately, there is no substitute for accurate estimates. Of course if the executor had the ability to switch from a nest loop to a hash join in mid query it would help a great deal, but that's a much bigger project, I think. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Josh Berkus writes: > Tom, >> (Personally, I use scripts based on start-scripts/osx/ for a number of >> services on my own machines, so if there's something wrong with them >> I'd definitely like to know what it is.) > I quote: > "# What to use to start up the postmaster (we do NOT use pg_ctl for this, > # as it adds no value and can cause the postmaster to misrecognize a stale > # lock file) > DAEMON="$prefix/bin/postmaster" And? That statement was and remains perfectly correct. We don't use pg_ctl to start the postmaster in the Linux initscripts, either. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Tom Lane wrote: > (Personally, I use scripts based on start-scripts/osx/ for a number of > services on my own machines, so if there's something wrong with them > I'd definitely like to know what it is.) What kind of "based on"? I mean, are there some changes of yours that could be applied to the contrib script? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
Tom, Greg, Robert, Here's my suggestion: 1. First, estimate the cost of the node with a very pessimistic (50%?) selectivity for the calculation. 2. If the cost hits a certain threshold, then run the calculation estimation on the histogram. That way, we avoid the subtransaction and other overhead on very small sets. also: > Trying it on the MCVs makes a lot of sense. I'm not so sure about > trying it on the histogram entries. There's no reason to assume that > those cluster in any way that will be useful. (For example, suppose > that I have the numbers 1 through 10,000 in some particular column and > my expression is col % 100.) Yes, but for seriously skewed column distributions, the difference in frequency between the MCV and a sample "random" distribution will be huge. And it's precisely those distributions which are currently failing in the query planner. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Josh Berkus wrote: > > ... for the simple reason that nobody is maintaining it. Wheeler just > pointed out to me today that the OSX startup script hasn't been updated > since 7.4 and contains misinformation and dangerous scripting. > > Other startup scripts there are equally dilapidated, and aren't used by > the linux distros regardless. If they are unmaintained, why not replace them with a pointer to some script that is maintained? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
On Aug 19, 2009, at 11:48 AM, Tom Lane wrote: (Personally, I use scripts based on start-scripts/osx/ for a number of services on my own machines, so if there's something wrong with them I'd definitely like to know what it is.) +1. Please don't remove the start scripts. I use them on every system on which I install PostgreSQL. They're very handy for those of us who don't use distro packages. But do fix them if they need it. Thanks, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
Greg Stark writes: > On Wed, Aug 19, 2009 at 7:22 PM, Tom Lane wrote: >> It may be that a subtransaction will actually be acceptable overhead, >> as long as we don't go overboard and invoke this mechanism for "simple" >> WHERE clauses. Hard to tell without coding it up and testing it though. > Are you looking for volunteers? Or have you already nearly finished? I have not started on it, and have a few other things on my plate first. I was planning to do it, but if you'd like to try, be my guest. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
On Wed, Aug 19, 2009 at 7:22 PM, Tom Lane wrote: > > It may be that a subtransaction will actually be acceptable overhead, > as long as we don't go overboard and invoke this mechanism for "simple" > WHERE clauses. Hard to tell without coding it up and testing it though. Are you looking for volunteers? Or have you already nearly finished? -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Chander Ganesan writes: > Josh Berkus wrote: >> ... for the simple reason that nobody is maintaining it. Wheeler just >> pointed out to me today that the OSX startup script hasn't been updated >> since 7.4 and contains misinformation and dangerous scripting. >> >> Other startup scripts there are equally dilapidated, and aren't used by >> the linux distros regardless. >> > IMHO, those scripts (at least the Linux one) has great value. There are > numerous people that compile and install from source, and for them at > least some of these scripts are used quite a bit. > AFAIK, the Linux one is functional (yes, sub-optimal in some ways, but > it works just fine)... There are lots of files in our distribution that don't get changed for years at a time. I think if there's something wrong with these, they should get fixed. I'm certainly not prepared to remove them on the basis of an unsubstantiated second-hand report of unspecified problems. (Personally, I use scripts based on start-scripts/osx/ for a number of services on my own machines, so if there's something wrong with them I'd definitely like to know what it is.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We should Axe /contrib/start-scripts
Josh Berkus wrote: ... for the simple reason that nobody is maintaining it. Wheeler just pointed out to me today that the OSX startup script hasn't been updated since 7.4 and contains misinformation and dangerous scripting. Other startup scripts there are equally dilapidated, and aren't used by the linux distros regardless. IMHO, those scripts (at least the Linux one) has great value. There are numerous people that compile and install from source, and for them at least some of these scripts are used quite a bit. AFAIK, the Linux one is functional (yes, sub-optimal in some ways, but it works just fine)... -- Chander Ganesan Open Technology Group, Inc. One Copley Parkway, Suite 210 Morrisville, NC 27560 919-463-0999/877-258-8987 http://www.otg-nc.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why ACL_EXECUTE is checked on FindConversion()?
KaiGai Kohei writes: > When FindConversion() is called, it also checks current user's ACL_EXECUTE > privilege on the conproc of the fetched conversion. > Why this check is applied on FindConversion(), instead of > FindDefaultConversion()? Does seem pretty inconsistent, doesn't it? The original idea may have been to provide a substitute for a USAGE ACL check on conversions, in which case it's not totally insane: if you make a conversion default then you're implicitly granting it to public. But there's no documentation about this. Offhand I see no really good reason to have a usage check on conversions, and would be happy with removing this one. The function permission check at CREATE CONVERSION time ought to be sufficient. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
Greg Stark writes: > Another thought. In the above case it would actually be fine to catch > the error with PG_TRY() without a subtransaction. As long as no shared > database state has been modified when the error is thrown then the > subtransaction isn't needed to roll them back. I think catching general errors coming out of possibly user-defined code without using a subtransaction is just too scary to contemplate. There's too much stuff that could need to be cleaned up and would not have been. It may be that a subtransaction will actually be acceptable overhead, as long as we don't go overboard and invoke this mechanism for "simple" WHERE clauses. Hard to tell without coding it up and testing it though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] We should Axe /contrib/start-scripts
... for the simple reason that nobody is maintaining it. Wheeler just pointed out to me today that the OSX startup script hasn't been updated since 7.4 and contains misinformation and dangerous scripting. Other startup scripts there are equally dilapidated, and aren't used by the linux distros regardless. Unless someone volunteers to be *permanent* maintainer of this directory, I vote that we remove it from 8.5. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha1 bundled -- please verify
Tom Lane wrote: Peter Eisentraut writes: Alpha1 has been bundled and is available at http://developer.postgresql.org/~petere/alpha/ Please check that it is sane. It looks like all the derived grammar files have been built with bison 2.4.1, which is not what's on svr1 (unless that's been updated recently). I'm not sure that this is something to worry about, but it does mean that people will be testing something that's a bit different from what an "official" tarball would look like. correct - svr1 has bison 1.875 and flex 2.5.35 (for the -HEAD builds) and flex 2.5.4 for the back branches. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha1 bundled -- please verify
Peter Eisentraut writes: > Alpha1 has been bundled and is available at > http://developer.postgresql.org/~petere/alpha/ > Please check that it is sane. It looks like all the derived grammar files have been built with bison 2.4.1, which is not what's on svr1 (unless that's been updated recently). I'm not sure that this is something to worry about, but it does mean that people will be testing something that's a bit different from what an "official" tarball would look like. Otherwise it looks good from here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] XLogInsert
Jeff Janes writes: > If I read the code correctly, the only thing that is irrevocable is > that it writes into > rdt->next, and if it saved an old copy of rdt first, then it could > revoke the changes just > by doing rdt_old->next=NULL. If that were done, then I think this > code could be > moved out of the section holding the WALInsertLock. Hmm, I recall that the changes are ... or were ... more complex. The tricky case I think is where we have to go back and redo the block-backup decisions after discovering that the checkpoint REDO pointer has just moved. If you can get the work out of the WALInsertLock section for just a few more instructions, it would definitely be worth doing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha1 bundled -- please verify
Kevin Grittner wrote: > Peter Eisentraut wrote: > > > Alpha1 has been bundled and is available at > > > > http://developer.postgresql.org/~petere/alpha/ > > > > Please check that it is sane. > > I downloaded it and did a make and make check on a machine without all > the packages to build from a source checkout. All 121 tests passed. > Anything else I can do to check sanity? (In particular, it sounds > like docs are a potential issue, but I'm not sure what I should > check.) Docs look good -- installed the HTML docs and manpages. The index seems fine. One little odd thing about the manpages: the SEE ALSO section of VACUUM.7 looks like this: SEE ALSO vacuumdb,Cost-Based Vacuum Delay , The Autovacuum Daemon Strangely, on the text it looks good: PostgreSQL includes an “autovacuum” facility which can automate routine vacuum maintenance. For more information about automatic and manual vacuuming, see Section 23.1, “Routine Vacuuming”, in the documentation. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] XLogInsert
In XLogInsert (src/backend/access/transam/xlog.c), the part that adds back-up blocks into the rdata chain is described: /* * Make additional rdata chain entries for the backup blocks, so that we * don't need to special-case them in the write loop. Note that we have * now irrevocably changed the input rdata chain. If I read the code correctly, the only thing that is irrevocable is that it writes into rdt->next, and if it saved an old copy of rdt first, then it could revoke the changes just by doing rdt_old->next=NULL. If that were done, then I think this code could be moved out of the section holding the WALInsertLock. It could probably be folded into the part of the code that computes the CRC. I don't think this wold be a big performance win, as that part of the code is pretty fast, but every bit helps in a highly contended lock, and I think the code would be simpler as well. Do you think it would be worth me giving this a try? Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha1 bundled -- please verify
Peter Eisentraut wrote: > Alpha1 has been bundled and is available at > > http://developer.postgresql.org/~petere/alpha/ > > Please check that it is sane. I downloaded it and did a make and make check on a machine without all the packages to build from a source checkout. All 121 tests passed. Anything else I can do to check sanity? (In particular, it sounds like docs are a potential issue, but I'm not sure what I should check.) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain root element for auto-explain
Andrew Dunstan writes: > Bruce Momjian wrote: >> Are we going to publish an XML DTD for EXPLAIN, or have we already? > Not a DTD, but I am working on an XML Schema (DTDs are a bit yesterday). +1 ... I asked for a spec for the output format before, and this would do fine. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Determining client_encoding from client locale
On Tue, Aug 18, 2009 at 6:49 AM, Heikki Linnakangas wrote: > > Hmm, are you sure you the right version of libpq is being loaded at > runtime? What does "ldd ./test-libpq" say? > i have to rebuild with the patch on linux and windows and i'm not sure i will have time until friday... once said that, in linux i'm very sure it was running with the right version of libpq... i checked with ldd after my original problems compiling the test program here: http://archives.postgresql.org/pgsql-hackers/2009-07/msg01496.php http://archives.postgresql.org/pgsql-hackers/2009-07/msg01501.php -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain root element for auto-explain
Andrew Dunstan wrote: > Bruce Momjian wrote: > > Are we going to publish an XML DTD for EXPLAIN, or have we already? > > Not a DTD, but I am working on an XML Schema (DTDs are a bit yesterday). OK, either one would be good. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain root element for auto-explain
Bruce Momjian wrote: Are we going to publish an XML DTD for EXPLAIN, or have we already? Not a DTD, but I am working on an XML Schema (DTDs are a bit yesterday). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] explain root element for auto-explain
Are we going to publish an XML DTD for EXPLAIN, or have we already?
---
Andrew Dunstan wrote:
>
> The attached tiny patch sets the root element for auto-explain
> XML output, so it looks something like this:
>
> http://www.postgresql.org/2009/explain";>
>
> Result
> 0.00
> 0.01
> 1
> 0
>
>
>
> The JSON output looks like this:
>
> [
> "Plan": {
> "Node Type": "Result",
> "Startup Cost": 0.00,
> "Total Cost": 0.01,
> "Plan Rows": 1,
> "Plan Width": 0
> }
> ]
>
> This is worth doing in itself in the XML case for reasons previously
> explained, but it also makes it relatively easy to add a Query-Text node
> or some such to the structured output, which is very much worth having,
> and would be my next proposed step.
>
> cheers
>
> andrew
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] REGRESS_OPTS versus MSVC build scripts
> -- Forwarded message --
> From: David Fetter
> To: Andrew Dunstan
> Date: Tue, 18 Aug 2009 11:31:41 -0700
> Subject: Re: REGRESS_OPTS versus MSVC build scripts
> On Tue, Aug 18, 2009 at 02:15:48PM -0400, Andrew Dunstan wrote:
>>
>>
>> Andrew Dunstan wrote:
>>>
>>
>> Here's an untested patch ... I think it should do the job, however, at
>> least for contrib modules, which is the immediate problem.
>>
>> cheers
>>
>> andrew
>>
>>
>
>> Index: src/tools/msvc/vcregress.pl
>> ===
>> RCS file: /cvsroot/pgsql/src/tools/msvc/vcregress.pl,v
>> retrieving revision 1.10
>> diff -c -r1.10 vcregress.pl
>> *** src/tools/msvc/vcregress.pl 1 Dec 2008 13:39:45 - 1.10
>> --- src/tools/msvc/vcregress.pl 18 Aug 2009 18:12:59 -
...
>> *** 198,203
>> --- 199,223
>> exit $mstat if $mstat;
>> }
>>
>> + sub fetchRegressOpts
>> + {
>> + my $handle;
>> + open($handle,"
> This section is probably better done with Tie::File.
No, absolutely not. I would have a hard time believing any computer
that would be
compiling or testing pg would not have enough memory to slurp a
Makefile into memory.
And if it did, Tie::File would not be the answer. For files with
small line lengths the
memory overhead of Tie::File is generally more than the memory savings.
And it gives us the added bonus of letting us accidentally change
files, when we
thought we were just changing an in memory copy.
The real solution in that highly unlikely case would be to read the
file line by line in
a loop, rather than slurping it, as the existing code doesn't search
across line
boundaries anyway.
Cheers,
Jeff
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW-based dblink (WIP)
Tom Lane wrote: > Pavel Stehule writes: >> 2009/8/19 Tom Lane : >>> I don't believe there is any consensus for integrating dblink into core, >>> and I for one will resist that strongly. Â Keep it in contrib. > >> if integration means, so I could to write query like >> SELECT * FROM otherdatabase.schema.table >> UPDATE otherdb.table SET ... >> I am for integration. > > That is not what "integrating dblink" means --- what Itagaki-san is > talking about is moving the dblink_xxx functions into core. What > you are talking about is actual SQL/MED functionality, which we should > indeed try to get into core someday. But dblink is a dead end as far > as standards compliance goes. Between that and the potential security > issues, we should not put it in core. +1 Joe signature.asc Description: OpenPGP digital signature
Re: [HACKERS] FDW-based dblink (WIP)
2009/8/19 Tom Lane : > Pavel Stehule writes: >> 2009/8/19 Tom Lane : >>> I don't believe there is any consensus for integrating dblink into core, >>> and I for one will resist that strongly. Keep it in contrib. > >> if integration means, so I could to write query like >> SELECT * FROM otherdatabase.schema.table >> UPDATE otherdb.table SET ... >> I am for integration. > > That is not what "integrating dblink" means --- what Itagaki-san is > talking about is moving the dblink_xxx functions into core. What > you are talking about is actual SQL/MED functionality, which we should > indeed try to get into core someday. But dblink is a dead end as far > as standards compliance goes. Between that and the potential security > issues, we should not put it in core. > aha, - ok regards Pavel Stehule > regards, tom lane > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW-based dblink (WIP)
Itagaki Takahiro wrote: > It integrates dblink module into core and adds a new functionality, > automatic transaction management. Why does it need to be integrated to core? I'd prefer to keep dblink out of core, unless there's a reason. > I want pretty much the automatic transaction management. It is useful to > write applied modules like materialized-view-over-network. But it should > be able to be turned off if we don't want it. I'll work on those parts next. The transaction management is very unsafe as it is, for the reasons I mentioned earlier. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW-based dblink (WIP)
Pavel Stehule writes: > 2009/8/19 Tom Lane : >> I don't believe there is any consensus for integrating dblink into core, >> and I for one will resist that strongly. Â Keep it in contrib. > if integration means, so I could to write query like > SELECT * FROM otherdatabase.schema.table > UPDATE otherdb.table SET ... > I am for integration. That is not what "integrating dblink" means --- what Itagaki-san is talking about is moving the dblink_xxx functions into core. What you are talking about is actual SQL/MED functionality, which we should indeed try to get into core someday. But dblink is a dead end as far as standards compliance goes. Between that and the potential security issues, we should not put it in core. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW-based dblink (WIP)
2009/8/19 Tom Lane : > Itagaki Takahiro writes: >> Here is a WIP patch for a foreign data wrapper based dblink. >> It integrates dblink module into core and adds a new functionality, >> automatic transaction management. > > I don't believe there is any consensus for integrating dblink into core, > and I for one will resist that strongly. Keep it in contrib. if integration means, so I could to write query like SELECT * FROM otherdatabase.schema.table UPDATE otherdb.table SET ... I am for integration. regards Pavel Stehule > > regards, tom lane > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW-based dblink (WIP)
Itagaki Takahiro writes: > Here is a WIP patch for a foreign data wrapper based dblink. > It integrates dblink module into core and adds a new functionality, > automatic transaction management. I don't believe there is any consensus for integrating dblink into core, and I for one will resist that strongly. Keep it in contrib. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
Greg Stark writes: > We could add another flag in pg_proc for functions which cannot throw > an error. Perhaps all index operator class operators be required to > use such functions too? It would be an extremely small set. For starters, anything that returns a pass-by-reference data type would be out automatically --- the palloc to return the result could hit out-of-memory. Now maybe we could define the flag as meaning "if this does throw an error it's okay for the calling query to fail too", which I think is reasonable for out-of-memory type errors. But it's getting pretty squishy at that point, and I think we'd have a lot of problems with mislabeled functions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
On Wed, Aug 19, 2009 at 3:16 PM, Greg Stark wrote: > On Wed, Aug 19, 2009 at 3:53 AM, Tom Lane wrote: >> * The expression might throw an error for some inputs, for instance >> (1 / field) < 0.5 >> which would fail on zero. We could recover by wrapping the whole >> estimation process in a subtransaction, but that seems really expensive. >> I thought about arguing that the failure would happen anyway at runtime, >> but that doesn't hold water --- for example, the user might have just >> deleted all the rows with field = 0, and would have grounds to complain >> if the query failed; but there would still be an entry for zero in the >> histogram. > > We could add another flag in pg_proc for functions which cannot throw > an error. Perhaps all index operator class operators be required to > use such functions too? Another thought. In the above case it would actually be fine to catch the error with PG_TRY() without a subtransaction. As long as no shared database state has been modified when the error is thrown then the subtransaction isn't needed to roll them back. In theory "immutable" should mean that's true, though there's the traditional question of whether invisible internal state changes count as mutations and it would be too fragile to depend on. What I'm wondering is if we can reliably detect any state changes that would need to be rolled back. If the PG_CATCH() fires and no state changes have occurred since the PG_TRY() then we can just continue and ignore that entry or treat it as false. If any have then we have to throw an error but if it requires very unusual types of functions then perhaps that's ok. I'm not sure if it's sufficient but does the lazy xid assignment provide a place to hook this in? We could add a counter to indicate how many times the xid was needed -- if that counter doesn't change while between the PG_TRY()/PG_CATCH() then we know there's nothing that would have needed a proper subtransaction to roll back. As I said I'm not sure that's actually sufficient. Are there other shared state that subtransactions are needed to roll back? Are there a small enumerable set or are we going to be constantly discovering new kinds of state that need to be protected against? -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea about estimating selectivity for single-column expressions
On Wed, Aug 19, 2009 at 3:53 AM, Tom Lane wrote: > * The expression might throw an error for some inputs, for instance > (1 / field) < 0.5 > which would fail on zero. We could recover by wrapping the whole > estimation process in a subtransaction, but that seems really expensive. > I thought about arguing that the failure would happen anyway at runtime, > but that doesn't hold water --- for example, the user might have just > deleted all the rows with field = 0, and would have grounds to complain > if the query failed; but there would still be an entry for zero in the > histogram. We could add another flag in pg_proc for functions which cannot throw an error. Perhaps all index operator class operators be required to use such functions too? -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_hba.conf: samehost and samenet
Magnus Hagander wrote:
> On Wed, Aug 19, 2009 at 03:58, Stef Walter wrote:
>> Attached is a new patch, which I hope addresses all the concerns raised.
>
> I think you forgot to actually attach the patch
Whoops. Here it is.
Stef
diff --git a/configure.in b/configure.in
index 505644a..bc37b1b 100644
*** a/configure.in
--- b/configure.in
*** AC_SUBST(OSSP_UUID_LIBS)
*** 962,968
##
dnl sys/socket.h is required by AC_FUNC_ACCEPT_ARGTYPES
! AC_CHECK_HEADERS([crypt.h dld.h fp_class.h getopt.h ieeefp.h langinfo.h poll.h pwd.h sys/ipc.h sys/poll.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/socket.h sys/shm.h sys/tas.h sys/time.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h kernel/OS.h kernel/image.h SupportDefs.h])
# At least on IRIX, cpp test for netinet/tcp.h will fail unless
# netinet/in.h is included first.
--- 962,968
##
dnl sys/socket.h is required by AC_FUNC_ACCEPT_ARGTYPES
! AC_CHECK_HEADERS([crypt.h dld.h fp_class.h getopt.h ieeefp.h langinfo.h poll.h pwd.h sys/ipc.h sys/poll.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/socket.h sys/shm.h sys/tas.h sys/time.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h kernel/OS.h kernel/image.h SupportDefs.h ifaddrs.h])
# At least on IRIX, cpp test for netinet/tcp.h will fail unless
# netinet/in.h is included first.
*** PGAC_VAR_INT_TIMEZONE
*** 1141,1147
AC_FUNC_ACCEPT_ARGTYPES
PGAC_FUNC_GETTIMEOFDAY_1ARG
! AC_CHECK_FUNCS([cbrt dlopen fcvt fdatasync getpeereid getpeerucred getrlimit memmove poll pstat readlink setproctitle setsid sigprocmask symlink sysconf towlower utime utimes waitpid wcstombs])
# posix_fadvise() is a no-op on Solaris, so don't incur function overhead
# by calling it, 2009-04-02
--- 1141,1147
AC_FUNC_ACCEPT_ARGTYPES
PGAC_FUNC_GETTIMEOFDAY_1ARG
! AC_CHECK_FUNCS([cbrt dlopen fcvt fdatasync getpeereid getpeerucred getrlimit memmove poll pstat readlink setproctitle setsid sigprocmask symlink sysconf towlower utime utimes waitpid wcstombs getifaddrs])
# posix_fadvise() is a no-op on Solaris, so don't incur function overhead
# by calling it, 2009-04-02
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index ad4d084..e88c796 100644
*** a/doc/src/sgml/client-auth.sgml
--- b/doc/src/sgml/client-auth.sgml
*** hostnossl database
+ Instead of an CIDR-address, you can specify
+the values samehost or samenet. To
+match any address on the subnets connected to the local machine, specify
+samenet. By specifying samehost, any
+addresses present on the network interfaces of local machine will match.
+
+
This field only applies to host,
hostssl, and hostnossl records.
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index b3df7ee..5d56603 100644
*** a/src/backend/libpq/hba.c
--- b/src/backend/libpq/hba.c
*** check_db(const char *dbname, const char
*** 564,569
--- 564,660
return false;
}
+ /*
+ * Check to see if a connecting IP matches the address and netmask.
+ */
+ static bool
+ check_ip (SockAddr *raddr, struct sockaddr *addr, struct sockaddr *mask)
+ {
+ if (raddr->addr.ss_family == addr->sa_family)
+ {
+ /* Same address family */
+ if (!pg_range_sockaddr(&raddr->addr, (struct sockaddr_storage*)addr,
+ (struct sockaddr_storage*)mask))
+ return false;
+ }
+ #ifdef HAVE_IPV6
+ else if (addr->sa_family == AF_INET &&
+ raddr->addr.ss_family == AF_INET6)
+ {
+ /*
+ * Wrong address family. We allow only one case: if the file
+ * has IPv4 and the port is IPv6, promote the file address to
+ * IPv6 and try to match that way.
+ */
+ struct sockaddr_storage addrcopy,
+ maskcopy;
+
+ memcpy(&addrcopy, &addr, sizeof(addrcopy));
+ memcpy(&maskcopy, &mask, sizeof(maskcopy));
+ pg_promote_v4_to_v6_addr(&addrcopy);
+ pg_promote_v4_to_v6_mask(&maskcopy);
+
+ if (!pg_range_sockaddr(&raddr->addr, &addrcopy, &maskcopy))
+ return false;
+ }
+ #endif /* HAVE_IPV6 */
+ else
+ {
+ /* Wrong address family, no IPV6 */
+ return false;
+ }
+
+ return true;
+ }
+
+ typedef struct CheckNetwork {
+ NetMethod method;
+ SockAddr *raddr;
+ bool result;
+ } CheckNetwork;
+
+ static void
+ callback_check_network (struct sockaddr *addr, struct sockaddr *netmask, void *data)
+ {
+ CheckNetwork *cn = data;
+ struct sockaddr_storage mask;
+
+ /* Already found a match */
+ if (cn->result)
+ return;
+
+ /* Make a fully 1's netmask of appropriate length */
+ if (cn->method == nmSameHost)
+ {
+ pg_sockaddr_cidr_mask (&mask, NULL, addr->sa_family);
+ cn->result = check_ip (cn->raddr, addr, (struct sockaddr*)&mask);
+ }
+
+ /* Use the netmask of the interface itself */
+ else
+ {
+ cn->result = check_ip (cn->raddr, addr, netmask);
+ }
+ }
+
+ static bool
+ check_same_host_or_net (SockAddr *ra
Re: [HACKERS] pg_hba.conf: samehost and samenet
On Wed, Aug 19, 2009 at 03:58, Stef Walter wrote: > Attached is a new patch, which I hope addresses all the concerns raised. I think you forgot to actually attach the patch (others have taken care of the question about login already I see) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] alpha1 bundled -- please verify
Alpha1 has been bundled and is available at http://developer.postgresql.org/~petere/alpha/ Please check that it is sane. Then, someone please move this to an appropriate place on the FTP server and make an announcement. See http://wiki.postgresql.org/wiki/Alpha_release_process for process details. I'm out for the rest of the day. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4961: pg_standby.exe crashes with no args
Magnus Hagander wrote: > This would amount to fairly major surgery for pg_standby on Win32. Is > that something we'd want to backpatch, or do we want to backpatch just > the removal of the signal() calls which would amount to not supporting > signals in pg_standby on win32? I think we should just remove the signals support for win32. The trigger file method still works, and the signal method has always been a bit iffy (it doesn't work when pg_standby isn't running, for example, which happens between processing of each WAL file, because there's no process to signal). Is pg_standby killed correctly when postmaster dies? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4961: pg_standby.exe crashes with no args
On Wed, Aug 19, 2009 at 08:38, Fujii Masao wrote:
> On Thu, Aug 13, 2009 at 2:24 AM, Magnus Hagander wrote:
>> Not sure. Potentially pure luck. SIGINT has never *worked*, though, it
>> just hasn't crashed.
>
> OK.
>
>> We could implement the same type of check in pg_standby, but it
>> requires something like CHECK_FOR_INTERRUPTS. And these interrupts
>> won't, by default, cause any kind of interruption of the process. In
>> the backend, we interrupt socket calls because we have the socket
>> wrapper layer, and nothing else. I don't know how doable this would be
>> in pg_standby - does it always block on a single thing where we could
>> stick some win32 synchronization code? If it's a single, or limited,
>> places we could implement something similar to the backend. But if we
>> need to interrupt at arbitrary locations, that's just not possible.
>
> I think that CHECK_FOR_INTERRUPTS should be placed just before
> checking the flag 'signaled' which may be enabled by the signal handler.
> Here is the pseudo-code.
>
>
> {
> /* Check for trigger file or signal first */
> CheckForExternalTrigger();
> + #ifdef WIN32
> + CHECK_FOR_INTERRUPTS();
> + #endif /* WIN32 */
> if (signaled)
> {
> Failover = FastFailover;
>
It's going to take a lot more than that, really. I looked a bit more
at it, and I think the best way is to do a win32 specific thing all
the way, not just emulate signals. Given that we only have a couple of
signal() calls anyway. I think that's going to be much clearer in what
it does, and also a lot simpler code in the end. The reason we have
the full emulation layer is that we use signals all over the place in
the backend.
From what I can tell, we don't actually need to interrupt anything
other than the sleep call on line 810, because we only check for the
signaled=true state in that loop anyway. Can someone more
knowledgeable in the pg_standby code comment on that?
This would amount to fairly major surgery for pg_standby on Win32. Is
that something we'd want to backpatch, or do we want to backpatch just
the removal of the signal() calls which would amount to not supporting
signals in pg_standby on win32?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Geometric bewilderment
Suspect that attaching large amounts of code is a breach of etiquette. So apologies in advance. Needed to write a contains(polygon,point) function that gave a level of accuracy greater than the current 'poly@>point' operator. The new function works just fine. While at it, thought it would be nice to add in all the 'box op point' and 'point op box' routines that are not currently defined. The code in contains.c implements these functions. In contains.sql you can find how these functions are defined, tided to operators, and then placed in an operator class. (Permission granted to copy any code in here you like, it's all trivial) The perl script test1 tests all the operators. It compares the output of "box op point" with "box op box(point,point)". It is called with one parameter, the name of the database to use. It creates a table called points within that database. It also creates two files in the current directory a.out and b.out. tkdiff is called to show the differences. Running this shows that all the operators work fine. The perl script test2 tests 1000 points against 1000 boxes. It is called with one parameter, the name of the database to use. It creates two tables called points and boxes within that database. It also creates two files in the current directory a.out and b.out. tkdiff is called to show the differences. Running this shows that the indexing works fine. (Explain says it uses the index) Problem: The following code works in production with 20,000,000 points against 10,000 polygons. SELECT W.geocode, F.state_code, F.area_code, F.area_name FROM work as W, boundary as B, features as F WHERE B.boundbox @> box(W.geocode,W.geocode) AND contains(B.boundary,W.geocode) AND B.boundout = 'T' AND (B.feature_id) NOT IN ( SELECT feature_id FROM boundary WHERE boundbox @> box(W.geocode,W.geocode) AND contains(boundary,W.geocode) AND boundout = 'F' ) AND B.feature_id = F.feature_id; However the following does not work. It returns an empty set. Note that all that has changed is that "box @> box(point,point)" has been changed to "box @> point". From test1 we know that the operators work, and from test2 we know the indexing works. Confused and bewildered at this point. SELECT W.geocode, F.state_code, F.area_code, F.area_name FROM work as W, boundary as B, features as F WHERE B.boundbox @> W.geocode AND contains(B.boundary,W.geocode) AND B.boundout = 'T' AND (B.feature_id) NOT IN ( SELECT feature_id FROM boundary WHERE boundbox @> W.geocode AND contains(boundary,W.geocode) AND boundout = 'F' ) AND B.feature_id = F.feature_id; /*= * Contains PostgreSQL extention * * PostgreSQL (poly@>point) operator is currently broken beyond a certain * precision. This module provides a contains(poly,point) to use as a * replacement. * * It addition it adds the currently missing box to point operators, and * conversly the point to box operators as well. * * (c) 2009 Paul Matthews. LGPL ?? .. (to be advised) *===*/ #include "postgres.h" #include "utils/geo_decls.h" #include "fmgr.h" #ifdef PG_MODULE_MAGIC PG_MODULE_MAGIC; #endif /*- * Box op Point routines * * strictly left of 1<box_point_notleft * strictly right of 5>>box_point_rightof * contains 7@>box_point_contains * does not extend above 9&<| box_point_notabove * strictly below 10 <<| box_point_below * strictly above 11 |>> box_point_above * does not extend below 12 |&> box_point_notbelow *---*/ PG_FUNCTION_INFO_V1(box_point_leftof); PG_FUNCTION_INFO_V1(box_point_notright); PG_FUNCTION_INFO_V1(box_point_notleft); PG_FUNCTION_INFO_V1(box_point_rightof); PG_FUNCTION_INFO_V1(box_point_contains); PG_FUNCTION_INFO_V1(box_point_notabove); PG_FUNCTION_INFO_V1(box_point_below); PG_FUNCTION_INFO_V1(box_point_above); PG_FUNCTION_INFO_V1(box_point_notbelow); /* * Box strictly left of point. box << point */ Datum box_point_leftof(PG_FUNCTION_ARGS) { BOX *box = PG_GETARG_BOX_P(0); Point *point = PG_GETARG_POINT_P(1); PG_RETURN_BOOL( box->high.x < point->x ); } /* * Box does not extend right of point. box &< point */ Datum box_point_notright(PG_FUNCTION_ARGS) { BOX *box = PG_GETARG_BOX_P(0);
[HACKERS] Why ACL_EXECUTE is checked on FindConversion()?
When FindConversion() is called, it also checks current user's ACL_EXECUTE privilege on the conproc of the fetched conversion. Why this check is applied on FindConversion(), instead of FindDefaultConversion()? The FindConversion() returns the OID of conversion for the given name and namespace, or InvalidOid if not found or user does not have ACL_EXECUTE privilege. It is called from DropConversionsCommand(), RenameConversion(), AlterConversionOwner() and CommentConversion(), to obtain OID of the target conversion to be modified by DDL statement. On the other hand, FindDefaultConversionProc() does not apply such kind of permission checks, though it is called from SetClientEncoding(). The conversion procedure is implicitly called when user communicates to the server backend, so it seems to me quite natural if FindDefaultConversionProc() checks user's ACL_EXECUTE privilege. But it is checked when we lookup the target conversion on DDL statement. It's unclear for me what is the intension of this check. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PQgetlength vs. octet_length()
Greg Stark wrote: > If you use binary encoding then you don't have to deal with that. > Though I seem to recall there is still a gotcha you have to worry > about if there are nul bytes in your datum. I don't recall exactly > what that meant you had to do though. As far as I know, it only means that you shouldn't use strcpy, strlen and friends on a binary string. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
