[HACKERS] Re: SPGiST versus hot standby - question about conflict resolution rules
On Mon, Mar 12, 2012 at 10:50:36PM -0400, Tom Lane wrote: > There is one more (known) stop-ship problem in SPGiST, which I'd kind of > like to get out of the way now before I let my knowledge of that code > get swapped out again. This is that SPGiST is unsafe for use by hot > standby slaves. I suspect that swap-out has passed, but ... > The problem comes from "redirect" tuples, which are short-lifespan > objects that replace a tuple that's been moved to another page. > A redirect tuple can be recycled as soon as no active indexscan could > be "in flight" from the parent index page to the moved tuple. SPGiST > implements this by marking each redirect tuple with the XID of the > creating transaction, and assuming that the tuple can be recycled once > that XID is below the OldestXmin horizon (implying that all active > transactions started after it ended). This is fine as far as > transactions on the master are concerned, but there is no guarantee that > the recycling WAL record couldn't be replayed on a hot standby slave > while there are still HS transactions that saw the old state of the > parent index tuple. > > Now, btree has a very similar problem with deciding when it's safe to > recycle a deleted index page: it has to wait out transactions that could > be in flight to the page, and it does that by marking deleted pages with > XIDs. I see that the problem has been patched for btree by emitting a > special WAL record just before a page is recycled. However, I'm a bit > nervous about copying that solution, because the details are a bit > different. In particular, I see that btree marks deleted pages with > ReadNewTransactionId() --- that is, the next-to-be-assigned XID --- > rather than the XID of the originating transaction, and then it > subtracts one from the XID before sending it to the WAL stream. > The comments about this are not clear enough for me, and so I'm Attempting to write an explanation for that btree code led me conclude that the code is incorrect. (FWIW, I caused that wrongness.) I will start a separate thread to fix it. > wondering whether it's okay to use the originating transaction XID > in a similar way, or if we need to modify SPGiST's rule for how to > mark redirection tuples. I think that the use of ReadNewTransactionId > is because btree page deletion happens in VACUUM, which does not have > its own XID; this is unlike the situation for SPGiST where creation of > redirects is caused by index tuple insertion, so there is a surrounding > transaction with a real XID. But it's not clear to me how > GetConflictingVirtualXIDs makes use of the limitXmin and whether a live > XID is okay to pass to it, or whether we actually need "next XID - 1". > > Info appreciated. GetConflictingVirtualXIDs() selects transactions with pgaxt->xmin <= limitXmin. The prototype use case was VACUUM, where limitXmin is the xmax of a dead tuple cleaned from a page. Transactions with xmin <= limitXmin might still regard the limitXmin XID as uncommitted; we conflict so they cannot fail to see the tuple we're about to purge. All hot standby transactions holding snapshots taken before the startup process applies the tuple-mover transaction's commit record will have xmin <= its XID. Therefore, passing that XID to ResolveRecoveryConflictWithSnapshot() meets the need here precisely. vacuum_defer_cleanup_age and hot_standby_feedback inform GetOldestXmin(), so the master will delay cleanup long enough to prevent conflicts when so configured. Thanks, nm -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join
On Tue, 2012-04-17 at 14:24 -0400, Robert Haas wrote: > I thought Jeff was parenthetically complaining about cases like A LEFT > JOIN (B INNER JOIN C ON b.y = c.y) ON a.x && b.x. That presumably > would require the parameterized-path stuff to have any chance of doing > partial index scans over B. However, I understand that's not the main > issue here. To take the mystery out of it, I was talking about any case where an index scan is impossible or impractical. For instance, let's say the ranges are computed values. Just to make it really impossible, let's say the ranges are computed from columns in two different tables joined in a subquery. But yes, the ability of the planner to find the plan is also an issue (hopefully less of one with the recent improvements). > One thing that I think needs some analysis is when the range join idea > is better or worse than a nested loop with inner index-scan, because > potentially those are the options the planner has to choose between, > and the costing model had better know enough to make the right thing > happen. It strikes me that the nested loop with inner index-scan is > likely to be a win when there are large chunks of the indexed relation > that the nestloop never needs to visit at all - imagine small JOIN big > ON small.a && big.a, for example. I suppose the really interesting > question is how much we can save when the entirety of both relations > has to be visited anyway - it seems promising, but I guess we won't > know for sure without testing it. Right, I will need to come up with a prototype that can at least test the executor piece. I suspect that the plan choice won't be all that different from an ordinary index nestloop versus mergejoin case, but with much worse cardinality estimates to work with. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Improving our clauseless-join heuristics
Amit Kapila writes: > The way I am telling was as below code. > With this extra paths will get generated, but it will as well consider for > joining c and d in query: > select * from a, b, c, d where a.x = b.y and (a.z = c.c or a.z = d.d) I think this would just be dead code as of HEAD. With the recent changes in the definition of have_relevant_joinclause, if we get into make_rels_by_clause_joins there should always be at least one other relation that the old_rel is considered to join to. In any case, I don't see the point of adding more logic to make_rels_by_clause_joins unless it allows us to take out as much or more work elsewhere. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments
Hello, this is new version of standby checkpoint_segments patch.
- xlog.c: Make StandbyMode shared.
- checkpointer.c: Use IsStandbyMode() to check if postmaster is
under standby mode.
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
== My e-mail address has been changed since Apr. 1, 2012.
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 8d0aabf..2457840 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -177,6 +177,12 @@ static bool LocalRecoveryInProgress = true;
static bool LocalHotStandbyActive = false;
/*
+ * Local copy of SharedIsStandbyMode variable. True actually means "not known,
+ * need to check the shared state".
+ */
+static bool LocalIsStandbyMode = true;
+
+/*
* Local state for XLogInsertAllowed():
* 1: unconditionally allowed to insert XLOG
* 0: unconditionally not allowed to insert XLOG
@@ -206,7 +212,6 @@ static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
/* options taken from recovery.conf for XLOG streaming */
-static bool StandbyMode = false;
static char *PrimaryConnInfo = NULL;
static char *TriggerFile = NULL;
@@ -427,6 +432,11 @@ typedef struct XLogCtlData
bool SharedHotStandbyActive;
/*
+ * SharedInStandbyMode indicates if we are running in standby mode.
+ */
+ bool SharedIsStandbyMode;
+
+ /*
* recoveryWakeupLatch is used to wake up the startup process to continue
* WAL replay, if it is waiting for WAL to arrive or failover trigger file
* to appear.
@@ -619,6 +629,7 @@ static void SetLatestXTime(TimestampTz xtime);
static void SetCurrentChunkStartTime(TimestampTz xtime);
static void CheckRequiredParameterValues(void);
static void XLogReportParameters(void);
+static void ExitStandbyMode(void);
static void LocalSetXLogInsertAllowed(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
static void KeepLogSeg(XLogRecPtr recptr, uint32 *logId, uint32 *logSeg);
@@ -3115,7 +3126,7 @@ RestoreArchivedFile(char *path, const char *xlogfname,
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
-if (StandbyMode && stat_buf.st_size < expectedSize)
+if (IsStandbyMode() && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
@@ -4072,7 +4083,7 @@ next_record_is_invalid:
}
/* In standby-mode, keep trying */
- if (StandbyMode)
+ if (IsStandbyMode())
goto retry;
else
return NULL;
@@ -5098,6 +5109,7 @@ XLOGShmemInit(void)
XLogCtl->XLogCacheBlck = XLOGbuffers - 1;
XLogCtl->SharedRecoveryInProgress = true;
XLogCtl->SharedHotStandbyActive = false;
+ XLogCtl->SharedIsStandbyMode = true;
XLogCtl->Insert.currpage = (XLogPageHeader) (XLogCtl->pages);
SpinLockInit(&XLogCtl->info_lck);
InitSharedLatch(&XLogCtl->recoveryWakeupLatch);
@@ -5289,6 +5301,7 @@ readRecoveryCommandFile(void)
FILE *fd;
TimeLineID rtli = 0;
bool rtliGiven = false;
+ boolstandby_mode = false;
ConfigVariable *item,
*head = NULL,
*tail = NULL;
@@ -5439,13 +5452,14 @@ readRecoveryCommandFile(void)
}
else if (strcmp(item->name, "standby_mode") == 0)
{
- if (!parse_bool(item->value, &StandbyMode))
+ if (!parse_bool(item->value, &standby_mode))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("parameter \"%s\" requires a Boolean value",
"standby_mode")));
ereport(DEBUG2,
(errmsg_internal("standby_mode = '%s'", item->value)));
+
}
else if (strcmp(item->name, "primary_conninfo") == 0)
{
@@ -5470,7 +5484,7 @@ readRecoveryCommandFile(void)
/*
* Check for compulsory parameters
*/
- if (StandbyMode)
+ if (standby_mode)
{
if (PrimaryConnInfo == NULL && recoveryRestoreCommand == NULL)
ereport(WARNING,
@@ -5480,6 +5494,7 @@ readRecoveryCommandFile(void)
}
else
{
+ ExitStandbyMode();
if (recoveryRestoreCommand == NULL)
ereport(FATAL,
(errmsg("recovery command file \"%s\" must specify restore_command when standby mode is not enabled",
@@ -6086,7 +6101,7 @@ StartupXLOG(void)
if (InArchiveRecovery)
{
- if (StandbyMode)
+ if (IsStandbyMode())
ereport(LOG,
(errmsg("entering standby mode")));
else if (recoveryTarget == RECOVERY_TARGET_XID)
@@ -6110,7 +6125,7 @@ StartupXLOG(void)
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
- if (StandbyMode)
+ if (IsStandbyMode())
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired,
@@ -6169,7 +6184,7 @@ StartupXLOG(void)
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
- else if (StandbyMode)
+ else if (IsStandbyMode())
{
/*
* The last valid checkpoint record required for a streaming
@@ -6683,7 +6698,7 @@ StartupXLOG(void)
* We don't need the latch anymore. It's not strictly nece
[HACKERS] clarification for generate join implied equalities
In function generate_join_implied_equalities_normal(), in which kind of query it can have all three outer_members, inner_members, new_members set? There is handling related to that in the same function down which I wanted to understand.
Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments
I convinced that current patch has a problem, and will come up with the new patch later. > > I tried that at first. But I suppose the requirement here is 'if > > reading segments comes via replication stream, enable throttling > > by checkpoint_segments.' and WalRcvInProgress() seems fit to > > check that. > > If so, what if replication is terminated and restarted repeatedly while > a restartpoint is running? It sometimes obeys and sometimes ignores > checkpoint_segments. Which seems strange behavior. I see your point. And agree on that is something not should be. > > Plus, adding SharedStartupStandbyMode into > > XLogCtlData seems accompanied with some annoyances which would > > not pay. > > Hmm... what are you worried about? I don't think that sharing the variable > via XLogCtl is so difficult. Please see the code to share > archiveCleanupCommand > from the startup process to the checkpointer. It looks very simple. The mechanism has nothing complex. I've been afraid of making so many variables with similar meaning sitting side by side on shared memory. But I convinced that additional shared variable is preferable because it makes the logic and behavior clear and sane. I will come up with updated patch soon. > > By the way, do you have some advise about GetStandbyFlushRecPtr() > > and the order of the locations? I'm embarrassed with that... > > Sorry. I could not parse this Could you explain this again? My point is, - Is the assumption correct that "restorePtr <= replayPtr <= receivePtr"? - If correct, what the code in GetStandbyFlushRecPtr() showing below means? > if (XLByteLT(receivePtr, replayPtr)) - Or if wrong, what situation would take place to break the expression "restorePtr <= replayPtr <= receivePtr"? regards, -- Kyotaro Horiguchi NTT Open Source Software Center == My e-mail address has been changed since Apr. 1, 2012. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Timsort performance, quicksort (was: Re: Memory usage during sorting)
On 17 April 2012 13:19, Greg Stark wrote:
> All in all I think it's handier to have a stable ORDER BY sort than an
> unstable one though. So I'm not necessarily opposed to it even if it
> means we're stuck using a stable sort indefinitely.
I think it might be useful to disguard the stability property if it's
not a real requirement, as I think doing so gives us additional leeway
to compose descending runs (pre-sorted subsets) that are not in what
is termed "strictly descending" order (that is, they can be a[0] >=
a[1] >= a[2] >= ... and not just a[0] > a[1] > a[2] > ...).
I'll share some preliminary findings on timsort performance (and,
indeed, quicksort performance). I decided on two intial tests - one
that tested performance of sorting against master for a reasonable
case, and the other for a rather sympathetic case.
The first test was for a pgbench run of a query against the dellstore
database, "select * from products order by actor offset 10001",
pgbench -T 300. Here, actors is a text column. I didn't look at scalar
types, because presumably quicksort is going to do much better there.
After running analyze on the table, the pg_stats.correlation is
-0.00493428. There is at best a very weak physical to logical
correlation for the column, as far as the random sampling of analyze
can tell, though there may well still be many individual "runs" of
ordered subsets - I should be able to instrument timsort to determine
how pre-sorted data already is in a well-principled fashion, but not
today.
master:
tps = 43.985745 (including connections establishing)
tps = 43.986028 (excluding connections establishing)
timsort:
tps = 39.181766 (including connections establishing)
tps = 39.182130 (excluding connections establishing)
Here, quicksort benefits somewhat from my earlier work (though there
is no SortSupport entry in any of the tests performed), as a full
tuplesort specialisation is used. timsort_arg is simply a drop-in
replacement for qsort_arg. It's fairly close, but quicksort does win
out here, which is perhaps not hugely surprising. Timsort does of
course have to maintain state like pending runs in a way that
quicksort does not, and quicksort is expected to take advantage of
memory locality to a greater degree. However, if we measure the
expense of the sorts in pure terms of number of comparisons, an
altogether different picture emerges:
timsort: 119,943
quicksort: 136,071
I'd like to see what happens when timsort_arg is further specialised
(not that I envisage multiple specialisations of it or anything - just
a single tuplesort one).
I contrived a very sympathetic test-case too. Namely, I created a new
numeric column for the dellstore database's orderlines table, with a
default value of "nextval('some_seq'), resulting in a
pg_stat.correlation of 1. Then, I changed the actual value of the very
last tuple in the table, with the intention of tripping up our
quicksort "check for pre-sorted array in a single bubblesort iteration
first" optimisation.
Now, I'll grant you that that was a very carefully placed banana skin,
but the results were even still quite surprising and interesting. With
the query "select * from orderlines order by test offset 60351", a
rather large gap in the performance of quicksort and timsort emerges:
Master:
=
(first run)
number of transactions actually processed: 1891
tps = 6.301991 (including connections establishing)
tps = 6.302041 (excluding connections establishing)
(second run)
number of transactions actually processed: 1878
tps = 6.259945 (including connections establishing)
tps = 6.260839 (excluding connections establishing)
timsort:
=
(first run)
number of transactions actually processed: 19997
tps = 66.655289 (including connections establishing)
tps = 66.655820 (excluding connections establishing)
(second run)
number of transactions actually processed: 19880
tps = 66.266234 (including connections establishing)
tps = 66.266810 (excluding connections establishing)
I can reproduce this result consistently - these two sets of figures
were produced hours apart.
Number of comparisons for single execution of this more sympathetic query:
Timsort: 60,351
Quicksort: 2,247,314 (yes, really)
The difference here is very big, and cannot be adequately explained by
the mere wastage of a single "bubble sort iteration to check if it's
presorted". I have heard a few stories about weird quicksort
edgecases, and you probably have too, but I don't know for sure which
one this might be right now. My theory is that we're paying a high
price for "going quadratic in the face of many duplicates" protection,
by following Sedgewick's advice and doing a partition swap in response
to the pivot and element being equal (which they pretty much always
are here). This normally isn't so obvious, because of the "check for
pre-sorted input" thing usually takes care of this instead.
My next step is to actually try out a benchmark with a more realistic
dataset, that still reasonably showcas
Re: [HACKERS] Parameterized-path cost comparisons need some work
So while testing this patch I've found out that there is a pretty nasty bug in HEAD as well as in my current formulation of the patch. Here is an example using the regression database: select count(*) from (tenk1 a join tenk1 b on a.unique1=b.unique2) left join tenk1 c on a.unique2 = b.unique1 and c.thousand = a.thousand join int4_tbl on b.thousand = f1; The correct answer to this query is 10, according to all previous PG branches, but HEAD is reporting zero. A look at the query plan soon finds the smoking gun: Aggregate (cost=264.29..264.30 rows=1 width=0) -> Nested Loop Left Join (cost=0.00..264.16 rows=50 width=0) Join Filter: (a.unique2 = b.unique1) -> Nested Loop (cost=0.00..234.21 rows=50 width=12) Join Filter: (b.unique2 = a.unique1) -> Nested Loop (cost=0.00..211.85 rows=50 width=8) -> Seq Scan on int4_tbl (cost=0.00..1.05 rows=5 width=4) -> Index Scan using tenk1_thous_tenthous on tenk1 b (cost=0.00..42.04 rows=10 width=12) Index Cond: (thousand = int4_tbl.f1) -> Index Scan using tenk1_unique2 on tenk1 a (cost=0.00..0.43 rows=1 width=12) Index Cond: (unique2 = b.unique1) -> Index Only Scan using tenk1_thous_tenthous on tenk1 c (cost=0.00..0.45 rows=10 width=4) Index Cond: (thousand = a.thousand) The condition a.unique2 = b.unique1 is getting pushed down into the a/b join, where it should not go; applying it there causes a/b join rows to be discarded when they ought to survive and then be null-extended at the left join. What this demonstrates is that the rule for identifying safe movable clauses that's used in HEAD is not good enough. It rejects clauses that reference relations that are on the inside of a left join relative to the target relation --- but the problematic clause doesn't actually reference c, so that doesn't trigger. The issue would go away if we examined required_relids instead of clause_relids, but that causes other, perfectly safe, optimizations to be discarded. I believe that the correct formulation of the join clause movement rule is "outer join clauses can't be pushed into the left-hand side of their outer join". However, the present representation of clauses doesn't provide enough information to test this cheaply during scan planning (indeed maybe it can't be done at all after distribute_qual_to_rels, because we don't retain any explicit memory of which clauses are above or below which outer joins). Looking at nullable_relids isn't the right thing because what that tells you about is what's on the right-hand side of the outer join, not the left side. So I'm coming to the conclusion that to get this right, we need to add another relids field to RestrictInfo and have it filled in during distribute_qual_to_rels. This is really probably going to end up cheaper than what we have today, because the join clause movement tests are somewhat expensive as the code stands, and it should be possible to reduce the number of operations needed there. What I have in mind is that the new field would be named something like outer_left_relids, and would list the relids of all base rels that are in the left-hand side of the outer join that the clause belongs to (or be NULL if the clause isn't an outer-join clause). This is cheaply constructable during distribute_qual_to_rels, we just are not bothering to record the information at present. Then the join clause movement checks can easily detect that it would be unsafe to push down such a clause to any of the listed relations. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug #6593, extensions, and proposed new patch policy
On Wed, Apr 18, 2012 at 7:54 PM, Tom Lane wrote: > However, ignoring that issue for the moment, this patch is making me > uncomfortable. I have a vague recollection that we deliberately omitted > ALTER EXTENSION OWNER because of security or definitional worries. > (Dimitri, does that ring any bells?) I wonder whether we should insist > that the new owner be a superuser, as the original owner must have been. Don't we have non-superuser extensions, that can be installed with just DBA privileges? Anyhow, it seems a bit nannyish, unless I'm missing something. If the current owner is a superuser and s/he wants to give the object to a non-superuser, you can't really stop them. They can just make the target user a superuser, give 'em the object, and make them not a superuser, all in one transaction no less. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug #6593, extensions, and proposed new patch policy
Robert Haas writes: > On Wed, Apr 18, 2012 at 5:27 PM, Alvaro Herrera > wrote: >> Here's a patch for that. > Looks sane on a quick once-over. I do wonder about the comment, > though. If we add ALTER EXTENSION .. OWNER, should that try to change > the ownership of the objects contained inside the extension? I would certainly think that not doing so would violate the principle of least astonishment. > Your > comment implies that the answer should be yes, but I'm not totally > convinced... what if the user has altered the ownership of the > objects manually, for example? So? ALTER OWNER doesn't care about the previous ownership of objects, it just reassigns them as told. So even if that had been done, I'd expect the post-ALTER state to be that everything has the new owner. However, ignoring that issue for the moment, this patch is making me uncomfortable. I have a vague recollection that we deliberately omitted ALTER EXTENSION OWNER because of security or definitional worries. (Dimitri, does that ring any bells?) I wonder whether we should insist that the new owner be a superuser, as the original owner must have been. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug #6593, extensions, and proposed new patch policy
On Wed, Apr 18, 2012 at 5:27 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mié abr 18 13:05:03 -0300 2012: >> On Wed, Apr 18, 2012 at 11:41 AM, Alvaro Herrera >> wrote: >> > Per bug #6593, REASSIGN OWNED fails when the affected role owns an >> > extension. This would be trivial to fix if extensions had support code >> > for ALTER EXTENSION / OWNER, but they don't. So the only back-patchable >> > fix right now seems to be to throw an error on REASSIGN OWNED when the >> > user owns an extension. (If anyone wants to claim that we ought to work >> > on a real fix that allows changing the owner internally from REASSIGN >> > OWNED, without introducing ALTER EXTENSION support for doing so, let me >> > know and I'll see about it.) >> >> I would be OK with the latter. > > Here's a patch for that. Looks sane on a quick once-over. I do wonder about the comment, though. If we add ALTER EXTENSION .. OWNER, should that try to change the ownership of the objects contained inside the extension? Your comment implies that the answer should be yes, but I'm not totally convinced... what if the user has altered the ownership of the objects manually, for example? I guess that's a question for another day, just wondering out loud. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug #6593, extensions, and proposed new patch policy
Excerpts from Robert Haas's message of mié abr 18 13:05:03 -0300 2012: > On Wed, Apr 18, 2012 at 11:41 AM, Alvaro Herrera > wrote: > > Per bug #6593, REASSIGN OWNED fails when the affected role owns an > > extension. This would be trivial to fix if extensions had support code > > for ALTER EXTENSION / OWNER, but they don't. So the only back-patchable > > fix right now seems to be to throw an error on REASSIGN OWNED when the > > user owns an extension. (If anyone wants to claim that we ought to work > > on a real fix that allows changing the owner internally from REASSIGN > > OWNED, without introducing ALTER EXTENSION support for doing so, let me > > know and I'll see about it.) > > I would be OK with the latter. Here's a patch for that. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support reassign-extension.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq URI and regression testing
Andrew Dunstan writes:
>>> That's one reason for that, but there are probably others in the way of
>>> making this fully portable and automatable.
>
> This test setup also appears to labor under the illusion that we live
> in a Unix-only world. And for no good reason that I can tell. The
> shell script should be ripped out and replaced by a perl script which
> could actually be used on any windows build host. The MSVC build
> system also needs adjusting to make it build the test driver, at
> least.
Good catch. Attached is my first take at writing Perl: replaces the
shell script, adds $libpq_uri_regress project to Mkvcbuild.pm.
I don't have access to a win32 box unfortunately, so if anyone who does
could try this out that'd be great.
--
Regards,
Alex
diff --git a/src/interfaces/libpq/test/Makefile b/src/interfaces/libpq/test/Makefile
index b9023c3..f569fc2 100644
--- a/src/interfaces/libpq/test/Makefile
+++ b/src/interfaces/libpq/test/Makefile
@@ -15,7 +15,7 @@ all: $(PROGS)
installcheck: all
SRCDIR='$(top_srcdir)' SUBDIR='$(subdir)' \
- $(SHELL) $(top_srcdir)/$(subdir)/regress.sh
+ $(top_srcdir)/$(subdir)/regress.pl
clean distclean maintainer-clean:
rm -f $(PROGS)
diff --git a/src/interfaces/libpq/test/regress.pl b/src/interfaces/libpq/test/regress.pl
new file mode 100755
index 000..74a877a
--- /dev/null
+++ b/src/interfaces/libpq/test/regress.pl
@@ -0,0 +1,56 @@
+#!/usr/bin/env perl
+use strict;
+
+# use of SRCDIR/SUBDIR is required for supporting VPath builds
+my $srcdir = $ENV{'SRCDIR'} or die '$SRCDIR environment variable is not set';
+my $subdir = $ENV{'SUBDIR'} or die '$SUBDIR environment variable is not set';
+
+my $regress_in = "$srcdir/$subdir/regress.in";
+my $expected_out = "$srcdir/$subdir/expected.out";
+
+# the output file should land in the build_dir of VPath, or just in
+# the current dir, if VPath isn't used
+my $regress_out = "regress.out";
+
+# open input file first, so possible error isn't sent to redirected STDERR
+open(REGRESS_IN, "<$regress_in") or die "Can't open <$regress_in: $!";
+
+# save STDOUT/ERR and redirect both to regress.out
+open(OLDOUT, ">&STDOUT") or die "Can't dup STDOUT: $!";
+open(OLDERR, ">&STDERR") or die "Can't dup STDERR: $!";
+
+open(STDOUT, ">$regress_out") or die "Can't open >$regress_out: $!";
+open(STDERR, ">&STDOUT") or die "Can't dup STDOUT: $!";
+
+# read lines from regress.in and run uri-regress on them
+while () {
+ chomp;
+ print "trying $_\n";
+ system("./uri-regress \"$_\"");
+ print "\n";
+}
+
+# restore STDOUT/ERR so we can print the outcome to the user
+open(STDERR, ">&OLDERR") or die; # can't complain as STDERR is still duped
+open(STDOUT, ">&OLDOUT") or die "Can't restore STDOUT: $!";
+
+# just in case
+close REGRESS_IN;
+
+my $diff_status = system("diff -c \"$srcdir/$subdir/expected.out\" regress.out >regress.diff");
+if ($diff_status == 0) {
+ print &1
-
-if diff -c "${SRCDIR}/${SUBDIR}/"expected.out regress.out >regress.diff; then
- echo ""
- echo "All tests passed"
- exit 0
-else
- echo ""
- echo "FAILED: the test result differs from the expected output"
- echo
- echo "Review the difference in ${SUBDIR}/regress.diff"
- echo ""
- exit 1
-fi
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f0fad43..e65971c 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -229,6 +229,15 @@ sub mkvcbuild
$libpq->ReplaceFile('src\interfaces\libpq\libpqrc.c','src\interfaces\libpq\libpq.rc');
$libpq->AddReference($libpgport);
+my $libpq_uri_regress = $solution->AddProject('libpq_uri_regress','exe','misc');
+$libpq_uri_regress->AddFile('src\interfaces\libpq\test\uri-regress.c');
+$libpq_uri_regress->AddIncludeDir('src\port');
+$libpq_uri_regress->AddIncludeDir('src\interfaces\libpq');
+$libpq_uri_regress->AddLibrary('wsock32.lib');
+$libpq_uri_regress->AddDefine('HOST_TUPLE="i686-pc-win32vc"');
+$libpq_uri_regress->AddDefine('FRONTEND');
+$libpq_uri_regress->AddReference($libpq,$libpgport);
+
my $libpqwalreceiver = $solution->AddProject('libpqwalreceiver', 'dll', '',
'src\backend\replication\libpqwalreceiver');
$libpqwalreceiver->AddIncludeDir('src\interfaces\libpq');
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq URI and regression testing
Peter Eisentraut writes: > On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote: >> What's the preferred way to make it automatically tested as much as >> possible? I know the buildfarm does not run "installcheck-world", so if >> we want it there, it'd need a bit more code on the client side. I think >> it would be wise to have it also run on installcheck-world. > > It was agreed during the patch discussion that it shouldn't be run > automatically. Hm... the testing program we've actually committed avoids any connection attempts and only deals with testing the parser routine. Does it change that? -- Regards, Alex -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] buffer locking fix for lazy_scan_heap
I discovered when researching the issue of index-only scans and Hot Standby that there's a bug (for which I'm responsible) in lazy_scan_heap[1]. Actually, the code has been like this for a long time, but I needed to change it when I did the crash-safe visibility map work, and I didn't. The problem is that lazy_scan_heap() releases the lock on the buffer it's whacking around before it sets the visibility map bit. Thus, it's possible for the page-level bit to be cleared by some other backend before the visibility map bit gets set. In previous releases that was arguably OK, since the worst thing that could happen is a postponement of vacuuming on that page until the next anti-wraparound cycle; but now that we have index-only scans, it can cause queries to return wrong answers. Attached is a patch to fix the problem, which rearranges things so that we set the bit in the visibility map before releasing the buffer lock. Similar work has already been done for inserts, updates, and deletes, which in previous releases would *clear* the visibility map bit after releasing the page lock, and no longer done. But the vacuum case, which *sets* the bit, was overlooked. Our convention in those related cases is that it's acceptable to hold the buffer lock while setting the visibility map bit and issuing the WAL log record, but there must be no possibility of an I/O to read in the visibility map page while the buffer lock is held. This turned out to be pretty simple because, as it turns out, lazy_scan_heap() is almost always holding a pin on the correct page anyway, so I only had to tweak things slightly to guarantee it. As a pleasant side effect, the new code is actually quite a bit simpler and more readable than the old code, at least IMHO. While I was at it, I made a couple of minor, related changes which I believe to be improvements. First, I arranged things so that, when we pause the first vacuum pass to do an index vac cycle, we release any buffer pin we're holding on the visibility map page. The fact that we haven't done that in the past is probably harmless, but I think there's something to be said for not holding onto buffer pins for long periods of time when it isn't really necessary. Second, I adjusted things so that we print a warning if the visibility map bit is set and the page-level bit is clear, since that should never happen in 9.2+. It is similar to the existing warning which catches the case where a page is marked all-visible despite containing dead tuples. Comments? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company http://archives.postgresql.org/pgsql-hackers/2012-04/msg00682.php lazy-scan-heap-locking-fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 1:48 PM, Magnus Hagander wrote: > The problem I've found with most tools is that they work reasonably > well if you let them control the entire workflow. But when you want to > do things your own way, and it doesn't match up with what they were > originally designed to do, it all comes falling down quickly... That's pretty much characteristic of the average SAP R/3 project. If you can change the organization's business processes to follow SAP's defined "best practices," then it's easy to install and use R/3. But that normally not being the case, every "SAP project" winds up having hideous customization costs to kludge the practices towards one another. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?" -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On 04/18/2012 01:26 PM, Robert Haas wrote: On Wed, Apr 18, 2012 at 1:08 PM, Josh Berkus wrote: BTW, given our heavy reliance on email, let me put a word in here for RT, which is 100% email-driven. RT has other limitations, but if your goal is to never log into a web interface, it's hard to beat. If your goal is to never log into a good web interface, it's even harder to beat. I actually don't mind logging into a web interface to update information. That doesn't slow me down much, as long as I can still also email *without* using the web interface. Now, RT's web interface is poor. But, if that's what everyone else likes, I could tolerate it. I think it's just frightful, TBH. And I do use the web for tracker interaction. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PgNext: Cancelled (sorry for the offtopic)
On Wed, Apr 18, 2012 at 8:12 PM, Joshua D. Drake wrote: > > On 04/18/2012 11:56 AM, Josh Berkus wrote: >> >> >> On 4/18/12 11:53 AM, Joshua D. Drake wrote: >>> >>> Lastly, there is a Denver PgDay in October. PgDay's are a great way to >>> meet locals and enjoy a smaller community setting but still maintain top >>> notch content. >> >> >> Date? Hopefully not the same time as pg.EU? >> >> I'd like to go this year. >> > > October 26th. Exactly the same time then. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PgNext: Cancelled (sorry for the offtopic)
On 04/18/2012 11:56 AM, Josh Berkus wrote: On 4/18/12 11:53 AM, Joshua D. Drake wrote: Lastly, there is a Denver PgDay in October. PgDay's are a great way to meet locals and enjoy a smaller community setting but still maintain top notch content. Date? Hopefully not the same time as pg.EU? I'd like to go this year. October 26th. -- Command Prompt, Inc. - http://www.commandprompt.com/ PostgreSQL Support, Training, Professional Services and Development The PostgreSQL Conference - http://www.postgresqlconference.org/ @cmdpromptinc - @postgresconf - 509-416-6579 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PgNext: Cancelled (sorry for the offtopic)
On 4/18/12 11:53 AM, Joshua D. Drake wrote: > Lastly, there is a Denver PgDay in October. PgDay's are a great way to > meet locals and enjoy a smaller community setting but still maintain top > notch content. Date? Hopefully not the same time as pg.EU? I'd like to go this year. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] PgNext: Cancelled (sorry for the offtopic)
04/18/2012 PostgreSQL Conference Next 2012, in Denver from Jun 26th - 29th is cancelled. If you are looking for another conference in this time frame we suggest: http://www.pgcon.org/ There are also other conferences coming up in the next 6 months: http://www.postgresopen.org/ http://www.pgconf.eu Lastly, there is a Denver PgDay in October. PgDay's are a great way to meet locals and enjoy a smaller community setting but still maintain top notch content. Sincerely, Joshua D. Drake -- Command Prompt, Inc. - http://www.commandprompt.com/ PostgreSQL Support, Training, Professional Services and Development The PostgreSQL Conference - http://www.postgresqlconference.org/ @cmdpromptinc - @postgresconf - 509-416-6579 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Desperately need mentors for GSOC FDW projects
On Wed, Apr 18, 2012 at 12:15 PM, Josh Berkus wrote: > All, > > We have 2 fairly promising projects for FDW work for this Google Summer > of code. One is for a Firebird FDW, and the more promising one is for a > "document collection" FDW from a previous successful GSOC student. > > The problem is, we don't have potential mentors for these. If you are > able to help with mentoring a Foreign Data Wrapper project, please get > back to me as soon as possible, so that we can accept one of these projects. I'm informally mentoring a GSoC applicant who is working on java-fdw project -- what's the procedure to become a mentor? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] urgent help required
Nagaraj, > i am nagaraj, i am newbi in this database world. i required your help. > 2 dyas back i formatted one of my client system. which is having postgresql > 8.2 database & that was having data. but i am not taken backup of the data. > 1) how to take the data from the formatted harddisk. ? > 2) how many folders or files will be their at base folder. When we install > postgresql 8.2? > 3) how to identify which folder contain which file. ? 1). the pgsql-hackers mailing list is not the place for this kind of question. Please use pgsql-admin or pgsql-general 2) Please use a subject line which indicates the nature of your issues. 3) PostgreSQL 8.2 is EOL and you should upgrade. 4) If you formatted the hard drive, your data is gone. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] urgent help required
Hi team, i am nagaraj, i am newbi in this database world. i required your help. 2 dyas back i formatted one of my client system. which is having postgresql 8.2 database & that was having data. but i am not taken backup of the data. 1) how to take the data from the formatted harddisk. ? 2) how many folders or files will be their at base folder. When we install postgresql 8.2? 3) how to identify which folder contain which file. ? Please help it is very urgent. thanks and regards -- Nagaraj V Shindagi
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 19:17, Stefan Kaltenbrunner wrote: > On 04/17/2012 11:29 PM, Andrew Dunstan wrote: >> >> >> On 04/17/2012 04:38 PM, Tom Lane wrote: >>> Jay Levitt writes: Greg Smith wrote: > Tracking when and how a bug is backported to older versions is one > hard part > of the problem here. That's a great point. Both GitHub and git itself have no real concept of releases, and can't tell you when a commit made it in. >>> We do actually have a somewhat-workable solution for that, see >>> src/tools/git_changelog. It relies on cooperation of the committers >>> to commit related patches with the same commit message and more or >>> less the same commit time, but that fits fairly well with our practices >>> anyway. If we did have an issue tracker I could see expecting commit >>> messages to include a reference to the issue number, and then it would >>> not be hard to adapt this program to key on that instead of matching >>> commit message texts. >>> >>> >> >> >> Yeah, that would be good. >> >> BTW, since we're discussing trackers yet again, let me put in a plug for >> Bugzilla, which has mature Postgres support, is written in Perl (which a >> large number of hackers are familiar with and which we use extensively), >> has a long history and a large organization behind it (Mozilla) and last >> but not least has out of the box support for creating updating and >> closing bugs via email (I just set up an instance of the latest release >> with this enabled to assure myself that it works, and it does.) It also >> has XML-RPC and JSON-RPC interfaces, as well as standard browser >> support, although I have not tested the RPC interfaces. > > years ago when I did the PoC install for PostgreSQL i used the > RPC-Interface for replacing the bug-reporting form on the main website, > it was prett ylimited back then (especially around authentication and a > way to actually make the report show up with the reporters name (which > very likely does not have a BZ account), but all those were solvable. BZ > really has the drawback that it is kind of a monster on the featureside > and you need to invest some significant time to make the UI > understandable before you can actually present it to a wider audience. What I saw from it, it would take more time and work to make the UI understandable and workable, and to get it to adapt to *our* process and workflow than to design a whole new tool from scratch... The problem I've found with most tools is that they work reasonably well if you let them control the entire workflow. But when you want to do things your own way, and it doesn't match up with what they were originally designed to do, it all comes falling down quickly... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 16:08, Peter Eisentraut wrote: > On tis, 2012-04-17 at 10:52 -0400, Jay Levitt wrote: >> Maybe I'm confused - Magnus et al, are we talking spammy issues/issue >> comments/etc, or are we talking more about exposed email addresses? > > My github.com account currently has 4264 notifications in the inbox. > Almost all of those are spam, growing constantly. Because of that, the > platform is currently fairly useless to me for actually communicating or > collaborating on code. That's about the same amount that I have. And yeah, it works great for hosting git repos. And it works great for browsing git repos and viewing differences and networks. But anything to do with pull requests, issues, comments or anything like that is completely useless these days. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 08:12, Tom Lane wrote: > Magnus Hagander writes: >> I think this cleraly outlines that we need to remember that there are >> *two* different patterns that people are trying tosolve with the >> bugtracker. > > Yeah, remember we drifted to this topic from discussion of management of > CF patches, which might be yet a third use-case. It's not obvious that > it's the same as tracking unfixed bugs, at least; though maybe the > requirements end up the same. > >> Any tool we'd go for should aim to cover *both* usecases. > > Not convinced that we should expect one tool to be good at both > (or all three) things. True. That means that it's more important than ever that whatever tools are used "plays well with others". Which in my experience very few of the existing trackers out there actually do. That can of course have changed since I last looked, but I have been very discouraged previously when looking, both for in the postgresql context and in others. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parameterized-path cost comparisons need some work
2012/4/18 Greg Stark > On Tue, Apr 17, 2012 at 5:14 PM, Tom Lane wrote: > > I've been hacking away on a patch to do this, and attached is something > > that I think is pretty close to committable. > > [..]Even when some of us don't > comment on some of the longer, more technical emails, we're definitely > still reading them and I at least find them invaluable for trying to > keep up with the changes over time. [..] > +1 hakan
Re: [HACKERS] Parameterized-path cost comparisons need some work
On Tue, Apr 17, 2012 at 5:14 PM, Tom Lane wrote: > I've been hacking away on a patch to do this, and attached is something > that I think is pretty close to committable. I have nothing to add but I just wanted to say thank you for taking the time to write up this explanation. Even when some of us don't comment on some of the longer, more technical emails, we're definitely still reading them and I at least find them invaluable for trying to keep up with the changes over time. It's also been key to establishing this practice as a good value for other patch authors to try to emulate. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 1:08 PM, Josh Berkus wrote: > BTW, given our heavy reliance on email, let me put a word in here for > RT, which is 100% email-driven. RT has other limitations, but if your > goal is to never log into a web interface, it's hard to beat. If your goal is to never log into a good web interface, it's even harder to beat. I actually don't mind logging into a web interface to update information. That doesn't slow me down much, as long as I can still also email *without* using the web interface. Now, RT's web interface is poor. But, if that's what everyone else likes, I could tolerate it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On 04/17/2012 11:29 PM, Andrew Dunstan wrote: > > > On 04/17/2012 04:38 PM, Tom Lane wrote: >> Jay Levitt writes: >>> Greg Smith wrote: Tracking when and how a bug is backported to older versions is one hard part of the problem here. >>> That's a great point. Both GitHub and git itself have no real concept of >>> releases, and can't tell you when a commit made it in. >> We do actually have a somewhat-workable solution for that, see >> src/tools/git_changelog. It relies on cooperation of the committers >> to commit related patches with the same commit message and more or >> less the same commit time, but that fits fairly well with our practices >> anyway. If we did have an issue tracker I could see expecting commit >> messages to include a reference to the issue number, and then it would >> not be hard to adapt this program to key on that instead of matching >> commit message texts. >> >> > > > Yeah, that would be good. > > BTW, since we're discussing trackers yet again, let me put in a plug for > Bugzilla, which has mature Postgres support, is written in Perl (which a > large number of hackers are familiar with and which we use extensively), > has a long history and a large organization behind it (Mozilla) and last > but not least has out of the box support for creating updating and > closing bugs via email (I just set up an instance of the latest release > with this enabled to assure myself that it works, and it does.) It also > has XML-RPC and JSON-RPC interfaces, as well as standard browser > support, although I have not tested the RPC interfaces. years ago when I did the PoC install for PostgreSQL i used the RPC-Interface for replacing the bug-reporting form on the main website, it was prett ylimited back then (especially around authentication and a way to actually make the report show up with the reporters name (which very likely does not have a BZ account), but all those were solvable. BZ really has the drawback that it is kind of a monster on the featureside and you need to invest some significant time to make the UI understandable before you can actually present it to a wider audience. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 1:08 PM, Josh Berkus wrote: >> 3. Otherwise, they drift forever in the bleakness of space. Seems to me that this line, is pretty close to being T-shirt-worthy. > wontfix. We don't need a system to help us ignore bug reports; our >> existing process handles that with admirable efficiency. > > I think I have this year's pgCon T-Shirt quote. ;-) Not bad, sir, not bad... -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?" -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Desperately need mentors for GSOC FDW projects
All, We have 2 fairly promising projects for FDW work for this Google Summer of code. One is for a Firebird FDW, and the more promising one is for a "document collection" FDW from a previous successful GSOC student. The problem is, we don't have potential mentors for these. If you are able to help with mentoring a Foreign Data Wrapper project, please get back to me as soon as possible, so that we can accept one of these projects. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
Robert, Peter, all: >>> IME, bug trackers typically work best when used by a tightly >>> integrated team. >> >> Well, very many loosely distributed open-source projects use bug >> trackers quite successfully. ... most of them, actually. >> Um, isn't half of the commitfest app about workflow? Patch is waiting >> for review, who is the reviewer, patch is waiting for author, who is the >> author, patch is ready for committer, who is the committer? And every >> week or so the commitfest manager (if any) produces a report on patch >> progress. Isn't that exactly what these "workflow management" systems >> provide? > > Yeah, but I thought we'd veered off into a digression about tracking > bug reports. Here's our workflow for bugs: I think assuming we can use the *same* tool for the CommitFests as for tracking bug reports would be a mistake. The workflow for both things is very different. > 1. If they seem interesting, Tom fixes them. > 2. Or occasionally someone else fixes them. > 3. Otherwise, they drift forever in the bleakness of space. Well, that *is* a workflow, even if it's a very simple one. And it could (should) be handled by a very simple bug tracker with a very simple workflow; as much as it irritates me otherwise, Bugzilla does this with its 4-5 bug "statuses", although the whole "assignment" approach wouldn't really work for us. Frankly, the concept of "assignment" doesn't work very well for OSS projects, period, in my experience. Often as not, it simply means that one person is ignoring the bug instead of a whole group of people. So a simple bug tracker which has four statuses ( "Submitted","Verified","Fixed","Invalid") would describe our entire bug workflow. Pretty much every bugtracker available can do this minimal level of bug tracking. BTW, given our heavy reliance on email, let me put a word in here for RT, which is 100% email-driven. RT has other limitations, but if your goal is to never log into a web interface, it's hard to beat. > wontfix. We don't need a system to help us ignore bug reports; our > existing process handles that with admirable efficiency. I think I have this year's pgCon T-Shirt quote. ;-) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments
On Wed, Apr 18, 2012 at 10:22 AM, Kyotaro HORIGUCHI wrote: > I tried that at first. But I suppose the requirement here is 'if > reading segments comes via replication stream, enable throttling > by checkpoint_segments.' and WalRcvInProgress() seems fit to > check that. If so, what if replication is terminated and restarted repeatedly while a restartpoint is running? It sometimes obeys and sometimes ignores checkpoint_segments. Which seems strange behavior. > Plus, adding SharedStartupStandbyMode into > XLogCtlData seems accompanied with some annoyances which would > not pay. Hmm... what are you worried about? I don't think that sharing the variable via XLogCtl is so difficult. Please see the code to share archiveCleanupCommand from the startup process to the checkpointer. It looks very simple. > By the way, do you have some advise about GetStandbyFlushRecPtr() > and the order of the locations? I'm embarrassed with that... Sorry. I could not parse this Could you explain this again? Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Devel 9.2] Index-only scan in count aggregation
On Wed, Apr 18, 2012 at 12:52 PM, Robert Haas wrote: > On Wed, Apr 18, 2012 at 12:40 PM, Emanuel Calvo > wrote: >>> I'm not sure what you're unhappy about. It seems that the query >>> planner picked the fastest plan (a sequential scan) and then when you >>> disabled that it picked the second-fastest plan (an index-only scan). >>> >>> The index-only scan would have a chance of beating the sequential scan >>> if the table had been recently vacuumed, but not in the case where >>> every row is going to require a heap fetch. >> >> Oh, I see now. Honestly, I thought it wasn't necessary to make a heap >> fetch. The table >> doesn't have any modifications, but with the vacuum the cost changed. > > Ah, I see. Yeah, I think you're not going to be the first person to > not realize that, especially since we haven't changed the rules for > autovacuuming, and therefore you can't count on autovac to correct the > problem for you. :-( And by "the first person" I of course meant "the last person". -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Devel 9.2] Index-only scan in count aggregation
On Wed, Apr 18, 2012 at 12:40 PM, Emanuel Calvo wrote: >> I'm not sure what you're unhappy about. It seems that the query >> planner picked the fastest plan (a sequential scan) and then when you >> disabled that it picked the second-fastest plan (an index-only scan). >> >> The index-only scan would have a chance of beating the sequential scan >> if the table had been recently vacuumed, but not in the case where >> every row is going to require a heap fetch. > > Oh, I see now. Honestly, I thought it wasn't necessary to make a heap > fetch. The table > doesn't have any modifications, but with the vacuum the cost changed. Ah, I see. Yeah, I think you're not going to be the first person to not realize that, especially since we haven't changed the rules for autovacuuming, and therefore you can't count on autovac to correct the problem for you. :-( -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Devel 9.2] Index-only scan in count aggregation
El día 18 de abril de 2012 18:17, Robert Haas escribió: > On Wed, Apr 18, 2012 at 12:13 PM, Emanuel Calvo > wrote: >> Hi guys, >> >> I'm one of the nightly sources of 9.2devel. I was trying some simple >> queries and I realized something: >> >> stuff=# explain (analyze true, costs true, buffers true, timing true, >> verbose true) select count(i) from lot_of_values; >> QUERY >> PLAN >> --- >> Aggregate (cost=213496.00..213496.01 rows=1 width=4) (actual >> time=60400.788..60400.791 rows=1 loops=1) >> Output: count(i) >> Buffers: shared hit=2400 read=86096 >> -> Seq Scan on public.lot_of_values (cost=0.00..188496.00 >> rows=1000 width=4) (actual time=0.371..32227.791 rows=1000 >> loops=1) >> Output: i, t1, r1, r2, r3, d1 >> Buffers: shared hit=2400 read=86096 >> Total runtime: 60402.460 ms >> (7 rows) >> >> stuff=# set enable_seqscan=off; >> SET >> stuff=# explain (analyze true, costs true, buffers true, timing true, >> verbose true) select count(i) from lot_of_values; >> >> QUERY PLAN >> >> Aggregate (cost=351292.03..351292.04 rows=1 width=4) (actual >> time=63278.472..63278.475 rows=1 loops=1) >> Output: count(i) >> Buffers: shared hit=1 read=110379 >> -> Index Only Scan using lot_of_values_pkey on >> public.lot_of_values (cost=0.00..326292.03 rows=1000 width=4) >> (actual time=42.028..35217.460 rows=1000 loops=1) >> Output: i >> Heap Fetches: 1000 >> Buffers: shared hit=1 read=110379 >> Total runtime: 63278.720 ms >> (8 rows) >> >> >> I know, still development. Just wanna know if there will be an >> improvement for this in the next patches or the idea is to maintain >> this behaviour. >> >> Cheers and thanks for the amazing work you all had done! > > I'm not sure what you're unhappy about. It seems that the query > planner picked the fastest plan (a sequential scan) and then when you > disabled that it picked the second-fastest plan (an index-only scan). > > The index-only scan would have a chance of beating the sequential scan > if the table had been recently vacuumed, but not in the case where > every row is going to require a heap fetch. > Oh, I see now. Honestly, I thought it wasn't necessary to make a heap fetch. The table doesn't have any modifications, but with the vacuum the cost changed. Checking the source code, I saw what you are talking about: /* * We can skip the heap fetch if the TID references a heap page on * which all tuples are known visible to everybody. In any case, * we'll use the index tuple not the heap tuple as the data source. */ Thanks for the information! -- -- Emanuel Calvo -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Devel 9.2] Index-only scan in count aggregation
On Wed, Apr 18, 2012 at 12:13 PM, Emanuel Calvo wrote: > Hi guys, > > I'm one of the nightly sources of 9.2devel. I was trying some simple > queries and I realized something: > > stuff=# explain (analyze true, costs true, buffers true, timing true, > verbose true) select count(i) from lot_of_values; > QUERY > PLAN > --- > Aggregate (cost=213496.00..213496.01 rows=1 width=4) (actual > time=60400.788..60400.791 rows=1 loops=1) > Output: count(i) > Buffers: shared hit=2400 read=86096 > -> Seq Scan on public.lot_of_values (cost=0.00..188496.00 > rows=1000 width=4) (actual time=0.371..32227.791 rows=1000 > loops=1) > Output: i, t1, r1, r2, r3, d1 > Buffers: shared hit=2400 read=86096 > Total runtime: 60402.460 ms > (7 rows) > > stuff=# set enable_seqscan=off; > SET > stuff=# explain (analyze true, costs true, buffers true, timing true, > verbose true) select count(i) from lot_of_values; > > QUERY PLAN > > Aggregate (cost=351292.03..351292.04 rows=1 width=4) (actual > time=63278.472..63278.475 rows=1 loops=1) > Output: count(i) > Buffers: shared hit=1 read=110379 > -> Index Only Scan using lot_of_values_pkey on > public.lot_of_values (cost=0.00..326292.03 rows=1000 width=4) > (actual time=42.028..35217.460 rows=1000 loops=1) > Output: i > Heap Fetches: 1000 > Buffers: shared hit=1 read=110379 > Total runtime: 63278.720 ms > (8 rows) > > > I know, still development. Just wanna know if there will be an > improvement for this in the next patches or the idea is to maintain > this behaviour. > > Cheers and thanks for the amazing work you all had done! I'm not sure what you're unhappy about. It seems that the query planner picked the fastest plan (a sequential scan) and then when you disabled that it picked the second-fastest plan (an index-only scan). The index-only scan would have a chance of beating the sequential scan if the table had been recently vacuumed, but not in the case where every row is going to require a heap fetch. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [Devel 9.2] Index-only scan in count aggregation
Hi guys, I'm one of the nightly sources of 9.2devel. I was trying some simple queries and I realized something: stuff=# explain (analyze true, costs true, buffers true, timing true, verbose true) select count(i) from lot_of_values; QUERY PLAN --- Aggregate (cost=213496.00..213496.01 rows=1 width=4) (actual time=60400.788..60400.791 rows=1 loops=1) Output: count(i) Buffers: shared hit=2400 read=86096 -> Seq Scan on public.lot_of_values (cost=0.00..188496.00 rows=1000 width=4) (actual time=0.371..32227.791 rows=1000 loops=1) Output: i, t1, r1, r2, r3, d1 Buffers: shared hit=2400 read=86096 Total runtime: 60402.460 ms (7 rows) stuff=# set enable_seqscan=off; SET stuff=# explain (analyze true, costs true, buffers true, timing true, verbose true) select count(i) from lot_of_values; QUERY PLAN Aggregate (cost=351292.03..351292.04 rows=1 width=4) (actual time=63278.472..63278.475 rows=1 loops=1) Output: count(i) Buffers: shared hit=1 read=110379 -> Index Only Scan using lot_of_values_pkey on public.lot_of_values (cost=0.00..326292.03 rows=1000 width=4) (actual time=42.028..35217.460 rows=1000 loops=1) Output: i Heap Fetches: 1000 Buffers: shared hit=1 read=110379 Total runtime: 63278.720 ms (8 rows) I know, still development. Just wanna know if there will be an improvement for this in the next patches or the idea is to maintain this behaviour. Cheers and thanks for the amazing work you all had done! -- -- Emanuel Calvo -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug #6593, extensions, and proposed new patch policy
On Wed, Apr 18, 2012 at 11:41 AM, Alvaro Herrera wrote: > Per bug #6593, REASSIGN OWNED fails when the affected role owns an > extension. This would be trivial to fix if extensions had support code > for ALTER EXTENSION / OWNER, but they don't. So the only back-patchable > fix right now seems to be to throw an error on REASSIGN OWNED when the > user owns an extension. (If anyone wants to claim that we ought to work > on a real fix that allows changing the owner internally from REASSIGN > OWNED, without introducing ALTER EXTENSION support for doing so, let me > know and I'll see about it.) I would be OK with the latter. > In HEAD we can do the more invasive fix of actually adding support code > for changing an extension's owner. And it seems to me that, going > further, we should have a policy that any ownable object type we add > must come with appropriate support for changing owner. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 path to repair_frag()?
Excerpts from Tom Lanyon's message of mié abr 18 12:44:11 -0300 2012: > Hi all, > > To satisfy my own curiosity, I was trying to find where 8.4 kicks into a > backwards scan of the data file during (auto-)vacuuming. > > This appears to be repair_frag() in backend/commands/vacuum.c, but only > appears to be called by a FULL vacuum. Autovacuum, however, appears to > explicitly only ever call a not-FULL vacuum, so I'm trying to see where and > how it would be possible for autovacuum to trigger this operation. Any > pointers? See lazy_truncate_heap in vacuumlazy.c. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Postgres install program files folder
Hello, Windows environment, has a problem in installing postgres in the Program Files folder? I have this doubt because Windows works with locking system to this folder.
Re: [HACKERS] Bug tracker tool we need
On 04/18/2012 11:29 AM, Kevin Grittner wrote: Peter Eisentraut wrote: On ons, 2012-04-18 at 13:33 +0300, Alex Shulgin wrote: I wonder why do people keep complaining how their bug tracker of choice sucks, instead of doing something about that. Lack of time, and to some degree a lack of clarity of what they want out of the thing. (Most people are very clear on what they don't want.) Personally, I haven't worked with one which had the data organized in what I would consider a sensible and useful way. Part of the trouble is that the whole area is fuzzy. So any organization that imposes some sort of order on the data will not fit well in some cases. Most tracker systems have ended up either trying to cater for increasingly complex and varied circumstances, or staying simple and more or less throwing in the towel on the complexity problem. That doesn't necessarily mean that it's not worth having a tracker, just that we need to recognize the limitations. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] 8.4 path to repair_frag()?
Hi all, To satisfy my own curiosity, I was trying to find where 8.4 kicks into a backwards scan of the data file during (auto-)vacuuming. This appears to be repair_frag() in backend/commands/vacuum.c, but only appears to be called by a FULL vacuum. Autovacuum, however, appears to explicitly only ever call a not-FULL vacuum, so I'm trying to see where and how it would be possible for autovacuum to trigger this operation. Any pointers? Thanks, Tom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Bug #6593, extensions, and proposed new patch policy
Hackers, Per bug #6593, REASSIGN OWNED fails when the affected role owns an extension. This would be trivial to fix if extensions had support code for ALTER EXTENSION / OWNER, but they don't. So the only back-patchable fix right now seems to be to throw an error on REASSIGN OWNED when the user owns an extension. (If anyone wants to claim that we ought to work on a real fix that allows changing the owner internally from REASSIGN OWNED, without introducing ALTER EXTENSION support for doing so, let me know and I'll see about it.) In HEAD we can do the more invasive fix of actually adding support code for changing an extension's owner. And it seems to me that, going further, we should have a policy that any ownable object type we add must come with appropriate support for changing owner. Thoughts? -- Álvaro Herrera -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
Peter Eisentraut wrote: > On ons, 2012-04-18 at 13:33 +0300, Alex Shulgin wrote: >> I wonder why do people keep complaining how their bug tracker of >> choice sucks, instead of doing something about that. > > Lack of time, and to some degree a lack of clarity of what they > want out of the thing. (Most people are very clear on what they > don't want.) Personally, I haven't worked with one which had the data organized in what I would consider a sensible and useful way. In my view there should be a *problem* report table, to describe the problem from a user perspective. What the user experienced, without any attempt to explain why. Error messages, steps to reproduce, environments in which it's been seen, independent confirmation of behavior, etc. This would be what you would primarily want to search when you hit a bug. There would be a separate table for an analysis of each contributing *cause* of the observed behavior. Describe why it caused or contributed to the observed behavior. There should be a list of these which exist independently of the problem statement, so that you can reference several causes for a problem report, and the cause can be linked from multiple reports. Each cause should include a separate section for user-oriented explanation of what to do when confronted by this issue -- limitations, workarounds, alternative approaches, documentation to read, etc. Each cause should maybe have a "Not a bug" check-box. Through a table linking problems to causes, not only could one easily look at all the contributing causes for a problem report, but all the problem reports with a given cause. In a field separate from the analysis, there could be a summary of what the community consensus on a solution is. Each cause should have a (possibly empty) *task* list describing what would need to be done by the community for resolution. Tasks would exist independently of problem statements or cause analysis and could be shared among various causes. (Of course, this being a relational database, one could easily find the related problem and cause rows that a to-do addressed.) I'm less clear on how work-flow management and resolution data would tie in, but it seems like there's plenty to attach that to. The muddling of problem description, cause analysis, solution design, tasks needed for correction, and assignments to tasks has been an insurmountable problem in the systems I've used so far (although there are a great many I've never seen, so maybe someone has this in what I would consider a reasonable structure). Any system which starts with a "problem description" and "solution description" as big text blobs and then jumps to a list of "assignments" (as one product I have to use) is hopelessly broken from the start, IMO. Now, possibly the problem is that other people think the above would be horrible for them, and that there *is* no design that works for everyone; but the above seems to me to model reality much better than any bug-tracking system I've used so far. And, as an aside, I think that calling an approach an anti-pattern is too often a cheap way to avoid serious thought about an issue which merits it, while pretending to the moral high ground. Better to leave that aside and stick to remarks with actual meaning and content. In other words, declaring something to be anti-pattern is, IMO, an anti-pattern. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #6572: The example of SPI_execute is bogus
On Sun, Apr 15, 2012 at 12:29 PM, Tom Lane wrote: >> I think it would be a good idea for UPDATE and DELETE to expose >> a LIMIT option, but I can't really see the virtue in making that >> functionality available only through SPI. > > FWIW, I'm not excited about that. You can get well-defined behavior > today from a SELECT/LIMIT drawing from a writable CTE (namely, that > the UPDATE/DELETE runs to completion but you only see a subset of > its RETURNING result). LIMIT directly on the UPDATE/DELETE would be > ill-defined, unless perhaps you want to also invent a way of specifying > the order in which rows get selected for update; but I don't want to > go there. In the use cases I'm thinking of, it doesn't matter which row you decide to update or delete, only that you pick a single one. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggressive memory consumption in {ts,array}_typanalyze
On Mon, Apr 16, 2012 at 4:58 PM, Noah Misch wrote: > The size hint I chose is fairly arbitrary. Any suggestions for principled > alternatives? Based on your test results, it doesn't seem like it matters very much what you put in there, so I'm inclined to think that num_mcelem is fine. I thought about maybe allowing for a little slop, like num_mcelem * 10, but maybe the way you did it is better. It's possible that people will set ridiculously overblown stats targets on some columns, and that's certainly going to cost something no matter what we do, but there's no point in making that worse than it has to be without some clear reason for doing so. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nodes/*funcs.c inconsistencies
Excerpts from Robert Haas's message of mié abr 18 11:47:37 -0300 2012: > On Mon, Apr 16, 2012 at 6:25 AM, Noah Misch wrote: > > I'd suggest backpatching the ReassignOwnedStmt() bits; the wrong code could > > produce crashes. The rest are for master only. > > Done, in the manner you suggest. Pah. I was just done with it and was about to push it ... -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nodes/*funcs.c inconsistencies
On Mon, Apr 16, 2012 at 6:25 AM, Noah Misch wrote: > I'd suggest backpatching the ReassignOwnedStmt() bits; the wrong code could > produce crashes. The rest are for master only. Done, in the manner you suggest. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 10:15 AM, Peter Eisentraut wrote: > On ons, 2012-04-18 at 08:28 -0400, Robert Haas wrote: >> It's very tempting to assume that the problem we're trying to solve >> must already have been well-solved by someone else, and therefore we >> ought to use their thing instead of inventing our own. But that >> presumes that our problem is exactly the same as other people's >> problem, which may not really be true. > > It's also very tempting to assume the opposite. ;-) Fair enough. >> IME, bug trackers typically work best when used by a tightly >> integrated team. > > Well, very many loosely distributed open-source projects use bug > trackers quite successfully. > >> So I think Greg has exactly the right idea: we shouldn't try to >> incorporate one of these systems that aims to manage workflow; > > Um, isn't half of the commitfest app about workflow? Patch is waiting > for review, who is the reviewer, patch is waiting for author, who is the > author, patch is ready for committer, who is the committer? And every > week or so the commitfest manager (if any) produces a report on patch > progress. Isn't that exactly what these "workflow management" systems > provide? Yeah, but I thought we'd veered off into a digression about tracking bug reports. Here's our workflow for bugs: 1. If they seem interesting, Tom fixes them. 2. Or occasionally someone else fixes them. 3. Otherwise, they drift forever in the bleakness of space. I've been conducting the experiment for a year or two now of leaving unresolved bug reports unread in my mailbox. I've got over 100 such emails now... and some of them may not really be bugs, but nobody's put in the work to figure that out. It would be useful to have a system that would keep track of which ones those are so people could look over them and maybe get inspired to go do some bug-fixing, or at least bug-analysis, but what good is workflow going to do us? We could "assign" all those bugs to people who are "supposed" to look at them, and maybe some of those people would actually do it, but a more likely outcome is that everybody's list of assigned issues would slowly converge to infinity, or they'd just start closing them as wontfix. We don't need a system to help us ignore bug reports; our existing process handles that with admirable efficiency. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen
On 04/18/2012 10:03 AM, Peter Eisentraut wrote: On ons, 2012-04-18 at 09:53 -0400, Tom Lane wrote: Peter Eisentraut writes: My vote is to revert this altogether and leave it be. In the alternative, make it an error. You mean in HEAD too? I don't agree with that, for sure. What this patch is accomplishing is to make sure that the less-commonly-used programs have similar command-line-parsing behavior to psql and pg_dump, where we long ago realized that failure to check this carefully could result in very confusing behavior. (Especially on machines where getopt is willing to rearrange the command line.) OK, if you care strongly about that, make it an error. But don't just ignore things. It won't be ignored. It will be caught by the "too many arguments" logic. The case where repeated arguments should be disallowed is a similar but different case that probably demands a much larger patch. I don't think its existence militates against this fix, however. I agree with Andrew that this is a bug fix. I can see the argument for not applying it to back branches, but not for declaring that it's not a bug. We shouldn't be backpatching things that are merely confusing. It works as designed at the time, after all. Improvements belong in master. If it was really intended to work this way then that's a piece of very poor design, IMNSHO. It looks to me much more like it was just an oversight. I don't have terribly strong feelings about this, since we've not had lots of complaints over the years, so I'll revert it in the back branches. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On ons, 2012-04-18 at 13:33 +0300, Alex Shulgin wrote: > I wonder why do people keep complaining how their bug tracker of > choice sucks, instead of doing something about that. Lack of time, and to some degree a lack of clarity of what they want out of the thing. (Most people are very clear on what they don't want.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On ons, 2012-04-18 at 08:28 -0400, Robert Haas wrote: > It's very tempting to assume that the problem we're trying to solve > must already have been well-solved by someone else, and therefore we > ought to use their thing instead of inventing our own. But that > presumes that our problem is exactly the same as other people's > problem, which may not really be true. It's also very tempting to assume the opposite. ;-) > IME, bug trackers typically work best when used by a tightly > integrated team. Well, very many loosely distributed open-source projects use bug trackers quite successfully. > So I think Greg has exactly the right idea: we shouldn't try to > incorporate one of these systems that aims to manage workflow; Um, isn't half of the commitfest app about workflow? Patch is waiting for review, who is the reviewer, patch is waiting for author, who is the author, patch is ready for committer, who is the committer? And every week or so the commitfest manager (if any) produces a report on patch progress. Isn't that exactly what these "workflow management" systems provide? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.
On Tue, Apr 17, 2012 at 3:29 PM, Fujii Masao wrote: > Okay, patch attached. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On tis, 2012-04-17 at 10:52 -0400, Jay Levitt wrote: > Maybe I'm confused - Magnus et al, are we talking spammy issues/issue > comments/etc, or are we talking more about exposed email addresses? My github.com account currently has 4264 notifications in the inbox. Almost all of those are spam, growing constantly. Because of that, the platform is currently fairly useless to me for actually communicating or collaborating on code. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen
On ons, 2012-04-18 at 09:53 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > My vote is to revert this altogether and leave it be. In the > > alternative, make it an error. > > You mean in HEAD too? I don't agree with that, for sure. What this > patch is accomplishing is to make sure that the less-commonly-used > programs have similar command-line-parsing behavior to psql and pg_dump, > where we long ago realized that failure to check this carefully could > result in very confusing behavior. (Especially on machines where > getopt is willing to rearrange the command line.) OK, if you care strongly about that, make it an error. But don't just ignore things. > I agree with Andrew that this is a bug fix. I can see the argument > for not applying it to back branches, but not for declaring that it's > not a bug. We shouldn't be backpatching things that are merely confusing. It works as designed at the time, after all. Improvements belong in master. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql: tab completions for 'WITH'
On tis, 2012-04-10 at 17:48 -0700, Josh Kupershmidt wrote: > > Hmm, but now you've set it up so that you can complete ALTER ROLE > foo > > WITH WITH. Were you aware of that? > > D'oh, I overlooked that. Attached is v2: the diff is a tad lengthier > now, but that should fix it. Committed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen
Peter Eisentraut writes: > My vote is to revert this altogether and leave it be. In the > alternative, make it an error. You mean in HEAD too? I don't agree with that, for sure. What this patch is accomplishing is to make sure that the less-commonly-used programs have similar command-line-parsing behavior to psql and pg_dump, where we long ago realized that failure to check this carefully could result in very confusing behavior. (Especially on machines where getopt is willing to rearrange the command line.) I agree with Andrew that this is a bug fix. I can see the argument for not applying it to back branches, but not for declaring that it's not a bug. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
On Wed, Apr 18, 2012 at 1:38 AM, Magnus Hagander wrote: >> At the same time, I think we'd likely be a lot better off squirting this >> data into bugzilla or another standard tracker, instead of building our >> own infrastructure. > > I'm somewhat doubtful. Me, too. It's very tempting to assume that the problem we're trying to solve must already have been well-solved by someone else, and therefore we ought to use their thing instead of inventing our own. But that presumes that our problem is exactly the same as other people's problem, which may not really be true. IME, bug trackers typically work best when used by a tightly integrated team. For example, EnterpriseDB uses a bug-tracker-ish system for tracking bugs and feature requests and another one for support requests. Those systems get the job done and, certainly, are better designed than Bugzilla (it doesn't take much), but a lot of what they manage is workflow. Person A assigns a ticket to person B, and person B is then responsible for taking some action, and if they don't then someone will come along and run a report and grouse at person B for failing to take that action. If those reports weren't run and people didn't get groused at, the system would degenerate into utter chaos; and of course in a community setting it's hard to imagine that any such thing could possibly occur, since this is an all-volunteer effort. So I think Greg has exactly the right idea: we shouldn't try to incorporate one of these systems that aims to manage workflow; we should just design something really simple that tracks what happened and lets people who wish to volunteer to do so help keep that tracking information up to date. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug tracker tool we need
Robert Haas writes: > On Tue, Apr 17, 2012 at 1:47 AM, Magnus Hagander wrote: >> That's probably one reason people aren't jumping on this. Because >> there is no tracker out there that people actually *like*... > > I think this is a point worth serious thought. I wonder why do people keep complaining how their bug tracker of choice sucks, instead of doing something about that. I can see a few possible factors: a) people do like to complain, and it's easier than submitting meaningful bug reports or feature requests, patches :-) b) the developers don't listen to their users, which happens far too often unfortunately c) (I had yet another idea here, but I forgot what it was :-p) d) a wild mix of the above However, this doesn't imply existing tracker software cannot be improved and more of that must be written from scratch (unless the code is cryptic and/or is written, probably poorly, in some rarely used programming language, and is unmaintainable.) Also, the reasons outlined above do not pertain only to bug tracker software somehow: any piece of software could suffer from that and I believe many of us have seen it. So maybe there's something fundamentally wrong with every existing bug tracker (e.g. they don't fix bugs for you?) Well, just kidding. ;-) -- Alex -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Improving our clauseless-join heuristics
>> Um ... wasn't that well enough explained already?
Yes, it was well explained and I understood also, but what I wanted to
understand the solution with which you have resolved the problem.
The way I am telling was as below code.
With this extra paths will get generated, but it will as well consider for
joining c and d in query:
select * from a, b, c, d where a.x = b.y and (a.z = c.c or a.z = d.d)
static void
make_rels_by_clause_joins(PlannerInfo *root,
RelOptInfo *old_rel,
ListCell *other_rels)
{
ListCell *l;
++ boolbIsold_relJointoanyother_rel = false;
for_each_cell(l, other_rels)
{
RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
if (!bms_overlap(old_rel->relids, other_rel->relids) &&
(have_relevant_joinclause(root, old_rel, other_rel)
||
have_join_order_restriction(root, old_rel,
other_rel)))
{
++bIsold_relJointoanyother_rel = true;
(void) make_join_rel(root, old_rel, other_rel);
}
}
++ /*if old_rel is not able to join with any other rel than try
joining it
++ with other_rels which has join clause.*/
++if(bIsold_relJointoanyother_rel == false)
++ {
++ for_each_cell(l, other_rels)
++ {
++ RelOptInfo *other_rel = (RelOptInfo *)
lfirst(l);
++ if (!bms_overlap(old_rel->relids,
other_rel->relids) &&
++ (has_join_restriction(root,
other_rel)||other_rel->joininfo != NIL))
++ {
++ (void) make_join_rel(root, old_rel,
other_rel);
++ }
++ }
++ }
}
-Original Message-
From: Tom Lane [mailto:t...@sss.pgh.pa.us]
Sent: Wednesday, April 18, 2012 11:59 AM
To: Amit Kapila
Cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Improving our clauseless-join heuristics
Amit Kapila writes:
>> I'm afraid I'm still not following you very well. Perhaps you could
>> submit a proposed patch?
> Before that can you please explain in little more detail (if possible with
> small example) about the idea you have told in original mail : "is there
any
> join clause that both these relations participate in?"
Um ... wasn't that well enough explained already?
I think there are basically two cases. You can have a join clause that
is immediately useful for joining two relations, say
select ... from a,b where a.x = b.y;
This is "immediate" in the sense that you can apply it when joining a
to b, regardless of any other relations involved in the query.
Or you can have a case like
select ... from a,b,c where (a.x + b.y) = c.z;
This clause is not immediately useful for joining any two of the three
relations in the query. It will be useful when we get to level 3,
particularly so if we chose to join a and b first and there's an index
on c.z. But we would have had to accept doing a cartesian join of a and
b to arrive at that situation. In this example, we have no alternative
except to do some cartesian join at level 2 --- but as soon as we add
some more tables and join clauses to the example, we could get
distracted from the possibility that a cartesian join of a and b might
be a good idea.
Given that make_rels_by_joins doesn't (and shouldn't IMO) have any
detailed understanding of the semantics of particular join clauses,
I would not expect it to realize that joining a to b is the most likely
option out of the three possible clauseless joins that are available
at level 2 in this query. It's going to have to generate all 3, and
then costing at the next level will figure out what's best to do.
However, I think it *does* need to understand that clauses relating
3 or more relations can work like this. In the code as it stood before
last week, it would actively reject joining a to b if there were any
additional relations in the query. That's just not right.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen
On tis, 2012-04-17 at 19:19 -0400, Andrew Dunstan wrote: > It was discussed. I think the previous behaviour is a bug. It can't be > sane to be allowed to do: > > initdb -D foo bar It's debatable whether it should be allowed. I don't see anything wrong with it. After all, later arguments usually override earlier arguments, and non-option arguments notionally come after option arguments. Also, if this should be disallowed, would you also disallow initdb -D foo -D bar? But what I think is worse is that the "bar" is silently ignored now. If you think that this is an error, it should produce an error. My vote is to revert this altogether and leave it be. In the alternative, make it an error. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
