[HACKERS] [PATCH] SE-PgSQL/lite rev.2163

2009-07-10 Thread KaiGai Kohei
The SE-PostgreSQL patches are updated as follows:

[1/5] http://sepgsql.googlecode.com/files/sepgsql-01-sysatt-8.5devel-r2163.patch
[2/5] http://sepgsql.googlecode.com/files/sepgsql-02-core-8.5devel-r2163.patch
[3/5] http://sepgsql.googlecode.com/files/sepgsql-03-gram-8.5devel-r2163.patch
[4/5] http://sepgsql.googlecode.com/files/sepgsql-04-tests-8.5devel-r2163.patch
[5/5] http://sepgsql.googlecode.com/files/sepgsql-05-docs-8.5devel-r2163.patch

List of updates:
* Patch set was organized to a few ones which provides only core features.
* Code base was upgraded to the latest CVS HEAD.
* Some of features in the fullset edition were separated, to focus on
  the core feature of SE-PostgreSQL at the first commit fest.

The full functional SE-PostgreSQL consists of ten patches sequentially
numbered. The patch with smaller number provide more fundamental features.
Robert Haas suggested we should focus on a part of patches on the first
commit fest, because all the patch set of SE-PostgreSQL is a bit large
to review within a single commit fest. I agreed and reorganized my patches.
Some of advanced features (such as row-level controls) are separated
from the features to be focused on the 1st commit fest.
I decided to call the small functional SE-PostgreSQL as SE-PgSQL/lite
to make clear what we discuss on.

The SE-PgSQL/lite contains the following features.
* Management of the security labels (1st patch)
 SELinux's security model requires all the subjects and objects are labeled.
 It enables to assign a certain security label on several kinds of database
 objects. The security label in text form is stored within the new system
 catalog (pg_security), and the catalog/pg_security.c provides a facility
 to translate it and the security identifier.

* Core facility to communicate with in-kernel SELinux and to make its
  decision on various kind of database objcets. (2nd patch)
 The second patch provides the core functionality to perform with SELinux.
 It deploys security hooks on the strategic points of PostgreSQL.
 The hooks invoke SE-PostgreSQL routines, when it is enabled. The routines
 makes its decision based on the system's security policy.
 The userspace access vector cache (src/backend/security/sepgsql/avc.c)
 minimizes the number of kernel space invocations, and enables to make
 a decision (previously asked) without context switching.
 The routines of security hooks (hooks.c and checker.c) pulls the security
 label of given database objects like a table, and asks the userspace AVC
 whether the required accesses to be allowed, or not.
 If denied, it returns an error status or raises error using ereport().

* SQL Extentions (3rd patch)
 When we create a database object, a default security label shall be given
 based on the security policy. But we can give an explicit security label
 for a new object, as far as user is allowed to create it with the given
 security label.
 This patch provide SECURITY_LABEL = '...' option for several kinds of
 CREATE or ALTER statement. It allows users to create database, schemas,
 tables, columns and procedures with a specified security label.

* Documentation patch (current 4th patch)
 It patches src/doc/sgml/*. Any descriptions corresponding to the row-level
 access controls and other upcoming features were separated.

* Test cases patch (current 5th patch)
 It provides test cases for SE-PgSQL/lite.

The SE-PgSQL/lite does NOT contain the following features, currently.

The row-level access controls provided by the 5th patch was separated from
the SE-PgSQL/lite. In addition, the writable system column support needed
by row-level controls provided by the 4th patch was also separated.
Some persons complained deployment of security hooks seem like row level
controls, such as sepgsqlHeapTupleInsert() from simple_heap_insert().
It was also separated from the SE-PgSQL/lite, and it checks permissions
outside of the simple_heap_insert(). For example, SE-PgSQL/lite put its
hook (sepgsqlCheckTableCreate()) on the DefineRelation() next to the DAC
permission checks. We can also keep completeness of the access controls
as far as security hooks checks all the routes users to create/alter/drop
tables and so on. However, it needed to apply a hardwired policy to prevent
users to modify system catalog by hand, instead of the design changes.

The advanced permission checks (in the 6th patch) were also separated
from the SE-PgSQL/lite. It includes file permission checks on COPY TO/FROM
statements, largeobjects accesses, installation of binary modules.

The functionality to reclaim orphan security labels (in the 7th patch)
was also separated.

Thanks,

-
FYI, scale of the patches

- sepgsql-01-sysatt-8.5devel-r2163.patch
  34 files changed, 723 insertions(+), 69 modifications(!)
- sepgsql-02-core-8.5devel-r2163.patch
  54 files changed, 4074 insertions(+), 128 modifications(!)
  (*) 88% of changesets are newlines at backend/security/sepgsql/*
  or its header.
- 

Re: [HACKERS] [pgsql-www] Launching commitfest.postgresql.org

2009-07-10 Thread Dave Page
On Fri, Jul 10, 2009 at 2:23 AM, Robert Haasrobertmh...@gmail.com wrote:

 We have a wildcard cert for the domain. I'm going on vacation at the
 weekend, and have to wrap up a bunch of stuff at work tomorrow, but if
 you talk nicely to Stefan...

 Stefan, you are mighty and have powerful administration skills. Indeed, you
 have knowledge of many passwords that are hidden from the foolish. I humbly
 beseech you for a copy of the aforementioned cert on coridan...

 Is that enough?  I can go on... :-)

Hey, that's really very good. Feel free to continue though, it's quite
amusing :-)

-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] Launching commitfest.postgresql.org

2009-07-10 Thread Stefan Kaltenbrunner

Robert Haas wrote:

On Jul 9, 2009, at 5:53 PM, Dave Page dp...@pgadmin.org wrote:

On Thu, Jul 9, 2009 at 11:31 PM, Robert Haasrobertmh...@gmail.com 
wrote:

On Jul 9, 2009, at 5:07 PM, Peter Eisentraut pete...@gmx.net wrote:


On Thursday 09 July 2009 19:06:35 Brendan Jurd wrote:


Your login details for the app == your community login == your wiki
login.


One thing I forgot to mention: Please set up an SSL server certificate
around
the login page.


Who will provide and pay for it?


We have a wildcard cert for the domain. I'm going on vacation at the
weekend, and have to wrap up a bunch of stuff at work tomorrow, but if
you talk nicely to Stefan...


Stefan, you are mighty and have powerful administration skills. Indeed, 
you have knowledge of many passwords that are hidden from the foolish. I 
humbly beseech you for a copy of the aforementioned cert on coridan...


Is that enough?  I can go on... :-)


not enough for getting it done immediately - but I will take a look 
later today...


Stefan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-10 Thread decibel

On Jul 9, 2009, at 12:35 PM, Brendan Jurd wrote:

We don't AFAIK collect data about these events.  However, we could
have certain actions trigger the creation of an automated comment
(e.g., Status changed to Committed by petere) and let the
aforementioned comment view suffice for a history.



Our main system at work does that; any kind of status is stored as a  
raw, text note. It sucks. It makes trying to query for specific  
kinds of events difficult, and it wastes a bunch of space.


It's a lot better to record machine-readable information for machine- 
created events. If you want to present it all as one, I suggest a  
union view that turns the machine-understood data into a human- 
understandable text format.

--
Decibel!, aka Jim C. Nasby, Database Architect  deci...@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Robert Haas
On Wed, Jul 8, 2009 at 4:57 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 Well, the reason I'm not voting for #3 is that it looks like a lot of
 work to implement something that would basically be a planner hint,
 which I'm generally against; furthermore, it's a hint that there's been
 no demand for.  (We're not even certain that anyone is using the ability
 to *fully* specify the join order, much less wanting some undetermined
 compromise between manual and automatic control.)  And anyway I didn't
 hear anyone volunteering to do it.  So the realistic alternatives are
 #1, #2, or do nothing; and out of those I like #2.

I took a look at this and it seems that #3 can be implemented with
essentially no additional code (the handful of lines I added where
more than balanced out by some simplifications in ruleutils.c).  Of
course you still don't have to like it.  :-)

Patch attached.

...Robert
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***
*** 2251,2313  SELECT * FROM parent WHERE key = 2400;
/listitem
   /varlistentry
  
-  varlistentry id=guc-from-collapse-limit xreflabel=from_collapse_limit
-   termvarnamefrom_collapse_limit/varname (typeinteger/type)/term
-   indexterm
-primaryvarnamefrom_collapse_limit/ configuration parameter/primary
-   /indexterm
-   listitem
-para
- The planner will merge sub-queries into upper queries if the
- resulting literalFROM/literal list would have no more than
- this many items.  Smaller values reduce planning time but might
- yield inferior query plans.  The default is eight.
- For more information see xref linkend=explicit-joins.
-/para
- 
-para
- Setting this value to xref linkend=guc-geqo-threshold or more
- may trigger use of the GEQO planner, resulting in nondeterministic
- plans.  See xref linkend=runtime-config-query-geqo.
-/para
-   /listitem
-  /varlistentry
- 
-  varlistentry id=guc-join-collapse-limit xreflabel=join_collapse_limit
-   termvarnamejoin_collapse_limit/varname (typeinteger/type)/term
-   indexterm
-primaryvarnamejoin_collapse_limit/ configuration parameter/primary
-   /indexterm
-   listitem
-para
- The planner will rewrite explicit literalJOIN/
- constructs (except literalFULL JOIN/s) into lists of
- literalFROM/ items whenever a list of no more than this many items
- would result.  Smaller values reduce planning time but might
- yield inferior query plans.
-/para
- 
-para
- By default, this variable is set the same as
- varnamefrom_collapse_limit/varname, which is appropriate
- for most uses. Setting it to 1 prevents any reordering of
- explicit literalJOIN/s. Thus, the explicit join order
- specified in the query will be the actual order in which the
- relations are joined. The query planner does not always choose
- the optimal join order; advanced users can elect to
- temporarily set this variable to 1, and then specify the join
- order they desire explicitly.
- For more information see xref linkend=explicit-joins.
-/para
- 
-para
- Setting this value to xref linkend=guc-geqo-threshold or more
- may trigger use of the GEQO planner, resulting in nondeterministic
- plans.  See xref linkend=runtime-config-query-geqo.
-/para
-   /listitem
-  /varlistentry
- 
   /variablelist
  /sect2
 /sect1
--- 2251,2256 
*** a/doc/src/sgml/perform.sgml
--- b/doc/src/sgml/perform.sgml
***
*** 599,606  WHERE tablename = 'road';
  
para
 It is possible
!to control the query planner to some extent by using the explicit literalJOIN/
!syntax.  To see why this matters, we first need some background.
/para
  
para
--- 599,607 
  
para
 It is possible
!to control the query planner to some extent by using literalJOIN/
!with the literalFORCE/ keyword.  To see why this matters, we first need
!some background.
/para
  
para
***
*** 675,681  SELECT * FROM a LEFT JOIN b ON (a.bid = b.id) LEFT JOIN c ON (a.cid = c.id);
para
 Even though most kinds of literalJOIN/ don't completely constrain
 the join order, it is possible to instruct the
!productnamePostgreSQL/productname query planner to treat all
 literalJOIN/ clauses as constraining the join order anyway.
 For example, these three queries are logically equivalent:
  programlisting
--- 676,682 
para
 Even though most kinds of literalJOIN/ don't completely constrain
 the join order, it is possible to instruct the
!productnamePostgreSQL/productname query planner to treat certain
 literalJOIN/ clauses as constraining the join order anyway.
 For example, these three queries are logically 

Re: [HACKERS] WIP: to_char, support for EEEE format

2009-07-10 Thread Brendan Jurd
2009/4/26 Brendan Jurd dire...@gmail.com:
 I've done some work updating Pavel's sci notation patch for to_char().

That patch again, now with a couple of minor tweaks to make it apply
cleanly against the current HEAD.

Cheers,
BJ


_3.diff.bz2
Description: BZip2 compressed data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Patch for contains/overlap of polygons

2009-07-10 Thread Teodor Sigaev

http://www.sigaev.ru/misc/polygon-0.2.gz
 Patch fixes wrong algorithms of contains and overlap operations over polygons.
 Regression tests contain an examples of such polygons.

--
Teodor Sigaev   E-mail: teo...@sigaev.ru
   WWW: http://www.sigaev.ru/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] Launching commitfest.postgresql.org

2009-07-10 Thread Stefan Kaltenbrunner

Stefan Kaltenbrunner wrote:

Robert Haas wrote:

On Jul 9, 2009, at 5:53 PM, Dave Page dp...@pgadmin.org wrote:

On Thu, Jul 9, 2009 at 11:31 PM, Robert Haasrobertmh...@gmail.com 
wrote:

On Jul 9, 2009, at 5:07 PM, Peter Eisentraut pete...@gmx.net wrote:


On Thursday 09 July 2009 19:06:35 Brendan Jurd wrote:


Your login details for the app == your community login == your wiki
login.


One thing I forgot to mention: Please set up an SSL server certificate
around
the login page.


Who will provide and pay for it?


We have a wildcard cert for the domain. I'm going on vacation at the
weekend, and have to wrap up a bunch of stuff at work tomorrow, but if
you talk nicely to Stefan...


Stefan, you are mighty and have powerful administration skills. 
Indeed, you have knowledge of many passwords that are hidden from the 
foolish. I humbly beseech you for a copy of the aforementioned cert on 
coridan...


Is that enough?  I can go on... :-)


not enough for getting it done immediately - but I will take a look 
later today...


https support is now available on that jail (maybe we should simply 
always redirect to the https url on all pages).
I also rearranged some of the configuration files as well as fixing a 
few omissions from the initial install(like erm - where the hell are the 
backups?!) so any new breakage is likely mine :)




Stefan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Dimitri Fontaine

Hi,

Le 10 juil. 09 à 17:22, Robert Haas a écrit :

I took a look at this and it seems that #3 can be implemented with
essentially no additional code (the handful of lines I added where
more than balanced out by some simplifications in ruleutils.c).  Of
course you still don't have to like it.  :-)


I see you're using the following syntax:
! SELECT * FROM a INNER FORCE JOIN (b INNER FORCE JOIN c ON (b.ref =  
c.id)) ON (a.id = b.id);


The only place I've seen that before is MySQL straight_join feature:
  http://dev.mysql.com/doc/refman/5.0/en/join.html

My first though at the time was what a lame optimiser if I'm to tell  
it where not to reorder joins, but perhaps this was because the  
option there, IIRC, could impact the results...


I guess I'm not going to like it, but nonetheless, if we're going to  
support the feature, what about calling it the same as MySQL, unless  
the standard has something to say?

--
dim
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 I took a look at this and it seems that #3 can be implemented with
 essentially no additional code (the handful of lines I added where
 more than balanced out by some simplifications in ruleutils.c).  Of
 course you still don't have to like it.  :-)

You're right, I don't.  Even if I thought this were a good idea, which
I most definitely do not, the need to add a nonstandard fully-reserved
word is sufficient reason to reject it.  (The patch tries to pretend
it's not going to reserve the word, but that only works because you have
carefully changed only one of the five JOIN productions, leading to
bizarrely non-orthogonal syntax.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Robert Haas
On Jul 10, 2009, at 11:48 AM, Dimitri Fontaine dfonta...@hi- 
media.com wrote:



Hi,

Le 10 juil. 09 à 17:22, Robert Haas a écrit :

I took a look at this and it seems that #3 can be implemented with
essentially no additional code (the handful of lines I added where
more than balanced out by some simplifications in ruleutils.c).  Of
course you still don't have to like it.  :-)


I see you're using the following syntax:
! SELECT * FROM a INNER FORCE JOIN (b INNER FORCE JOIN c ON (b.ref =  
c.id)) ON (a.id = b.id);


The only place I've seen that before is MySQL straight_join feature:
 http://dev.mysql.com/doc/refman/5.0/en/join.html

My first though at the time was what a lame optimiser if I'm to  
tell it where not to reorder joins, but perhaps this was because  
the option there, IIRC, could impact the results...


I guess I'm not going to like it, but nonetheless, if we're going to  
support the feature, what about calling it the same as MySQL, unless  
the standard has something to say?


Well, I think we would be well-advised to get past the threshold issue  
of whether or not the overall design sucks before quibbling over  
details like my particular choice of keyword.  That having been said,  
if the MySQL feature to which you linked actually does the same thing  
as what I implemented here, then it's amazingly poorly documented. We  
certainly don't guarantee anything about the order in which the input  
tables are read; I'd ask what that even means except I don't care.  
We're only making a promise about where that join will be implemented  
in the plan tree as compared with *other joins*.


It was reported upthread that Oracle uses ORDERED for this but I don't  
know whether either the syntax or the semantics match what I did  
here.  At any rate the particular choice of keyword seems rather  
insignificant; I picked it because it was already a keyword and seemed  
vaguely appropriate, but it could easily be changed.


...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lock Wait Statistics (next commitfest)

2009-07-10 Thread Jaime Casanova
On Sun, Jan 25, 2009 at 6:57 PM, Mark Kirkwoodmar...@paradise.net.nz wrote:

 So here is my initial attempt at this, at this point merely to spark
 discussion (attached patch)


this patch doesn't apply cleanly to head... can you update it, please?

-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: 
 
 At any rate the particular choice of keyword seems rather  
 insignificant; I picked it because it was already a keyword and
 seemed vaguely appropriate, but it could easily be changed.
 
Actually, if we were going to add fine-grained optimizer hints for
this (which I'm not at all convinced is a good idea), I'd be tempted
to go with what I saw a few years ago in SAP-DB (later rebranded as
MySQL Max-DB): treat parentheses around JOIN operations as optimizer
hints.  I would only consider this as remotely sane if there was an
enable_* GUC to disable the normal reordering of joins.  It introduces
no new keywords.  The queries are still standard compliant.  We would
just have a configuration option which treats an optional part of the
standard syntax as a hint.
 
In other words:
 
select * from (t1 join t2 on whatever) join t3 on whatever;
 
would limit optimizer choice from those available with:
 
select * from t1 join t2 on whatever join t3 on whatever;
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 Actually, if we were going to add fine-grained optimizer hints for
 this (which I'm not at all convinced is a good idea), I'd be tempted
 to go with what I saw a few years ago in SAP-DB (later rebranded as
 MySQL Max-DB): treat parentheses around JOIN operations as optimizer
 hints.

That's a *truly* horrid idea, as sometimes you need them simply to
get the precedence correct.  Such awful design from SAP doesn't surprise
me, and MySQL copying a bad idea surprises me even less, but let's not
go there.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: 
 Kevin Grittner kevin.gritt...@wicourts.gov writes:
 treat parentheses around JOIN operations as optimizer hints.
 
 That's a *truly* horrid idea, as sometimes you need them simply to
 get the precedence correct.
 
You do, but it's been pretty rare in my experience, and we're
considering alternatives which give a lot less flexibility that this.
 
The *truly* awful thing about the SAP-DB implementation is that it
wasn't optional -- parentheses in this part of a query always limited
optimizer options; I sure wouldn't want to go there again.  I thought
we were talking about options for what to do when an enable_* setting
was off for diagnostic purposes
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] git.postgresql.org vs. REL8_1_STABLE

2009-07-10 Thread Joshua Tolley
Am I the only one having problems building 8.1 from git? (Am I the only one
building 8.1 from git?) In a clean repository, I've checked out REL8_1_STABLE,
configured with only one argument, to set --prefix, and make gives me this:

make[4]: Entering directory
`/home/josh/devel/pgsrc/pg81/src/backend/access/common'
gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline
-Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv
-I../../../../src/include -D_GNU_SOURCE   -c -o indexvalid.o indexvalid.c
In file included from ../../../../src/include/executor/execdesc.h:19,
 from ../../../../src/include/executor/executor.h:17,
 from ../../../../src/include/executor/execdebug.h:17,
 from indexvalid.c:19:
../../../../src/include/nodes/execnodes.h:23:29: error: nodes/tidbitmap.h: No
such file or directory
In file included from ../../../../src/include/executor/execdesc.h:19,
 from ../../../../src/include/executor/executor.h:17,
 from ../../../../src/include/executor/execdebug.h:17,
 from indexvalid.c:19:
../../../../src/include/nodes/execnodes.h:934: error: expected
specifier-qualifier-list before ‘TIDBitmap’
../../../../src/include/nodes/execnodes.h:959: error: expected
specifier-qualifier-list before ‘TIDBitmap’
make[4]: *** [indexvalid.o] Error 1
make[4]: Leaving directory
`/home/josh/devel/pgsrc/pg81/src/backend/access/common'
make[3]: *** [common-recursive] Error 2
make[3]: Leaving directory `/home/josh/devel/pgsrc/pg81/src/backend/access'
make[2]: *** [access-recursive] Error 2
make[2]: Leaving directory `/home/josh/devel/pgsrc/pg81/src/backend'
make[1]: *** [all] Error 2 
make[1]: Leaving directory `/home/josh/devel/pgsrc/pg81/src'
make: *** [all] Error 2 

Indeed, src/include/nodes has no tidbitmap.h file (it shows up in
REL8_2_STABLE).

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com


signature.asc
Description: Digital signature


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Robert Haas

On Jul 10, 2009, at 12:06 PM, Tom Lane t...@sss.pgh.pa.us wrote:


Robert Haas robertmh...@gmail.com writes:

I took a look at this and it seems that #3 can be implemented with
essentially no additional code (the handful of lines I added where
more than balanced out by some simplifications in ruleutils.c).  Of
course you still don't have to like it.  :-)


You're right, I don't.  Even if I thought this were a good idea, which
I most definitely do not, the need to add a nonstandard fully-reserved
word is sufficient reason to reject it.  (The patch tries to pretend
it's not going to reserve the word, but that only works because you  
have

carefully changed only one of the five JOIN productions, leading to
bizarrely non-orthogonal syntax.)


Well, it's pretty obvious that only one of those productions is  
actually a problem, and that is the one which produces an undecorated  
JOIN. The others could all be changed easily enough, but they add no  
expressive power, so I didn't think it very worthwhile to add MORE non- 
standard syntax.  In any event, the current non-orthogonality is  
exponentially more bizarre: you can constrain the join order by  
setting join_collapse_limit to 1, but then you must write the joins  
you want constrained using JOIN and the others as FROM items, which of  
course doesn't work at all for LEFT or RIGHT joins and will have  
random and unpredictable effects on subqueries pulled in via VIEWs.   
Needing to write INNER to be able to use FORCE pales by comparison.


That having been said, I'm not excited about pushing water up a hill.   
The important thing here is to remove the collapse limits; providing a  
tool to control the join order that won't make you want to beat your  
head in a brick is just something that can be trivially done with no  
extra code, not my primary goal.


...Robert

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Jaime Casanova
On Fri, Jul 10, 2009 at 10:22 AM, Robert Haasrobertmh...@gmail.com wrote:
 I took a look at this and it seems that #3 can be implemented with
 essentially no additional code (the handful of lines I added where
 more than balanced out by some simplifications in ruleutils.c).  Of
 course you still don't have to like it.  :-)

 Patch attached.


! SELECT * FROM a INNER FORCE JOIN (b INNER FORCE JOIN c ON (b.ref =
c.id)) ON (a.id = b.id);

what's next? FORCE INDEX?
once we implement one hint like this the complaints about the others
will lose force


-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 You do, but it's been pretty rare in my experience, and we're
 considering alternatives which give a lot less flexibility that this.

I dunno about considering.  We've already wasted vastly more time on
this than it's worth.  AFAIR there has never been one single user
request for the ability to partially constrain join order.  I think we
should do an enable_join_ordering boolean and quit wasting brainpower on
the issue.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-10 Thread Josh Berkus

  I think we might be better off just

leaving the closed commitfests up on the wiki, and putting a notice on
the app saying commitfests prior to July 2009 can be found at
wiki.postgresql.org.


+1.  That's why we're switching technogies at the beginning of a dev cycle.


--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] git.postgresql.org vs. REL8_1_STABLE

2009-07-10 Thread James Pye

On Jul 10, 2009, at 11:03 AM, Joshua Tolley wrote:

Am I the only one having problems building 8.1 from git? (Am I the  
only one
building 8.1 from git?) In a clean repository, I've checked out  
REL8_1_STABLE,
configured with only one argument, to set --prefix, and make gives  
me this:


While not this exact issue, I have had issues compiling older _STABLE  
branches from git.
At the time, I wasn't sure if I somehow borked my clone or not, so I  
just moved on.


[iirc, my problem was with dbcommand.c. the repo seemed to have  
checked out an older header file..]


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Ron Mayer
Tom Lane wrote:
 Kevin Grittner kevin.gritt...@wicourts.gov writes:
 You do, but it's been pretty rare in my experience, and we're
 considering alternatives which give a lot less flexibility that this.
 
 I dunno about considering.  We've already wasted vastly more time on
 this than it's worth.  AFAIR there has never been one single user
 request for the ability to partially constrain join order.  I think we
 should do an enable_join_ordering boolean and quit wasting brainpower on
 the issue.

I think I've found it useful in the past[1], but I also think we
already have a way to give postgres such hints using subselects
and offset 0.

Instead of SAP-DB's
 select * from (t1 join t2 on whatever) join t3 on whatever;
ISTM we can already do
 select * from (select t1 join t2 on whatever offset 0) as a join t3 on 
 whatever;
which seems like a reasonably way of hinting which parenthesis
can be reordered and which can't.


Would these new proposals give (guc's or syntax hacks) anything that
I can't already do?



[1] http://archives.postgresql.org/pgsql-performance/2007-12/msg00088.php

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Josh Berkus

All,

I'd suggest that we publish an official policy, with the following dates 
for EOL:


7.4   2009-08-01
8.0   2010-02-01
8.1   2011-01-01
8.2   2012-01-01
8.3   2013-03-01
8.4   2014-08-01

EOL would be defined as:

After the above dates, the PostgreSQL Project will not promise to 
provide any minor (patch or update) releases of the respective versions, 
whether for security, data loss, or any other issue.  Individual 
companies might choose to continue backporting patches to earlier 
versions, and if they contribute these update releases we will make them 
available.  However, after the final minor release date, there is no 
guarantee that updates will be available and users should upgrade.



--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Commitfest Code Sprint with PUGs

2009-07-10 Thread Josh Berkus

All,

Gabrielle of PDXPUG volunteered that PUG to hold a code sprint in 
coordination with a commitfest sometime soon.  For that event, the 
PDXPUG members would take on a dozen or so patches, compile and review 
them and submit the results.


The reason I mention this here is that they would need to reserve 
several patches we felt were suitable for beginners for the event.  This 
would require reviewers to hold off on those patches until after the 
code sprint to make sure that the PUG would have a good event and 
contribute usefully.  For this reason, the PUG event would need to be at 
the beginning of the CF.


Obviously PDXPUG won't be ready for July 15th.  But could we plan this 
for the Sept. 15th CF?


Also, do any other PUGs want to do this?

--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread David E. Wheeler

On Jul 10, 2009, at 4:01 PM, Josh Berkus wrote:


All,

I'd suggest that we publish an official policy, with the following  
dates for EOL:


7.4   2009-08-01
8.0   2010-02-01
8.1   2011-01-01
8.2   2012-01-01
8.3   2013-03-01
8.4   2014-08-01

EOL would be defined as:

After the above dates, the PostgreSQL Project will not promise to  
provide any minor (patch or update) releases of the respective  
versions, whether for security, data loss, or any other issue.   
Individual companies might choose to continue backporting patches to  
earlier versions, and if they contribute these update releases we  
will make them available.  However, after the final minor release  
date, there is no guarantee that updates will be available and users  
should upgrade.


+1

It's useful to have the version number and date of the most recent  
release too, but this is the important stuff.


Best,

David


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes:
 I'd suggest that we publish an official policy, with the following dates 
 for EOL:

 7.4   2009-08-01
 8.0   2010-02-01
 8.1   2011-01-01
 8.2   2012-01-01
 8.3   2013-03-01
 8.4   2014-08-01

I have no objection to setting an EOL date for 7.4 now, but I think it
is premature to set EOL dates for later versions.  I suppose the policy
you have in mind here (but are not spelling out) is that versions will
be EOL'd five years after release no matter what.  I'm not convinced
that we need to have a policy for that at all; but if we were to set
one, I'd prefer to define it in terms of the maximum number of back
branches we want to deal with.  (So it'd go more like a version will be
EOL'd shortly after the release of the fifth subsequent major version.)
Or, if you want something calendar-based, it should be driven off the
release of the immediately following major version (i.e., not less than
four years after the release of the next major version), so that people
would always know that they have at least N years' support when they
adopt the current major version.

But on the whole I think we should NOT have such a policy, at all.
If we'd enunciated such a thing in 2005, we'd still be on the hook to
support 8.0 on Windows; or else have had to go back on our word.  The
truth of the matter is that the community will make reasonable efforts
to support back branches but we are not going to set anything in stone.
If someone wants a guaranteed EOL date, they ought to be contracting
with a commercial support company and paying appropriately.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Bruce Momjian
David E. Wheeler wrote:
 On Jul 10, 2009, at 4:01 PM, Josh Berkus wrote:
 
  All,
 
  I'd suggest that we publish an official policy, with the following  
  dates for EOL:
 
  7.4   2009-08-01
  8.0   2010-02-01
  8.1   2011-01-01
  8.2   2012-01-01
  8.3   2013-03-01
  8.4   2014-08-01
 
  EOL would be defined as:
 
  After the above dates, the PostgreSQL Project will not promise to  
  provide any minor (patch or update) releases of the respective  
  versions, whether for security, data loss, or any other issue.   
  Individual companies might choose to continue backporting patches to  
  earlier versions, and if they contribute these update releases we  
  will make them available.  However, after the final minor release  
  date, there is no guarantee that updates will be available and users  
  should upgrade.
 
 +1
 
 It's useful to have the version number and date of the most recent  
 release too, but this is the important stuff.

We haven't committed to something like this in the past but it is
probably time where we can't avoid it.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Robert Haas

On Jul 10, 2009, at 6:01 PM, Josh Berkus j...@agliodbs.com wrote:


All,

I'd suggest that we publish an official policy, with the following  
dates for EOL:


7.4   2009-08-01
8.0   2010-02-01
8.1   2011-01-01
8.2   2012-01-01
8.3   2013-03-01
8.4   2014-08-01

EOL would be defined as:

After the above dates, the PostgreSQL Project will not promise to  
provide any minor (patch or update) releases of the respective  
versions, whether for security, data loss, or any other issue.   
Individual companies might choose to continue backporting patches to  
earlier versions, and if they contribute these update releases we  
will make them available.  However, after the final minor release  
date, there is no guarantee that updates will be available and users  
should upgrade.


It seems pretty speculative to propose end of support dates for every  
active release at this point; what if it takes us longer than planned  
to get the next few releases out?


I think we could try to set dates for the older releases.

...Robert

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Andrew Dunstan



Tom Lane wrote:

But on the whole I think we should NOT have such a policy, at all.
If we'd enunciated such a thing in 2005, we'd still be on the hook to
support 8.0 on Windows; or else have had to go back on our word.  The
truth of the matter is that the community will make reasonable efforts
to support back branches but we are not going to set anything in stone.
If someone wants a guaranteed EOL date, they ought to be contracting
with a commercial support company and paying appropriately.


  


I think we can avoid most of these problems by making a best effort 
policy rather than a hard promise.  But it can be moderately specific 
about what we will make best efforts towards. I agree that anyone who 
wants a hard promise should be getting commercial support.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes:
 I think we can avoid most of these problems by making a best effort 
 policy rather than a hard promise.  But it can be moderately specific 
 about what we will make best efforts towards. I agree that anyone who 
 wants a hard promise should be getting commercial support.

I don't mind the idea of saying our intention is to support new
releases for about five years, or something equally squishy.
But a list of dates in black and white does not look reasonable,
especially not dates that are four or five years out for versions
that have zero track record.  We have no idea whatsoever what the
future will bring.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread Steve Crawford

Tom Lane wrote:

Andrew Dunstan and...@dunslane.net writes:
  
I think we can avoid most of these problems by making a best effort 
policy rather than a hard promise.  But it can be moderately specific 
about what we will make best efforts towards. I agree that anyone who 
wants a hard promise should be getting commercial support.



I don't mind the idea of saying our intention is to support new
releases for about five years, or something equally squishy.
But a list of dates in black and white does not look reasonable,
especially not dates that are four or five years out for versions
that have zero track record.  We have no idea whatsoever what the
future will bring.
  
Would it be reasonable to have the squishy intention coupled with a 
more firm policy of ...EOL will be announced X months in 
advance...Users requiring firm long-term EOL commitments are advised to 
purchase commercial support...


Perhaps the postgresql.org home-page should be modified slightly. 
Instead of Latest Releases (which doesn't even list 7.4 when I just 
looked), it could be something like Current Releases. Then when EOL is 
announced, the release could be suffixed with the EOL date (i.e. 7.4.25 
EOL 2009-12-31 - maybe even with the EOL date in bold and/or red) which 
would link to the EOL announcement or general EOL statement page.


I think that a EOL Statement link to a page with the generic statement 
placed just below the oldest release could be helpful as well.


Cheers,
Steve



Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread David E. Wheeler

On Jul 10, 2009, at 5:39 PM, Tom Lane wrote:


I don't mind the idea of saying our intention is to support new
releases for about five years, or something equally squishy.
But a list of dates in black and white does not look reasonable,
especially not dates that are four or five years out for versions
that have zero track record.  We have no idea whatsoever what the
future will bring.


I think that it would be useful to have the squish comment, but also a  
list of versions that are definitely no longer supported, and when  
support ceased.


Best,

David

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Maintenance Policy?

2009-07-10 Thread D'Arcy J.M. Cain
On Fri, 10 Jul 2009 19:51:31 -0400
Tom Lane t...@sss.pgh.pa.us wrote:
 Josh Berkus j...@agliodbs.com writes:
  I'd suggest that we publish an official policy, with the following dates 
  for EOL:
 
 I have no objection to setting an EOL date for 7.4 now, but I think it
 is premature to set EOL dates for later versions.  I suppose the policy
 you have in mind here (but are not spelling out) is that versions will
 be EOL'd five years after release no matter what.  I'm not convinced

How about five (or four or...) years after the next version is
released? That takes into account longer release schedules.  That way
we aren't guaranteeing support for a hard term for a release but rather
that we will support it for a specified time from the date it is
superceded by the next version.

-- 
D'Arcy J.M. Cain da...@druid.net |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] concurrent index builds unneeded lock?

2009-07-10 Thread Theo Schlossnagle
We just ran into a case where we were performing two concurrent index  
builds on two different tables in two different schemas in the same  
database (no relational constraints between them).


One of the index builds locked on the other.

The first index build started...
The second index build started...
The first one locked on the second one
The second one finished...
The first one was allows to continue and finish.

quux=# select *  from pg_locks where pid IN (25264, 20108);
  locktype  | database | relation | page | tuple | virtualxid |  
transactionid | classid | objid | objsubid | virtualtransaction |   
pid  |   mode   | granted
+--+--+--+---+ 
+---+-+---+--+ 
+---+--+-
 relation   |16385 |25852 |  |   | 
|   | |   |  | 9/3041 |  
20108 | RowExclusiveLock | t
 relation   |16385 |25861 |  |   | 
|   | |   |  | 1/15735|  
25264 | RowExclusiveLock | t
 relation   |16385 |16421 |  |   | 
|   | |   |  | 9/3041 |  
20108 | ShareUpdateExclusiveLock | t
 virtualxid |  |  |  |   | 9/3041  
|   | |   |  | 9/3041 |  
20108 | ExclusiveLock| t
 virtualxid |  |  |  |   | 1/15735 
|   | |   |  | 1/15735|  
25264 | ExclusiveLock| t
 virtualxid |  |  |  |   | 9/3041  
|   | |   |  | 1/15735|  
25264 | ShareLock| f
 relation   |16385 |16528 |  |   | 
|   | |   |  | 1/15735|  
25264 | ShareUpdateExclusiveLock | t

(7 rows)

Reading the comments in the concurrent index build code, it seems that  
in prep for phase 3 of the index build it looks for any open txns that  
could feasibly see deleted tuples prior to the snap.


I would think it would be txns that would be reading that table, but  
I'm thinking it is a bit to aggressive.  Am I reading the code wrong  
there?  I'm thinking it should be more selective about vxids it  
chooses to block on.  I'd expect it to block on vxids touching the  
same table only.


Thoughts?

--
Theo Schlossnagle
http://omniti.com/is/theo-schlossnagle
p: +1.443.325.1357 x201   f: +1.410.872.4911






--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] concurrent index builds unneeded lock?

2009-07-10 Thread Tom Lane
Theo Schlossnagle je...@omniti.com writes:
 I would think it would be txns that would be reading that table, but  
 I'm thinking it is a bit to aggressive.  Am I reading the code wrong  
 there?  I'm thinking it should be more selective about vxids it  
 chooses to block on.  I'd expect it to block on vxids touching the  
 same table only.

There is no way to know whether a currently active vxid will try to look
at the other table later.  We can not just ignore this case...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Robert Haas
On Fri, Jul 10, 2009 at 2:44 PM, Jaime
Casanovajcasa...@systemguards.com.ec wrote:
 On Fri, Jul 10, 2009 at 10:22 AM, Robert Haasrobertmh...@gmail.com wrote:
 I took a look at this and it seems that #3 can be implemented with
 essentially no additional code (the handful of lines I added where
 more than balanced out by some simplifications in ruleutils.c).  Of
 course you still don't have to like it.  :-)

 Patch attached.


 ! SELECT * FROM a INNER FORCE JOIN (b INNER FORCE JOIN c ON (b.ref =
 c.id)) ON (a.id = b.id);

 what's next? FORCE INDEX?
 once we implement one hint like this the complaints about the others
 will lose force

That would be pretty useless, because while it might work for trivial
cases, in a complex plan, it's almost certainly not going to do what
you want unless you specify every detail of the entire plan along with
it, which kind of defeats the purpose of using a database with a
sophisticated planner.  Actually, it seems to me that if we were to
add hints, the place to start would be with selectivity, since in my
experience bad selectivity estimates (often by 3, 4, 5, or 6 orders of
magnitude) are often the reason for a bad plan, and if you can fix
that, the rest takes care of itself - but fixing it is often very
difficult.  Of course, improving the selectivity estimator would be
even better, because time spent applying hints to queries is time that
could be better spent doing something else, and in fact one of the
reasons why I started using PostgreSQL is because complex queries Just
Work.

Constraining the join order does seem a lot less useful, because (for
example) it won't compel the planner to use a hash join instead of a
nest join, or to make a sensible decision about which relations should
be on the inner side of the join.  But I still think that it's worth
considering, because:

1. We basically already support the functionality - it's just broken.
Today, you can control the join order by setting join_collapse_limit =
1; then, the joins you want to order you specify using JOIN syntax;
the ones you don't want to order, you write as FROM items.  But all of
your outer joins will necessarily have the order forced, because
there's no way to write those as FROM items.  And if you want to use a
view that was written without this convention in mind, you're hosed.
So I think we either ought to remove it or fix it.

2. It's actually LESS code to support it completely than it is to
leave it the way it is now while removing from_collapse_limit and
replacing join_collapse_limit with enable_join_ordering.  Compare the
patch I sent earlier with the one that I'm about to send.

3. The point of this particular type of hint, at least as I see it, is
not so much to force the planner into the correct query plan as it is
to constrain the planning time.  There are very few tools available
for this today: join_collapse_limit and from_collapse_limit do it by
unexpectedly producing terrible plans, and the only other option is
GEQO.  In a case like the one posted upthread where you have ten or so
joins that can be reordered almost arbitrarily, you could shove a
single FORCE right into the middle and split it into two problems that
can be planned consecutively.  This isn't that different from what
join_collapse_limit does today, but it doesn't force a uniform
threshold on you across the board; you can apply where, when, and to
the extent necessary to control planning time for a particular query,
and you have to explicitly ask for it so you can't complain you were
ambushed if it doesn't work out.

But I can see I'm outvoted, so I give up!

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-10 Thread Robert Haas
On Fri, Jul 10, 2009 at 2:48 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 Kevin Grittner kevin.gritt...@wicourts.gov writes:
 You do, but it's been pretty rare in my experience, and we're
 considering alternatives which give a lot less flexibility that this.

 I dunno about considering.  We've already wasted vastly more time on
 this than it's worth.  AFAIR there has never been one single user
 request for the ability to partially constrain join order.  I think we
 should do an enable_join_ordering boolean and quit wasting brainpower on
 the issue.

Patch attached.

...Robert
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***
*** 1798,1803  archive_command = 'copy %p C:\\server\\archivedir\\%f'  # Windows
--- 1798,1823 
/listitem
   /varlistentry
  
+  varlistentry id=guc-enable-join-ordering xreflabel=enable_join_ordering
+   termvarnameenable_join_ordering/varname (typeboolean/type)/term
+   indexterm
+primaryvarnameenable_join_ordering/ configuration parameter/primary
+   /indexterm
+   listitem
+para
+ Allows the planner to reorder joins, which is generally appropriate
+ for most uses. Setting it to false prevents any reordering of
+ explicit literalJOIN/s. Thus, the explicit join order
+ specified in the query will be the actual order in which the
+ relations are joined. The query planner does not always choose
+ the optimal join order; advanced users can elect to
+ temporarily set this variable to false, and then specify the join
+ order they desire explicitly.
+ For more information see xref linkend=explicit-joins.
+/para
+   /listitem
+  /varlistentry
+ 
   varlistentry id=guc-enable-mergejoin xreflabel=enable_mergejoin
termvarnameenable_mergejoin/varname (typeboolean/type)/term
indexterm
***
*** 2251,2313  SELECT * FROM parent WHERE key = 2400;
/listitem
   /varlistentry
  
-  varlistentry id=guc-from-collapse-limit xreflabel=from_collapse_limit
-   termvarnamefrom_collapse_limit/varname (typeinteger/type)/term
-   indexterm
-primaryvarnamefrom_collapse_limit/ configuration parameter/primary
-   /indexterm
-   listitem
-para
- The planner will merge sub-queries into upper queries if the
- resulting literalFROM/literal list would have no more than
- this many items.  Smaller values reduce planning time but might
- yield inferior query plans.  The default is eight.
- For more information see xref linkend=explicit-joins.
-/para
- 
-para
- Setting this value to xref linkend=guc-geqo-threshold or more
- may trigger use of the GEQO planner, resulting in nondeterministic
- plans.  See xref linkend=runtime-config-query-geqo.
-/para
-   /listitem
-  /varlistentry
- 
-  varlistentry id=guc-join-collapse-limit xreflabel=join_collapse_limit
-   termvarnamejoin_collapse_limit/varname (typeinteger/type)/term
-   indexterm
-primaryvarnamejoin_collapse_limit/ configuration parameter/primary
-   /indexterm
-   listitem
-para
- The planner will rewrite explicit literalJOIN/
- constructs (except literalFULL JOIN/s) into lists of
- literalFROM/ items whenever a list of no more than this many items
- would result.  Smaller values reduce planning time but might
- yield inferior query plans.
-/para
- 
-para
- By default, this variable is set the same as
- varnamefrom_collapse_limit/varname, which is appropriate
- for most uses. Setting it to 1 prevents any reordering of
- explicit literalJOIN/s. Thus, the explicit join order
- specified in the query will be the actual order in which the
- relations are joined. The query planner does not always choose
- the optimal join order; advanced users can elect to
- temporarily set this variable to 1, and then specify the join
- order they desire explicitly.
- For more information see xref linkend=explicit-joins.
-/para
- 
-para
- Setting this value to xref linkend=guc-geqo-threshold or more
- may trigger use of the GEQO planner, resulting in nondeterministic
- plans.  See xref linkend=runtime-config-query-geqo.
-/para
-   /listitem
-  /varlistentry
- 
   /variablelist
  /sect2
 /sect1
--- 2271,2276 
*** a/doc/src/sgml/perform.sgml
--- b/doc/src/sgml/perform.sgml
***
*** 692,699  SELECT * FROM a JOIN (b JOIN c ON (b.ref = c.id)) ON (a.id = b.id);
para
 To force the planner to follow the join order laid out by explicit
 literalJOIN/s,
!set the xref linkend=guc-join-collapse-limit run-time parameter to 1.
!(Other possible values are discussed below.)
/para
  

Re: [HACKERS] concurrent index builds unneeded lock?

2009-07-10 Thread Theo Schlossnagle



On Jul 11, 2009, at 12:12 AM, Tom Lane wrote:


Theo Schlossnagle je...@omniti.com writes:

I would think it would be txns that would be reading that table, but
I'm thinking it is a bit to aggressive.  Am I reading the code wrong
there?  I'm thinking it should be more selective about vxids it
chooses to block on.  I'd expect it to block on vxids touching the
same table only.


There is no way to know whether a currently active vxid will try to  
look

at the other table later.  We can not just ignore this case...

regards, tom lane



Can't you know that if the other active query in question is a  
concurrent index build?


Concurrent index builds by their current implementation cannot exist  
within another transaction, so you know everythere there is to know  
about that transaction by looking at it (no risk of prior or future  
work).


While very much unlike a vacuum (a special exclusion in concurrent  
index builds), they still seem to constitute a special case for  
exclusion.


Happy to be wrong here, I really haven't completely digested the code.

--
Theo Schlossnagle
http://omniti.com/is/theo-schlossnagle
p: +1.443.325.1357 x201   f: +1.410.872.4911






--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers