Re: [HACKERS] ERROR: GIN indexes do not support whole-index scans

2010-05-21 Thread Kevin Flanagan
Ah - you mentioning index definitions has suddenly made it clearer just what that error message might mean. The source_lang_code and target_lang_code columns didn't yet each have an index. If I create an index for either one of them, the error then goes away, I'm guessing because the query

Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 4:20 , Florian Pflug wrote: On May 19, 2010, at 2:15 , Florian Pflug wrote: On May 17, 2010, at 3:30 , Robert Haas wrote: On Sun, May 16, 2010 at 9:07 PM, Florian Pflug f...@phlo.org wrote: On May 14, 2010, at 22:54 , Robert Haas wrote: On Thu, May 13, 2010 at 5:39 PM, Tom

[HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
Folks, I feel dumb. I have been looking for a document which specifies what trusted and untrusted PLs must do and forbid, so far without result. Where do we document this, and if we don't where *should* we document this? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: I have been looking for a document which specifies what trusted and untrusted PLs must do and forbid, so far without result. I think you might have been missing the tree for the forest in this case.. :) I'm sure you've seen this, but perhaps you weren't

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Peter Geoghegan
That's about it- a language is TRUSTED if there's no way for a user to be able to write a function which will give them access to things they're not supposed to have.  Practically, this includes things like any kind of direct I/O (files, network, etc). The fact that plpythonu used to be

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Peter Geoghegan peter.geoghega...@gmail.com writes: That's about it- a language is TRUSTED if there's no way for a user to be able to write a function which will give them access to things they're not supposed to have.  Practically, this includes things like any kind of direct I/O (files,

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Robert Haas
2010/5/20 Pavel baro...@seznam.cz: For this summer I have plan to make patch inplementing snapshot materialized views (MV). I believe it will not be end of effort to implement more of MV. But I / we need discuss MV syntax and exact behaviour so I have some questions about that for all of you:

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Massa, Harald Armin
Pavel, b) create MV syntax? - CREATE MATERIALIZED VIEW mvname AS ..., I think it is quite obvious to do so, but I had to ask please do not fortget the: create or replace MATERIALIZED VIEW option. And also the DROP if exists for the drop-command Best wishes Harald -- GHUM Harald

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Tom Lane
Massa, Harald Armin c...@ghum.de writes: please do not fortget the: create or replace MATERIALIZED VIEW option. Please do. For something as complex as a table or view, CREATE OR REPLACE is a lot more complicated than it is for simple objects like functions. (See flamewar just a couple weeks

[HACKERS] changed source files.

2010-05-21 Thread MMK
Hello: How does one find out what source files were modified between two releases? For example, what .c and .h changed between 8.4.3 and 8.4.4? The Notes do not mention specific source files. Thanks, MMK.

Re: [HACKERS] changed source files.

2010-05-21 Thread Stephen Frost
MMK, * MMK (bom...@yahoo.com) wrote: How does one find out what source files were modified between two releases? For example, what .c and .h changed between 8.4.3 and 8.4.4? The Notes do not mention specific source files. You can pull down the CVS tags for each and then run a recursive diff..

[HACKERS] unnailing shared relations (was Re: global temporary tables)

2010-05-21 Thread Robert Haas
On Sat, Apr 24, 2010 at 6:53 PM, Robert Haas robertmh...@gmail.com wrote (in reply to Tom Lane):  If we create, e.g. pg_shared_class and pg_shared_attribute, then we can un-nail the catalogs you just nailed to make the authentication process able to work without selecting a database. Actually,

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 15:59 , Robert Haas wrote: 2010/5/20 Pavel baro...@seznam.cz: For this summer I have plan to make patch inplementing snapshot materialized views (MV). I believe it will not be end of effort to implement more of MV. But I / we need discuss MV syntax and exact behaviour so I

Re: [HACKERS] unnailing shared relations (was Re: global temporary tables)

2010-05-21 Thread Pavel Stehule
2010/5/21 Robert Haas robertmh...@gmail.com: On Sat, Apr 24, 2010 at 6:53 PM, Robert Haas robertmh...@gmail.com wrote (in reply to Tom Lane):  If we create, e.g. pg_shared_class and pg_shared_attribute, then we can un-nail the catalogs you just nailed to make the authentication process able to

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Josh Berkus
So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports 3) uses only the SPI interface to interact with postgresql tables etc. 4) does any logging only using elog to the postgres log Questions: a) it seems like there should be some

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Magnus Hagander
On Fri, May 21, 2010 at 11:55 AM, Josh Berkus j...@agliodbs.com wrote: So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports To make that more covering, don't yu really need something like cannot communicate with outside processes?

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Josh Berkus
On 05/21/2010 11:57 AM, Magnus Hagander wrote: On Fri, May 21, 2010 at 11:55 AM, Josh Berkusj...@agliodbs.com wrote: So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports To make that more covering, don't yu really need something

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 11:57:33AM -0400, Magnus Hagander wrote: On Fri, May 21, 2010 at 11:55 AM, Josh Berkus j...@agliodbs.com wrote: So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports To make that more covering, don't

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: These need to be testable conditions, and new tests need to get added any time we find that we've missed something. Making this concept fuzzier is exactly the wrong direction to go. I'm really not sure that we want to be in the business of writing a

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Magnus Hagander
On Fri, May 21, 2010 at 12:22 PM, David Fetter da...@fetter.org wrote: On Fri, May 21, 2010 at 11:57:33AM -0400, Magnus Hagander wrote: On Fri, May 21, 2010 at 11:55 AM, Josh Berkus j...@agliodbs.com wrote: So, here's a working definition: 1) cannot directly read or write files on the

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 12:26:24PM -0400, Stephen Frost wrote: * David Fetter (da...@fetter.org) wrote: These need to be testable conditions, and new tests need to get added any time we find that we've missed something. Making this concept fuzzier is exactly the wrong direction to go.

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: That is *precisely* the business we need to be in, at least for the languages we ship, and it would behoove us to test languages we don't ship so we can warn people when they don't pass. k, let's start with something simpler first tho- I'm sure we can

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 01:45:45PM -0400, Stephen Frost wrote: * David Fetter (da...@fetter.org) wrote: That is *precisely* the business we need to be in, at least for the languages we ship, and it would behoove us to test languages we don't ship so we can warn people when they don't pass.

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 18:26 , Stephen Frost wrote: * David Fetter (da...@fetter.org) wrote: These need to be testable conditions, and new tests need to get added any time we find that we've missed something. Making this concept fuzzier is exactly the wrong direction to go. I'm really not

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* David Fetter (da...@fetter.org) wrote: On Fri, May 21, 2010 at 01:45:45PM -0400, Stephen Frost wrote: k, let's start with something simpler first tho- I'm sure we can pull in the glibc regression tests and run them too. You know, just in case there's a bug there, somewhere. That's

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
David Fetter da...@fetter.org writes: On Fri, May 21, 2010 at 12:26:24PM -0400, Stephen Frost wrote: I'm really not sure that we want to be in the business of writing a ton of regression tests to see if languages which claim to be trusted really are.. That is *precisely* the business we need

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Robert Haas
On Fri, May 21, 2010 at 1:58 PM, David Fetter da...@fetter.org wrote: On Fri, May 21, 2010 at 01:45:45PM -0400, Stephen Frost wrote: * David Fetter (da...@fetter.org) wrote: That is *precisely* the business we need to be in, at least for the languages we ship, and it would behoove us to test

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: So... can we get back to coming up with a reasonable definition, (1) no access to system calls (including file and network I/O) (2) no access to process memory, other than variables defined within the PL. What else? regards,

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Robert Haas
On Fri, May 21, 2010 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So... can we get back to coming up with a reasonable definition, (1) no access to system calls (including file and network I/O) (2) no access to process memory, other than variables

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Well, the best way to define what a trusted language can do is to define a *whitelist* of what it can do, not a blacklist of what it can't do. That's the only way to get a complete definition. It's then up to the implementation step to

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: So... can we get back to coming up with a reasonable definition, and Guess I'm wondering if we could steal such a definition from one of the languages we allow as trusted already.. Just a thought. I certainly think we should make sure that we

[HACKERS] small exclusion constraints patch

2010-05-21 Thread Jeff Davis
Currently, the check for exclusion constraints performs a sanity check that's slightly too strict -- it assumes that a tuple will conflict with itself. That is not always the case: the operator might be , in which case it's perfectly valid for the search for conflicts to not find itself. This

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Greg Sabino Mullane g...@turnstep.com writes: Well, the best way to define what a trusted language can do is to define a *whitelist* of what it can do, not a blacklist of what it can't do. No, that's exactly backwards. We can't define all the things a language can do, but we can certainly

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, May 21, 2010 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: (1) no access to system calls (including file and network I/O) (2) no access to process memory, other than variables defined within the PL. What else? Doesn't subvert the general

Re: [HACKERS] changed source files.

2010-05-21 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 How does one find out what source files were modified between two releases? For example, what .c and .h changed between 8.4.3 and 8.4.4? The Notes do not mention specific source files. You can pull down the CVS tags for each and then run a

Re: [HACKERS] small exclusion constraints patch

2010-05-21 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: Currently, the check for exclusion constraints performs a sanity check that's slightly too strict -- it assumes that a tuple will conflict with itself. That is not always the case: the operator might be , in which case it's perfectly valid for the search

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 03:15:27PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Fri, May 21, 2010 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: (1) no access to system calls (including file and network I/O) (2) no access to process memory, other than variables

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread David Fetter
On Fri, May 21, 2010 at 12:36:50PM -0700, David Fetter wrote: On Fri, May 21, 2010 at 03:15:27PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Fri, May 21, 2010 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: (1) no access to system calls (including file and network

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Joshua Tolley
On Fri, May 21, 2010 at 1:36 PM, David Fetter da...@fetter.org wrote: On Fri, May 21, 2010 at 03:15:27PM -0400, Tom Lane wrote: As long as you can't do database access except via SPI, that should be covered.  So I guess the next item on the list is no, or at least restricted, access to

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Tom Lane
Joshua Tolley eggyk...@gmail.com writes: Agreed. As long as a trusted language can do things outside the database only by going through a database and calling some function to which the user has rights, in an untrusted language, that seems decent to me. A user with permissions to

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Jonathan Leto
Howdy, On Fri, May 21, 2010 at 11:21 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So... can we get back to coming up with a reasonable definition, (1) no access to system calls (including file and network I/O) (2) no access to process memory, other than

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Joshua Tolley
On Fri, May 21, 2010 at 2:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Joshua Tolley eggyk...@gmail.com writes: Agreed. As long as a trusted language can do things outside the database only by going through a database and calling some function to which the user has rights, in an untrusted

Re: [HACKERS] Specification for Trusted PLs?

2010-05-21 Thread Jan Wieck
The original idea was that a trusted language does not allow an unprivileged user to gain access to any object or data, he does not have access to without that language. This does not include data transformation functionality, like string processing or the like. As long as the user had

Re: [HACKERS] unnailing shared relations (was Re: global temporary tables)

2010-05-21 Thread Robert Haas
On Fri, May 21, 2010 at 11:10 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2010/5/21 Robert Haas robertmh...@gmail.com: On Sat, Apr 24, 2010 at 6:53 PM, Robert Haas robertmh...@gmail.com wrote (in reply to Tom Lane):  If we create, e.g. pg_shared_class and pg_shared_attribute, then we can

[HACKERS] (9.1) btree_gist support for searching on not equals

2010-05-21 Thread Jeff Davis
This patch adds support to btree_gist for searching on (not equals). This allows an interesting use of exclusion constraints: Say you have a table: create table zoo ( cage int, animal text, exclude using gist (cage with =, animal with ) ); That will permit you to add as

[HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-21 Thread Josh Berkus
From a discussion at dinner at pgcon, I wanted to send this to the list for people to poke holes in it: Problem: currently, if your database has a large amount of cold data, such as 350GB of 3-year-old sales transactions, in 8.4 vacuum no longer needs to touch it thanks to the visibility map.

Re: [HACKERS] (9.1) btree_gist support for searching on not equals

2010-05-21 Thread Marko Tiikkaja
On 5/21/10 11:47 PM +0300, Jeff Davis wrote: It also allows you to enforce the constraint that only one tuple exists in a table by doing something like: create table a ( i int, exclude using gist (i with), unique (i) ); FWIW, this is achievable a lot more easily:

[HACKERS] beta testing - planner bug - ERROR: XX000: failed to build any 2-way joins

2010-05-21 Thread Pavel Stehule
Hello there are maybe planner bug. test case: CREATE TABLE person ( id SERIAL PRIMARY KEY, name VARCHAR(64) ) WITHOUT OIDS; CREATE TABLE person_data ( id SERIAL PRIMARY KEY ) WITHOUT OIDS; SELECT name FROM person WHERE name IN (

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-21 Thread Florian Pflug
On May 21, 2010, at 23:57 , Josh Berkus wrote: From a discussion at dinner at pgcon, I wanted to send this to the list for people to poke holes in it: Problem: currently, if your database has a large amount of cold data, such as 350GB of 3-year-old sales transactions, in 8.4 vacuum no

[HACKERS] beta testing - pg_upgrade bug fix - double free

2010-05-21 Thread Pavel Stehule
Hello it fixes bug pg_upgrade(13359) malloc: *** error for object 0x801600: non-page-aligned, non-allocated pointer being freed *** set a breakpoint in malloc_error_break to debug arget 03:31 /usr/local/src/postgresql/contrib/pg_upgrade git diff . diff --git a/contrib/pg_upgrade/check.c