Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Simon Riggs
On Sun, 2008-01-13 at 17:44 -0800, Sean Utt wrote: It is not my contention that the core developers need to be different in any way. It is also not my contention that the users need to be different in any way. First, this is an open forum, so thank you for expressing your views openly in

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-14 Thread Simon Riggs
On Sat, 2008-01-12 at 16:22 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: ... So if we did this via an installable module approach then we would be able to use it much sooner for upgrading to 8.3, rather than waiting for 8.4 I think it borders on ludicrous to imagine

Re: [HACKERS] Declarative partitioning grammar

2008-01-14 Thread Markus Schiltknecht
Hi, Jeff Cohen wrote: We did look at allowing general functions for partitioning and this was one concern. The other is that we want to enforce that a row only gets inserted into a single partition, so we wanted a declarative syntax where it was relatively easy to check that range and list

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Csaba Nagy
On Mon, 2008-01-14 at 09:22 +, Simon Riggs wrote: So I support Mark Mielke's views on writing code. Anybody who wants to code, can. There's probably a project of a size and complexity that's right for your first project. The main problem is that usually that initial thing is not what you

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Jean-Michel Pouré
Dear Friends, I hope that this flame war can stop, as it is useless. The logic of free software is that developers pick-up issues, based on their skills and interest. The power of the cummunity is to gather very talented developers from all over the planet. Freedom is the logic and there is no

[HACKERS] Using tapes on tuplesort.c

2008-01-14 Thread mac_man2005
Hi to all. It seems that the current PostgreSQL implementation of the Replacement Selection (RS) algorithm [Knuth] changes a logical tape for each run built. I'm trying to implement that refinement to RS using 2 heaps instead of just one (2Way RS). Recall each heap is aimed at building its

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Hannes Dorbath
Tom Lane wrote: I've committed a patch to do that. Please test CVS HEAD and see if you still see problems. I'm happy to hear you found something and I will try CVS HEAD in a minute. In the meantime let me report that the cluster issue happens with GIST as well. I have load 5 million rows in

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Oooh ... I can't be sure that this is what's biting you, but I definitely see a bug that seems to match the symptoms. As the comments in index.c point out, CREATE INDEX CONCURRENTLY works like this: * validate_index() works by first gathering all the

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Mark Mielke
Jean-Michel Pouré wrote: When posting this thread, I hope that a talented developer would some day pick-up the materialized view issue and work on it, during a process of discussion. In case it was lost in the noise - there are several capable people that have been personally on this

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Alvaro Herrera
Mark Mielke wrote: FYI, my triggers are perhaps 10 lines each, and I believe I have three triggers in the 1500 ms - 1 ms example. I have a view and a summary table. I update the summary table from the view. In my opinion, this solution is very manageable given the 1500:1 performance

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Roberts, Jon
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Sunday, January 13, 2008 8:18 PM To: Sean Utt Cc: Andrew Dunstan; Joshua D. Drake; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Postgresql Materialized views Sean Utt [EMAIL PROTECTED] writes: My point is

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Martijn van Oosterhout
On Mon, Jan 14, 2008 at 08:30:53AM -0600, Roberts, Jon wrote: My point is that you should be able to query _table and the system should automatically use the view, without you saying so (except by initially creating them). I agree! From a BI perspective, a materialized view is worthless

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Zeugswetter Andreas ADI SD
But you had to modify your queries. I would think that a materialized views implementation worth its salt would put the view to work on the original, unmodified queries. I might be slow today (everyday? :-) ) - but what do you mean by this? The only difference between *_table and

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Andrew Dunstan
Roberts, Jon wrote: What gets implemented is whatever individual contributors choose to work on, either because they find it interesting or (in some cases) because someone pays them to do something specific. Certainly, some contributors pay attention to what's being requested, but I see no

[HACKERS] Pl/Java broken since Postgresql 8.3-rc1

2008-01-14 Thread [EMAIL PROTECTED]
Hi there, as you now is plJava broken with the actual security releases. There is a pljava.dll at http://www.ejurka.com/pgsql/pljava/83rc1/ to fix it for version 8.3RC1. Is a pljava.dll for version 8.2.6 out? It's very important for me, need it for my office. Juergen

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Tom Lane
Hannes Dorbath [EMAIL PROTECTED] writes: In the meantime let me report that the cluster issue happens with GIST as well. ... But as far as I understood this is already covered by your thesis. Right, the bug is independent of which index AM you use (though the symptoms may vary).

Re: [HACKERS] could not open relation: Invalid argument

2008-01-14 Thread Alvaro Herrera
Roberts, Jon wrote: Version: PostgreSQL 8.2.5 on i686-pc-mingw32 I recently started getting this error message randomly, could not open relation 42904/42906/42985: Invalid argument. I also got it for a couple of other files. All three files are related to tables that have just a single

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 My point is simply this: The lack of a clear formal process for feature requests leads to this degradation in the conversation. Without a formalized structure, the conversation devolves rapidly into an argument over semantics and word

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Tom Lane
Zeugswetter Andreas ADI SD [EMAIL PROTECTED] writes: Traditionally materialized views exist, so that you do not need to code differently. Your queries still run on the detail table, but are silently answered by a suitable MV. The MV might have count + other aggregated columns grouped by some

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Joshua D. Drake
Roberts, Jon wrote: 2) In the end, this is an open source *community*; no amount of formal feature requesting will have any material impact on what actually gets implemented, because there isn't any central control. Wow. Being new to Open Source, this amazes me. Well it depends on what

Re: [HACKERS] Pl/Java broken since Postgresql 8.3-rc1

2008-01-14 Thread Josh Berkus
Juergen, Is a pljava.dll for version 8.2.6 out? It's very important for me, need it for my office. Try e-mailing pgsql-jdbc mailing list and asking there. --Josh ---(end of broadcast)--- TIP 4: Have you searched our list archives?

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Zeugswetter Andreas ADI SD
Traditionally materialized views exist, so that you do not need to code differently. Your queries still run on the detail table, but are silently answered by a suitable MV. The MV might have count + other aggregated columns grouped by some columns, and thus be able e.g. shortcircuit a

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Ron Mayer
Mark Mielke wrote: Mark Mielke wrote: Counts, because as we all know, PostgreSQL count(*) is slow, and in any case, my count(*) is not on the whole table, but on a subset. Doing this in a general way seems complex to me as it would need to be able to evaluate whether a given INSERT or UPDATE

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Hannes Dorbath
Tom Lane wrote: I wrote: I think it's okay for CREATE INDEX CONCURRENTLY to use bulk-read access strategy (that is, seqscan using a limited number of buffers), but it has to be able to force the scan to start at page zero. I've committed a patch to do that. Please test CVS HEAD and see if

Re: [HACKERS] scan.l: check_escape_warning()

2008-01-14 Thread Tom Lane
Michael Meskes [EMAIL PROTECTED] writes: On Fri, Jan 11, 2008 at 10:41:17AM -0500, Tom Lane wrote: Perhaps there's some discrepancy between the ecpg and backend lexers as to where these are called? You're right. There is no way to (un)select standard conforming strings which makes up for the

Re: [HACKERS] timestamp refactor effort

2008-01-14 Thread Warren Turkal
On Jan 13, 2008 9:21 AM, Tom Lane [EMAIL PROTECTED] wrote: Warren Turkal [EMAIL PROTECTED] writes: I have a question. Are the low level representations of Timestamp and TimestampTZ the same? They're the same but the interpretations are different, which is why I think it's useful to have

Re: [HACKERS] Re: [COMMITTERS] pgsql: Most recent Postgres version is 8.2.6, per report from Robert

2008-01-14 Thread Magnus Hagander
Alvaro Herrera wrote: Bruce Momjian wrote: Log Message: --- Most recent Postgres version is 8.2.6, per report from Robert Treat. Can't we make this automatically somehow? Easiest way would be to have it refer to www.postgresql.org, where the frontpage already states this...

[HACKERS] Re: [COMMITTERS] pgsql: Most recent Postgres version is 8.2.6, per report from Robert

2008-01-14 Thread Alvaro Herrera
Bruce Momjian wrote: Log Message: --- Most recent Postgres version is 8.2.6, per report from Robert Treat. Can't we make this automatically somehow? Are we going to move to having the FAQ in XML Docbook FAQ format? That would rock and it would be easier to keep translations. --

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Jeff Davis
On Sun, 2008-01-13 at 18:52 -0500, Tom Lane wrote: The scan is done using the regular heapscan code, which in 8.3 has been modified to enable synchronized scanning, which means it might start from the middle of the table and wrap around. If that happens, the merge join will get totally

Re: [HACKERS] to_char incompatibility

2008-01-14 Thread Roberts, Jon
You'll have to explain to Oracle and their customers that Oracle's security model is not a great idea then. I'd love to, and in fact *do* whenever I'm given the chance. In fact, Oracle's security model is pretty bad; the reason why Oracle advertises Unbreakable so hard is that they

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: On Sun, 2008-01-13 at 18:52 -0500, Tom Lane wrote: I wonder whether there are any other places that are silently assuming that heapscans start from page zero ... I considered that question when implementing sync scans, but I could not think of any

Re: [HACKERS] to_char incompatibility

2008-01-14 Thread Josh Berkus
Jon, You'll have to explain to Oracle and their customers that Oracle's security model is not a great idea then. I'd love to, and in fact *do* whenever I'm given the chance. In fact, Oracle's security model is pretty bad; the reason why Oracle advertises Unbreakable so hard is that they

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-14 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: It has been reported that the data transmission overhead is much less than the connection establishing overhead, which is measured here. But this is certainly not an encouraging measurement, if we want to put this close to the default path of use. I

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Bruce Momjian
Added to TODO: * Add the ability to automatically create materialized views Right now materialized views require the user to create triggers on the main table to keep the summary table current. SQL syntax should be able to manager the triggers

Re: [HACKERS] to_char incompatibility

2008-01-14 Thread Josh Berkus
Jon, Also, there is no need to argue this because we can have it both ways. Security definer is an option and I recommend to always use it over the default. If you don't want to use it, don't. Security Definer has ramifications in PostgreSQL which I don't think it does in Oracle.

Re: [HACKERS] to_char incompatibility

2008-01-14 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Security Definer has ramifications in PostgreSQL which I don't think it does in Oracle. Particularly, see: http://www.postgresql.org/docs/techdocs.77 BTW, that article needs to be updated to show the (much easier) way to do it as of 8.3. I concur that

Re: [HACKERS] Bug: Unreferenced temp tables disables vacuum to update xid

2008-01-14 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Hmm ... that would be strange. Off-the-cuff idea: we introduced code to advance relfrozenxid in CLUSTER, TRUNCATE and table-rewriting forms of ALTER TABLE. Perhaps the problem is that we're neglecting to update it for the toast table there. AFAIR I

Re: [HACKERS] Declarative partitioning grammar

2008-01-14 Thread Gavin Sherry
On Sat, Jan 12, 2008 at 04:01:19PM +0530, NikhilS wrote: Hi, We did look at allowing general functions for partitioning and this was one concern. The other is that we want to enforce that a row only gets inserted into a single partition, so we wanted a declarative syntax where it was

Re: [HACKERS] Bug: Unreferenced temp tables disables vacuum to update xid

2008-01-14 Thread Alvaro Herrera
Tom Lane wrote: regression=# cluster foo_pkey on foo; CLUSTER regression=# select relname, relkind, relfrozenxid from pg_class order by oid desc limit 6; relname| relkind | relfrozenxid ---+-+-- pg_toast_707231_index | i |

Re: [HACKERS] Bug: Unreferenced temp tables disables vacuum to update xid

2008-01-14 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Perhaps what we could do is take the relfrozenxid from the old relation and copy it over, if it's later than FreezeXid? It certainly doesn't seem to make any sense to allow the rel's relfrozenxid to go backwards. Indeed this coding lets it end up less

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Tom Lane
Zeugswetter Andreas ADI SD [EMAIL PROTECTED] writes: Note that you just raised the minimum bar for implementation of the feature by a couple orders of magnitude. Yes, unfortunately. But don't you also think that this is what makes it a worthwhile feature ? Well, my point is that taking

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Jeff Davis [EMAIL PROTECTED] writes: On Sun, 2008-01-13 at 18:52 -0500, Tom Lane wrote: I wonder whether there are any other places that are silently assuming that heapscans start from page zero ... I considered that question when implementing sync scans,

[HACKERS] 8.3RC1 on windows missing descriptive Event handle names

2008-01-14 Thread Stephen Denne
On Windows XP, using Process Explorer with the lower pane showing Handles, not all postgres.exe processes are including an Event type with a description of what the process is doing. At the moment, I have ten postgres processes shown as being at the bottom of the tree, three of which do not

Re: [HACKERS] Declarative partitioning grammar

2008-01-14 Thread Jeff Cohen
On Jan 14, 2008, at 1:49 AM, Markus Schiltknecht wrote: I don't think the separation into list, hash and range partitioning is adequate. What is the system supposed to do, if you try to insert a row which doesn't fit any of the values in your list or doesn't fit any of the ranges you

Re: [HACKERS] to_char incompatibility

2008-01-14 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Josh Berkus [EMAIL PROTECTED] writes: Security Definer has ramifications in PostgreSQL which I don't think it does in Oracle. Particularly, see: http://www.postgresql.org/docs/techdocs.77 BTW, that article needs to be updated to show the (much easier)

Re: [HACKERS] Index trouble with 8.3b4

2008-01-14 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I went through all of the heap_beginscan calls in the code last night. pgstattuple was broken but AFAICS none of the other callers care about the visitation order. I wonder though about third-party add-ons :-(

Re: [HACKERS] could not open relation: Invalid argument

2008-01-14 Thread Jaime Casanova
On Jan 14, 2008 11:03 AM, Alvaro Herrera [EMAIL PROTECTED] wrote: Roberts, Jon wrote: Version: PostgreSQL 8.2.5 on i686-pc-mingw32 I recently started getting this error message randomly, could not open relation 42904/42906/42985: Invalid argument. I also got it for a couple of other

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-14 Thread Bruce Momjian
Tom Lane wrote: Conclusions: * SSL, even without real authentication, is *way* too expensive to enable by default. * The extra cost of going across a local TCP connection is measurable, but it's insignificant compared to the cost of turning on SSL. (This is on a Fedora 8 kernel BTW ...

Re: [HACKERS] Declarative partitioning grammar

2008-01-14 Thread Tom Lane
Jeff Cohen [EMAIL PROTECTED] writes: In the proposed solution, hash and list partitions work for all types that support an equality operator, and range partitions work for all types that support fully-ordered comparison. Surely a hashing method would require a *hashable* equality operator,

Re: [HACKERS] Declarative partitioning grammar

2008-01-14 Thread Jeff Cohen
On Jan 12, 2008, at 9:34 AM, Peter Eisentraut wrote: Well, with an extensible system such as PostgreSQL you will need to have a partitioning scheme that can deal with extensions. Perhaps people want to partition by XML, GIS, text-search data, or whatever someone might come up with in

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-14 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Yea, I figured using protected directories for the socket was the zero-cost solution, and if you have to do SSL, might as well just use TCP too. (If you moved the socket file to a protected directory I think you could use

[HACKERS] Re: [COMMITTERS] pgsql: Most recent Postgres version is 8.2.6, per report from Robert

2008-01-14 Thread Bruce Momjian
Alvaro Herrera wrote: Bruce Momjian wrote: Log Message: --- Most recent Postgres version is 8.2.6, per report from Robert Treat. Can't we make this automatically somehow? Hmmm, well, that file is pulled to our web site on every change so I am not sure how we could do this

Re: [HACKERS] Pl/Java broken since Postgresql 8.3-rc1

2008-01-14 Thread Kris Jurka
On Mon, 14 Jan 2008, Josh Berkus wrote: Juergen, Is a pljava.dll for version 8.2.6 out? It's very important for me, need it for my office. Try e-mailing pgsql-jdbc mailing list and asking there. The correct list is actually [EMAIL PROTECTED] The JDBC driver and the server side

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-14 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Yea, I figured using protected directories for the socket was the zero-cost solution, and if you have to do SSL, might as well just use TCP too. (If you moved the socket file to a protected directory I think you could use

Re: [HACKERS] Array behavior oddities

2008-01-14 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: If any dimension is written as a slice, i.e. contains a colon, then all dimensions are treated as slices. Is the the behavior of assuming an entry with no colon is a slice what we want, or are we just stuck with it? Why do you find that surprising?

[HACKERS] Array behavior oddities

2008-01-14 Thread Bruce Momjian
In reading our array documentation I came across two unusual behaviors. The issue relates to slices: We can also access arbitrary rectangular slices of an array, or subarrays. An array slice is denoted by writing

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-14 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: Yeah, all of this is about confusion and error-proneness. I still think that the real problem is that we don't have full control over client-side code, and therefore can't just write off the problem of a client deciding to connect to