Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-08 Thread Kohei KaiGai
2013/1/7 Robert Haas : > On Mon, Jan 7, 2013 at 2:14 PM, Alvaro Herrera > wrote: >> Kohei KaiGai escribió: >> >>> Function and collation are candidates of this special case handling; >>> here are just two kinds of object. >>> >>> Another idea is to add a function-pointer as argument of >>> AlterN

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Takeshi Yamamuro
Hi, Why would that be a good tradeoff to make? Larger stored values require more I/O, which is likely to swamp any CPU savings in the compression step. Not to mention that a value once written may be read many times, so the extra I/O cost could be multiplied many times over

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Takeshi Yamamuro
So why don't we use LZ4? +1 Agree though, I think there're still patent issues there. regards, -- Takeshi Yamamuro NTT Cyber Communications Laboratory Group Software Innovation Center (Open Source Software Center) Tel: +81-3-5860-5057 Fax: +81-3-5463-5490 Mail:yamamuro.take...@lab.ntt.c

Re: [HACKERS] lazy_vacuum_heap()'s removal of HEAPTUPLE_DEAD tuples

2013-01-08 Thread Pavan Deolasee
On Tue, Jan 8, 2013 at 8:19 AM, Noah Misch wrote: > > At that point in the investigation, I realized that the cost of being able > to > remove entire tuples in lazy_vacuum_heap() greatly exceeds the benefit. > Again, the benefit is being able to remove tuples whose inserting > transaction > abort

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Takeshi Yamamuro
Hi, (2013/01/07 22:36), Greg Stark wrote: On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to remember somebody even posting a patch t

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Magnus Hagander
On Tue, Jan 8, 2013 at 3:58 AM, Peter Eisentraut wrote: > For debugging PL/Python functions, I'm often tempted to write something > like > > rv = plpy.execute(...) > plpy.info(rv) > > which prints something unhelpful like > > > > By implementing a "str" handler for the result object, it now print

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Hannu Krosing
On 01/08/2013 10:19 AM, Takeshi Yamamuro wrote: Hi, (2013/01/07 22:36), Greg Stark wrote: On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I

Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 4:05 AM, Kohei KaiGai wrote: > Does it make sense an idea to invoke AlterFunctionNamespace_oid() > or AlterCollationNamespace_oid() from AlterObjectNamespace_internal() > for checks of namespace conflicts? > It can handle special cases with keeping modularity between common

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 4:04 AM, Takeshi Yamamuro wrote: > Apart from my patch, what I care is that the current one might > be much slow against I/O. For example, when compressing > and writing large values, compressing data (20-40MiB/s) might be > a dragger against writing data in disks (50-80MiB/

Re: [HACKERS] Re: Proposal: Store "timestamptz" of database creation on "pg_database"

2013-01-08 Thread Robert Haas
On Fri, Jan 4, 2013 at 1:07 PM, Peter Eisentraut wrote: > On 1/3/13 3:26 PM, Robert Haas wrote: >> It's true, as we've often >> said here, that leveraging the OS facilities means that we get the >> benefit of improving OS facilities "for free" - but it also means that >> we never exceed what the O

Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-08 Thread Kohei KaiGai
2013/1/8 Robert Haas : > On Tue, Jan 8, 2013 at 4:05 AM, Kohei KaiGai wrote: >> Does it make sense an idea to invoke AlterFunctionNamespace_oid() >> or AlterCollationNamespace_oid() from AlterObjectNamespace_internal() >> for checks of namespace conflicts? >> It can handle special cases with keepi

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Amit Kapila
On Monday, January 07, 2013 7:15 PM Andres Freund wrote: > On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: > > On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: > > > On 7 January 2013 12:39, Amit Kapila > wrote: > > > > > > > So We can modify to change this in function LogStandbySnapshot as

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Andres Freund
On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: > On Monday, January 07, 2013 7:15 PM Andres Freund wrote: > > On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: > > > On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: > > > > On 7 January 2013 12:39, Amit Kapila > > wrote: > > > > > > > > > So

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Claudio Freire
On Tue, Jan 8, 2013 at 10:20 AM, Robert Haas wrote: > On Tue, Jan 8, 2013 at 4:04 AM, Takeshi Yamamuro > wrote: >> Apart from my patch, what I care is that the current one might >> be much slow against I/O. For example, when compressing >> and writing large values, compressing data (20-40MiB/s) m

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Claudio Freire
On Tue, Jan 8, 2013 at 11:39 AM, Merlin Moncure wrote: > Reference: > http://postgresql.1045698.n5.nabble.com/Simple-join-doesn-t-use-index-td5738689.html > > This is a pretty common gotcha: user sets shared_buffers but misses > the esoteric but important effective_cache_size. ISTM > effective_c

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 01:45 AM, james wrote: The processing functions have been extended to provide populate_record() and populate_recordset() functions.The latter in particular could be useful in decomposing a piece of json representing an array of flat objects (a fairly common pattern) into a set o

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Amit Kapila
On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: > On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: > > On Monday, January 07, 2013 7:15 PM Andres Freund wrote: > > > On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: > > > > On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: > > > > > O

Re: [HACKERS] Re: Proposal: Store "timestamptz" of database creation on "pg_database"

2013-01-08 Thread Robert Haas
On Sat, Jan 5, 2013 at 11:04 AM, Stephen Frost wrote: > * Fabrízio de Royes Mello (fabriziome...@gmail.com) wrote: >> * also we discuss about create two new catalogs, one local and another >> shared (like pg_description and pg_shdescription) to track creation times >> of all database objects. > >

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Andres Freund
On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: > On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: > > On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: > > > On Monday, January 07, 2013 7:15 PM Andres Freund wrote: > > > > On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: > > > > > On Mond

[HACKERS] Weird Assert failure in GetLockStatusData()

2013-01-08 Thread Tom Lane
This is a bit disturbing: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bushpig&dt=2013-01-07%2019%3A15%3A02 The key bit is [50eb2156.651e:6] LOG: execute isolationtester_waiting: SELECT 1 FROM pg_locks holder, pg_locks waiter WHERE NOT waiter.granted AND waiter.pid = $1 AND holder.gr

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Daniele Varrazzo
On Tue, Jan 8, 2013 at 9:32 AM, Magnus Hagander wrote: > On Tue, Jan 8, 2013 at 3:58 AM, Peter Eisentraut wrote: >> For debugging PL/Python functions, I'm often tempted to write something >> like >> >> rv = plpy.execute(...) >> plpy.info(rv) >> >> which prints something unhelpful like >> >> >> >

[HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Tom Lane
In a tree in which I previously ran "make check" in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master # Untracked files: # (use "git add ..." to include in what will be committed) # # contrib/pg_upgrade/pg_upgrade_dump_1.log # contrib/pg_upgrade/pg_upgrade_dump_

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Josh Berkus
On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas wrote: I'm sure it's possible; I don't *think* it's terribly easy. I'm inclined to agree that this isn't a terribly pressing issue. Certainly, the need to introduce a bunch of new infrastructure to detect this ca

[HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Andres Freund
From: Andres Freund Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader & xlogdump with changes both from Heikki and me. Changes: * windows build support for pg_xlogdump * xlogdump moved to contrib * xlogdump option parsing enhancements *

[HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
From: Andres Freund c.h already had parts of the assert support (StaticAssert*) and its the shared file between postgres.h and postgres_fe.h. This makes it easier to build frontend programs which have to do the hack. --- src/include/c.h | 65

[HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
From: Andres Freund relpathbackend() (via some of its wrappers) is used in *_desc routines which we want to be useable without a backend environment arround. Change signature to return a 'const char *' to make misuse easier to detect. That necessicates also changing the 'FileName' typedef to 'co

[HACKERS] [PATCH 5/5] remove spurious space in running_xact's _desc function

2013-01-08 Thread Andres Freund
From: Andres Freund --- src/backend/access/rmgrdesc/standbydesc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index c38892b..5fb6f54 100644 --- a/src/backend/access/rmgrdesc/standbydesc

[HACKERS] [PATCH 4/5] Add pg_xlogdump contrib module

2013-01-08 Thread Andres Freund
From: Andres Freund Authors: Andres Freund, Heikki Linnakangas --- contrib/Makefile | 1 + contrib/pg_xlogdump/Makefile | 37 +++ contrib/pg_xlogdump/compat.c | 58 contrib/pg_xlogdump/pg_xlogdump.c | 654 ++ contrib/pg_xlog

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Thom Brown
On 8 January 2013 19:09, Andres Freund wrote: > From: Andres Freund > Subject: [PATCH] xlogreader-v4 > In-Reply-To: > > Hi, > > this is the latest and obviously best version of xlogreader & xlogdump with > changes both from Heikki and me. Aren't you forgetting something? -- Thom

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Thom Brown
On 8 January 2013 19:15, Thom Brown wrote: > On 8 January 2013 19:09, Andres Freund wrote: > >> From: Andres Freund >> Subject: [PATCH] xlogreader-v4 >> In-Reply-To: >> >> Hi, >> >> this is the latest and obviously best version of xlogreader & xlogdump >> with >> changes both from Heikki and me

Re: [HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Tom Lane
Andres Freund writes: > From: Andres Freund > c.h already had parts of the assert support (StaticAssert*) and its the shared > file between postgres.h and postgres_fe.h. This makes it easier to build > frontend programs which have to do the hack. This patch seems unnecessary given that we alread

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Andres Freund
On 2013-01-08 20:09:42 +0100, Andres Freund wrote: > From: Andres Freund > Subject: [PATCH] xlogreader-v4 > In-Reply-To: > > Hi, > > this is the latest and obviously best version of xlogreader & xlogdump with > changes both from Heikki and me. > > Changes: > * windows build support for pg_xlog

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund writes: maxpg> From: Andres Freund > relpathbackend() (via some of its wrappers) is used in *_desc routines which > we > want to be useable without a backend environment arround. I'm 100% unimpressed with making relpathbackend return a pointer to a static buffer. Who's to say whe

[HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 14:25:06 -0500, Tom Lane wrote: > Andres Freund writes: > > From: Andres Freund > > c.h already had parts of the assert support (StaticAssert*) and its the > > shared > > file between postgres.h and postgres_fe.h. This makes it easier to build > > frontend programs which have to do

Re: [HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Tom Lane
Andres Freund writes: > On 2013-01-08 14:25:06 -0500, Tom Lane wrote: >> This patch seems unnecessary given that we already put a version of Assert() >> into postgres_fe.h. > The problem is that some (including existing) pieces of code need to > include postgres.h itself, those can't easily inclu

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 14:28:14 -0500, Tom Lane wrote: > Andres Freund writes: > maxpg> From: Andres Freund > > relpathbackend() (via some of its wrappers) is used in *_desc routines > > which we > > want to be useable without a backend environment arround. > > I'm 100% unimpressed with making relpathbac

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Simon Riggs
On 8 January 2013 18:46, Josh Berkus wrote: > On 1/5/13 1:21 PM, Peter Geoghegan wrote: >> >> On 21 December 2012 14:08, Robert Haas wrote: >>> >>> I'm sure it's possible; I don't *think* it's terribly easy. >> >> >> I'm inclined to agree that this isn't a terribly pressing issue. >> Certainly, t

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund writes: > On 2013-01-08 14:28:14 -0500, Tom Lane wrote: >> I'm 100% unimpressed with making relpathbackend return a pointer to a >> static buffer. Who's to say whether that won't create bugs due to >> overlapping usages? > I say it ;). I've gone through all callers and checked. Not

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread David Fetter
On Tue, Jan 08, 2013 at 10:46:12AM -0800, Josh Berkus wrote: > On 1/5/13 1:21 PM, Peter Geoghegan wrote: > >On 21 December 2012 14:08, Robert Haas wrote: > >>I'm sure it's possible; I don't *think* it's terribly easy. > > > >I'm inclined to agree that this isn't a terribly pressing issue. > >Certa

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 14:53:29 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-01-08 14:28:14 -0500, Tom Lane wrote: > >> I'm 100% unimpressed with making relpathbackend return a pointer to a > >> static buffer. Who's to say whether that won't create bugs due to > >> overlapping usages? > > >

Re: [HACKERS] json api WIP patch

2013-01-08 Thread james
I had been wondering how to do such an insertion efficiently in the context of SPI, but it seems that there is no SPI_copy equiv that would allow a query parse and plan to be avoided. Your query above would need to be planned too, although the plan will be trivial. Ah yes, I meant that I h

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 09:58 AM, Andrew Dunstan wrote: If you have such a datum, parsing it involves having it in memory and then taking a copy (I wonder if we could avoid that step - will take a look). Here is a Proof Of Concept patch against my development tip on what's involved in getting the J

[HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 14:35:12 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-01-08 14:25:06 -0500, Tom Lane wrote: > >> This patch seems unnecessary given that we already put a version of > >> Assert() > >> into postgres_fe.h. > > > The problem is that some (including existing) pieces of co

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 03:12 PM, Andrew Dunstan wrote: On 01/08/2013 09:58 AM, Andrew Dunstan wrote: If you have such a datum, parsing it involves having it in memory and then taking a copy (I wonder if we could avoid that step - will take a look). Here is a Proof Of Concept patch against my deve

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 03:07 PM, james wrote: Yes - but I don't think I can use COPY from a stored proc context can I? If I could use binary COPY from a stored proc that has received a binary param and unpacked to the data, it would be handy. You can use COPY from a stored procedure, but only to

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund writes: > Uhm, we don't have & need palloc support and I don't think > relpathbackend() is a good justification for adding it. I've said from the very beginning of this effort that it would be impossible to share any meaningful amount of code between frontend and backend environment

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Simon Riggs
On 8 January 2013 19:53, David Fetter wrote: > On Tue, Jan 08, 2013 at 10:46:12AM -0800, Josh Berkus wrote: >> On 1/5/13 1:21 PM, Peter Geoghegan wrote: >> >On 21 December 2012 14:08, Robert Haas wrote: >> >>I'm sure it's possible; I don't *think* it's terribly easy. >> > >> >I'm inclined to agre

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-01-08 14:35:12 -0500, Tom Lane wrote: > > Andres Freund writes: > > > On 2013-01-08 14:25:06 -0500, Tom Lane wrote: > > >> This patch seems unnecessary given that we already put a version of > > >> Assert() > > >> into postgres_fe.h. > > > > > The problem is that s

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Alvaro Herrera
Tom Lane wrote: > Andres Freund writes: > > Uhm, we don't have & need palloc support and I don't think > > relpathbackend() is a good justification for adding it. > > I've said from the very beginning of this effort that it would be > impossible to share any meaningful amount of code between fron

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 15:27:23 -0500, Tom Lane wrote: > Andres Freund writes: > > Uhm, we don't have & need palloc support and I don't think > > relpathbackend() is a good justification for adding it. > > I've said from the very beginning of this effort that it would be > impossible to share any meaningf

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 17:36:19 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2013-01-08 14:35:12 -0500, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2013-01-08 14:25:06 -0500, Tom Lane wrote: > > > >> This patch seems unnecessary given that we already put a version of > > > >> Asse

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund writes: > To what extent do you want palloc et al. emulation? Provide actual pools > or just make redirect to malloc and provide the required symbols (at the > very least CurrentMemoryContext)? I don't see any need for memory pools, at least not for frontend applications of the curr

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Heikki Linnakangas
On 08.01.2013 22:39, Andres Freund wrote: On 2013-01-08 15:27:23 -0500, Tom Lane wrote: Andres Freund writes: Uhm, we don't have& need palloc support and I don't think relpathbackend() is a good justification for adding it. I've said from the very beginning of this effort that it would be i

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Tom Lane
Alvaro Herrera writes: > Andres Freund wrote: >> Sorry, misremembered the problem somewhat. The problem is that code that >> includes postgres.h atm ends up with ExceptionalCondition() et >> al. declared even if FRONTEND is defined. So if anything uses an assert >> you need to provide wrappers for

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Tomas Vondra
On 8.1.2013 03:47, Shigeru Hanada wrote: >>> * naming of DROP_RELATIONS_BSEARCH_LIMIT (or off-by-one bug?) >>> IIUC bsearch is used when # of relations to be dropped is *more* than >>> the value of DROP_RELATIONS_BSEARCH_LIMIT (10). IMO this behavior is >>> not what the macro name implies; I thoug

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 22:47:43 +0200, Heikki Linnakangas wrote: > On 08.01.2013 22:39, Andres Freund wrote: > >On 2013-01-08 15:27:23 -0500, Tom Lane wrote: > >>Andres Freund writes: > >>>Uhm, we don't have& need palloc support and I don't think > >>>relpathbackend() is a good justification for adding it

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Heikki Linnakangas
On 08.01.2013 23:00, Andres Freund wrote: Note that the xlogreader facility doesn't need any more emulation. I'm quite satisfied with that part of the patch now. However, the rmgr desc routines do, and I'm not very happy with those. Not sure what to do about it. As you said, we could add enough i

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: > In a tree in which I previously ran "make check" in contrib/pg_upgrade: > > $ make -s distclean > $ git status > # On branch master > # Untracked files: > # (use "git add ..." to include in what will be committed) > # > # contrib/

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Tom Lane
Bruce Momjian writes: > On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: >> In a tree in which I previously ran "make check" in contrib/pg_upgrade: >> >> $ make -s distclean >> $ git status >> # On branch master >> # Untracked files: >> # (use "git add ..." to include in what will be c

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 15:45:07 -0500, Tom Lane wrote: > Andres Freund writes: > > To what extent do you want palloc et al. emulation? Provide actual pools > > or just make redirect to malloc and provide the required symbols (at the > > very least CurrentMemoryContext)? > > I don't see any need for memory

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Peter Eisentraut
On 1/8/13 4:04 PM, Bruce Momjian wrote: > On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: >> In a tree in which I previously ran "make check" in contrib/pg_upgrade: >> >> $ make -s distclean >> $ git status >> # On branch master >> # Untracked files: >> # (use "git add ..." to include i

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 04:08:42PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: > >> In a tree in which I previously ran "make check" in contrib/pg_upgrade: > >> > >> $ make -s distclean > >> $ git status > >> # On branch master > >>

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 04:11:41PM -0500, Peter Eisentraut wrote: > On 1/8/13 4:04 PM, Bruce Momjian wrote: > > On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: > >> In a tree in which I previously ran "make check" in contrib/pg_upgrade: > >> > >> $ make -s distclean > >> $ git status > >>

[HACKERS] Re: [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 23:02:15 +0200, Heikki Linnakangas wrote: > On 08.01.2013 23:00, Andres Freund wrote: > >>Note that the xlogreader facility doesn't need any more emulation. I'm quite > >>satisfied with that part of the patch now. However, the rmgr desc routines > >>do, and I'm not very happy with tho

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Peter Eisentraut
On 1/7/13 5:15 PM, Andrew Dunstan wrote: > You (Merlin) have kindly volunteered to work on documentation, so before > we go too far with that any bikeshedding on names, or on the > functionality being provided, should now take place. Hmm, I was going to say, this patch contains no documentation, s

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Peter Eisentraut
On 1/8/13 4:32 AM, Magnus Hagander wrote: > How does it work if there are many rows in there? Say the result > contains 10,000 rows - will the string contain all of them? If so, > might it be worthwhile to cap the number of rows shown and then follow > with a "..." or something? I don't think so.

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Magnus Hagander
On Tue, Jan 8, 2013 at 10:23 PM, Peter Eisentraut wrote: > On 1/8/13 4:32 AM, Magnus Hagander wrote: >> How does it work if there are many rows in there? Say the result >> contains 10,000 rows - will the string contain all of them? If so, >> might it be worthwhile to cap the number of rows shown a

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Peter Eisentraut
On 1/8/13 11:55 AM, Daniele Varrazzo wrote: >>> >> 'bar': '22'}]> > This looks more a repr-style format to me (if you implement repr but > not str, the latter will default to the former). The repr style was the only guideline I found. There is no guideline for how str should look like when it's n

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Alvaro Herrera
Tomas Vondra wrote: > On 8.1.2013 03:47, Shigeru Hanada wrote: > >>> * +1 for Alvaro's suggestion about avoiding palloc traffic by starting > >>> with 8 elements or so. > >> > >> Done. > > > > Not yet. Initial size of srels array is still 1 element. > > I've checked the patch and I see there 'm

Re: [HACKERS] json api WIP patch

2013-01-08 Thread james
You can use COPY from a stored procedure, but only to and from files. I think that's in the chocolate fireguard realm though as far as efficiency for this sort of scenario goes, even if its handled by retaining an mmap'd file as workspace. If SPI provided a way to perform a copy to a temp

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Merlin Moncure
On Tue, Jan 8, 2013 at 3:19 PM, Peter Eisentraut wrote: > On 1/7/13 5:15 PM, Andrew Dunstan wrote: >> You (Merlin) have kindly volunteered to work on documentation, so before >> we go too far with that any bikeshedding on names, or on the >> functionality being provided, should now take place. > >

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Tomas Vondra
On 8.1.2013 22:30, Alvaro Herrera wrote: > Tomas Vondra wrote: >> On 8.1.2013 03:47, Shigeru Hanada wrote: > > * +1 for Alvaro's suggestion about avoiding palloc traffic by starting > with 8 elements or so. Done. >>> >>> Not yet. Initial size of srels array is still 1 element. >

Re: [HACKERS] Weird Assert failure in GetLockStatusData()

2013-01-08 Thread Tom Lane
I wrote: > This is a bit disturbing: > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bushpig&dt=2013-01-07%2019%3A15%3A02 > ... > The assertion failure seems to indicate that the number of > LockMethodProcLockHash entries found by hash_seq_search didn't match the > number that had been cou

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 04:32 PM, Merlin Moncure wrote: On Tue, Jan 8, 2013 at 3:19 PM, Peter Eisentraut wrote: On 1/7/13 5:15 PM, Andrew Dunstan wrote: You (Merlin) have kindly volunteered to work on documentation, so before we go too far with that any bikeshedding on names, or on the functionality be

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Daniel Farina
On Tue, Jan 8, 2013 at 11:51 AM, Simon Riggs wrote: > On 8 January 2013 18:46, Josh Berkus wrote: >> On 1/5/13 1:21 PM, Peter Geoghegan wrote: >>> >>> On 21 December 2012 14:08, Robert Haas wrote: I'm sure it's possible; I don't *think* it's terribly easy. >>> >>> >>> I'm inclined to a

Re: [HACKERS] Re: Proposal: Store "timestamptz" of database creation on "pg_database"

2013-01-08 Thread Peter Eisentraut
On 1/5/13 11:04 AM, Stephen Frost wrote: > Creating a separate catalog (or two) every time we want to track XYZ for > all objects is rather overkill... Thinking about this a bit more, and > noting that pg_description/shdescription more-or-less already exist as a > framework for tracking 'something

[HACKERS] Index build temp files

2013-01-08 Thread Stephen Frost
Greetings, We were surprised recently to note that the temp files that are created during a CREATE INDEX don't go into either a temp tablespace set for the user or into the tablespace which the CREATE INDEX specifies. Instead, they go only into base/pgsql_tmp/. This doesn't allow for a

Re: [HACKERS] I s this a bug of spgist index in a heavy write condition?

2013-01-08 Thread Tom Lane
=?gb2312?B?wO66o8H6?= writes: > I am very excited to say that I may have created a test case! I've been running variants of this example for most of the afternoon, and have not seen a failure :-(. So I'm afraid there is some aspect of your situation that you've not provided enough information to

Re: [HACKERS] Re: Proposal: Store "timestamptz" of database creation on "pg_database"

2013-01-08 Thread Pavel Stehule
2013/1/8 Peter Eisentraut : > On 1/5/13 11:04 AM, Stephen Frost wrote: >> Creating a separate catalog (or two) every time we want to track XYZ for >> all objects is rather overkill... Thinking about this a bit more, and >> noting that pg_description/shdescription more-or-less already exist as a >>

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 9:51 AM, Claudio Freire wrote: > On Tue, Jan 8, 2013 at 10:20 AM, Robert Haas wrote: >> On Tue, Jan 8, 2013 at 4:04 AM, Takeshi Yamamuro >> wrote: >>> Apart from my patch, what I care is that the current one might >>> be much slow against I/O. For example, when compressing

Re: [HACKERS] Index build temp files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 05:09:47PM -0500, Stephen Frost wrote: > Greetings, > > We were surprised recently to note that the temp files that are > created during a CREATE INDEX don't go into either a temp tablespace > set for the user or into the tablespace which the CREATE INDEX > specifie

Re: [HACKERS] Re: Proposal: Store "timestamptz" of database creation on "pg_database"

2013-01-08 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: > 2013/1/8 Peter Eisentraut : > > On 1/5/13 11:04 AM, Stephen Frost wrote: > > Yeah, actually, the other day I was thinking we should get rid of all > > the system catalogs and use a big EAV-like schema instead. We're not > > getting any relational-

Re: [HACKERS] Index build temp files

2013-01-08 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jan 8, 2013 at 05:09:47PM -0500, Stephen Frost wrote: > > Greetings, > > > > We were surprised recently to note that the temp files that are > > created during a CREATE INDEX don't go into either a temp tablespace > > set for the user or i

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 9:53 AM, Claudio Freire wrote: > On Tue, Jan 8, 2013 at 11:39 AM, Merlin Moncure wrote: >> Reference: >> http://postgresql.1045698.n5.nabble.com/Simple-join-doesn-t-use-index-td5738689.html >> >> This is a pretty common gotcha: user sets shared_buffers but misses >> the es

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 3:00 PM, Andres Freund wrote: > Uhm, we don't have & need palloc support and I don't think > relpathbackend() is a good justification for adding it. FWIW, I'm with Tom on this one. Any meaningful code sharing is going to need that, so we might as well bite the bullet. And

Re: [HACKERS] Weird Assert failure in GetLockStatusData()

2013-01-08 Thread Tom Lane
I wrote: > After digging around a bit, I can find only one place where it looks > like somebody might be messing with the LockMethodProcLockHash table > while not holding the appropriate lock-partition LWLock(s): > 1. VirtualXactLock finds target xact holds its VXID lock fast-path. > 2. VirtualXac

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Alvaro Herrera
Robert Haas escribió: > And functions that return static buffers are evil incarnate. I've > spent way too much of my life dealing with the supreme idiocy that is > fmtId(). +1 > If someone ever finds a way to make that go away, I will buy > them a beverage of their choice at the next conference

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 17:28:33 -0500, Robert Haas wrote: > On Tue, Jan 8, 2013 at 3:00 PM, Andres Freund wrote: > > Uhm, we don't have & need palloc support and I don't think > > relpathbackend() is a good justification for adding it. > > FWIW, I'm with Tom on this one. Any meaningful code sharing is goi

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 05:23:36PM -0500, Robert Haas wrote: > > Rather, I'd propose the default setting should be "-1" or something > > "default" and "automagic" that works most of the time (but not all). > > +1. I've found that a value of three-quarters of system memory works > pretty well most

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Robert Haas writes: > And functions that return static buffers are evil incarnate. I've > spent way too much of my life dealing with the supreme idiocy that is > fmtId(). If someone ever finds a way to make that go away, I will buy > them a beverage of their choice at the next conference we're b

Re: [HACKERS] Index build temp files

2013-01-08 Thread Tom Lane
Stephen Frost writes: > * Bruce Momjian (br...@momjian.us) wrote: >> Well, our docs for temp_tablespaces says: >> This variable specifies tablespaces in which to create temporary >> objects (temp tables and indexes on temp tables) when a >> CREATE command does not explicitly specify a tablespace.

Re: [HACKERS] Index build temp files

2013-01-08 Thread Simon Riggs
On 9 January 2013 00:04, Tom Lane wrote: > I don't think it's a bug. What you seem to be proposing is that CREATE > INDEX ought to ignore temp_tablespaces and instead always put its temp > files in the tablespace where the finished index will reside. I don't think that's what he said. -- Sim

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 8, 2013 at 9:53 AM, Claudio Freire wrote: >> Rather, I'd propose the default setting should be "-1" or something >> "default" and "automagic" that works most of the time (but not all). > A cruder heuristic that might be useful is 3 * shared_buffers. Both parts

Re: [HACKERS] Index build temp files

2013-01-08 Thread Tom Lane
Simon Riggs writes: > On 9 January 2013 00:04, Tom Lane wrote: >> I don't think it's a bug. What you seem to be proposing is that CREATE >> INDEX ought to ignore temp_tablespaces and instead always put its temp >> files in the tablespace where the finished index will reside. > I don't think tha

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 6:23 PM, Tom Lane wrote: > Robert Haas writes: >> And functions that return static buffers are evil incarnate. I've >> spent way too much of my life dealing with the supreme idiocy that is >> fmtId(). If someone ever finds a way to make that go away, I will buy >> them a

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 7:17 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 8, 2013 at 9:53 AM, Claudio Freire >> wrote: >>> Rather, I'd propose the default setting should be "-1" or something >>> "default" and "automagic" that works most of the time (but not all). > >> A cruder heuris

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Robert Haas writes: > I was thinking more about a sprintf()-type function that only > understands a handful of escapes, but adds the additional and novel > escapes %I (quote as identifier) and %L (quote as literal). I think > that would allow a great deal of code simplification, and it'd be more

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 7:57 PM, Tom Lane wrote: > Robert Haas writes: >> I was thinking more about a sprintf()-type function that only >> understands a handful of escapes, but adds the additional and novel >> escapes %I (quote as identifier) and %L (quote as literal). I think >> that would allow

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 8, 2013 at 7:17 PM, Tom Lane wrote: >> ... And I don't especially like the idea of trying to >> make it depend directly on the box's physical RAM, for the same >> practical reasons Robert mentioned. > For the record, I don't believe those problems would be part

Re: [HACKERS] pg_dump transaction's read-only mode

2013-01-08 Thread Gurjeet Singh
On Mon, Dec 31, 2012 at 1:38 AM, Pavan Deolasee wrote: > On Sun, Dec 30, 2012 at 12:38 AM, Stephen Frost > wrote: > > * Pavan Deolasee (pavan.deola...@gmail.com) wrote: > >> On Fri, Sep 7, 2012 at 6:06 PM, Kevin Grittner > >> > That makes sense to me. The reason I didn't make that change when I

  1   2   >