Re: [HACKERS] per-column FDW options, v5

2011-08-08 Thread Shigeru Hanada
Sorry, I've missed sending copy to list, so I quoted off-list discussion. On Aug 5, 2011, at 7:59 PM, Shigeru Hanadashigeru.han...@gmail.com wrote: 2011/8/6 Robert Haasrobertmh...@gmail.com: Done. Thanks! Incidentally, I notice that if you do: \d some_foreign_table ...the table-level

[HACKERS] force_not_null option support for file_fdw

2011-08-08 Thread Shigeru Hanada
Hi, I propose to support force_not_null option for file_fdw too. In 9.1 development cycle, file_fdw had been implemented with exported COPY FROM routines, but only force_not_null option has not been  supported yet. Originally, in COPY FROM, force_not_null is specified as a list of column which

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Kohei KaiGai
2011/8/7 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: I'm under implementation of this code according to the suggestion. However, I'm not sure whether it is really portable way (at least, GCC accepts), and whether the interface is simpler than as Robert suggested at

Re: [HACKERS] per-column FDW options, v5

2011-08-08 Thread Shigeru Hanada
(2011/07/29 17:37), Shigeru Hanada wrote: I also attached a rebased version of force_not_null patch, which adds force_not_null option support to file_fdw. This is a use case of per-column FDW option. [just for redirection] Robert has committed only per_column_option patch. So I posted

Re: [HACKERS] plperl crash with Debian 6 (64 bit), pl/perlu, libwww and https

2011-08-08 Thread Alex Hunsaker
On Sun, Aug 7, 2011 at 17:06, Tim Bunce tim.bu...@pobox.com wrote: On Sat, Aug 06, 2011 at 12:37:28PM -0600, Alex Hunsaker wrote: ... Find attached a version that does the equivalent of local %SIG for each pl/perl(u) call. +     gv = gv_fetchpv(SIG, 0, SVt_PVHV); +     save_hash(gv);      

Re: [HACKERS] plperl crash with Debian 6 (64 bit), pl/perlu, libwww and https

2011-08-08 Thread Tim Bunce
On Mon, Aug 08, 2011 at 01:23:08AM -0600, Alex Hunsaker wrote: On Sun, Aug 7, 2011 at 17:06, Tim Bunce tim.bu...@pobox.com wrote: Localizing an individual element of %SIG works fine. In C that's something like this (untested):    hv = gv_fetchpv(SIG, 0, SVt_PVHV);    keysv = ...SV

Re: [HACKERS] Transient plans versus the SPI API

2011-08-08 Thread Anssi Kääriäinen
On 08/07/2011 12:25 PM, Hannu Krosing wrote: On Sun, 2011-08-07 at 11:15 +0200, Hannu Krosing wrote: On Wed, 2011-08-03 at 15:19 -0400, Tom Lane wrote: Hm, you mean reverse-engineering the parameterization of the query? Yes, basically re-generate the query after (or while) parsing, replacing

Re: [HACKERS] WIP: Fast GiST index build

2011-08-08 Thread Heikki Linnakangas
On 07.08.2011 22:28, Alexander Korotkov wrote: There is last version of patch. There is the list of most significant changes in comparison with your version of patch: 1) Reference counting of path items was added. It should helps against possible accumulation of path items. Ok. 2) Neighbor

Re: [HACKERS] Transient plans versus the SPI API

2011-08-08 Thread Hannu Krosing
On Mon, 2011-08-08 at 11:39 +0300, Anssi Kääriäinen wrote: On 08/07/2011 12:25 PM, Hannu Krosing wrote: On Sun, 2011-08-07 at 11:15 +0200, Hannu Krosing wrote: On Wed, 2011-08-03 at 15:19 -0400, Tom Lane wrote: Hm, you mean reverse-engineering the parameterization of the query? Yes,

Re: [HACKERS] WIP: Fast GiST index build

2011-08-08 Thread Alexander Korotkov
On Mon, Aug 8, 2011 at 1:23 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: 2) Neighbor relocation was added. Ok. I think we're going to need some sort of a heuristic on when to enable neighbor relocation. If I remember the performance tests correctly, it improves the

[HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alexander Korotkov
String-formatted relopts was never used before, but I've used it in buffering GiST index build patch and encountered with following compiler warnings: reloptions.c:259: warning: initializer-string for array of chars is too long reloptions.c:259: warning: (near initialization for

Re: [HACKERS] Transient plans versus the SPI API

2011-08-08 Thread Anssi Kääriäinen
On 08/08/2011 01:07 PM, Hannu Krosing wrote: That is why I think it is best done in the main parser - it has to parse and analyse the query anyway and likely knows which constants are arguments to the query As far as I understand the problem, the parsing must transform table references to

Re: [HACKERS] Yes, WaitLatch is vulnerable to weak-memory-ordering bugs

2011-08-08 Thread Robert Haas
On Sun, Aug 7, 2011 at 1:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Any protocol of that sort has *obviously* got a race condition between changes of the latch state and changes of the separate flag state, if run in a weak-memory-ordering machine. At least on the hardware I'm testing, it seems

Re: [HACKERS] vacuumlo patch

2011-08-08 Thread Robert Haas
On Sun, Aug 7, 2011 at 7:53 PM, Tim elatl...@gmail.com wrote: Thanks Josh, I like the ability to bail out on PQTRANS_INERROR, and I think it's a small enough fix to be appropriate to include in this patch. I did consider it before but did not implement it because I am still new to

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Robert Haas
On Sat, Aug 6, 2011 at 1:43 PM, Jeff Janes jeff.ja...@gmail.com wrote: The approach is to move the important things from a LWLock to a spinlock, and to not do any locking for increments to clock-hand increment and numBufferAllocs. That means that some buffers might occasionally get inspected

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Robert Haas
On Sat, Aug 6, 2011 at 2:16 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: It would be nice if the Linux guys would fix this problem for us, but I'm not sure whether they will.  For those who may be curious, the problem is in generic_file_llseek()

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Robert Haas
On Sat, Aug 6, 2011 at 3:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: My experiments have shown that the freelist proper is not substantially faster than the freelist clocksweep--and that is even under the assumption that putting things back into the

[HACKERS] fstat vs. lseek

2011-08-08 Thread Robert Haas
In response to my blog post on lseek contention, someone posted a comment wherein they proposed using fstat() rather than lseek() to get file sizes. http://rhaas.blogspot.com/2011/08/linux-and-glibc-scalability.html I tried that on a RHEL 6.1 machine with 64-cores running

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: In response to my blog post on lseek contention, someone posted a comment wherein they proposed using fstat() rather than lseek() to get file sizes. Patch and test results are attached. Test runs are 5-minute runs with scale factor 100 and

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Andres Freund
On Monday, August 08, 2011 10:30:38 Robert Haas wrote: In response to my blog post on lseek contention, someone posted a comment wherein they proposed using fstat() rather than lseek() to get file sizes. Thoughts? I don't think its a good idea to replace lseek with fstat in the long run. The

Re: [HACKERS] WIP fix proposal for bug #6123

2011-08-08 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Florian Pflug f...@phlo.org wrote: Going down that road opens the door to a *lot* of subtle semantic differences between currently equivalent queries. For example, UPDATE T SET f=f, a=1 would behave differently then UPDATE T SET a=1 because in

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Jesper Krogh
On 2011-08-08 15:29, Robert Haas wrote: On Sat, Aug 6, 2011 at 2:16 PM, Dimitri Fontainedimi...@2ndquadrant.fr wrote: Robert Haasrobertmh...@gmail.com writes: It would be nice if the Linux guys would fix this problem for us, but I'm not sure whether they will. For those who may be curious,

Re: [HACKERS] Yes, WaitLatch is vulnerable to weak-memory-ordering bugs

2011-08-08 Thread Peter Geoghegan
On 8 August 2011 13:47, Robert Haas robertmh...@gmail.com wrote: On the flip side, I'm not sure that anyone ever really expected that a latch could be safely used this way.  Normally, I'd expect the flag to be some piece of state protected by an LWLock, and I think that ought to be OK provided

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 10:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm a bit concerned by the fact that you've only tested this on one operating system, and thus the performance characteristics could be quite different elsewhere.  The comment in mdextend also points out a way in which this

Re: [HACKERS] WIP fix proposal for bug #6123

2011-08-08 Thread Florian Pflug
On Aug8, 2011, at 17:02 , Kevin Grittner wrote: So, we have three options on the table here: (1) We (the Wisconsin Courts) are using a very simple fix to work around the issue so we can move forward with conversion to PostgreSQL triggers. A DELETE is allowed to complete if the BEFORE

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alvaro Herrera
Excerpts from Alexander Korotkov's message of lun ago 08 06:27:33 -0400 2011: String-formatted relopts was never used before, but I've used it in buffering GiST index build patch and encountered with following compiler warnings: reloptions.c:259: warning: initializer-string for array of

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alexander Korotkov
On Mon, Aug 8, 2011 at 7:43 PM, Alvaro Herrera alvhe...@commandprompt.comwrote: Maybe this needs to use the new FLEXIBLE_ARRAY_MEMBER stuff. Can you try that please? typedef struct relopt_string { relopt_gen gen; int default_len; bool default_isnull; validate_string_relopt validate_cb;

Re: [HACKERS] Yes, WaitLatch is vulnerable to weak-memory-ordering bugs

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 11:28 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: Maybe we should try to document the contract here a little better; I think it's just that there must be a full memory barrier (such as LWLockRelease) in both processes involved in the interaction. Or, maybe we

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Alvaro Herrera
Excerpts from Kohei KaiGai's message of lun ago 08 03:12:20 -0400 2011: Thanks for your suggestion. So, it seems to me the interface should return a pointer to the entry of array being specified, rather than above approach. E.g, the above macro could be probably rewritten as follows:

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 11:57 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Kohei KaiGai's message of lun ago 08 03:12:20 -0400 2011: Thanks for your suggestion. So, it seems to me the interface should return a pointer to the entry of array being specified, rather than

Re: [HACKERS] per-column FDW options, v5

2011-08-08 Thread Robert Haas
2011/8/8 Shigeru Hanada shigeru.han...@gmail.com: Currently table-level options are showin in result of \det+ command (only verbose mode), in same style as fdw and foreign servers. But \d is more popular for table describing, so moving table-level options from \det+ to \d might be better.  

Re: [HACKERS] psql document fix about showing FDW options

2011-08-08 Thread Robert Haas
2011/8/8 Shigeru Hanada shigeru.han...@gmail.com: I noticed that psql document wrongly says that \d+ command shows per-table FDW options of a foreign table, but in fact, per-table FDW options are shown only in the result of \det+ command.  Attached patch removes this wrong description. This

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Alvaro Herrera
Excerpts from Robert Haas's message of lun ago 08 12:05:05 -0400 2011: We could do that, but what the heck is the point? What benefit are we trying to get by not returning a pointer to the structure? I feel like we're making this ludicrously complicated with no real justification of why

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alvaro Herrera
Excerpts from Alexander Korotkov's message of lun ago 08 11:50:53 -0400 2011: On Mon, Aug 8, 2011 at 7:43 PM, Alvaro Herrera alvhe...@commandprompt.comwrote: Maybe this needs to use the new FLEXIBLE_ARRAY_MEMBER stuff. Can you try that please? typedef struct relopt_string {

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 12:22 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun ago 08 12:05:05 -0400 2011: We could do that, but what the heck is the point?   What benefit are we trying to get by not returning a pointer to the structure?  I feel

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Alvaro Herrera
Excerpts from Kohei KaiGai's message of lun ago 08 12:18:47 -0400 2011: 2011/8/8 Robert Haas robertmh...@gmail.com: We could do that, but what the heck is the point?   What benefit are we trying to get by not returning a pointer to the structure?  I feel like we're making this ludicrously

Re: [HACKERS] Yes, WaitLatch is vulnerable to weak-memory-ordering bugs

2011-08-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Aug 7, 2011 at 1:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Any protocol of that sort has *obviously* got a race condition between changes of the latch state and changes of the separate flag state, if run in a weak-memory-ordering machine. On

Re: [HACKERS] Yes, WaitLatch is vulnerable to weak-memory-ordering bugs

2011-08-08 Thread Heikki Linnakangas
On 08.08.2011 19:39, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: On the flip side, I'm not sure that anyone ever really expected that a latch could be safely used this way. Normally, I'd expect the flag to be some piece of state protected by an LWLock, and I think that ought to

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Kohei KaiGai
2011/8/8 Robert Haas robertmh...@gmail.com: On Mon, Aug 8, 2011 at 11:57 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Kohei KaiGai's message of lun ago 08 03:12:20 -0400 2011: Thanks for your suggestion. So, it seems to me the interface should return a pointer to the

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Alvaro Herrera
Excerpts from Robert Haas's message of lun ago 08 12:33:45 -0400 2011: On Mon, Aug 8, 2011 at 12:22 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun ago 08 12:05:05 -0400 2011: We could do that, but what the heck is the point?   What benefit are

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Andres Freund
On Monday, August 08, 2011 11:33:29 Robert Haas wrote: On Mon, Aug 8, 2011 at 10:49 AM, Andres Freund and...@anarazel.de wrote: I don't think its a good idea to replace lseek with fstat in the long run. The likelihood that the lockless generic_file_llseek will get included seems rather

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: We could do that, but what the heck is the point? What benefit are we trying to get by not returning a pointer to the structure? Not having an ABI break if we find it necessary to add members to the struct ... which I grant is unlikely to happen in a

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 1:10 PM, Andres Freund and...@anarazel.de wrote: There doesn't seem to have been any activity to inlude it in 3.1. The merge window for 3.1 just ended. The next one will open for about a week after the release. Its also not yet included in linux-next which is a preview

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alexander Korotkov
On Mon, Aug 8, 2011 at 8:27 PM, Alvaro Herrera alvhe...@commandprompt.comwrote: An array of relopt_string? Isn't that a bit strange? If I recall correctly, the point of this was to be able to allocate the relopt_string struct and the char array itself as a single palloc unit, in a single

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 1:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: We could do that, but what the heck is the point?   What benefit are we trying to get by not returning a pointer to the structure? Not having an ABI break if we find it necessary to

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Not really. I do have root access to a 64-core box at the moment, and I could probably get permission to reboot it, but if it didn't come back on-line that would be awkward. Red Hat has some test hardware that I can use (... pokes around ...) Hmm,

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Andres Freund
On Monday, August 08, 2011 13:19:13 Robert Haas wrote: On Mon, Aug 8, 2011 at 1:10 PM, Andres Freund and...@anarazel.de wrote: There doesn't seem to have been any activity to inlude it in 3.1. The merge window for 3.1 just ended. The next one will open for about a week after the release.

Re: [HACKERS] [RFC] Common object property boards

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 12:49 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun ago 08 12:33:45 -0400 2011: On Mon, Aug 8, 2011 at 12:22 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun ago 08 12:05:05

[HACKERS] index sizes: single table vs partitioned

2011-08-08 Thread Andrew Hammond
For a large table, should there be a difference in index sizes between a single table representation and representation based on multiple partitions with identical indexes? A

Re: [HACKERS] fstat vs. lseek

2011-08-08 Thread Robert Haas
On Mon, Aug 8, 2011 at 1:31 PM, Andres Freund and...@anarazel.de wrote: If its ok I will write a mail to lkml referencing this thread and your numbers inline (with attribution obviously). That would be great. Please go ahead. I don't think it will be that hard to convince them. But I

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alvaro Herrera
Excerpts from Alexander Korotkov's message of lun ago 08 13:21:17 -0400 2011: On Mon, Aug 8, 2011 at 8:27 PM, Alvaro Herrera alvhe...@commandprompt.comwrote: An array of relopt_string? Isn't that a bit strange? If I recall correctly, the point of this was to be able to allocate the

Re: [HACKERS] WIP fix proposal for bug #6123

2011-08-08 Thread Florian Pflug
On Aug8, 2011, at 17:35 , Florian Pflug wrote: I think it would be helpful if we had a more precise idea about the intended use-cases. So far, the only use-case that has been described in detail is the one which made Kevin aware of the problem. But if I understood Kevin correctly, that fact

Re: [HACKERS] psql: display of object comments

2011-08-08 Thread Robert Haas
On Fri, Aug 5, 2011 at 7:25 PM, Josh Kupershmidt schmi...@gmail.com wrote: On Fri, Aug 5, 2011 at 8:32 AM, Robert Haas robertmh...@gmail.com wrote: I guess my vote is to make the SQL/MED stuff show the description only in verbose mode, and always at the end; and revise what we did with \dL to

Re: [HACKERS] psql: display of object comments

2011-08-08 Thread Josh Kupershmidt
On Mon, Aug 8, 2011 at 4:34 PM, Robert Haas robertmh...@gmail.com wrote: OK, I've now committed most of this, with some additions to the documentation.  Remaining bits attached. Looks good, thanks. I am a bit confused as to why we have both \det and \dE.  They seem redundant.  Shouldn't we

Re: [HACKERS] psql: display of object comments

2011-08-08 Thread Josh Kupershmidt
On Mon, Aug 8, 2011 at 6:01 PM, Josh Kupershmidt schmi...@gmail.com wrote: (i.e. add Options column to \dE+ if we keep that one). Oh nevermind, Options is displayed by \d+ foreign_table_name. Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Selecting user-defined CASTs

2011-08-08 Thread Joe Abbate
Hi, I'm trying to query the catalogs to select only the user-defined CASTs (my test db only has one such CAST). Looking at pg_dump.c, I've come up with the following so far: SELECT castsource::regtype AS source, casttarget::regtype AS target, castfunc::regprocedure AS

Re: [HACKERS] Selecting user-defined CASTs

2011-08-08 Thread Joe Abbate
On 08/08/2011 06:31 PM, Joe Abbate wrote: It seems the only way out is to do something like a 9-way join between pg_cast, pg_type, pg_proc and pg_namespace to test the source, target and function namespaces much as dumpCast() does in pg_dump.c. Before I go that route, I'd thought I'd check

Re: [HACKERS] Selecting user-defined CASTs

2011-08-08 Thread Tom Lane
Joe Abbate j...@freedomcircle.com writes: I'm trying to query the catalogs to select only the user-defined CASTs This is rather difficult to do, actually, because pg_cast stores neither an owner nor a schema for casts, which eliminates all of the principled ways in which you might decide that a