Re: [HACKERS] Itanium HP-UX build failure, register stack

2011-04-14 Thread Josh Berkus
On 4/13/11 5:46 AM, Heikki Linnakangas wrote: On 13.04.2011 14:22, Andrew Dunstan wrote: I wish we could get some buildfarm coverage for HPUX. I've whined about this in the past, but nobody's ever made an offer to provide suitable platform(s) that I know of. I only have temporary access to

[HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Greg Smith
This week several list regulars here waded into the MySQL Convention. I decided to revisit PostgreSQL vs. MySQL performance using the sysbench program as part of that. It's not important to what I'm going to describe to understand exactly what statements sysbench runs here or how to use it,

[HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Pavel Stehule
Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) { snprintf(buffer, sizeof(buffer), %s%s\n, row[0], row[1]); copy_result =

Re: [HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Pavel Stehule
Hello The way COPY works is that PQputCopyData just sends the data to the server, and the server will buffer it in its internal buffer and processes it when it feels like it. The PQputCopyData() calls don't even need to match line boundaries. Yes, it is current behave - then documentation

Re: [HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Heikki Linnakangas
On 14.04.2011 10:15, Pavel Stehule wrote: Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) { snprintf(buffer, sizeof(buffer), %s%s\n, row[0], row[1]);

Re: [HACKERS] Itanium HP-UX build failure, register stack

2011-04-14 Thread Andrew Dunstan
On 04/14/2011 02:04 AM, Josh Berkus wrote: On 4/13/11 5:46 AM, Heikki Linnakangas wrote: On 13.04.2011 14:22, Andrew Dunstan wrote: I wish we could get some buildfarm coverage for HPUX. I've whined about this in the past, but nobody's ever made an offer to provide suitable platform(s) that I

Re: [HACKERS] Typed table DDL loose ends

2011-04-14 Thread Noah Misch
On Wed, Apr 13, 2011 at 07:57:29PM -0700, Robert Haas wrote: On Sat, Apr 9, 2011 at 6:57 PM, Noah Misch n...@leadboat.com wrote: While looking at the typed table/pg_upgrade problem, I ran into a few smaller problems in the area. ?I'm not envisioning a need for much code shift to fix

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Florian Weimer
* Tom Lane: Well, the fundamental point is that ignoring NFS is not the real world. We can't tell people not to put data directories on NFS, and even if we did tell them not to, they'd still do it. And NFS locking is not trustworthy, because the remote lock daemon can crash and restart

Re: [HACKERS] pg_dump --binary-upgrade vs. ALTER TYPE ... DROP ATTRIBUTE

2011-04-14 Thread Noah Misch
On Wed, Apr 13, 2011 at 11:46:45PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: If we adopt the elsewhere-proposed approach of forbidding the use of rowtypes to create typed tables, the circularity-checking logic here can become simpler. I think it's not actually

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 13, 2011, at 9:30 PM, Robert Haas wrote: On Wed, Apr 13, 2011 at 6:11 PM, A.M. age...@themactionfaction.com wrote: I don't see why we need to get rid of SysV shared memory; needing less of it seems just as good. 1. As long one keeps SysV shared memory around, the postgresql project

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 14, 2011, at 8:22 AM, Florian Weimer wrote: * Tom Lane: Well, the fundamental point is that ignoring NFS is not the real world. We can't tell people not to put data directories on NFS, and even if we did tell them not to, they'd still do it. And NFS locking is not trustworthy,

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: samples %image name symbol name 53548 6.7609 postgres AllocSetAlloc 32787 4.1396 postgres MemoryContextAllocZeroAligned 26330 3.3244 postgres base_yyparse 21723

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 13, 2011, at 11:37 PM, Tom Lane wrote: A.M. age...@themactionfaction.com writes: 1. As long one keeps SysV shared memory around, the postgresql project has to maintain the annoying platform-specific document on how to configure the poorly named kernel parameters. No, if it's just a

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Martijn van Oosterhout
On Thu, Apr 14, 2011 at 10:26:33AM -0400, A.M. wrote: 1) the SysV nattch method's primary purpose is to protect the shmem region. This is no longer necessary in my patch because the shared memory in unlinked immediately after creation, so only the initial postmaster and its children have

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread David Fetter
On Thu, Apr 14, 2011 at 10:43:16AM -0400, Tom Lane wrote: I doubt that it's possible to make AllocSetAlloc radically cheaper. I think the more likely route to improvement there is going to be to find a way to do fewer pallocs. For instance, if we had more rigorous rules about which data

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 7:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: I doubt that it's possible to make AllocSetAlloc radically cheaper. I think the more likely route to improvement there is going to be to find a way to do fewer pallocs.  For instance, if we had more rigorous rules about which

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 7:26 AM, A.M. age...@themactionfaction.com wrote: From what I understood, the primary purpose of the SysV check was to protect the shared memory from multiple stompers. The interlock was a neat side-effect. Not really - the purpose of the interlock is to protect the

Re: [HACKERS] Typed table DDL loose ends

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 5:18 AM, Noah Misch n...@leadboat.com wrote: I guess my gut feeling is that it would make more sense to forbid it outright for 9.1, and we can look at relaxing that restriction later if we're so inclined. Much as with the problem Tom fixed in commit

[HACKERS] small bug in recoveryStopsHere()

2011-04-14 Thread Robert Haas
I discovered while fooling around the other night that the named restore point patch introduced a small bug into recoveryStopsHere(): the test at the top of the function now lets through two resource-manager IDs rather than one, but the remainder of the function tests only the record_info flag and

Re: [DOCS] [HACKERS] Uppercase SGML entity declarations

2011-04-14 Thread Robert Haas
On Tue, Apr 5, 2011 at 6:55 PM, Josh Kupershmidt schmi...@gmail.com wrote: On Mon, Apr 4, 2011 at 3:02 PM, Robert Haas robertmh...@gmail.com wrote:  In theory, we have documentation that explains this: http://www.postgresql.org/docs/current/static/docguide-toolsets.html While we're on the

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: rhaas=# CREATE OR REPLACE FUNCTION developer_lookup(id integer) RETURNS text AS $$SELECT name FROM developer WHERE id = $1$$ LANGUAGE sql STABLE; Now, when this person attempts to recreate this function on a hypothetical version of PostgreSQL that

Re: [HACKERS] Foreign table permissions and cloning

2011-04-14 Thread Robert Haas
On Fri, Apr 1, 2011 at 5:13 AM, Thom Brown t...@linux.com wrote: On 1 April 2011 12:57, Shigeru HANADA han...@metrosystems.co.jp wrote: NOT NULL constraint on foreign table is just declaration and can't force data integrity.  And I noticed that CREATE FOREIGN TABLE document doesn't mention

Re: [HACKERS] Foreign table permissions and cloning

2011-04-14 Thread Robert Haas
On Fri, Apr 1, 2011 at 1:29 AM, Shigeru HANADA han...@metrosystems.co.jp wrote: In addition to the 2nd GRANT above, GRANT SELECT (colour) ON stuff TO user_a (omitting TABLE) will succeed too because parser assumes that the target object is a regular table if object type was TABLE or omitted.

Re: [HACKERS] Proposal for GSoC : ADJ dashboard (Administration related software)

2011-04-14 Thread Robert Haas
On Wed, Apr 6, 2011 at 11:06 AM, Erdinc Akkaya hz.r...@gmail.com wrote: Project Title: ADJ Dashboard Name : Erdinc AKKAYA Email: erdinc.akk...@gmail.com Synopsis AnyDBJSP is a database monitoring and reporting solution with a browser based interface. ADJ dashboard mainly will be written

Re: [HACKERS] proposal: a validator for configuration files

2011-04-14 Thread Robert Haas
On Mon, Apr 4, 2011 at 2:03 PM, Alexey Klyukin al...@commandprompt.com wrote: Here's the update of Selena's patch, which also shows all errors in configuration parameters (as well as parser errors) during reload. You should add this here:

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 11:37 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: rhaas=# CREATE OR REPLACE FUNCTION developer_lookup(id integer) RETURNS text AS $$SELECT name FROM developer WHERE id = $1$$ LANGUAGE sql STABLE; Now, when this person

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Heikki Linnakangas
On 14.04.2011 17:43, Tom Lane wrote: Greg Smithg...@2ndquadrant.com writes: samples %image name symbol name 53548 6.7609 postgres AllocSetAlloc 32787 4.1396 postgres MemoryContextAllocZeroAligned 26330 3.3244 postgres

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Noah Misch
On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: It's fairly far down in the noise on this particular profile, but in the low-hanging fruit department, I think we should fix ScanKeywordLookup to use a smarter algorithm that is more like O(1) rather than O(lg n) in the number of

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: There's one very low-hanging fruit here, though. I profiled the pgbench case, with -M prepared, and found that like in Greg Smith's profile, hash_seq_search pops up quite high in the list. Those calls are coming from

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Andres Freund
Hi, On Thursday 14 April 2011 16:43:16 Tom Lane wrote: I doubt that it's possible to make AllocSetAlloc radically cheaper. I actually doubt your doubt. I think you could add some SLAB like interface for common allocation sizes making it significantly faster for some uses (because

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue abr 14 17:08:34 -0300 2011: Hi, On Thursday 14 April 2011 16:43:16 Tom Lane wrote: I doubt that it's possible to make AllocSetAlloc radically cheaper. I actually doubt your doubt. I think you could add some SLAB like interface for common

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Greg Smith
Heikki Linnakangas wrote: In this case you could just use prepared statements and get rid of all the parser related overhead, which includes much of the allocations. Trying that gives me around 9200 TPS instead of 5500 on my laptop, so a pretty big gain here. Will have to include that in my

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-14 Thread jagan
Hi, Where is the OID of tuple stored in a WAL record of a tuple? If not with xl_heap_header, where is it stored? Is it stored at all? It's stored in the tuple data portion. I see it now. I was having alignment issues, which I resolved. Thanks for your help. I am still not sure if I

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Merlin Moncure
On Thu, Apr 14, 2011 at 2:08 AM, Greg Smith g...@2ndquadrant.com wrote: This week several list regulars here waded into the MySQL Convention.  I decided to revisit PostgreSQL vs. MySQL performance using the sysbench program as part of that.  It's not important to what I'm going to describe to

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Andres Freund
On Thursday 14 April 2011 22:21:26 Alvaro Herrera wrote: Excerpts from Andres Freund's message of jue abr 14 17:08:34 -0300 2011: Hi, On Thursday 14 April 2011 16:43:16 Tom Lane wrote: I doubt that it's possible to make AllocSetAlloc radically cheaper. I actually doubt your doubt.

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Andres Freund and...@anarazel.de writes: Where I am with you is that its quite possible that it will not make sense (performancewise) for all contexts. Which is quite annoying. The mcxt stuff was designed from day one to support multiple types of contexts, so it wouldn't be very hard at all to

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Apr 14, 2011 at 11:37 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Unless we make it so that no such version ever exists.  Meaning that the code works fine as is or using WHERE id = developer_lookup.id.  AS id can't ever be the parameter

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 12:45 PM, Noah Misch n...@leadboat.com wrote: It shouldn't be terribly difficult to come up with some kind of hash function based on, say, the first two characters of the keyword that would be a lot faster than what we're doing now. I'd look at `gperf', which

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 1:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Apr 14, 2011 at 11:37 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Unless we make it so that no such version ever exists.  Meaning that the code works fine as is or

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: It shouldn't be terribly difficult to come up with some kind of hash function based on, say, the first two characters of the keyword that would be a lot faster than what we're doing now. I'd

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Merlin Moncure
On Thu, Apr 14, 2011 at 4:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Noah Misch n...@leadboat.com writes: On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: It shouldn't be terribly difficult to come up with some kind of hash function based on, say, the first two characters of the

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Apr 14, 2011 at 1:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm, what I read Dimitri to be proposing is that we *require* parameter names to be qualified with the function name.  I don't recall hearing that before.  It would solve the problem

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So far the most promising proposal I've seen seems to be to let id mean the parameter called id only when it can't refer to anything in the query. Yeah, I've come round to that position too. I think allowing

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Andres Freund
On Thursday 14 April 2011 23:10:41 Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: It shouldn't be terribly difficult to come up with some kind of hash function based on, say, the first two characters of the keyword that

Re: [HACKERS] Typed table DDL loose ends

2011-04-14 Thread Noah Misch
On Thu, Apr 14, 2011 at 11:23:49AM -0700, Robert Haas wrote: On Thu, Apr 14, 2011 at 5:18 AM, Noah Misch n...@leadboat.com wrote: I guess my gut feeling is that it would make more sense to forbid it outright for 9.1, and we can look at relaxing that restriction later if we're so inclined.

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Jim Nasby
On Apr 14, 2011, at 4:20 PM, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So far the most promising proposal I've seen seems to be to let id mean the parameter called id only when it can't refer to anything in the query. Yeah, I've

Re: [HACKERS] Proposal for GSoC : ADJ dashboard (Administration related software)

2011-04-14 Thread Jim Nasby
On Apr 14, 2011, at 1:45 PM, Robert Haas wrote: On Wed, Apr 6, 2011 at 11:06 AM, Erdinc Akkaya hz.r...@gmail.com wrote: Project Title: ADJ Dashboard Name : Erdinc AKKAYA Email: erdinc.akk...@gmail.com Synopsis AnyDBJSP is a database monitoring and reporting solution with a browser based

Re: [HACKERS] using a lot of maintenance_work_mem

2011-04-14 Thread Jim Nasby
On Apr 9, 2011, at 9:23 PM, Stephen Frost wrote: Actually, Tom has a point in that work_mem can be set above 1GB (which is where I had it set previously..). I didn't think it'd actually do anything given the MaxAlloc limit, but suprisingly, it does (at least, under 8.4). I'm currently trying

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Pavel Stehule
2011/4/15 Jim Nasby j...@nasby.net: On Apr 14, 2011, at 4:20 PM, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So far the most promising proposal I've seen seems to be to let id mean the parameter called id only when it can't refer to

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
David Fetter da...@fetter.org writes: On Thu, Apr 14, 2011 at 10:43:16AM -0400, Tom Lane wrote: ... I think a lot of this ultimately traces to the extensible, data-type-agnostic design philosophy. The fact that we don't know what an integer is until we look in pg_type, and don't know what an

Re: [HACKERS] Foreign table permissions and cloning

2011-04-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Apr 1, 2011 at 5:13 AM, Thom Brown t...@linux.com wrote: On 1 April 2011 12:57, Shigeru HANADA han...@metrosystems.co.jp wrote: NOT NULL constraint on foreign table is just declaration and can't force data integrity. And I noticed that CREATE

Re: [HACKERS] small bug in recoveryStopsHere()

2011-04-14 Thread Jaime Casanova
On Thu, Apr 14, 2011 at 1:30 PM, Robert Haas robertmh...@gmail.com wrote: I discovered while fooling around the other night that the named restore point patch introduced a small bug into recoveryStopsHere(): the test at the top of the function now lets through two resource-manager IDs rather