Re: [HACKERS] Get more from indices.

2013-12-05 Thread Kyotaro HORIGUCHI
Hello, I've modified the patch to work in such a way. Also, as ISTM the patch is more complicated than what the patch really does, I've simplified the patch. I've revised the patch a bit. Please find attached the patch. Thank you, but it seems to me too simplified. You made two major

[HACKERS] Regression tests failing if not launched on db regression

2013-12-05 Thread Michael Paquier
Hi all, It happens that the following regression tests are failing if they are run on a database not named regression: - updatable_views - foreign_data - sequence Those tests are failing because some relations of information_schemas contain information that are database-dependent. Please see the

Re: [HACKERS] Regression tests failing if not launched on db regression

2013-12-05 Thread Michael Paquier
On Thu, Dec 5, 2013 at 5:12 PM, Michael Paquier michael.paqu...@gmail.com wrote: Those tests are failing because some relations of information_schemas contain information that are database-dependent. I forgot to mention that it is our QE team at VMware that reported me a portion of this failure,

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread KONDO Mitsumasa
(2013/12/04 16:39), Claudio Freire wrote: On Wed, Dec 4, 2013 at 4:28 AM, Tatsuo Ishii is...@postgresql.org wrote: Can we avoid the Linux kernel problem by simply increasing our shared buffer size, say up to 80% of memory? It will be swap more easier. Is that the case? If the system has not

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 11:07 AM, Josh Berkus j...@agliodbs.com wrote: I also wasn't exaggerating the reception I got when I tried to talk about IO and PostgreSQL at LinuxCon and other events. The majority of Linux hackers I've talked to simply don't want to be bothered with PostgreSQL's

Re: [HACKERS] Time-Delayed Standbys

2013-12-05 Thread Magnus Hagander
On Thu, Dec 5, 2013 at 1:45 AM, Simon Riggs si...@2ndquadrant.com wrote: On 3 December 2013 18:46, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
- When we increased NUM_BUFFER_PARTITIONS to 1024, this problem is disappeared for 8 core machines and come back with 16 core machines on Amazon EC2. Would it be related with PostgreSQL locking mechanism? If we build with -DLWLOCK_STATS to print locking stats from PostgreSQL, we see tons of

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Andres Freund
On 2013-12-05 11:15:20 +0200, Metin Doslu wrote: - When we increased NUM_BUFFER_PARTITIONS to 1024, this problem is disappeared for 8 core machines and come back with 16 core machines on Amazon EC2. Would it be related with PostgreSQL locking mechanism? If we build with -DLWLOCK_STATS to

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
Is your workload bigger than RAM? RAM is bigger than workload (more than a couple of times). I think a good bit of the contention you're seeing in that listing is populating shared_buffers - and might actually vanish once you're halfway cached. From what I've seen so far the bigger problem

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Andres Freund
On 2013-12-05 11:33:29 +0200, Metin Doslu wrote: Is your workload bigger than RAM? RAM is bigger than workload (more than a couple of times). I think a good bit of the contention you're seeing in that listing is populating shared_buffers - and might actually vanish once you're halfway

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
Hi Robert, On 2013-12-04 23:32:27 -0500, Robert Haas wrote: But I'm also learning painfully that this kind of thing only goes so far. For example, I spent some time looking at what it would take to provide a dynamic shared memory equivalent of palloc/pfree, a facility that I feel fairly sure

Re: [HACKERS] Time-Delayed Standbys

2013-12-05 Thread Simon Riggs
On 5 December 2013 08:51, Magnus Hagander mag...@hagander.net wrote: Not recalling the older thread, but it seems the breaks on clock drift, I think we can fairly easily make that situation good enough. Just have IDENTIFY_SYSTEM return the current timestamp on the master, and refuse to start

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-12-05 Thread Pavel Stehule
Hello 2013/12/5 Peter Eisentraut pete...@gmx.net Can someone in this thread clarify the commit fest situation? I see two entries that appear to be the same: https://commitfest.postgresql.org/action/patch_view?id=1174 https://commitfest.postgresql.org/action/patch_view?id=1175 I think

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-12-05 Thread Pavel Stehule
2013/12/5 Dean Rasheed dean.a.rash...@gmail.com On 5 December 2013 01:33, Peter Eisentraut pete...@gmx.net wrote: Can someone in this thread clarify the commit fest situation? I see two entries that appear to be the same: https://commitfest.postgresql.org/action/patch_view?id=1174

Re: [HACKERS] Changes in Trigger Firing

2013-12-05 Thread Sameer Kumar
I read somewhere that the best editor is the one you master (1) :) 1: http://www.postgresql.org/message-id/m2wrs6giyp@hi-media.com Thanks, I am using eclipse now. Any comments about the utility of this feature? Or is it just me who thinks this can be useful? I think users/developers

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 5:28 PM, Peter Geoghegan p...@heroku.com wrote: I'm also curious about the impact on insertion into primary key indexes. Presently, we hold an exclusive buffer lock for the duration of a couple of operations when checkUnique != UNIQUE_CHECK_NO. _bt_binsrch() is one such

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Andres Freund
On 2013-12-04 20:25:53 -0500, Tom Lane wrote: Tatsuo Ishii is...@sraoss.co.jp writes: I would like to add a variant of regclass, which is exactly same as current regclass except it does not raise an error when the target table is not found. Instead it returns InvalidOid (0). I've

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Pavel Stehule
2013/12/5 Andres Freund and...@2ndquadrant.com On 2013-12-04 20:25:53 -0500, Tom Lane wrote: Tatsuo Ishii is...@sraoss.co.jp writes: I would like to add a variant of regclass, which is exactly same as current regclass except it does not raise an error when the target table is not

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Andres Freund
On 2013-12-05 11:54:20 +0100, Pavel Stehule wrote: 2013/12/5 Andres Freund and...@2ndquadrant.com We can introduce some assert polymorphic function CREATE OR REPLACE FUNCTION notnull(any, message text) RETURNS any, that can be used for check inside SQL Uh. How is that going to help

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Simon Riggs
On 5 December 2013 01:55, Peter Eisentraut pete...@gmx.net wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it doesn't, those connection parameters will

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-04 20:55:08 -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it doesn't, those connection parameters will be

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Pavel Stehule
2013/12/5 Andres Freund and...@2ndquadrant.com On 2013-12-05 11:54:20 +0100, Pavel Stehule wrote: 2013/12/5 Andres Freund and...@2ndquadrant.com We can introduce some assert polymorphic function CREATE OR REPLACE FUNCTION notnull(any, message text) RETURNS any, that can be used for

Re: [HACKERS] better atomics - v0.2

2013-12-05 Thread Andres Freund
On 2013-11-19 10:37:35 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: The only animal we have that doesn't support quiet inlines today is HP-UX/ac++, and I think - as in patch 1 in the series - we might be able to simply suppress the warning there. Or just not worry

Re: [HACKERS] Get more from indices.

2013-12-05 Thread Etsuro Fujita
Kyotaro HORIGUCHI wrote: Thank you, but it seems to me too simplified. You made two major functional changes. Thank you for the comments! One is, you put the added code for getrelation_info() out of the block for the condition (info-relam == BTREE_AM_OID) (though amcanorder would be

Re: [HACKERS] Get more from indices.

2013-12-05 Thread Etsuro Fujita
I wrote: Kyotaro HORIGUCHI wrote: Another is, you changed pathkeys expantion to be all-or-nothing decision. While this change should simplify the code slightly, it also dismisses the oppotunity for partially-extended pathkeys. Could you let me know the reason why you did so. At first

Re: [HACKERS] [bug fix] pg_ctl stop times out when it should respond quickly

2013-12-05 Thread MauMau
From: Tom Lane t...@sss.pgh.pa.us If you're going to do a postmaster_is_alive check, why bother with repeated get_pgpid()? As I said yesterday, I removed get_pgpid() calls. I'll add this patch to 2014-1 commitfest this weekend if it is not committed until then. Regards MauMau

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-04 18:48:44 -0500, Robert Haas wrote: * When a type narrower than Datum is stored in a Datum, we place it in the * low-order bits and are careful that the DatumGetXXX macro for it discards * the unused high-order bits (as opposed to, say, assuming they are zero). * This is

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Pavel Golub
Hello, Andres. You wrote: AF On 2013-12-04 20:25:53 -0500, Tom Lane wrote: Tatsuo Ishii is...@sraoss.co.jp writes: I would like to add a variant of regclass, which is exactly same as current regclass except it does not raise an error when the target table is not found. Instead it returns

Re: [HACKERS] tracking commit timestamps

2013-12-05 Thread Andres Freund
On 2013-12-02 02:39:55 -0500, Jaime Casanova wrote: === performance === i expected a regression on performance with the module turned on because of the new XLOG records and wrote of files in pg_committs but the performance drop is excessive. Master 437.835674 tps Patch, guc off

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 4:56 AM, Andres Freund and...@2ndquadrant.com wrote: Hi Robert, On 2013-12-04 23:32:27 -0500, Robert Haas wrote: But I'm also learning painfully that this kind of thing only goes so far. For example, I spent some time looking at what it would take to provide a dynamic

Re: [HACKERS] [bug fix] pg_ctl fails with config-only directory

2013-12-05 Thread MauMau
From: Amit Kapila amit.kapil...@gmail.com On Wed, Dec 4, 2013 at 7:57 PM, MauMau maumau...@gmail.com wrote: * Approach 1 When postgres starts, it removes Administrator privileges from its own process. But is this possible at all? Windows security API is complex and provides many functions.

Re: [HACKERS] logical changeset generation v6.7

2013-12-05 Thread Kyotaro HORIGUCHI
Hello, Will send the rebased version as soon as I've addressed your comments. Thank you. = 0001: - You assined HeapTupleGetOid(tuple) into relid to read in several points but no modification. Nevertheless, you left HeapTupleGetOid not replaced there. I think 'relid' just

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 07:30 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: Lazy people? I'm not in a hurry to drop it; it's not costing us much to just sit there, other than in this connection which we see how to fix. Actually, I

Re: [HACKERS] shared memory message queues

2013-12-05 Thread Andres Freund
Hi, Planned to look at this for a while... Not a detailed review, just some thoughts. I'll let what I read sink in and possibly comment later. On 2013-10-31 12:21:31 -0400, Robert Haas wrote: The attached patches attempt to rectify some of these problems. Well, I wouldn't call it problems.

Re: [HACKERS] Changes in Trigger Firing

2013-12-05 Thread Sameer Kumar
I have been finally able to get the right set of files. I going with below approach: 1) Add a new column in pg_trigger called tgiscascaded 2) Change pg_trigger.h for this 3) Made changes in trigger.c to insert this values 4) Corresponding changes made in - reltrigger.h

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-05 Thread Alvaro Herrera
Andres Freund wrote: On 2013-12-03 19:55:40 -0300, Alvaro Herrera wrote: I added a new isolation spec to test this specific case, and noticed something that seems curious to me when that test is run in REPEATABLE READ mode: when the UPDATE is aborted, the concurrent FOR UPDATE gets a

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: On Wed, Dec 4, 2013 at 11:07 AM, Josh Berkus j...@agliodbs.com wrote: But you know what? 2.6, overall, still performs better than any kernel in the 3.X series, at least for Postgres. What about the fseek() scalability issue? Not to mention that

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-05 Thread Andres Freund
Hi, On 2013-12-05 10:42:35 -0300, Alvaro Herrera wrote: I intend to apply these two to 9.3 and master, and then apply your freeze fix on top (which I'm cleaning up a bit -- will resend later.) I sure hope it get's cleaned up - it's an evening's hack, with a good glass of wine ontop. Do you

Re: [HACKERS] Changes in Trigger Firing

2013-12-05 Thread Sameer Kumar
One scenario where I can forsee an issue is when someone uses the tablename with-in the trigger function on which the trigger was fired. I am not sure how and what issue might crop up but this will be one of my test cases. -- View this message in context:

[HACKERS] Feature request: Logging SSL connections

2013-12-05 Thread Dr. Andreas Kunert
Hello, we were really missing the information in our log files if (and which of) our users are using SSL during their connections. The attached patch is a very simple solution to this problem - it just tests if the ssl pointer in Port is null. If no, it adds SSL to the logfile, otherwise it

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 06:32 AM, Robert Haas wrote: During development of the dynamic shared memory facility, Noah and I spent a lot of time arguing about whether it was practical to ensure that a dynamic shared memory segment got mapped at the same address in every backend that used it. My vote goes

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-04 18:48:44 -0500, Robert Haas wrote: And record_image_eq does a rather elaborate dance around here, calling the appropriate GET_x_BYTES macro depending on the type-width. If we can really count on the high-order bits to be zero, that's

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
On 2013-12-05 15:57:22 +0200, Heikki Linnakangas wrote: As a side-note, I've been thinking that we don't really need same-address mapping for shared_buffers either. Getting rid of it wouldn't buy us anything right now, but if we wanted e.g to make shared_buffers changeable without a restart,

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 08:58:55 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I don't think we can get rid of that dance in record_image_eq - it very well could used on records originally generated when those bits haven't been guaranteed to be zeroed. No, you're failing to

Re: [HACKERS] Regression tests failing if not launched on db regression

2013-12-05 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: It happens that the following regression tests are failing if they are run on a database not named regression: This does not seem like a bug to me, although maybe we'd better update the documentation to specify that you need to use a DB named

[HACKERS] [bug fix] pg_ctl always uses the same event source

2013-12-05 Thread MauMau
Hello, I've removed a limitation regarding event log on Windows with the attached patch. I hesitate to admit this is a bug fix and want to regard this an improvement, but maybe it's a bug fix from users' perspective. Actually, I received problem reports from some users. [Problem] pg_ctl

Re: [HACKERS] Backup throttling

2013-12-05 Thread Antonin Houska
On 12/02/2013 02:23 PM, Boszormenyi Zoltan wrote: Hi, I am reviewing your patch. Thanks. New version attached. * Does it follow the project coding guidelines? Yes. A nitpicking: this else branch below might need brackets because there is also a comment in that branch: +

Re: [HACKERS] More legacy code: pg_ctl

2013-12-05 Thread Peter Eisentraut
On 11/18/13, 8:09 PM, Josh Berkus wrote: a) by default, it returns to the caller without waiting for postgres to actually start/stop/restart. In this mode, it also always returns success regardless of result. The reason for this is that until sometime recently (PQping) we didn't have a

Re: [HACKERS] More legacy code: pg_ctl

2013-12-05 Thread Peter Eisentraut
On 11/18/13, 8:20 PM, Josh Berkus wrote: c) that stop defaults to smart mode, instead of fast mode. This has been discussed many times already, so you'd need to check the archives. (I'm not in favor of smart mode either.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
On 2013-12-05 07:44:27 -0500, Robert Haas wrote: On Thu, Dec 5, 2013 at 4:56 AM, Andres Freund and...@2ndquadrant.com wrote: Hi Robert, On 2013-12-04 23:32:27 -0500, Robert Haas wrote: But I'm also learning painfully that this kind of thing only goes so far. For example, I spent some

Re: [HACKERS] Feature request: Logging SSL connections

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 8:53 AM, Dr. Andreas Kunert wrote: we were really missing the information in our log files if (and which of) our users are using SSL during their connections. The attached patch is a very simple solution to this problem - it just tests if the ssl pointer in Port is null. If no,

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Greg Stark
On Thu, Dec 5, 2013 at 8:35 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Yes. And using something efficiently DirectIO is more difficult than BufferedIO. If we change write() flag with direct IO in PostgreSQL, it will execute hardest ugly randomIO. Using DirectIO presumes you're

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Tom Lane
Pavel Golub pa...@microolap.com writes: I personally see two approaches: 1. Implement GUC variable controling this behaviour per session 2. Introduce new safe reg* variables, e.g. sregclass, sregtype etc. I don't think new types are a good idea. If we are afraid to change the behavior of the

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
On 2013-12-05 15:44:34 +0100, Andres Freund wrote: On 2013-12-05 07:44:27 -0500, Robert Haas wrote: And then I thought, boy, it sucks not to be able to declare what kind of a thing we're pointing *at* here, but apart from using C++ I see no solution to that problem. I guess we could do

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Claudio Freire
On Thu, Dec 5, 2013 at 11:42 AM, Greg Stark st...@mit.edu wrote: (b) is the way more interesting research project though. I don't think anyone's tried it and the kernel interface to provide the kinds of information Postgres needs requires a lot of thought. If it's done right then Postgres

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 9:41 AM, Tom Lane wrote: Pavel Golub pa...@microolap.com writes: I personally see two approaches: 1. Implement GUC variable controling this behaviour per session 2. Introduce new safe reg* variables, e.g. sregclass, sregtype etc. I don't think new types are a good idea. If we

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 8:57 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/05/2013 06:32 AM, Robert Haas wrote: During development of the dynamic shared memory facility, Noah and I spent a lot of time arguing about whether it was practical to ensure that a dynamic shared memory

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 08:58:55 -0500, Tom Lane wrote: I'm a bit worried that somebody, particularly third-party code, might've sloppily written return foo in a V1 function when return Int32GetDatum(foo) would be correct. In that case, the resultant Datum

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: We could invent some sneaky syntax variants, like 'pg_klass'::regclass errors, but '?pg_klass'::regclass does not. Hmm ... cute idea, but shoehorning it into regoperator might be problematic. You'd have to pick a flag character that wasn't a valid

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 10:02:56 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 08:58:55 -0500, Tom Lane wrote: I'm a bit worried that somebody, particularly third-party code, might've sloppily written return foo in a V1 function when return Int32GetDatum(foo)

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 9:44 AM, Andres Freund and...@2ndquadrant.com wrote: Why? Lots of people have written lots of programs that do just that. Well, but we're a database, not a generic programming library ;) I think we're arguably both. But what's your alternative if you have a

[HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread MauMau
Hello, My customers and colleagues sometimes (or often?) ask about the following message: FATAL: the database system is starting up This message is often output dozens of times during a failover or PITR. The users seem to be worried because the message level is FATAL and they don't know

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 9:41 AM, Tom Lane t...@sss.pgh.pa.us wrote: Pavel Golub pa...@microolap.com writes: I personally see two approaches: 1. Implement GUC variable controling this behaviour per session 2. Introduce new safe reg* variables, e.g. sregclass, sregtype etc. I don't think new

Re: [HACKERS] shared memory message queues

2013-12-05 Thread Kohei KaiGai
Sorry for my late. I checked the part-3 (shm-mq-v1.patc) portion as below. Your comments towards part-1 and part-2 are reasonable for me, so I have no argue on this portion. Even though shm_mq_create() expects the address is aligned, however, no mechanism to ensure. How about to put Assert()

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I was actually thinking about making Datum (and some other types we have) structs or unions. Currently it's far, far to easy to mix them. We throw away pretty much all of the little typesafety C has by typedef'ing them to integral types with lots of

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Dec 5, 2013 at 9:41 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't think new types are a good idea. If we are afraid to change the behavior of the input converters, what we should do is introduce new functions, eg toregclass(text) returns

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Greg Stark
On Thu, Dec 5, 2013 at 2:54 PM, Claudio Freire klaussfre...@gmail.com wrote: That's a bad idea in the current state of affairs. MM files haven't been designed for that usage, and getting stable performance out of that will be way too difficult. I'm talking about long-term goals here. Either of

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 10:25 AM, MauMau wrote: Report these as FATAL to the client because the client wants to know the reason. But don't output them to server log because they are not necessary for DBAs Yeah, this is part of a more general problem, which you have characterized correctly: What is fatal

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Tom Lane
MauMau maumau...@gmail.com writes: Shouldn't we lower the severity or avoiding those messages to server log? No. They are FATAL so far as the individual session is concerned. Possibly some documentation effort is needed here, but I don't think any change in the code behavior would be an

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
You could try my lwlock-scalability improvement patches - for some workloads here, the improvements have been rather noticeable. Which version are you testing? I tried your patches on next link. As you suspect I didn't see any improvements. I tested it on PostgreSQL 9.2 Stable.

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
On 2013-12-05 10:17:18 -0500, Robert Haas wrote: On Thu, Dec 5, 2013 at 9:44 AM, Andres Freund and...@2ndquadrant.com wrote: Why? Lots of people have written lots of programs that do just that. Well, but we're a database, not a generic programming library ;) I think we're arguably both.

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Andres Freund
On 2013-12-05 17:46:44 +0200, Metin Doslu wrote: I tried your patches on next link. As you suspect I didn't see any improvements. I tested it on PostgreSQL 9.2 Stable. You tested the correct branch, right? Which commit does git rev-parse HEAD show? But generally, as long as your profile hides

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 6:07 AM, Simon Riggs wrote: On 5 December 2013 01:55, Peter Eisentraut pete...@gmx.net wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Another advantage of this approach is that, IIUC, type input functions can't return a NULL value. So 'pg_klass'::regclass could return 0, but not NULL. On the other hand, toregclass('pg_klass') *could* return NULL, which seems conceptually cleaner.

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
You tested the correct branch, right? Which commit does git rev-parse HEAD show? I applied last two patches manually on PostgreSQL 9.2 Stable.

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Tom Lane
Greg Stark st...@mit.edu writes: I think the way to use mmap would be to mmap very large chunks, possibly whole tables. We would need some way to control page flushes that doesn't involve splitting mappings and can be efficiently controlled without having the kernel storing arbitrarily large

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
From what I've seen so far the bigger problem than contention in the lwlocks itself, is the spinlock protecting the lwlocks... Postgres 9.3.1 also reports spindelay, it seems that there is no contention on spinlocks. PID 21121 lwlock 0: shacq 0 exacq 33 blk 1 spindelay 0 PID 21121 lwlock 33:

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 10:34:16 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I was actually thinking about making Datum (and some other types we have) structs or unions. Currently it's far, far to easy to mix them. We throw away pretty much all of the little typesafety C

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 10:34:16 -0500, Tom Lane wrote: In any case, the number of bugs I can remember that such a thing would've prevented is negligible. Cases talked about upthread, where a plain datatype is returned as a Datum instead of using

Re: [HACKERS] Dynamic Shared Memory stuff

2013-12-05 Thread Heikki Linnakangas
On 11/20/2013 09:58 PM, Robert Haas wrote: On Wed, Nov 20, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: How many allocations? What size will they have have typically, minimum and maximum? The facility is intended to be general, so the answer could vary widely by

Re: [HACKERS] Feature request: Logging SSL connections

2013-12-05 Thread Marko Kreen
On Thu, Dec 05, 2013 at 09:43:31AM -0500, Peter Eisentraut wrote: On 12/5/13, 8:53 AM, Dr. Andreas Kunert wrote: we were really missing the information in our log files if (and which of) our users are using SSL during their connections. The attached patch is a very simple solution to

[HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread David Johnston
Tom Lane-2 wrote MauMau lt; maumau307@ gt; writes: Shouldn't we lower the severity or avoiding those messages to server log? No. They are FATAL so far as the individual session is concerned. Possibly some documentation effort is needed here, but I don't think any change in the code

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Peter Eisentraut
I think this proposal is a bit deadlocked now. - There are technical concerns about launching a server executable from within a client. - There are conceptual concerns about promoting an embedded database mode. On the other hand: - Everyone would like to have a way to use psql (and other basic

Re: [HACKERS] logical changeset generation v6.7

2013-12-05 Thread Andres Freund
Hi, On 2013-12-05 22:03:51 +0900, Kyotaro HORIGUCHI wrote: - You assined HeapTupleGetOid(tuple) into relid to read in several points but no modification. Nevertheless, you left HeapTupleGetOid not replaced there. I think 'relid' just for repeated reading has far small merit

Re: [HACKERS] Extension Templates S03E11

2013-12-05 Thread Peter Eisentraut
On 12/1/13, 10:47 PM, Stephen Frost wrote: Having a management system for sets of objects is a *great* idea- and one which we already have through schemas. What we don't have is any kind of versioning system built-in or other metadata about it, nor do we have good tooling which leverages such

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-05 11:39:29 -0500, Peter Eisentraut wrote: I think this proposal is a bit deadlocked now. - There are technical concerns about launching a server executable from within a client. - There are conceptual concerns about promoting an embedded database mode. On the other hand:

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 10:08 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: We could invent some sneaky syntax variants, like 'pg_klass'::regclass errors, but '?pg_klass'::regclass does not. Hmm ... cute idea, but shoehorning it into regoperator might be problematic. You'd have to pick

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-05 Thread J Smith
On Thu, Nov 28, 2013 at 5:15 AM, Andres Freund and...@2ndquadrant.com wrote: Hi, Do you still have the core file around? If so could you 'p *ShmemVariableCache' and 'p *ControlFile'? So sorry, I didn't see this message until just today. Seems it was accidentally archived before hitting my

Re: [HACKERS] Time-Delayed Standbys

2013-12-05 Thread Fabrízio de Royes Mello
On Thu, Dec 5, 2013 at 7:57 AM, Simon Riggs si...@2ndquadrant.com wrote: On 5 December 2013 08:51, Magnus Hagander mag...@hagander.net wrote: Not recalling the older thread, but it seems the breaks on clock drift, I think we can fairly easily make that situation good enough. Just have

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread David E. Wheeler
On Dec 5, 2013, at 7:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, another arguable advantage of fixing this via new functions is that users could write equivalent (though no doubt slower) functions for use in pre-9.4 releases, and thus not need to maintain multiple versions of app code that

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Josh Berkus
On 12/05/2013 07:40 AM, Greg Stark wrote: On Thu, Dec 5, 2013 at 2:54 PM, Claudio Freire klaussfre...@gmail.com wrote: That's a bad idea in the current state of affairs. MM files haven't been designed for that usage, and getting stable performance out of that will be way too difficult. I'm

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Josh Berkus
On 12/05/2013 05:48 AM, Stephen Frost wrote: * Peter Geoghegan (p...@heroku.com) wrote: On Wed, Dec 4, 2013 at 11:07 AM, Josh Berkus j...@agliodbs.com wrote: But you know what? 2.6, overall, still performs better than any kernel in the 3.X series, at least for Postgres. What about the

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Claudio Freire
On Thu, Dec 5, 2013 at 1:03 PM, Metin Doslu me...@citusdata.com wrote: From what I've seen so far the bigger problem than contention in the lwlocks itself, is the spinlock protecting the lwlocks... Postgres 9.3.1 also reports spindelay, it seems that there is no contention on spinlocks. Did

Re: [HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Stephen Frost
* David Johnston (pol...@yahoo.com) wrote: ISTM that instituting some level of categorization for messages would be helpful. Then logging and reporting frameworks would be able to identify and segregate the logs in whatever way they and the configuration deems appropriate. I've wanted to do

Re: [HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Josh Berkus
On 12/05/2013 10:21 AM, Stephen Frost wrote: * David Johnston (pol...@yahoo.com) wrote: ISTM that instituting some level of categorization for messages would be helpful. Then logging and reporting frameworks would be able to identify and segregate the logs in whatever way they and the

Re: [HACKERS] Problem with displaying wide tables in psql

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 1:09 AM, Sergey Muraviov sergey.k.murav...@gmail.com wrote: And my patch affects the row view only. To help us avoid forgetting about this patch, please add it here: https://commitfest.postgresql.org/action/commitfest_view/open -- Robert Haas EnterpriseDB:

Re: [HACKERS] Regression tests failing if not launched on db regression

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 9:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: It happens that the following regression tests are failing if they are run on a database not named regression: This does not seem like a bug to me, although maybe we'd better

Re: [HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 12/05/2013 10:21 AM, Stephen Frost wrote: But ... if we set a firm policy on this, then we could gradually clean up the error messages piecemeal over the next couple of major versions. We could also make sure that any new features complied with the

Re: [HACKERS] Regression tests failing if not launched on db regression

2013-12-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Dec 5, 2013 at 9:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: It happens that the following regression tests are failing if they are run on a database not named regression: This does not seem

Re: [HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Josh Berkus
On 12/05/2013 10:46 AM, Tom Lane wrote: Before we could get very far we'd need a better understanding than we have of what cases a DBA might be interested in. To take the specific example that started this thread, there wouldn't be a lot of value IMO in a classification like connection

  1   2   >