Re: [HACKERS] GIN improvements part 1: additional information

2014-01-06 Thread Amit Langote
On Sat, Dec 21, 2013 at 4:36 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Yet another version. The encoding/decoding code is now quite isolated in ginpostinglist.c, so it's easy to experiment with different encodings. This patch uses varbyte encoding again. I got a bit carried away,

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Heikki Linnakangas
On 01/05/2014 07:56 PM, Robert Haas wrote: Right now, storing spinlocks in dynamic shared memory *almost* works, but there are problems with --disable-spinlocks. In that configuration, we use semaphores to simulate spinlocks. Every time someone calls SpinLockInit(), it's going to allocate a

Re: [HACKERS] [PATCH] Support for pg_stat_archiver view

2014-01-06 Thread Gabriele Bartolini
Hi Fabrizio, Il 05/01/14 20:46, Fabrizio Mello ha scritto: I don't see your code yet, but I would like to know if is possible to implement this view as an extension. I wanted to do it as an extension - so that I could backport that to previous versions of Postgres. I do not think it is a

Re: [HACKERS] Compiling extensions on Windows

2014-01-06 Thread Dave Page
On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is

Re: [HACKERS] cleanup in code

2014-01-06 Thread Heikki Linnakangas
On 01/04/2014 07:20 AM, Amit Kapila wrote: 1. compiling with msvc shows warning in relcache.c 1e:\workspace\postgresql\master\postgresql\src\backend\utils\cache\relcache.c(3959): warning C4715: 'RelationGetIndexAttrBitmap' : not all control paths return a value Attached patch

Re: [HACKERS] cleanup in code

2014-01-06 Thread David Rowley
On Mon, Jan 6, 2014 at 11:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/04/2014 07:20 AM, Amit Kapila wrote: 1. compiling with msvc shows warning in relcache.c 1e:\workspace\postgresql\master\postgresql\src\backend\ utils\cache\relcache.c(3959): warning C4715:

Re: [HACKERS] Compiling extensions on Windows

2014-01-06 Thread Sandeep Thakkar
Sure. I'll make the changes so that the next available Windows installers include lbintl.h in $Installdir/include. How about the changes with respect to NLS? On Mon, Jan 6, 2014 at 2:44 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com

Re: [HACKERS] cleanup in code

2014-01-06 Thread Andres Freund
On 2014-01-06 23:51:52 +1300, David Rowley wrote: On Mon, Jan 6, 2014 at 11:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/04/2014 07:20 AM, Amit Kapila wrote: 1. compiling with msvc shows warning in relcache.c 1e:\workspace\postgresql\master\postgresql\src\backend\

Re: [HACKERS] Compiling extensions on Windows

2014-01-06 Thread Dave Page
On Mon, Jan 6, 2014 at 10:57 AM, Sandeep Thakkar sandeep.thak...@enterprisedb.com wrote: Sure. I'll make the changes so that the next available Windows installers include lbintl.h in $Installdir/include. How about the changes with respect to NLS? No, there's nothing to change there. Craig was

Re: [HACKERS] Compiling extensions on Windows

2014-01-06 Thread Craig Ringer
If libintl.h and any headers it in turn includes are bundled, there is no longer an issue with NLS. That was just a workaround for building exts when Pg's headers tried to refer to nonexistent headers when NLS was enabled. On 6 Jan 2014 18:57, Sandeep Thakkar sandeep.thak...@enterprisedb.com

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Andres Freund
On 2014-01-06 10:35:59 +0200, Heikki Linnakangas wrote: On 01/05/2014 07:56 PM, Robert Haas wrote: Right now, storing spinlocks in dynamic shared memory *almost* works, but there are problems with --disable-spinlocks. In that configuration, we use semaphores to simulate spinlocks. Every time

[HACKERS] PostgreSQL Service on Windows does not start if data directory given is relative path

2014-01-06 Thread Rajeev rastogi
I have found a case that PostgreSQL as win32 service does not start, if the data directory given as relative path. Error observed in this case is: The PostgreSQL on Local Computer started and then stopped. This may happen because relative path given will be

Re: [HACKERS] Compiling extensions on Windows

2014-01-06 Thread Sandeep Thakkar
Okay. BTW, I just checked that Windows 32bit installer ships the libintl.h. Did you try if it works for you? Or it requires more headers? Somehow, Windows 64bit installer installer missed it. I'll fix the build script. On Mon, Jan 6, 2014 at 4:55 PM, Craig Ringer cr...@2ndquadrant.com wrote:

[HACKERS] Convert Datum* to char*

2014-01-06 Thread Masterprojekt Naumann1
Hi, I want to read an attribute value from a TupleTableSlot. When I try to convert an attribute of SQL type varchar from Datum* to char* with the help of the method TextDatumGetCString(...), sometimes there is a segmentation fault. The segmentation fault comes from the method

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Heikki Linnakangas
On 01/06/2014 03:09 PM, Masterprojekt Naumann1 wrote: I want to read an attribute value from a TupleTableSlot. When I try to convert an attribute of SQL type varchar from Datum* to char* with the help of the method TextDatumGetCString(...), sometimes there is a segmentation fault. The

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Masterprojekt Naumann1
Yes, in some cases, Datum is 0, which I test before conversion. Additionally, I looked at tts_isnull but it does not prevent the segmentation fault in some cases. The problem is. that sometimes the value is random, but I don't know why and how I can detect that case. 2014/1/6 Heikki Linnakangas

Re: [HACKERS] In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread Steve Singer
On 01/05/2014 09:13 PM, Michael Paquier wrote: On Mon, Jan 6, 2014 at 4:51 AM, Mark Dilger markdil...@yahoo.com mailto:markdil...@yahoo.com wrote: I am building a regression test system for replication and came across this email thread. I have gotten pretty far into my implementation, but

Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread Andres Freund
On 2014-01-06 01:25:57 +, Greg Stark wrote: -- greg On 5 Jan 2014 14:54, Mark Dilger markdil...@yahoo.com wrote: I am building a regression test system for replication and came across this email thread. I have gotten pretty far into my implementation, but would be happy to make

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Craig Ringer
On 01/06/2014 09:09 PM, Masterprojekt Naumann1 wrote: Why is the memory of the variable uninitialized? Are there any other patches you've made to the running PostgreSQL instance? I'd want to run under Valgrind and see what turned up. This might be a bit tricky with an intermittent fault during

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Thu, Jan 2, 2014 at 4:15 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-02 16:05:09 -0500, Robert Haas wrote: On Thu, Jan 2, 2014 at 3:19 PM, Andres Freund and...@2ndquadrant.com wrote: I was wondering if we could somehow arrange to not release the subtransaction's

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 09:10:48 -0500, Robert Haas wrote: On Thu, Jan 2, 2014 at 4:15 PM, Andres Freund and...@2ndquadrant.com wrote: I think the only principled fixes are to either retain the lock or forcibly detoast before releasing it. I don't think that's sufficient. Unless I miss something

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2014-01-06 Thread Robert Haas
On Sun, Dec 22, 2013 at 10:02 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Sun, Dec 22, 2013 at 3:01 PM, Fujii Masao masao.fu...@gmail.com wrote: I found the bug of ALTER SYSTEM SET patch. The procedure to reproduce it is here. $ psql =# ALTER SYSTEM SET shared_buffers = '512MB'; ALTER

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Masterprojekt Naumann1
2014/1/6 Craig Ringer cr...@2ndquadrant.com On 01/06/2014 09:09 PM, Masterprojekt Naumann1 wrote: Why is the memory of the variable uninitialized? Are there any other patches you've made to the running PostgreSQL instance? I'd want to run under Valgrind and see what turned up. This might

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Thomas Fanghaenel
On Mon, Jan 6, 2014 at 8:09 AM, Masterprojekt Naumann1 mpws201...@gmail.com wrote: Why is the memory of the variable uninitialized? Are you checking that i = slot-tts_nvalid before accessing the tts_values and tts_isnull arrays? -- Sent via pgsql-hackers mailing list

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 09:43:45 -0500, Robert Haas wrote: I actually vote for not allowing doing so at all by erroring out when accessing a plpgsql variable created in an aborted subxact, unless you explicitly signal that you want to do do so by calling some function deleting the information about

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Another idea is to include some identifying information in the lwlock. That was my immediate reaction to this issue... For example, each lwlock could have a char *name in it, and we could print the name. In theory, this could be a big step

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:19 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-06 09:10:48 -0500, Robert Haas wrote: On Thu, Jan 2, 2014 at 4:15 PM, Andres Freund and...@2ndquadrant.com wrote: I think the only principled fixes are to either retain the lock or forcibly detoast before

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2014-01-06 Thread Amit Kapila
On Mon, Jan 6, 2014 at 7:58 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Dec 22, 2013 at 10:02 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Sun, Dec 22, 2013 at 3:01 PM, Fujii Masao masao.fu...@gmail.com wrote: I found the bug of ALTER SYSTEM SET patch. The procedure to reproduce

Re: [HACKERS] cleanup in code

2014-01-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Mon, Jan 6, 2014 at 11:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Hmm, I thought we gave enough hints in the elog macro to tell the compiler that elog(ERROR) does no return, since commit b853eb97182079dcd30b4f52576bd5d6c275ee71.

[HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Andres Freund
Hi, Does anybody have an opinion about introducing generic pseudotype IO functions? Pseudotype.c/pg_proc.h are slowly growing a number of pretty useless/redundant copypasted functions... Most for cases that are pretty damn unlikely to be hit by users not knowing what they do. What about adding a

Re: [HACKERS] [PATCH] SQL assertions prototype

2014-01-06 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: Peter Eisentraut pete...@gmx.net schrieb: On 12/18/13, 2:22 PM, Andres Freund wrote: It would only force serialization for transactions that modify tables covered by the assert, that doesn't seem to bad. Anything covered by an assert shoulnd't be

Re: [HACKERS] cleanup in code

2014-01-06 Thread Andres Freund
On 2014-01-06 09:54:15 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On Mon, Jan 6, 2014 at 11:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Hmm, I thought we gave enough hints in the elog macro to tell the compiler that elog(ERROR) does no return, since

Re: [HACKERS] [PATCH] SQL assertions prototype

2014-01-06 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: For an example of how things can look OK in terms of enforced constraints as of different moments in time, yet those moments in time could be inconsistent, see this section: http://wiki.postgresql.org/wiki/SSI#Enforcing_Business_Rules_in_Triggers

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-01-06 10:35:59 +0200, Heikki Linnakangas wrote: That assumes that you never hold more than one spinlock at a time, otherwise you can get deadlocks. I think that assumptions holds currently, because acquiring two spinlocks at a time would be

Re: [HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Andres Freund
On 2014-01-06 10:29:06 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Does anybody have an opinion about introducing generic pseudotype IO functions? Yes: -1. Ok, fine with me. Pseudotype.c/pg_proc.h are slowly growing a number of pretty useless/redundant

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Masterprojekt Naumann1
2014/1/6 Thomas Fanghaenel tfanghae...@salesforce.com On Mon, Jan 6, 2014 at 8:09 AM, Masterprojekt Naumann1 mpws201...@gmail.com wrote: Why is the memory of the variable uninitialized? Are you checking that i = slot-tts_nvalid before accessing the tts_values and tts_isnull arrays?

Re: [HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Does anybody have an opinion about introducing generic pseudotype IO functions? Yes: -1. Pseudotype.c/pg_proc.h are slowly growing a number of pretty useless/redundant copypasted functions... Most for cases that are pretty damn unlikely to be hit

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:47 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-06 09:43:45 -0500, Robert Haas wrote: I actually vote for not allowing doing so at all by erroring out when accessing a plpgsql variable created in an aborted subxact, unless you explicitly signal that

Re: [HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Peter Eisentraut
On 1/6/14, 10:29 AM, Tom Lane wrote: Pseudotype.c/pg_proc.h are slowly growing a number of pretty useless/redundant copypasted functions... Most for cases that are pretty damn unlikely to be hit by users not knowing what they do. That's hardly the largest cost associated with inventing a new

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I guess the question boils down to: why are we keeping --disable-spinlocks around? If we're expecting that people might really use it for serious work, then it needs to remain and it needs to work with dynamic shared memory. If we're expecting that

Re: [HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 1/6/14, 10:29 AM, Tom Lane wrote: This will break some of the function sanity checks in opr_sanity, Then the tests can be changed. That will weaken their ability to detect actual mistakes, no? If there were a large benefit to merging the pseudotype

Re: [HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Andres Freund
On 2014-01-06 11:28:29 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On 1/6/14, 10:29 AM, Tom Lane wrote: This will break some of the function sanity checks in opr_sanity, Then the tests can be changed. That will weaken their ability to detect actual mistakes, no?

Re: [HACKERS] Compiling extensions on Windows

2014-01-06 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2014 07:32 PM, Craig Ringer wrote: Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 11:08:41 -0500, Robert Haas wrote: On Mon, Jan 6, 2014 at 9:47 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-06 09:43:45 -0500, Robert Haas wrote: I actually vote for not allowing doing so at all by erroring out when accessing a plpgsql variable created in an

Re: [HACKERS] generic pseudotype IO functions?

2014-01-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I think I am less concerned about pseudotypes.c than about bloating pg_proc.h even further and about the annoyance of editing it - but I guess that should rather be fixed by storing it in a more sensible format at some point... Yeah, getting rid of

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Andres Freund
On 2014-01-06 09:59:49 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-01-06 10:35:59 +0200, Heikki Linnakangas wrote: That assumes that you never hold more than one spinlock at a time, otherwise you can get deadlocks. I think that assumptions holds currently,

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Tom Lane
Masterprojekt Naumann1 mpws201...@gmail.com writes: Nevertheless, slot-tts_nvalid is always 0. I hope that there is no other problem. You should not be touching the tts_values/tts_isnull arrays without having first called slot_getsomeattrs or slot_getallattrs. See comments in

Re: [HACKERS] In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread Mark Dilger
The reason I was going to all the trouble of creating chrooted environments was to be able to replicate clusters that have tablespaces.  Not doing so makes the test code simpler at the expense of reducing test coverage. I am using the same binaries.  The chroot directories are not chroot jails. 

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-01-06 10:35:59 +0200, Heikki Linnakangas wrote: That assumes that you never hold more than one spinlock at a time, otherwise you can get deadlocks. I think that assumptions

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-06 11:08:41 -0500, Robert Haas wrote: On Mon, Jan 6, 2014 at 9:47 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-06 09:43:45 -0500, Robert Haas wrote: I actually vote for not allowing doing

Re: [HACKERS] Add CREATE support to event triggers

2014-01-06 Thread Robert Haas
On Fri, Jan 3, 2014 at 9:05 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: The other thing that bothers me here is that, while a normalized command string sounds great in theory, as soon as you want to allow (for example) mapping schema A on node 1 to schema B on node 2, the wheels come

Re: [HACKERS] In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread Mark Dilger
I agree that merely setting up masters and slaves is the tip of the iceberg.  It seems to be what needs to be tackled first, though, because until we have a common framework, we cannot all contribute tests to it. I imagine setting up a whole hierarchy of master, hotstandbys, warmstandbys, etc.,

Re: [HACKERS] In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread Andres Freund
On 2014-01-06 09:12:03 -0800, Mark Dilger wrote: The reason I was going to all the trouble of creating chrooted environments was to be able to replicate clusters that have tablespaces.  Not doing so makes the test code simpler at the expense of reducing test coverage. I am using the same

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Is forcibly detoast everything a complete no-go? I realize there are performance concerns with that approach, but I'm not sure how realistic a worry it actually is. It's certainly possible to think of scenarios under which it'd be painful, eg, you

Re: [HACKERS] Add CREATE support to event triggers

2014-01-06 Thread Alvaro Herrera
Robert Haas escribió: I think this direction has some potential. I'm not sure it's right in detail. The exact scheme you propose above won't work if you want to leave out the schema name altogether, and more generally it's not going to help very much with anything other than substituting in

[HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread David Johnston
Andres Freund-3 wrote On 2014-01-06 09:12:03 -0800, Mark Dilger wrote: The reason I was going to all the trouble of creating chrooted environments was to be able to replicate clusters that have tablespaces.  Not doing so makes the test code simpler at the expense of reducing test coverage.

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 11:22 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think we can eliminate the first of those. Semaphores for spinlocks were a performance disaster fifteen years ago, and the situation has surely only gotten worse since then. I do, however, believe that --disable-spinlocks

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 12:40:25 -0500, Robert Haas wrote: On Mon, Jan 6, 2014 at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-06 11:08:41 -0500, Robert Haas wrote: Yea. But at least it would fail reliably instead of just under concurrency and other strange circumstances - and

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-01-05 14:06:52 -0500, Tom Lane wrote: I seem to recall that there was some good reason for keeping all the LWLocks in an array, back when the facility was first designed. I'm too lazy to research the point right now, but you might want to

Re: [HACKERS] Add CREATE support to event triggers

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 1:17 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I agree, except that I would prefer not to have one function for each DDL statement type; instead we would have a single function that knows to expand arbitrary strings using arbitrary JSON parameter objects, and

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Robert Haas
On Sat, Jan 4, 2014 at 12:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: As far as back-patching the GUCs, my thought would be to back-patch them but mark them GUC_NOT_IN_SAMPLE in 9.3, so we don't have to touch the default postgresql.conf. That seems

Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc. Michael Paquier

2014-01-06 Thread Mark Dilger
I was already starting to consider making the chroot logic optional, based on the resistence expressed by folks on this thread. How about the following: During the configure phase, it checks for chroot and setuid and friends that it will need. The regression suite has config parameters to

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:48 AM, Amit Kapila amit.kapil...@gmail.com wrote: Couldn't we also handle this by postponing FreeConfigVariables until after the if (error) block? Wouldn't doing that way can lead to bigger memory leak, if error level is ERROR. Though in current fix also it can

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-06 Thread Robert Haas
On Sun, Jan 5, 2014 at 10:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: =?ISO-8859-1?Q?Fabr=EDzio_de_Royes_Mello?= fabriziome...@gmail.com writes: You are correct. pg_dump export reloptions using WITH clause of CREATE TABLE statement. I.e.: CREATE TABLE foo ( ) WITH (autovacuum_enabled=false,

[HACKERS] How to reproduce serialization failure for a read only transaction.

2014-01-06 Thread AK
If two transactions both read and write, I can easily reproduce the following: could not serialize access due to read/write dependencies among transactions. However, the 9.3 documentation says that When relying on Serializable transactions to prevent anomalies, it is important that any data read

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, I took a look at this and it turns out not to be very hard, so here's a patch. Currently, we allocate 3 semaphore per shared buffer and a bunch of others, but the 3 per shared buffer dominates, so you end up with ~49k spinlocks for the default

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Jan 4, 2014 at 12:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Keep in mind that 9.3 is still wet behind the ears and many many people haven't adopted it yet. If we do what you're suggesting then we're creating a completely useless inconsistency

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 2:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, I took a look at this and it turns out not to be very hard, so here's a patch. Currently, we allocate 3 semaphore per shared buffer and a bunch of others, but the 3 per shared

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Jan 5, 2014 at 10:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, if bdr is installed but the validation doesn't happen unless bdr is loaded in some sense, then that is an implementation deficiency that I think we can insist be rectified before

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 1:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the LWLock mechanism has been stable for long enough now that we can probably suppose this struct is no more subject to churn than any other widely-known one, so maybe that consideration is no longer significant. On the

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-01-06 12:40:25 -0500, Robert Haas wrote: Is forcibly detoast everything a complete no-go? I realize there are performance concerns with that approach, but I'm not sure how realistic a worry it actually is. The scenario I am primarily

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 2:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: -1 for the any_spinlock_held business (useless overhead IMO, as it doesn't have anything whatsoever to do with enforcing the actual coding rule). Hmm. I thought that was a pretty

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 1:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the LWLock mechanism has been stable for long enough now that we can probably suppose this struct is no more subject to churn than any other widely-known one, so maybe that

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 3:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 2:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: -1 for the any_spinlock_held business (useless overhead IMO, as it doesn't have anything whatsoever to do with

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 3:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 1:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the LWLock mechanism has been stable for long enough now that we can probably suppose this struct is no more

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 3:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree it'd be nicer if we had some better way than mere manual inspection to enforce proper use of spinlocks; but this change doesn't seem to me to move the ball downfield by any

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 3:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 3:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree it'd be nicer if we had some better way than mere manual inspection to enforce proper use of spinlocks; but

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-06 Thread james
On 06/01/2014 03:14, Robert Haas wrote: That's up to the application. After calling dsm_create(), you call dsm_segment_handle() to get the 32-bit integer handle for that segment. Then you have to get that to the other process(es) somehow. If you're trying to share a handle with a background

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-06 Thread james
On 06/01/2014 04:20, Amit Kapila wrote: Duplicate handle should work, but we need to communicate the handle to other process using IPC. Only if the other process needs to use it. The IPC is not to transfer the handle to the other process, just to tell it which slot in its handle table

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 4:04 PM, james ja...@mansionfamily.plus.com wrote: The point remains that you need to duplicate it into every process that might want to use it subsequently, so it makes sense to DuplicateHandle into the parent, and then to advertise that handle value publicly so that

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Jan 5, 2014 at 10:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, if bdr is installed but the validation doesn't happen unless bdr is loaded in some sense, then that is an

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 2:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Jan 4, 2014 at 12:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Keep in mind that 9.3 is still wet behind the ears and many many people haven't adopted it yet. If we do what you're

Re: [HACKERS] How to reproduce serialization failure for a read only transaction.

2014-01-06 Thread Florian Pflug
On Jan6, 2014, at 20:41 , AK alk...@gmail.com wrote: If two transactions both read and write, I can easily reproduce the following: could not serialize access due to read/write dependencies among transactions. However, the 9.3 documentation says that When relying on Serializable transactions

Re: [HACKERS] Convert Datum* to char*

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 11:34 AM, Tom Lane t...@sss.pgh.pa.us wrote: Masterprojekt Naumann1 mpws201...@gmail.com writes: Nevertheless, slot-tts_nvalid is always 0. I hope that there is no other problem. You should not be touching the tts_values/tts_isnull arrays without having first called

[HACKERS] Re: How to reproduce serialization failure for a read only transaction.

2014-01-06 Thread AK
Hi Florian, can you explain why do you state that default isolation level is assumed to be serializable, of course, when you explicitly specify isolation level for every session - why should he default matter at all? When I am trying to reproduce the scenario which you have posted, I am

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:48 AM, Stephen Frost sfr...@snowman.net wrote: None of these ideas are a complete solution for LWLOCK_STATS. In the other three cases noted above, we only need an identifier for the lock instantaneously, so that we can pass it off to the logger or dtrace or whatever.

Re: [HACKERS] Re: How to reproduce serialization failure for a read only transaction.

2014-01-06 Thread Florian Pflug
On Jan6, 2014, at 23:28 , AK alk...@gmail.com wrote: can you explain why do you state that default isolation level is assumed to be serializable, of course, when you explicitly specify isolation level for every session - why should he default matter at all? Sorry, that was a leftover - I

Re: [HACKERS] Re: How to reproduce serialization failure for a read only transaction.

2014-01-06 Thread Jim Nasby
On 1/6/14, 5:27 PM, Florian Pflug wrote: On Jan6, 2014, at 23:28 , AK alk...@gmail.com wrote: First, dependency tracking can produce false positives, i.e. assume that dependencies exist between transactions which are actually independent. In my example, postgres fails to realize that W2 can

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Jim Nasby
On 1/6/14, 2:59 PM, Robert Haas wrote: On Mon, Jan 6, 2014 at 3:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 6, 2014 at 3:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree it'd be nicer if we had some better way than mere manual inspection to

Re: [HACKERS] dynamic shared memory and locks

2014-01-06 Thread Alvaro Herrera
Jim Nasby escribió: On 1/6/14, 2:59 PM, Robert Haas wrote: On Mon, Jan 6, 2014 at 3:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: The point I'm making is that no such code should get past review, whether it's got an obvious performance problem or not. Sure, I agree, but we all make mistakes.

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-06 Thread Jim Nasby
On 1/4/14, 12:00 PM, Tom Lane wrote: If you have some settings that need to be table-specific, then I agree that the reloptions infrastructure is a nice place to track them. What's actually missing here is some compelling examples of such settings for plausible extensions. I've got a real

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Jim Nasby
On 1/4/14, 8:19 AM, Robert Haas wrote: Also, while multixactid_freeze_min_age should be low, perhaps a million as you suggest, multixactid_freeze_table_age should NOT be lowered to 3 million or anything like it. If you do that, people who are actually doing lots of row locking will start

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Jim Nasby
On 1/2/14, 1:32 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: The simplest fix would be to just detoast everything on assignment but that was rejected on performance grounds in that previous thread. I don't see any other realistic way to fix this, however, so maybe we

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Jim Nasby
On 1/6/14, 2:21 PM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-01-06 12:40:25 -0500, Robert Haas wrote: Is forcibly detoast everything a complete no-go? I realize there are performance concerns with that approach, but I'm not sure how realistic a worry it actually

[HACKERS] Re: How to reproduce serialization failure for a read only transaction.

2014-01-06 Thread AK
This worked for me - thank you so much! The SELECT did fail. Also I cannot reproduce a scenario when applications must not depend on results read during a transaction that later aborted;. In this example the SELECT itself has failed. Can you show an example where a SELECT completes, but the

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 7:50 PM, Jim Nasby j...@nasby.net wrote: On 1/4/14, 8:19 AM, Robert Haas wrote: Also, while multixactid_freeze_min_age should be low, perhaps a million as you suggest, multixactid_freeze_table_age should NOT be lowered to 3 million or anything like it. If you do that,

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 8:02 PM, Jim Nasby j...@nasby.net wrote: If concurrent TRUNCATE isn't safe outside of this case then why do we allow it? IE: why doesn't TRUNCATE exclusive lock the relation? It *does*. The problem is that the *other* transaction that's reading the relation can still

[HACKERS] Fixing bug #8228 (set-valued function called in context that cannot accept a set)

2014-01-06 Thread Tom Lane
I kinda forgot about this bug when I went off on vacation: http://www.postgresql.org/message-id/e1uncv4-0007of...@wrigleys.postgresql.org The way to fix it seems to be to do static prechecking of whether a function's input arguments can return sets, rather than relying on their behavior during

Re: [HACKERS] Get more from indices.

2014-01-06 Thread Etsuro Fujita
Tom Lane wrote: Etsuro Fujita fujita.ets...@lab.ntt.co.jp writes: [ pathkey_and_uniqueindx_v7_20131203.patch ] I started to look at this patch. I don't understand the reason for the foreach loop in index_pathkeys_are_extensible (and the complete lack of comments in the patch isn't

[HACKERS] [HITB-Announce] HITB Magazine Issue 10 Out Now

2014-01-06 Thread Hafez Kamal
Issue #10 is now available! Hello readers and welcome to the somewhat overdue Issue 010 of HITB Magazine. As they say, better late than never! Since the last issue, we've also changed the HITB Security Conference Call for Papers submission guidelines to now require speakers to submit a research

  1   2   >