Re: [HACKERS] Retain dynamic shared memory segments for postmaster lifetime

2014-02-12 Thread Kyotaro HORIGUCHI
Hello, Please find new version of patch attached with this mail containing above changes. This patch applies cleanly on current HEAD and build completed successfully on both Windows and Linux. (but master needed to be rewinded to some time ago for some compile errors.) This works correctly as

Re: [HACKERS] Patch: compiling the docs under Gentoo

2014-02-12 Thread Christian Kruse
Hi, On Tuesday 11 February 2014 16:04:30 Peter Eisentraut wrote: On 1/30/14, 2:42 AM, Christian Kruse wrote: +Since Gentoo often supports different versions of a package to be +installed you have to tell the PostgreSQL build environment where the +Docbook DTD is located:

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Tomonari Katsumata
Hi Heikki, I need PostgreSQL9.3 which fixed this problem. It didn't happen in PostgreSQL9.2, so I agree with your proposal which changes are done against 93_STABLE and master. Can you fix this in next release(9.3.3)? Tomonari Katsumata (2014/01/13 20:16), Heikki Linnakangas

Re: [HACKERS] Retain dynamic shared memory segments for postmaster lifetime

2014-02-12 Thread Amit Kapila
On Wed, Feb 12, 2014 at 2:02 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, Please find new version of patch attached with this mail containing above changes. This patch applies cleanly on current HEAD and build completed successfully on both Windows and Linux. (but

Re: [HACKERS] Patch: show xid and xmin in pg_stat_activity and pg_stat_replication

2014-02-12 Thread Andres Freund
On 2014-02-11 09:15:45 -0500, Robert Haas wrote: If I understand correctly, modifying PgBackendStatus adds additional fields to the shared memory data structure that are never used and will be returned by functions like pgstat_fetch_stat_beentry() unitialized. That seems both inefficient and

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Christoph Berg
Re: Heikki Linnakangas 2014-01-13 52d3caff.3010...@vmware.com Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server is still up and running, and

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-12 Thread MauMau
From: Andres Freund and...@2ndquadrant.com It's x86, right? Then it's unlikely to be actual unordered memory accesses, but if the compiler reordered: LOG_LWDEBUG(LWLockRelease, T_NAME(l), T_ID(l), release waiter); proc = head; head = proc-lwWaitLink; proc-lwWaitLink = NULL;

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-12 Thread Andres Freund
On 2014-02-12 20:55:32 +0900, MauMau wrote: Dump of assembler code for function LWLockRelease: could you try if you get more readable dumps by using disassemble/m? That might at least print line numbers if you have debug info installed. Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-12 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-02-02 23:50:40 +0900, Fujii Masao wrote: Right. If standby_mode is enabled, checkpoint_segment can trigger the restartpoint. But the problem is that the timing of restartpoint depends on not only the checkpoint parameters (i.e.,

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-12 Thread MauMau
From: Andres Freund and...@2ndquadrant.com could you try if you get more readable dumps by using disassemble/m? That might at least print line numbers if you have debug info installed. OK, I'll try that tomorrow. However, the debug info is not available, because they use PostgreSQL built by

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-12 Thread Andres Freund
On 2014-02-12 21:23:54 +0900, MauMau wrote: Maybe we could consider in that direction, but there is a problem. Archive recovery slows down compared to 9.1, because of repeated restartpoints. Archive recovery should be as fast as possible, because it typically applies dozens or hundreds of WAL

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Greg Stark
So I think I've come up with a scenario that could cause this. I don't think it's exactly what happened here but maybe something analogous happened with our base backup restore. On the primary you extend a table a bunch, including adding new segments, but crash before committing (or

[HACKERS] Could you include bug fix for Windows ASLR issue in the next release?

2014-02-12 Thread MauMau
Could you include this simple patch in the upcoming minor release 9.3.3/9.2.7/9.1.12? This is very scary. This problem would cause much random trouble among Windows users. https://commitfest.postgresql.org/action/patch_view?id=1409 Regards MauMau -- Sent via pgsql-hackers mailing list

Re: [HACKERS] DATE type output does not follow datestyle parameter

2014-02-12 Thread MauMau
From: Bruce Momjian br...@momjian.us On Mon, Dec 2, 2013 at 10:22:47PM +0900, MauMau wrote: So, my suggestion is to just add the following sentence right after the above one. The Postgres style is an exception: the output of the date type is either MM-DD- (e.g. 12-17-1997) or DD-MM-

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-12 Thread Florian Pflug
On Feb12, 2014, at 12:55 , MauMau maumau...@gmail.com wrote: From: Andres Freund and...@2ndquadrant.com It's x86, right? Then it's unlikely to be actual unordered memory accesses, but if the compiler reordered: LOG_LWDEBUG(LWLockRelease, T_NAME(l), T_ID(l), release waiter); proc = head;

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-12 Thread knizhnik
On 02/12/2014 05:42 PM, Florian Pflug wrote: On Feb12, 2014, at 12:55 , MauMau maumau...@gmail.com wrote: From: Andres Freund and...@2ndquadrant.com It's x86, right? Then it's unlikely to be actual unordered memory accesses, but if the compiler reordered: LOG_LWDEBUG(LWLockRelease,

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-12 Thread Ants Aasma
On Wed, Feb 12, 2014 at 4:04 PM, knizhnik knizh...@garret.ru wrote: Even if reordering was not done by compiler, it still can happen while execution. There is no warranty that two subsequent assignments will be observed by all CPU cores in the same order. The x86 memory model (total store

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Greg Stark
So here's my attempt to rewrite this logic. I ended up refactoring a bit because I found it unnecessarily confusing having the mode branches in several places. I think it's much clearer just having two separate pieces of logic for RBM_NEW and the extension cases since all they have in common is

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-12 Thread Bruce Momjian
On Wed, Feb 12, 2014 at 10:02:32AM +0530, Amit Kapila wrote: By issue, I assume you mean to say, which compression algorithm is best for this patch. For this patch, currently we have 2 algorithm's for which results have been posted. As far as I understand Heikki is pretty sure that the latest

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-12 Thread Kohei KaiGai
2014-02-12 14:59 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: I reviewed all the three patches. The first 1 and 2 core PostgreSQL patches are fine. And I have comments in the third patch related to cache scan. Thanks for your volunteering. 1. +# contrib/dbcache/Makefile Makefile

Re: [HACKERS] Changeset Extraction v7.5

2014-02-12 Thread Andres Freund
On 2014-02-11 11:22:24 -0500, Robert Haas wrote: + context = AllocSetContextCreate(CurrentMemoryContext, + Changeset Extraction Context, + ALLOCSET_DEFAULT_MINSIZE, + ALLOCSET_DEFAULT_INITSIZE, + ALLOCSET_DEFAULT_MAXSIZE); I have my doubts about whether it's wise to make this

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: Great, that's what I was hoping to see - proper errors where we've omitted things, not silent miscompilation. And, just to make sure that ain't nobody happy ... brolga, our one operational Cygwin animal, is still building without complaints. We really

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: Great, that's what I was hoping to see - proper errors where we've omitted things, not silent miscompilation. And, just to make sure that ain't nobody happy ... brolga, our one operational Cygwin

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of it seems to be on

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big

Re: [HACKERS] DATE type output does not follow datestyle parameter

2014-02-12 Thread Bruce Momjian
On Wed, Feb 12, 2014 at 10:08:26PM +0900, MauMau wrote: From: Bruce Momjian br...@momjian.us On Mon, Dec 2, 2013 at 10:22:47PM +0900, MauMau wrote: So, my suggestion is to just add the following sentence right after the above one. The Postgres style is an exception: the output of the date

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On 2014-02-12 11:39:43 -0500, Tom Lane wrote: I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. The manpage also has a --disable-auto-import, but doesn't document wheter enabling or disabling is the default...

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Bruce Momjian
On Wed, Feb 12, 2014 at 11:39:43AM -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On 2014-02-12 11:50:41 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:39:43 -0500, Tom Lane wrote: I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. The manpage

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Some release notes I just found seem to suggest it is http://sourceware.org/binutils/docs-2.17/ld/WIN32.html : This feature is enabled with the `--enable-auto-import' command-line option, although it is enabled by default on cygwin/mingw. Curious

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:39:43 -0500, Tom Lane wrote: I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. The manpage also has a --disable-auto-import, but doesn't document

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
Greg Stark st...@mit.edu writes: So here's my attempt to rewrite this logic. I ended up refactoring a bit because I found it unnecessarily confusing having the mode branches in several places. I think it's much clearer just having two separate pieces of logic for RBM_NEW and the extension

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
Greg Stark st...@mit.edu writes: So I think I've come up with a scenario that could cause this. I don't think it's exactly what happened here but maybe something analogous happened with our base backup restore. I agree it seems like a good idea for XLogReadBufferExtended to defend itself

Re: [HACKERS] Terminating pg_basebackup background streamer

2014-02-12 Thread Magnus Hagander
On Mon, Feb 10, 2014 at 7:39 PM, Magnus Hagander mag...@hagander.netwrote: On Mon, Feb 10, 2014 at 7:29 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 02/09/2014 02:17 PM, Magnus Hagander wrote: If an error occurs in the foreground (backup) process of pg_basebackup, and we exit

Re: [HACKERS] truncating pg_multixact/members

2014-02-12 Thread Alvaro Herrera
In this new version, I added a couple of fields to VacuumStmt node. How strongly do we feel this would cause an ABI break? Would we be more comfortable if I put them at the end of the struct for 9.3 instead? Do we expect third-party code to be calling vacuum()? Also, AutoVacOpts (used as part

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Marco Atzeri
On 12/02/2014 17:26, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of it seems to

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: On 12/02/2014 17:26, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? If I am not wrong

Re: [HACKERS] truncating pg_multixact/members

2014-02-12 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: In this new version, I added a couple of fields to VacuumStmt node. How strongly do we feel this would cause an ABI break? Would we be more comfortable if I put them at the end of the struct for 9.3 instead? In the past we've usually added such

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Greg Stark
On Wed, Feb 12, 2014 at 5:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: How about the attached instead? This does possibly allocate an extra block past the target block. I'm not sure how surprising that would be for the rest of the code. For what it's worth I've confirmed the bug in wal-e caused

Re: [HACKERS] WIP patch for Todo Item : Provide fallback_application_name in contrib/pgbench, oid2name, and dblink

2014-02-12 Thread Robert Haas
On Mon, Feb 10, 2014 at 12:14 PM, Jeff Janes jeff.ja...@gmail.com wrote: Presumably whatever behavior difference you are seeing is somehow related to the use of PQconnectdbParams() rather than PQsetdbLogin(), but the fine manual does not appear to document a different between those functions

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
I wrote: Greg Stark st...@mit.edu writes: (Or maybe the hot backup process could just catch the files in this state if a table is rapidly growing and it doesn't take care to avoid picking up new files that appear after it starts?) That's a possible explanation I guess, but it doesn't seem

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Wed, Feb 12, 2014 at 5:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: How about the attached instead? This does possibly allocate an extra block past the target block. I'm not sure how surprising that would be for the rest of the code. Should be fine; we could

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Marco Atzeri
On 12/02/2014 19:19, Andres Freund wrote: On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: On 12/02/2014 17:26, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Marco Atzeri marco.atz...@gmail.com writes: On 12/02/2014 19:19, Andres Freund wrote: On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: About PGDLLIMPORT , my build log is full of warning: ‘optarg’ redeclared without dllimport attribute: previous dllimport ignored That should be fixed

Re: [HACKERS] dynamic shared memory and locks

2014-02-12 Thread Robert Haas
On Mon, Feb 10, 2014 at 7:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Does it make another problem if dsm_detach() also releases lwlocks being allocated on the dsm segment to be released? Lwlocks being held is tracked in the held_lwlocks[] array; its length is usually 100. In case when

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Greg Stark
On Wed, Feb 12, 2014 at 6:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark st...@mit.edu writes: On Wed, Feb 12, 2014 at 5:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: How about the attached instead? This does possibly allocate an extra block past the target block. I'm not sure how

[HACKERS] issue with gininsert under very high load

2014-02-12 Thread Andrew Dunstan
A client of ours encountered this problem when upgrading (via slony) from 8.4 to 9.2, and then from 8.4 to to 9.3. The application is a telephony app that inserts call records at pretty hight volume in the busiest part of the day, which usually starts around 10.00 am to 11.00 am US eastern

Re: [HACKERS] issue with gininsert under very high load

2014-02-12 Thread Andres Freund
On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I found that a number of processes were locked waiting for one wedged process to end its transaction, which never happened (this transaction should normally take milliseconds). oprofile revealed that postgres was spending 87%

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Wed, Feb 12, 2014 at 6:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark st...@mit.edu writes: This does possibly allocate an extra block past the target block. I'm not sure how surprising that would be for the rest of the code. Should be fine; we could

Re: [HACKERS] issue with gininsert under very high load

2014-02-12 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: There's previously been talk about changing the limits to something more reasonable but it got stalled in bikeshedding IIRC. As I recall, there was argument that we didn't really need a new GUC for this (which was the proposal) but rather just

Re: [HACKERS] issue with gininsert under very high load

2014-02-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I found that a number of processes were locked waiting for one wedged process to end its transaction, which never happened (this transaction should normally take milliseconds).

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
I wrote: Greg Stark st...@mit.edu writes: WAL-E actually didn't restore a whole 1GB file due to a transient S3 problem, in fact a bunch of them. Hah. Okay, I think we can write this issue off as closed then. Oh, wait a minute. It's not just a matter of whether we find the right block: we

Re: [HACKERS] issue with gininsert under very high load

2014-02-12 Thread Andrew Dunstan
On 02/12/2014 02:50 PM, Andres Freund wrote: On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I found that a number of processes were locked waiting for one wedged process to end its transaction, which never happened (this transaction should normally take milliseconds).

Re: [HACKERS] truncating pg_multixact/members

2014-02-12 Thread Alvaro Herrera
Tom Lane escribió: Alvaro Herrera alvhe...@2ndquadrant.com writes: In this new version, I added a couple of fields to VacuumStmt node. How strongly do we feel this would cause an ABI break? Would we be more comfortable if I put them at the end of the struct for 9.3 instead? In the past

Re: [HACKERS] issue with gininsert under very high load

2014-02-12 Thread Andres Freund
On February 12, 2014 9:33:38 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I found that a number of processes were locked waiting for one wedged process to end its transaction, which

Re: [HACKERS] issue with gininsert under very high load

2014-02-12 Thread Heikki Linnakangas
On 02/12/2014 10:50 PM, Andres Freund wrote: On February 12, 2014 9:33:38 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I found that a number of processes were locked waiting for

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Peter Eisentraut
On 2/11/14, 7:04 PM, Craig Ringer wrote: I don't see any use for that with plperl, but it might be a valid thing to be doing for (e.g.) hstore.dll. Though you can't really link to it from another module anyway, you have to go through the fmgr to get access to its symbols at rutime, so we can

Re: [HACKERS] Terminating pg_basebackup background streamer

2014-02-12 Thread Peter Eisentraut
On 2/12/14, 12:47 PM, Magnus Hagander wrote: Since there were no other objections, I've applied this patch. I'm getting a compiler warning: pg_basebackup.c:105:3: error: implicit declaration of function 'kill' [-Werror=implicit-function-declaration] -- Sent via pgsql-hackers mailing list

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: On 2/11/14, 7:04 PM, Craig Ringer wrote: I don't see any use for that with plperl, but it might be a valid thing to be doing for (e.g.) hstore.dll. Though you can't really link to it from another module anyway, you

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
I wrote: What I think we probably want to do is forcibly cause the target page to exist, using a P_NEW loop like what I committed, and then decide on the basis of whether it's all-zeroes whether to consider it invalid or not. This seems sane on the grounds that it's just the extension to the

Re: [HACKERS] Terminating pg_basebackup background streamer

2014-02-12 Thread Magnus Hagander
On Wed, Feb 12, 2014 at 10:28 PM, Peter Eisentraut pete...@gmx.net wrote: On 2/12/14, 12:47 PM, Magnus Hagander wrote: Since there were no other objections, I've applied this patch. I'm getting a compiler warning: pg_basebackup.c:105:3: error: implicit declaration of function 'kill'

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither

Re: [HACKERS] memory usage of pg_upgrade

2014-02-12 Thread Bruce Momjian
On Mon, Feb 3, 2014 at 09:14:10PM -0500, Bruce Momjian wrote: On Mon, Sep 9, 2013 at 07:39:00PM -0400, Bruce Momjian wrote: In the case of tablespaces, I should have thought you could keep a hash table of the names and just store an entry id in the table structure. But that's just my

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Peter Eisentraut
On 2/12/14, 4:30 PM, Andres Freund wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: There are cases where one module needs symbols from another directly. Would that be affected by this? It works reasonably well on other platforms. Of course, we can barely build extension modules on Windows, so maybe this is a bit much to ask. But

Re: [DOCS] [HACKERS] Viability of text HISTORY/INSTALL/regression README files (was Re: [COMMITTERS] pgsql: Document a few more regression test hazards.)

2014-02-12 Thread Peter Eisentraut
On 2/8/14, 4:41 PM, Tom Lane wrote: diff --git a/HISTORY b/HISTORY index ...360c7f6 . *** a/HISTORY --- b/HISTORY *** *** 0 --- 1,6 + Release notes for all versions of PostgreSQL can be found on-line at + http://www.postgresql.org/docs/devel/static/release.html

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On February 12, 2014 10:35:06 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: There are cases where one module needs symbols from another directly. Would that be affected by

Re: [HACKERS] Terminating pg_basebackup background streamer

2014-02-12 Thread Andres Freund
On February 12, 2014 10:34:47 PM CET, Magnus Hagander mag...@hagander.net wrote: On Wed, Feb 12, 2014 at 10:28 PM, Peter Eisentraut pete...@gmx.net wrote: On 2/12/14, 12:47 PM, Magnus Hagander wrote: Since there were no other objections, I've applied this patch. I'm getting a compiler

Re: [HACKERS] Terminating pg_basebackup background streamer

2014-02-12 Thread Peter Eisentraut
On 2/12/14, 4:34 PM, Magnus Hagander wrote: On Wed, Feb 12, 2014 at 10:28 PM, Peter Eisentraut pete...@gmx.net mailto:pete...@gmx.net wrote: On 2/12/14, 12:47 PM, Magnus Hagander wrote: Since there were no other objections, I've applied this patch. I'm getting a compiler

Re: [HACKERS] PoC: Partial sort

2014-02-12 Thread Marti Raudsepp
On Mon, Feb 10, 2014 at 8:59 PM, Alexander Korotkov aekorot...@gmail.com wrote: Done. Patch is splitted. Thanks! I think the 1st patch now has a bug in initial_cost_mergejoin; you still pass the presorted_keys argument to cost_sort, making it calculate a partial sort cost, but generated plans

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

2014-02-12 Thread Bruce Momjian
On Wed, Sep 11, 2013 at 02:10:45PM -0400, Robert Haas wrote: On Tue, Sep 10, 2013 at 11:33 PM, Noah Misch n...@leadboat.com wrote: I'm thinking to preserve postmaster.pid at immediate shutdown in all released versions, but I'm less sure about back-patching a change to make

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Craig Ringer
On 02/12/2014 02:31 PM, Inoue, Hiroshi wrote: Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Wow. Has anyone been in touch with the Python package

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Craig Ringer
On 02/13/2014 12:39 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other

Re: [DOCS] [HACKERS] Viability of text HISTORY/INSTALL/regression README files (was Re: [COMMITTERS] pgsql: Document a few more regression test hazards.)

2014-02-12 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 2/8/14, 4:41 PM, Tom Lane wrote: + Release notes for all versions of PostgreSQL can be found on-line at + http://www.postgresql.org/docs/devel/static/release.html Should be current instead of devel? + + In a distribution file set, release notes

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Craig Ringer
On 02/13/2014 05:23 AM, Peter Eisentraut wrote: On 2/11/14, 7:04 PM, Craig Ringer wrote: I don't see any use for that with plperl, but it might be a valid thing to be doing for (e.g.) hstore.dll. Though you can't really link to it from another module anyway, you have to go through the fmgr to

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: On 02/12/2014 02:31 PM, Inoue, Hiroshi wrote: Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Wow. Has anyone

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Craig Ringer
On 02/13/2014 05:35 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-12 Thread Haribabu Kommi
On Thu, Feb 13, 2014 at 2:42 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2014-02-12 14:59 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: 7. In ccache_find_tuple function this Assert(i_min + 1 cchunk-ntups); can go wrong when only one tuple present in the block with the equal item

Re: [HACKERS] Making strxfrm() blobs in indexes work

2014-02-12 Thread Martijn van Oosterhout
On Sun, Feb 02, 2014 at 05:09:06PM -0800, Peter Geoghegan wrote: However, it also occurs to me that strxfrm() blobs have another useful property: We (as, say, the author of an equality operator on text, an operator intended for a btree operator class) *can* trust a strcmp()'s result on blobs,

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Craig Ringer
On 02/13/2014 05:35 AM, Peter Eisentraut wrote: On 2/12/14, 4:30 PM, Andres Freund wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several

Re: [HACKERS] Making strxfrm() blobs in indexes work

2014-02-12 Thread Peter Geoghegan
On Wed, Feb 12, 2014 at 3:30 PM, Martijn van Oosterhout klep...@svana.org wrote: (A bit late to the party). This idea has come up before and the most annoying thing is that braindead strxfrm api. Namely, to strxfrm a large strings you need to strxfrm it completely even if you only want the

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: However, from the reading I've done recently, I'm pretty sure that if you fail to declare __declspec(dllimport) on the importing side, you actually land up statically linking to a thunk function that in turn calls the real function in the DLL. So it

Re: [HACKERS] Retain dynamic shared memory segments for postmaster lifetime

2014-02-12 Thread Kyotaro HORIGUCHI
Hello, I've marked this patch as 'Ready for committer'. To be honest, I see no harm in changing the name as per your suggestion, as it can improve segment naming for dynamic shared memory segments, however there is no clear problem with current name as well, so I don't want to change in

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Andres Freund
On 2014-02-13 07:58:09 +0800, Craig Ringer wrote: On 02/13/2014 05:35 AM, Peter Eisentraut wrote: On 2/12/14, 4:30 PM, Andres Freund wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Craig Ringer
On 02/13/2014 08:26 AM, Andres Freund wrote: It gets worse, too. Say you want hstore to export a couple of symbols. Those symbols must be __declspec(dllexport) while everything else in headers must be __declspec(dllimport). This means you can't just use PGDLLIMPORT. You must define a

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-12 Thread Hiroshi Inoue
(2014/02/12 12:28), Inoue, Hiroshi wrote: (2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting

Re: [HACKERS] psql should show disabled internal triggers

2014-02-12 Thread Bruce Momjian
On Thu, Nov 21, 2013 at 11:59:51PM -0200, Fabrízio de Royes Mello wrote: On Fri, Oct 25, 2013 at 3:37 PM, fabriziomello fabriziome...@gmail.com wrote: On 2013-09-18 15:15:55 +0200, Bernd Helmle wrote: On 2013-09-18 15:15:55 +0200, Bernd Helmle wrote: --On 18. September 2013 13:52:29

Re: [HACKERS] Dead code or buggy code?

2014-02-12 Thread Bruce Momjian
On Fri, Sep 20, 2013 at 12:13:18AM +0100, Greg Stark wrote: So I'm just going to make the code defensive and assume NULL is possible when if maybe it isn't. In case it's not clear, this is one of the thing's Xi Wang's took picked up. There not to many but it turns out they are indeed not all

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Greg Stark
On Wed, Feb 12, 2014 at 8:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Oh, wait a minute. It's not just a matter of whether we find the right block: we also have to consider whether XLogReadBufferExtended will apply the right mode behavior. Currently, it supposes that all pages past the

Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones

2014-02-12 Thread Bruce Momjian
On Fri, Nov 15, 2013 at 10:40:20AM +0200, Heikki Linnakangas wrote: The BTRFS_IOC_CLONE ioctl operates on file level and can be used to clone files anywhere in a btrfs filesystem. Hmm, you can also do cp --reflog -r data92 data-tmp I think you mean --reflink here. pg_upgrade --link

Re: [HACKERS] Prevent pg_basebackup -Fp -D -?

2014-02-12 Thread Bruce Momjian
On Thu, Oct 3, 2013 at 06:50:57AM +0200, Magnus Hagander wrote: On Oct 3, 2013 2:47 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Oct 2, 2013 at 11:31 PM, Magnus Hagander mag...@hagander.net wrote: Right now, if you use pg_basebackup -Ft -D - you get a

Re: [HACKERS] psql should show disabled internal triggers

2014-02-12 Thread Fabrízio de Royes Mello
On Thu, Feb 13, 2014 at 12:04 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Nov 21, 2013 at 11:59:51PM -0200, Fabrízio de Royes Mello wrote: On Fri, Oct 25, 2013 at 3:37 PM, fabriziomello fabriziome...@gmail.com wrote: On 2013-09-18 15:15:55 +0200, Bernd Helmle wrote: On

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-12 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Wed, Feb 12, 2014 at 8:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Oh, wait a minute. It's not just a matter of whether we find the right block: we also have to consider whether XLogReadBufferExtended will apply the right mode behavior. Currently, it

Re: [HACKERS] Row-security on updatable s.b. views

2014-02-12 Thread Craig Ringer
On 02/11/2014 08:19 PM, Yeb Havinga wrote: I compared output of psql -ef of the minirim.sql script posted earlier in http://www.postgresql.org/message-id/52f54927.1040...@gmail.com between v4 and v7. Not everything is ok. +psql:/home/m/minirim2.sql:409: ERROR: attribute 6 has wrong type

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-12 Thread Amit Kapila
On Wed, Feb 12, 2014 at 8:19 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Feb 12, 2014 at 10:02:32AM +0530, Amit Kapila wrote: I think 99.9% of users are never going to adjust this so we had better choose something we are happy to enable for effectively everyone. In my reading,

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-12 Thread Kouhei Kaigai
8. I am not able to find a protection mechanism in insert/delete and etc of a tuple in Ttree. As this is a shared memory it can cause problems. For design simplification, I put a giant lock per columnar-cache. So, routines in cscan.c acquires exclusive

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-12 Thread Claudio Freire
On Thu, Feb 13, 2014 at 1:20 AM, Amit Kapila amit.kapil...@gmail.com wrote: Here one of the improvements which can be done is that after prefix-suffix match, instead of going byte-by-byte copy as per LZ format we can directly copy all the remaining part of tuple but I think that would require

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-12 Thread Amit Kapila
On Thu, Feb 13, 2014 at 10:07 AM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, Feb 13, 2014 at 1:20 AM, Amit Kapila amit.kapil...@gmail.com wrote: Here one of the improvements which can be done is that after prefix-suffix match, instead of going byte-by-byte copy as per LZ format we can

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-12 Thread Robert Haas
On Tue, Feb 11, 2014 at 11:37 AM, Bruce Momjian br...@momjian.us wrote: Yes, that was my point. I though the compression of full-page images was a huge win and that compression was pretty straight-forward, except for the compression algorithm. If the compression algorithm issue is resolved,

  1   2   >