Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-12 Thread Michael Paquier
On Thu, Jun 11, 2015 at 5:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Jun 11, 2015 at 2:14 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jun 11, 2015 at 1:51 AM, Fujii Masao masao.fu...@gmail.com wrote: Shouldn't pg_rewind ignore that failure of operation? If the

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-12 Thread Abhijit Menon-Sen
At 2015-06-11 14:38:03 +0900, langote_amit...@lab.ntt.co.jp wrote: On the other hand, I don't like the idea of doing (3) by adding command line arguments to pg_basebackup and adding a new option to the command. I don't think that level of flexibility is justified; it would also make it

Re: [HACKERS] Comfortably check BackendPID with psql

2015-06-12 Thread Naoya Anzai
Not a big fan of that abbreviation itself. What I'd wondered about instead - and actually had patched into my psql at some point - is adding an appropriate escape to psql's PROMPT. I think that'd serve your purpose as well? +3.14159; that would be hugely helpful when using gdb. You

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-12 Thread Fujii Masao
On Fri, Jun 12, 2015 at 3:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jun 11, 2015 at 5:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Jun 11, 2015 at 2:14 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jun 11, 2015 at 1:51 AM, Fujii Masao

Re: [HACKERS] The Future of Aggregation

2015-06-12 Thread David Rowley
On 11 June 2015 at 01:39, Kevin Grittner kgri...@ymail.com wrote: David Rowley david.row...@2ndquadrant.com wrote: /* setup */ create table millionrowtable as select generate_series(1,100)::numeric as x; /* test 1 */ SELECT sum(x) / count(x) from millionrowtable; /* test 2 */

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-12 Thread Michael Meskes
On Mon, Jun 08, 2015 at 10:50:25PM +0900, Michael Paquier wrote: And the caller needs to be sure that str actually allocates enough space.. That's not directly ECPG's business, still it feels But there is no way for us to fix this as we want to implement the API as defined in Informix.

Re: [HACKERS] Aggregate Supporting Functions

2015-06-12 Thread David Rowley
On 10 June 2015 at 02:26, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: David Rowley david.row...@2ndquadrant.com wrote: [ avoid duplicate calculations for related aggregates ] From the information you have proposed storing, with cost factors associated

[HACKERS] [Proposal] Progress bar for pg_dump/pg_restore

2015-06-12 Thread Taiki Kondo
Hi, all. I am newbie in hackers. I have an idea from my point of view as one user, I would like to propose the following. Progress bar for pg_dump / pg_restore = Motivation -- pg_dump and pg_restore show nothing if users don't specify verbose (-v)

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Petr Jelinek
Hi, + listitem + para +Add typeJSONB/ functions functionjsonb_set()/ and +functionjsonb_pretty/ (Dmitry Dolgov, Andrew Dunstan) + /para + /listitem I believe I should be 3rd author for this one as I rewrote large parts of this functionality as part of the

Re: [HACKERS] Why does replication need the old history file?

2015-06-12 Thread Fujii Masao
On Fri, Jun 12, 2015 at 5:18 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Jun 12, 2015 at 4:56 AM, Josh Berkus j...@agliodbs.com wrote: Hackers, Sequence of events: 1. PITR backup of server on timeline 2. 2. Restored the backup to a new server, new-master. 3. Restored the

Re: [HACKERS] On columnar storage

2015-06-12 Thread Alvaro Herrera
Merlin Moncure wrote: On Thu, Jun 11, 2015 at 6:03 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: We hope to have a chance to discuss this during the upcoming developer unconference in Ottawa. Here are some preliminary ideas to shed some light on what we're trying to do. Quick

Re: [HACKERS] Hash index creation warning

2015-06-12 Thread Thom Brown
On 18 October 2014 at 15:36, Bruce Momjian br...@momjian.us wrote: On Fri, Oct 17, 2014 at 02:36:55PM -0400, Bruce Momjian wrote: On Fri, Oct 17, 2014 at 12:56:52PM -0400, Tom Lane wrote: David G Johnston david.g.johns...@gmail.com writes: The question is whether we explain the implications

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-12 Thread Peter Geoghegan
On Thu, Jun 4, 2015 at 5:43 PM, Peter Geoghegan p...@heroku.com wrote: BTW, there is a bug here -- strtol() needs additional defenses [1] (before casting to int): postgres=# select jsonb_set('[1, 2, 3, 4, 5,6,7,8,9,10,11,12,13,14,15,16,17,18]', '{9223372036854775806}'::text[], 'Input

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-12 Thread Peter Geoghegan
On Fri, Jun 12, 2015 at 12:26 PM, Andrew Dunstan and...@dunslane.net wrote: Here's some code for the count piece of that. Thanks. I'll look into integrating this with what I have. BTW, on reflection I'm not so sure about my decision to not touch the logic within jsonb_delete_idx() (commit

Re: [HACKERS] On columnar storage

2015-06-12 Thread Michael Nolan
On Thu, Jun 11, 2015 at 7:03 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: We hope to have a chance to discuss this during the upcoming developer unconference in Ottawa. Here are some preliminary ideas to shed some light on what we're trying to do. I've been trying to figure out a

Re: [HACKERS] The Future of Aggregation

2015-06-12 Thread Kevin Grittner
David Rowley david.row...@2ndquadrant.com wrote: I am a little curious what sort of machine you're running on, because my i7 is much slower. I ran a few other tests with your table for perspective. Assert enabled build? Mystery solved. Too often I forget to reconfigure with optimization

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-12 Thread Sehrope Sarkuni
The JDBC driver tries to handle this by estimating how much data has been buffered. It mainly comes up when executing batch INSERTS as a large number of statements may be sent to the backend prior to reading back any results. There's a nice write up of the potential deadlock and the driver's

Re: [HACKERS] Time to fully remove heap_formtuple() and friends?

2015-06-12 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: Attached patch actually removes heap_formtuple() and friends. Does this seem worthwhile? Seems reasonable, but at this point I would say this is 9.6 material; third-party-module authors have enough to do with the API breaks we've already created for 9.5.

Re: [HACKERS] Time to fully remove heap_formtuple() and friends?

2015-06-12 Thread Peter Geoghegan
On Fri, Jun 12, 2015 at 8:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan p...@heroku.com writes: Attached patch actually removes heap_formtuple() and friends. Does this seem worthwhile? Seems reasonable, but at this point I would say this is 9.6 material; third-party-module

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-12 Thread Michael Paquier
On Fri, Jun 12, 2015 at 10:01 PM, Michael Meskes mes...@postgresql.org wrote: On Mon, Jun 08, 2015 at 10:50:25PM +0900, Michael Paquier wrote: Right, that's because they were developed before free received the safeguard, or the programmer simply didn't know at that point in time. Unless I'm

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-12 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: On Fri, Jun 12, 2015 at 10:01 PM, Michael Meskes mes...@postgresql.org wrote: More seriously, though, does anyone know of any platform where free(NULL) is *not* a noop? I recall reading that some past versions of SunOS crashed, but it is

Re: [HACKERS] Incompatible trig error handling

2015-06-12 Thread Noah Misch
On Wed, Apr 29, 2015 at 05:11:48PM -0700, Tom Lane wrote: John Gorman johngorm...@gmail.com writes: Two of the trigonometry functions have differing error condition behavior between Linux and OSX. The Linux behavior follows the standard set by the other trig functions. We have never

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Bruce Momjian
On Thu, Jun 11, 2015 at 01:31:01PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Jun 11, 2015 at 11:32 AM, Bruce Momjian br...@momjian.us wrote: Improve hash creation and lookup performance (Tomas Vondra, Teodor Sigaev, Tom Lane, Robert Haas) I suggest haveing

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Bruce Momjian
On Thu, Jun 11, 2015 at 09:02:35PM +0200, Magnus Hagander wrote: On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus j...@agliodbs.com wrote: On 06/10/2015 09:50 PM, Amit Kapila wrote: Also shall we mention about below in Migrations to 9.5 section pg_basebackup will not not work

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Bruce Momjian
On Thu, Jun 11, 2015 at 01:27:23PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Thu, Jun 11, 2015 at 05:16:07PM +1200, David Rowley wrote: Would you also be able to mention something about�f15821e and�d222585 ? I am going to defer to Tom on that. I have added

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-12 Thread Thomas Munro
Hi Since the multixact equivalent of this problem[1] fell through the cracks on the multixact mega-thread, here is an updated patch that addresses this problem for both pg_subtrans and pg_multixact/offsets using the same approach: always step back one multixact/xid (rather than doing so only if

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Bruce Momjian
On Thu, Jun 11, 2015 at 05:17:54PM -0400, Robert Haas wrote: On Thu, Jun 11, 2015 at 4:23 PM, Peter Geoghegan p...@heroku.com wrote: Secondly, Robert didn't credit himself as an author in his commit message for the abbreviated keys infrastructure + text opclass support *at all*. However, I

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Bruce Momjian
On Fri, Jun 12, 2015 at 02:47:22PM +0200, Petr Jelinek wrote: Hi, + listitem + para +Add typeJSONB/ functions functionjsonb_set()/ and +functionjsonb_pretty/ (Dmitry Dolgov, Andrew Dunstan) + /para + /listitem I believe I should be 3rd author for

Re: [HACKERS] [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-12 Thread Steve Kehlet
Just wanted to report that I rolled back my VM to where it was with 9.4.2 installed and it wouldn't start. I installed 9.4.4 and now it starts up just fine: 2015-06-12 16:05:58 PDT [6453]: [1-1] LOG: database system was shut down at 2015-05-27 13:12:55 PDT 2015-06-12 16:05:58 PDT [6453]: [2-1]

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-12 Thread Peter Geoghegan
On Fri, Jun 12, 2015 at 4:31 PM, Andrew Dunstan and...@dunslane.net wrote: OK, pushed, although you'd have to be trying really hard to break this. Still, it's reasonable to defend against. I was trying really hard. :-) Thanks -- Peter Geoghegan -- Sent via pgsql-hackers mailing list

[HACKERS] Time to fully remove heap_formtuple() and friends?

2015-06-12 Thread Peter Geoghegan
Commit 902d1cb3, made in 2008, established that the functions heap_formtuple(), heap_modifytuple(), and heap_deformtuple() were deprecated. The commit also actually removed those routines, replacing them with simple wrappers around their real replacements, which are spelled slightly differently

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Amit Kapila
On Sat, Jun 13, 2015 at 7:47 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Jun 11, 2015 at 09:02:35PM +0200, Magnus Hagander wrote: On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus j...@agliodbs.com wrote: On 06/10/2015 09:50 PM, Amit Kapila wrote: Also shall we mention about

Re: [HACKERS] 9.5 release notes

2015-06-12 Thread Bruce Momjian
On Fri, Jun 12, 2015 at 12:49:11PM +0900, Fujii Masao wrote: On Thu, Jun 11, 2015 at 1:15 PM, Bruce Momjian br...@momjian.us wrote: I have committed the first draft of the 9.5 release notes. You can view the output here: http://momjian.us/pgsql_docs/release-9-5.html and it

Re: [HACKERS] On columnar storage

2015-06-12 Thread Merlin Moncure
On Thu, Jun 11, 2015 at 6:03 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: We hope to have a chance to discuss this during the upcoming developer unconference in Ottawa. Here are some preliminary ideas to shed some light on what we're trying to do. Quick thought. We already support out

Re: [HACKERS] git push hook to check for outdated timestamps

2015-06-12 Thread Andrew Dunstan
On 06/12/2015 09:31 AM, Robert Haas wrote: Could we update our git hook to refuse a push of a new commit whose timestamp is more than, say, 24 hours in the past? Our commit history has some timestamps in it now that are over a month off, and it's really easy to do, because when you rebase a

Re: [HACKERS] Further issues with jsonb semantics, documentation

2015-06-12 Thread Andrew Dunstan
On 06/10/2015 04:02 PM, Peter Geoghegan wrote: On Wed, Jun 10, 2015 at 11:48 AM, Andrew Dunstan and...@dunslane.net wrote: Sorry for the delay on this. I've been mostly off the grid, having an all too rare visit from Tom Mr Enum Dunstan, and I misunderstood what you were suggesting, Thank you

Re: [HACKERS] [Proposal] Progress bar for pg_dump/pg_restore

2015-06-12 Thread Andres Freund
Hi, On 2015-06-12 12:45:50 +, Taiki Kondo wrote: Design API When pg_dump / pg_restore is running, progress bar and estimated time to finish is shown on screen like following. = (50%) 15:50 The bar (= in above) and percentage value (50% in above)

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-12 Thread Fujii Masao
On Fri, Jun 12, 2015 at 4:29 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Jun 12, 2015 at 3:50 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Jun 12, 2015 at 3:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jun 11, 2015 at 5:48 PM, Fujii Masao

Re: [HACKERS] git push hook to check for outdated timestamps

2015-06-12 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 06/12/2015 09:31 AM, Robert Haas wrote: Could we update our git hook to refuse a push of a new commit whose timestamp is more than, say, 24 hours in the past? Our commit history has some timestamps in it now that are over a month off, and it's

Re: [HACKERS] The purpose of the core team

2015-06-12 Thread Robert Haas
On Fri, Jun 12, 2015 at 1:21 AM, Simon Riggs si...@2ndquadrant.com wrote: Deciding WHAT goes in the next release? is what Committers do, by definition. It seems strange to have a different mailing list for WHEN is the next release needed?, so those two things should be combined. Core team

Re: [HACKERS] Why does replication need the old history file?

2015-06-12 Thread Josh Berkus
Questions: A. Why does the replica need 0002.history? Shouldn't it only need 0003.history? From where is the base backup taken in case of the node started at 5? It is the same backup used to restore the master, restored to a point in time 5 minutes earlier just to make sure the

Re: [HACKERS] On columnar storage

2015-06-12 Thread Alvaro Herrera
Amit Kapila wrote: On Fri, Jun 12, 2015 at 4:33 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: There are several parts to this: 1. the CSM API 2. Cataloguing column stores 3. Query processing: rewriter, optimizer, executor I think another important point is about the format

Re: [HACKERS] Reconsidering the behavior of ALTER COLUMN TYPE

2015-06-12 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Thu, Jun 11, 2015 at 03:41:49PM -0500, Merlin Moncure wrote: On Thu, Jun 11, 2015 at 3:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, we oughta use two different error messages for the two cases, as per my comment in the above thread. That seems

Re: [HACKERS] reaper should restart archiver even on standby

2015-06-12 Thread Fujii Masao
On Thu, Jun 11, 2015 at 1:39 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Fujii Masao wrote: Agreed. The attached patch defines the macro to check whether archiver is allowed to start up or not, and uses it everywhere except sigusr1_handler. I made sigusr1_handler use a different

[HACKERS] git push hook to check for outdated timestamps

2015-06-12 Thread Robert Haas
Could we update our git hook to refuse a push of a new commit whose timestamp is more than, say, 24 hours in the past? Our commit history has some timestamps in it now that are over a month off, and it's really easy to do, because when you rebase a commit, it keeps the old timestamp. If you then

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-12 Thread Simon Riggs
On 11 June 2015 at 22:12, Shay Rojansky r...@roji.org wrote: Thanks everyone for your time (or rather sorry for having wasted it). Just in case it's interesting to you... The reason we implemented things this way is in order to avoid a deadlock situation - if we send two queries as

Re: [HACKERS] On columnar storage

2015-06-12 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Amit Kapila wrote: Will the column store obey snapshot model similar to current heap tuples, if so will it derive the transaction information from heap tuple? Yes, visibility will be tied to the heap tuple -- a value is accessed only when its