Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Greg Stark
On Thu, May 23, 2013 at 1:03 PM, Simon Riggs si...@2ndquadrant.com wrote: The COPYRIGHT file shows that VMware is claiming copyright on unstated parts of the code for this. As such, its not a normal submission to the PostgreSQL project, which involves placing copyright with the PGDG. Fwiw I

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Wed, Jun 5, 2013 at 10:12:17AM +0100, Greg Stark wrote: I'm not a lawyer and I make no judgement on how solid a practice this is but that's VMware doesn't seem to be doing anything special here. They can retain copyright ownership of their

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Bruce Momjian
On Wed, Jun 5, 2013 at 10:12:17AM +0100, Greg Stark wrote: On Thu, May 23, 2013 at 1:03 PM, Simon Riggs si...@2ndquadrant.com wrote: The COPYRIGHT file shows that VMware is claiming copyright on unstated parts of the code for this. As such, its not a normal submission to the PostgreSQL

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Josh Berkus
I'm not a lawyer and I make no judgement on how solid a practice this is but that's VMware doesn't seem to be doing anything special here. They can retain copyright ownership of their contributions as long as they're happy releasing it under the Postgres copyright. Ideally they wold also be

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Dave Page
On Wed, Jun 5, 2013 at 6:10 PM, Josh Berkus j...@agliodbs.com wrote: I'm not a lawyer and I make no judgement on how solid a practice this is but that's VMware doesn't seem to be doing anything special here. They can retain copyright ownership of their contributions as long as they're happy

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-04 Thread Josh Berkus
On 05/28/2013 11:32 AM, Bruce Momjian wrote: I think Simon has a good point, as VMWare has asserted patents on some changes to their version of Postgres in the past, so if the copyright ... which I'll point out that they *didn't* contribute, and which may yet get resolved in a way that benefits

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-04 Thread Joshua D. Drake
On 06/04/2013 01:55 PM, Josh Berkus wrote: That seems rather like a catch-22 Bruce. If they don't check with the legal department, it's dangerous, but if they do check, it's dangerous? Presumably if they checked with the legal department, it's cleared. We should be wary of stuff contributed

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-04 Thread Bruce Momjian
On Tue, Jun 4, 2013 at 01:55:27PM -0700, Josh Berkus wrote: On 05/28/2013 11:32 AM, Bruce Momjian wrote: I think Simon has a good point, as VMWare has asserted patents on some changes to their version of Postgres in the past, so if the copyright ... which I'll point out that they *didn't*

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Thu, May 23, 2013 at 01:48:24PM -0400, Heikki Linnakangas wrote: On 23.05.2013 08:03, Simon Riggs wrote: On 23 May 2013 12:10, Heikki Linnakangashlinnakan...@vmware.com wrote: Please take a look: https://github.com/vmware/pg_rewind The COPYRIGHT file shows that VMware is claiming

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Andres Freund
On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: We have a lot of code in PostgreSQL source tree with different copyright notices, and there's no problem with that as long as the coe is licensed under the PostgreSQL license. For patches that add Really? Where? I think we have removed

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 08:37:44PM +0200, Andres Freund wrote: On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: We have a lot of code in PostgreSQL source tree with different copyright notices, and there's no problem with that as long as the coe is licensed under the PostgreSQL

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Andres Freund
On 2013-05-28 14:50:57 -0400, Bruce Momjian wrote: On Tue, May 28, 2013 at 08:37:44PM +0200, Andres Freund wrote: On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: We have a lot of code in PostgreSQL source tree with different copyright notices, and there's no problem with that as long

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Alvaro Herrera
Bruce Momjian wrote: Oh, I see. Have we historically been OK with these as long as it is clear it is the PG copyright? I know we had do some cleanups in the past, but I don't remember the details, obviously. We've had request from companies because they wanted to distribute Postgres and

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Christophe Pettus
On May 28, 2013, at 12:49 PM, Alvaro Herrera wrote: We've had request from companies because they wanted to distribute Postgres and lawyers weren't comfortable with copyright statements in assorted files. In those cases we've asked the people mentioned in such copyright statements, got

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 03:49:14PM -0400, Alvaro Herrera wrote: Bruce Momjian wrote: Oh, I see. Have we historically been OK with these as long as it is clear it is the PG copyright? I know we had do some cleanups in the past, but I don't remember the details, obviously. We've had

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-25 Thread Amit kapila
On Thursday, May 23, 2013 4:40 PM Heikki Linnakangas wrote: Hi, I've been hacking on a tool to allow resynchronizing an old master server after failover. The need to do a full backup/restore has been a common complaint ever since we've had streaming replication. I saw on the wiki that this

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-25 Thread Ants Aasma
On Sat, May 25, 2013 at 10:05 AM, Amit kapila amit.kap...@huawei.com wrote: In a nutshell, the idea is to do copy everything that has changed between the cluster, like rsync does, but instead of reading through all files, use the WAL to determine what has changed. Here's a somewhat more

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-24 Thread Thom Brown
On 23 May 2013 10:03, Thom Brown t...@linux.com wrote: On 23 May 2013 07:10, Heikki Linnakangas hlinnakan...@vmware.com wrote: Hi, I've been hacking on a tool to allow resynchronizing an old master server after failover. The need to do a full backup/restore has been a common complaint ever

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-24 Thread Andres Freund
On 2013-05-24 10:57:22 -0400, Thom Brown wrote: By the way, without any data inserted I get: thom@swift /tmp $ pg_rewind --target-pgdata=/tmp/primary --source-server='host=localhost port=5531 dbname=postgres' -v connected to remote server fetched file global/pg_control, length 8192 fetched

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-24 Thread Jim Nasby
On 5/23/13 12:51 PM, Pavan Deolasee wrote: On Thu, May 23, 2013 at 11:10 PM, Heikki Linnakangas hlinnakan...@vmware.com mailto:hlinnakan...@vmware.com wrote: On 23.05.2013 07:55, Robert Haas wrote: On Thu, May 23, 2013 at 7:10 AM, Heikki Linnakangas

[HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Heikki Linnakangas
Hi, I've been hacking on a tool to allow resynchronizing an old master server after failover. The need to do a full backup/restore has been a common complaint ever since we've had streaming replication. I saw on the wiki that this was discussed in the dev meeting; too bad I couldn't make it.

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Robert Haas
On Thu, May 23, 2013 at 7:10 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Scan the WAL log of the old cluster, starting from the point where the new cluster's timeline history forked off from the old cluster. For each WAL record, make a note of the data blocks that are touched. This

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Simon Riggs
On 23 May 2013 12:10, Heikki Linnakangas hlinnakan...@vmware.com wrote: Please take a look: https://github.com/vmware/pg_rewind The COPYRIGHT file shows that VMware is claiming copyright on unstated parts of the code for this. As such, its not a normal submission to the PostgreSQL project,

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Thom Brown
On 23 May 2013 07:10, Heikki Linnakangas hlinnakan...@vmware.com wrote: Hi, I've been hacking on a tool to allow resynchronizing an old master server after failover. The need to do a full backup/restore has been a common complaint ever since we've had streaming replication. I saw on the wiki

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Fujii Masao
On Thu, May 23, 2013 at 8:55 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 23, 2013 at 7:10 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Scan the WAL log of the old cluster, starting from the point where the new cluster's timeline history forked off from the old cluster.

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Robert Haas
On Thu, May 23, 2013 at 11:34 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, May 23, 2013 at 8:55 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 23, 2013 at 7:10 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Scan the WAL log of the old cluster, starting from the

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Heikki Linnakangas
On 23.05.2013 07:55, Robert Haas wrote: On Thu, May 23, 2013 at 7:10 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Scan the WAL log of the old cluster, starting from the point where the new cluster's timeline history forked off from the old cluster. For each WAL record, make a note

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Heikki Linnakangas
On 23.05.2013 08:03, Simon Riggs wrote: On 23 May 2013 12:10, Heikki Linnakangashlinnakan...@vmware.com wrote: Please take a look: https://github.com/vmware/pg_rewind The COPYRIGHT file shows that VMware is claiming copyright on unstated parts of the code for this. As such, its not a normal

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-23 Thread Pavan Deolasee
On Thu, May 23, 2013 at 11:10 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 23.05.2013 07:55, Robert Haas wrote: On Thu, May 23, 2013 at 7:10 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Scan the WAL log of the old cluster, starting from the point where the new