Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-28 Thread Bruce Momjian
On Fri, Jul 27, 2012 at 02:13:31PM +1000, Chris Angelico wrote: > On Fri, Jul 27, 2012 at 9:57 AM, Chris Angelico wrote: > > On Fri, Jul 27, 2012 at 9:53 AM, Bruce Momjian wrote: > >> You might want to look at the hackers list thread I started about the > >> same topic a week before your post: >

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-26 Thread Chris Angelico
On Fri, Jul 27, 2012 at 9:57 AM, Chris Angelico wrote: > On Fri, Jul 27, 2012 at 9:53 AM, Bruce Momjian wrote: >> You might want to look at the hackers list thread I started about the >> same topic a week before your post: >> >> http://archives.postgresql.org/pgsql-hackers/2012-07/msg0041

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-26 Thread Bruce Momjian
On Fri, Jul 27, 2012 at 09:57:55AM +1000, Chris Angelico wrote: > On Fri, Jul 27, 2012 at 9:53 AM, Bruce Momjian wrote: > > You might want to look at the hackers list thread I started about the > > same topic a week before your post: > > > > http://archives.postgresql.org/pgsql-hackers/201

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-26 Thread Chris Angelico
On Fri, Jul 27, 2012 at 9:53 AM, Bruce Momjian wrote: > You might want to look at the hackers list thread I started about the > same topic a week before your post: > > http://archives.postgresql.org/pgsql-hackers/2012-07/msg00416.php > > Basically, you can only use mtime/size if you are re

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-26 Thread Bruce Momjian
On Tue, Jul 17, 2012 at 07:42:38AM +1000, Chris Angelico wrote: > On Tue, Jul 17, 2012 at 4:35 AM, Sergey Konoplev > wrote: > > On Mon, Jul 16, 2012 at 8:01 PM, Chris Angelico wrote: > >> On Tue, Jul 17, 2012 at 1:58 AM, Michael Nolan wrote: > >>> As I understand the docs for rsync, it will use

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Michael Nolan
On 7/16/12, Steven Schlansker wrote: > I think it's pretty easy to show that timestamp+size isn't good enough to do > this 100% reliably. That may not be a problem if the slave server synchronization code always starts to play back WAL entries at a time before the worst case for timestamp precisi

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread John R Pierce
On 07/16/12 2:42 PM, Chris Angelico wrote: On Tue, Jul 17, 2012 at 4:35 AM, Sergey Konoplev wrote: >I wonder if it is correct in general to use mtime and size to perform >these checks from the point of view of PostgreSQL. > >If it works with the current version then is there a guaranty that it

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Steven Schlansker
I think it's pretty easy to show that timestamp+size isn't good enough to do this 100% reliably. Imagine that your timestamps have a millisecond resolution. I assume this will vary based on OS / filesystem, but the point remains the same no matter what size it is. You can have multiple writes

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Chris Angelico
On Tue, Jul 17, 2012 at 4:35 AM, Sergey Konoplev wrote: > On Mon, Jul 16, 2012 at 8:01 PM, Chris Angelico wrote: >> On Tue, Jul 17, 2012 at 1:58 AM, Michael Nolan wrote: >>> As I understand the docs for rsync, it will use both mod time and file size >>> if told not to do checksums. > > I wonder

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Michael Nolan
On 7/16/12, Sergey Konoplev wrote: > On Mon, Jul 16, 2012 at 8:01 PM, Chris Angelico wrote: >> On Tue, Jul 17, 2012 at 1:58 AM, Michael Nolan wrote: >>> As I understand the docs for rsync, it will use both mod time and file >>> size >>> if told not to do checksums. > > I wonder if it is correct

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Sergey Konoplev
On Mon, Jul 16, 2012 at 8:01 PM, Chris Angelico wrote: > On Tue, Jul 17, 2012 at 1:58 AM, Michael Nolan wrote: >> As I understand the docs for rsync, it will use both mod time and file size >> if told not to do checksums. I wonder if it is correct in general to use mtime and size to perform thes

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Chris Angelico
On Tue, Jul 17, 2012 at 1:58 AM, Michael Nolan wrote: > As I understand the docs for rsync, it will use both mod time and file size > if told not to do checksums. Oh, so it does, I misread. Thanks! Time+size it is. ChrisA -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Michael Nolan
On 7/16/12, Chris Angelico wrote: > On Tue, Jul 17, 2012 at 1:40 AM, Michael Nolan wrote: >> I did several weeks of tests on 9.1.3 using mod time and file size >> rather than checksumming the files, that did not appear to cause any >> problems >> and it sped up the rsync considerably. (This was

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Chris Angelico
On Tue, Jul 17, 2012 at 1:40 AM, Michael Nolan wrote: > I did several weeks of tests on 9.1.3 using mod time and file size > rather than checksumming the files, that did not appear to cause any problems > and it sped up the rsync considerably. (This was about a 40 GB database.) Thanks! Is file s

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Michael Nolan
On 7/16/12, Chris Angelico wrote: > I'm speccing up a three-node database for reliability, making use of > streaming replication, and it's all working but I have a bit of a > performance concern. > > > Can the individual files' modification timestamps be relied upon? If > so, it'd potentially mean

[GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Chris Angelico
I'm speccing up a three-node database for reliability, making use of streaming replication, and it's all working but I have a bit of a performance concern. Suppose a node dies and is removed from the cluster, but then returns (say, a day or two later). I could, of course, utterly wipe the existing