On Wed, May 31, 2006 at 10:01:26AM -0400, Rod Taylor wrote: > I've been thinking of the initial COPY process. > > The problem is that with a large amount of data you end up with a very > large transaction on the data provider. The transaction on the > subscriber isn't as important since it will normally be an otherwise > idle database. > > COPY in is one part, but building indexes on the subscriber is the > painful part and during much of this process the data provider has an > idle connection. Pardon my ignorance, but is the provider actually sitting in a transaction while the subscriber is building indexes, and if so, why? ISTM there's no reason you'd need indexes (or RI for that matter) while loading data into a subscriber. > I haven't looked at the log shipping support, but I suspect it holds > part of the answer. The data provider does not need to be active while > the new subscriber is catching up to the transaction log when log > shipping is enabled. > > Is there any reason you could not convert between a shipped log and the > standard "online" mode? > > If possible, that would reduce the initial transaction size on the data > provider to the time required to dump out the initial copy of the > tables. Still a significant amount of time, one transaction per table > would be better, but much shorter than today. > > > A first approximation of a transition between log shipping and standard > modes would be to write the data to a temporary area on the data > provider machine (avoid network overhead) and later on send it across > the wire without being connected to the data provider database. > > -- > > _______________________________________________ > Slony1-general mailing list > [email protected] > http://gborg.postgresql.org/mailman/listinfo/slony1-general >
-- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 _______________________________________________ Slony1-general mailing list [email protected] http://gborg.postgresql.org/mailman/listinfo/slony1-general
