We do a similar trick in our code; when we don't overwrite existing data, the whole process completes much, much faster. At least 100x, maybe 1000 - not sure, but we ended up deciding it was OK to take the hit of iterating over all the uris.
-Mike On 05/27/2011 03:00 PM, Michael Blakeley wrote: > Looking at the code, SKIP_EXISTING is checked in SessionWriter. In theory any > AbstractWriter subclass could check it, but the included PackageWriter does > not. > > With XQSync's processing model, there isn't much room for an AbstractReader > subclass to use SKIP_EXISTING. I suppose the manager code might be extended > to check each URI before queuing the sync action, but that would be complex > to implement in a scalable fashion. My hunch is that the writes are the > expensive part, and that the extra complexity isn't worthwhile. > > -- Mike > > On 27 May 2011, at 05:46 , Paul M wrote: > > >> When using XQSYNC with the option SKIP_EXISTING set to TRUE, what handling >> changes occur internally with regards to reading from the >> INPUT_CONNECTION_STRING repository and sending to the >> OUTPUT_CONNECTION_STRING repository? It seems to be the same amount of time >> regardless of whether SKIP_EXISTING is set to TRUE or FALSE...records still >> read and set just not written? >> >> >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general >> >> > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
