Parallel export will write the data to files on the bucket primary for each bucket, distributing the work (and therefore files) to all the members. That would be a big enough deviation from the current behavior (single file on single machine), that I think it makes it worth having the additional options (but I agree: less options is generally better).
On Tue, Aug 22, 2017 at 1:59 PM, Jacob Barrett <jbarr...@pivotal.io> wrote: > On Tue, Aug 22, 2017 at 1:49 PM Nick Reich <nre...@pivotal.io> wrote: > > > The idea of deprecating —file in favor of path is interesting. I wonder > if > > instead of making them mutually exclusive to start, having —path be able > to > > support both modes from the start would be better? That way —file could > > still be used for the existing mode, but —path could be used instead (and > > override —file is both given?): that would provide a clear path forward > for > > how the command should be used, while fully supporting existing > workflows. > > > > This is what I meant by deprecating. Maybe even providing a message that if > --file is set that it is deprecated for --path. > > > > We need to continue to support both modes, as only Partitioned Regions > can > > make use of parallel export (it is parallelized on a bucket level). > > > > Ok, so why not just make parallel the only mode for partitioned. Then you > remove the need for --parallel and --path would work for any region, > non-partitioned would create a single file at that path and partitioned > would create several? I am all for less options. ;) > > -Jake >
