With minimal code change, it is possible to enable the use of —dir for both standard and parallel export/import, allowing —file to function only for standard exports (and optionally, make it depricated in favor of the —dir option).
While not inherently opposed to forcing all Partitioned Region snapshots to be parallel, that seems to be a significant enough change to current functionality (one file one one member to multiple files on multiple members), I am hesitant to do so without united community backing for that approach. On Tue, Aug 22, 2017 at 2:24 PM, Michael Stolz <mst...@pivotal.io> wrote: > One other idea that hasn't been mentioned is making parallel the only way > for Partitioned Regions, and having --file serve the purpose of defining > both a path and a filename pattern where the bucket ID or whatever we're > using gets automatically inserted before the .gfd extension. > > No need for a new option (--parallel). > No need for a new option (--path). > > In fact, no need for a change to gfsh command at all. > > > -- > Mike Stolz > Principal Engineer, GemFire Product Manager > Mobile: +1-631-835-4771 > > On Tue, Aug 22, 2017 at 2:15 PM, Nick Reich <nre...@pivotal.io> wrote: > > > Parallel export will write the data to files on the bucket primary for > each > > bucket, distributing the work (and therefore files) to all the members. > > That would be a big enough deviation from the current behavior (single > file > > on single machine), that I think it makes it worth having the additional > > options (but I agree: less options is generally better). > > > > On Tue, Aug 22, 2017 at 1:59 PM, Jacob Barrett <jbarr...@pivotal.io> > > wrote: > > > > > On Tue, Aug 22, 2017 at 1:49 PM Nick Reich <nre...@pivotal.io> wrote: > > > > > > > The idea of deprecating —file in favor of path is interesting. I > wonder > > > if > > > > instead of making them mutually exclusive to start, having —path be > > able > > > to > > > > support both modes from the start would be better? That way —file > could > > > > still be used for the existing mode, but —path could be used instead > > (and > > > > override —file is both given?): that would provide a clear path > forward > > > for > > > > how the command should be used, while fully supporting existing > > > workflows. > > > > > > > > > > This is what I meant by deprecating. Maybe even providing a message > that > > if > > > --file is set that it is deprecated for --path. > > > > > > > > > > We need to continue to support both modes, as only Partitioned > Regions > > > can > > > > make use of parallel export (it is parallelized on a bucket level). > > > > > > > > > > Ok, so why not just make parallel the only mode for partitioned. Then > you > > > remove the need for --parallel and --path would work for any region, > > > non-partitioned would create a single file at that path and partitioned > > > would create several? I am all for less options. ;) > > > > > > -Jake > > > > > >
