>> One other idea that hasn't been mentioned is making parallel the only way
My vote is to support both option; we could make parallel default but having an option to take snapshot at one node may be useful for use-cases where: - Easier to manage snapshot at one file location; in a large cluster environment. - Control on file level security - A node can be used for snapshot without impacting other nodes (disk i/o ops). -Anil. On Tue, Aug 22, 2017 at 3:42 PM, Nick Reich <nre...@pivotal.io> wrote: > With minimal code change, it is possible to enable the use of —dir for both > standard and parallel export/import, allowing —file to function only for > standard exports (and optionally, make it depricated in favor of the —dir > option). > > While not inherently opposed to forcing all Partitioned Region snapshots to > be parallel, that seems to be a significant enough change to current > functionality (one file one one member to multiple files on multiple > members), I am hesitant to do so without united community backing for that > approach. > > On Tue, Aug 22, 2017 at 2:24 PM, Michael Stolz <mst...@pivotal.io> wrote: > > > One other idea that hasn't been mentioned is making parallel the only way > > for Partitioned Regions, and having --file serve the purpose of defining > > both a path and a filename pattern where the bucket ID or whatever we're > > using gets automatically inserted before the .gfd extension. > > > > No need for a new option (--parallel). > > No need for a new option (--path). > > > > In fact, no need for a change to gfsh command at all. > > > > > > -- > > Mike Stolz > > Principal Engineer, GemFire Product Manager > > Mobile: +1-631-835-4771 > > > > On Tue, Aug 22, 2017 at 2:15 PM, Nick Reich <nre...@pivotal.io> wrote: > > > > > Parallel export will write the data to files on the bucket primary for > > each > > > bucket, distributing the work (and therefore files) to all the members. > > > That would be a big enough deviation from the current behavior (single > > file > > > on single machine), that I think it makes it worth having the > additional > > > options (but I agree: less options is generally better). > > > > > > On Tue, Aug 22, 2017 at 1:59 PM, Jacob Barrett <jbarr...@pivotal.io> > > > wrote: > > > > > > > On Tue, Aug 22, 2017 at 1:49 PM Nick Reich <nre...@pivotal.io> > wrote: > > > > > > > > > The idea of deprecating —file in favor of path is interesting. I > > wonder > > > > if > > > > > instead of making them mutually exclusive to start, having —path be > > > able > > > > to > > > > > support both modes from the start would be better? That way —file > > could > > > > > still be used for the existing mode, but —path could be used > instead > > > (and > > > > > override —file is both given?): that would provide a clear path > > forward > > > > for > > > > > how the command should be used, while fully supporting existing > > > > workflows. > > > > > > > > > > > > > This is what I meant by deprecating. Maybe even providing a message > > that > > > if > > > > --file is set that it is deprecated for --path. > > > > > > > > > > > > > We need to continue to support both modes, as only Partitioned > > Regions > > > > can > > > > > make use of parallel export (it is parallelized on a bucket level). > > > > > > > > > > > > > Ok, so why not just make parallel the only mode for partitioned. Then > > you > > > > remove the need for --parallel and --path would work for any region, > > > > non-partitioned would create a single file at that path and > partitioned > > > > would create several? I am all for less options. ;) > > > > > > > > -Jake > > > > > > > > > >
