Yes, maybe we could write WAL to SQS and HFile to S3, then we can deploy HBase on AWS without any local storage volumes...
But we also need a good abstraction for Replication, as the current design is file based... 2018-07-27 1:28 GMT+08:00 Zach York <[email protected]>: > I would REALLY hope that the WAL interface/API changes would go into master > even if the feature work for Ratis is going in a feature branch. Not only > would this enable other backends to be developed in parallel with the Ratis > solution if there are other good fits for a non-HDFS WAL, but also it would > save the burden of having to rebase these core changes onto the latest > master to maintain compatibility. I'm assuming the Ratis portion of the > code would be mostly new files so these would be less of a concern. > > On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser <[email protected]> wrote: > > > On 7/26/18 1:00 AM, Stack wrote: > > > >> All this said, I'd like to start moving toward the point where we start > >>> breaking out this work into a feature-branch off of master and start > >>> building code. My hope is that this is amenable to everyone, with the > >>> acknowledge that the Ratis work is considered "experimental" and not an > >>> attempt to make all of HBase use Ratis-backed WALs. > >>> > >>> > >>> Go for it. > >> > >> The branch would have WAL API changes only or would it include Ratis WAL > >> dev? (If the latter, would that be better done over on Ratis project?). > >> > > > > I think we would start with WAL API changes, get those "blessed", and > then > > continue Ratis WAL dev after that. > > >
