Does S3 need sync if it's "strongly consistent"? Does "strongly" imply "immediate"? If so, you might just need to use a noop log closer. But I'm not sure.
On Wed, Oct 12, 2022, 20:05 <[email protected]> wrote: > I believe that S3 Guard is OBE, but you still need to put the WAL on HDFS > as S3 does not support sync. If you put your WAL in S3, and you have a > tserver failure, then it’s possible that you will lose data. > > > > *From:* Christopher <[email protected]> > *Sent:* Wednesday, October 12, 2022 4:12 PM > *To:* [email protected] > *Subject:* Re: Accumulo On S3 > > > > Since S3 became strongly consistent, I think it would probably just work. > But, obviously, we can't make any guarantees, especially about the behavior > of software outside of our control. So, your experience may vary. > > > > On Wed, Oct 12, 2022 at 12:28 PM Josh Clum <[email protected]> wrote: > > Hi, > > > > Question on this post: > https://accumulo.apache.org/blog/2019/09/10/accumulo-S3-notes.html > > > > It's been a long time and it looks like several of the merge requests are > merged into master. > > > > The point about S3 Guard being needed seems OBE since S3 is strongly > consistent and S3 Guard is deprecated at this point: > https://issues.apache.org/jira/browse/HADOOP-17480 > > > > Should Accumulo on S3 with the metadata table and WAL logs in S3 work or > are there still commits somewhere that won't be available until 2.1.0? > > > > Thanks, > > Josh > >
