Hi Steve and Chao, Konstatin has updated HDFS-15751 and attached patch for documentation msync API, Would you mind to TAL. I want to involve this update to 3.2.2 release, but for now it seems there are not enough committers to review. Please check when you have bandwidth. Thanks all.
- He Xiaoqiao On Fri, Dec 25, 2020 at 5:40 AM Konstantin Shvachko <shv.had...@gmail.com> wrote: > Hi Steve, > > I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751> > for > documenting msync API. > Would appreciate your suggestions. > > Stay safe, > --Konstantin > > On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran <ste...@cloudera.com> > wrote: > > > > > > > On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.had...@gmail.com> > > wrote: > > > >> Hey Steve, > >> > >> Thanks for the references. I was reading but still need to understand > how > >> exactly this applies to msync. > >> > > > > mainly: pull it up and it becomes part of the broader API, so needs to be > > specified in a way which can be understood by users and for implementors > of > > others stores: to give their own stores the same semantics. > > > > What does the HDFS one do? > > > > > > > >> Will come up with a plan and post it on a new jira. > >> Will make sure to create it under HADOOP and ping Hadoop Common list for > >> visibility. > >> > >> > > thanks > > > > > >> You are right about ViewFS. The impl should make sure it calls msync() > on > >> all mount points that enabled observer reads. > >> > >> > > That's the kind of issue this process aims to resolve. Another is to > > identify where we have HDFS-layer "quirks" and at least document them > (e.g. > > how hdfs streams are thread safe, rename isn't Posix, ...) and list what > we > > know breaks if you don't re-implement > > >