Hi Konstantin and Steve, HDFS-15751 has committed to related branches including 3.2.2. I will prepare 3.2.2-RC shortly. Please let me know if any other issues for doc or deployment do you meet. Thanks.
- He Xiaoqiao On Wed, Dec 30, 2020 at 6:40 PM Xiaoqiao He <xq.he2...@gmail.com> wrote: > Hi Steve and Chao, > > Konstatin has updated HDFS-15751 and attached patch for documentation > msync API, Would you mind to TAL. I want to involve this update to 3.2.2 > release, but for now it seems there are not enough committers to review. > Please check when you have bandwidth. Thanks all. > > - He Xiaoqiao > > On Fri, Dec 25, 2020 at 5:40 AM Konstantin Shvachko <shv.had...@gmail.com> > wrote: > >> Hi Steve, >> >> I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751> >> for >> documenting msync API. >> Would appreciate your suggestions. >> >> Stay safe, >> --Konstantin >> >> On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran <ste...@cloudera.com> >> wrote: >> >> > >> > >> > On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.had...@gmail.com >> > >> > wrote: >> > >> >> Hey Steve, >> >> >> >> Thanks for the references. I was reading but still need to understand >> how >> >> exactly this applies to msync. >> >> >> > >> > mainly: pull it up and it becomes part of the broader API, so needs to >> be >> > specified in a way which can be understood by users and for >> implementors of >> > others stores: to give their own stores the same semantics. >> > >> > What does the HDFS one do? >> > >> > >> > >> >> Will come up with a plan and post it on a new jira. >> >> Will make sure to create it under HADOOP and ping Hadoop Common list >> for >> >> visibility. >> >> >> >> >> > thanks >> > >> > >> >> You are right about ViewFS. The impl should make sure it calls msync() >> on >> >> all mount points that enabled observer reads. >> >> >> >> >> > That's the kind of issue this process aims to resolve. Another is to >> > identify where we have HDFS-layer "quirks" and at least document them >> (e.g. >> > how hdfs streams are thread safe, rename isn't Posix, ...) and list >> what we >> > know breaks if you don't re-implement >> > >> >