On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.had...@gmail.com> wrote:
> Hey Steve, > > Thanks for the references. I was reading but still need to understand how > exactly this applies to msync. > mainly: pull it up and it becomes part of the broader API, so needs to be specified in a way which can be understood by users and for implementors of others stores: to give their own stores the same semantics. What does the HDFS one do? > Will come up with a plan and post it on a new jira. > Will make sure to create it under HADOOP and ping Hadoop Common list for > visibility. > > thanks > You are right about ViewFS. The impl should make sure it calls msync() on > all mount points that enabled observer reads. > > That's the kind of issue this process aims to resolve. Another is to identify where we have HDFS-layer "quirks" and at least document them (e.g. how hdfs streams are thread safe, rename isn't Posix, ...) and list what we know breaks if you don't re-implement
