Ayush,
Yes, reflections are a part of Java. Why we have to define the FileSystem APIs
but not simply use reflections all the times?
Reflection is good for dealing with unknown code such as loading a plugin, code
analysis, etc. However, it probably is not a good way to define APIs.
Tsz-Wo
On Tuesday, March 21, 2023, 01:00:20 PM GMT+8, Ayush Saxena
<[email protected]> wrote:
I am not sure what classifies as a Hack and what not, I thought reflections
are part of Java.
Whatever solution but pulling in just the HDFS specific stuff to FileSystem
just for Ozone, because Hbase guys didn’t agree and we have people in
Hadoop who we can convince, I am -1 to such an approach and mindset. Hbase
wants ozone, they should give way for it like they do for HDFS
Explore ways in Hbase, explore the Utils and ways by the links that Steve
shared, try ViewDFS, When we have some more convincing reasons, we can
discuss more over here to pull them to FileSystem as the last option
-Ayush
On Fri, 17 Mar 2023 at 2:26 AM, Wei-Chiu Chuang <[email protected]> wrote:
> Hi,
>
> Stephen and I are working on a project to make HBase to run on Ozone.
>
> HBase, born out of the Hadoop project, depends on a number of HDFS specific
> APIs, including recoverLease() and isInSafeMode(). The HBase community [1]
> strongly voiced that they don't want the project to have direct dependency
> on additional FS implementations due to dependency and vulnerability
> management concerns.
>
> To make this project successful, we're exploring options, to push up these
> APIs to the FileSystem abstraction. Eventually, it would make HBase FS
> implementation agnostic, and perhaps enable HBase to support other storage
> systems in the future.
>
> We'd use the PathCapabilities API to probe if the underlying FS
> implementation supports these APIs, and would then invoke the corresponding
> FileSystem APIs. This is straightforward but the FileSystem would become
> bloated.
>
> Another option is to create a "RecoverableFileSystem" interface, and have
> both DistributedFileSystem (HDFS) and RootedOzoneFileSystem (Ozone). This
> way the impact to the Hadoop project and the FileSystem abstraction is even
> smaller.
>
> Thoughts?
>
> [1] https://lists.apache.org/thread/tcrp8vxxs3z12y36mpzx35txhpp7tvxv
>
