+1 (binding) .. Owen
On Sun, Mar 11, 2018 at 6:20 PM, Chris Douglas <cdoug...@apache.org> wrote: > +1 (binding) -C > > On Thu, Mar 8, 2018 at 9:31 AM, Jim Clampffer <james.clampf...@gmail.com> > wrote: > > Hi Everyone, > > > > The feedback was generally positive on the discussion thread [1] so I'd > > like to start a formal vote for merging HDFS-8707 (libhdfs++) into trunk. > > The vote will be open for 7 days and end 6PM EST on 3/15/18. > > > > This branch includes a C++ implementation of an HDFS client for use in > > applications that don't run an in-process JVM. Right now the branch only > > supports reads and metadata calls. > > > > Features (paraphrasing the list from the discussion thread): > > -Avoiding the JVM means applications that use libhdfs++ can explicitly > > control resources (memory, FDs, threads). The driving goal for this > > project was to let C/C++ applications access HDFS while maintaining a > > single heap. > > -Includes support for Kerberos authentication. > > -Includes a libhdfs/libhdfs3 compatible C API as well as a C++ API that > > supports asynchronous operations. Applications that only do reads may be > > able to use this as a drop in replacement for libhdfs. > > -Asynchronous IO is built on top of boost::asio which in turn uses > > select/epoll so many sockets can be monitored from a single thread (or > > thread pool) rather than spawning a thread to sleep on a blocked socket. > > -Includes a set of utilities written in C++ that mirror the CLI tools > (e.g. > > ./hdfs dfs -ls). These have a 3 order of magnitude lower startup time > than > > java client which is useful for scripts that need to work with many > files. > > -Support for cancelable reads that release associated resources > > immediately. Useful for applications that need to be responsive to > > interactive users. > > > > Other points: > > -This is almost all new code in a new subdirectory. No Java source for > the > > rest of hadoop was changed so there's no risk of regressions there. The > > only changes outside of that subdirectory were integrating the build in > > some of the pom files and adding a couple dependencies to the DockerFile. > > -The library has had plenty of burn-in time. It's been used in > production > > for well over a year and is indirectly being distributed as part of the > > Apache ORC project (in the form of a third party dependency). > > -There isn't much in the way of well formatted documentation right now. > > The documentation for the libhdfs API is applicable to the libhdfs++ C > API. > > Header files describe various component including details about threading > > and lifecycle expectations for important objects. Good places to start > are > > hdfspp.h, filesystem.h, filehandle.h, rpc_connection.h and rpc_enginel.h. > > > > I'll start with my +1 (binding). > > > > [1] > > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/ > 201803.mbox/browser > > (second message in thread, can't figure out how to link directly to mine) > > > > Thanks! > > --------------------------------------------------------------------- > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
