Hi Tomer, Thanks for your advice about creating a Hadoop FileSystem implementation. I just finished a prototypical implementation of a Hadoop file system for Manta: https://github.com/dekobon/hadoop-manta <https://github.com/dekobon/hadoop-manta>
I see the example for enabling S3 with Apache Drill and I’ve verified that it works. However, when I attempt to replicate the configuration of S3 for Manta, I’m unable to get the Hadoop FileSystem driver to load. I’ve verified that the FileSystem driver works in Hadoop by checking all of the hdfs dfs -* commands and I’ve got a fair bit of automated testing around it. What’s the magic to get it turned on with Drill? Do I need to do something to make the jar load other than copy it into jars/3rdparty? Right now, I’m just testing in drill-embedded for what it is worth. Thanks, Elijah Zupancic > On May 6, 2016, at 4:11 PM, Tomer Shiran <tshi...@dremio.com> wrote: > > Does Manta have a Hadoop FileSystem API implementation? That's what Drill > uses for S3, HDFS, MapR-FS, Azure Blob Storage, etc. You could potentially > write a Drill storage plugin, but you get a lot for free if you already > have the file system implementation. > > On Fri, May 6, 2016 at 9:43 AM, Elijah Zupancic <eli...@apache.org> wrote: > >> I'm trying to get started contributing to Apache Drill. I've got the >> project checked out and it is building to my satisfaction. Right now, I'm >> trying to add support for the open source object store Manta ( >> https://github.com/joyent/manta). I thought that this would be a good >> learning project. >> >> Initially, I want to add support in the same way that S3 has support. >> However, I can't seem to find a reference to the S3 storage driver in the >> code base. Is the s3 storage driver part of a different project? How would >> you suggest that I get started? >> >> Thank you, >> Elijah Zupancic >>
