Hi there,I've been looking into this. It's almost working except for some issues when reading lots of parquet files, but I'm not sure if that is Alluxio's fault.
I have a couple of WIP Docker images that's I'm using on Kubernetes. I can share if you want, but it's all very hackish right now and I'm still changing things very often.
Cheers, On Wed, Sep 14, 2016 at 11:13:49AM -0700, Sudheesh Katkam wrote:
Based on the documentation [1], Alluxio not only uses HDFS as a underlying storage system but also provides a “Hadoop API” to clients, and Drill uses this API. So it should be possible. Create a storage plugin named “alluxio” whose contents match “dfs” plugin, and then make changes. Not sure what exactly, but [2] should be helpful. See “S3 Storage Plugin” as an example as well [3]. Once you get things to work, please contribute by adding a section to the Drill documentation. Thank you, Sudheesh [1] http://www.alluxio.org/docs/master/en/File-System-API.html#hadoop-api <http://www.alluxio.org/docs/master/en/File-System-API.html#hadoop-api> [2] http://www.alluxio.org/docs/master/en/Configuration-Settings.html <http://www.alluxio.org/docs/master/en/Configuration-Settings.html> [3] http://drill.apache.org/docs/s3-storage-plugin/ <http://drill.apache.org/docs/s3-storage-plugin/>On Sep 14, 2016, at 10:07 AM, Edmon Begoli <[email protected]> wrote: Is it possible to use Alluxio as a filesystem backend in Drill's storage formats, and if so how? Thanks.
--
Oscar Morante
"Self-education is, I firmly believe, the only kind of education there is."
-- Isaac Asimov.
signature.asc
Description: PGP signature
