That's very helpful Paul ! But I can't access the wiki page you referenced. May be it's set to private ?
*---------------------* *Muhammad Gelbana* http://www.linkedin.com/in/mgelbana On Thu, Mar 16, 2017 at 1:50 AM, Paul Rogers <prog...@mapr.com> wrote: > Hi Muhammad, > > I know of no tutorial. I recently updated the “mock” storage plugin and so > have some experience with this interface. You can find my notes at [1]. > > Unfortunately, creating a storage plugin seems to require a significant > commitment of time and effort because you must understand: > > * The storage plugin structure. Some bits are a bit unusual (such as > bindings between the various bits and pieces.) > * Enough about Calcite to provide it with the required plan-time > information. > * The Jackson-serialization structure for various components. > * The rather complex process by which you work with ScanBatch and value > vectors to get data from your data source into value vectors. > * Drill as a whole so you can build Drill and debug it. The only way to > test a plugin is by running it inside Drill. > > Your best approach is to carefully study existing plugins. You can start > simple, say with the mock plugin. Replace the bits of the mock > implementation with your own. Try to get it to work for a single table. > Then, add other functionality gradually. > > By the time you are done you will be well on your way to being an expert > in some of Drill’s internals. > > Thanks, > > - Paul > > [1] https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model > > > On Mar 15, 2017, at 10:35 AM, Muhammad Gelbana <m.gelb...@gmail.com> > wrote: > > > > Everyone, > > > > Is there a tutorial on how to write a custom storage plugin to support > some > > sort of a proprietary data source ? > > > > I understand I can configure a storage plugin based on pre-shipped > storage > > plugins such as the one for MongoDB, MySQL\JDBC, HBase, Hadoop HDFS..etc, > > but that's not what I need. > > > > I need to write a new plugin to support a storage that is not publicly > > available. > > > > The best that I've got so far is the information on these pages: > > > > - > > https://github.com/apache/drill/blob/master/exec/java- > exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java > > - > > https://github.com/apache/drill/tree/master/contrib/ > storage-kudu/src/main/java/org/apache/drill/exec/store/kudu > > > > > > But this isn't enough to understand what needs to be done or troubleshoot > > errors while developing the plugin. > > > > *---------------------* > > *Muhammad Gelbana* > > http://www.linkedin.com/in/mgelbana > >
