Hi Rafael, Glad you're getting some value from Drill. Repackaging that directory as a truly pluggable jar is tricky. A few questions: 1. Did you copy the contrib/storage-http into its own folder and then do a build from that? 2. Did it build successfully? 3. Did you copy the JARs into your Drill jars/3rdparty folder? 4. You'll also have to get JARs of any dependencies as well and copy them to the jars/3rdparty. Have you done that?
I actually have a pre-built version of Drill with the storage-http plugin available here: https://github.com/cgivre/drill/releases <https://github.com/cgivre/drill/releases>. Please do not use that in any kind of production setup. If you're just wanting to try this out, it might be easier to d/l that and use that. -- C > On Mar 31, 2020, at 12:57 PM, Jaimes, Rafael - 0993 - MITLL > <[email protected]> wrote: > > Hi Charles, > > I am trying to use the http-storage plugin from your branch. I put the > storage plug-in files in a jar and tried to keep the jar directory structure > the same as other plug-ins. Upon starting drill-embedded I’m getting the > error below. I am using your drill-module.conf and > bootstrap-storage-plugins.json from your branch. Is there another step I need > to perform to get Drill to recognize the plug-in? I am using 1.17 release. > > Error: Failure in starting embedded Drillbit: > java.lang.IllegalStateException: > com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve > type id 'http' as a subtype of [simple type, class > org.apache.drill.common.logical.StoragePluginConfig]: known type ids = > [InfoSchemaConfig, SystemTablePluginConfig, file, hbase, hive, jdbc, kafka, > kudu, mock, mongo, named, openTSDB] (for POJO property 'storage') > at [Source: (String)"{ > "storage":{ > "http" : { > "type":"http", > "connections": {}, > "enabled": false > } > } > } > "; line: 4, column: 14] (through reference chain: > org.apache.drill.exec.planner.logical.StoragePlugins["storage"]->java.util.LinkedHashMap["http"]) > (state=,code=0) > > Paul, > > I don’t know much about this REST service quite yet (it is internal). We > utilize REST API where all responses are returned as JSON formatted strings > in many places, I don’t think it is very sophisticated. I am not sure how it > will handle projection and filter issues. My current pipeline involves using > python requests.get() and then unpacking the response string. It does have an > authentication layer, so I am mildly concerned that the HTTP-storage-plugin > will have a hiccup – although it looks like it can use “Basic”. If I can get > Drill to query the endpoint I will report back if I find anything else that > might be useful to you. > > Thanks both for your great work with Drill! > > - Rafael
