paul-rogers commented on a change in pull request #1892: Drill-7437: Storage
Plugin for Generic HTTP REST API
URL: https://github.com/apache/drill/pull/1892#discussion_r352253759
##########
File path: contrib/storage-http/README.md
##########
@@ -0,0 +1,31 @@
+
+# Generic API Storage Plugin
+This plugin is intended to enable you to query APIs over HTTP/REST. At this
point, the API reader will only accept JSON as input however in the future, it
may be possible to
+ add additional format readers to allow for APIs which return XML, CSV or
other formats.
Review comment:
I presume the response has to look like what the JSON reader expects? As
JSON objects with the fields delimited by object fields?
Unfortunately, the JSON which the JSON reader expects is not valid since
files usually contain a list of objects, which is not valid JSON. Valid JSON
would be an array of objects, which the JSON reader can handle. That is, the
following is a typical JSON file:
```
{a: 10, b: "foo"}
{a: 20, b: "bar"}
```
But the following is a typical JSON message response:
```
[ {a: 10, b: "foo"},
{a: 20, b: "bar"} ]
```
Or:
```
{ status: "OK",
records: [ {a: 10, b: "foo"},
{a: 20, b: "bar"} ]
}
```
How does this plugin handle those? Even worse, does it handle common forms
like:
```
{ status: "OK",
description: "Request accepted",
records: [ {a: 10, b: "foo"},
{a: 20, b: "bar"} ]
}
```
Or:
```
{ status: "OK",
records: [
{values: [ {key: "a", value: 10}, {key: "b", value: "foo"} ],
[ {key: "a", value: 20}, {key: "b", value: "bar"} ] ]
} }
```
That is, records are expressed as a series of key/value pairs so that the
column schema is not fixed. In some cases, I've seen such messages include a
"schema" section to explain the dynamic schema.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
