[
https://issues.apache.org/jira/browse/HAWQ-459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivram Mani updated HAWQ-459:
------------------------------
Description:
Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes
*table* as a path parameter and returns metadata for one given table.
Idea is to enhance this endpoint to support retrieving metadata for generic
datasources, i.e. Hive, HBase, HDFS, etc.
h1.Method supported
||*name*||
|GET|
h1.Path mapped
pxf/<protocolVersion>/Metadata/getMetadata
h1.Input parameters:
||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values||
|source|Type of source to fetch data from, should support Hive.|Yes|No|hive|
|container|Database, schema, folder, entity which describes logically separated
set of objects.|Yes|Yes|Valid schema/database/folder name or pattern including
asterisk|
|object|Table, relation, file, logically separated set of
records.|Yes|Yes|Valid table/file name or pattern including asterisk|
In case of Hive, object corresponds to table and container corresponds to
database or schema.
In case of HBase, object=table, container=schema.
In case of HDFS, object=file, container=folder.
h1. HTTP return codes:
||*Code*||*Meaning*||
|200|found some result set|
|400|not all required parameters passed|
|404|didn't find any results for given parameters, empty result set|
|405|called method other than GET|
|500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource
not supported, some other exceptions|
h1. Return data format for 200 code:
{code}
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
h1. Return data format for 404 code:
{code}
{
"PXFMetadata":[]
}
{code}
h1. Return data format for 500 code:
{code}
{
"PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
}
{code}
h1. Examples
Request:
{code}
curl -i
"http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*";
{code}
Response:
{code}
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 26 Feb 2016 03:33:53 GMT
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
was:
Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes
*table* as a path parameter and returns metadata for one given table.
Idea is to enhance this endpoint to support retrieving metadata for generic
datasources, i.e. Hive, HBase, HDFS, etc.
h1.Method supported
||*name*||
|GET|
h1.Path mapped
pxf/<protocolVersion>/Metadata/getMetadata
h1.Input parameters:
||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values||
|source|Type of source to fetch data from, should support Hive.|Yes|No|hive|
|container|Database, schema, folder, entity which describes logically separated
set of objects.|Yes|Yes|Valid schema/database/folder name or pattern including
asterisk|
|object|Table, relation, file, logically separated set of
records.|Yes|Yes|Valid table/file name or pattern including asterisk|
In case of Hive, object corresponds to table and container corresponds to
database or schema.
In case of HBase, object=table, container=schema.
In case of HDFS, object=file, container=folder.
h1. HTTP return codes:
||*code*||*meaning*||
|200|found some result set|
|400|not all required parameters passed|
|404|didn't find any results for given parameters, empty result set|
|405|called method other than GET|
|500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource
not supported, some other exceptions|
h1. Return data format for 200 code:
{code}
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
h1. Return data format for 404 code:
{code}
{
"PXFMetadata":[]
}
{code}
h1. Return data format for 500 code:
{code}
{
"PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
}
{code}
h1. Examples
Request:
{code}
curl -i
"http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*";
{code}
Response:
{code}
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 26 Feb 2016 03:33:53 GMT
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
> Enhance Metadata Rest API to support table(s)
> ---------------------------------------------
>
> Key: HAWQ-459
> URL: https://issues.apache.org/jira/browse/HAWQ-459
> Project: Apache HAWQ
> Issue Type: Sub-task
> Components: Hcatalog, PXF
> Reporter: Shivram Mani
> Assignee: Shivram Mani
> Fix For: 2.0.0
>
>
> Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes
> *table* as a path parameter and returns metadata for one given table.
> Idea is to enhance this endpoint to support retrieving metadata for generic
> datasources, i.e. Hive, HBase, HDFS, etc.
> h1.Method supported
> ||*name*||
> |GET|
> h1.Path mapped
> pxf/<protocolVersion>/Metadata/getMetadata
> h1.Input parameters:
> ||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values||
> |source|Type of source to fetch data from, should support Hive.|Yes|No|hive|
> |container|Database, schema, folder, entity which describes logically
> separated set of objects.|Yes|Yes|Valid schema/database/folder name or
> pattern including asterisk|
> |object|Table, relation, file, logically separated set of
> records.|Yes|Yes|Valid table/file name or pattern including asterisk|
> In case of Hive, object corresponds to table and container corresponds to
> database or schema.
> In case of HBase, object=table, container=schema.
> In case of HDFS, object=file, container=folder.
> h1. HTTP return codes:
> ||*Code*||*Meaning*||
> |200|found some result set|
> |400|not all required parameters passed|
> |404|didn't find any results for given parameters, empty result set|
> |405|called method other than GET|
> |500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource
> not supported, some other exceptions|
> h1. Return data format for 200 code:
> {code}
> {
> "PXFMetadata":[
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abc"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdef"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdefghk"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> }
> ]
> }
> {code}
> h1. Return data format for 404 code:
> {code}
> {
> "PXFMetadata":[]
> }
> {code}
> h1. Return data format for 500 code:
> {code}
> {
> "PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
> }
> {code}
> h1. Examples
> Request:
> {code}
> curl -i
> "http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*";
>
> {code}
> Response:
> {code}
> HTTP/1.1 200 OK
> Server: Apache-Coyote/1.1
> Content-Type: application/json
> Transfer-Encoding: chunked
> Date: Fri, 26 Feb 2016 03:33:53 GMT
> {
> "PXFMetadata":[
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abc"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdef"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdefghk"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
