[
https://issues.apache.org/jira/browse/HAWQ-459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivram Mani updated HAWQ-459:
------------------------------
Description:
Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes
*table* as a path parameter and returns metadata for one given table.
Idea is to enhance this endpoint to support retrieving metadata for generic
datasources, i.e. Hive, HBase, HDFS, etc and support retrieving metadata for
multiple objects based on wildcard pattern
h1.Method supported
||*name*||
|GET|
h1.Path mapped
pxf/<protocolVersion>/Metadata/getMetadata
h1.Input parameters:
||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values||
|source|Type of source to fetch data from, should support Hive.|Yes|No|hive|
|container|Database, schema, folder, entity which describes logically separated
set of objects.|Yes|Yes|Valid schema/database/folder name or pattern including
asterisk|
|object|Table, relation, file, logically separated set of
records.|Yes|Yes|Valid table/file name or pattern including asterisk|
In case of Hive, object corresponds to table and container corresponds to
database or schema.
In case of HBase, object=table, container=schema.
In case of HDFS, object=file, container=folder.
h1. HTTP return codes:
||*Code*||*Meaning*||
|200|found some result set|
|400|not all required parameters passed|
|404|didn't find any results for given parameters, empty result set|
|405|called method other than GET|
|500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource
not supported, some other exceptions|
h1. Return data format for 200 code:
{code}
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
h1. Return data format for 404 code:
{code}
{
"PXFMetadata":[]
}
{code}
h1. Return data format for 500 code:
{code}
{
"PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
}
{code}
h1. Examples
Request:
{code}
curl -i
"http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*";
{code}
Response:
{code}
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 26 Feb 2016 03:33:53 GMT
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
was:
Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes
*table* as a path parameter and returns metadata for one given table.
Idea is to enhance this endpoint to support retrieving metadata for generic
datasources, i.e. Hive, HBase, HDFS, etc.
h1.Method supported
||*name*||
|GET|
h1.Path mapped
pxf/<protocolVersion>/Metadata/getMetadata
h1.Input parameters:
||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values||
|source|Type of source to fetch data from, should support Hive.|Yes|No|hive|
|container|Database, schema, folder, entity which describes logically separated
set of objects.|Yes|Yes|Valid schema/database/folder name or pattern including
asterisk|
|object|Table, relation, file, logically separated set of
records.|Yes|Yes|Valid table/file name or pattern including asterisk|
In case of Hive, object corresponds to table and container corresponds to
database or schema.
In case of HBase, object=table, container=schema.
In case of HDFS, object=file, container=folder.
h1. HTTP return codes:
||*Code*||*Meaning*||
|200|found some result set|
|400|not all required parameters passed|
|404|didn't find any results for given parameters, empty result set|
|405|called method other than GET|
|500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource
not supported, some other exceptions|
h1. Return data format for 200 code:
{code}
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
h1. Return data format for 404 code:
{code}
{
"PXFMetadata":[]
}
{code}
h1. Return data format for 500 code:
{code}
{
"PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
}
{code}
h1. Examples
Request:
{code}
curl -i
"http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*";
{code}
Response:
{code}
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 26 Feb 2016 03:33:53 GMT
{
"PXFMetadata":[
{
"object":{
"source":"hive",
"container":"default",
"object":"abc"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdef"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
},
{
"object":{
"source":"hive",
"container":"default",
"object":"abcdefghk"
},
"fields":[
{
"name":"a",
"type":"int"
},
{
"name":"b",
"type":"float"
}
]
}
]
}
{code}
> Enhance Metadata Rest API to support table(s)
> ---------------------------------------------
>
> Key: HAWQ-459
> URL: https://issues.apache.org/jira/browse/HAWQ-459
> Project: Apache HAWQ
> Issue Type: Sub-task
> Components: Hcatalog, PXF
> Reporter: Shivram Mani
> Assignee: Shivram Mani
> Fix For: 2.0.0
>
>
> Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes
> *table* as a path parameter and returns metadata for one given table.
> Idea is to enhance this endpoint to support retrieving metadata for generic
> datasources, i.e. Hive, HBase, HDFS, etc and support retrieving metadata for
> multiple objects based on wildcard pattern
> h1.Method supported
> ||*name*||
> |GET|
> h1.Path mapped
> pxf/<protocolVersion>/Metadata/getMetadata
> h1.Input parameters:
> ||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values||
> |source|Type of source to fetch data from, should support Hive.|Yes|No|hive|
> |container|Database, schema, folder, entity which describes logically
> separated set of objects.|Yes|Yes|Valid schema/database/folder name or
> pattern including asterisk|
> |object|Table, relation, file, logically separated set of
> records.|Yes|Yes|Valid table/file name or pattern including asterisk|
> In case of Hive, object corresponds to table and container corresponds to
> database or schema.
> In case of HBase, object=table, container=schema.
> In case of HDFS, object=file, container=folder.
> h1. HTTP return codes:
> ||*Code*||*Meaning*||
> |200|found some result set|
> |400|not all required parameters passed|
> |404|didn't find any results for given parameters, empty result set|
> |405|called method other than GET|
> |500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource
> not supported, some other exceptions|
> h1. Return data format for 200 code:
> {code}
> {
> "PXFMetadata":[
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abc"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdef"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdefghk"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> }
> ]
> }
> {code}
> h1. Return data format for 404 code:
> {code}
> {
> "PXFMetadata":[]
> }
> {code}
> h1. Return data format for 500 code:
> {code}
> {
> "PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
> }
> {code}
> h1. Examples
> Request:
> {code}
> curl -i
> "http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*";
>
> {code}
> Response:
> {code}
> HTTP/1.1 200 OK
> Server: Apache-Coyote/1.1
> Content-Type: application/json
> Transfer-Encoding: chunked
> Date: Fri, 26 Feb 2016 03:33:53 GMT
> {
> "PXFMetadata":[
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abc"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdef"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> },
> {
> "object":{
> "source":"hive",
> "container":"default",
> "object":"abcdefghk"
> },
> "fields":[
> {
> "name":"a",
> "type":"int"
> },
> {
> "name":"b",
> "type":"float"
> }
> ]
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
