[ https://issues.apache.org/jira/browse/HAWQ-459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shivram Mani updated HAWQ-459: ------------------------------ Description: Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes *table* as a path parameter and returns metadata for one given table. Idea is to enhance this endpoint to support retrieving metadata for generic datasources, i.e. Hive, HBase, HDFS, etc. h1.Method supported ||*name*|| |GET| h1.Path mapped pxf/<protocolVersion>/Metadata/getMetadata h1.Input parameters: ||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values|| |source|Type of source to fetch data from, should support Hive.|Yes|No|hive| |container|Database, schema, folder, entity which describes logically separated set of objects.|Yes|Yes|Valid schema/database/folder name or pattern including asterisk| |object|Table, relation, file, logically separated set of records.|Yes|Yes|Valid table/file name or pattern including asterisk| In case of Hive, object corresponds to table and container corresponds to database or schema. In case of HBase, object=table, container=schema. In case of HDFS, object=file, container=folder. h1. HTTP return codes: ||*Code*||*Meaning*|| |200|found some result set| |400|not all required parameters passed| |404|didn't find any results for given parameters, empty result set| |405|called method other than GET| |500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource not supported, some other exceptions| h1. Return data format for 200 code: {code} { "PXFMetadata":[ { "object":{ "source":"hive", "container":"default", "object":"abc" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdef" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdefghk" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] } ] } {code} h1. Return data format for 404 code: {code} { "PXFMetadata":[] } {code} h1. Return data format for 500 code: {code} { "PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"} } {code} h1. Examples Request: {code} curl -i "http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*" {code} Response: {code} HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Content-Type: application/json Transfer-Encoding: chunked Date: Fri, 26 Feb 2016 03:33:53 GMT { "PXFMetadata":[ { "object":{ "source":"hive", "container":"default", "object":"abc" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdef" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdefghk" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] } ] } {code} was: Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes *table* as a path parameter and returns metadata for one given table. Idea is to enhance this endpoint to support retrieving metadata for generic datasources, i.e. Hive, HBase, HDFS, etc. h1.Method supported ||*name*|| |GET| h1.Path mapped pxf/<protocolVersion>/Metadata/getMetadata h1.Input parameters: ||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values|| |source|Type of source to fetch data from, should support Hive.|Yes|No|hive| |container|Database, schema, folder, entity which describes logically separated set of objects.|Yes|Yes|Valid schema/database/folder name or pattern including asterisk| |object|Table, relation, file, logically separated set of records.|Yes|Yes|Valid table/file name or pattern including asterisk| In case of Hive, object corresponds to table and container corresponds to database or schema. In case of HBase, object=table, container=schema. In case of HDFS, object=file, container=folder. h1. HTTP return codes: ||*code*||*meaning*|| |200|found some result set| |400|not all required parameters passed| |404|didn't find any results for given parameters, empty result set| |405|called method other than GET| |500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource not supported, some other exceptions| h1. Return data format for 200 code: {code} { "PXFMetadata":[ { "object":{ "source":"hive", "container":"default", "object":"abc" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdef" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdefghk" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] } ] } {code} h1. Return data format for 404 code: {code} { "PXFMetadata":[] } {code} h1. Return data format for 500 code: {code} { "PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"} } {code} h1. Examples Request: {code} curl -i "http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*" {code} Response: {code} HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Content-Type: application/json Transfer-Encoding: chunked Date: Fri, 26 Feb 2016 03:33:53 GMT { "PXFMetadata":[ { "object":{ "source":"hive", "container":"default", "object":"abc" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdef" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] }, { "object":{ "source":"hive", "container":"default", "object":"abcdefghk" }, "fields":[ { "name":"a", "type":"int" }, { "name":"b", "type":"float" } ] } ] } {code} > Enhance Metadata Rest API to support table(s) > --------------------------------------------- > > Key: HAWQ-459 > URL: https://issues.apache.org/jira/browse/HAWQ-459 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Hcatalog, PXF > Reporter: Shivram Mani > Assignee: Shivram Mani > Fix For: 2.0.0 > > > Currently we have *pxf/v14/Metadata/getTableMetadata* endpoint, which takes > *table* as a path parameter and returns metadata for one given table. > Idea is to enhance this endpoint to support retrieving metadata for generic > datasources, i.e. Hive, HBase, HDFS, etc. > h1.Method supported > ||*name*|| > |GET| > h1.Path mapped > pxf/<protocolVersion>/Metadata/getMetadata > h1.Input parameters: > ||Name||Description||Is mandatory?||Could be a wildcard?||Valid input values|| > |source|Type of source to fetch data from, should support Hive.|Yes|No|hive| > |container|Database, schema, folder, entity which describes logically > separated set of objects.|Yes|Yes|Valid schema/database/folder name or > pattern including asterisk| > |object|Table, relation, file, logically separated set of > records.|Yes|Yes|Valid table/file name or pattern including asterisk| > In case of Hive, object corresponds to table and container corresponds to > database or schema. > In case of HBase, object=table, container=schema. > In case of HDFS, object=file, container=folder. > h1. HTTP return codes: > ||*Code*||*Meaning*|| > |200|found some result set| > |400|not all required parameters passed| > |404|didn't find any results for given parameters, empty result set| > |405|called method other than GET| > |500|unable to access underlying metastore(hive, hbase, hdfs etc), datasource > not supported, some other exceptions| > h1. Return data format for 200 code: > {code} > { > "PXFMetadata":[ > { > "object":{ > "source":"hive", > "container":"default", > "object":"abc" > }, > "fields":[ > { > "name":"a", > "type":"int" > }, > { > "name":"b", > "type":"float" > } > ] > }, > { > "object":{ > "source":"hive", > "container":"default", > "object":"abcdef" > }, > "fields":[ > { > "name":"a", > "type":"int" > }, > { > "name":"b", > "type":"float" > } > ] > }, > { > "object":{ > "source":"hive", > "container":"default", > "object":"abcdefghk" > }, > "fields":[ > { > "name":"a", > "type":"int" > }, > { > "name":"b", > "type":"float" > } > ] > } > ] > } > {code} > h1. Return data format for 404 code: > {code} > { > "PXFMetadata":[] > } > {code} > h1. Return data format for 500 code: > {code} > { > "PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"} > } > {code} > h1. Examples > Request: > {code} > curl -i > "http://localhost:51200/pxf/v14/Metadata/getMetadata?source=hive&container=default&object=abc*" > > {code} > Response: > {code} > HTTP/1.1 200 OK > Server: Apache-Coyote/1.1 > Content-Type: application/json > Transfer-Encoding: chunked > Date: Fri, 26 Feb 2016 03:33:53 GMT > { > "PXFMetadata":[ > { > "object":{ > "source":"hive", > "container":"default", > "object":"abc" > }, > "fields":[ > { > "name":"a", > "type":"int" > }, > { > "name":"b", > "type":"float" > } > ] > }, > { > "object":{ > "source":"hive", > "container":"default", > "object":"abcdef" > }, > "fields":[ > { > "name":"a", > "type":"int" > }, > { > "name":"b", > "type":"float" > } > ] > }, > { > "object":{ > "source":"hive", > "container":"default", > "object":"abcdefghk" > }, > "fields":[ > { > "name":"a", > "type":"int" > }, > { > "name":"b", > "type":"float" > } > ] > } > ] > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)