Pranav Kumar Agarwal created LENS-511:
-----------------------------------------
Summary: identify facts associated with underlying hive/jdbc tables
Key: LENS-511
URL: https://issues.apache.org/jira/browse/LENS-511
Project: Apache Lens
Issue Type: Improvement
Components: api, cube
Affects Versions: 2.0
Reporter: Pranav Kumar Agarwal
Consider following storage table defn inside a fact:
<x_fact_table cube_name="sample_cube" name="fact2" weight="200.0"
xmlns="uri:lens:cube:0.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
...
<storage_tables>
<storage_table>
<update_periods>
<update_period>HOURLY</update_period>
<update_period>DAILY</update_period>
</update_periods>
<storage_name>local</storage_name>
<table_desc external="true" field_delimiter=","
table_location="/tmp/examples/fact2_local">
<part_cols>
<column comment="Time column" name="dt" type="STRING"/>
</part_cols>
<time_part_cols>dt</time_part_cols>
</table_desc>
</storage_table>
</storage_tables>
</x_fact_table>
In an event a new partition is added to the external table location
"/tmp/examples/fact2_local" then I wish to add a new partition on the
fact2, however I have no way to find what all facts are built on
external table location "/tmp/examples/fact2_local". We can possibly do
it by matching the location, however that doesn't seem to be quite
nice.. Consider for a JDBC source the table_location is kind of dummy as
its not really used to query the content from that location. JDBCDriver
expects a table with the name as storageName_factName in the target
datastore, thus no indication on which all facts to be updated.
Problem Statement: Current storage_table definition doesn't give me
enough detail to find where all partitions needs to be added given a new
partition is added to an external table.
I propose that we add a table_name property as part of table_desc and
provide following API's:
GET /storages/{storageName}/tableNames/
GET /storages/{storageName}/tableNames/{tableName}/facts
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
