[
https://issues.apache.org/jira/browse/PHOENIX-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658835#comment-15658835
]
James Taylor edited comment on PHOENIX-3475 at 11/12/16 2:39 AM:
-----------------------------------------------------------------
Wanted to summarize our offline conversation. If you want to come up with the
HBase scan to get all index data (including indexes on views), you could do
something like this:
- Run "global" view and index creation on both source and target clusters. This
ensures that the expected HBase tables will be created.
- Iterate through all non multi-tenant indexes with a statement like this:
{code}
conn.getMetaData().getTables("", "", null, new String[] {"INDEX"});
{code}
- For each row you get back, you could construct a query like this: {{SELECT *
FROM <table_schem>.<table_name>}}
- Instead of running the query, you can just compile it like this:
{code}
QueryPlan plan = statement.unwrap(PhoenixStatement.class).compileQuery(query);
String hbaseTableName =
plan.getContext().getCurrentTable().getTable().getPhysicalName();
Scan scan = plan.getContext().getScan();
{code}
- Use the information in the scan (start/stop row, column family map, etc) as
needed
FYI, we're also working on an asynchronous UPSERT SELECT (which would run a MR
job behind the scenes). You can talk with [~singamteja] about that. You could
then just issue an UPSERT SELECT for every index instead of dipping below the
supported APIs to get to the physical table and scan information.
was (Author: jamestaylor):
Wanted to summarize our offline conversation. If you want to come up with the
HBase scan to get all index data (including indexes on views), you could do
something like this:
- Run "global" view and index creation on both source and target clusters. This
ensures that the expected HBase tables will be created.
- Iterate through all non multi-tenant indexes with a statement like this:
{code}
conn.getMetaData().getTables("", "", null, new String[] {"INDEX"});
{code}
- For each row you get back, you could construct a query like this: {{SELECT *
FROM <table_schem>.<table_name>}}
- Instead of running the query, you can just compile it like this:
{code}
QueryPlan plan = statement.unwrap(PhoenixStatement.class).compileQuery(query);
String hbaseTableName =
plan.getContext().getCurrentTable().getTable().getPhysicalName();
Scan scan = plan.getContext().getScan();
{code}
> MetaData #getTables() API doesn't return view indexes
> -----------------------------------------------------
>
> Key: PHOENIX-3475
> URL: https://issues.apache.org/jira/browse/PHOENIX-3475
> Project: Phoenix
> Issue Type: Bug
> Reporter: Akshita Malhotra
> Fix For: 4.9.0
>
>
> HBase migration tool uses DatabaseMetadata#getTables() API to retrieve the
> tables for copying data. We have found that API doesn't return base index
> tables ( _IDX_<base table name>)
> For testing purposes, we issue following DDL to generate the view and the
> corresponding view index:
> -CREATE VIEW IF NOT EXISTS MIGRATIONTEST_VIEW (OLD_VALUE_VIEW varchar) AS
> SELECT * FROM MIGRATIONTEST WHERE OLD_VALUE like 'E%'
> -CREATE INDEX IF NOT EXISTS MIGRATIONTEST_VIEW_IDX ON MIGRATIONTEST_VIEW
> (OLD_VALUE_VIEW)
> By using HBase API, we were able to confirm that base index table
> (_IDX_MIGRATIONTEST) is created.
> Both jdbc DatabaseMetadata API and P* getMetaDataCache API doesn't seem to
> be returning view indexes. Also P*MetaData #getTableRef API return
> "TableNotFoundException" when attempted to fetch PTable corresponding to the
> base index table name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
