[
https://issues.apache.org/jira/browse/HAWQ-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongwu updated HAWQ-991:
------------------------
Description:
User should be able to use hawq register to register table files into a new
HAWQ cluster. It is some kind of protecting against corruption from users'
perspective. Users use the last-known-good metadata to update the portion of
catalog managing HDFS blocks. The table files or dictionary should be
backuped(such as using distcp) into the same path in the new HDFS setting. And
in this case, both AO and Parquet formats are supported.
Usage:
hawq extract -o t1.yml t1; // in HAWQ Cluster A
hawq register -h localhost -p 5432 -u me -d postgres -c t1.yml t1; // in HAWQ
Cluster B
was:
User should be able to use HAWQ Register utility to register HAWQ table
files/directories into a new HAWQ cluster so that the data can be copied from
one cluster to another, and the HAWQ catalog metadata is synchronized with
these HDFS HAWQ files.
The ask for this feature is basically to pass `hawq register` an input file (or
set of files) containing the last-known-good metadata that it can use to update
the portion of the catalog managing HDFS blocks. Prior to every new data load,
the user can leverage the `hawq extract` command to snapshot the metadata for
every table to protect against corruption / divergence.
> Add support for "HAWQ register" that could register tables by using "hawq
> extract" output
> -----------------------------------------------------------------------------------------
>
> Key: HAWQ-991
> URL: https://issues.apache.org/jira/browse/HAWQ-991
> Project: Apache HAWQ
> Issue Type: New Feature
> Components: Command Line Tools, External Tables
> Affects Versions: 2.0.1.0-incubating
> Reporter: hongwu
> Assignee: hongwu
> Fix For: 2.0.1.0-incubating
>
>
> User should be able to use hawq register to register table files into a new
> HAWQ cluster. It is some kind of protecting against corruption from users'
> perspective. Users use the last-known-good metadata to update the portion of
> catalog managing HDFS blocks. The table files or dictionary should be
> backuped(such as using distcp) into the same path in the new HDFS setting.
> And in this case, both AO and Parquet formats are supported.
> Usage:
> hawq extract -o t1.yml t1; // in HAWQ Cluster A
> hawq register -h localhost -p 5432 -u me -d postgres -c t1.yml t1; // in HAWQ
> Cluster B
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
