Quanlong Huang has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/20110 )
Change subject: IMPALA-11901: Support COPY TESTCASE in local catalog mode ...................................................................... IMPALA-11901: Support COPY TESTCASE in local catalog mode COPY TESTCASE TO statement is used to export the metadata required for planning a given query. It will export catalog objects cached in coordinator side to a file. Users can transfer the metadata file to another debug cluster and import it by the COPY TESTCASE FROM statement. The statement will import the metadata to the new catalogd, including the topology of the original cluster. So an EXPLAIN on the same query can produce the same query plan for debugging frontend issues. However, the COPY TESTCASE statements are not supported in LocalCatalog mode. In the legacy catalog mode, coordinator has the same catalog representation of catalogd, i.e. coordinator just mirrors all the metadata from catalogd. So the export is simple - we can dump the metadata from coordinator side and they can be ingested into catalogd. However, in the local catalog mode, coordinator caches the metadata in a finer grained manner which differs a lot to the metadata in catalogd. To export the metadata that can be imported in catalogd, we need some transformation to convert them into catalog objects used in catalogd. This patch uses the fine-graiend metadata to construct the full thrift objects of THdfsTable. So coordinator can export them in the same way as we did in the legacy catalog mode. Tests: - We don't have two clusters in our dev env to verify the real use cases. Add sanity tests for exporting and improting testcase metadata. Change-Id: I02c1c76d7af15f28bdbc8d98d92a1553570e9e27 --- M fe/src/main/java/org/apache/impala/analysis/CopyTestCaseStmt.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java M tests/metadata/test_testcase_builder.py 8 files changed, 113 insertions(+), 29 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/20110/2 -- To view, visit http://gerrit.cloudera.org:8080/20110 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I02c1c76d7af15f28bdbc8d98d92a1553570e9e27 Gerrit-Change-Number: 20110 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <[email protected]>
