Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9445
Change subject: IMPALA-6579: Always force reload Kudu tables for dataload ...................................................................... IMPALA-6579: Always force reload Kudu tables for dataload When loading from an up-to-date snapshot, dataload will load all of the metadata and load data into HDFS. Then, it will skip load-data.py for functional/exhaustive, tpch/core, and tpcds/core. It will invoke a special round of load-data.py calls to populate Kudu tables, and it always runs these with a force reload. However, when loading from an old snapshot, dataload will still load all of the metadata and load the data into HDFS, but then it will still invoke load-data.py for functional/exhaustive, tpch/core, and tpcds/core. These invocations mostly do DDLs with very few load statements. However, these invocations are a problem for Kudu. The metadata of Impala tables referencing Kudu entities have been imported along with all the other metadata, but the Kudu entities have not been created, as they are separate from HDFS. This means that Kudu tables are not really valid in this circumstance. Since Kudu has been added to the list of data formats for tpch/core (see IMPALA-6475), load-data.py with tpch/core will attempt to insert into these invalid Kudu tables. To avoid this, always force reload any Kudu tables. generate-schema-statements.py will always generate a drop table statement before any create of a Kudu table. This guarantees that the create will also create the corresponding Kudu entity. Change-Id: I2d07f3513c543e2590f2f62b96b37472316868ee --- M testdata/bin/generate-schema-statements.py 1 file changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/9445/1 -- To view, visit http://gerrit.cloudera.org:8080/9445 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2d07f3513c543e2590f2f62b96b37472316868ee Gerrit-Change-Number: 9445 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com>
