-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14890/
-----------------------------------------------------------
(Updated Nov. 14, 2014, 12:03 a.m.)
Review request for hive, Ashutosh Chauhan and Thejas Nair.
Changes
-------
Rebased on latest trunk.
Bugs: HIVE-5631
https://issues.apache.org/jira/browse/HIVE-5631
Repository: hive-git
Description (updated)
-------
Repro steps:
CREATE DATABASE skewtest;
USE skewtest;
CREATE TABLE skew (id bigint, acct string) SKEWED BY (acct) ON ('CC','CH');
CREATE INDEX skew_indx ON TABLE skew (id) as
'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED
REBUILD;
Last DDL fails with following error.
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. InvalidObjectException(message:Invalid
skew column [acct])
When creating a table, Hive has sanity tests to make sure the columns have
proper names and the skewed columns are subset of the table columns. Here we
fail because index table has skewed column info. Index tables's skewed columns
include {acct} and the columns are {id, _bucketname, _offsets}. As the skewed
column {acct} is not part of the table columns Hive throws the exception.
The reason why Index table got skewed column info even though its definition
has no such info is: When creating the index table a deep copy of the base
table's StorageDescriptor (SD) (in this case 'skew') is made. And in that
copied SD, index specific parameters are set and unrelated parameters are
reset. Here skewed column info is not reset (there are few other params that
are not reset). That's why the index table contains the skewed column info.
Fix: Instead of deep copying the base table StorageDescriptor, create a new one
from gathered info. This way it avoids the index table to inherit unnecessary
properties in SD from base table.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java b900627
ql/src/test/queries/clientpositive/index_skewtable.q PRE-CREATION
ql/src/test/results/clientpositive/index_skewtable.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/14890/diff/
Testing
-------
Added unittest and ran the index related unittest queries
Thanks,
Venki Korukanti
