Soumyakanti Das created HIVE-28938:
--------------------------------------
Summary: Error in LATERAL VIEW with non native tables due to
presence of incorrect virtual columns in RowResolver
Key: HIVE-28938
URL: https://issues.apache.org/jira/browse/HIVE-28938
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 4.0.1
Reporter: Soumyakanti Das
Assignee: Soumyakanti Das
To reproduce, run:
{noformat}
create external table test(id int, arr array<string>) stored by iceberg;
insert into test values (1, array("a", "b")), (2, array("c", "d")), (3,
array("e", "f"));
select * from test
lateral view explode(arr) tbl1 as name
lateral view explode(arr) tbl2 as name1;
{noformat}
Error:
{noformat}
org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Invalid column
reference 'BLOCK__OFFSET__INSIDE__FILE'
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:14032)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:13971)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:5027)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4801)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:11888)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:11827)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12812)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12665)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12696)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12678)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:688)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13572)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:489)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:324)
at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:227)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:108){noformat}
BLOCK__OFFSET__INSIDE__FILE should not be present for non native tables, but it
is present because of the misclassification of ICEBERG tables as
TableType.NATIVE in the method [obtainTableType(Table
tabMetaData)|https://github.com/apache/hive/blob/e7cdc0cac935b8e2c573edad9d6e790d8beeee40/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L3272].
This returns NATIVE for ICEBERG tables and subsequently adds incorrect virtual
columns in
[genTableLogicalPlan|https://github.com/apache/hive/blob/e7cdc0cac935b8e2c573edad9d6e790d8beeee40/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L3079].
Virtual column adding logic should be similar to
[here|https://github.com/apache/hive/blob/e7cdc0cac935b8e2c573edad9d6e790d8beeee40/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L11990].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
