Rajesh Balamohan created SPARK-14551:
----------------------------------------
Summary: Reduce number of NN calls in OrcRelation with
FileSourceStrategy mode
Key: SPARK-14551
URL: https://issues.apache.org/jira/browse/SPARK-14551
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Rajesh Balamohan
Priority: Minor
When FileSourceStrategy is used, record reader is created which incurs a NN
call internally. Later in OrcRelation.unwrapOrcStructs, it ends ups reading the
file information to get the ObjectInspector. This incurs additional NN call. It
would be good to avoid this additional NN call (specifically for partitioned
datasets)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
