[
https://issues.apache.org/jira/browse/HIVE-25159?focusedWorklogId=602091&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602091
]
ASF GitHub Bot logged work on HIVE-25159:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/May/21 04:48
Start Date: 26/May/21 04:48
Worklog Time Spent: 10m
Work Description: ShubhamChaurasia opened a new pull request #2318:
URL: https://github.com/apache/hive/pull/2318
The issue with order by in llap external client is described in
https://issues.apache.org/jira/browse/HIVE-25159
### What changes were proposed in this pull request?
To resolve the above issue, we propose removing of order by handling in the
llap external client library and let the clients do it at their end.
### Why are the changes needed?
1. These changes are needed to make llap external client queries more
performant, earlier they used to generate only a single split for order by
queries.
2. Removing order by support also helps us to get rid of exception arising
to due false detection of order by clause (it was specific to spark-llap and
was done for that only).
### Does this PR introduce _any_ user-facing change?
It removes a config - hive.llap.external.splits.order.by.force.single.split
### How was this patch tested?
Through existing unit tests
- org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits
- org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 602091)
Remaining Estimate: 0h
Time Spent: 10m
> Remove support for ordered results in llap external client library
> ------------------------------------------------------------------
>
> Key: HIVE-25159
> URL: https://issues.apache.org/jira/browse/HIVE-25159
> Project: Hive
> Issue Type: Bug
> Components: Clients, Hive
> Reporter: Shubham Chaurasia
> Assignee: Shubham Chaurasia
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently when querying via llap external client framework, in case of order
> by queries -
> 1. Due to the fact that spark-llap used to wrap actual query in a subquery as
> mentioned in [HIVE-19794|https://issues.apache.org/jira/browse/HIVE-19794]
> a) We had to detect order by like -
> {code}
> orderByQuery = plan.getQueryProperties().hasOrderBy() ||
> plan.getQueryProperties().hasOuterOrderBy();
> {code}
> Due to this we recently saw an exception like below for one of the queries
> that did not have an outer order by (It was having an order by in a subquery)
> {code}
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.lang.IllegalStateException: Requested to generate single split. Paths
> and fileStatuses are expected to be 1. Got paths: 1 fileStatuses: 7
> {code}
> b) Also we had to disable following optimization -
> {code}
> HiveConf.setBoolVar(conf, ConfVars.HIVE_REMOVE_ORDERBY_IN_SUBQUERY, false);
> {code}
> 2. By default we have
> {{hive.llap.external.splits.order.by.force.single.split=true}} which forces
> us to generate single split leading to performance bottleneck.
> We should remove ordering support altogether from llap external client repo
> and let clients handle it at their end.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
