kgyrtkirk commented on a change in pull request #787: HIVE-22239
URL: https://github.com/apache/hive/pull/787#discussion_r333892055
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
##########
@@ -944,7 +948,7 @@ else
if(colTypeLowerCase.equals(serdeConstants.SMALLINT_TYPE_NAME)){
} else if (colTypeLowerCase.equals(serdeConstants.DATE_TYPE_NAME)) {
cs.setAvgColLen(JavaDataModel.get().lengthOfDate());
// epoch, days since epoch
- cs.setRange(0, 25201);
+ cs.setRange(DATE_RANGE_LOWER_LIMIT, DATE_RANGE_UPPER_LIMIT);
Review comment:
I feel like we should be setting this range to the maximum as possible
* let's say the user has data from 1920-1940
* and submits a query to <1930 which would mean 1/2 of the rows
* if the stats are estimated hive will go straight to 0 rows with the new
uniform estimation logic (earlier it was 1/3 or something)
so I think either this should be set for the whole range; or have toggle to
change how this supposed to work....or our archelogist end users will get angry
and grab some rusty :dagger: to cut our necks :D
...or we should tell them to calculate statistics properly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
