[
https://issues.apache.org/jira/browse/HIVE-19671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521790#comment-16521790
]
Rui Li commented on HIVE-19671:
-------------------------------
We can check all RS and look for non-deterministic UDF in partition keys --
{{FunctionRegistry::isDeterministic}} can be used.
I noted Hive itself may also use non-deterministic partitioning, e.g. to handle
skewed GBY, we first shuffle randomly to do partial aggregation. Do you think
it makes sense to print a warning for that?
> Distribute by rand() can lead to data inconsistency
> ---------------------------------------------------
>
> Key: HIVE-19671
> URL: https://issues.apache.org/jira/browse/HIVE-19671
> Project: Hive
> Issue Type: Bug
> Reporter: Rui Li
> Assignee: Rui Li
> Priority: Major
>
> Noticed the following queries can give different results:
> {code}
> select count(*) from tbl;
> select count(*) from (select * from tbl distribute by rand()) a;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
