[ https://issues.apache.org/jira/browse/IMPALA-8005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on IMPALA-8005 started by Anurag Mantripragada. ---------------------------------------------------- > Randomize partitioning exchanges destinations > --------------------------------------------- > > Key: IMPALA-8005 > URL: https://issues.apache.org/jira/browse/IMPALA-8005 > Project: IMPALA > Issue Type: Improvement > Components: Distributed Exec > Affects Versions: Impala 3.1.0 > Reporter: Michael Ho > Assignee: Anurag Mantripragada > Priority: Major > Labels: ramp-up > > Currently, we use the same hash seed for partitioning exchanges at the > sender. For a table with skew in distribution in the shuffling keys, multiple > queries using the same shuffling keys for exchanges will end up hashing to > the same destination fragments running on particular host and potentially > overloading that host. > We should consider using the query id or other query specific information to > seed the hashing function to randomize the destinations for different > queries. Thanks to [~tlipcon] for pointing this problem out. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org