[jira] [Updated] (HIVE-16151) BytesBytesHashTable allocates large arrays
[ https://issues.apache.org/jira/browse/HIVE-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16151: Attachment: HIVE-16151.01.patch Updated to eliminate the null check on read path. > BytesBytesHashTable allocates large arrays > -- > > Key: HIVE-16151 > URL: https://issues.apache.org/jira/browse/HIVE-16151 > Project: Hive > Issue Type: Bug >Reporter: Prasanth Jayachandran >Assignee: Sergey Shelukhin > Attachments: HIVE-16151.01.patch, HIVE-16151.patch > > > These arrays cause GC pressure and also impose key count limitations on the > table. Wrt the latter, we won't be able to get rid of it without a 64-bit > hash function, but for now we can get rid of the former. If we need the > latter we'd add murmur64 and probably account for it differently for resize > (we don't want to blow up the hashtable by 4 bytes/key in the common case > where #of keys is less than ~1.5B :)) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16151) BytesBytesHashTable allocates large arrays
[ https://issues.apache.org/jira/browse/HIVE-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16151: Attachment: HIVE-16151.patch The patch. [~mmccline] can you take a look? > BytesBytesHashTable allocates large arrays > -- > > Key: HIVE-16151 > URL: https://issues.apache.org/jira/browse/HIVE-16151 > Project: Hive > Issue Type: Bug >Reporter: Prasanth Jayachandran >Assignee: Sergey Shelukhin > Attachments: HIVE-16151.patch > > > These arrays cause GC pressure and also impose key count limitations on the > table. Wrt the latter, we won't be able to get rid of it without a 64-bit > hash function, but for now we can get rid of the former. If we need the > latter we'd add murmur64 and probably account for it differently for resize > (we don't want to blow up the hashtable by 4 bytes/key in the common case > where #of keys is less than ~1.5B :)) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16151) BytesBytesHashTable allocates large arrays
[ https://issues.apache.org/jira/browse/HIVE-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16151: Status: Patch Available (was: Open) > BytesBytesHashTable allocates large arrays > -- > > Key: HIVE-16151 > URL: https://issues.apache.org/jira/browse/HIVE-16151 > Project: Hive > Issue Type: Bug >Reporter: Prasanth Jayachandran >Assignee: Sergey Shelukhin > Attachments: HIVE-16151.patch > > > These arrays cause GC pressure and also impose key count limitations on the > table. Wrt the latter, we won't be able to get rid of it without a 64-bit > hash function, but for now we can get rid of the former. If we need the > latter we'd add murmur64 and probably account for it differently for resize > (we don't want to blow up the hashtable by 4 bytes/key in the common case > where #of keys is less than ~1.5B :)) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16151) BytesBytesHashTable allocates large arrays
[ https://issues.apache.org/jira/browse/HIVE-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16151: Description: These arrays cause GC pressure and also impose key count limitations on the table. Wrt the latter, we won't be able to get rid of it without a 64-bit hash function, but for now we can get rid of the former. If we need the latter we'd add murmur64 and probably account for it differently for resize (we don't want to blow up the hashtable by 4 bytes/key in the common case where #of keys is less than ~1.5B :)) (was: These arrays cause GC pressure and also impose key count limitations on the table. Wrt the latter, we won't be able to get rid of it without a 64-bit hash function, but for now we can get rid of the former.) > BytesBytesHashTable allocates large arrays > -- > > Key: HIVE-16151 > URL: https://issues.apache.org/jira/browse/HIVE-16151 > Project: Hive > Issue Type: Bug >Reporter: Prasanth Jayachandran >Assignee: Sergey Shelukhin > > These arrays cause GC pressure and also impose key count limitations on the > table. Wrt the latter, we won't be able to get rid of it without a 64-bit > hash function, but for now we can get rid of the former. If we need the > latter we'd add murmur64 and probably account for it differently for resize > (we don't want to blow up the hashtable by 4 bytes/key in the common case > where #of keys is less than ~1.5B :)) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
