[
https://issues.apache.org/jira/browse/HIVE-27782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pravin Sinha updated HIVE-27782:
--------------------------------
Description:
The actual buffer size during data transfer from Hs2 to beeline connection
is calculated based on (data Size *
numberOfByteCharIsRepresentedBasedOnCharSet). In UTF-8, each char is
represented in 2 bytes and hence when the response size is >1GB (in char data
type) it gets translated to >2GB (in byte[] representation) and hence hits the
2GB limit of thrift layer in SASL transport impl.
If the HS2 is running with {{*-XX:+ExitOnOutOfMemoryError*}} flag, the HS2
process in above scenario will crash with OOM regarding of how much amount of
heap it is configured with.
was:
The actual buffer size during data transfer from Hs2 to beeline connection
is calculated based on (data Size *
numberOfByteCharIsRepresentedBasedOnCharSet). In UTF-8, each char is
represented in 2 bytes and hence when the response size is >1GB (in char data
type) it gets translated to >2GB (in byte[] representation) and hence hits the
2GB limit of thrift layer in SASL transport impl.
> HS2 crashes with OOM even though there is enough heap space
> -----------------------------------------------------------
>
> Key: HIVE-27782
> URL: https://issues.apache.org/jira/browse/HIVE-27782
> Project: Hive
> Issue Type: Bug
> Reporter: Pravin Sinha
> Assignee: Pravin Sinha
> Priority: Major
>
> The actual buffer size during data transfer from Hs2 to beeline connection
> is calculated based on (data Size *
> numberOfByteCharIsRepresentedBasedOnCharSet). In UTF-8, each char is
> represented in 2 bytes and hence when the response size is >1GB (in char data
> type) it gets translated to >2GB (in byte[] representation) and hence hits
> the 2GB limit of thrift layer in SASL transport impl.
> If the HS2 is running with {{*-XX:+ExitOnOutOfMemoryError*}} flag, the HS2
> process in above scenario will crash with OOM regarding of how much amount of
> heap it is configured with.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
