[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Fix Version/s: 0.14.0 > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Assignee: Prasanth J >Priority: Critical > Fix For: 0.14.0 > > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.2.patch, HIVE-8078.3.patch, HIVE-8078.4.patch, HIVE-8078.5.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks [~gopalv]! > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Assignee: Prasanth J >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.2.patch, HIVE-8078.3.patch, HIVE-8078.4.patch, HIVE-8078.5.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: HIVE-8078.5.patch fixes relevant test failures. > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Assignee: Prasanth J >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.2.patch, HIVE-8078.3.patch, HIVE-8078.4.patch, HIVE-8078.5.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: HIVE-8078.4.patch > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Assignee: Prasanth J >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.2.patch, HIVE-8078.3.patch, HIVE-8078.4.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: HIVE-8078.3.patch > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Assignee: Prasanth J >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.2.patch, HIVE-8078.3.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: HIVE-8078.2.patch Fixes HIVE QA test failures. > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.2.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: (was: HIVE-8078.1.patch) > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Status: Patch Available (was: Open) > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.1, 0.14.0 >Reporter: Tim Patterson >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.1.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: HIVE-8078.1.patch > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch, > HIVE-8078.1.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-8078: - Attachment: HIVE-8078.1.patch [~tim.patterson] Thanks for reporting the issue and the test case. Here is the patch that should fix the overflow issue. > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Priority: Critical > Attachments: HIVE-8078-testcase.patch, HIVE-8078.1.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Patterson updated HIVE-8078: Attachment: HIVE-8078-testcase.patch Unittest to show problematic behaviour > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.14.0, 0.13.1 >Reporter: Tim Patterson >Priority: Critical > Attachments: HIVE-8078-testcase.patch > > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Patterson updated HIVE-8078: Status: Open (was: Patch Available) > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.1, 0.14.0 >Reporter: Tim Patterson >Priority: Critical > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8078) ORC Delta encoding corrupts data when delta overflows long
[ https://issues.apache.org/jira/browse/HIVE-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Patterson updated HIVE-8078: Status: Patch Available (was: Open) Unit test patch > ORC Delta encoding corrupts data when delta overflows long > -- > > Key: HIVE-8078 > URL: https://issues.apache.org/jira/browse/HIVE-8078 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.1, 0.14.0 >Reporter: Tim Patterson >Priority: Critical > > There is an issue with the integer encoding that can cause corruption in > certain cases. > The following 3 longs cause this failure. > 4513343538618202711 > 2911390882471569739 > -9181829309989854913 > I believe that even though the numbers are in decreasing order, the delta > between the last two numbers overflows causing a positive delta, in this case > the last digit ends up being corrupted (the delta is applied for the wrong > sign resulting in -3442132998776557225 instead of -9181829309989854913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)