[
https://issues.apache.org/jira/browse/TRAFODION-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396662#comment-15396662
]
David Wayne Birdsall edited comment on TRAFODION-2131 at 7/27/16 11:37 PM:
---------------------------------------------------------------------------
Further debugging and testing revealed that in the first case UPDATE STATS was
using a SELECT and casting the interval column to a TINYINT.
In both cases, UPDATE STATS was using a rowset to retrieve the data values. It
assumed the values would be 1 byte each. However, the compiler casts a TINYINT
output to a 2-byte integer. The row set fetches would write two bytes per value
to the rowset buffer. The high byte of one value would be overwritten by the
low byte of the next (since this is was on a little-endian machine), and so the
values would be correctly written to the rowset buffer until the last value.
The last value would result in the high order byte being written past the end
of the buffer.
Sometimes this proved innocuous (the high order byte was often zero, for
example, and the byte it was overwriting happened to be zero much of the time).
However, with row counts of a multiple of 8, the overrun would typically
clobber a memory pointer leading to a core.
The fix is to change UPDATE STATS to use SMALLINT outputs for these data types
instead.
Another issue encountered while debugging this code on a later version of
Trafodion was that when casting an INTERVAL YEAR(1) TO MONTH value down to a
TINYINT, we would core in the compiler, in Cast::synthesizeType. The problem
here was that a new routine, ExpGenerator::handleUnsupportedCast
(generator/GenExpGenerator.cpp), was assuming that unsupported casts for code
generation involving 8-byte integers always involved another numeric type. The
fix is to take care of the case of interval types as well. Thanks to Anoop
Sharma for diagnosing and providing the fix to this latter problem.
was (Author: davebirdsall):
Further debugging and testing revealed that in the first case UPDATE STATS was
using a SELECT and casting the interval column to a TINYINT.
In both cases, UPDATE STATS was using a rowset to retrieve the data values. It
assumed the values would be 1 byte each. However, the compiler casts a TINYINT
output to a 2-byte integer. The row set fetches would write two bytes per value
to the rowset buffer. The high byte of one value would be overwritten by the
low byte of the next (since this is was on a little-endian machine), and so the
values would be correctly written to the rowset buffer until the last value.
The last value would result in the high order byte being written past the end
of the buffer.
Sometimes this proved innocuous (the high order byte was often zero, for
example, and the byte it was overwriting happened to be zero much of the time).
However, with row counts of a multiple of 8, the overrun would typically
clobber a memory pointer leading to a core.
The fix is to change UPDATE STATS to use SMALLINT outputs for these data types
instead.
Another issue encountered while debugging this code on a later version of
Trafodion was that when casting an INTERVAL YEAR(1) TO MONTH value down to a
TINYINT, we would core in the compiler, in Cast::synthesizeType. The problem
here was that a new routine, ExpGenerator::handleUnsupportedCast
(generator/GenExpGenerator.cpp), was assuming that unsupported casts for code
generation involving 8-byte integers always involved another numeric type. The
fix is to take care of the case of interval types as well.
> UPDATE STATS cores, buffer overrun, on TINYINT and small INTERVAL cols
> ----------------------------------------------------------------------
>
> Key: TRAFODION-2131
> URL: https://issues.apache.org/jira/browse/TRAFODION-2131
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.1-incubating
> Environment: All
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
