[
https://issues.apache.org/jira/browse/IGNITE-26491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-26491:
--------------------------------------
Description:
The behavior of the embedded client is inconsistent with the thin client when
trying to read a narrower integer type from tuple that contains long value.
For example, when we have table with bigint column val.
{code:sql}
create table test_int (id bigint primary key, val bigint not null)
{code}
Getting existing row from view
{code:java}
tuple = kvVIew.get(null, key);
{code}
the next
{code:java}
tuple.intValue("VAL")
{code}
in thin client produces
{noformat}
ClassCastException: Column with name 'VAL' has type INT64 but INT32 was
requested
{noformat}
but does not produces any errors in embedded client.
(need to check the same case with record view also)
Full reproducer
{code:java}
sql("create table test_int (id bigint primary key, val bigint)");
KeyValueView<Tuple, Tuple> kvVIew =
client.tables().table("test_int").keyValueView();
KeyValueView<Tuple, Tuple> kvVIewEmbedded =
CLUSTER.aliveNode().tables().table("test_int").keyValueView();
Tuple key = Tuple.create().set("id", 1L);
Tuple val = Tuple.create().set("val", 1L);
kvVIew.put(null, key, val);
Tuple thinReturnedVal = kvVIew.get(null, key);
Tuple embeddedReturnedVal = kvVIewEmbedded.get(null, key);
assertEquals(1, embeddedReturnedVal.longValue(0));
assertEquals(1, thinReturnedVal.longValue(0));
// The following statement should fail, but does not fail.
assertEquals(1, embeddedReturnedVal.intValue("VAL"));
// The following statement fails with ClassCastException.
assertEquals(1, thinReturnedVal.intValue("VAL"));
{code}
Exception stacktrace:
{noformat}
java.lang.ClassCastException: Column with name 'VAL' has type INT64 but INT32
was requested
at
org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.binaryTupleIndex(MutableTupleBinaryTupleAdapter.java:468)
at
org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.validateSchemaColumnType(MutableTupleBinaryTupleAdapter.java:477)
at
org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.intValue(MutableTupleBinaryTupleAdapter.java:202)
{noformat}
It is necessary to determine which behavior is correct and eliminate
inconsistencies.
h2. UPDATE
We want to make the behavior consistent in all clients:
# align the behavior with the [column upgrade
rules|https://docs.google.com/document/d/17dx7hKmKLMGQ460O3i4k5ZNXO9ylW1pHlCyD8xg23L0/edit?tab=t.0]
# relax current validation to support downcast where possible because we don't
want to break column upgrade scenarios
For example the following scenario should work:
* The user upgraded the column in the database from int to long, the user
application continues to treat the column as an int and this shouldn't break
application during reading or writing.
* The user updates the application
h3. Rules
For the following type families:
# Integer types: {*}INT8{*}{*},{*} {*}INT16{*}, {*}INT32{*}, *INT64*
# Floating‑point types: {*}FLOAT{*}, *DOUBLE*
Within each family the following implicit conversion rules apply:
# *Implicit widening conversions* are always permitted.
# *Implicit narrowing conversions* are permitted only if the source value can
be exactly represented in the target type without loss of precision or overflow.
These rules essentially reflect how we write numeric types in binary format.
h3. Example
{code:java}
Column VAL has type INT16.
Implicit conversion when reading:
// allowed for all non null values
// for null values throws NullPointerException
long val = tuple.longValue(“val”);
int val = tuple.intValue(“val”);
short val = tuple.shortValue(“val”);
// allowed only if the value is in range [-127, 127],
// if the value is null throws NullPointerException
// otherwise throws ClassCastException
byte val = tuple.byteValue(“val"); {code}
h3. Implementation notes
We have several Tuple implementations. The behavior should be the same across
all implementations.
The following table shows the base implementation classes that must be updated
to support implicit conversion and consistent type checking.
|Class|Mutable|Usage|Implementation details|
|TupleImpl|yes| |Map|
|MutableTupleBinaryTupleAdapter
|yes|java thin
child classes:
* ClientSqlRow
* ClientHandlerTuple
* ClientTuple|BinaryTupleReader for reading
TupleImpl for modifications|
|MutableRowTupleAdapter|yes|Embedded client
binary table views.
child classes:
* TableRow
* KeyRowChunk
* ValueRowChunk|BinaryRow for reading
TupleImpl for modifications|
|SqlRowImpl|no|Embedded
SQL resultset| |
This class is not aware about schema and is currently used as the main
implementation of Tuple in the public API, and is also used in the internal API
when modifying an existing Tuple (for example, in MutableRowTupleAdapter, etc.)
Currently it does not support implicit type conversion and throws a
ClassCastException if the value type does not match the required one. The
behavior of this class must conform to the specified implicit type conversion
rules. We must keep this class {*}schema unaware{*}.
The “set” method will remain unchanged and is expected not to generate any
exceptions.
h3. SqlRowImpl
This class always reads the boxed value according to the schema, and then casts
it to the desired type,
Similar to TupleImpl it throws ClassCastException if the value type doesn’t
match the required type. And similar to TupleImpl we must support implicit
conversion for numeric types.
Below is a pseudo code that can be used for implement implicit conversion when
reading short and float values (in TupleImpl and SqlRowImpl):
{code:java}
public short shortValue(String columnName) {
Number number = (Number) row.get(columnIndexChecked(columnName));
if (obj instanceof Long
|| obj instanceof Integer
|| obj instanceof Byte) {
long longVal = number.longValue();
short shortVal = number.shortValue();
if (longVal == shortVal)
{ return shortVal; }
throw new ArithmeticException("short value overflow");
}
return (short) number;
}
public float floatValue(String columnName) {
Object obj = row.get(columnIndexChecked(columnName));
if (obj instanceof Double) {
Double doubleValue = (Double) obj;
float floatValue = doubleValue.floatValue();
if (doubleValue == floatValue) {
return floatValue;
}
throw new ArithmeticException("float value overflow");
}
return (float) obj;
} {code}
MutableTupleBinaryTupleAdapter
This implementation is used in the thin client and, unlike SqlRow, can read
primitives without value boxing.
This class performs strict schema validation before reading value from the
binary tuple.
Validation needs to be relaxed and behavior needs to be aligned with the
described type conversion rules. At the same time, the ability to read
primitives without value boxing must be preserved.
{code:java}
public float floatValue(String columnName) {
if (tuple != null) {
return tuple.floatValue(columnName);
}
var binaryTupleIndex = binaryTupleIndex(columnName);
// validate index
// ...
ColumnType actualType = schemaColumnType(binaryTupleIndex);
if (actualType == ColumnType.FLOAT) {
return binaryTuple.floatValue(binaryTupleIndex);
}
if (actualType == ColumnType.DOUBLE) {
double doubleValue = binaryTuple.doubleValue(binaryTupleIndex);
float floatValue = (float) doubleValue;
if (doubleValue == floatValue) {
return floatValue;
}
}
throw new ClassCastException("...");
} {code}
Note: validation in the serializer must also be relaxed to comply with the
specified rules (see ClientTupleSerializer.writeTupleRow).
h3. MutableRowTupleAdapter
Similar to MutableTupleBinaryTupleAdapter this class is aware about schema and
able to read values without unboxing, but doesn’t perform any validation
currently.
We need to align its behavior according to the described rules, similar to
MutableTupleBinaryTupleAdapter.
Note: validation at the marshaller-level must also be relaxed to comply with
the specified rules (see Column.validate).
was:
The behavior of the embedded client is inconsistent with the thin client when
trying to read a narrower integer type from tuple that contains long value.
For example, when we have table with bigint column val.
{code:sql}
create table test_int (id bigint primary key, val bigint not null)
{code}
Getting existing row from view
{code:java}
tuple = kvVIew.get(null, key);
{code}
the next
{code:java}
tuple.intValue("VAL")
{code}
in thin client produces
{noformat}
ClassCastException: Column with name 'VAL' has type INT64 but INT32 was
requested
{noformat}
but does not produces any errors in embedded client.
(need to check the same case with record view also)
Full reproducer
{code:java}
sql("create table test_int (id bigint primary key, val bigint)");
KeyValueView<Tuple, Tuple> kvVIew =
client.tables().table("test_int").keyValueView();
KeyValueView<Tuple, Tuple> kvVIewEmbedded =
CLUSTER.aliveNode().tables().table("test_int").keyValueView();
Tuple key = Tuple.create().set("id", 1L);
Tuple val = Tuple.create().set("val", 1L);
kvVIew.put(null, key, val);
Tuple thinReturnedVal = kvVIew.get(null, key);
Tuple embeddedReturnedVal = kvVIewEmbedded.get(null, key);
assertEquals(1, embeddedReturnedVal.longValue(0));
assertEquals(1, thinReturnedVal.longValue(0));
// The following statement should fail, but does not fail.
assertEquals(1, embeddedReturnedVal.intValue("VAL"));
// The following statement fails with ClassCastException.
assertEquals(1, thinReturnedVal.intValue("VAL"));
{code}
Exception stacktrace:
{noformat}
java.lang.ClassCastException: Column with name 'VAL' has type INT64 but INT32
was requested
at
org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.binaryTupleIndex(MutableTupleBinaryTupleAdapter.java:468)
at
org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.validateSchemaColumnType(MutableTupleBinaryTupleAdapter.java:477)
at
org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.intValue(MutableTupleBinaryTupleAdapter.java:202)
{noformat}
It is necessary to determine which behavior is correct and eliminate
inconsistencies.
h2. UPDATE
We want to make the behavior consistent in all clients:
# align the behavior with the [column upgrade
rules|https://docs.google.com/document/d/17dx7hKmKLMGQ460O3i4k5ZNXO9ylW1pHlCyD8xg23L0/edit?tab=t.0]
# relax current validation to support downcast where possible because we don't
want to break column upgrade scenarios
For example the following scenario should work:
* The user upgraded the column in the database from int to long, the user
application continues to treat the column as an int and this shouldn't break
application during reading or writing.
* The user updates the application
h3. Rules
For the following type families:
# Integer types: {*}INT8{*}{*},{*} {*}INT16{*}, {*}INT32{*}, *INT64*
# Floating‑point types: {*}FLOAT{*}, *DOUBLE*
Within each family the following implicit conversion rules apply:
# *Implicit widening conversions* are always permitted.
# *Implicit narrowing conversions* are permitted only if the source value can
be exactly represented in the target type without loss of precision or overflow.
These rules essentially reflect how we write numeric types in binary format.
h3. Example
{code:java}
Column VAL has type INT16.
Implicit conversion when reading:
// allowed for all non null values
// for null values throws NullPointerException
long val = tuple.longValue(“val”);
int val = tuple.intValue(“val”);
short val = tuple.shortValue(“val”);
// allowed only if the value is in range [-127, 127],
// if the value is null throws NullPointerException
// otherwise throws ClassCastException
byte val = tuple.byteValue(“val"); {code}
h3. Implementation notes
We have several Tuple implementations. The behavior should be the same across
all implementations.
The following table shows the base implementation classes that must be updated
to support implicit conversion and consistent type checking.
|Class|Mutable|Usage|Implementation details|Usage|Implementation details|
| | | | | | |
| | | | | | |
| | | | | | |
|Col A1| | |Col A2| | |
This class is not aware about schema and is currently used as the main
implementation of Tuple in the public API, and is also used in the internal API
when modifying an existing Tuple (for example, in MutableRowTupleAdapter, etc.)
Currently it does not support implicit type conversion and throws a
ClassCastException if the value type does not match the required one. The
behavior of this class must conform to the specified implicit type conversion
rules. We must keep this class {*}schema unaware{*}.
The “set” method will remain unchanged and is expected not to generate any
exceptions.
h3. SqlRowImpl
This class always reads the boxed value according to the schema, and then casts
it to the desired type,
Similar to TupleImpl it throws ClassCastException if the value type doesn’t
match the required type. And similar to TupleImpl we must support implicit
conversion for numeric types.
Below is a pseudo code that can be used for implement implicit conversion when
reading short and float values (in TupleImpl and SqlRowImpl):
public short shortValue(String columnName) {
Number number = (Number) row.get(columnIndexChecked(columnName));
if (obj instanceof Long
|| obj instanceof Integer
|| obj instanceof Byte) {
long longVal = number.longValue();
short shortVal = number.shortValue();
if (longVal == shortVal)
{ return shortVal; }
throw new ArithmeticException("short value overflow");
}
return (short) number;
}
public float floatValue(String columnName) {
Object obj = row.get(columnIndexChecked(columnName));
if (obj instanceof Double) {
Double doubleValue = (Double) obj;
float floatValue = doubleValue.floatValue();
if (doubleValue == floatValue)
{ return floatValue; }
throw new ArithmeticException("float value overflow");
}
return (float) obj;
}
h3. MutableTupleBinaryTupleAdapter
This implementation is used in the thin client and, unlike SqlRow, can read
primitives without value boxing.
This class performs strict schema validation before reading value from the
binary tuple.
Validation needs to be relaxed and behavior needs to be aligned with the
described type conversion rules. At the same time, the ability to read
primitives without value boxing must be preserved.
public float floatValue(String columnName) {
if (tuple != null) \{ return tuple.floatValue(columnName); }
var binaryTupleIndex = binaryTupleIndex(columnName);
// validate index
// ...
ColumnType actualType = schemaColumnType(binaryTupleIndex);
if (actualType == ColumnType.FLOAT) \{ return
binaryTuple.floatValue(binaryTupleIndex); }
if (actualType == ColumnType.DOUBLE) {
double doubleValue = binaryTuple.doubleValue(binaryTupleIndex);
float floatValue = (float) doubleValue;
if (doubleValue == floatValue) \{ return floatValue;
}
}
throw new ClassCastException("...");
}
Note: validation in the serializer must also be relaxed to comply with the
specified rules (see ClientTupleSerializer.writeTupleRow).
h3. MutableRowTupleAdapter
Similar to MutableTupleBinaryTupleAdapter this class is aware about schema and
able to read values without unboxing, but doesn’t perform any validation
currently.
We need to align its behavior according to the described rules, similar to
MutableTupleBinaryTupleAdapter.
Note: validation at the marshaller-level must also be relaxed to comply with
the specified rules (see Column.validate).
> Inconsistent behavior of thin/embedded client when reading tuple value
> ----------------------------------------------------------------------
>
> Key: IGNITE-26491
> URL: https://issues.apache.org/jira/browse/IGNITE-26491
> Project: Ignite
> Issue Type: Bug
> Components: sql ai3
> Reporter: Pavel Pereslegin
> Priority: Major
> Labels: ignite-3
>
>
> The behavior of the embedded client is inconsistent with the thin client when
> trying to read a narrower integer type from tuple that contains long value.
> For example, when we have table with bigint column val.
> {code:sql}
> create table test_int (id bigint primary key, val bigint not null)
> {code}
> Getting existing row from view
> {code:java}
> tuple = kvVIew.get(null, key);
> {code}
> the next
> {code:java}
> tuple.intValue("VAL")
> {code}
> in thin client produces
> {noformat}
> ClassCastException: Column with name 'VAL' has type INT64 but INT32 was
> requested
> {noformat}
> but does not produces any errors in embedded client.
> (need to check the same case with record view also)
> Full reproducer
> {code:java}
> sql("create table test_int (id bigint primary key, val bigint)");
> KeyValueView<Tuple, Tuple> kvVIew =
> client.tables().table("test_int").keyValueView();
> KeyValueView<Tuple, Tuple> kvVIewEmbedded =
> CLUSTER.aliveNode().tables().table("test_int").keyValueView();
> Tuple key = Tuple.create().set("id", 1L);
> Tuple val = Tuple.create().set("val", 1L);
> kvVIew.put(null, key, val);
> Tuple thinReturnedVal = kvVIew.get(null, key);
> Tuple embeddedReturnedVal = kvVIewEmbedded.get(null, key);
> assertEquals(1, embeddedReturnedVal.longValue(0));
> assertEquals(1, thinReturnedVal.longValue(0));
> // The following statement should fail, but does not fail.
> assertEquals(1, embeddedReturnedVal.intValue("VAL"));
> // The following statement fails with ClassCastException.
> assertEquals(1, thinReturnedVal.intValue("VAL"));
> {code}
> Exception stacktrace:
> {noformat}
> java.lang.ClassCastException: Column with name 'VAL' has type INT64 but INT32
> was requested
> at
> org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.binaryTupleIndex(MutableTupleBinaryTupleAdapter.java:468)
> at
> org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.validateSchemaColumnType(MutableTupleBinaryTupleAdapter.java:477)
> at
> org.apache.ignite.internal.client.table.MutableTupleBinaryTupleAdapter.intValue(MutableTupleBinaryTupleAdapter.java:202)
> {noformat}
> It is necessary to determine which behavior is correct and eliminate
> inconsistencies.
> h2. UPDATE
> We want to make the behavior consistent in all clients:
> # align the behavior with the [column upgrade
> rules|https://docs.google.com/document/d/17dx7hKmKLMGQ460O3i4k5ZNXO9ylW1pHlCyD8xg23L0/edit?tab=t.0]
> # relax current validation to support downcast where possible because we
> don't want to break column upgrade scenarios
> For example the following scenario should work:
> * The user upgraded the column in the database from int to long, the user
> application continues to treat the column as an int and this shouldn't break
> application during reading or writing.
> * The user updates the application
> h3. Rules
> For the following type families:
> # Integer types: {*}INT8{*}{*},{*} {*}INT16{*}, {*}INT32{*}, *INT64*
> # Floating‑point types: {*}FLOAT{*}, *DOUBLE*
> Within each family the following implicit conversion rules apply:
> # *Implicit widening conversions* are always permitted.
> # *Implicit narrowing conversions* are permitted only if the source value
> can be exactly represented in the target type without loss of precision or
> overflow.
>
> These rules essentially reflect how we write numeric types in binary format.
> h3. Example
> {code:java}
> Column VAL has type INT16.
>
> Implicit conversion when reading:
>
> // allowed for all non null values
> // for null values throws NullPointerException
> long val = tuple.longValue(“val”);
> int val = tuple.intValue(“val”);
> short val = tuple.shortValue(“val”);
>
> // allowed only if the value is in range [-127, 127],
> // if the value is null throws NullPointerException
> // otherwise throws ClassCastException
> byte val = tuple.byteValue(“val"); {code}
>
> h3. Implementation notes
> We have several Tuple implementations. The behavior should be the same across
> all implementations.
> The following table shows the base implementation classes that must be
> updated to support implicit conversion and consistent type checking.
>
> |Class|Mutable|Usage|Implementation details|
> |TupleImpl|yes| |Map|
> |MutableTupleBinaryTupleAdapter
> |yes|java thin
> child classes:
> * ClientSqlRow
> * ClientHandlerTuple
> * ClientTuple|BinaryTupleReader for reading
> TupleImpl for modifications|
> |MutableRowTupleAdapter|yes|Embedded client
> binary table views.
> child classes:
> * TableRow
> * KeyRowChunk
> * ValueRowChunk|BinaryRow for reading
> TupleImpl for modifications|
> |SqlRowImpl|no|Embedded
> SQL resultset| |
>
> This class is not aware about schema and is currently used as the main
> implementation of Tuple in the public API, and is also used in the internal
> API when modifying an existing Tuple (for example, in MutableRowTupleAdapter,
> etc.)
> Currently it does not support implicit type conversion and throws a
> ClassCastException if the value type does not match the required one. The
> behavior of this class must conform to the specified implicit type conversion
> rules. We must keep this class {*}schema unaware{*}.
> The “set” method will remain unchanged and is expected not to generate any
> exceptions.
> h3. SqlRowImpl
> This class always reads the boxed value according to the schema, and then
> casts it to the desired type,
> Similar to TupleImpl it throws ClassCastException if the value type doesn’t
> match the required type. And similar to TupleImpl we must support implicit
> conversion for numeric types.
>
> Below is a pseudo code that can be used for implement implicit conversion
> when reading short and float values (in TupleImpl and SqlRowImpl):
>
> {code:java}
> public short shortValue(String columnName) {
> Number number = (Number) row.get(columnIndexChecked(columnName));
>
> if (obj instanceof Long
> || obj instanceof Integer
> || obj instanceof Byte) {
> long longVal = number.longValue();
> short shortVal = number.shortValue();
>
> if (longVal == shortVal)
> { return shortVal; }
>
> throw new ArithmeticException("short value overflow");
> }
>
> return (short) number;
> }
>
> public float floatValue(String columnName) {
> Object obj = row.get(columnIndexChecked(columnName));
>
> if (obj instanceof Double) {
> Double doubleValue = (Double) obj;
> float floatValue = doubleValue.floatValue();
>
> if (doubleValue == floatValue) {
> return floatValue;
> }
>
> throw new ArithmeticException("float value overflow");
> }
>
> return (float) obj;
> } {code}
>
> MutableTupleBinaryTupleAdapter
> This implementation is used in the thin client and, unlike SqlRow, can read
> primitives without value boxing.
> This class performs strict schema validation before reading value from the
> binary tuple.
> Validation needs to be relaxed and behavior needs to be aligned with the
> described type conversion rules. At the same time, the ability to read
> primitives without value boxing must be preserved.
>
> {code:java}
>
> public float floatValue(String columnName) {
> if (tuple != null) {
> return tuple.floatValue(columnName);
> }
>
> var binaryTupleIndex = binaryTupleIndex(columnName);
>
> // validate index
> // ...
>
> ColumnType actualType = schemaColumnType(binaryTupleIndex);
>
> if (actualType == ColumnType.FLOAT) {
> return binaryTuple.floatValue(binaryTupleIndex);
> }
>
> if (actualType == ColumnType.DOUBLE) {
> double doubleValue = binaryTuple.doubleValue(binaryTupleIndex);
> float floatValue = (float) doubleValue;
> if (doubleValue == floatValue) {
> return floatValue;
> }
> }
>
> throw new ClassCastException("...");
> } {code}
> Note: validation in the serializer must also be relaxed to comply with the
> specified rules (see ClientTupleSerializer.writeTupleRow).
>
> h3. MutableRowTupleAdapter
> Similar to MutableTupleBinaryTupleAdapter this class is aware about schema
> and able to read values without unboxing, but doesn’t perform any validation
> currently.
> We need to align its behavior according to the described rules, similar to
> MutableTupleBinaryTupleAdapter.
>
> Note: validation at the marshaller-level must also be relaxed to comply with
> the specified rules (see Column.validate).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
