Gabriel39 commented on code in PR #67784:
URL: https://github.com/apache/doris/pull/67784#discussion_r4056549743
##########
be/src/util/raw_value.h:
##########
@@ -55,6 +55,8 @@ inline uint32_t RawValue::zlib_crc32(const void* v, size_t
len, const PrimitiveT
case TYPE_VARCHAR:
case TYPE_HLL:
case TYPE_STRING:
+ // Tablet routing and FE bucket pruning must hash the same unmodified
binary bytes.
Review Comment:
Addressed in 19bb5f231e.
Removed binary tablet-routing hash computation. RawValue::zlib_crc32 now
explicitly rejects VARBINARY, including NULL input, instead of reaching the
default-type assertion. ColumnVarbinary also inherits the unsupported
column-hash implementations; tests cover every removed hash method.
##########
be/src/format_v2/table/paimon_reader.cpp:
##########
@@ -179,6 +180,47 @@ Status annotate_paimon_variants(const
std::vector<ColumnDefinition>& table_schem
return Status::OK();
}
+DataTypePtr nullable_like_original(const DataTypePtr& original, DataTypePtr
nested) {
+ return original != nullptr && original->is_nullable() ?
make_nullable(nested) : nested;
+}
+
+DataTypePtr apply_paimon_timestamp_semantics(format::ColumnDefinition* column)
{
Review Comment:
Addressed in 19bb5f231e.
Documented how unannotated INT96 relies on Paimon table semantics, why the
timestamp leaf and all parent container types must be rebuilt together, and how
precision and nullability are preserved. Added nested ARRAY/MAP timestamp
coverage.
##########
be/src/format_v2/table/paimon_reader.cpp:
##########
@@ -179,6 +180,47 @@ Status annotate_paimon_variants(const
std::vector<ColumnDefinition>& table_schem
return Status::OK();
}
+DataTypePtr nullable_like_original(const DataTypePtr& original, DataTypePtr
nested) {
+ return original != nullptr && original->is_nullable() ?
make_nullable(nested) : nested;
+}
+
+DataTypePtr apply_paimon_timestamp_semantics(format::ColumnDefinition* column)
{
+ DORIS_CHECK(column != nullptr);
+ DORIS_CHECK(column->type != nullptr);
+ const auto primitive = remove_nullable(column->type)->get_primitive_type();
+ if (column->timestamp_is_adjusted_to_utc.has_value() &&
+ (primitive == TYPE_DATETIMEV2 || primitive == TYPE_TIMESTAMPTZ)) {
+ const auto target =
+ *column->timestamp_is_adjusted_to_utc ? TYPE_TIMESTAMPTZ :
TYPE_DATETIMEV2;
+ column->type = DataTypeFactory::instance().create_data_type(
+ target, column->type->is_nullable(), 0,
column->type->get_scale());
+ return column->type;
+ }
+
+ std::vector<DataTypePtr> child_types;
+ child_types.reserve(column->children.size());
+ for (auto& child : column->children) {
+ child_types.push_back(apply_paimon_timestamp_semantics(&child));
+ }
+ if (primitive == TYPE_ARRAY && child_types.size() == 1) {
Review Comment:
Addressed in 19bb5f231e.
Changed timestamp schema application to return Status. ARRAY/MAP child
counts are checked before children are modified, and malformed structures now
return an explicit error instead of silently retaining a stale parent type.
Added malformed arity cases and valid nested-type coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
