github-actions[bot] commented on code in PR #69032:
URL: https://github.com/apache/doris/pull/69032#discussion_r4306515716
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Ndv.java:
##########
@@ -63,6 +63,10 @@ private Ndv(AggregateFunctionParams functionParams) {
@Override
public void checkLegalityBeforeTypeCoercion() {
+ // The BE kernel has no spatial value-state dispatch; fail before
casts hide the input type.
+ if (getArgumentType(0).isSpatialType()) {
+ throw new AnalysisException("ndv does not support spatial
arguments");
Review Comment:
[P2] Exclude spatial columns from statistics collection
For an Iceberg table containing GEOMETRY or GEOGRAPHY, `ANALYZE TABLE t WITH
FULL WITH SYNC` still selects the spatial columns because
`StatisticsUtil.isUnsupportedType` and `ColumnStatistic.UNSUPPORTED_TYPE` omit
them. `PluginDrivenExternalTable` creates an `ExternalAnalysisTask`, and both
full-analysis templates unconditionally emit `NDV(shape)`, which now fails here
and makes the ANALYZE job fail. Enabled automatic analysis can schedule the
same unsupported tasks. Keep this explicit user-expression rejection and
exclude both spatial primitive kinds from statistics eligibility, covering
every CRS/algorithm, with a mixed ordinary/spatial ANALYZE regression.
##########
be/src/format/transformer/viceberg_parquet_writer.cpp:
##########
@@ -65,6 +193,28 @@ VIcebergParquetWriter::VIcebergParquetWriter(RuntimeState*
state, io::FileWriter
_iceberg_schema_json(iceberg_schema_json == nullptr ? "" :
*iceberg_schema_json),
_nan_count_field_ids(nan_count_field_ids) {}
+arrow::Status VIcebergParquetWriter::_open_file_writer() {
+ bool has_spatial_column = false;
+ for (const auto& column : _iceberg_schema.columns()) {
+ has_spatial_column = has_spatial_column ||
contains_spatial_type(column.field_type());
+ }
+ if (!has_spatial_column) {
+ return VParquetWriter::_open_file_writer();
+ }
+
+ ARROW_ASSIGN_OR_RAISE(
+ auto parquet_schema,
+ make_iceberg_parquet_schema(_arrow_convertor()->arrow_schema(),
_iceberg_schema,
Review Comment:
[P2] Finish failed writer cleanup before deleting the S3 object
A dynamic-partition INSERT SELECT into a matching `projjson:` spatial target
fails here after `VIcebergPartitionWriter::open` has created its S3 writer and
`ParquetOutputStream`. The open-error path deletes the object while the stream
remains open, and `create_and_open_writer` returns before registering the
partition writer. Destruction then closes the stream;
`S3FileWriter::_close_impl` uploads an empty object when `bytes_appended == 0`,
recreating it after deletion. The failed writer is in neither the active map
nor the closed-file cleanup list, so rejected attempts leave physical S3
orphans. Validate the schema before object creation, or close/abort the writer
before the final deletion, and cover dynamic partitions using physical object
listings; the existing unpartitioned `$files` check only sees committed files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
