github-actions[bot] commented on code in PR #64011: URL: https://github.com/apache/doris/pull/64011#discussion_r3366567017
########## regression-test/suites/datatype_p0/complex_types/test_struct_field_align.groovy: ########## @@ -0,0 +1,91 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +// Regression test for OPENSOURCE-374: when loading JSON into a STRUCT column, sub-fields must +// be matched by field name. Out-of-order JSON keys in Stream Load used to turn the whole struct +// column into NULL; a missing field is now filled with NULL and an unknown field is ignored +// (consistent with PostgreSQL / Spark / Trino). Struct-to-struct CAST stays positional, which +// also matches those engines. +suite("test_struct_field_align") { + def tableName = "test_struct_field_align" + + sql "DROP TABLE IF EXISTS ${tableName}" + sql """ + CREATE TABLE ${tableName} ( + id INT NOT NULL, + c_struct STRUCT<f1:INT,f2:FLOAT,f3:STRING> NULL + ) + UNIQUE KEY(id) + DISTRIBUTED BY HASH(id) BUCKETS 1 + PROPERTIES ("replication_allocation" = "tag.location.default: 1") + """ + + def doStreamLoad = { fileName -> + streamLoad { + table tableName + set 'format', 'json' + set 'strip_outer_array', 'true' + set 'columns', 'id, c_struct' + file fileName + time 10000 + check { result, exception, startTime, endTime -> + if (exception != null) { + throw exception + } + def json = parseJson(result) + assertEquals("success", json.Status.toLowerCase()) + assertEquals(0, json.NumberFilteredRows) + assertTrue(json.NumberLoadedRows > 0) + } + } + } + + // 1) ordered keys (baseline) + doStreamLoad "test_struct_field_align_ordered.json" Review Comment: This test loads JSON objects, but the code change only updates `DataTypeStructSerDe::_from_string`. For stream-load JSON, `NewJsonReader::_simdjson_write_data_to_column` still takes the `_is_load` branch, converts non-string values with `simdjson::to_json_string(value)`, and calls `data_serde->deserialize_one_cell_from_json(...)`. The struct implementation of `deserialize_one_cell_from_json` is still positional/order-sensitive (`elem_names[field_pos]`) and still rejects missing fields, so this swapped-key file and the omitted `f3` row will not be handled by the new `_from_string` logic. Please either update the JSON deserialize path to use the same by-name/missing-field semantics or route load-time struct JSON objects through the existing by-name struct object handling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
