el-hult commented on code in PR #55351:
URL: https://github.com/apache/arrow/pull/55351#discussion_r4437421813
##########
cpp/src/parquet/arrow/arrow_reader_writer_test.cc:
##########
@@ -3453,6 +3453,22 @@ TEST(ArrowReadWrite, ListWithNoValues) {
CheckSimpleRoundtrip(table, 2, props_store_schema);
}
+TEST(ArrowReadWrite, NestedDictionaryMultipleRowGroups) {
+ // GH-45087: reading several row groups at once yields one chunk per row
group
+ // for a nested dictionary leaf, and the chunks have different dictionaries.
+ using ::arrow::field;
+
+ auto dict_type = ::arrow::dictionary(::arrow::int32(), ::arrow::utf8());
+ for (const auto& type : {list(dict_type), ::arrow::large_list(dict_type)}) {
+ ARROW_SCOPED_TRACE("type = ", type->ToString());
+ auto array =
+ ::arrow::ArrayFromJSON(type, R"([["a"], ["b", "a"], ["c", "a"],
["b"]])");
+ auto table = ::arrow::Table::Make(::arrow::schema({field("root", type)}),
{array});
Review Comment:
Thanks, I want to make sure I understand. I’d add a separate test that:
1. builds an input ChunkedArray of list<dictionary<utf8>> with two chunks,
each with its own dictionary ([a, b] and [c, a, b]);
2. writes it with a row group size equal to the chunk length, so each chunk
becomes one row group;
3. reads it back with ReadTable();
4. compares against an expected table built separately, from the same rows,
with a single dictionary.
My concern is that comparing dictionary-typed arrays also compares the
dictionary order, so the test would then depend on how Concatenate orders the
merged dictionary. That is what the fix delegates to, but it isn’t what it is
meant to guarantee. Would it be fine to cast the result to list<utf8> and
compare the decoded values against an expected list<utf8>, and keep a schema
check so the result is still the dictionary type?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
