tachyonwill commented on a change in pull request #11984:
URL: https://github.com/apache/arrow/pull/11984#discussion_r775573699
##########
File path: cpp/src/parquet/column_reader.cc
##########
@@ -940,7 +940,7 @@ int64_t
TypedColumnReaderImpl<DType>::ReadBatchWithDictionary(
int64_t* indices_read, const T** dict, int32_t* dict_len) {
bool has_dict_output = dict != nullptr && dict_len != nullptr;
// Similar logic as ReadValues to get pages.
- if (!HasNext()) {
+ if (batch_size == 0 || !HasNext()) {
Review comment:
We should probably support this. I think the more important question is
should we throw an exception for bad pages or just skip over the phantom
values. Parquet-mr seems to do the latter but does other validation on the
reported value counts. It would also make the batch size 0 case easier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
