wgtmac commented on code in PR #34193:
URL: https://github.com/apache/arrow/pull/34193#discussion_r1113769699
##########
cpp/src/parquet/column_writer.cc:
##########
@@ -1014,11 +1014,58 @@ template <typename Action>
inline void DoInBatches(int64_t total, int64_t batch_size, Action&& action) {
int64_t num_batches = static_cast<int>(total / batch_size);
for (int round = 0; round < num_batches; round++) {
- action(round * batch_size, batch_size);
+ action(round * batch_size, batch_size, /*check_page=*/true);
}
// Write the remaining values
if (total % batch_size > 0) {
- action(num_batches * batch_size, total % batch_size);
+ action(num_batches * batch_size, total % batch_size, /*check_page=*/true);
+ }
+}
+
+template <typename Action>
+inline void DoInBatches(const int16_t* def_levels, const int16_t* rep_levels,
+ int64_t num_levels, int64_t batch_size, Action&&
action,
Review Comment:
Yes, `num_levels` is the length of `def_levels` and `rep_levels`. It is used
by various functions in this file so it would be better to be consistent.
```cpp
Status WriteArrowDictionary(const int16_t* def_levels, const int16_t*
rep_levels,
int64_t num_levels, const ::arrow::Array&
array,
ArrowWriteContext* context, bool
maybe_parent_nulls);
Status WriteArrowDense(const int16_t* def_levels, const int16_t*
rep_levels,
int64_t num_levels, const ::arrow::Array& array,
ArrowWriteContext* context, bool
maybe_parent_nulls);
```
IMHO, "depth" is the value at each slot of `def_levels` and `rep_levels`. So
the name `num_levels` makes sense to me. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
