curioustien commented on code in PR #45351:
URL: https://github.com/apache/arrow/pull/45351#discussion_r2019993180
##########
cpp/src/parquet/arrow/schema_internal.h:
##########
@@ -29,12 +29,16 @@ namespace parquet::arrow {
using ::arrow::Result;
-Result<std::shared_ptr<::arrow::DataType>> FromByteArray(const LogicalType&
logical_type,
- bool
use_known_arrow_extensions);
-Result<std::shared_ptr<::arrow::DataType>> FromFLBA(const LogicalType&
logical_type,
- int32_t physical_length);
-Result<std::shared_ptr<::arrow::DataType>> FromInt32(const LogicalType&
logical_type);
-Result<std::shared_ptr<::arrow::DataType>> FromInt64(const LogicalType&
logical_type);
+Result<std::shared_ptr<::arrow::DataType>> FromByteArray(
+ const LogicalType& logical_type, bool arrow_extensions_enabled = false,
+ bool smallest_decimal_enabled = false);
Review Comment:
I can remove them for `FromByteArray` and `FromFLBA` to use
`ArrowReaderProperties` instead. However, for `FromInt32` and `FromInt64`,
they're also used
[FromInt32Statistics](https://github.com/apache/arrow/blob/main/cpp/src/parquet/arrow/reader_internal.cc#L156)
and
[FromInt64Statistics](https://github.com/apache/arrow/blob/main/cpp/src/parquet/arrow/reader_internal.cc#L178)
from
[StatisticsAsScalars](https://github.com/apache/arrow/blob/main/cpp/src/parquet/arrow/reader_internal.cc#L277-L316).
It doesn't look like I can stream the `ArrowReaderProperties` from there, so I
have to use default parameters
##########
cpp/src/parquet/arrow/test_util.h:
##########
@@ -47,24 +47,58 @@ using ::arrow::Array;
using ::arrow::ChunkedArray;
using ::arrow::Status;
-template <int32_t PRECISION>
-struct Decimal128WithPrecisionAndScale {
- static_assert(PRECISION >= 1 && PRECISION <= 38, "Invalid precision value");
+struct BaseDecimalWithPrecisionAndScale {};
+
+template <int32_t PRECISION, bool SMALLEST_DECIMAL_ENABLED = false>
+struct Decimal32WithPrecisionAndScale : BaseDecimalWithPrecisionAndScale {
+ static_assert(PRECISION >= ::arrow::Decimal32Type::kMinPrecision &&
+ PRECISION <= ::arrow::Decimal32Type::kMaxPrecision,
+ "Invalid precision value");
+
+ using type = ::arrow::Decimal32Type;
+ static constexpr ::arrow::Type::type type_id =
::arrow::Decimal32Type::type_id;
+ static constexpr int32_t precision = PRECISION;
+ static constexpr int32_t scale = PRECISION - 1;
+ static constexpr bool smallest_decimal_enabled = SMALLEST_DECIMAL_ENABLED;
Review Comment:
This is mainly used for the tests in `arrow_reader_writer_test.cc` for
`TestParquetIO.TestTypes` from line 984 to 997 where I used this variable to
trigger whether the test type requires the reader to use the smallest decimal
or not
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
