zhouyuan commented on code in PR #12915:
URL: https://github.com/apache/gluten/pull/12915#discussion_r3958475849
##########
cpp/velox/utils/ConfigExtractor.cc:
##########
@@ -273,6 +273,8 @@ std::shared_ptr<facebook::velox::config::ConfigBase>
createHiveConnectorSessionC
conf->get<bool>(kAllowInt32Narrowing, true) ? "true" : "false";
configs[parquetSessionProperty(facebook::velox::parquet::ParquetConfig::kWriterPageSizeSession)]
=
conf->get<std::string>(kWriteParquetPageSizeBytes, "1MB");
+
configs[parquetSessionProperty(facebook::velox::parquet::ParquetConfig::kWriterRowGroupSizeSession)]
=
+ conf->get<std::string>(kColumnarParquetWriteBlockSize, "128MB");
Review Comment:
velox expect this to be a string like "32MB", however in Spark/Gluten it
will expanded into digits only when passing to native, this will be a bug?
https://github.com/apache/gluten/blob/a44947e7ecc4732bc7c82a723c73717e57c38cbc/gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala#L1238
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
