zeroshade commented on code in PR #369:
URL: https://github.com/apache/iceberg-go/pull/369#discussion_r2023420124
##########
table/internal/parquet_files.go:
##########
@@ -170,6 +172,86 @@ func (parquetFormat) PrimitiveTypeToPhysicalType(typ
iceberg.PrimitiveType) stri
}
}
+func (parquetFormat) GetWriteProperties(props iceberg.Properties) any {
+ writerProps := []parquet.WriterProperty{
+ parquet.WithDictionaryDefault(false),
+
parquet.WithMaxRowGroupLength(int64(props.GetInt("write.parquet.row-group-limit",
1048576))),
+
parquet.WithDataPageSize(int64(props.GetInt("write.parquet.page-size-bytes",
1024*1024))),
+ parquet.WithDataPageVersion(parquet.DataPageV2),
+
parquet.WithBatchSize(int64(props.GetInt("write.parquet.page-row-limit",
20000))),
+
parquet.WithDictionaryPageSizeLimit(int64(props.GetInt("write.parquet.dict-size-bytes",
2*1024*1024))),
Review Comment:
it is, and I have constants in `table/properties.go`, the problem is that I
can't actually use them there without creating a cyclical import issue.
I guess I could just define these constants in here and then just expose
them in the `table` package, not as convenient for someone who just wants to
see the value of the constant in the docs, but it works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
