RussellSpitzer commented on code in PR #3935:
URL: https://github.com/apache/parquet-java/pull/3935#discussion_r4557310869
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java:
##########
@@ -1385,16 +1422,126 @@ public void writeDataPageV2(
byte[] pageHeaderAAD,
SizeStatistics sizeStatistics)
throws IOException {
+ writeDataPageV2Parts(
+ rowCount,
+ nullCount,
+ valueCount,
+ toIntWithCheck(repetitionLevels.size(), "page repetition levels"),
+ toIntWithCheck(definitionLevels.size(), "page definition levels"),
+ repetitionLevels,
+ definitionLevels,
+ dataEncoding,
+ bytes,
+ compressed,
+ uncompressedDataSize,
+ statistics,
+ metadataBlockEncryptor,
+ pageHeaderAAD,
+ sizeStatistics);
+ }
+
+ /**
+ * Writes a single v2 data page. If {@code pageBlockEncryptor} is not null,
the repetition levels,
+ * definition levels and data bytes are encrypted together as one module.
+ *
+ * @param rowCount count of rows
+ * @param nullCount count of nulls
+ * @param valueCount count of values
+ * @param repetitionLevels repetition level bytes
+ * @param definitionLevels definition level bytes
+ * @param dataEncoding encoding for data
+ * @param bytes data bytes, not yet encrypted
+ * @param compressed whether the data bytes is compressed
+ * @param uncompressedDataSize the size of uncompressed data
+ * @param statistics the statistics of the page
+ * @param metadataBlockEncryptor encryptor for the page header
+ * @param pageHeaderAAD pageHeader AAD
+ * @param sizeStatistics size statistics for the page
+ * @param pageBlockEncryptor encryptor for the levels and data bytes, or
null if the column is not
+ * encrypted; null if and only if {@code
metadataBlockEncryptor} is null
+ * @param dataPageAAD data page AAD, updated for this page's ordinal;
required if
+ * {@code pageBlockEncryptor} is not null
+ * @throws IOException if any I/O error occurs during writing the file
+ */
+ public void writeDataPageV2(
Review Comment:
The old methods expected that you were passing through encrypted bytes with
the levels coming in separately. That's no good see we need to also encrypt the
levels along with that data. So now we have a method which lets us send a
metadata encryptor and a page block encryptor along with the levels so we can
correctly encrypt it all together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]