[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-05 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-669725393 > extend ParquetWriteSupport which converts crypto setting in schema.. @shangxinli your previous pull request (already merged) added a very nice Encryption Factory in

Re: [VOTE] Release Apache Parquet 1.11.1 RC1

2020-08-05 Thread Ryan Blue
I’ll change my vote to a +1. If I run the tests using -pl :parquet-hadoop, they pass, as do the rest of the tests. That makes it harder to debug, but gives me confidence. The error appears to be with the Parquet build and not the code, so I think this won’t affect downstream users. And Iceberg tes

[GitHub] [parquet-mr] shangxinli commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-05 Thread GitBox
shangxinli commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-669333155 The way it works is SchemaCryptoPropertiesFactory should have RPC call with KMS to get the column key for keyId/metadata defined in metastore, and build up encryption proper

[jira] [Commented] (PARQUET-1878) [C++] lz4 codec is not compatible with Hadoop Lz4Codec

2020-08-05 Thread Antoine Pitrou (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171536#comment-17171536 ] Antoine Pitrou commented on PARQUET-1878: - In https://github.com/apache/hadoop

Re: [VOTE] Release Apache Parquet 1.11.1 RC1

2020-08-05 Thread Gabor Szadovszky
Hi Ryan, Did you have time to investigate this issue? Is it reproducible with the 1.11.0 release as well? Any other voters? We only have two binding votes and one is a +0. Thanks a lot, Gabor On Fri, Jul 31, 2020 at 9:40 AM Gabor Szadovszky wrote: > Hi Ryan, > > I have no idea. We are using t

[jira] [Commented] (PARQUET-1559) Add way to manually commit already written data to disk

2020-08-05 Thread Gabor Szadovszky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171442#comment-17171442 ] Gabor Szadovszky commented on PARQUET-1559: --- [~wxmimperio], as I've said, par

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #802: PARQUET-1885: Pass descriptor to ProtoWriteSupport constructor

2020-08-05 Thread GitBox
gszadovszky commented on a change in pull request #802: URL: https://github.com/apache/parquet-mr/pull/802#discussion_r465664459 ## File path: parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoDescriptorSupport.java ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apach

[jira] [Commented] (PARQUET-1885) [parquet-protobuf] Pass descriptor to ProtoWriteSupport constructor

2020-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171438#comment-17171438 ] ASF GitHub Bot commented on PARQUET-1885: - gszadovszky commented on a change in

[jira] [Commented] (PARQUET-1559) Add way to manually commit already written data to disk

2020-08-05 Thread wxmimperio (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171434#comment-17171434 ] wxmimperio commented on PARQUET-1559: - [~gszadovszky] Hi, If I need to write multi

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-05 Thread GitBox
gszadovszky commented on a change in pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#discussion_r465644605 ## File path: parquet-column/src/main/java/org/apache/parquet/schema/Type.java ## @@ -363,4 +365,11 @@ void checkContains(Type subType) { */