Re: parquet-mr build fail with jdk7; move to jdk8?

2018-01-18 Thread Gabor Szadovszky
Created PARQUET-1198 to track moving to java8. Cheers, Gabor > On 18 Jan 2018, at 22:16, Ryan Blue wrote: > > I think we should move. JDK7 has been EOL for a couple years now and it's a > pain to even install an

[jira] [Created] (PARQUET-1198) Bump java source and target to java8

2018-01-18 Thread Gabor Szadovszky (JIRA)
Gabor Szadovszky created PARQUET-1198: - Summary: Bump java source and target to java8 Key: PARQUET-1198 URL: https://issues.apache.org/jira/browse/PARQUET-1198 Project: Parquet Issue

[jira] [Commented] (PARQUET-41) Add bloom filters to parquet statistics

2018-01-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331602#comment-16331602 ] ASF GitHub Bot commented on PARQUET-41: --- cjjnjust closed pull request #431: PARQUET-41: Add bloom

[jira] [Commented] (PARQUET-41) Add bloom filters to parquet statistics

2018-01-18 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331598#comment-16331598 ] ASF GitHub Bot commented on PARQUET-41: --- cjjnjust opened a new pull request #431: PARQUET-41: Add

Request for PR Review

2018-01-18 Thread Pradeep Gollakota
Hi All, Can one of you review my PR at https://github.com/apache/parquet-mr/pull/447 please? Thanks, Pradeep

Re: Null count not set if min/max is to large ?

2018-01-18 Thread Ryan Blue
If I remember correctly, the Java Statistics objects assume that if the null count is present and not equal to the value count, then the min and max must be there. That's why we don't write any stats when min and max won't be written. I wouldn't worry about this too much. There are a couple of

Re: parquet-mr build fail with jdk7; move to jdk8?

2018-01-18 Thread Ryan Blue
I think we should move. JDK7 has been EOL for a couple years now and it's a pain to even install an old JDK7 these days. rb On Thu, Jan 18, 2018 at 2:31 AM, Gabor Szadovszky < gabor.szadovs...@cloudera.com> wrote: > Hi, > > The last commit in parquet-mr master >

[jira] [Commented] (PARQUET-1189) Release Parquet Java 1.10

2018-01-18 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331233#comment-16331233 ] Ryan Blue commented on PARQUET-1189: We don't typically publish snapshots, but it isn't difficult to

[jira] [Updated] (PARQUET-1178) Parquet modular encryption

2018-01-18 Thread Gidon Gershinsky (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-1178: -- Description: A mechanism for modular encryption and decryption of Parquet files.

Re: Parquet modular encryption

2018-01-18 Thread Gidon Gershinsky
I have posted a link to the design draft at the Jira. All comments are welcome. Thanks to Julien for an initial feedback and suggestions, at our chat during my SF trip last week. The design document is relatively detailed, with 8 pages its actually longer than the code used to implement it :).

parquet-mr build fail with jdk7; move to jdk8?

2018-01-18 Thread Gabor Szadovszky
Hi, The last commit in parquet-mr master (c6764c4a0848abf1d581e22df8b33e28ee9f2ced) does not build with jdk7 only with jdk8. We did not catch the issue because either Travis and me use jdk8 to build parquet-mr. (The source level in the pom.xml is set to 1.7 so both jdk7 and jdk8 should be able