Re: The largest table that Parquet can support

2016-01-07 Thread Reuben Kuhnert
.. > .. > {"name": "attribute50", "type": "int"} > ] > } > > For each record in the table, it may not have every attribute valid. For > example, a record of Profile may only have M1, M20 and M

Re: The largest table that Parquet can support

2016-01-06 Thread Reuben Kuhnert
Hi Yan, So the primary concern here would be the 'row group' size that you're using for your table. The row group is basically what determines how much information is stored in memory before being flushed to disk (this becomes an even greater issue if you have multiple parquet files open simultane

Re: Parquet sync up

2015-12-16 Thread Reuben Kuhnert
ay at 10am PT > https://plus.google.com/events/ca6mpc8n9akhhunnp6u9q1g1bh4 > > On Mon, Dec 14, 2015 at 5:28 PM, Jason Altekruse > > wrote: > > > +1 > > > > On Mon, Dec 14, 2015 at 4:19 PM, Daniel Weeks > > > wrote: > > > > > W

[jira] [Assigned] (PARQUET-406) Counter Initialization causes NPE

2015-12-15 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert reassigned PARQUET-406: -- Assignee: Reuben Kuhnert > Counter Initialization causes

Re: Error due to null Counter

2015-12-15 Thread Reuben Kuhnert
Oh, and *groupName *missed that one. Thanks On Tue, Dec 15, 2015 at 9:58 AM, Reuben Kuhnert wrote: > Hi again, > > So I'm looking into your issue and I'm wondering if you can't send me a > few pieces of information. > > (1) Can I get a stacktrace when

Re: Error due to null Counter

2015-12-15 Thread Reuben Kuhnert
Hi again, So I'm looking into your issue and I'm wondering if you can't send me a few pieces of information. (1) Can I get a stacktrace when this line is called? public MapReduceCounterAdapter(Counter adaptee) { *this.adaptee = adaptee;* } (2) Can you send me information about this:

[jira] [Created] (PARQUET-406) Counter Initialization causes NPE

2015-12-14 Thread Reuben Kuhnert (JIRA)
Reuben Kuhnert created PARQUET-406: -- Summary: Counter Initialization causes NPE Key: PARQUET-406 URL: https://issues.apache.org/jira/browse/PARQUET-406 Project: Parquet Issue Type: Bug

Re: Error due to null Counter

2015-12-14 Thread Reuben Kuhnert
Hi Stephen, I created ticket: https://issues.apache.org/jira/browse/PARQUET-406 to track your issue. We'll take a look to track down your issue and then get back to you. Thanks, and let us know if you have any other questions. Reuben On Mon, Dec 14, 2015 at 12:22 PM, Stephen Bly wrote: > Greet

Re: Parquet sync up

2015-12-14 Thread Reuben Kuhnert
I can make that. Thanks On Mon, Dec 14, 2015 at 12:27 PM, Ryan Blue wrote: > Works for me. > > > On 12/12/2015 03:20 PM, Julien Le Dem wrote: > >> The next parquet sync up is scheduled for next week Wednesday at 10 am PT >> Any objection to move it to Thursday same time? I have a conflict. >> >>

Re: [DISCUSS] Weekly triage rotation

2015-12-01 Thread Reuben Kuhnert
Yeah, I'm in as well. On Tue, Dec 1, 2015 at 3:33 PM, Jason Altekruse wrote: > While I am not a committer, and would need to ask some questions along the > way, I would be willing to sign up for triaging to get more involved with > the project. > > On Tue, Dec 1, 2015 at 11:55 AM, Daniel Weeks

Re: [ANNOUNCE] New Parquet committer: Sergio Peña

2015-11-23 Thread Reuben Kuhnert
Awesome, well done Sergio! On Mon, Nov 23, 2015 at 1:58 PM, Mohammad Islam wrote: > Congrats Sergio! > ---Mohammad > > > On Monday, November 23, 2015 11:55 AM, Ryan Blue > wrote: > > > On behalf of the Parquet PMC, I'm happy to announce that Sergio Peña has > been invited to be a committer

[jira] [Commented] (PARQUET-388) ProtoRecordConverter might wrongly cast a Message.Builder to Message

2015-11-03 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987875#comment-14987875 ] Reuben Kuhnert commented on PARQUET-388: Just as a general comment, retur

[jira] [Updated] (PARQUET-112) RunLengthBitPackingHybridDecoder: Reading past RLE/BitPacking stream.

2015-11-02 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert updated PARQUET-112: --- Description: I am using Avro and Crunch 0.11 to write data into Hadoop CDH 4.6 in parquet

[jira] [Assigned] (PARQUET-112) RunLengthBitPackingHybridDecoder: Reading past RLE/BitPacking stream.

2015-11-02 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert reassigned PARQUET-112: -- Assignee: Reuben Kuhnert > RunLengthBitPackingHybridDecoder: Reading past

[jira] [Commented] (PARQUET-388) ProtoRecordConverter might wrongly cast a Message.Builder to Message

2015-11-02 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985793#comment-14985793 ] Reuben Kuhnert commented on PARQUET-388: I'm a little confused on

[jira] [Updated] (PARQUET-388) ProtoRecordConverter might wrongly cast a Message.Builder to Message

2015-11-02 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert updated PARQUET-388: --- Description: ProtoRecordConverter returns current record as follows: {code} public T

[jira] [Assigned] (PARQUET-388) ProtoRecordConverter might wrongly cast a Message.Builder to Message

2015-10-21 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert reassigned PARQUET-388: -- Assignee: Reuben Kuhnert > ProtoRecordConverter might wrongly cast a Message.Buil

[jira] [Commented] (PARQUET-349) VersionParser does not handle versions like "parquet-mr 1.6.0rc4"

2015-10-19 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963892#comment-14963892 ] Reuben Kuhnert commented on PARQUET-349: Added patch (PR-283). > Versio

[jira] [Assigned] (PARQUET-349) VersionParser does not handle versions like "parquet-mr 1.6.0rc4"

2015-10-19 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert reassigned PARQUET-349: -- Assignee: Reuben Kuhnert > VersionParser does not handle versions like "pa

[jira] [Commented] (PARQUET-367) "parquet-cat -j" doesn't show all records

2015-10-16 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960968#comment-14960968 ] Reuben Kuhnert commented on PARQUET-367: Added pull request. > "pa

[jira] [Assigned] (PARQUET-367) "parquet-cat -j" doesn't show all records

2015-10-15 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert reassigned PARQUET-367: -- Assignee: Reuben Kuhnert > "parquet-cat -j" doesn'

[jira] [Commented] (PARQUET-220) Unnecessary warning in ParquetRecordReader.initialize

2015-10-08 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949147#comment-14949147 ] Reuben Kuhnert commented on PARQUET-220: Added new pull request for this i

[jira] [Assigned] (PARQUET-220) Unnecessary warning in ParquetRecordReader.initialize

2015-10-08 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reuben Kuhnert reassigned PARQUET-220: -- Assignee: Reuben Kuhnert > Unnecessary warning in ParquetRecordReader.initial

[jira] [Commented] (PARQUET-355) Create Integration tests to validate statistics

2015-08-21 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706784#comment-14706784 ] Reuben Kuhnert commented on PARQUET-355: New update available. >

[jira] [Commented] (PARQUET-355) Create Integration tests to validate statistics

2015-08-17 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699537#comment-14699537 ] Reuben Kuhnert commented on PARQUET-355: Updated Pull request with cha

[jira] [Commented] (PARQUET-355) Create Integration tests to validate statistics

2015-08-07 Thread Reuben Kuhnert (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662053#comment-14662053 ] Reuben Kuhnert commented on PARQUET-355: [Updated|https://github.com/ap

[jira] [Created] (PARQUET-355) Create Integration tests to validate statistics

2015-08-07 Thread Reuben Kuhnert (JIRA)
Reuben Kuhnert created PARQUET-355: -- Summary: Create Integration tests to validate statistics Key: PARQUET-355 URL: https://issues.apache.org/jira/browse/PARQUET-355 Project: Parquet Issue