[jira] [Assigned] (PARQUET-1102) Travis CI builds are failing for parquet-format PRs

2017-09-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned PARQUET-1102: --- Assignee: Cheng Lian > Travis CI builds are failing for parquet-format PRs >

[jira] [Resolved] (PARQUET-1091) Wrong and broken links in README

2017-09-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-1091. - Resolution: Fixed Fix Version/s: format-2.3.2 Issue resolved by pull request 65

[jira] [Resolved] (PARQUET-1102) Travis CI builds are failing for parquet-format PRs

2017-09-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-1102. - Resolution: Fixed Fix Version/s: format-2.3.2 Issue resolved by pull request 66

[jira] [Updated] (PARQUET-1102) Travis CI builds are failing for parquet-format PRs

2017-09-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-1102: Priority: Blocker (was: Major) > Travis CI builds are failing for parquet-format PRs >

[jira] [Created] (PARQUET-1102) Travis CI builds are failing for parquet-format PRs

2017-09-12 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-1102: --- Summary: Travis CI builds are failing for parquet-format PRs Key: PARQUET-1102 URL: https://issues.apache.org/jira/browse/PARQUET-1102 Project: Parquet Issue

[jira] [Created] (PARQUET-1091) Wrong and broken links in README

2017-09-07 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-1091: --- Summary: Wrong and broken links in README Key: PARQUET-1091 URL: https://issues.apache.org/jira/browse/PARQUET-1091 Project: Parquet Issue Type: Bug

[jira] [Comment Edited] (PARQUET-980) Cannot read row group larger than 2GB

2017-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007326#comment-16007326 ] Cheng Lian edited comment on PARQUET-980 at 5/11/17 10:46 PM: -- The current

[jira] [Commented] (PARQUET-980) Cannot read row group larger than 2GB

2017-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007326#comment-16007326 ] Cheng Lian commented on PARQUET-980: The current write path ensures that it never writes a page that

[jira] [Updated] (PARQUET-980) Cannot read row group larger than 2GB

2017-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-980: --- Affects Version/s: 1.8.1 1.8.2 > Cannot read row group larger than 2GB >

[jira] [Updated] (PARQUET-893) GroupColumnIO.getFirst() doesn't check for empty groups

2017-02-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-893: --- Description: The following Spark snippet reproduces this issue with Spark 2.1 (with parquet-mr

[jira] [Created] (PARQUET-893) GroupColumnIO.getFirst() doesn't check for empty groups

2017-02-22 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-893: -- Summary: GroupColumnIO.getFirst() doesn't check for empty groups Key: PARQUET-893 URL: https://issues.apache.org/jira/browse/PARQUET-893 Project: Parquet Issue

[jira] [Created] (PARQUET-754) Deprecate the "strict" argument in MessageType.union()

2016-10-17 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-754: -- Summary: Deprecate the "strict" argument in MessageType.union() Key: PARQUET-754 URL: https://issues.apache.org/jira/browse/PARQUET-754 Project: Parquet Issue

[jira] [Commented] (PARQUET-753) GroupType.union() doesn't merge the original type

2016-10-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583942#comment-15583942 ] Cheng Lian commented on PARQUET-753: PARQUET-379 resolves the {{union}} issue related to primitive

[jira] [Updated] (PARQUET-655) The LogicalTypes.md link in README.md points to the old Parquet GitHub repository

2016-07-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-655: --- Component/s: parquet-format > The LogicalTypes.md link in README.md points to the old Parquet GitHub

[jira] [Created] (PARQUET-655) The LogicalTypes.md link in README.md points to the old Parquet GitHub repository

2016-07-08 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-655: -- Summary: The LogicalTypes.md link in README.md points to the old Parquet GitHub repository Key: PARQUET-655 URL: https://issues.apache.org/jira/browse/PARQUET-655

[jira] [Created] (PARQUET-654) Make record-level filtering optional

2016-07-08 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-654: -- Summary: Make record-level filtering optional Key: PARQUET-654 URL: https://issues.apache.org/jira/browse/PARQUET-654 Project: Parquet Issue Type: Improvement

[jira] [Updated] (PARQUET-651) Parquet-avro fails to decode array of record with a single field name "element" correctly

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-651: --- Affects Version/s: 1.9.0 > Parquet-avro fails to decode array of record with a single field name >

[jira] [Updated] (PARQUET-651) Parquet-avro fails to decode array of record with a single field name "element" correctly

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-651: --- Description: Found this issue while investigating SPARK-16344. For the following Parquet schema

[jira] [Updated] (PARQUET-651) Parquet-avro fails to decode array of record with a single field name "element" correctly

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-651: --- Description: Found this issue while investigating SPARK-16344. For the following Parquet schema

[jira] [Resolved] (PARQUET-528) Fix flush() for RecordConsumer and implementations

2016-03-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-528. Resolution: Fixed Issue resolved by pull request 325

[jira] [Commented] (PARQUET-401) Deprecate Log and move to SLF4J Logger

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127668#comment-15127668 ] Cheng Lian commented on PARQUET-401: Fix of this issue is nice to have but probably shouldn't block

[jira] [Resolved] (PARQUET-495) Fix mismatches in Types class comments

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-495. Resolution: Fixed Issue resolved by pull request 317

[jira] [Resolved] (PARQUET-432) Complete a todo for method ColumnDescriptor.compareTo()

2016-01-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-432. Resolution: Fixed Issue resolved by pull request 314

[jira] [Created] (PARQUET-398) Testing JIRA ticket for testing committership

2015-12-02 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-398: -- Summary: Testing JIRA ticket for testing committership Key: PARQUET-398 URL: https://issues.apache.org/jira/browse/PARQUET-398 Project: Parquet Issue Type: Test

[jira] [Created] (PARQUET-389) Filter predicates should work with missing columns

2015-10-28 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-389: -- Summary: Filter predicates should work with missing columns Key: PARQUET-389 URL: https://issues.apache.org/jira/browse/PARQUET-389 Project: Parquet Issue Type:

[jira] [Comment Edited] (PARQUET-379) PrimitiveType.union erases original type

2015-09-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14933630#comment-14933630 ] Cheng Lian edited comment on PARQUET-379 at 9/28/15 5:34 PM: - While trying to

[jira] [Created] (PARQUET-385) PrimitiveType.union accepts fixed_len_byte_array fields with different length when strict mode is on

2015-09-28 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-385: -- Summary: PrimitiveType.union accepts fixed_len_byte_array fields with different length when strict mode is on Key: PARQUET-385 URL: https://issues.apache.org/jira/browse/PARQUET-385

[jira] [Commented] (PARQUET-379) PrimitiveType.union erases original type

2015-09-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14933630#comment-14933630 ] Cheng Lian commented on PARQUET-379: While trying to fix this issue, I got a problem regarding to the

[jira] [Commented] (PARQUET-379) PrimitiveType.union erases original type

2015-09-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934084#comment-14934084 ] Cheng Lian commented on PARQUET-379: So deprecating non-strict schema merging seems to be reasonable?

[jira] [Updated] (PARQUET-385) PrimitiveType.union accepts fixed_len_byte_array fields with different lengths when strict mode is on

2015-09-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-385: --- Summary: PrimitiveType.union accepts fixed_len_byte_array fields with different lengths when strict

[jira] [Updated] (PARQUET-379) PrimitiveType.union erases original type

2015-09-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-379: --- Description: The following ScalaTest test case {code} test("merge primitive types") { val

[jira] [Created] (PARQUET-379) PrimitiveType.union erases original type

2015-09-23 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-379: -- Summary: PrimitiveType.union erases original type Key: PARQUET-379 URL: https://issues.apache.org/jira/browse/PARQUET-379 Project: Parquet Issue Type: Bug

[jira] [Updated] (PARQUET-379) PrimitiveType.union erases original type

2015-09-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-379: --- Description: The following ScalaTest test case {code} test("merge primitive types") { val

[jira] [Updated] (PARQUET-371) Bumps Thrift version to 0.9.0

2015-09-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-371: --- Summary: Bumps Thrift version to 0.9.0 (was: Add thrift9 Maven profile for parquet-format) > Bumps

[jira] [Updated] (PARQUET-371) Bumps Thrift version to 0.9.0

2015-09-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-371: --- Description: Thrift 0.7.0 is too old a version, and it doesn't compile on Mac. Would be nice to bump

[jira] [Comment Edited] (PARQUET-370) Nested records are not properly read if none of their fields are requested

2015-09-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734568#comment-14734568 ] Cheng Lian edited comment on PARQUET-370 at 9/10/15 11:43 AM: -- A complete

[jira] [Commented] (PARQUET-371) Add thrift9 Maven profile for parquet-format

2015-09-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740068#comment-14740068 ] Cheng Lian commented on PARQUET-371: That would be even nicer. I'll update my PR. > Add thrift9

[jira] [Created] (PARQUET-371) Add thrift9 Maven profile for parquet-format

2015-09-09 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-371: -- Summary: Add thrift9 Maven profile for parquet-format Key: PARQUET-371 URL: https://issues.apache.org/jira/browse/PARQUET-371 Project: Parquet Issue Type:

[jira] [Commented] (PARQUET-370) Nested records are not properly read if none of their fields are requested

2015-09-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734568#comment-14734568 ] Cheng Lian commented on PARQUET-370: A complete sample code for reproducing this issue against

[jira] [Commented] (PARQUET-369) Shading SLF4J prevents SLF4J locating org.slf4j.impl.StaticLoggerBinder

2015-09-07 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733506#comment-14733506 ] Cheng Lian commented on PARQUET-369: Here is a more concrete version in another thread

[jira] [Updated] (PARQUET-369) Shading SLF4J prevents SLF4J locating org.slf4j.impl.StaticLoggerBinder

2015-09-07 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-369: --- Description: Parquet-format shades SLF4J to {{parquet.org.slf4j}} (see

[jira] [Created] (PARQUET-369) Shading SLF4J prevents SLF4J locating org.slf4j.impl.StaticLoggerBinder

2015-09-05 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-369: -- Summary: Shading SLF4J prevents SLF4J locating org.slf4j.impl.StaticLoggerBinder Key: PARQUET-369 URL: https://issues.apache.org/jira/browse/PARQUET-369 Project: Parquet

[jira] [Updated] (PARQUET-364) Parquet-avro cannot decode Avro/Thrift array of primitive array (e.g. array<array>)

2015-09-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-364: --- Summary: Parquet-avro cannot decode Avro/Thrift array of primitive array (e.g. array) (was:

[jira] [Created] (PARQUET-367) parquet-cat -j doesn't show all records

2015-08-27 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-367: -- Summary: parquet-cat -j doesn't show all records Key: PARQUET-367 URL: https://issues.apache.org/jira/browse/PARQUET-367 Project: Parquet Issue Type: Bug

[jira] [Commented] (PARQUET-364) Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint)

2015-08-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708387#comment-14708387 ] Cheng Lian commented on PARQUET-364: Sent out a PR

[jira] [Commented] (PARQUET-364) Parque-avro cannot decode Avro array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706766#comment-14706766 ] Cheng Lian commented on PARQUET-364: Although I haven't verified it yet, I suspect

[jira] [Commented] (PARQUET-364) Parque-avro cannot decode Avro array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706760#comment-14706760 ] Cheng Lian commented on PARQUET-364: Tried to write a test case in parquet-mr, but

[jira] [Updated] (PARQUET-364) Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-364: --- Description: The problematic Avro and Thrift schemas are: {noformat} record AvroArrayOfArray {

[jira] [Commented] (PARQUET-364) Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706959#comment-14706959 ] Cheng Lian commented on PARQUET-364: I tested the Thrift case with Thrift 0.9.2

[jira] [Updated] (PARQUET-364) Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-364: --- Description: The problematic Avro and Thrift schemas are: {noformat} record AvroArrayOfArray {

[jira] [Updated] (PARQUET-364) Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-364: --- Summary: Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint) (was:

[jira] [Commented] (PARQUET-364) Parque-avro cannot decode Avro array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706938#comment-14706938 ] Cheng Lian commented on PARQUET-364: Verified that parquet-avro doesn't correctly

[jira] [Commented] (PARQUET-364) Parque-avro cannot decode Avro/Thrift array of primitive array (e.g. arrayarrayint)

2015-08-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707150#comment-14707150 ] Cheng Lian commented on PARQUET-364: [~rdblue] The suggested fix has been verified by

[jira] [Created] (PARQUET-363) Cannot construct empty MessageType for ReadContext.requestedSchema

2015-08-21 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-363: -- Summary: Cannot construct empty MessageType for ReadContext.requestedSchema Key: PARQUET-363 URL: https://issues.apache.org/jira/browse/PARQUET-363 Project: Parquet

[jira] [Updated] (PARQUET-173) StatisticsFilter doesn't handle And properly

2015-08-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-173: --- Description: I guess it's [a pretty straightforward

[jira] [Updated] (PARQUET-136) NPE thrown in StatisticsFilter when all values in a string/binary column trunk are null

2015-08-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-136: --- Description: For a string or a binary column, if all values in a single column trunk are null, so

[jira] [Commented] (PARQUET-70) PARQUET #36: Pig Schema Storage to UDFContext

2015-07-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622600#comment-14622600 ] Cheng Lian commented on PARQUET-70: --- Just remove the incubator- part of the URL:

[jira] [Comment Edited] (PARQUET-222) parquet writer runs into OOM during writing when calling DataFrame.saveAsParquetFile in Spark SQL

2015-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580765#comment-14580765 ] Cheng Lian edited comment on PARQUET-222 at 6/10/15 4:37 PM: -

[jira] [Commented] (PARQUET-222) parquet writer runs into OOM during writing when calling DataFrame.saveAsParquetFile in Spark SQL

2015-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580765#comment-14580765 ] Cheng Lian commented on PARQUET-222: Hey [~rdblue], it seems that you are referring

[jira] [Comment Edited] (PARQUET-222) parquet writer runs into OOM during writing when calling DataFrame.saveAsParquetFile in Spark SQL

2015-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580339#comment-14580339 ] Cheng Lian edited comment on PARQUET-222 at 6/10/15 4:39 PM: -

[jira] [Created] (PARQUET-305) Logger instantiated for package org.apache.parquet may be GC-ed

2015-06-09 Thread Cheng Lian (JIRA)
Cheng Lian created PARQUET-305: -- Summary: Logger instantiated for package org.apache.parquet may be GC-ed Key: PARQUET-305 URL: https://issues.apache.org/jira/browse/PARQUET-305 Project: Parquet

[jira] [Comment Edited] (PARQUET-222) parquet writer runs into OOM during writing when calling DataFrame.saveAsParquetFile in Spark SQL

2015-06-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577257#comment-14577257 ] Cheng Lian edited comment on PARQUET-222 at 6/8/15 2:33 PM:

[jira] [Commented] (PARQUET-294) NPE in ParquetInputFormat.getSplits when no .parquet files exist

2015-06-07 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576258#comment-14576258 ] Cheng Lian commented on PARQUET-294: Is this related to PARQUET-151? NPE in

[jira] [Commented] (PARQUET-222) parquet writer runs into OOM during writing when calling DataFrame.saveAsParquetFile in Spark SQL

2015-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575783#comment-14575783 ] Cheng Lian commented on PARQUET-222: There are several ways to alleviate this.

[jira] [Updated] (PARQUET-222) parquet writer runs into OOM during writing when calling DataFrame.saveAsParquetFile in Spark SQL

2015-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-222: --- Description: In Spark SQL, there is a function {{saveAsParquetFile}} in {{DataFrame}} or

[jira] [Updated] (PARQUET-293) ScalaReflectionException when trying to convert an RDD of Scrooge to a DataFrame

2015-05-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-293: --- Description: I get scala.ScalaReflectionException: none is not a term when I try to convert an RDD

[jira] [Commented] (PARQUET-293) ScalaReflectionException when trying to convert an RDD of Scrooge to a DataFrame

2015-05-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564332#comment-14564332 ] Cheng Lian commented on PARQUET-293: Hm, it's possible. But the context is a little