[jira] [Commented] (PARQUET-1679) Invalid SchemaException for UUID while using AvroParquetWriter

2019-10-28 Thread Qinghui Xu (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961457#comment-16961457 ] Qinghui Xu commented on PARQUET-1679: - Sorry for the delay. Yes, you got it right

PARQUET-1685:Truncate Min/Max for Statistics

2019-10-28 Thread Xinli shang
Hi all, If you know any Parquet application requires the min/max value of statistics to be real column value, please let me know. Also, If you have any questions or concerns about this feature, feel free to raise comments in the design document

[jira] [Commented] (PARQUET-1679) Invalid SchemaException for UUID while using AvroParquetWriter

2019-10-28 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961429#comment-16961429 ] Felix Kizhakkel Jose commented on PARQUET-1679: --- [~q.xu] what do you thin

[jira] [Commented] (PARQUET-1680) Parquet Java Serialization is very slow

2019-10-28 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961428#comment-16961428 ] Felix Kizhakkel Jose commented on PARQUET-1680: --- Any help would be much a

Re: block-vs-frame LZ4 in Parquet

2019-10-28 Thread Wes McKinney
If the specification is not clear, I think we should first clarify what's there and then try to have multiple compatible implementations of the specification. On Sun, Oct 27, 2019 at 3:35 PM Michael Heuer wrote: > > Thank you, Wes. > > What about adding separate LZ4_BLOCK and LZ4_FRAME definition

[jira] [Commented] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961207#comment-16961207 ] Xinli Shang commented on PARQUET-1685: -- Sounds good [~gszadovszky] and [~rdblue].

[jira] [Commented] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Ryan Blue (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961187#comment-16961187 ] Ryan Blue commented on PARQUET-1685: Looks like Gabor is right. The stats fields us

[jira] [Commented] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Gabor Szadovszky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961177#comment-16961177 ] Gabor Szadovszky commented on PARQUET-1685: --- Hi [~sha...@uber.com], I am fin

[jira] [Comment Edited] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961165#comment-16961165 ] Xinli Shang edited comment on PARQUET-1685 at 10/28/19 3:35 PM: -

[jira] [Comment Edited] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961165#comment-16961165 ] Xinli Shang edited comment on PARQUET-1685 at 10/28/19 3:35 PM: -

[jira] [Commented] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961165#comment-16961165 ] Xinli Shang commented on PARQUET-1685: -- Hi [~gszadovszky] Thanks for your reply!  

[jira] [Commented] (PARQUET-1685) Truncate the stored min and max for String statistics to reduce the footer size

2019-10-28 Thread Gabor Szadovszky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960878#comment-16960878 ] Gabor Szadovszky commented on PARQUET-1685: --- We implemented a similar feature