[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311363#comment-15311363
]
Cheng Lian commented on SPARK-11153:
Yea, right. Can we do it later on master to minimize merge
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311252#comment-15311252
]
Mark Hamstra commented on SPARK-11153:
--
If I am not mistaken, Parquet 1.8.1 and filter push-down for
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311231#comment-15311231
]
Cheng Lian commented on SPARK-11153:
Unfortunately we just decided to revert Parquet 1.8.1. See:
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1530#comment-1530
]
Ian commented on SPARK-11153:
-
Cheng,
Can we revisit SPARK-11784?
CC [~markhamstra]
> Turns off Parquet
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008897#comment-15008897
]
Ian commented on SPARK-11153:
-
Cheng:
Any plan to enable pushdown for timestamps?
Should I open a ticket
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008827#comment-15008827
]
Cheng Lian commented on SPARK-11153:
You are right that Impala and Hive write timestamps as Parquet
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009057#comment-15009057
]
Cheng Lian commented on SPARK-11153:
Yes please file a ticket.
One complication here is that,
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009279#comment-15009279
]
Ian commented on SPARK-11153:
-
SPARK-11784 is now tracking the need of timestamp pushdown.
> Turns off
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008074#comment-15008074
]
Ian commented on SPARK-11153:
-
Hi, Cheng:
Filter pushdown for Timestamp type?
It appeared that the stats of
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005542#comment-15005542
]
Mark Hamstra commented on SPARK-11153:
--
Thanks.
> Turns off Parquet filter push-down for string and
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005004#comment-15005004
]
Cheng Lian commented on SPARK-11153:
Yes.
> Turns off Parquet filter push-down for string and binary
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005005#comment-15005005
]
Cheng Lian commented on SPARK-11153:
Yes.
> Turns off Parquet filter push-down for string and binary
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004944#comment-15004944
]
Mark Hamstra commented on SPARK-11153:
--
Is there a reason why parquet.version hasn't been pushed up
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005023#comment-15005023
]
Cheng Lian commented on SPARK-11153:
Good question. We tried, see [PR
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966306#comment-14966306
]
Romi Kuntsman commented on SPARK-11153:
---
Does this mean that all Spark 1.5.1 are recommended to set
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961147#comment-14961147
]
Michael Armbrust commented on SPARK-11153:
--
Its actually corrupted statistics in data that is
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961300#comment-14961300
]
Apache Spark commented on SPARK-11153:
--
User 'liancheng' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961281#comment-14961281
]
Cheng Lian commented on SPARK-11153:
Yes, it's the statistics information that is corrupted. And yes,
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961371#comment-14961371
]
Felix Cheung commented on SPARK-11153:
--
so the corrupted stats data would still be a problem when
[
https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961374#comment-14961374
]
Felix Cheung commented on SPARK-11153:
--
re-read what you said, I think it makes sense. I assume it
20 matches
Mail list logo
