[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-14 Thread Lukasz Gajowy (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907334#comment-16907334 ] Lukasz Gajowy commented on BEAM-5164: - I'm fine with adding the docs as you suggested. I left comments

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-14 Thread Ryan Skraba (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907232#comment-16907232 ] Ryan Skraba commented on BEAM-5164: --- OK -- one huge complication :/ Relocating the parquet library

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-13 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906445#comment-16906445 ] Luke Cwik commented on BEAM-5164: - In this specific case I think we could shade the parquet library.  

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-13 Thread Ryan Skraba (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906154#comment-16906154 ] Ryan Skraba commented on BEAM-5164: --- I am not confident on the overall strategy with respect to

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-13 Thread Lukasz Gajowy (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906058#comment-16906058 ] Lukasz Gajowy commented on BEAM-5164: - Do you know what is the situation on Flink runner? Should we

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-13 Thread Ryan Skraba (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906020#comment-16906020 ] Ryan Skraba commented on BEAM-5164: --- I checked with a spark local run on Spark 2.4.3, there is no issue

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-13 Thread Lukasz Gajowy (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905999#comment-16905999 ] Lukasz Gajowy commented on BEAM-5164: - > Were you running with your own spark installation? At the

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-13 Thread Ryan Skraba (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905907#comment-16905907 ] Ryan Skraba commented on BEAM-5164: --- Thanks for the link for the context! Is it possible that

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-12 Thread Lukasz Gajowy (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905107#comment-16905107 ] Lukasz Gajowy commented on BEAM-5164: - I think so but I'm not very confident when it comes to these

[jira] [Commented] (BEAM-5164) ParquetIOIT fails on Spark and Flink

2019-08-09 Thread Ryan Skraba (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903656#comment-16903656 ] Ryan Skraba commented on BEAM-5164: --- See question and possible workarounds on [stack