[jira] [Commented] (SPARK-30162) Filter is not being pushed down for Parquet files

2019-12-07 Thread Nasir Ali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990741#comment-16990741 ] Nasir Ali commented on SPARK-30162: --- No, I only use pyspark shell. > Filter is not be

[jira] [Commented] (SPARK-30162) Filter is not being pushed down for Parquet files

2019-12-07 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990725#comment-16990725 ] t oo commented on SPARK-30162: -- did u try on scala sparkshell? > Filter is not being pushe

[jira] [Commented] (SPARK-27623) Provider org.apache.spark.sql.avro.AvroFileFormat could not be instantiated

2019-12-07 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990723#comment-16990723 ] t oo commented on SPARK-27623: -- bump > Provider org.apache.spark.sql.avro.AvroFileFormat c

[jira] [Commented] (SPARK-26346) Upgrade parquet to 1.11.0

2019-12-07 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990722#comment-16990722 ] t oo commented on SPARK-26346: -- bump > Upgrade parquet to 1.11.0 > ---

[jira] [Updated] (SPARK-30158) Resolve Array + reference type compile problems in 2.13, with sc.parallelize

2019-12-07 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30158: - Description: Scala 2.13 has some different rules about resolving Arrays as Seqs when the array

[jira] [Assigned] (SPARK-30154) PySpark UDF to convert MLlib vectors to dense arrays

2019-12-07 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-30154: -- Assignee: Weichen Xu > PySpark UDF to convert MLlib vectors to dense arrays > ---

[jira] [Commented] (SPARK-30154) PySpark UDF to convert MLlib vectors to dense arrays

2019-12-07 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990696#comment-16990696 ] Weichen Xu commented on SPARK-30154: I began working on this. Thanks :) > PySpark U

[jira] [Commented] (SPARK-27292) Spark Job Fails with Unknown Error writing to S3 from AWS EMR

2019-12-07 Thread Kaihui Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990688#comment-16990688 ] Kaihui Gao commented on SPARK-27292: HiĀ [~padmakarm9], I am facing a similar issue w

[jira] [Updated] (SPARK-30164) Fix document generation in "sbt unidoc"

2019-12-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-30164: --- Component/s: (was: Build) Tests > Fix document generation in "sbt unido

[jira] [Created] (SPARK-30164) Fix document generation in "sbt unidoc"

2019-12-07 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-30164: -- Summary: Fix document generation in "sbt unidoc" Key: SPARK-30164 URL: https://issues.apache.org/jira/browse/SPARK-30164 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-30163) Use Google Maven mirror in GitHub Action

2019-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30163. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26793 [https://

[jira] [Assigned] (SPARK-30163) Use Google Maven mirror in GitHub Action

2019-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30163: - Assignee: Dongjoon Hyun > Use Google Maven mirror in GitHub Action > --

[jira] [Commented] (SPARK-30108) Add robust accumulator for observable metrics

2019-12-07 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990595#comment-16990595 ] Ankit Raj Boudh commented on SPARK-30108: - I will start working on this feature

[jira] [Created] (SPARK-30163) Use Google Maven mirror in GitHub Action

2019-12-07 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30163: - Summary: Use Google Maven mirror in GitHub Action Key: SPARK-30163 URL: https://issues.apache.org/jira/browse/SPARK-30163 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-30162) Filter is not being pushed down for Parquet files

2019-12-07 Thread Nasir Ali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-30162: -- Description: Filters are not pushed down in Spark 3.0 preview. Also the output of "explain" method is