[jira] [Resolved] (SPARK-25817) Dataset encoder should support combination of map and product type

[ https://issues.apache.org/jira/browse/SPARK-25817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25817. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22812

[jira] [Commented] (SPARK-25833) Views without column names created by Hive are not readable by Spark

[ https://issues.apache.org/jira/browse/SPARK-25833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1291#comment-1291 ] Chenxiao Mao commented on SPARK-25833: -- [~dkbiswal] Thanks for you comments. I think you are right

[jira] [Updated] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from $PWD to HDFS

[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Senia updated SPARK-25778: --- Attachment: SPARK-25778.patch > WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of

[jira] [Commented] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from $PWD to HDFS

[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1285#comment-1285 ] Apache Spark commented on SPARK-25778: -- User 'gss2002' has created a pull request for this issue:

[jira] [Commented] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from $PWD to HDFS

[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1284#comment-1284 ] Apache Spark commented on SPARK-25778: -- User 'gss2002' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from $PWD to HDFS

[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25778: Assignee: Apache Spark > WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack

[jira] [Assigned] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from $PWD to HDFS

[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25778: Assignee: (was: Apache Spark) > WriteAheadLogBackedBlockRDD in YARN Cluster Mode

[jira] [Updated] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from $PWD to HDFS

[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Senia updated SPARK-25778: --- Summary: WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to tmpDir from

[jira] [Assigned] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19851: --- Assignee: Dilip Biswal > Add support for EVERY and ANY (SOME) aggregates >

[jira] [Resolved] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19851. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22809

[jira] [Commented] (SPARK-12172) Consider removing SparkR internal RDD APIs

[ https://issues.apache.org/jira/browse/SPARK-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1252#comment-1252 ] Apache Spark commented on SPARK-12172: -- User 'felixcheung' has created a pull request for this

[jira] [Commented] (SPARK-12172) Consider removing SparkR internal RDD APIs

[ https://issues.apache.org/jira/browse/SPARK-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1251#comment-1251 ] Apache Spark commented on SPARK-12172: -- User 'felixcheung' has created a pull request for this

[jira] [Assigned] (SPARK-12172) Consider removing SparkR internal RDD APIs

[ https://issues.apache.org/jira/browse/SPARK-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12172: Assignee: (was: Apache Spark) > Consider removing SparkR internal RDD APIs >

[jira] [Assigned] (SPARK-12172) Consider removing SparkR internal RDD APIs

[ https://issues.apache.org/jira/browse/SPARK-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12172: Assignee: Apache Spark > Consider removing SparkR internal RDD APIs >

[jira] [Resolved] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-25859. -- Resolution: Fixed Assignee: Huaxin Gao Fix Version/s: 2.4.0

[jira] [Resolved] (SPARK-16693) Remove R deprecated methods

[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-16693. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 3.0.0 > Remove R

[jira] [Updated] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25823: -- Priority: Critical (was: Blocker) > map_filter can generate incorrect data >

[jira] [Updated] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25823: -- Affects Version/s: (was: 2.4.0) 3.0.0 > map_filter can generate

[jira] [Comment Edited] (SPARK-25833) Views without column names created by Hive are not readable by Spark

[ https://issues.apache.org/jira/browse/SPARK-25833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1178#comment-1178 ] Dilip Biswal edited comment on SPARK-25833 at 10/27/18 8:39 PM: This

[jira] [Resolved] (SPARK-25858) Passing Field Metadata to Parquet

[ https://issues.apache.org/jira/browse/SPARK-25858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved SPARK-25858. - Resolution: Later It is a little early to open this issue. I will re-open it after the

[jira] [Commented] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1211#comment-1211 ] Huaxin Gao commented on SPARK-25859: PowerIterationClustering is not in the doc either. Do I need to

[jira] [Assigned] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25861: Assignee: (was: Apache Spark) > Remove unused refreshInterval parameter from the

[jira] [Assigned] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25861: Assignee: Apache Spark > Remove unused refreshInterval parameter from the

[jira] [Commented] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1196#comment-1196 ] Apache Spark commented on SPARK-25861: -- User 'shahidki31' has created a pull request for this

[jira] [Commented] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1197#comment-1197 ] Apache Spark commented on SPARK-25861: -- User 'shahidki31' has created a pull request for this

[jira] [Created] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

shahid created SPARK-25861: -- Summary: Remove unused refreshInterval parameter from the headerSparkPage method. Key: SPARK-25861 URL: https://issues.apache.org/jira/browse/SPARK-25861 Project: Spark

[jira] [Assigned] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25859: Assignee: Apache Spark > add scala/java/python example and doc for PrefixSpan >

[jira] [Assigned] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25859: Assignee: (was: Apache Spark) > add scala/java/python example and doc for PrefixSpan

[jira] [Commented] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1189#comment-1189 ] Apache Spark commented on SPARK-25859: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1188#comment-1188 ] Apache Spark commented on SPARK-25859: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-25833) Views without column names created by Hive are not readable by Spark

[ https://issues.apache.org/jira/browse/SPARK-25833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1178#comment-1178 ] Dilip Biswal commented on SPARK-25833: -- This looks like a duplicate of 

[jira] [Commented] (SPARK-25661) Refactor AvroWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1147#comment-1147 ] Apache Spark commented on SPARK-25661: -- User 'yucai' has created a pull request for this issue:

[jira] [Commented] (SPARK-25663) Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1148#comment-1148 ] yucai commented on SPARK-25663: --- [~Gengliang.Wang] I make an improvement on this, could you help review?

[jira] [Assigned] (SPARK-25663) Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25663: Assignee: (was: Apache Spark) > Refactor BuiltInDataSourceWriteBenchmark and

[jira] [Assigned] (SPARK-25661) Refactor AvroWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25661: Assignee: (was: Apache Spark) > Refactor AvroWriteBenchmark to use main method >

[jira] [Commented] (SPARK-25663) Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1144#comment-1144 ] Apache Spark commented on SPARK-25663: -- User 'yucai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25661) Refactor AvroWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25661: Assignee: Apache Spark > Refactor AvroWriteBenchmark to use main method >

[jira] [Assigned] (SPARK-25663) Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25663: Assignee: Apache Spark > Refactor BuiltInDataSourceWriteBenchmark and

[jira] [Commented] (SPARK-25661) Refactor AvroWriteBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1146#comment-1146 ] Apache Spark commented on SPARK-25661: -- User 'yucai' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23367) Include python document style checking

[ https://issues.apache.org/jira/browse/SPARK-23367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23367. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22425

[jira] [Assigned] (SPARK-23367) Include python document style checking

[ https://issues.apache.org/jira/browse/SPARK-23367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23367: - Assignee: Rekha Joshi > Include python document style checking >

[jira] [Commented] (SPARK-25816) Functions does not resolve Columns correctly

[ https://issues.apache.org/jira/browse/SPARK-25816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1058#comment-1058 ] Peter Toth commented on SPARK-25816: Thanks [~bzhang], It seems both are regressions from 2.2 to 2.3

[jira] [Commented] (SPARK-24709) Inferring schema from JSON string literal

[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1033#comment-1033 ] Apache Spark commented on SPARK-24709: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1032#comment-1032 ] Apache Spark commented on SPARK-25860: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25860: Assignee: (was: Apache Spark) > Replace Literal(null, _) with FalseLiteral whenever

[jira] [Assigned] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25860: Assignee: Apache Spark > Replace Literal(null, _) with FalseLiteral whenever possible >

[jira] [Commented] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1031#comment-1031 ] Apache Spark commented on SPARK-25860: -- User 'aokolnychyi' has created a pull request for this

[jira] [Created] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

Anton Okolnychyi created SPARK-25860: Summary: Replace Literal(null, _) with FalseLiteral whenever possible Key: SPARK-25860 URL: https://issues.apache.org/jira/browse/SPARK-25860 Project: Spark

[jira] [Updated] (SPARK-25259) Left/Right join support push down during-join predicates

[ https://issues.apache.org/jira/browse/SPARK-25259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25259: Description: For example: {code:sql} create temporary view EMPLOYEE as select * from values

[jira] [Updated] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-25859: --- Description: scala/java/python examples and doc for PrefixSpan are added in 3.0 in 

[jira] [Commented] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665973#comment-16665973 ] Huaxin Gao commented on SPARK-25859: [~felixcheung] I had some problems to submit a PR for 

[jira] [Created] (SPARK-25859) add scala/java/python example and doc for PrefixSpan

Huaxin Gao created SPARK-25859: -- Summary: add scala/java/python example and doc for PrefixSpan Key: SPARK-25859 URL: https://issues.apache.org/jira/browse/SPARK-25859 Project: Spark Issue Type: