[jira] [Commented] (SPARK-14503) spark.ml Scala API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906122#comment-15906122 ] yuhao yang commented on SPARK-14503: Thanks for reporting that. I just found there's a misplaced

[jira] [Commented] (SPARK-19914) Spark Scala - Calling persist after reading a parquet file makes certain spark.sql queries return empty results

[ https://issues.apache.org/jira/browse/SPARK-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906121#comment-15906121 ] Takeshi Yamamuro commented on SPARK-19914: -- I couldn't reproduce this and do I miss anything?

[jira] [Comment Edited] (SPARK-19914) Spark Scala - Calling persist after reading a parquet file makes certain spark.sql queries return empty results

[ https://issues.apache.org/jira/browse/SPARK-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906121#comment-15906121 ] Takeshi Yamamuro edited comment on SPARK-19914 at 3/11/17 7:21 AM: --- I

[jira] [Assigned] (SPARK-19919) Defer input path validation into DataSource in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-19919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19919: Assignee: Apache Spark > Defer input path validation into DataSource in CSV datasource >

[jira] [Assigned] (SPARK-19919) Defer input path validation into DataSource in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-19919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19919: Assignee: (was: Apache Spark) > Defer input path validation into DataSource in CSV

[jira] [Commented] (SPARK-19919) Defer input path validation into DataSource in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-19919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906115#comment-15906115 ] Apache Spark commented on SPARK-19919: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-19919) Defer input path validation into DataSource in CSV datasource

Hyukjin Kwon created SPARK-19919: Summary: Defer input path validation into DataSource in CSV datasource Key: SPARK-19919 URL: https://issues.apache.org/jira/browse/SPARK-19919 Project: Spark

[jira] [Assigned] (SPARK-19918) Use TextFileFormat in implementation of JsonFileFormat

[ https://issues.apache.org/jira/browse/SPARK-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19918: Assignee: Apache Spark > Use TextFileFormat in implementation of JsonFileFormat >

[jira] [Assigned] (SPARK-19918) Use TextFileFormat in implementation of JsonFileFormat

[ https://issues.apache.org/jira/browse/SPARK-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19918: Assignee: (was: Apache Spark) > Use TextFileFormat in implementation of

[jira] [Commented] (SPARK-19918) Use TextFileFormat in implementation of JsonFileFormat

[ https://issues.apache.org/jira/browse/SPARK-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906098#comment-15906098 ] Apache Spark commented on SPARK-19918: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-19918) Use TextFileFormat in implementation of JsonFileFormat

Hyukjin Kwon created SPARK-19918: Summary: Use TextFileFormat in implementation of JsonFileFormat Key: SPARK-19918 URL: https://issues.apache.org/jira/browse/SPARK-19918 Project: Spark Issue

[jira] [Resolved] (SPARK-19901) Clean up the clunky method signature of acquireMemory

[ https://issues.apache.org/jira/browse/SPARK-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19901. --- Resolution: Not A Problem > Clean up the clunky method signature of acquireMemory >

[jira] [Assigned] (SPARK-19723) create table for data source tables should work with an non-existent location

[ https://issues.apache.org/jira/browse/SPARK-19723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19723: --- Assignee: Song Jun > create table for data source tables should work with an non-existent

[jira] [Resolved] (SPARK-19723) create table for data source tables should work with an non-existent location

[ https://issues.apache.org/jira/browse/SPARK-19723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19723. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17055

[jira] [Assigned] (SPARK-19917) qualified partition location stored in catalog

[ https://issues.apache.org/jira/browse/SPARK-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19917: Assignee: Apache Spark > qualified partition location stored in catalog >

[jira] [Assigned] (SPARK-19917) qualified partition location stored in catalog

[ https://issues.apache.org/jira/browse/SPARK-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19917: Assignee: (was: Apache Spark) > qualified partition location stored in catalog >

[jira] [Commented] (SPARK-19917) qualified partition location stored in catalog

[ https://issues.apache.org/jira/browse/SPARK-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906072#comment-15906072 ] Apache Spark commented on SPARK-19917: -- User 'windpiger' has created a pull request for this issue:

[jira] [Created] (SPARK-19917) qualified partition location stored in catalog

Song Jun created SPARK-19917: Summary: qualified partition location stored in catalog Key: SPARK-19917 URL: https://issues.apache.org/jira/browse/SPARK-19917 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-19916) simplify bad file handling

[ https://issues.apache.org/jira/browse/SPARK-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19916: Assignee: Apache Spark (was: Wenchen Fan) > simplify bad file handling >

[jira] [Assigned] (SPARK-19916) simplify bad file handling

[ https://issues.apache.org/jira/browse/SPARK-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19916: Assignee: Wenchen Fan (was: Apache Spark) > simplify bad file handling >

[jira] [Commented] (SPARK-19916) simplify bad file handling

[ https://issues.apache.org/jira/browse/SPARK-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906048#comment-15906048 ] Apache Spark commented on SPARK-19916: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-19916) simplify bad file handling

Wenchen Fan created SPARK-19916: --- Summary: simplify bad file handling Key: SPARK-19916 URL: https://issues.apache.org/jira/browse/SPARK-19916 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-6634) Allow replacing columns in Transformers

[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904445#comment-15904445 ] Tree Field edited comment on SPARK-6634 at 3/11/17 2:59 AM: I want this

[jira] [Assigned] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19915: Assignee: Apache Spark > Improve join reorder: simplify cost evaluation, postpone column

[jira] [Assigned] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19915: Assignee: (was: Apache Spark) > Improve join reorder: simplify cost evaluation,

[jira] [Updated] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-19915: - Description: 1. Usually cardinality is more important than size, we can simplify cost

[jira] [Commented] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

[ https://issues.apache.org/jira/browse/SPARK-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906018#comment-15906018 ] Apache Spark commented on SPARK-19915: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Created] (SPARK-19915) Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product

Zhenhua Wang created SPARK-19915: Summary: Improve join reorder: simplify cost evaluation, postpone column pruning, exclude cartesian product Key: SPARK-19915 URL:

[jira] [Commented] (SPARK-19863) Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application

[ https://issues.apache.org/jira/browse/SPARK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905991#comment-15905991 ] LvDongrong commented on SPARK-19863: I see your comment on that issue(SPARK-19185), and I am agree

[jira] [Created] (SPARK-19914) Spark Scala - Calling persist after reading a parquet file makes certain spark.sql queries return empty results

Yifeng Li created SPARK-19914: - Summary: Spark Scala - Calling persist after reading a parquet file makes certain spark.sql queries return empty results Key: SPARK-19914 URL:

[jira] [Updated] (SPARK-19611) Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files

[ https://issues.apache.org/jira/browse/SPARK-19611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19611: Fix Version/s: 2.1.1 > Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files >

[jira] [Resolved] (SPARK-19893) should not run DataFrame set oprations with map type

[ https://issues.apache.org/jira/browse/SPARK-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19893. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 2.0.3

[jira] [Assigned] (SPARK-19913) Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach

[ https://issues.apache.org/jira/browse/SPARK-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19913: Assignee: (was: Apache Spark) > Log warning rather than throw AnalysisException when

[jira] [Assigned] (SPARK-19913) Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach

[ https://issues.apache.org/jira/browse/SPARK-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19913: Assignee: Apache Spark > Log warning rather than throw AnalysisException when output is

[jira] [Commented] (SPARK-19913) Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach

[ https://issues.apache.org/jira/browse/SPARK-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905904#comment-15905904 ] Apache Spark commented on SPARK-19913: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-19913) Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach

Kousuke Saruta created SPARK-19913: -- Summary: Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach Key: SPARK-19913 URL:

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905886#comment-15905886 ] Shixiong Zhu commented on SPARK-18057: -- > Based on previous kafka client upgrades I wouldn't expect

[jira] [Updated] (SPARK-19912) String literals are not escaped while performing Hive metastore level partition pruning

[ https://issues.apache.org/jira/browse/SPARK-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19912: --- Summary: String literals are not escaped while performing Hive metastore level partition pruning

[jira] [Resolved] (SPARK-19905) Dataset.inputFiles is broken for Hive SerDe tables

[ https://issues.apache.org/jira/browse/SPARK-19905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19905. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17247

[jira] [Updated] (SPARK-19912) String literals are not escaped while performing partition pruning at Hive metastore level

[ https://issues.apache.org/jira/browse/SPARK-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19912: --- Description: {{Shim_v0_13.convertFilters()}} doesn't escape string literals while generating Hive

[jira] [Updated] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19887: --- Labels: correctness (was: ) > __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value

[jira] [Created] (SPARK-19912) String literals are not escaped while performing partition pruning at Hive metastore level

Cheng Lian created SPARK-19912: -- Summary: String literals are not escaped while performing partition pruning at Hive metastore level Key: SPARK-19912 URL: https://issues.apache.org/jira/browse/SPARK-19912

[jira] [Commented] (SPARK-14503) spark.ml Scala API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905840#comment-15905840 ] Maciej Szymkiewicz commented on SPARK-14503: I think we should keep only unique predictions

[jira] [Commented] (SPARK-19899) FPGrowth input column naming

[ https://issues.apache.org/jira/browse/SPARK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905834#comment-15905834 ] Maciej Szymkiewicz commented on SPARK-19899: Thanks [~yuhaoyan]. > FPGrowth input column

[jira] [Updated] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19887: --- Affects Version/s: 2.2.0 > __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905774#comment-15905774 ] Cody Koeninger commented on SPARK-18057: Based on previous kafka client upgrades I wouldn't

[jira] [Assigned] (SPARK-19910) `stack` should not reject NULL values due to type mismatch

[ https://issues.apache.org/jira/browse/SPARK-19910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19910: Assignee: (was: Apache Spark) > `stack` should not reject NULL values due to type

[jira] [Assigned] (SPARK-19910) `stack` should not reject NULL values due to type mismatch

[ https://issues.apache.org/jira/browse/SPARK-19910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19910: Assignee: Apache Spark > `stack` should not reject NULL values due to type mismatch >

[jira] [Commented] (SPARK-19910) `stack` should not reject NULL values due to type mismatch

[ https://issues.apache.org/jira/browse/SPARK-19910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905766#comment-15905766 ] Apache Spark commented on SPARK-19910: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905755#comment-15905755 ] Michael Armbrust commented on SPARK-18057: -- It seems like we can upgrade the existing Kafka10

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905747#comment-15905747 ] Cody Koeninger commented on SPARK-18057: I think the bigger question is once there's a kafka

[jira] [Assigned] (SPARK-19911) Add builder interface for Kinesis DStreams

[ https://issues.apache.org/jira/browse/SPARK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19911: Assignee: Apache Spark > Add builder interface for Kinesis DStreams >

[jira] [Assigned] (SPARK-19911) Add builder interface for Kinesis DStreams

[ https://issues.apache.org/jira/browse/SPARK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19911: Assignee: (was: Apache Spark) > Add builder interface for Kinesis DStreams >

[jira] [Commented] (SPARK-19911) Add builder interface for Kinesis DStreams

[ https://issues.apache.org/jira/browse/SPARK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905741#comment-15905741 ] Apache Spark commented on SPARK-19911: -- User 'budde' has created a pull request for this issue:

[jira] [Resolved] (SPARK-17979) Remove deprecated support for config SPARK_YARN_USER_ENV

[ https://issues.apache.org/jira/browse/SPARK-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17979. Resolution: Fixed Assignee: Yong Tang Fix Version/s: 2.2.0 > Remove

[jira] [Resolved] (SPARK-14453) Remove SPARK_JAVA_OPTS environment variable

[ https://issues.apache.org/jira/browse/SPARK-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-14453. Resolution: Fixed Assignee: Yong Tang Fix Version/s: 2.2.0 > Remove

[jira] [Commented] (SPARK-19888) Seeing offsets not resetting even when reset policy is configured explicitly

[ https://issues.apache.org/jira/browse/SPARK-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905730#comment-15905730 ] Cody Koeninger commented on SPARK-19888: That stacktrace also shows a concurrent modification

[jira] [Created] (SPARK-19911) Add builder interface for Kinesis DStreams

Adam Budde created SPARK-19911: -- Summary: Add builder interface for Kinesis DStreams Key: SPARK-19911 URL: https://issues.apache.org/jira/browse/SPARK-19911 Project: Spark Issue Type: New

[jira] [Comment Edited] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905716#comment-15905716 ] Shixiong Zhu edited comment on SPARK-18057 at 3/10/17 9:21 PM: --- I did some

[jira] [Comment Edited] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905716#comment-15905716 ] Shixiong Zhu edited comment on SPARK-18057 at 3/10/17 9:21 PM: --- I did some

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905716#comment-15905716 ] Shixiong Zhu commented on SPARK-18057: -- I did some investigation yesterday, and found one issue in

[jira] [Commented] (SPARK-19611) Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files

[ https://issues.apache.org/jira/browse/SPARK-19611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905705#comment-15905705 ] Apache Spark commented on SPARK-19611: -- User 'budde' has created a pull request for this issue:

[jira] [Created] (SPARK-19910) `stack` should not reject NULL values due to type mismatch

Dongjoon Hyun created SPARK-19910: - Summary: `stack` should not reject NULL values due to type mismatch Key: SPARK-19910 URL: https://issues.apache.org/jira/browse/SPARK-19910 Project: Spark

[jira] [Updated] (SPARK-19888) Seeing offsets not resetting even when reset policy is configured explicitly

[ https://issues.apache.org/jira/browse/SPARK-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-19888: - Component/s: (was: Spark Core) DStreams > Seeing offsets not

[jira] [Commented] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905641#comment-15905641 ] Apache Spark commented on SPARK-19909: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19909: Assignee: (was: Apache Spark) > Batches will fail in case that temporary checkpoint

[jira] [Assigned] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19909: Assignee: Apache Spark > Batches will fail in case that temporary checkpoint dir is on

[jira] [Created] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

Kousuke Saruta created SPARK-19909: -- Summary: Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS Key: SPARK-19909 URL:

[jira] [Assigned] (SPARK-19905) Dataset.inputFiles is broken for Hive SerDe tables

[ https://issues.apache.org/jira/browse/SPARK-19905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19905: Assignee: Cheng Lian (was: Apache Spark) > Dataset.inputFiles is broken for Hive SerDe

[jira] [Assigned] (SPARK-19905) Dataset.inputFiles is broken for Hive SerDe tables

[ https://issues.apache.org/jira/browse/SPARK-19905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19905: Assignee: Apache Spark (was: Cheng Lian) > Dataset.inputFiles is broken for Hive SerDe

[jira] [Commented] (SPARK-19905) Dataset.inputFiles is broken for Hive SerDe tables

[ https://issues.apache.org/jira/browse/SPARK-19905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905608#comment-15905608 ] Apache Spark commented on SPARK-19905: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-19863) Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application

[ https://issues.apache.org/jira/browse/SPARK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905603#comment-15905603 ] Cody Koeninger commented on SPARK-19863: Isn't this basically a duplicate of SPARK-19185 with the

[jira] [Created] (SPARK-19908) Direct buffer memory OOM should not cause stage retries.

Zhan Zhang created SPARK-19908: -- Summary: Direct buffer memory OOM should not cause stage retries. Key: SPARK-19908 URL: https://issues.apache.org/jira/browse/SPARK-19908 Project: Spark Issue

[jira] [Updated] (SPARK-19904) SPIP Add Spark Project Improvement Proposal doc to website

[ https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-19904: --- Description: see

[jira] [Closed] (SPARK-19907) Spark Submit Does not pick up the HBase Jars

[ https://issues.apache.org/jira/browse/SPARK-19907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-19907. - > Spark Submit Does not pick up the HBase Jars > > >

[jira] [Resolved] (SPARK-19907) Spark Submit Does not pick up the HBase Jars

[ https://issues.apache.org/jira/browse/SPARK-19907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19907. --- Resolution: Invalid Target Version/s: (was: 2.0.0) A huge dump of your config and logs

[jira] [Assigned] (SPARK-19906) Add Documentation for Kafka Write paths

[ https://issues.apache.org/jira/browse/SPARK-19906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19906: Assignee: Apache Spark > Add Documentation for Kafka Write paths >

[jira] [Created] (SPARK-19907) Spark Submit Does not pick up the HBase Jars

Ramchandhar Rapolu created SPARK-19907: -- Summary: Spark Submit Does not pick up the HBase Jars Key: SPARK-19907 URL: https://issues.apache.org/jira/browse/SPARK-19907 Project: Spark

[jira] [Assigned] (SPARK-19906) Add Documentation for Kafka Write paths

[ https://issues.apache.org/jira/browse/SPARK-19906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19906: Assignee: (was: Apache Spark) > Add Documentation for Kafka Write paths >

[jira] [Commented] (SPARK-19906) Add Documentation for Kafka Write paths

[ https://issues.apache.org/jira/browse/SPARK-19906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905580#comment-15905580 ] Apache Spark commented on SPARK-19906: -- User 'tcondie' has created a pull request for this issue:

[jira] [Created] (SPARK-19906) Add Documentation for Kafka Write paths

Tyson Condie created SPARK-19906: Summary: Add Documentation for Kafka Write paths Key: SPARK-19906 URL: https://issues.apache.org/jira/browse/SPARK-19906 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-19620) Incorrect exchange coordinator Id in physical plan

[ https://issues.apache.org/jira/browse/SPARK-19620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-19620: Assignee: Carson Wang > Incorrect exchange coordinator Id in physical plan >

[jira] [Resolved] (SPARK-19620) Incorrect exchange coordinator Id in physical plan

[ https://issues.apache.org/jira/browse/SPARK-19620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-19620. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16952

[jira] [Resolved] (SPARK-19885) The config ignoreCorruptFiles doesn't work for CSV

[ https://issues.apache.org/jira/browse/SPARK-19885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19885. - Resolution: Fixed Fix Version/s: 2.2.0 > The config ignoreCorruptFiles doesn't work for

[jira] [Created] (SPARK-19905) Dataset.inputFiles is broken for Hive SerDe tables

Cheng Lian created SPARK-19905: -- Summary: Dataset.inputFiles is broken for Hive SerDe tables Key: SPARK-19905 URL: https://issues.apache.org/jira/browse/SPARK-19905 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19885) The config ignoreCorruptFiles doesn't work for CSV

[ https://issues.apache.org/jira/browse/SPARK-19885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905564#comment-15905564 ] Wenchen Fan commented on SPARK-19885: - Oh, so this issue is already fixed by SPARK-18362 in Spark 2.2

[jira] [Updated] (SPARK-19893) should not run DataFrame set oprations with map type

[ https://issues.apache.org/jira/browse/SPARK-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19893: Summary: should not run DataFrame set oprations with map type (was: Cannot run

[jira] [Commented] (SPARK-14453) Remove SPARK_JAVA_OPTS environment variable

[ https://issues.apache.org/jira/browse/SPARK-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905518#comment-15905518 ] Apache Spark commented on SPARK-14453: -- User 'yongtang' has created a pull request for this issue:

[jira] [Updated] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19887: --- Description: The following Spark shell snippet under Spark 2.1 reproduces this issue: {code} val

[jira] [Commented] (SPARK-19899) FPGrowth input column naming

[ https://issues.apache.org/jira/browse/SPARK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905492#comment-15905492 ] yuhao yang commented on SPARK-19899: also cc [~podongfeng] since I recalled he mentioned to use

[jira] [Commented] (SPARK-19899) FPGrowth input column naming

[ https://issues.apache.org/jira/browse/SPARK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905484#comment-15905484 ] yuhao yang commented on SPARK-19899: Thanks for the reply. We can wait for some time to see if people

[jira] [Updated] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19887: --- Description: The following Spark shell snippet under Spark 2.1 reproduces this issue: {code} val

[jira] [Commented] (SPARK-19899) FPGrowth input column naming

[ https://issues.apache.org/jira/browse/SPARK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905480#comment-15905480 ] Maciej Szymkiewicz commented on SPARK-19899: This is just an idea, but I would start with: -

[jira] [Commented] (SPARK-19899) FPGrowth input column naming

[ https://issues.apache.org/jira/browse/SPARK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905451#comment-15905451 ] yuhao yang commented on SPARK-19899: {quote} if we mix-in HasFeaturesCol the featuresCol should be

[jira] [Created] (SPARK-19904) SPIP Add Spark Project Improvement Proposal doc to website

Cody Koeninger created SPARK-19904: -- Summary: SPIP Add Spark Project Improvement Proposal doc to website Key: SPARK-19904 URL: https://issues.apache.org/jira/browse/SPARK-19904 Project: Spark

[jira] [Resolved] (SPARK-19786) Facilitate loop optimizations in a JIT compiler regarding range()

[ https://issues.apache.org/jira/browse/SPARK-19786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19786. --- Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.2.0 >

[jira] [Assigned] (SPARK-19850) Support aliased expressions in function parameters

[ https://issues.apache.org/jira/browse/SPARK-19850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19850: Assignee: Herman van Hovell (was: Apache Spark) > Support aliased expressions in

[jira] [Commented] (SPARK-19899) FPGrowth input column naming

[ https://issues.apache.org/jira/browse/SPARK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905405#comment-15905405 ] Maciej Szymkiewicz commented on SPARK-19899: In my opinion a trait for each input category

[jira] [Assigned] (SPARK-19850) Support aliased expressions in function parameters

[ https://issues.apache.org/jira/browse/SPARK-19850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19850: Assignee: Apache Spark (was: Herman van Hovell) > Support aliased expressions in

[jira] [Commented] (SPARK-19850) Support aliased expressions in function parameters

[ https://issues.apache.org/jira/browse/SPARK-19850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905404#comment-15905404 ] Apache Spark commented on SPARK-19850: -- User 'hvanhovell' has created a pull request for this issue:

  1   2   >