[jira] [Resolved] (SPARK-24997) Support MINUS ALL

[ https://issues.apache.org/jira/browse/SPARK-24997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24997. - Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.4.0 > Support MINUS ALL >

[jira] [Resolved] (SPARK-24788) RelationalGroupedDataset.toString throws errors when grouping by UnresolvedAttribute

[ https://issues.apache.org/jira/browse/SPARK-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24788. - Resolution: Fixed Assignee: Chris Horn Fix Version/s: 2.4.0 >

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Priority: Blocker (was: Major) > SHS filters wrongly some applications due to permission check >

[jira] [Resolved] (SPARK-25002) Avro: revise the output record namespace

[ https://issues.apache.org/jira/browse/SPARK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25002. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21974

[jira] [Assigned] (SPARK-25002) Avro: revise the output record namespace

[ https://issues.apache.org/jira/browse/SPARK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25002: --- Assignee: Gengliang Wang > Avro: revise the output record namespace >

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Target Version/s: 2.2.3, 2.3.2, 2.4.0 > SHS filters wrongly some applications due to permission

[jira] [Commented] (SPARK-23911) High-order function: reduce(array, initialState S, inputFunction, outputFunction) → R

[ https://issues.apache.org/jira/browse/SPARK-23911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567791#comment-16567791 ] Takuya Ueshin commented on SPARK-23911: --- [~smilegator] [~hvanhovell] I'd use {{aggregate}} instead

[jira] [Created] (SPARK-25011) Add PrefixSpan to __all__

yuhao yang created SPARK-25011: -- Summary: Add PrefixSpan to __all__ Key: SPARK-25011 URL: https://issues.apache.org/jira/browse/SPARK-25011 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-24966) Fix the precedence rule for set operations.

[ https://issues.apache.org/jira/browse/SPARK-24966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24966. - Resolution: Fixed Fix Version/s: 2.4.0 > Fix the precedence rule for set operations. >

[jira] [Assigned] (SPARK-24966) Fix the precedence rule for set operations.

[ https://issues.apache.org/jira/browse/SPARK-24966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24966: --- Assignee: Dilip Biswal > Fix the precedence rule for set operations. >

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567708#comment-16567708 ] Hyukjin Kwon commented on SPARK-24924: -- Similar discussion was made in SPARK-20590 when we port

[jira] [Resolved] (SPARK-24977) input_file_name() result can't save and use for partitionBy()

[ https://issues.apache.org/jira/browse/SPARK-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24977. --- Resolution: Not A Problem This isn't nearly sufficient detail for a JIRA, and evidence it isn't a

[jira] [Updated] (SPARK-24977) input_file_name() result can't save and use for partitionBy()

[ https://issues.apache.org/jira/browse/SPARK-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24977: -- Target Version/s: (was: 2.2.1) > input_file_name() result can't save and use for partitionBy() >

[jira] [Updated] (SPARK-24910) Spark Bloom Filter Closure Serialization improvement for very high volume of Data

[ https://issues.apache.org/jira/browse/SPARK-24910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24910: -- Shepherd: (was: Sean Owen) Flags: (was: Patch) Labels: (was: bloom-filter)

[jira] [Commented] (SPARK-10413) ML models should support prediction on single instances

[ https://issues.apache.org/jira/browse/SPARK-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567683#comment-16567683 ] zhengruifeng commented on SPARK-10413: -- Is there a plan to expose predict in clustering

[jira] [Commented] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567677#comment-16567677 ] Imran Rashid commented on SPARK-25009: -- [~devaraj.k] I don't think SPARK-22941 is in 2.3.1, so I

[jira] [Updated] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25009: - Affects Version/s: (was: 2.3.1) 2.4.0 > Standalone Cluster mode

[jira] [Updated] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25009: - Priority: Critical (was: Blocker) > Standalone Cluster mode application submit is not working

[jira] [Updated] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25009: - Priority: Blocker (was: Major) > Standalone Cluster mode application submit is not working >

[jira] [Assigned] (SPARK-24945) Switch to uniVocity >= 2.7.2

[ https://issues.apache.org/jira/browse/SPARK-24945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24945: Assignee: Maxim Gekk > Switch to uniVocity >= 2.7.2 > > >

[jira] [Resolved] (SPARK-24945) Switch to uniVocity >= 2.7.2

[ https://issues.apache.org/jira/browse/SPARK-24945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24945. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21969

[jira] [Assigned] (SPARK-24773) support reading AVRO logical types - Timestamp with different precisions

[ https://issues.apache.org/jira/browse/SPARK-24773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24773: Assignee: Gengliang Wang > support reading AVRO logical types - Timestamp with different

[jira] [Resolved] (SPARK-24773) support reading AVRO logical types - Timestamp with different precisions

[ https://issues.apache.org/jira/browse/SPARK-24773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24773. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21935

[jira] [Commented] (SPARK-25010) Rand/Randn should produce different values for each execution in streaming query

[ https://issues.apache.org/jira/browse/SPARK-25010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567592#comment-16567592 ] Apache Spark commented on SPARK-25010: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25010) Rand/Randn should produce different values for each execution in streaming query

[ https://issues.apache.org/jira/browse/SPARK-25010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25010: Assignee: (was: Apache Spark) > Rand/Randn should produce different values for each

[jira] [Assigned] (SPARK-25010) Rand/Randn should produce different values for each execution in streaming query

[ https://issues.apache.org/jira/browse/SPARK-25010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25010: Assignee: Apache Spark > Rand/Randn should produce different values for each execution

[jira] [Resolved] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou resolved SPARK-21961. - Resolution: Won't Fix > Filter out BlockStatuses Accumulators during replaying history logs in Spark >

[jira] [Commented] (SPARK-24928) spark sql cross join running time too long

[ https://issues.apache.org/jira/browse/SPARK-24928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567578#comment-16567578 ] Matthew Normyle commented on SPARK-24928: - {color:#cc7832}val {color}largeRDD =

[jira] [Created] (SPARK-25010) Rand/Randn should produce different values for each execution in streaming query

Liang-Chi Hsieh created SPARK-25010: --- Summary: Rand/Randn should produce different values for each execution in streaming query Key: SPARK-25010 URL: https://issues.apache.org/jira/browse/SPARK-25010

[jira] [Resolved] (SPARK-22219) Refector "spark.sql.codegen.comments"

[ https://issues.apache.org/jira/browse/SPARK-22219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22219. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 19449

[jira] [Assigned] (SPARK-22219) Refector "spark.sql.codegen.comments"

[ https://issues.apache.org/jira/browse/SPARK-22219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-22219: - Assignee: Kazuaki Ishizaki > Refector "spark.sql.codegen.comments" >

[jira] [Updated] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Target Version/s: 2.4.0 > Spark scheduler can hang when fetch failures, executor lost, task

[jira] [Resolved] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

[ https://issues.apache.org/jira/browse/SPARK-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-24896. -- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.4.0 > Uuid

[jira] [Updated] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

[ https://issues.apache.org/jira/browse/SPARK-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-24896: - Affects Version/s: (was: 2.4.0) 2.3.0 2.3.1 >

[jira] [Assigned] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24909: Assignee: Apache Spark > Spark scheduler can hang when fetch failures, executor lost,

[jira] [Assigned] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24909: Assignee: (was: Apache Spark) > Spark scheduler can hang when fetch failures,

[jira] [Assigned] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25009: Assignee: (was: Apache Spark) > Standalone Cluster mode application submit is not

[jira] [Assigned] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25009: Assignee: Apache Spark > Standalone Cluster mode application submit is not working >

[jira] [Commented] (SPARK-25009) Standalone Cluster mode application submit is not working

[ https://issues.apache.org/jira/browse/SPARK-25009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567536#comment-16567536 ] Apache Spark commented on SPARK-25009: -- User 'devaraj-kavali' has created a pull request for this

[jira] [Created] (SPARK-25009) Standalone Cluster mode application submit is not working

Devaraj K created SPARK-25009: - Summary: Standalone Cluster mode application submit is not working Key: SPARK-25009 URL: https://issues.apache.org/jira/browse/SPARK-25009 Project: Spark Issue

[jira] [Assigned] (SPARK-25001) Fix build miscellaneous warnings

[ https://issues.apache.org/jira/browse/SPARK-25001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25001: Assignee: (was: Apache Spark) > Fix build miscellaneous warnings >

[jira] [Assigned] (SPARK-25001) Fix build miscellaneous warnings

[ https://issues.apache.org/jira/browse/SPARK-25001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25001: Assignee: Apache Spark > Fix build miscellaneous warnings >

[jira] [Created] (SPARK-25008) Add memory mode info to showMemoryUsage in TaskMemoryManager

Ankur Gupta created SPARK-25008: --- Summary: Add memory mode info to showMemoryUsage in TaskMemoryManager Key: SPARK-25008 URL: https://issues.apache.org/jira/browse/SPARK-25008 Project: Spark

[jira] [Commented] (SPARK-25007) Add transform / array_except /array_union / array_shuffle to SparkR

[ https://issues.apache.org/jira/browse/SPARK-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567521#comment-16567521 ] Huaxin Gao commented on SPARK-25007: I will work on this. Thanks! > Add transform / array_except

[jira] [Created] (SPARK-25007) Add transform / array_except /array_union / array_shuffle to SparkR

Huaxin Gao created SPARK-25007: -- Summary: Add transform / array_except /array_union / array_shuffle to SparkR Key: SPARK-25007 URL: https://issues.apache.org/jira/browse/SPARK-25007 Project: Spark

[jira] [Assigned] (SPARK-25006) Add optional catalog to TableIdentifier

[ https://issues.apache.org/jira/browse/SPARK-25006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25006: Assignee: Apache Spark > Add optional catalog to TableIdentifier >

[jira] [Commented] (SPARK-25006) Add optional catalog to TableIdentifier

[ https://issues.apache.org/jira/browse/SPARK-25006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567513#comment-16567513 ] Apache Spark commented on SPARK-25006: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25006) Add optional catalog to TableIdentifier

[ https://issues.apache.org/jira/browse/SPARK-25006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25006: Assignee: (was: Apache Spark) > Add optional catalog to TableIdentifier >

[jira] [Assigned] (SPARK-23908) High-order function: transform(array, function) → array

[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23908: - Assignee: Takuya Ueshin (was: Herman van Hovell) > High-order function:

[jira] [Created] (SPARK-25006) Add optional catalog to TableIdentifier

Ryan Blue created SPARK-25006: - Summary: Add optional catalog to TableIdentifier Key: SPARK-25006 URL: https://issues.apache.org/jira/browse/SPARK-25006 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

Quentin Ambard created SPARK-25005: -- Summary: Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers) Key: SPARK-25005 URL:

[jira] [Updated] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Ambard updated SPARK-24720: --- Component/s: (was: Structured Streaming) DStreams > kafka transaction

[jira] [Resolved] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24705. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 >

[jira] [Updated] (SPARK-20964) Make some keywords reserved along with the ANSI/SQL standard

[ https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20964: Target Version/s: 3.0.0 > Make some keywords reserved along with the ANSI/SQL standard >

[jira] [Resolved] (SPARK-23908) High-order function: transform(array, function) → array

[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23908. - Resolution: Fixed Fix Version/s: 2.4.0 > High-order function: transform(array, function) → array

[jira] [Assigned] (SPARK-25004) Add spark.executor.pyspark.memory config to set resource.RLIMIT_AS

[ https://issues.apache.org/jira/browse/SPARK-25004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25004: Assignee: (was: Apache Spark) > Add spark.executor.pyspark.memory config to set

[jira] [Commented] (SPARK-25004) Add spark.executor.pyspark.memory config to set resource.RLIMIT_AS

[ https://issues.apache.org/jira/browse/SPARK-25004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567396#comment-16567396 ] Apache Spark commented on SPARK-25004: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25004) Add spark.executor.pyspark.memory config to set resource.RLIMIT_AS

[ https://issues.apache.org/jira/browse/SPARK-25004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25004: Assignee: Apache Spark > Add spark.executor.pyspark.memory config to set

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567393#comment-16567393 ] Mingjie Tang commented on SPARK-24615: -- >From user's perspective, user only concern about the GPU

[jira] [Updated] (SPARK-25004) Add spark.executor.pyspark.memory config to set resource.RLIMIT_AS

[ https://issues.apache.org/jira/browse/SPARK-25004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Blue updated SPARK-25004: -- Description: Some platforms support limiting Python's addressable memory space by limiting

[jira] [Created] (SPARK-25004) Add spark.executor.pyspark.memory config to set resource.RLIMIT_AS

Ryan Blue created SPARK-25004: - Summary: Add spark.executor.pyspark.memory config to set resource.RLIMIT_AS Key: SPARK-25004 URL: https://issues.apache.org/jira/browse/SPARK-25004 Project: Spark

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567383#comment-16567383 ] Reynold Xin commented on SPARK-14220: - This is awesome! Congrats!   > Build and test Spark against

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567367#comment-16567367 ] Stavros Kontopoulos commented on SPARK-24434: - Btw I have started working on a PR on this so

[jira] [Commented] (SPARK-24817) Implement BarrierTaskContext.barrier()

[ https://issues.apache.org/jira/browse/SPARK-24817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567358#comment-16567358 ] Erik Erlandson commented on SPARK-24817: I have been looking at the use cases for barrier-mode

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567335#comment-16567335 ] Stavros Kontopoulos commented on SPARK-24434: - Thanks [~rvesse] I will have a look. >

[jira] [Commented] (SPARK-24817) Implement BarrierTaskContext.barrier()

[ https://issues.apache.org/jira/browse/SPARK-24817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567301#comment-16567301 ] Erik Erlandson commented on SPARK-24817: Thanks [~jiangxb] - I'd expect that design to work

[jira] [Resolved] (SPARK-24988) Add a castBySchema method which casts all the values of a DataFrame based on the DataTypes of a StructType

[ https://issues.apache.org/jira/browse/SPARK-24988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24988. -- Resolution: Won't Fix > Add a castBySchema method which casts all the values of a DataFrame

[jira] [Created] (SPARK-25003) Pyspark Does not use Spark Sql Extensions

Russell Spitzer created SPARK-25003: --- Summary: Pyspark Does not use Spark Sql Extensions Key: SPARK-25003 URL: https://issues.apache.org/jira/browse/SPARK-25003 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25001) Fix build miscellaneous warnings

[ https://issues.apache.org/jira/browse/SPARK-25001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567136#comment-16567136 ] Apache Spark commented on SPARK-25001: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-25002) Avro: revise the output record namespace

[ https://issues.apache.org/jira/browse/SPARK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25002: Assignee: (was: Apache Spark) > Avro: revise the output record namespace >

[jira] [Assigned] (SPARK-25002) Avro: revise the output record namespace

[ https://issues.apache.org/jira/browse/SPARK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25002: Assignee: Apache Spark > Avro: revise the output record namespace >

[jira] [Commented] (SPARK-25002) Avro: revise the output record namespace

[ https://issues.apache.org/jira/browse/SPARK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567134#comment-16567134 ] Apache Spark commented on SPARK-25002: -- User 'gengliangwang' has created a pull request for this

[jira] [Updated] (SPARK-25001) Fix build miscellaneous warnings

[ https://issues.apache.org/jira/browse/SPARK-25001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25001: - Summary: Fix build miscellaneous warnings (was: Handle build warnings in common, core,

[jira] [Updated] (SPARK-25002) Avro: revise the output record namespace

[ https://issues.apache.org/jira/browse/SPARK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-25002: --- Summary: Avro: revise the output record namespace (was: Avro: revise the output namespace)

[jira] [Created] (SPARK-25002) Avro: revise the output namespace

Gengliang Wang created SPARK-25002: -- Summary: Avro: revise the output namespace Key: SPARK-25002 URL: https://issues.apache.org/jira/browse/SPARK-25002 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-25001) Handle build warnings in common, core, launcher, mllib, sql

[ https://issues.apache.org/jira/browse/SPARK-25001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25001: - Summary: Handle build warnings in common, core, launcher, mllib, sql (was: Remove build

[jira] [Created] (SPARK-25001) Remove build warnings in common, core, launcher, mllib, sql

Hyukjin Kwon created SPARK-25001: Summary: Remove build warnings in common, core, launcher, mllib, sql Key: SPARK-25001 URL: https://issues.apache.org/jira/browse/SPARK-25001 Project: Spark

[jira] [Updated] (SPARK-24940) Coalesce Hint for SQL Queries

[ https://issues.apache.org/jira/browse/SPARK-24940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24940: Target Version/s: 2.4.0 > Coalesce Hint for SQL Queries > - > >

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567062#comment-16567062 ] Kildiev Rustam commented on SPARK-14220: Hurra > Build and test Spark against Scala

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567045#comment-16567045 ] Thomas Graves commented on SPARK-24924: --- why are we doing this? If a user ships the spark-avro

[jira] [Resolved] (SPARK-24821) Fail fast when submitted job compute on a subset of all the partitions for a barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-24821. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21927

[jira] [Assigned] (SPARK-24821) Fail fast when submitted job compute on a subset of all the partitions for a barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24821: - Assignee: Jiang Xingbo > Fail fast when submitted job compute on a subset of all the

[jira] [Assigned] (SPARK-24820) Fail fast when submitted job contains PartitionPruningRDD in a barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24820: - Assignee: Jiang Xingbo > Fail fast when submitted job contains PartitionPruningRDD in

[jira] [Resolved] (SPARK-24820) Fail fast when submitted job contains PartitionPruningRDD in a barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-24820. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21927

[jira] [Resolved] (SPARK-24598) SPARK SQL:Datatype overflow conditions gives incorrect result

[ https://issues.apache.org/jira/browse/SPARK-24598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24598. - Resolution: Done Assignee: Marco Gaido Fix Version/s: 2.4.0 > SPARK SQL:Datatype

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567017#comment-16567017 ] Rob Vesse commented on SPARK-24434: --- [~skonto] Added a couple more comments based on some issues I've

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567015#comment-16567015 ] Sean Owen commented on SPARK-18057: --- [~zsxwing] hm, it seems weird that Spark is then using two

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567010#comment-16567010 ] Shixiong Zhu commented on SPARK-18057: -- [~srowen] Could you create a new ticket for DStreams Kafka?

[jira] [Comment Edited] (SPARK-24826) Self-Join not working in Apache Spark 2.2.2

[ https://issues.apache.org/jira/browse/SPARK-24826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566981#comment-16566981 ] Joseph Fourny edited comment on SPARK-24826 at 8/2/18 3:57 PM: --- I was able

[jira] [Commented] (SPARK-24826) Self-Join not working in Apache Spark 2.2.2

[ https://issues.apache.org/jira/browse/SPARK-24826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566981#comment-16566981 ] Joseph Fourny commented on SPARK-24826: --- I was able to reproduce this defect with an inner-join of

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566954#comment-16566954 ] Jackey Lee commented on SPARK-24630: [~uncleGen] Are you willing to assist in the code review? I can

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566948#comment-16566948 ] Simeon H.K. Fitch commented on SPARK-14540: --- Congratulations! A long, difficult haul... Cheers

[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds

[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566949#comment-16566949 ] shane knapp commented on SPARK-24980: - looking pretty good (you have to go to the very bottom of the

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566945#comment-16566945 ] Simeon H.K. Fitch commented on SPARK-14220: ---

[jira] [Issue Comment Deleted] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon H.K. Fitch updated SPARK-14220: -- Comment: was deleted (was: (flag)(*)(*r)(*g)(*b)(*y):D(*y)(*b)(*g)(*r)(*)(flag) Way

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566942#comment-16566942 ] Simeon H.K. Fitch commented on SPARK-14220: ---

[jira] [Commented] (SPARK-24795) Implement barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-24795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566935#comment-16566935 ] Apache Spark commented on SPARK-24795: -- User 'jiangxb1987' has created a pull request for this

[jira] [Assigned] (SPARK-24947) aggregateAsync and foldAsync for RDD

[ https://issues.apache.org/jira/browse/SPARK-24947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24947: Assignee: (was: Apache Spark) > aggregateAsync and foldAsync for RDD >

[jira] [Assigned] (SPARK-24947) aggregateAsync and foldAsync for RDD

[ https://issues.apache.org/jira/browse/SPARK-24947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24947: Assignee: Apache Spark > aggregateAsync and foldAsync for RDD >

[jira] [Commented] (SPARK-24947) aggregateAsync and foldAsync for RDD

[ https://issues.apache.org/jira/browse/SPARK-24947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566867#comment-16566867 ] Apache Spark commented on SPARK-24947: -- User 'ceedubs' has created a pull request for this issue:

  1   2   >