[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523284#comment-16523284 ] Sivakumar commented on SPARK-24631: --- The table that I queried was a view. That view wa

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Description: An end-user SQL client tool (ie in the screenshot) can list tables from hiveserver2 and major DBs

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Description: An end-user SQL client tool (ie in the screenshot) can list tables from hiveserver2 and major DBs

[jira] [Comment Edited] (SPARK-24382) Spark Structured Streaming aggregation on old timestamp data

[ https://issues.apache.org/jira/browse/SPARK-24382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523227#comment-16523227 ] Richard Yu edited comment on SPARK-24382 at 6/26/18 5:34 AM: -

[jira] [Commented] (SPARK-24382) Spark Structured Streaming aggregation on old timestamp data

[ https://issues.apache.org/jira/browse/SPARK-24382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523227#comment-16523227 ] Richard Yu commented on SPARK-24382: [~karthikus] Yes, the old dates would in fact m

[jira] [Assigned] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24658: Assignee: (was: Apache Spark) > Remove workaround for ANTLR bug > ---

[jira] [Assigned] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24658: Assignee: Apache Spark > Remove workaround for ANTLR bug > --

[jira] [Commented] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523223#comment-16523223 ] Apache Spark commented on SPARK-24658: -- User 'wangyum' has created a pull request f

[jira] [Created] (SPARK-24658) Remove workaround for ANTLR bug

Yuming Wang created SPARK-24658: --- Summary: Remove workaround for ANTLR bug Key: SPARK-24658 URL: https://issues.apache.org/jira/browse/SPARK-24658 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24657) SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleanup resource when finished the merge join

[ https://issues.apache.org/jira/browse/SPARK-24657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-24657: - Description: In my sql, It join three tables, and all these tables are small table (about 2mb).

[jira] [Updated] (SPARK-24657) SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleanup resource when finished the merge join

[ https://issues.apache.org/jira/browse/SPARK-24657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-24657: - Description: In my sql, It join three tables, and all these tables are small table (about 2mb).

[jira] [Created] (SPARK-24657) SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleaning resource when finished the merge join

Joshuawangzj created SPARK-24657: Summary: SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleaning resource when finished the merge join Key: SPARK-24657 URL: https://issues.apache.o

[jira] [Issue Comment Deleted] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish updated SPARK-18681: --- Comment: was deleted (was: [~michael] I see the same issue in 2.3.1. I have hive partitioned table with int

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523064#comment-16523064 ] Li Yuanjian commented on SPARK-24630: - cc [~zsxwing] and [~tdas] We have some pract

[jira] [Resolved] (SPARK-24636) Type Coercion of Arrays for array_join Function

[ https://issues.apache.org/jira/browse/SPARK-24636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24636. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21620 [https://gi

[jira] [Assigned] (SPARK-24636) Type Coercion of Arrays for array_join Function

[ https://issues.apache.org/jira/browse/SPARK-24636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24636: Assignee: Marek Novotny > Type Coercion of Arrays for array_join Function > -

[jira] [Resolved] (SPARK-24418) Upgrade to Scala 2.11.12

[ https://issues.apache.org/jira/browse/SPARK-24418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24418. - Resolution: Fixed Issue resolved by pull request 21495 [https://github.com/apache/spark/pull/214

[jira] [Assigned] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23776: Assignee: Bruce Robbins > pyspark-sql tests should display build instructions when compon

[jira] [Resolved] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23776. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21628 [https://gi

[jira] [Commented] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523049#comment-16523049 ] Harish commented on SPARK-18681: [~michael] I see the same issue in 2.3.1. I have hive p

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523023#comment-16523023 ] vaquar khan commented on SPARK-24631: - Why pull request? Please close jira no issue

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523022#comment-16523022 ] Saisai Shao commented on SPARK-24646: - Hi [~vanzin], here is a specific example: We

[jira] [Resolved] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24552. Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.4.0

[jira] [Resolved] (SPARK-24652) Strange ALS Implementation for Implicit Feedback

[ https://issues.apache.org/jira/browse/SPARK-24652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam resolved SPARK-24652. --- Resolution: Not A Problem > Strange ALS Implementation for Implicit Feedback > -

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522946#comment-16522946 ] Marcelo Vanzin commented on SPARK-24646: I'm not sure I understand what is the p

[jira] [Commented] (SPARK-23858) Need to apply pyarrow adjustments to complex types with DateType/TimestampType

[ https://issues.apache.org/jira/browse/SPARK-23858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522916#comment-16522916 ] Bryan Cutler commented on SPARK-23858: -- [~semanticbeeng] sorry, there aren't failin

[jira] [Commented] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522877#comment-16522877 ] Apache Spark commented on SPARK-24654: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24654: Assignee: Sean Owen (was: Apache Spark) > Update, fix LICENSE and NOTICE, and specialize

[jira] [Assigned] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24654: Assignee: Apache Spark (was: Sean Owen) > Update, fix LICENSE and NOTICE, and specialize

[jira] [Created] (SPARK-24656) SparkML Transformers and Estimators with multiple columns

Michael Dreibelbis created SPARK-24656: -- Summary: SparkML Transformers and Estimators with multiple columns Key: SPARK-24656 URL: https://issues.apache.org/jira/browse/SPARK-24656 Project: Spark

[jira] [Created] (SPARK-24655) [K8S] Custom Docker Image Expectations and Documentation

Matt Cheah created SPARK-24655: -- Summary: [K8S] Custom Docker Image Expectations and Documentation Key: SPARK-24655 URL: https://issues.apache.org/jira/browse/SPARK-24655 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24631: Assignee: Apache Spark > Cannot up cast column from bigint to smallint as it may truncate

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24631: Assignee: (was: Apache Spark) > Cannot up cast column from bigint to smallint as it m

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522871#comment-16522871 ] Apache Spark commented on SPARK-24631: -- User 'vanzin' has created a pull request fo

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522869#comment-16522869 ] Bryan Cutler commented on SPARK-24579: -- I left some comments on the shared doc, ove

[jira] [Resolved] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-24648. --- Resolution: Fixed Assignee: Stacy Kerkela Fix Version/s: 2.4.0 > SQL

[jira] [Created] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

Sean Owen created SPARK-24654: - Summary: Update, fix LICENSE and NOTICE, and specialize for source vs binary Key: SPARK-24654 URL: https://issues.apache.org/jira/browse/SPARK-24654 Project: Spark

[jira] [Created] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

Marcelo Vanzin created SPARK-24653: -- Summary: Flaky test "JoinSuite.test SortMergeJoin (with spill)" Key: SPARK-24653 URL: https://issues.apache.org/jira/browse/SPARK-24653 Project: Spark Is

[jira] [Updated] (SPARK-24651) Add ability to write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Summary: Add ability to write null values while writing JSON (was: Write null values while writ

[jira] [Updated] (SPARK-24651) Write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Description: Hello,  Spark is configured to ignore the null values when writing JSON based off

[jira] [Created] (SPARK-24652) Strange ALS Implementation for Implicit Feedback

Jerry Lam created SPARK-24652: - Summary: Strange ALS Implementation for Implicit Feedback Key: SPARK-24652 URL: https://issues.apache.org/jira/browse/SPARK-24652 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24651) Write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Summary: Write null values while writing JSON (was: Write null values when writing JSON) > Wri

[jira] [Updated] (SPARK-24651) Write null values when writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Description: Hello,  Spark is configured to ignore the null values when writing JSON based off

[jira] [Created] (SPARK-24651) Write null values when writing JSON

Matthew Liem created SPARK-24651: Summary: Write null values when writing JSON Key: SPARK-24651 URL: https://issues.apache.org/jira/browse/SPARK-24651 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22357: Assignee: (was: Apache Spark) > SparkContext.binaryFiles ignore minPartitions paramet

[jira] [Commented] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522754#comment-16522754 ] Apache Spark commented on SPARK-22357: -- User 'bomeng' has created a pull request fo

[jira] [Assigned] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22357: Assignee: Apache Spark > SparkContext.binaryFiles ignore minPartitions parameter > --

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522661#comment-16522661 ] Xiangrui Meng commented on SPARK-24530: --- [~dongjoon] [~hyukjin.kwon] Could you rep

[jira] [Commented] (SPARK-24324) Pandas Grouped Map UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522617#comment-16522617 ] Apache Spark commented on SPARK-24324: -- User 'BryanCutler' has created a pull reque

[jira] [Created] (SPARK-24650) GroupingSet

Mihir Sahu created SPARK-24650: -- Summary: GroupingSet Key: SPARK-24650 URL: https://issues.apache.org/jira/browse/SPARK-24650 Project: Spark Issue Type: Improvement Components: SQL

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Target Version/s: 2.4.0 > Sink Should Return OffsetSeqs For ProgressReporting >

[jira] [Commented] (SPARK-21478) Unpersist a DF also unpersists related DFs

[ https://issues.apache.org/jira/browse/SPARK-21478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522563#comment-16522563 ] Xiao Li commented on SPARK-21478: - [~roberto.mirizzi] The issue has been resolved by  ht

[jira] [Commented] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522562#comment-16522562 ] Xiao Li commented on SPARK-19765: - The reported issue has been resolved by  https://issu

[jira] [Resolved] (SPARK-21607) Can dropTempView function add a param like dropTempView(viewName: String, dropSelfOnly: Boolean)

[ https://issues.apache.org/jira/browse/SPARK-21607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21607. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 2.4.0 > Can dropTempView function

[jira] [Commented] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522557#comment-16522557 ] Joseph K. Bradley commented on SPARK-24632: --- CC [~yanboliang], [~holden.ka...@

[jira] [Commented] (SPARK-21607) Can dropTempView function add a param like dropTempView(viewName: String, dropSelfOnly: Boolean)

[ https://issues.apache.org/jira/browse/SPARK-21607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522559#comment-16522559 ] Xiao Li commented on SPARK-21607: - This is resolved by https://issues.apache.org/jira/br

[jira] [Assigned] (SPARK-24596) Non-cascading Cache Invalidation

[ https://issues.apache.org/jira/browse/SPARK-24596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24596: --- Assignee: Maryann Xue > Non-cascading Cache Invalidation > > >

[jira] [Resolved] (SPARK-24596) Non-cascading Cache Invalidation

[ https://issues.apache.org/jira/browse/SPARK-24596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24596. - Resolution: Fixed Target Version/s: 2.4.0 > Non-cascading Cache Invalidation > ---

[jira] [Updated] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24632: -- Description: This is a follow-up for [SPARK-17025], which allowed users to implement P

[jira] [Updated] (SPARK-24649) SparkUDF.unapply is not backwards compatable

[ https://issues.apache.org/jira/browse/SPARK-24649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon H.K. Fitch updated SPARK-24649: -- Priority: Minor (was: Major) > SparkUDF.unapply is not backwards compatable > ---

[jira] [Assigned] (SPARK-24633) arrays_zip function's code generator splits input processing incorrectly

[ https://issues.apache.org/jira/browse/SPARK-24633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24633: --- Assignee: Marco Gaido > arrays_zip function's code generator splits input processing incorr

[jira] [Resolved] (SPARK-24633) arrays_zip function's code generator splits input processing incorrectly

[ https://issues.apache.org/jira/browse/SPARK-24633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24633. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21621 [https://gith

[jira] [Updated] (SPARK-24649) SparkUDF.unapply is not backwards compatable

[ https://issues.apache.org/jira/browse/SPARK-24649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon H.K. Fitch updated SPARK-24649: -- Description: The shape of the `ScalaUDF` case class changed in 2.3.0. A secondary co

[jira] [Assigned] (SPARK-24533) typesafe has rebranded to lightbend. change the build/mvn endpoint from downloads.typesafe.com to downloads.lightbend.com

[ https://issues.apache.org/jira/browse/SPARK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24533: Assignee: Apache Spark > typesafe has rebranded to lightbend. change the build/mvn endpoi

[jira] [Commented] (SPARK-24533) typesafe has rebranded to lightbend. change the build/mvn endpoint from downloads.typesafe.com to downloads.lightbend.com

[ https://issues.apache.org/jira/browse/SPARK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522349#comment-16522349 ] Apache Spark commented on SPARK-24533: -- User 'redsanket' has created a pull request

[jira] [Assigned] (SPARK-24533) typesafe has rebranded to lightbend. change the build/mvn endpoint from downloads.typesafe.com to downloads.lightbend.com

[ https://issues.apache.org/jira/browse/SPARK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24533: Assignee: (was: Apache Spark) > typesafe has rebranded to lightbend. change the build

[jira] [Created] (SPARK-24649) SparkUDF.unapply is not backwards compatable

Simeon H.K. Fitch created SPARK-24649: - Summary: SparkUDF.unapply is not backwards compatable Key: SPARK-24649 URL: https://issues.apache.org/jira/browse/SPARK-24649 Project: Spark Issue

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522339#comment-16522339 ] Hyukjin Kwon commented on SPARK-23710: -- That was a rather initial work having some

[jira] [Assigned] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24594: Assignee: (was: Apache Spark) > Introduce metrics for YARN executor allocation proble

[jira] [Commented] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522330#comment-16522330 ] Apache Spark commented on SPARK-24594: -- User 'attilapiros' has created a pull reque

[jira] [Assigned] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24594: Assignee: Apache Spark > Introduce metrics for YARN executor allocation problems > -

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522319#comment-16522319 ] Darek commented on SPARK-23710: --- The work was done in [PR20659|https://github.com/apache/s

[jira] [Commented] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522251#comment-16522251 ] Apache Spark commented on SPARK-24648: -- User 'dbkerkela' has created a pull request

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Affects Version/s: (was: 2.4.0) 2.3.1 > Sink Should Return OffsetSeqs For

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Fix Version/s: 2.4.0 > Sink Should Return OffsetSeqs For ProgressReporting > ---

[jira] [Assigned] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24648: Assignee: (was: Apache Spark) > SQLMetrics counters are not thread safe > ---

[jira] [Assigned] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24648: Assignee: Apache Spark > SQLMetrics counters are not thread safe > --

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement t

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Description: To be able to track data lineage for Structured Streaming (I intend to implement this

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Description: To be able to track data lineage for Structured Streaming (I intend to implement this

[jira] [Commented] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1654#comment-1654 ] Stacy Kerkela commented on SPARK-24648: --- I have a PR prepared for this. > SQLMetr

[jira] [Created] (SPARK-24648) SQLMetrics counters are not thread safe

Stacy Kerkela created SPARK-24648: - Summary: SQLMetrics counters are not thread safe Key: SPARK-24648 URL: https://issues.apache.org/jira/browse/SPARK-24648 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement t

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement t

[jira] [Commented] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522218#comment-16522218 ] Vaclav Kosar commented on SPARK-24647: -- [~c...@koeninger.org] and [~jlaskowski] you

[jira] [Created] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

Vaclav Kosar created SPARK-24647: Summary: Sink Should Return OffsetSeqs For ProgressReporting Key: SPARK-24647 URL: https://issues.apache.org/jira/browse/SPARK-24647 Project: Spark Issue Typ

[jira] [Commented] (SPARK-9775) Query Mesos for number of CPUs to set default parallelism

[ https://issues.apache.org/jira/browse/SPARK-9775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522064#comment-16522064 ] Maxim Gekk commented on SPARK-9775: --- Please, change another related methods like propos

[jira] [Commented] (SPARK-22150) PeriodicCheckpointer fails with FileNotFoundException in case of dependant RDDs

[ https://issues.apache.org/jira/browse/SPARK-22150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522015#comment-16522015 ] Sergey Zhemzhitsky commented on SPARK-22150: Just a kind remainder... > Per

[jira] [Commented] (SPARK-22184) GraphX fails in case of insufficient memory and checkpoints enabled

[ https://issues.apache.org/jira/browse/SPARK-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522017#comment-16522017 ] Sergey Zhemzhitsky commented on SPARK-22184: Just a kind remainder... > Gra

[jira] [Updated] (SPARK-24628) Typos of the example code in docs/mllib-data-types.md

[ https://issues.apache.org/jira/browse/SPARK-24628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weizhe Huang updated SPARK-24628: - Summary: Typos of the example code in docs/mllib-data-types.md (was: The example given to creat