[jira] [Updated] (SPARK-27802) SparkUI throws NoSuchElementException when inconsistency appears between `ExecutorStageSummaryWrapper`s and `ExecutorSummaryWrapper`s

2019-05-22 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-27802: - Description: Recently, we hit this issue when testing spark2.3. It report the following error m

[jira] [Commented] (SPARK-16738) Queryable state for Spark State Store

2019-05-22 Thread Patrick McGloin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845594#comment-16845594 ] Patrick McGloin commented on SPARK-16738: - For what it's worth [~skonto] I think

[jira] [Commented] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-05-22 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845603#comment-16845603 ] Josh Rosen commented on SPARK-19468: Chiming in to add a strong +1 here, since this

[jira] [Created] (SPARK-27803) fix column pruning for python UDF

2019-05-22 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-27803: --- Summary: fix column pruning for python UDF Key: SPARK-27803 URL: https://issues.apache.org/jira/browse/SPARK-27803 Project: Spark Issue Type: Bug Com

[jira] [Assigned] (SPARK-27803) fix column pruning for python UDF

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27803: Assignee: Wenchen Fan (was: Apache Spark) > fix column pruning for python UDF >

[jira] [Assigned] (SPARK-27803) fix column pruning for python UDF

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27803: Assignee: Apache Spark (was: Wenchen Fan) > fix column pruning for python UDF >

[jira] [Commented] (SPARK-16738) Queryable state for Spark State Store

2019-05-22 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845719#comment-16845719 ] Stavros Kontopoulos commented on SPARK-16738: - [~hyukjin.kwon] Both Kafka St

[jira] [Commented] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

2019-05-22 Thread Renze Post (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845784#comment-16845784 ] Renze Post commented on SPARK-26045: [~srowen] I think the issue here is that the Sp

[jira] [Created] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread duyu (JIRA)
duyu created SPARK-27804: Summary: LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition Key: SPARK-27804 URL: https://issues.apache.org/jira/browse/SPARK-27804 Project: Spark

[jira] [Updated] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread duyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyu updated SPARK-27804: - Description: LiveListenerBus.scala {code} private[spark] def addToQueue( listener: SparkListenerInterfac

[jira] [Commented] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845826#comment-16845826 ] Jungtaek Lim commented on SPARK-27804: -- Did you encounter this actually? The reason

[jira] [Commented] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

2019-05-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845832#comment-16845832 ] Sean Owen commented on SPARK-26045: --- Yes this is consistent with the thread on dev@ ye

[jira] [Assigned] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27804: Assignee: Apache Spark > LiveListenerBus#addToQueue : create multiple AsyncEventQueues un

[jira] [Assigned] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27804: Assignee: (was: Apache Spark) > LiveListenerBus#addToQueue : create multiple AsyncEve

[jira] [Commented] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread duyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845899#comment-16845899 ] duyu commented on SPARK-27804: -- I didn't. You say this method is guided with synchronized,

[jira] [Comment Edited] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread duyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845899#comment-16845899 ] duyu edited comment on SPARK-27804 at 5/22/19 2:08 PM: --- I didn't.

[jira] [Created] (SPARK-27805) toPandas does not propagate SparkExceptions with arrow enabled

2019-05-22 Thread David Vogelbacher (JIRA)
David Vogelbacher created SPARK-27805: - Summary: toPandas does not propagate SparkExceptions with arrow enabled Key: SPARK-27805 URL: https://issues.apache.org/jira/browse/SPARK-27805 Project: Spa

[jira] [Assigned] (SPARK-27805) toPandas does not propagate SparkExceptions with arrow enabled

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27805: Assignee: (was: Apache Spark) > toPandas does not propagate SparkExceptions with arro

[jira] [Assigned] (SPARK-27805) toPandas does not propagate SparkExceptions with arrow enabled

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27805: Assignee: Apache Spark > toPandas does not propagate SparkExceptions with arrow enabled >

[jira] [Commented] (SPARK-27714) Support Join Reorder based on Genetic Algorithm when the # of joined tables > 12

2019-05-22 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845988#comment-16845988 ] Nandor Kollar commented on SPARK-27714: --- Is this something similar that is impleme

[jira] [Updated] (SPARK-25139) PythonRunner#WriterThread released block after TaskRunner finally block which invoke BlockManager#releaseAllLocksForTask

2019-05-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25139: -- Fix Version/s: 2.3.4 > PythonRunner#WriterThread released block after TaskRunner finally block

[jira] [Created] (SPARK-27806) byName/byPosition should apply to struct fields as well

2019-05-22 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-27806: --- Summary: byName/byPosition should apply to struct fields as well Key: SPARK-27806 URL: https://issues.apache.org/jira/browse/SPARK-27806 Project: Spark Issue T

[jira] [Commented] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread duyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846016#comment-16846016 ] duyu commented on SPARK-27804: -- thank you [~kabhwan]. I ignored the word {color:#FF}sy

[jira] [Assigned] (SPARK-27806) byName/byPosition should apply to struct fields as well

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27806: Assignee: Wenchen Fan (was: Apache Spark) > byName/byPosition should apply to struct fie

[jira] [Assigned] (SPARK-27806) byName/byPosition should apply to struct fields as well

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27806: Assignee: Apache Spark (was: Wenchen Fan) > byName/byPosition should apply to struct fie

[jira] [Created] (SPARK-27807) Parallel resolve leaf statuses in InMemoryFileIndex

2019-05-22 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27807: --- Summary: Parallel resolve leaf statuses in InMemoryFileIndex Key: SPARK-27807 URL: https://issues.apache.org/jira/browse/SPARK-27807 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-27807) Parallel resolve leaf statuses in InMemoryFileIndex

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27807: Assignee: Apache Spark > Parallel resolve leaf statuses in InMemoryFileIndex > --

[jira] [Commented] (SPARK-27807) Parallel resolve leaf statuses in InMemoryFileIndex

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846041#comment-16846041 ] Apache Spark commented on SPARK-27807: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-27807) Parallel resolve leaf statuses in InMemoryFileIndex

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27807: Assignee: (was: Apache Spark) > Parallel resolve leaf statuses in InMemoryFileIndex >

[jira] [Commented] (SPARK-27797) Shuffle service metric "registeredConnections" not tracked correctly

2019-05-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846068#comment-16846068 ] Marcelo Vanzin commented on SPARK-27797: You're right. Looks just like some dead

[jira] [Created] (SPARK-27808) Ability to ignore existing files for structured streaming

2019-05-22 Thread Vladimir Matveev (JIRA)
Vladimir Matveev created SPARK-27808: Summary: Ability to ignore existing files for structured streaming Key: SPARK-27808 URL: https://issues.apache.org/jira/browse/SPARK-27808 Project: Spark

[jira] [Updated] (SPARK-27808) Ability to ignore existing files for structured streaming

2019-05-22 Thread Vladimir Matveev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Matveev updated SPARK-27808: - Description: Currently it is not easily possible to make a structured streaming query to

[jira] [Updated] (SPARK-27808) Ability to ignore existing files for structured streaming

2019-05-22 Thread Vladimir Matveev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Matveev updated SPARK-27808: - Description: Currently it is not easily possible to make a structured streaming query to

[jira] [Updated] (SPARK-27808) Ability to ignore existing files for structured streaming

2019-05-22 Thread Vladimir Matveev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Matveev updated SPARK-27808: - Description: Currently it is not easily possible to make a structured streaming query to

[jira] [Reopened] (SPARK-16824) Add API docs for VectorUDT

2019-05-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-16824: -- Reviewing the links here, it seems that VectorUDT has been in use since 2016 at the earli

[jira] [Updated] (SPARK-16824) Add API docs for VectorUDT

2019-05-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-16824: - Affects Version/s: 2.4.3 > Add API docs for VectorUDT > -- > >

[jira] [Updated] (SPARK-16824) Add API docs for VectorUDT

2019-05-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-16824: - Labels: (was: bulk-closed) > Add API docs for VectorUDT > -- >

[jira] [Resolved] (SPARK-27804) LiveListenerBus#addToQueue : create multiple AsyncEventQueues under race condition

2019-05-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27804. Resolution: Not A Problem > LiveListenerBus#addToQueue : create multiple AsyncEventQueues

[jira] [Assigned] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26045: Assignee: Apache Spark > Error in the spark 2.4 release package with the spark-avro_2.11

[jira] [Assigned] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26045: Assignee: (was: Apache Spark) > Error in the spark 2.4 release package with the spark

[jira] [Created] (SPARK-27809) Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement

2019-05-22 Thread Yesheng Ma (JIRA)
Yesheng Ma created SPARK-27809: -- Summary: Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement Key: SPARK-27809 URL: https://issues.apache.org/jira/browse/SPARK-27809 Project: S

[jira] [Assigned] (SPARK-27809) Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27809: Assignee: Apache Spark > Make optional clauses order insensitive for CREATE DATABASE/VIEW

[jira] [Assigned] (SPARK-27809) Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27809: Assignee: (was: Apache Spark) > Make optional clauses order insensitive for CREATE DA

[jira] [Assigned] (SPARK-20547) ExecutorClassLoader's findClass may not work correctly when a task is cancelled.

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20547: Assignee: Shixiong Zhu (was: Apache Spark) > ExecutorClassLoader's findClass may not wor

[jira] [Assigned] (SPARK-20547) ExecutorClassLoader's findClass may not work correctly when a task is cancelled.

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20547: Assignee: Apache Spark (was: Shixiong Zhu) > ExecutorClassLoader's findClass may not wor

[jira] [Updated] (SPARK-20547) ExecutorClassLoader's findClass may not work correctly when a task is cancelled.

2019-05-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20547: - Labels: (was: bulk-closed) > ExecutorClassLoader's findClass may not work correctly when a tas

[jira] [Resolved] (SPARK-23153) Support application dependencies in submission client's local file system

2019-05-22 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-23153. Resolution: Fixed > Support application dependencies in submission client's local file sys

[jira] [Assigned] (SPARK-23153) Support application dependencies in submission client's local file system

2019-05-22 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson reassigned SPARK-23153: -- Assignee: Stavros Kontopoulos Fix Version/s: 3.0.0 > Support application dep

[jira] [Created] (SPARK-27810) PySpark breaks Cloudpickle serialization of collections.namedtuple objects

2019-05-22 Thread Travis Addair (JIRA)
Travis Addair created SPARK-27810: - Summary: PySpark breaks Cloudpickle serialization of collections.namedtuple objects Key: SPARK-27810 URL: https://issues.apache.org/jira/browse/SPARK-27810 Project:

[jira] [Updated] (SPARK-27810) PySpark breaks Cloudpickle serialization of collections.namedtuple objects

2019-05-22 Thread Travis Addair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Addair updated SPARK-27810: -- Description: After importing pyspark, cloudpickle is no longer able to properly serialize obj

[jira] [Commented] (SPARK-15015) Log statements lack file name/number

2019-05-22 Thread John-Michael Reed (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846389#comment-16846389 ] John-Michael Reed commented on SPARK-15015: --- I would be willing to work to add

[jira] [Reopened] (SPARK-15015) Log statements lack file name/number

2019-05-22 Thread John-Michael Reed (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John-Michael Reed reopened SPARK-15015: --- See last comment. > Log statements lack file name/number >

[jira] [Resolved] (SPARK-27807) Parallel resolve leaf statuses in InMemoryFileIndex

2019-05-22 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-27807. - Resolution: Duplicate > Parallel resolve leaf statuses in InMemoryFileIndex > --

[jira] [Created] (SPARK-27811) Docs of spark.driver.memoryOverhead and spark.executor.memoryOverhead exists a little ambiguity

2019-05-22 Thread jiaan.geng (JIRA)
jiaan.geng created SPARK-27811: -- Summary: Docs of spark.driver.memoryOverhead and spark.executor.memoryOverhead exists a little ambiguity Key: SPARK-27811 URL: https://issues.apache.org/jira/browse/SPARK-27811

[jira] [Commented] (SPARK-27811) Docs of spark.driver.memoryOverhead and spark.executor.memoryOverhead exists a little ambiguity

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846415#comment-16846415 ] Apache Spark commented on SPARK-27811: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-27811) Docs of spark.driver.memoryOverhead and spark.executor.memoryOverhead exists a little ambiguity

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27811: Assignee: Apache Spark > Docs of spark.driver.memoryOverhead and spark.executor.memoryOv

[jira] [Assigned] (SPARK-27811) Docs of spark.driver.memoryOverhead and spark.executor.memoryOverhead exists a little ambiguity

2019-05-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27811: Assignee: (was: Apache Spark) > Docs of spark.driver.memoryOverhead and spark.execut

[jira] [Updated] (SPARK-27322) Select from multiple catalogs

2019-05-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27322: --- Description: Support multi-catalog in the following SELECT code paths: * SELECT * FROM catalog.db.t

[jira] [Updated] (SPARK-27810) PySpark breaks Cloudpickle serialization of collections.namedtuple objects

2019-05-22 Thread Travis Addair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Addair updated SPARK-27810: -- Description: After importing pyspark, cloudpickle is no longer able to properly serialize obj

[jira] [Updated] (SPARK-27810) PySpark breaks Cloudpickle serialization of collections.namedtuple objects

2019-05-22 Thread Travis Addair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Addair updated SPARK-27810: -- Description: After importing pyspark, cloudpickle is no longer able to properly serialize obj

[jira] [Commented] (SPARK-27705) kubernetes integration test break on osx when test PVTestsSuite

2019-05-22 Thread Henry Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846458#comment-16846458 ] Henry Yu commented on SPARK-27705: -- [~dongjoon] That's great! Could you make a pr?  An

[jira] [Commented] (SPARK-26388) No support for "alter table .. replace columns" to drop columns

2019-05-22 Thread Mathew Wicks (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846469#comment-16846469 ] Mathew Wicks commented on SPARK-26388: -- This appears to still be an issue in Spark

[jira] [Commented] (SPARK-27657) ml.util.Instrumentation.logFailure doesn't log error message

2019-05-22 Thread Wesley Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846476#comment-16846476 ] Wesley Tang commented on SPARK-27657: - If you won't mind, i can send a PR to fix thi

[jira] [Created] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

2019-05-22 Thread Henry Yu (JIRA)
Henry Yu created SPARK-27812: Summary: kubernetes client import non-daemon thread which block jvm exit. Key: SPARK-27812 URL: https://issues.apache.org/jira/browse/SPARK-27812 Project: Spark Iss