[jira] [Comment Edited] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859710#comment-16859710 ] Henry Yu edited comment on SPARK-27812 at 6/10/19 5:55 AM: --- In our private

[jira] [Commented] (SPARK-27697) KubernetesClientApplication alway exit with 0

[ https://issues.apache.org/jira/browse/SPARK-27697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859717#comment-16859717 ] Henry Yu commented on SPARK-27697: -- @[~dongjoon] I fix it by adding a pod phase judgement . If driver

[jira] [Commented] (SPARK-27960) DataSourceV2 ORC implementation doesn't handle schemas correctly

[ https://issues.apache.org/jira/browse/SPARK-27960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859714#comment-16859714 ] Gengliang Wang commented on SPARK-27960: I think we can resolve it in this way:

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859712#comment-16859712 ] Henry Yu commented on SPARK-27812: -- According to Okhttp Committer 

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859710#comment-16859710 ] Henry Yu commented on SPARK-27812: -- In our private branch, I fix this and potential non-daemon thread

[jira] [Assigned] (SPARK-27988) Add aggregates.sql - Part3

[ https://issues.apache.org/jira/browse/SPARK-27988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27988: Assignee: (was: Apache Spark) > Add aggregates.sql - Part3 >

[jira] [Assigned] (SPARK-27988) Add aggregates.sql - Part3

[ https://issues.apache.org/jira/browse/SPARK-27988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27988: Assignee: Apache Spark > Add aggregates.sql - Part3 > -- > >

[jira] [Created] (SPARK-27988) Add aggregates.sql - Part3

Yuming Wang created SPARK-27988: --- Summary: Add aggregates.sql - Part3 Key: SPARK-27988 URL: https://issues.apache.org/jira/browse/SPARK-27988 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-27980) Add built-in Ordered-Set Aggregate Functions: percentile_cont

[ https://issues.apache.org/jira/browse/SPARK-27980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27980: Description: ||Function||Direct Argument Type(s)||Aggregated Argument Type(s)||Return

[jira] [Closed] (SPARK-27983) defer the initialization of kafka producer until used

[ https://issues.apache.org/jira/browse/SPARK-27983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-27983. - > defer the initialization of kafka producer until used >

[jira] [Resolved] (SPARK-27983) defer the initialization of kafka producer until used

[ https://issues.apache.org/jira/browse/SPARK-27983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27983. --- Resolution: Invalid This is closed by author after some discussion. Please see the PR

[jira] [Commented] (SPARK-24791) Spark Structured Streaming randomly does not process batch

[ https://issues.apache.org/jira/browse/SPARK-24791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859683#comment-16859683 ] Apache Spark commented on SPARK-24791: -- User 'zhangmeng0426' has created a pull request for this

[jira] [Assigned] (SPARK-24791) Spark Structured Streaming randomly does not process batch

[ https://issues.apache.org/jira/browse/SPARK-24791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24791: Assignee: Apache Spark > Spark Structured Streaming randomly does not process batch >

[jira] [Assigned] (SPARK-24791) Spark Structured Streaming randomly does not process batch

[ https://issues.apache.org/jira/browse/SPARK-24791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24791: Assignee: (was: Apache Spark) > Spark Structured Streaming randomly does not process

[jira] [Comment Edited] (SPARK-20894) Error while checkpointing to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859672#comment-16859672 ] phan minh duc edited comment on SPARK-20894 at 6/10/19 2:57 AM: I'm

[jira] [Comment Edited] (SPARK-20894) Error while checkpointing to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859672#comment-16859672 ] phan minh duc edited comment on SPARK-20894 at 6/10/19 2:52 AM: I'm

[jira] [Commented] (SPARK-20894) Error while checkpointing to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859672#comment-16859672 ] phan minh duc commented on SPARK-20894: --- I'm using spark 2.4.0 and facing the same issue when i

[jira] [Commented] (SPARK-27985) List of Spark releases in SdkMan gone stale

[ https://issues.apache.org/jira/browse/SPARK-27985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859643#comment-16859643 ] Hyukjin Kwon commented on SPARK-27985: -- I don't think our official release is made there. See

[jira] [Resolved] (SPARK-27985) List of Spark releases in SdkMan gone stale

[ https://issues.apache.org/jira/browse/SPARK-27985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27985. -- Resolution: Invalid > List of Spark releases in SdkMan gone stale >

[jira] [Created] (SPARK-27987) Support POSIX Regular Expressions

Yuming Wang created SPARK-27987: --- Summary: Support POSIX Regular Expressions Key: SPARK-27987 URL: https://issues.apache.org/jira/browse/SPARK-27987 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-27846) Eagerly compute Configuration.properties in sc.hadoopConfiguration

[ https://issues.apache.org/jira/browse/SPARK-27846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27846. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via

[jira] [Updated] (SPARK-27846) Eagerly compute Configuration.properties in sc.hadoopConfiguration

[ https://issues.apache.org/jira/browse/SPARK-27846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27846: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Eagerly compute

[jira] [Commented] (SPARK-27930) Add built-in Math Function: RANDOM

[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859628#comment-16859628 ] Yuming Wang commented on SPARK-27930: - Workaround: {code:sql} select reflect("java.lang.Math",

[jira] [Resolved] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27546. -- Resolution: Not A Problem > Should repalce DateTimeUtils#defaultTimeZoneuse with

[jira] [Commented] (SPARK-27759) Do not auto cast array to np.array in vectorized udf

[ https://issues.apache.org/jira/browse/SPARK-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859625#comment-16859625 ] Hyukjin Kwon commented on SPARK-27759: -- BTW, currently Pandas UDFs don't support type-widening out

[jira] [Created] (SPARK-27986) Support Aggregate Expressions

Yuming Wang created SPARK-27986: --- Summary: Support Aggregate Expressions Key: SPARK-27986 URL: https://issues.apache.org/jira/browse/SPARK-27986 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-27985) List of Spark releases in SdkMan gone stale

Sergii Mikhtoniuk created SPARK-27985: - Summary: List of Spark releases in SdkMan gone stale Key: SPARK-27985 URL: https://issues.apache.org/jira/browse/SPARK-27985 Project: Spark Issue

[jira] [Updated] (SPARK-27081) Support launching executors in existed Pods

[ https://issues.apache.org/jira/browse/SPARK-27081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27081: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support launching

[jira] [Updated] (SPARK-23720) Leverage shuffle service when running in non-host networking mode in hadoop 3 docker support

[ https://issues.apache.org/jira/browse/SPARK-23720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23720: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Leverage shuffle service

[jira] [Updated] (SPARK-23719) Use correct hostname in non-host networking mode in hadoop 3 docker support

[ https://issues.apache.org/jira/browse/SPARK-23719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23719: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Use correct hostname in

[jira] [Updated] (SPARK-23721) Enhance BlockManagerId to include container's underlying host machine hostname

[ https://issues.apache.org/jira/browse/SPARK-23721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23721: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Enhance BlockManagerId to

[jira] [Updated] (SPARK-23718) Document using docker in host networking mode in hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23718: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Document using docker in

[jira] [Updated] (SPARK-23717) Leverage docker support in Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23717: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Leverage docker support in

[jira] [Updated] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-9: -- Affects Version/s: (was: 2.4.0) (was: 2.3.0) > SPIP: RDMA

[jira] [Assigned] (SPARK-27870) Flush each batch for pandas UDF (for improving pandas UDFs pipeline)

[ https://issues.apache.org/jira/browse/SPARK-27870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27870: Assignee: (was: Apache Spark) > Flush each batch for pandas UDF (for improving

[jira] [Assigned] (SPARK-27870) Flush each batch for pandas UDF (for improving pandas UDFs pipeline)

[ https://issues.apache.org/jira/browse/SPARK-27870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27870: Assignee: Apache Spark > Flush each batch for pandas UDF (for improving pandas UDFs

[jira] [Updated] (SPARK-25039) Binary comparison behavior should refer to Teradata

[ https://issues.apache.org/jira/browse/SPARK-25039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25039: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Binary comparison behavior

[jira] [Updated] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24942: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Improve cluster resource

[jira] [Updated] (SPARK-25342) Support rolling back a result stage

[ https://issues.apache.org/jira/browse/SPARK-25342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25342: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support rolling back a

[jira] [Updated] (SPARK-24941) Add RDDBarrier.coalesce() function

[ https://issues.apache.org/jira/browse/SPARK-24941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24941: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add RDDBarrier.coalesce()

[jira] [Updated] (SPARK-26988) Spark overwrites spark.scheduler.pool if set in configs

[ https://issues.apache.org/jira/browse/SPARK-26988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26988: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Spark overwrites

[jira] [Commented] (SPARK-26988) Spark overwrites spark.scheduler.pool if set in configs

[ https://issues.apache.org/jira/browse/SPARK-26988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859615#comment-16859615 ] Dongjoon Hyun commented on SPARK-26988: --- If there is no problem when we use `--conf` option or

[jira] [Commented] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859613#comment-16859613 ] Dongjoon Hyun commented on SPARK-25053: --- Hi, [~skonto]. Could you add the JIRA issue here and

[jira] [Updated] (SPARK-25153) Improve error messages for columns with dots/periods

[ https://issues.apache.org/jira/browse/SPARK-25153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25153: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Improve error messages for

[jira] [Updated] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25732: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Allow specifying a

[jira] [Updated] (SPARK-25643) Performance issues querying wide rows

[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25643: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Performance issues

[jira] [Updated] (SPARK-25878) Document existing k8s features and how to add new features

[ https://issues.apache.org/jira/browse/SPARK-25878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25878: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Document existing k8s

[jira] [Updated] (SPARK-25878) Document existing k8s features and how to add new features

[ https://issues.apache.org/jira/browse/SPARK-25878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25878: -- Component/s: Documentation > Document existing k8s features and how to add new features >

[jira] [Updated] (SPARK-25752) Add trait to easily whitelist logical operators that produce named output from CleanupAliases

[ https://issues.apache.org/jira/browse/SPARK-25752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25752: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add trait to easily

[jira] [Updated] (SPARK-25049) Support custom schema in `to_avro`

[ https://issues.apache.org/jira/browse/SPARK-25049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25049: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support custom schema in

[jira] [Updated] (SPARK-25722) Support a backtick character in column names

[ https://issues.apache.org/jira/browse/SPARK-25722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25722: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support a backtick

[jira] [Updated] (SPARK-26058) Incorrect logging class loaded for all the logs.

[ https://issues.apache.org/jira/browse/SPARK-26058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26058: -- Affects Version/s: (was: 2.4.0) > Incorrect logging class loaded for all the logs. >

[jira] [Updated] (SPARK-26111) Support ANOVA F-value between label/feature for the continuous distribution feature selection

[ https://issues.apache.org/jira/browse/SPARK-26111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26111: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support ANOVA F-value

[jira] [Updated] (SPARK-26305) Breakthrough the memory limitation of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-26305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26305: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Breakthrough the memory

[jira] [Closed] (SPARK-26062) Rename spark-avro external module to spark-sql-avro (to match spark-sql-kafka)

[ https://issues.apache.org/jira/browse/SPARK-26062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26062. - > Rename spark-avro external module to spark-sql-avro (to match spark-sql-kafka) >

[jira] [Updated] (SPARK-26309) Verification of Data source options

[ https://issues.apache.org/jira/browse/SPARK-26309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26309: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Verification of Data

[jira] [Resolved] (SPARK-26062) Rename spark-avro external module to spark-sql-avro (to match spark-sql-kafka)

[ https://issues.apache.org/jira/browse/SPARK-26062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26062. --- Resolution: Duplicate SPARK-24768 decided the opposite direction; `spark-avro` instead of

[jira] [Updated] (SPARK-26238) Set SPARK_CONF_DIR to be ${SPARK_HOME}/conf for K8S

[ https://issues.apache.org/jira/browse/SPARK-26238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26238: -- Priority: Minor (was: Major) > Set SPARK_CONF_DIR to be ${SPARK_HOME}/conf for K8S >

[jira] [Updated] (SPARK-26238) Set SPARK_CONF_DIR to be ${SPARK_HOME}/conf for K8S

[ https://issues.apache.org/jira/browse/SPARK-26238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26238: -- Affects Version/s: (was: 2.4.0) > Set SPARK_CONF_DIR to be ${SPARK_HOME}/conf for K8S >

[jira] [Updated] (SPARK-26209) Allow for dataframe bucketization without Hive

[ https://issues.apache.org/jira/browse/SPARK-26209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26209: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Allow for dataframe

[jira] [Updated] (SPARK-26342) Support for NFS mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26342: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support for NFS mount for

[jira] [Updated] (SPARK-26425) Add more constraint checks in file streaming source to avoid checkpoint corruption

[ https://issues.apache.org/jira/browse/SPARK-26425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26425: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add more constraint checks

[jira] [Updated] (SPARK-26344) Support for flexVolume mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26344: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support for flexVolume

[jira] [Updated] (SPARK-26639) The reuse subquery function maybe does not work in SPARK SQL

[ https://issues.apache.org/jira/browse/SPARK-26639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26639: -- Affects Version/s: (was: 2.3.2) (was: 2.4.0)

[jira] [Updated] (SPARK-26639) The reuse subquery function maybe does not work in SPARK SQL

[ https://issues.apache.org/jira/browse/SPARK-26639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26639: -- Component/s: (was: Web UI) > The reuse subquery function maybe does not work in SPARK SQL

[jira] [Updated] (SPARK-26679) Deconflict spark.executor.pyspark.memory and spark.python.worker.memory

[ https://issues.apache.org/jira/browse/SPARK-26679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26679: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Deconflict

[jira] [Commented] (SPARK-26533) Support query auto cancel on thriftserver

[ https://issues.apache.org/jira/browse/SPARK-26533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859605#comment-16859605 ] Dongjoon Hyun commented on SPARK-26533: --- Thank you for filing a JIRA, [~cane]. For new features,

[jira] [Updated] (SPARK-26533) Support query auto cancel on thriftserver

[ https://issues.apache.org/jira/browse/SPARK-26533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26533: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support query auto cancel

[jira] [Updated] (SPARK-26505) Catalog class Function is missing "database" field

[ https://issues.apache.org/jira/browse/SPARK-26505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26505: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Catalog class Function is

[jira] [Updated] (SPARK-26833) Kubernetes RBAC documentation is unclear on exact RBAC requirements

[ https://issues.apache.org/jira/browse/SPARK-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26833: -- Affects Version/s: (was: 2.3.2) (was: 2.3.1)

[jira] [Updated] (SPARK-26833) Kubernetes RBAC documentation is unclear on exact RBAC requirements

[ https://issues.apache.org/jira/browse/SPARK-26833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26833: -- Component/s: Documentation > Kubernetes RBAC documentation is unclear on exact RBAC

[jira] [Comment Edited] (SPARK-27334) Support specify scheduler name for executor pods when submit

[ https://issues.apache.org/jira/browse/SPARK-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859599#comment-16859599 ] Dongjoon Hyun edited comment on SPARK-27334 at 6/9/19 11:16 PM: As

[jira] [Closed] (SPARK-27334) Support specify scheduler name for executor pods when submit

[ https://issues.apache.org/jira/browse/SPARK-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-27334. - > Support specify scheduler name for executor pods when submit >

[jira] [Resolved] (SPARK-27334) Support specify scheduler name for executor pods when submit

[ https://issues.apache.org/jira/browse/SPARK-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27334. --- Resolution: Duplicate Fix Version/s: 3.0.0 As [~Alexander_Fedosov] pointed out,

[jira] [Commented] (SPARK-27424) Joining of one stream against the most recent update in another stream

[ https://issues.apache.org/jira/browse/SPARK-27424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859593#comment-16859593 ] Thilo Schneider commented on SPARK-27424: - Sehr geehrte Damen und Herren, vielen Dank für Ihre

[jira] [Commented] (SPARK-27424) Joining of one stream against the most recent update in another stream

[ https://issues.apache.org/jira/browse/SPARK-27424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859594#comment-16859594 ] Dongjoon Hyun commented on SPARK-27424: --- Thank you for filing a JIRA and document,

[jira] [Updated] (SPARK-27424) Joining of one stream against the most recent update in another stream

[ https://issues.apache.org/jira/browse/SPARK-27424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27424: -- Affects Version/s: (was: 2.4.1) 3.0.0 > Joining of one stream

[jira] [Updated] (SPARK-18569) Support R formula arithmetic

[ https://issues.apache.org/jira/browse/SPARK-18569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18569: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Support R formula

[jira] [Commented] (SPARK-27455) spark-submit and friends should allow main artifact to be specified as a package

[ https://issues.apache.org/jira/browse/SPARK-27455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859577#comment-16859577 ] Dongjoon Hyun commented on SPARK-27455: --- Hi, [~lindblombr] and [~dbtsai]. Could you make a PR for

[jira] [Resolved] (SPARK-27872) Driver and executors use a different service account breaking pull secrets

[ https://issues.apache.org/jira/browse/SPARK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27872. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24748

[jira] [Updated] (SPARK-27455) spark-submit and friends should allow main artifact to be specified as a package

[ https://issues.apache.org/jira/browse/SPARK-27455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27455: -- Affects Version/s: (was: 2.4.1) 3.0.0 > spark-submit and friends

[jira] [Assigned] (SPARK-27872) Driver and executors use a different service account breaking pull secrets

[ https://issues.apache.org/jira/browse/SPARK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27872: - Assignee: Stavros Kontopoulos > Driver and executors use a different service account breaking

[jira] [Commented] (SPARK-27456) Support commitSync for offsets in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-27456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859572#comment-16859572 ] Dongjoon Hyun commented on SPARK-27456: --- Thank you for filing a JIRA. Since new feature is not

[jira] [Updated] (SPARK-27456) Support commitSync for offsets in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-27456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27456: -- Affects Version/s: (was: 2.4.1) 3.0.0 > Support commitSync for

[jira] [Resolved] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27499. --- Resolution: Duplicate Fix Version/s: 2.4.0 Hi, [~junjie]. Thank you for reporting,

[jira] [Updated] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27499: -- Affects Version/s: (was: 2.4.1) 3.0.0 > Support mapping

[jira] [Commented] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859564#comment-16859564 ] Dongjoon Hyun commented on SPARK-27546: --- Hi, [~Aron.tao]. The given example looks irrelevant to

[jira] [Updated] (SPARK-27543) Support getRequiredJars and getRequiredFiles APIs for Hive UDFs

[ https://issues.apache.org/jira/browse/SPARK-27543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27543: -- Affects Version/s: (was: 2.4.1) (was: 2.0.0)

[jira] [Updated] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27546: -- Affects Version/s: (was: 2.4.1) 3.0.0 > Should repalce

[jira] [Updated] (SPARK-27573) Skip partial aggregation when data is already partitioned (or collapse adjacent partial and final aggregates)

[ https://issues.apache.org/jira/browse/SPARK-27573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27573: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Skip partial aggregation

[jira] [Updated] (SPARK-27599) DataFrameWriter.partitionBy should be optional when writing to a hive table

[ https://issues.apache.org/jira/browse/SPARK-27599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27599: -- Affects Version/s: (was: 2.4.1) 3.0.0 >

[jira] [Updated] (SPARK-27708) Add documentation for v2 data sources

[ https://issues.apache.org/jira/browse/SPARK-27708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27708: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Add documentation for v2

[jira] [Updated] (SPARK-27471) Reorganize public v2 catalog API

[ https://issues.apache.org/jira/browse/SPARK-27471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27471: -- Affects Version/s: (was: 2.4.1) 3.0.0 > Reorganize public v2

[jira] [Updated] (SPARK-27709) AppStatusListener.cleanupExecutors should remove dead executors in an ordering that makes sense, not a random order

[ https://issues.apache.org/jira/browse/SPARK-27709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27709: -- Affects Version/s: (was: 2.4.0) 3.0.0 >

[jira] [Updated] (SPARK-27602) SparkSQL CBO can't get true size of partition table after partition pruning

[ https://issues.apache.org/jira/browse/SPARK-27602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27602: -- Affects Version/s: (was: 2.4.0) (was: 2.3.0)

[jira] [Updated] (SPARK-27714) Support Join Reorder based on Genetic Algorithm when the # of joined tables > 12

[ https://issues.apache.org/jira/browse/SPARK-27714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27714: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Support Join Reorder based

[jira] [Commented] (SPARK-27719) Set maxDisplayLogSize for spark history server

[ https://issues.apache.org/jira/browse/SPARK-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859558#comment-16859558 ] Dongjoon Hyun commented on SPARK-27719: --- Hi, [~hao.li]. Do you have big log files over 200GB? In

[jira] [Updated] (SPARK-27759) Do not auto cast array to np.array in vectorized udf

[ https://issues.apache.org/jira/browse/SPARK-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27759: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Do not auto cast array to

[jira] [Updated] (SPARK-27719) Set maxDisplayLogSize for spark history server

[ https://issues.apache.org/jira/browse/SPARK-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27719: -- Affects Version/s: (was: 2.4.1) 3.0.0 > Set maxDisplayLogSize for

[jira] [Commented] (SPARK-27775) Support multiple return values for udf

[ https://issues.apache.org/jira/browse/SPARK-27775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859553#comment-16859553 ] Dongjoon Hyun commented on SPARK-27775: --- Hi, [~advancedxy]. Thank you for filing JIRA issue. I

  1   2   >