[jira] [Updated] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27068: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Support failed jobs ui and

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860605#comment-16860605 ] HonglunChen commented on SPARK-18112: - [~dongjoon],I replaced the hive-1.x jars(hiv

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860602#comment-16860602 ] Dongjoon Hyun commented on SPARK-27812: --- If SPARK-26742 is the root cause and ther

[jira] [Updated] (SPARK-27997) kubernetes client token expired

[ https://issues.apache.org/jira/browse/SPARK-27997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27997: -- Description: Hi , when I try to submit spark to k8s in cluster mode, I need an authtoken to t

[jira] [Updated] (SPARK-27997) kubernetes client token expired

[ https://issues.apache.org/jira/browse/SPARK-27997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27997: -- Shepherd: (was: Dongjoon Hyun) > kubernetes client token expired >

[jira] [Updated] (SPARK-27997) kubernetes client token expired

[ https://issues.apache.org/jira/browse/SPARK-27997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27997: -- Issue Type: Improvement (was: Bug) > kubernetes client token expired > -

[jira] [Updated] (SPARK-27997) kubernetes client token expired

[ https://issues.apache.org/jira/browse/SPARK-27997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27997: -- Affects Version/s: (was: 2.4.3) 3.0.0 > kubernetes client token exp

[jira] [Resolved] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27499. --- Resolution: Invalid > Support mapping spark.local.dir to hostPath volume > -

[jira] [Comment Edited] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860598#comment-16860598 ] Dongjoon Hyun edited comment on SPARK-27499 at 6/11/19 6:33 AM: --

[jira] [Closed] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-27499. - > Support mapping spark.local.dir to hostPath volume > -

[jira] [Comment Edited] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860598#comment-16860598 ] Dongjoon Hyun edited comment on SPARK-27499 at 6/11/19 6:33 AM: --

[jira] [Commented] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860598#comment-16860598 ] Dongjoon Hyun commented on SPARK-27499: --- No, I think you didn't read the patch of

[jira] [Comment Edited] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860595#comment-16860595 ] Dongjoon Hyun edited comment on SPARK-18112 at 6/11/19 6:29 AM: --

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860595#comment-16860595 ] Dongjoon Hyun commented on SPARK-18112: --- What is your issue, [~honglun]? In this i

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860580#comment-16860580 ] Henry Yu commented on SPARK-27812: -- So, any other idea to fix it? or I will try to make

[jira] [Commented] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860579#comment-16860579 ] zhengruifeng commented on SPARK-27018: -- I test on both local env and a cluster env,

[jira] [Created] (SPARK-27997) kubernetes client token expired

Henry Yu created SPARK-27997: Summary: kubernetes client token expired Key: SPARK-27997 URL: https://issues.apache.org/jira/browse/SPARK-27997 Project: Spark Issue Type: Bug Components

[jira] [Comment Edited] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860519#comment-16860519 ] Junjie Chen edited comment on SPARK-27499 at 6/11/19 5:20 AM:

[jira] [Updated] (SPARK-27845) DataSourceV2: InsertTable

[ https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27845: --- Summary: DataSourceV2: InsertTable (was: DataSourceV2: Insert into tables in multiple catalogs) >

[jira] [Updated] (SPARK-27322) DataSourceV2 table relation

[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27322: --- Summary: DataSourceV2 table relation (was: DataSourceV2: Select from multiple catalogs) > DataSour

[jira] [Updated] (SPARK-27372) Standalone executor process-level isolation to support GPU scheduling

[ https://issues.apache.org/jira/browse/SPARK-27372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27372: -- Issue Type: Story (was: Sub-task) Parent: (was: SPARK-27360) > Standalone executo

[jira] [Updated] (SPARK-27371) Standalone master receives resource info from worker and allocate driver/executor properly

[ https://issues.apache.org/jira/browse/SPARK-27371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27371: -- Summary: Standalone master receives resource info from worker and allocate driver/executor pro

[jira] [Updated] (SPARK-27371) Master receives resource info from worker and allocate driver/executor properly

[ https://issues.apache.org/jira/browse/SPARK-27371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27371: -- Summary: Master receives resource info from worker and allocate driver/executor properly (was

[jira] [Deleted] (SPARK-27370) spark-submit requests GPUs in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-27370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng deleted SPARK-27370: -- > spark-submit requests GPUs in standalone mode > - >

[jira] [Updated] (SPARK-27369) Standalone worker can load resource conf and discover resources

[ https://issues.apache.org/jira/browse/SPARK-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27369: -- Summary: Standalone worker can load resource conf and discover resources (was: Standalone sup

[jira] [Reopened] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junjie Chen reopened SPARK-27499: - > Support mapping spark.local.dir to hostPath volume > -

[jira] [Commented] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860519#comment-16860519 ] Junjie Chen commented on SPARK-27499: - Hi, [~dongjoon], I know SPARK_LOCAL_DIRS can

[jira] [Updated] (SPARK-27368) Design: Standalone supports GPU scheduling

[ https://issues.apache.org/jira/browse/SPARK-27368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27368: -- Description: Design draft: Scenarios: * client-mode, worker might create one or more executor

[jira] [Updated] (SPARK-27368) Design: Standalone supports GPU scheduling

[ https://issues.apache.org/jira/browse/SPARK-27368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27368: -- Description: Design draft: Scenarios: * client-mode, worker might create one or more executor

[jira] [Updated] (SPARK-27368) Design: Standalone supports GPU scheduling

[ https://issues.apache.org/jira/browse/SPARK-27368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-27368: -- Description: Design draft: Scenarios: * client-mode, worker might create one or more executor

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860512#comment-16860512 ] HonglunChen commented on SPARK-18112: - Why do my issue still exist，I use the Spark-2

[jira] [Created] (SPARK-27996) Spark UI redirect will be failed behind the https reverse proxy

Saisai Shao created SPARK-27996: --- Summary: Spark UI redirect will be failed behind the https reverse proxy Key: SPARK-27996 URL: https://issues.apache.org/jira/browse/SPARK-27996 Project: Spark

[jira] [Assigned] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized toPandas

[ https://issues.apache.org/jira/browse/SPARK-27995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27995: Assignee: (was: Apache Spark) > Note the difference between str of Python 2 and 3 at

[jira] [Assigned] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized toPandas

[ https://issues.apache.org/jira/browse/SPARK-27995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27995: Assignee: Apache Spark > Note the difference between str of Python 2 and 3 at Arrow optim

[jira] [Updated] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized toPandas

[ https://issues.apache.org/jira/browse/SPARK-27995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27995: - Description: When Arrow optimization is enabled in Python 2.7, {code} import pandas pdf = pand

[jira] [Created] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized toPandas

Hyukjin Kwon created SPARK-27995: Summary: Note the difference between str of Python 2 and 3 at Arrow optimized toPandas Key: SPARK-27995 URL: https://issues.apache.org/jira/browse/SPARK-27995 Project

[jira] [Comment Edited] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860491#comment-16860491 ] Jiatao Tao edited comment on SPARK-27546 at 6/11/19 2:46 AM: -

[jira] [Commented] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860491#comment-16860491 ] Jiatao Tao commented on SPARK-27546: After reading the code,  Although I have set "

[jira] [Comment Edited] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860491#comment-16860491 ] Jiatao Tao edited comment on SPARK-27546 at 6/11/19 2:42 AM: -

[jira] [Comment Edited] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860489#comment-16860489 ] Jiatao Tao edited comment on SPARK-27546 at 6/11/19 2:40 AM: -

[jira] [Commented] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860489#comment-16860489 ] Jiatao Tao commented on SPARK-27546: Hi [~dongjoon]   What I get is "ts", not the

[jira] [Issue Comment Deleted] (SPARK-20894) Error while checkpointing to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] phan minh duc updated SPARK-20894: -- Comment: was deleted (was: I'm using spark 2.4.0 and facing the same issue when i submit struc

[jira] [Updated] (SPARK-27825) spark thriftserver session's first username cause the impersonation issue.

[ https://issues.apache.org/jira/browse/SPARK-27825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxinxin updated SPARK-27825: --- Attachment: test.zip > spark thriftserver session's first username cause the impersonation issue.

[jira] [Updated] (SPARK-27825) spark thriftserver session's first username cause the impersonation issue.

[ https://issues.apache.org/jira/browse/SPARK-27825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangxinxin updated SPARK-27825: --- Attachment: (was: test.zip) > spark thriftserver session's first username cause the impersonat

[jira] [Resolved] (SPARK-27979) Remove deprecated `--force` option in `build/mvn` and `run-tests.py`

[ https://issues.apache.org/jira/browse/SPARK-27979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27979. --- Resolution: Fixed This is resolved back via https://github.com/apache/spark/pull/24833 > Re

[jira] [Commented] (SPARK-27994) Spark Avro Failed to read logical type decimal backed by bytes

[ https://issues.apache.org/jira/browse/SPARK-27994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860442#comment-16860442 ] Nicolas Pascal commented on SPARK-27994: I'll push a PR with failing test cases

[jira] [Closed] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-25053. - Thank you, [~skonto]. > Allow additional port forwarding on Spark on K8S as needed > --

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860439#comment-16860439 ] Dongjoon Hyun commented on SPARK-27812: --- Thank you for update, [~Andrew HUALI]. >

[jira] [Commented] (SPARK-27697) KubernetesClientApplication alway exit with 0

[ https://issues.apache.org/jira/browse/SPARK-27697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860438#comment-16860438 ] Dongjoon Hyun commented on SPARK-27697: --- Please make a PR. Let's see there, [~Andr

[jira] [Assigned] (SPARK-27993) Port HIVE-12981 to hive-thriftserver

[ https://issues.apache.org/jira/browse/SPARK-27993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27993: Assignee: (was: Apache Spark) > Port HIVE-12981 to hive-thriftserver > --

[jira] [Assigned] (SPARK-27993) Port HIVE-12981 to hive-thriftserver

[ https://issues.apache.org/jira/browse/SPARK-27993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27993: Assignee: Apache Spark > Port HIVE-12981 to hive-thriftserver > -

[jira] [Created] (SPARK-27994) Spark Avro Failed to read logical type decimal backed by bytes

Nicolas Pascal created SPARK-27994: -- Summary: Spark Avro Failed to read logical type decimal backed by bytes Key: SPARK-27994 URL: https://issues.apache.org/jira/browse/SPARK-27994 Project: Spark

[jira] [Created] (SPARK-27993) Port HIVE-12981 to hive-thriftserver

Yuming Wang created SPARK-27993: --- Summary: Port HIVE-12981 to hive-thriftserver Key: SPARK-27993 URL: https://issues.apache.org/jira/browse/SPARK-27993 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-27217) Nested schema pruning doesn't work for aggregation e.g. `sum`.

[ https://issues.apache.org/jira/browse/SPARK-27217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27217: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Nested schema pruning doesn

[jira] [Updated] (SPARK-27217) Nested schema pruning doesn't work for aggregation e.g. `sum`.

[ https://issues.apache.org/jira/browse/SPARK-27217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27217: -- Priority: Major (was: Minor) > Nested schema pruning doesn't work for aggregation e.g. `sum`.

[jira] [Updated] (SPARK-27217) Nested schema pruning doesn't work for aggregation e.g. `sum`.

[ https://issues.apache.org/jira/browse/SPARK-27217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27217: -- Description: Since SPARK-4502 is fixed, I would expect queries such as `select sum(b.x)` doe

[jira] [Updated] (SPARK-27217) Nested schema pruning doesn't work for aggregation e.g. `sum`.

[ https://issues.apache.org/jira/browse/SPARK-27217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27217: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-25603 > Nested schema pruning d

[jira] [Updated] (SPARK-27290) remove unneed sort under Aggregate

[ https://issues.apache.org/jira/browse/SPARK-27290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27290: -- Affects Version/s: (was: 2.4.0) 3.0.0 > remove unneed sort under Ag

[jira] [Updated] (SPARK-19799) Support WITH clause in subqueries

[ https://issues.apache.org/jira/browse/SPARK-19799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19799: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-27764 > Support WITH clause in

[jira] [Updated] (SPARK-27970) Support Hive 3.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-27970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27970: -- Description: It seems that some users are using Hive 3.0.0, at least HDP 3.0.0 was: It se

[jira] [Resolved] (SPARK-27917) Semantic equals of CaseWhen is failing with case sensitivity of column Names

[ https://issues.apache.org/jira/browse/SPARK-27917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27917. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via https://github.com/apache

[jira] [Assigned] (SPARK-27917) Semantic equals of CaseWhen is failing with case sensitivity of column Names

[ https://issues.apache.org/jira/browse/SPARK-27917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-27917: - Assignee: Sandeep Katta > Semantic equals of CaseWhen is failing with case sensitivity

[jira] [Assigned] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27992: Assignee: (was: Apache Spark) > PySpark socket server should sync with JVM connection

[jira] [Assigned] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27992: Assignee: Apache Spark > PySpark socket server should sync with JVM connection thread fut

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Affects Version/s: (was: 2.4.3) 3.0.0 > PySpark socket server should

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Description: Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spark job ar

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Environment: (was: Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spa

[jira] [Created] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

Bryan Cutler created SPARK-27992: Summary: PySpark socket server should sync with JVM connection thread future Key: SPARK-27992 URL: https://issues.apache.org/jira/browse/SPARK-27992 Project: Spark

[jira] [Updated] (SPARK-27947) Enhance redactOptions to accept any Map type

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27947: -- Priority: Major (was: Minor) > Enhance redactOptions to accept any Map type > ---

[jira] [Resolved] (SPARK-27947) Enhance redactOptions to accept any Map type

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27947. --- Resolution: Fixed Assignee: John Zhuge Fix Version/s: 3.0.0 This is resolved

[jira] [Updated] (SPARK-27947) Enhance redactOptions to accept any Map type

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27947: -- Summary: Enhance redactOptions to accept any Map type (was: ParsedStatement subclass toString

[jira] [Updated] (SPARK-27947) Enhance redactOptions to accept any Map type

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27947: -- Issue Type: Improvement (was: Bug) > Enhance redactOptions to accept any Map type > -

[jira] [Assigned] (SPARK-27845) DataSourceV2: Insert into tables in multiple catalogs

[ https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27845: Assignee: Apache Spark > DataSourceV2: Insert into tables in multiple catalogs >

[jira] [Assigned] (SPARK-27845) DataSourceV2: Insert into tables in multiple catalogs

[ https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27845: Assignee: (was: Apache Spark) > DataSourceV2: Insert into tables in multiple catalogs

[jira] [Commented] (SPARK-27845) DataSourceV2: Insert into tables in multiple catalogs

[ https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860221#comment-16860221 ] Apache Spark commented on SPARK-27845: -- User 'jzhuge' has created a pull request fo

[jira] [Updated] (SPARK-27845) DataSourceV2: Insert into tables in multiple catalogs

[ https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27845: --- Summary: DataSourceV2: Insert into tables in multiple catalogs (was: DataSourceV2: InsertInto multi

[jira] [Created] (SPARK-27991) ShuffleBlockFetcherIterator should take Netty constant-factor overheads into account when limiting number of simultaneous block fetches

Josh Rosen created SPARK-27991: -- Summary: ShuffleBlockFetcherIterator should take Netty constant-factor overheads into account when limiting number of simultaneous block fetches Key: SPARK-27991 URL: https://issues.

[jira] [Closed] (SPARK-27635) Prevent from splitting too many partitions smaller than row group size in Parquet file format

[ https://issues.apache.org/jira/browse/SPARK-27635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-27635. - > Prevent from splitting too many partitions smaller than row group size in > Parquet file format >

[jira] [Resolved] (SPARK-27635) Prevent from splitting too many partitions smaller than row group size in Parquet file format

[ https://issues.apache.org/jira/browse/SPARK-27635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27635. --- Resolution: Invalid Please see the discussion on the PR. > Prevent from splitting too many

[jira] [Resolved] (SPARK-27949) Support SUBSTRING(str FROM n1 [FOR n2]) syntax

[ https://issues.apache.org/jira/browse/SPARK-27949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27949. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via https://github.com/apache

[jira] [Comment Edited] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos edited comment on SPARK-25053 at 6/10/19 1:59 PM:

[jira] [Comment Edited] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos edited comment on SPARK-25053 at 6/10/19 1:58 PM:

[jira] [Resolved] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos resolved SPARK-25053. - Resolution: Duplicate > Allow additional port forwarding on Spark on K8S as need

[jira] [Comment Edited] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos edited comment on SPARK-25053 at 6/10/19 1:57 PM:

[jira] [Comment Edited] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos edited comment on SPARK-25053 at 6/10/19 1:57 PM:

[jira] [Comment Edited] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos edited comment on SPARK-25053 at 6/10/19 1:56 PM:

[jira] [Comment Edited] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos edited comment on SPARK-25053 at 6/10/19 1:55 PM:

[jira] [Commented] (SPARK-25053) Allow additional port forwarding on Spark on K8S as needed

[ https://issues.apache.org/jira/browse/SPARK-25053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860014#comment-16860014 ] Stavros Kontopoulos commented on SPARK-25053: - Sure [~dongjoon]. With the po

[jira] [Commented] (SPARK-27708) Add documentation for v2 data sources

[ https://issues.apache.org/jira/browse/SPARK-27708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860013#comment-16860013 ] Jacek Laskowski commented on SPARK-27708: - [~rdblue] Mind if I asked you to upda

[jira] [Comment Edited] (SPARK-27930) Add built-in Math Function: RANDOM

[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859628#comment-16859628 ] Yuming Wang edited comment on SPARK-27930 at 6/10/19 12:54 PM: ---

[jira] [Resolved] (SPARK-27425) Add count_if functions

[ https://issues.apache.org/jira/browse/SPARK-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27425. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24335 [https://gi

[jira] [Assigned] (SPARK-27425) Add count_if functions

[ https://issues.apache.org/jira/browse/SPARK-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27425: Assignee: Chaerim Yeo > Add count_if functions > -- > >

[jira] [Commented] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859889#comment-16859889 ] zhengruifeng commented on SPARK-27018: -- [~pkolaczk]  With the codes you provided, I

[jira] [Commented] (SPARK-27400) LinearSVC only supports binary classification

[ https://issues.apache.org/jira/browse/SPARK-27400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859878#comment-16859878 ] zhengruifeng commented on SPARK-27400: -- According to current design, LinearSVC only

[jira] [Assigned] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27990: Assignee: (was: Apache Spark) > Provide a way to recursively load data from datasourc

[jira] [Resolved] (SPARK-27867) RegressionEvaluator cache lastest RegressionMetrics to avoid duplicated computation

[ https://issues.apache.org/jira/browse/SPARK-27867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-27867. -- Resolution: Not A Problem > RegressionEvaluator cache lastest RegressionMetrics to avoid dupli

[jira] [Assigned] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27990: Assignee: Apache Spark > Provide a way to recursively load data from datasource > ---

[jira] [Created] (SPARK-27990) Provide a way to recursively load data from datasource

Weichen Xu created SPARK-27990: -- Summary: Provide a way to recursively load data from datasource Key: SPARK-27990 URL: https://issues.apache.org/jira/browse/SPARK-27990 Project: Spark Issue Type

[jira] [Commented] (SPARK-27303) PropertyGraph construction (Scala/Java)

[ https://issues.apache.org/jira/browse/SPARK-27303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859853#comment-16859853 ] Martin Junghanns commented on SPARK-27303: -- [~mengxr] I will work on this, you

[jira] [Assigned] (SPARK-27989) Add retries on the connection to the driver

[ https://issues.apache.org/jira/browse/SPARK-27989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27989: Assignee: (was: Apache Spark) > Add retries on the connection to the driver > ---

  1   2   >