[jira] [Commented] (SPARK-26353) Add typed aggregate functions:max&

[ https://issues.apache.org/jira/browse/SPARK-26353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721033#comment-16721033 ] ASF GitHub Bot commented on SPARK-26353: 10110346 opened a new pull request #23304:

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721107#comment-16721107 ] Kaspar Tint commented on SPARK-26359: - This seems to be related to the fact that we decided to try

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721213#comment-16721213 ] Kaspar Tint commented on SPARK-26359: - Means we changed the configuration

[jira] [Created] (SPARK-26371) Increase Kafka ConfigUpdater test coverage

Gabor Somogyi created SPARK-26371: - Summary: Increase Kafka ConfigUpdater test coverage Key: SPARK-26371 URL: https://issues.apache.org/jira/browse/SPARK-26371 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-26371) Increase Kafka ConfigUpdater test coverage

[ https://issues.apache.org/jira/browse/SPARK-26371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26371: Assignee: (was: Apache Spark) > Increase Kafka ConfigUpdater test coverage >

[jira] [Assigned] (SPARK-26371) Increase Kafka ConfigUpdater test coverage

[ https://issues.apache.org/jira/browse/SPARK-26371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26371: Assignee: Apache Spark > Increase Kafka ConfigUpdater test coverage >

[jira] [Commented] (SPARK-26371) Increase Kafka ConfigUpdater test coverage

[ https://issues.apache.org/jira/browse/SPARK-26371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721240#comment-16721240 ] ASF GitHub Bot commented on SPARK-26371: gaborgsomogyi opened a new pull request #23321:

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721251#comment-16721251 ] Gabor Somogyi commented on SPARK-26359: --- There is also a possibility (apart from delete the

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721195#comment-16721195 ] Gabor Somogyi commented on SPARK-26359: --- Yeah, after the recovery it's advised to switch that

[jira] [Commented] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721258#comment-16721258 ] Steve Loughran commented on SPARK-19739: BTW: I'm not backporting HADOOP-14556 to any non-trunk

[jira] [Assigned] (SPARK-26372) CSV parsing uses previous good value for bad input field

[ https://issues.apache.org/jira/browse/SPARK-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26372: Assignee: Apache Spark > CSV parsing uses previous good value for bad input field >

[jira] [Created] (SPARK-26373) Spark UI 'environment' tab - column to indicate default vs overridden values

t oo created SPARK-26373: Summary: Spark UI 'environment' tab - column to indicate default vs overridden values Key: SPARK-26373 URL: https://issues.apache.org/jira/browse/SPARK-26373 Project: Spark

[jira] [Assigned] (SPARK-26372) CSV parsing uses previous good value for bad input field

[ https://issues.apache.org/jira/browse/SPARK-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26372: Assignee: (was: Apache Spark) > CSV parsing uses previous good value for bad input

[jira] [Commented] (SPARK-26327) Metrics in FileSourceScanExec not update correctly while relation.partitionSchema is set

[ https://issues.apache.org/jira/browse/SPARK-26327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721785#comment-16721785 ] ASF GitHub Bot commented on SPARK-26327: dongjoon-hyun closed pull request #23299:

[jira] [Reopened] (SPARK-20130) Flaky test: BlockManagerProactiveReplicationSuite

[ https://issues.apache.org/jira/browse/SPARK-20130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-20130: Reopening since I've seen this a few times recently. e.g.:

[jira] [Assigned] (SPARK-26267) Kafka source may reprocess data

[ https://issues.apache.org/jira/browse/SPARK-26267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-26267: Assignee: Shixiong Zhu > Kafka source may reprocess data >

[jira] [Commented] (SPARK-24942) Improve cluster resource management with jobs containing barrier stage

[ https://issues.apache.org/jira/browse/SPARK-24942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721841#comment-16721841 ] Ilya Matiach commented on SPARK-24942: -- Would really like to see this resolved.  It would be great

[jira] [Commented] (SPARK-26342) Support for NFS mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721990#comment-16721990 ] Eric Carlson commented on SPARK-26342: -- (discussion also applies for flexvolume improvement

[jira] [Assigned] (SPARK-25922) [K8] Spark Driver/Executor "spark-app-selector" label mismatch

[ https://issues.apache.org/jira/browse/SPARK-25922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25922: Assignee: Apache Spark > [K8] Spark Driver/Executor "spark-app-selector" label mismatch

[jira] [Commented] (SPARK-25922) [K8] Spark Driver/Executor "spark-app-selector" label mismatch

[ https://issues.apache.org/jira/browse/SPARK-25922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721349#comment-16721349 ] Wang, Xinglong commented on SPARK-25922: Hi [~liyinan926], Recently while evaluating spark on

[jira] [Commented] (SPARK-21453) Cached Kafka consumer may be closed too early

[ https://issues.apache.org/jira/browse/SPARK-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721386#comment-16721386 ] Julian commented on SPARK-21453: Also the issue is marked as Major on the other thread, but here only

[jira] [Commented] (SPARK-20890) Add min and max functions for dataset aggregation

[ https://issues.apache.org/jira/browse/SPARK-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721387#comment-16721387 ] ASF GitHub Bot commented on SPARK-20890: srowen closed pull request #18113: [SPARK-20890][SQL]

[jira] [Assigned] (SPARK-25922) [K8] Spark Driver/Executor "spark-app-selector" label mismatch

[ https://issues.apache.org/jira/browse/SPARK-25922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25922: Assignee: (was: Apache Spark) > [K8] Spark Driver/Executor "spark-app-selector"

[jira] [Commented] (SPARK-25922) [K8] Spark Driver/Executor "spark-app-selector" label mismatch

[ https://issues.apache.org/jira/browse/SPARK-25922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721346#comment-16721346 ] ASF GitHub Bot commented on SPARK-25922: suxingfate opened a new pull request #23322:

[jira] [Commented] (SPARK-21453) Cached Kafka consumer may be closed too early

[ https://issues.apache.org/jira/browse/SPARK-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721380#comment-16721380 ] Julian commented on SPARK-21453: I see this issue is also raised against Kafka under 

[jira] [Comment Edited] (SPARK-26342) Support for NFS mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721990#comment-16721990 ] Eric Carlson edited comment on SPARK-26342 at 12/15/18 4:04 AM:

[jira] [Commented] (SPARK-26344) Support for flexVolume mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721992#comment-16721992 ] Eric Carlson commented on SPARK-26344: -- (Discussion at SPARK-26342 - question on whether any types

[jira] [Commented] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722028#comment-16722028 ] ASF GitHub Bot commented on SPARK-26362: asfgit closed pull request #23311: [SPARK-26362][CORE]

[jira] [Commented] (SPARK-26342) Support for NFS mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721997#comment-16721997 ] Yinan Li commented on SPARK-26342: -- Yes, that's true. Feel free to create a PR to add nfs and flex. >

[jira] [Commented] (SPARK-26265) deadlock between TaskMemoryManager and BytesToBytesMap$MapIterator

[ https://issues.apache.org/jira/browse/SPARK-26265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722025#comment-16722025 ] ASF GitHub Bot commented on SPARK-26265: asfgit closed pull request #23294:

[jira] [Resolved] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26362. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23311

[jira] [Assigned] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26362: Assignee: Hyukjin Kwon > Remove 'spark.driver.allowMultipleContexts' to disallow

[jira] [Commented] (SPARK-26372) CSV parsing uses previous good value for bad input field

[ https://issues.apache.org/jira/browse/SPARK-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721858#comment-16721858 ] ASF GitHub Bot commented on SPARK-26372: bersprockets opened a new pull request #23323:

[jira] [Assigned] (SPARK-26267) Kafka source may reprocess data

[ https://issues.apache.org/jira/browse/SPARK-26267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26267: Assignee: Shixiong Zhu (was: Apache Spark) > Kafka source may reprocess data >

[jira] [Assigned] (SPARK-26267) Kafka source may reprocess data

[ https://issues.apache.org/jira/browse/SPARK-26267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26267: Assignee: Apache Spark (was: Shixiong Zhu) > Kafka source may reprocess data >

[jira] [Commented] (SPARK-26267) Kafka source may reprocess data

[ https://issues.apache.org/jira/browse/SPARK-26267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721869#comment-16721869 ] ASF GitHub Bot commented on SPARK-26267: zsxwing opened a new pull request #23324:

[jira] [Commented] (SPARK-26344) Support for flexVolume mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721893#comment-16721893 ] Yinan Li commented on SPARK-26344: -- This is covered by SPARK-24434, which enables using a pod template

[jira] [Commented] (SPARK-26342) Support for NFS mount for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721894#comment-16721894 ] Yinan Li commented on SPARK-26342: -- So basically what you want is a generic way to mount arbitrary

[jira] [Resolved] (SPARK-26290) [K8s] Driver Pods no mounted volumes on submissions from older spark versions

[ https://issues.apache.org/jira/browse/SPARK-26290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li resolved SPARK-26290. -- Resolution: Not A Bug > [K8s] Driver Pods no mounted volumes on submissions from older spark versions

[jira] [Commented] (SPARK-20628) Keep track of nodes which are going to be shut down & avoid scheduling new tasks

[ https://issues.apache.org/jira/browse/SPARK-20628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721909#comment-16721909 ] ASF GitHub Bot commented on SPARK-20628: vanzin closed pull request #19267:

[jira] [Commented] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

[ https://issues.apache.org/jira/browse/SPARK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721575#comment-16721575 ] ASF GitHub Bot commented on SPARK-26370: asfgit closed pull request #23320: [SPARK-26370][SQL]

[jira] [Assigned] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

[ https://issues.apache.org/jira/browse/SPARK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26370: --- Assignee: Takuya Ueshin > Fix resolution of higher-order function for the same identifier.

[jira] [Updated] (SPARK-26372) CSV parsing uses previous good value for bad input field

[ https://issues.apache.org/jira/browse/SPARK-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26372: -- Summary: CSV parsing uses previous good value for bad input field (was: CSV Parsing uses

[jira] [Resolved] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

[ https://issues.apache.org/jira/browse/SPARK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26370. - Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by pull

[jira] [Created] (SPARK-26372) CSV Parsing uses previous good value for bad input field

Bruce Robbins created SPARK-26372: - Summary: CSV Parsing uses previous good value for bad input field Key: SPARK-26372 URL: https://issues.apache.org/jira/browse/SPARK-26372 Project: Spark

[jira] [Commented] (SPARK-26372) CSV parsing uses previous good value for bad input field

[ https://issues.apache.org/jira/browse/SPARK-26372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721593#comment-16721593 ] Bruce Robbins commented on SPARK-26372: --- I can prep a PR, unless someone thinks this needs a

[jira] [Commented] (SPARK-26327) Metrics in FileSourceScanExec not update correctly while relation.partitionSchema is set

[ https://issues.apache.org/jira/browse/SPARK-26327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721788#comment-16721788 ] ASF GitHub Bot commented on SPARK-26327: dongjoon-hyun closed pull request #23300:

[jira] [Updated] (SPARK-26327) Metrics in FileSourceScanExec not update correctly while relation.partitionSchema is set

[ https://issues.apache.org/jira/browse/SPARK-26327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26327: -- Fix Version/s: 2.4.1 2.3.3 2.2.3 > Metrics in

[jira] [Commented] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721656#comment-16721656 ] Howard Wong commented on SPARK-25694: - [~boyangwa], do you know of a workaround for this? I'm facing

[jira] [Commented] (SPARK-21239) Support WAL recover in windows

[ https://issues.apache.org/jira/browse/SPARK-21239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721662#comment-16721662 ] ASF GitHub Bot commented on SPARK-21239: vanzin closed pull request #18452:

[jira] [Resolved] (SPARK-21239) Support WAL recover in windows

[ https://issues.apache.org/jira/browse/SPARK-21239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21239. Resolution: Duplicate As far as my understanding goes the fix for SPARK-25778 should also