[jira] [Commented] (SPARK-25282) Fix support for spark-shell with K8s

[ https://issues.apache.org/jira/browse/SPARK-25282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629816#comment-16629816 ] Prashant Sharma commented on SPARK-25282: - The reason, I was running into this,

[jira] [Resolved] (SPARK-25485) Refactor UnsafeProjectionBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25485. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22493 [https://

[jira] [Assigned] (SPARK-25485) Refactor UnsafeProjectionBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25485: - Assignee: yucai > Refactor UnsafeProjectionBenchmark to use main method > -

[jira] [Commented] (SPARK-23002) SparkUI inconsistent driver hostname compare with other executors

[ https://issues.apache.org/jira/browse/SPARK-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629804#comment-16629804 ] Adam Wang commented on SPARK-23002: --- I find that this bug existed in the version spark

[jira] [Updated] (SPARK-23002) SparkUI inconsistent driver hostname compare with other executors

[ https://issues.apache.org/jira/browse/SPARK-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Wang updated SPARK-23002: -- Attachment: image-2018-09-27-14-14-10-640.png > SparkUI inconsistent driver hostname compare with othe

[jira] [Updated] (SPARK-23002) SparkUI inconsistent driver hostname compare with other executors

[ https://issues.apache.org/jira/browse/SPARK-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Wang updated SPARK-23002: -- Attachment: image-2018-09-27-14-12-52-854.png > SparkUI inconsistent driver hostname compare with othe

[jira] [Assigned] (SPARK-25468) Highlight current page index in the history server

[ https://issues.apache.org/jira/browse/SPARK-25468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25468: - Assignee: Adam Wang > Highlight current page index in the history server >

[jira] [Resolved] (SPARK-25468) Highlight current page index in the history server

[ https://issues.apache.org/jira/browse/SPARK-25468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25468. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22516 [https://github.c

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629749#comment-16629749 ] Hyukjin Kwon commented on SPARK-18112: -- Hive 3 support. See https://github.com/apac

[jira] [Comment Edited] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629742#comment-16629742 ] Hyukjin Kwon edited comment on SPARK-18112 at 9/27/18 4:42 AM: ---

[jira] [Updated] (SPARK-25540) Make HiveContext in PySpark behave as the same as Scala.

[ https://issues.apache.org/jira/browse/SPARK-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25540: - Fix Version/s: (was: 2.4.0) 2.5.0 > Make HiveContext in PySpark behave as

[jira] [Assigned] (SPARK-25525) Do not update conf for existing SparkContext in SparkSession.getOrCreate.

[ https://issues.apache.org/jira/browse/SPARK-25525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25525: Assignee: Takuya Ueshin > Do not update conf for existing SparkContext in SparkSession.ge

[jira] [Resolved] (SPARK-25525) Do not update conf for existing SparkContext in SparkSession.getOrCreate.

[ https://issues.apache.org/jira/browse/SPARK-25525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25525. -- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22545 [https://gi

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629743#comment-16629743 ] Hyukjin Kwon commented on SPARK-18112: -- Re: https://issues.apache.org/jira/browse/

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629742#comment-16629742 ] Hyukjin Kwon commented on SPARK-18112: -- Hive 3 support is blocked by Hadoop 3 profi

[jira] [Commented] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629740#comment-16629740 ] Dongjoon Hyun commented on SPARK-25536: --- Issue resolved by pull request 22555 [htt

[jira] [Assigned] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25536: - Assignee: shahid > executorSource.METRIC read wrong record in Executor.scala Line444 >

[jira] [Resolved] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25536. --- Resolution: Fixed > executorSource.METRIC read wrong record in Executor.scala Line444 >

[jira] [Updated] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25536: -- Fix Version/s: 2.4.0 2.3.3 > executorSource.METRIC read wrong record in Exe

[jira] [Resolved] (SPARK-25481) Refactor ColumnarBatchBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25481. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22490 [https://

[jira] [Assigned] (SPARK-25481) Refactor ColumnarBatchBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25481: - Assignee: yucai > Refactor ColumnarBatchBenchmark to use main method >

[jira] [Commented] (SPARK-25549) High level API to collect RDD statistics

[ https://issues.apache.org/jira/browse/SPARK-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629702#comment-16629702 ] Liang-Chi Hsieh commented on SPARK-25549: - cc [~cloud_fan]   > High level API

[jira] [Commented] (SPARK-25549) High level API to collect RDD statistics

[ https://issues.apache.org/jira/browse/SPARK-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629700#comment-16629700 ] Liang-Chi Hsieh commented on SPARK-25549: - The design doc is at: https://docs.g

[jira] [Created] (SPARK-25549) High level API to collect RDD statistics

Liang-Chi Hsieh created SPARK-25549: --- Summary: High level API to collect RDD statistics Key: SPARK-25549 URL: https://issues.apache.org/jira/browse/SPARK-25549 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-24341) Codegen compile error from predicate subquery

[ https://issues.apache.org/jira/browse/SPARK-24341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629698#comment-16629698 ] Apache Spark commented on SPARK-24341: -- User 'cloud-fan' has created a pull request

[jira] [Updated] (SPARK-25541) CaseInsensitiveMap should be serializable after '-' operator

[ https://issues.apache.org/jira/browse/SPARK-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-25541: --- Summary: CaseInsensitiveMap should be serializable after '-' operator (was: CaseInsensitive

[jira] [Commented] (SPARK-25541) CaseInsensitiveMap should be serializable after '-' or 'filterKeys'

[ https://issues.apache.org/jira/browse/SPARK-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629690#comment-16629690 ] Apache Spark commented on SPARK-25541: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25548: Assignee: Apache Spark > In the PruneFileSourcePartitions optimizer, replace the nonParti

[jira] [Assigned] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25548: Assignee: Apache Spark > In the PruneFileSourcePartitions optimizer, replace the nonParti

[jira] [Assigned] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25548: Assignee: (was: Apache Spark) > In the PruneFileSourcePartitions optimizer, replace t

[jira] [Commented] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629655#comment-16629655 ] Apache Spark commented on SPARK-25548: -- User 'eatoncys' has created a pull request

[jira] [Created] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

eaton created SPARK-25548: - Summary: In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned Key: SPARK-25548

[jira] [Resolved] (SPARK-25540) Make HiveContext in PySpark behave as the same as Scala.

[ https://issues.apache.org/jira/browse/SPARK-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25540. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22552 [https://gith

[jira] [Assigned] (SPARK-25540) Make HiveContext in PySpark behave as the same as Scala.

[ https://issues.apache.org/jira/browse/SPARK-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25540: --- Assignee: Takuya Ueshin > Make HiveContext in PySpark behave as the same as Scala. > --

[jira] [Reopened] (SPARK-25454) Division between operands with negative scale can cause precision loss

[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-25454: - Assignee: (was: Wenchen Fan) I'm reopening it, since the bug is not fully fixed. But we do

[jira] [Updated] (SPARK-25454) Division between operands with negative scale can cause precision loss

[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25454: Fix Version/s: (was: 2.3.3) (was: 2.4.0) > Division between operands wi

[jira] [Resolved] (SPARK-25454) Division between operands with negative scale can cause precision loss

[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25454. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.4.0 2.3.3 >

[jira] [Commented] (SPARK-25351) Handle Pandas category type when converting from Python with Arrow

[ https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629572#comment-16629572 ] Bryan Cutler commented on SPARK-25351: -- Hi [~pgadige], yes please go ahead with thi

[jira] [Assigned] (SPARK-25372) Deprecate Yarn-specific configs in regards to keytab login for SparkSubmit

[ https://issues.apache.org/jira/browse/SPARK-25372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25372: -- Assignee: Ilan Filonenko > Deprecate Yarn-specific configs in regards to keytab login

[jira] [Resolved] (SPARK-25372) Deprecate Yarn-specific configs in regards to keytab login for SparkSubmit

[ https://issues.apache.org/jira/browse/SPARK-25372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25372. Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22362 [https:

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629561#comment-16629561 ] Steven Rand commented on SPARK-25538: - [~kiszk], yes, the schema is:   {code} scala

[jira] [Commented] (SPARK-25531) new write APIs for data source v2

[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629550#comment-16629550 ] Wenchen Fan commented on SPARK-25531: - I want to have a more structured view of the

[jira] [Updated] (SPARK-24285) Flaky test: ContinuousSuite.query without test harness

[ https://issues.apache.org/jira/browse/SPARK-24285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24285: -- Description: *2.5.0-SNAPSHOT* - [https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestB

[jira] [Updated] (SPARK-24285) Flaky test: ContinuousSuite.query without test harness

[ https://issues.apache.org/jira/browse/SPARK-24285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24285: -- Description: *2.5.0-SNAPSHOT* - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBui

[jira] [Assigned] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25547: Assignee: (was: Apache Spark) > Pluggable jdbc connection factory > -

[jira] [Assigned] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25547: Assignee: Apache Spark > Pluggable jdbc connection factory >

[jira] [Commented] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629425#comment-16629425 ] Apache Spark commented on SPARK-25547: -- User 'fsauer65' has created a pull request

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629420#comment-16629420 ] t oo commented on SPARK-18112: -- here, here! > Spark2.x does not support read data from Hiv

[jira] [Created] (SPARK-25547) Pluggable jdbc connection factory

Frank Sauer created SPARK-25547: --- Summary: Pluggable jdbc connection factory Key: SPARK-25547 URL: https://issues.apache.org/jira/browse/SPARK-25547 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25531) new write APIs for data source v2

[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629418#comment-16629418 ] Ryan Blue commented on SPARK-25531: --- [~cloud_fan], what was the intent for this umbrel

[jira] [Commented] (SPARK-17952) SparkSession createDataFrame method throws exception for nested JavaBeans

[ https://issues.apache.org/jira/browse/SPARK-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629321#comment-16629321 ] Michal Šenkýř commented on SPARK-17952: --- Implemented nested bean support in pull r

[jira] [Commented] (SPARK-25501) Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-25501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629320#comment-16629320 ] Mingjie Tang commented on SPARK-25501: -- [~gsomogyi] Thanks for your reply. At fir

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629281#comment-16629281 ] Kazuaki Ishizaki commented on SPARK-25538: -- Hi [~Steven Rand], would it be poss

[jira] [Commented] (SPARK-25533) Inconsistent message for Completed Jobs in the JobUI, when there are failed jobs, compared to spark2.2

[ https://issues.apache.org/jira/browse/SPARK-25533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629243#comment-16629243 ] Marcelo Vanzin commented on SPARK-25533: This is merged to master. I'll backport

[jira] [Assigned] (SPARK-25533) Inconsistent message for Completed Jobs in the JobUI, when there are failed jobs, compared to spark2.2

[ https://issues.apache.org/jira/browse/SPARK-25533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25533: -- Assignee: shahid > Inconsistent message for Completed Jobs in the JobUI, when there

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629231#comment-16629231 ] David Spies commented on SPARK-18492: - Ran into this as well. It seems like this is

[jira] [Issue Comment Deleted] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-25546: --- Comment: was deleted (was: User 'vanzin' has created a pull request for this issue: https://

[jira] [Commented] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629218#comment-16629218 ] Apache Spark commented on SPARK-25546: -- User 'vanzin' has created a pull request fo

[jira] [Commented] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629216#comment-16629216 ] Apache Spark commented on SPARK-25546: -- User 'vanzin' has created a pull request fo

[jira] [Assigned] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25546: Assignee: (was: Apache Spark) > RDDInfo uses SparkEnv before it may have been initial

[jira] [Assigned] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25546: Assignee: Apache Spark > RDDInfo uses SparkEnv before it may have been initialized >

[jira] [Created] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

Marcelo Vanzin created SPARK-25546: -- Summary: RDDInfo uses SparkEnv before it may have been initialized Key: SPARK-25546 URL: https://issues.apache.org/jira/browse/SPARK-25546 Project: Spark

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629198#comment-16629198 ] Huaxin Gao commented on SPARK-21291: [~felixcheung] I will submit a PR for bucketBy.

[jira] [Updated] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25536: -- Affects Version/s: 2.3.0 2.3.1 > executorSource.METRIC read wrong recor

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629182#comment-16629182 ] Leo Gallucci commented on SPARK-18112: -- And to get things worse Hive is already in

[jira] [Commented] (SPARK-25535) Work around bad error checking in commons-crypto

[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629174#comment-16629174 ] Apache Spark commented on SPARK-25535: -- User 'vanzin' has created a pull request fo

[jira] [Assigned] (SPARK-25535) Work around bad error checking in commons-crypto

[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25535: Assignee: (was: Apache Spark) > Work around bad error checking in commons-crypto > --

[jira] [Assigned] (SPARK-25535) Work around bad error checking in commons-crypto

[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25535: Assignee: Apache Spark > Work around bad error checking in commons-crypto > -

[jira] [Resolved] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

[ https://issues.apache.org/jira/browse/SPARK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25318. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22325 [https:

[jira] [Assigned] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

[ https://issues.apache.org/jira/browse/SPARK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25318: -- Assignee: Reza Safi > Add exception handling when wrapping the input stream during th

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629000#comment-16629000 ] Eugeniu commented on SPARK-18112: - I can only describe my situation. I am using AWS EMR

[jira] [Updated] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Crosby updated SPARK-25544: -- Description: The LinearRegression and LogisticRegression estimators in Spark ML can take a la

[jira] [Resolved] (SPARK-25509) SHS V2 cannot enabled in Windows, because POSIX permissions is not support.

[ https://issues.apache.org/jira/browse/SPARK-25509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25509. --- Resolution: Fixed Fix Version/s: 2.4.0 2.3.3 Issue resolved by pull reques

[jira] [Assigned] (SPARK-25509) SHS V2 cannot enabled in Windows, because POSIX permissions is not support.

[ https://issues.apache.org/jira/browse/SPARK-25509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25509: - Assignee: Rong Tang > SHS V2 cannot enabled in Windows, because POSIX permissions is not suppor

[jira] [Commented] (SPARK-25545) CSV loading with DROPMALFORMED mode doesn't correctly drop rows that do not confirm to non-nullable schema fields

[ https://issues.apache.org/jira/browse/SPARK-25545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628949#comment-16628949 ] Steven Bakhtiari commented on SPARK-25545: -- Somebody on SO pointed me to this o

[jira] [Created] (SPARK-25545) CSV loading with DROPMALFORMED mode doesn't correctly drop rows that do not confirm to non-nullable schema fields

Steven Bakhtiari created SPARK-25545: Summary: CSV loading with DROPMALFORMED mode doesn't correctly drop rows that do not confirm to non-nullable schema fields Key: SPARK-25545 URL: https://issues.apache.org/

[jira] [Updated] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Crosby updated SPARK-25544: -- Description: The LinearRegression and LogisticRegression estimators in Spark ML can take a la

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628865#comment-16628865 ] Hyukjin Kwon commented on SPARK-18112: -- Can you post reproducer steps please before

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628852#comment-16628852 ] Eugeniu commented on SPARK-18112: - This issue should be reopened. As already commented

[jira] [Assigned] (SPARK-20937) Describe spark.sql.parquet.writeLegacyFormat property in Spark SQL, DataFrames and Datasets Guide

[ https://issues.apache.org/jira/browse/SPARK-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-20937: Assignee: Chenxiao Mao > Describe spark.sql.parquet.writeLegacyFormat property in Spark S

[jira] [Resolved] (SPARK-20937) Describe spark.sql.parquet.writeLegacyFormat property in Spark SQL, DataFrames and Datasets Guide

[ https://issues.apache.org/jira/browse/SPARK-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20937. -- Resolution: Fixed Fix Version/s: 2.4.1 2.5.0 Issue resolved by pull

[jira] [Updated] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Crosby updated SPARK-25544: -- Description: The LinearRegression and LogisticRegression estimators in Spark ML can take a la

[jira] [Created] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

Andrew Crosby created SPARK-25544: - Summary: Slow/failed convergence in Spark ML models due to internal predictor scaling Key: SPARK-25544 URL: https://issues.apache.org/jira/browse/SPARK-25544 Projec

[jira] [Resolved] (SPARK-25379) Improve ColumnPruning performance

[ https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25379. - Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22364 [https://gith

[jira] [Assigned] (SPARK-25379) Improve ColumnPruning performance

[ https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25379: --- Assignee: Marco Gaido > Improve ColumnPruning performance > ---

[jira] [Updated] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-25392: - Description: Steps: 1.Enable spark.scheduler.mode = FAIR 2.Submitted beeline jo

[jira] [Commented] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628742#comment-16628742 ] sandeep katta commented on SPARK-25392: --- [~abhishek.akg] as per current design poo

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628744#comment-16628744 ] Wenchen Fan commented on SPARK-25538: - cc [~kiszk] as well > incorrect row counts a

[jira] [Commented] (SPARK-23401) Improve test cases for all supported types and unsupported types

[ https://issues.apache.org/jira/browse/SPARK-23401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628727#comment-16628727 ] Aleksandr Koriagin commented on SPARK-23401: I will take a look > Improve t

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628721#comment-16628721 ] Felix Cheung commented on SPARK-21291: -- The PR did not have bucketBy? > R bucke

[jira] [Resolved] (SPARK-25541) CaseInsensitiveMap should be serializable after '-' or 'filterKeys'

[ https://issues.apache.org/jira/browse/SPARK-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25541. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.5.0 > CaseInsensitiv

[jira] [Commented] (SPARK-24440) When use constant as column we may get wrong answer versus impala

[ https://issues.apache.org/jira/browse/SPARK-24440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628608#comment-16628608 ] Marco Gaido commented on SPARK-24440: - Can you provide a sample repro which can be r

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628588#comment-16628588 ] shahid commented on SPARK-25502: [~toopt4] No. please refer the PR, to see the fix > [S

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628561#comment-16628561 ] t oo commented on SPARK-25502: -- related https://jira.apache.org/jira/browse/SPARK-16859 ?

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:46 AM: bump w

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:45 AM: bump [~a

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:43 AM: bump @sh

[jira] [Commented] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628554#comment-16628554 ] t oo commented on SPARK-16859: -- bump > History Server storage information is missing > ---

[jira] [Commented] (SPARK-25452) Query with where clause is giving unexpected result in case of float column

[ https://issues.apache.org/jira/browse/SPARK-25452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628525#comment-16628525 ] Ayush Anubhava commented on SPARK-25452: Hi HyukjiKwon This issue does not seem

[jira] [Created] (SPARK-25543) Confusing log messages at DEBUG level, in K8s mode.

Prashant Sharma created SPARK-25543: --- Summary: Confusing log messages at DEBUG level, in K8s mode. Key: SPARK-25543 URL: https://issues.apache.org/jira/browse/SPARK-25543 Project: Spark Iss

  1   2   >