[jira] [Created] (SPARK-22751) Improve ML RandomForest shuffle performance

lucio35 created SPARK-22751: --- Summary: Improve ML RandomForest shuffle performance Key: SPARK-22751 URL: https://issues.apache.org/jira/browse/SPARK-22751 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285631#comment-16285631 ] lucio35 commented on SPARK-22751: - If this improvement is necessary, I can do this work,

[jira] [Resolved] (SPARK-22718) Expose mechanism for testing timeout based testcases in StreamTest

[ https://issues.apache.org/jira/browse/SPARK-22718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Ahern resolved SPARK-22718. - Resolution: Not A Problem > Expose mechanism for testing timeout based testcases in StreamTest >

[jira] [Commented] (SPARK-22718) Expose mechanism for testing timeout based testcases in StreamTest

[ https://issues.apache.org/jira/browse/SPARK-22718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285691#comment-16285691 ] David Ahern commented on SPARK-22718: - ignore - i found how to test this https://gith

[jira] [Created] (SPARK-22752) FileNotFoundException while reading from Kafka

Marco Gaido created SPARK-22752: --- Summary: FileNotFoundException while reading from Kafka Key: SPARK-22752 URL: https://issues.apache.org/jira/browse/SPARK-22752 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285729#comment-16285729 ] Marco Gaido commented on SPARK-22751: - You can submit a PR on github, if you have a w

[jira] [Commented] (SPARK-9299) percentile and percentile_approx aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285755#comment-16285755 ] Herman van Hovell commented on SPARK-9299: -- [~hyukjin.kwon] This can be safely cl

[jira] [Resolved] (SPARK-9299) percentile and percentile_approx aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-9299. -- Resolution: Fixed Fix Version/s: 2.1.0 > percentile and percentile_approx aggrega

[jira] [Commented] (SPARK-16738) Queryable state for Spark State Store

[ https://issues.apache.org/jira/browse/SPARK-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285791#comment-16285791 ] Stavros Kontopoulos commented on SPARK-16738: - I have opened recently the [h

[jira] [Comment Edited] (SPARK-16738) Queryable state for Spark State Store

[ https://issues.apache.org/jira/browse/SPARK-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285791#comment-16285791 ] Stavros Kontopoulos edited comment on SPARK-16738 at 12/11/17 11:26 AM: ---

[jira] [Commented] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

[ https://issues.apache.org/jira/browse/SPARK-13809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285798#comment-16285798 ] Stavros Kontopoulos commented on SPARK-13809: - Any reason why RockDB or some

[jira] [Comment Edited] (SPARK-16738) Queryable state for Spark State Store

[ https://issues.apache.org/jira/browse/SPARK-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285791#comment-16285791 ] Stavros Kontopoulos edited comment on SPARK-16738 at 12/11/17 11:31 AM: ---

[jira] [Comment Edited] (SPARK-13809) State Store: A new framework for state management for computing Streaming Aggregates

[ https://issues.apache.org/jira/browse/SPARK-13809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285798#comment-16285798 ] Stavros Kontopoulos edited comment on SPARK-13809 at 12/11/17 11:32 AM: ---

[jira] [Updated] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22751: -- Priority: Minor (was: Major) It looks somewhat difficult to change this at first glance, but findSpli

[jira] [Created] (SPARK-22753) Get rid of dataSource.writeAndRead

Li Yuanjian created SPARK-22753: --- Summary: Get rid of dataSource.writeAndRead Key: SPARK-22753 URL: https://issues.apache.org/jira/browse/SPARK-22753 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-22753) Get rid of dataSource.writeAndRead

[ https://issues.apache.org/jira/browse/SPARK-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285833#comment-16285833 ] Apache Spark commented on SPARK-22753: -- User 'xuanyuanking' has created a pull reque

[jira] [Assigned] (SPARK-22753) Get rid of dataSource.writeAndRead

[ https://issues.apache.org/jira/browse/SPARK-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22753: Assignee: Apache Spark > Get rid of dataSource.writeAndRead >

[jira] [Assigned] (SPARK-22753) Get rid of dataSource.writeAndRead

[ https://issues.apache.org/jira/browse/SPARK-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22753: Assignee: (was: Apache Spark) > Get rid of dataSource.writeAndRead > -

[jira] [Resolved] (SPARK-22727) spark.executor.instances's default value should be 2

[ https://issues.apache.org/jira/browse/SPARK-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22727. --- Resolution: Not A Problem > spark.executor.instances's default value should be 2 >

[jira] [Updated] (SPARK-22691) Custom HttpFileSystem, issue with question-marks in path

[ https://issues.apache.org/jira/browse/SPARK-22691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22691: -- Issue Type: Improvement (was: Bug) I doubt the code will change to accommodate this, especially if it

[jira] [Commented] (SPARK-1145) Memory mapping with many small blocks can cause JVM allocation failures

[ https://issues.apache.org/jira/browse/SPARK-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285846#comment-16285846 ] Ismael Juma commented on SPARK-1145: Thanks for the additional information. The issue

[jira] [Created] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

zhoukang created SPARK-22754: Summary: Check spark.executor.heartbeatInterval setting in case of ExecutorLost Key: SPARK-22754 URL: https://issues.apache.org/jira/browse/SPARK-22754 Project: Spark

[jira] [Commented] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

[ https://issues.apache.org/jira/browse/SPARK-22754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285853#comment-16285853 ] Apache Spark commented on SPARK-22754: -- User 'caneGuy' has created a pull request fo

[jira] [Assigned] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

[ https://issues.apache.org/jira/browse/SPARK-22754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22754: Assignee: (was: Apache Spark) > Check spark.executor.heartbeatInterval setting in case

[jira] [Assigned] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

[ https://issues.apache.org/jira/browse/SPARK-22754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22754: Assignee: Apache Spark > Check spark.executor.heartbeatInterval setting in case of Executo

[jira] [Commented] (SPARK-22751) Improve ML RandomForest shuffle performance

[ https://issues.apache.org/jira/browse/SPARK-22751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285858#comment-16285858 ] lucio35 commented on SPARK-22751: - The reason i suggest reduceByKey is that we only need

[jira] [Issue Comment Deleted] (SPARK-1145) Memory mapping with many small blocks can cause JVM allocation failures

[ https://issues.apache.org/jira/browse/SPARK-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated SPARK-1145: --- Comment: was deleted (was: Thanks for the additional information. The issue seems similar to SPARK-11

[jira] [Commented] (SPARK-14228) Lost executor of RPC disassociated, and occurs exception: Could not find CoarseGrainedScheduler or it has been stopped

[ https://issues.apache.org/jira/browse/SPARK-14228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285920#comment-16285920 ] KaiXinXIaoLei commented on SPARK-14228: --- Using this patch, this problem is still ex

[jira] [Assigned] (SPARK-22267) Spark SQL incorrectly reads ORC file when column order is different

[ https://issues.apache.org/jira/browse/SPARK-22267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22267: --- Assignee: Dongjoon Hyun > Spark SQL incorrectly reads ORC file when column order is differen

[jira] [Resolved] (SPARK-22267) Spark SQL incorrectly reads ORC file when column order is different

[ https://issues.apache.org/jira/browse/SPARK-22267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22267. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19928 [https://githu

[jira] [Commented] (SPARK-21638) Warning message of RF is not accurate

[ https://issues.apache.org/jira/browse/SPARK-21638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285942#comment-16285942 ] Apache Spark commented on SPARK-21638: -- User 'mpjlu' has created a pull request for

[jira] [Updated] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

[ https://issues.apache.org/jira/browse/SPARK-22754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-22754: - Description: If spark.executor.heartbeatInterval bigger than spark.network.timeout,it will almost always

[jira] [Comment Edited] (SPARK-14228) Lost executor of RPC disassociated, and occurs exception: Could not find CoarseGrainedScheduler or it has been stopped

[ https://issues.apache.org/jira/browse/SPARK-14228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285920#comment-16285920 ] KaiXinXIaoLei edited comment on SPARK-14228 at 12/11/17 3:17 PM: --

[jira] [Commented] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286032#comment-16286032 ] Julien Cuquemelle commented on SPARK-22683: --- I'd like to point out that when re

[jira] [Comment Edited] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286032#comment-16286032 ] Julien Cuquemelle edited comment on SPARK-22683 at 12/11/17 3:19 PM: --

[jira] [Comment Edited] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286032#comment-16286032 ] Julien Cuquemelle edited comment on SPARK-22683 at 12/11/17 3:20 PM: --

[jira] [Comment Edited] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286032#comment-16286032 ] Julien Cuquemelle edited comment on SPARK-22683 at 12/11/17 3:21 PM: --

[jira] [Commented] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286058#comment-16286058 ] Sean Owen commented on SPARK-22683: --- This is still optimizing for a particular type of

[jira] [Created] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

Kevin Zhang created SPARK-22755: --- Summary: Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results Key: SPARK-22755 URL: https://issues.apache.org/jira/browse/SPARK-22755

[jira] [Commented] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286253#comment-16286253 ] Julien Cuquemelle commented on SPARK-22683: --- The impression I get from our disc

[jira] [Commented] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286256#comment-16286256 ] Sean Owen commented on SPARK-22755: --- I assume the problem is precision, that the last e

[jira] [Commented] (SPARK-14228) Lost executor of RPC disassociated, and occurs exception: Could not find CoarseGrainedScheduler or it has been stopped

[ https://issues.apache.org/jira/browse/SPARK-14228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286282#comment-16286282 ] Devaraj K commented on SPARK-14228: --- [~KaiXinXIaoLei], Thanks for checking this. Is the

[jira] [Commented] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

[ https://issues.apache.org/jira/browse/SPARK-22742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286455#comment-16286455 ] Xiao Li commented on SPARK-22742: - This is for supporting Hive metastore 2.2 and 2.3. The

[jira] [Reopened] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

[ https://issues.apache.org/jira/browse/SPARK-22742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-22742: - > Spark2.x does not support read data from Hive 2.2 and 2.3 > ---

[jira] [Updated] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

[ https://issues.apache.org/jira/browse/SPARK-22742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22742: Issue Type: New Feature (was: Bug) > Spark2.x does not support read data from Hive 2.2 and 2.3 > -

[jira] [Updated] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

[ https://issues.apache.org/jira/browse/SPARK-22742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22742: Description: Hive has been release latest version 2.3.2 but spark doesn't support read from metadata store

[jira] [Assigned] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

[ https://issues.apache.org/jira/browse/SPARK-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-22642: - Assignee: zuotingbing > the createdTempDir will not be deleted if an exception occurs >

[jira] [Resolved] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

[ https://issues.apache.org/jira/browse/SPARK-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22642. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19841 [https://github.co

[jira] [Commented] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286471#comment-16286471 ] Sunitha Kambhampati commented on SPARK-22755: - I just tried this in sql on th

[jira] [Updated] (SPARK-22606) There may be two or more tasks in one executor will use the same kafka consumer at the same time, then it will throw an exception: "KafkaConsumer is not safe for multi-t

[ https://issues.apache.org/jira/browse/SPARK-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22606: - Component/s: (was: Structured Streaming) DStreams > There may be two or more

[jira] [Created] (SPARK-22756) Run SparkR tests if hive_thriftserver module has code changes

Xiao Li created SPARK-22756: --- Summary: Run SparkR tests if hive_thriftserver module has code changes Key: SPARK-22756 URL: https://issues.apache.org/jira/browse/SPARK-22756 Project: Spark Issue Ty

[jira] [Created] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

Yinan Li created SPARK-22757: Summary: Init-container in the driver/executor pods for downloading remote dependencies Key: SPARK-22757 URL: https://issues.apache.org/jira/browse/SPARK-22757 Project: Spark

[jira] [Commented] (SPARK-22752) FileNotFoundException while reading from Kafka

[ https://issues.apache.org/jira/browse/SPARK-22752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286584#comment-16286584 ] Shixiong Zhu commented on SPARK-22752: -- What's your "checkpointLocation"? Is it usin

[jira] [Commented] (SPARK-16060) Vectorized Orc reader

[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286695#comment-16286695 ] Apache Spark commented on SPARK-16060: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286680#comment-16286680 ] Ruslan Dautkhanov commented on SPARK-4502: -- Would somebody be available to review

[jira] [Created] (SPARK-22758) New Spark Jira component for Kubernetes

Yinan Li created SPARK-22758: Summary: New Spark Jira component for Kubernetes Key: SPARK-22758 URL: https://issues.apache.org/jira/browse/SPARK-22758 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-22758) New Spark Jira component for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-22758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22758: -- Component/s: Kubernetes > New Spark Jira component for Kubernetes > ---

[jira] [Updated] (SPARK-22646) Spark on Kubernetes - basic submission client

[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-22646: --- Component/s: (was: Scheduler) Kubernetes > Spark on Kubernetes - basic s

[jira] [Updated] (SPARK-22756) Run SparkR tests if hive_thriftserver module has code changes

[ https://issues.apache.org/jira/browse/SPARK-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22756: Description: The recent PR change in hive_thriftserver caused the test failure in CRAN requirements. To so

[jira] [Assigned] (SPARK-22756) Run SparkR tests if hive_thriftserver module has code changes

[ https://issues.apache.org/jira/browse/SPARK-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22756: Assignee: Xiao Li (was: Apache Spark) > Run SparkR tests if hive_thriftserver module has

[jira] [Commented] (SPARK-22756) Run SparkR tests if hive_thriftserver module has code changes

[ https://issues.apache.org/jira/browse/SPARK-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286742#comment-16286742 ] Apache Spark commented on SPARK-22756: -- User 'gatorsmile' has created a pull request

[jira] [Updated] (SPARK-22756) Run SparkR tests if hive_thriftserver module has code changes

[ https://issues.apache.org/jira/browse/SPARK-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22756: Description: SparkR module depends on hive_thriftserver module, so we should run hive_thriftserver tests i

[jira] [Assigned] (SPARK-22756) Run SparkR tests if hive_thriftserver module has code changes

[ https://issues.apache.org/jira/browse/SPARK-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22756: Assignee: Apache Spark (was: Xiao Li) > Run SparkR tests if hive_thriftserver module has

[jira] [Updated] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-22757: - Component/s: Kubernetes > Init-container in the driver/executor pods for downloading remote dependencies

[jira] [Assigned] (SPARK-22646) Spark on Kubernetes - basic submission client

[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22646: -- Assignee: Yinan Li > Spark on Kubernetes - basic submission client > -

[jira] [Commented] (SPARK-20640) Make rpc timeout and retry for shuffle registration configurable

[ https://issues.apache.org/jira/browse/SPARK-20640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286807#comment-16286807 ] Xuefu Zhang commented on SPARK-20640: - [~lyc], thanks for fixing this. I'm wondering

[jira] [Resolved] (SPARK-22646) Spark on Kubernetes - basic submission client

[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22646. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19717 [https:/

[jira] [Resolved] (SPARK-22746) Avoid the generation of useless mutable states by SortMergeJoin

[ https://issues.apache.org/jira/browse/SPARK-22746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22746. - Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.3.0 > Avoid the generation

[jira] [Resolved] (SPARK-22758) New Spark Jira component for Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-22758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22758. Resolution: Fixed Done. > New Spark Jira component for Kubernetes > --

[jira] [Commented] (SPARK-22648) Documentation for Kubernetes Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286846#comment-16286846 ] Apache Spark commented on SPARK-22648: -- User 'foxish' has created a pull request for

[jira] [Resolved] (SPARK-20557) JdbcUtils doesn't support java.sql.Types.TIMESTAMP_WITH_TIMEZONE

[ https://issues.apache.org/jira/browse/SPARK-20557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20557. - Resolution: Fixed > JdbcUtils doesn't support java.sql.Types.TIMESTAMP_WITH_TIMEZONE > --

[jira] [Resolved] (SPARK-22726) Basic tests for Binary Comparison and ImplicitTypeCasts

[ https://issues.apache.org/jira/browse/SPARK-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22726. - Resolution: Fixed > Basic tests for Binary Comparison and ImplicitTypeCasts > ---

[jira] [Updated] (SPARK-22647) Docker files for image creation

[ https://issues.apache.org/jira/browse/SPARK-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-22647: --- Component/s: (was: Scheduler) Kubernetes > Docker files for image creati

[jira] [Updated] (SPARK-22648) Documentation for Kubernetes Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-22648: --- Component/s: Kubernetes > Documentation for Kubernetes Scheduler Backend > --

[jira] [Closed] (SPARK-22174) Support to automatically create the directory where the event logs go (`spark.eventLog.dir`)

[ https://issues.apache.org/jira/browse/SPARK-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing closed SPARK-22174. --- > Support to automatically create the directory where the event logs go > (`spark.eventLog.dir`) >

[jira] [Commented] (SPARK-22759) Filters can be combined iff both are deterministic

[ https://issues.apache.org/jira/browse/SPARK-22759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286885#comment-16286885 ] Apache Spark commented on SPARK-22759: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-22759) Filters can be combined iff both are deterministic

[ https://issues.apache.org/jira/browse/SPARK-22759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22759: Assignee: Xiao Li (was: Apache Spark) > Filters can be combined iff both are deterministi

[jira] [Created] (SPARK-22759) Filters can be combined iff both are deterministic

Xiao Li created SPARK-22759: --- Summary: Filters can be combined iff both are deterministic Key: SPARK-22759 URL: https://issues.apache.org/jira/browse/SPARK-22759 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-22759) Filters can be combined iff both are deterministic

[ https://issues.apache.org/jira/browse/SPARK-22759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22759: Assignee: Apache Spark (was: Xiao Li) > Filters can be combined iff both are deterministi

[jira] [Updated] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-22757: --- Component/s: (was: Deploy) > Init-container in the driver/executor pods for downloading r

[jira] [Assigned] (SPARK-22648) Documentation for Kubernetes Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22648: Assignee: Apache Spark > Documentation for Kubernetes Scheduler Backend >

[jira] [Commented] (SPARK-14228) Lost executor of RPC disassociated, and occurs exception: Could not find CoarseGrainedScheduler or it has been stopped

[ https://issues.apache.org/jira/browse/SPARK-14228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286798#comment-16286798 ] Apache Spark commented on SPARK-14228: -- User 'KaiXinXiaoLei' has created a pull requ

[jira] [Assigned] (SPARK-22648) Documentation for Kubernetes Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22648: Assignee: (was: Apache Spark) > Documentation for Kubernetes Scheduler Backend > -

[jira] [Updated] (SPARK-19809) NullPointerException on zero-size ORC file

[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19809: -- Summary: NullPointerException on zero-size ORC file (was: NullPointerException on empty ORC fi

[jira] [Created] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem,

KaiXinXIaoLei created SPARK-22760: - Summary: where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem, Key: SPARK-22760 URL: https://issues.apache.org/jira/b

[jira] [Assigned] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem,

[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22760: Assignee: Apache Spark > where driver is stopping, and some executors lost because of > Y

[jira] [Updated] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21867: Attachment: Async ShuffleExternalSorter.pdf Here is a design proposal to implement this per

[jira] [Commented] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem,

[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286928#comment-16286928 ] Apache Spark commented on SPARK-22760: -- User 'KaiXinXiaoLei' has created a pull requ

[jira] [Assigned] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem,

[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22760: Assignee: (was: Apache Spark) > where driver is stopping, and some executors lost beca

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem,

[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Use SPARK-14228 , i find a problem: 17/12/11 22:38:33 WARN YarnSchedulerBackend$Y

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem,

[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Attachment: 微信图片_20171212094100.jpg > where driver is stopping, and some executors lost because

[jira] [Commented] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286961#comment-16286961 ] Kevin Zhang commented on SPARK-22755: - Thanks for reply. In hive and presto the resul

[jira] [Commented] (SPARK-19809) NullPointerException on zero-size ORC file

[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286965#comment-16286965 ] Apache Spark commented on SPARK-19809: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-19809) NullPointerException on zero-size ORC file

[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19809: Assignee: Apache Spark > NullPointerException on zero-size ORC file >

[jira] [Comment Edited] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286961#comment-16286961 ] Kevin Zhang edited comment on SPARK-22755 at 12/12/17 1:58 AM:

[jira] [Assigned] (SPARK-19809) NullPointerException on zero-size ORC file

[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19809: Assignee: (was: Apache Spark) > NullPointerException on zero-size ORC file > -

[jira] [Comment Edited] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286961#comment-16286961 ] Kevin Zhang edited comment on SPARK-22755 at 12/12/17 2:03 AM:

[jira] [Commented] (SPARK-15282) UDF executed twice when filter on new column created by withColumn and the final value may be not correct

[ https://issues.apache.org/jira/browse/SPARK-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286951#comment-16286951 ] Wenchen Fan commented on SPARK-15282: - Shall we resolve this ticket as now users can

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Summary: where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop

[jira] [Commented] (SPARK-19566) Error initializing SparkContext under a Windows SYSTEM user

[ https://issues.apache.org/jira/browse/SPARK-19566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16287015#comment-16287015 ] Joseph Fourny commented on SPARK-19566: --- Did this ever get resolved? I am stuck wit

  1   2   >