[jira] [Updated] (SPARK-20869) Master should clear failed apps when worker down

[ https://issues.apache.org/jira/browse/SPARK-20869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lyc updated SPARK-20869: Description: In `Master.removeWorker`, master clears executor and driver state, but does not clear app state. App

[jira] [Updated] (SPARK-20869) Master should clear failed apps when worker down

[ https://issues.apache.org/jira/browse/SPARK-20869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lyc updated SPARK-20869: Description: In `Master.removeWorker`, master clears executor and driver state, but does not clear app state. App

[jira] [Comment Edited] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049942#comment-16049942 ] Li Yichao edited comment on SPARK-19900 at 6/15/17 3:26 AM: M

[jira] [Comment Edited] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049942#comment-16049942 ] Li Yichao edited comment on SPARK-19900 at 6/15/17 3:29 AM: M

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049965#comment-16049965 ] DjvuLee commented on SPARK-21082: - Data locality, input size for task, scheduling order a

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Affects Version/s: (was: 2.2.1) 2.3.0 > Consider Executor's memory usage when sc

[jira] [Commented] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049977#comment-16049977 ] Takeshi Yamamuro commented on SPARK-21101: -- Since JIRA is not a place for questi

[jira] [Comment Edited] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049977#comment-16049977 ] Takeshi Yamamuro edited comment on SPARK-21101 at 6/15/17 4:16 AM:

[jira] [Closed] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro closed SPARK-21101. Resolution: Not A Problem > Error running Hive temporary UDTF on latest Spark 2.2 > ---

[jira] [Commented] (SPARK-15905) Driver hung while writing to console progress bar

[ https://issues.apache.org/jira/browse/SPARK-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049979#comment-16049979 ] remoteServer commented on SPARK-15905: -- I faced the same issue. Increasing driver me

[jira] [Resolved] (SPARK-21092) Wire SQLConf in logical plan and expressions

[ https://issues.apache.org/jira/browse/SPARK-21092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-21092. - Resolution: Fixed Fix Version/s: 2.3.0 > Wire SQLConf in logical plan and expressions > --

[jira] [Updated] (SPARK-21074) Parquet files are read fully even though only count() is requested

[ https://issues.apache.org/jira/browse/SPARK-21074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21074: - Issue Type: Improvement (was: Bug) > Parquet files are read fully even though only count

[jira] [Commented] (SPARK-21074) Parquet files are read fully even though only count() is requested

[ https://issues.apache.org/jira/browse/SPARK-21074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050005#comment-16050005 ] Takeshi Yamamuro commented on SPARK-21074: -- Since this is an expected behaviour

[jira] [Resolved] (SPARK-20980) Rename the option `wholeFile` to `multiLine` for JSON and CSV

[ https://issues.apache.org/jira/browse/SPARK-20980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20980. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18202 [https://githu

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050016#comment-16050016 ] Saisai Shao commented on SPARK-21082: - That's fine if the storage memory is not enoug

[jira] [Commented] (SPARK-20980) Rename the option `wholeFile` to `multiLine` for JSON and CSV

[ https://issues.apache.org/jira/browse/SPARK-20980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050023#comment-16050023 ] Apache Spark commented on SPARK-20980: -- User 'felixcheung' has created a pull reques

[jira] [Resolved] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18016. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18075 [https://githu

[jira] [Commented] (SPARK-20851) Drop spark table failed if a column name is a numeric string

[ https://issues.apache.org/jira/browse/SPARK-20851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050028#comment-16050028 ] Chen Gong commented on SPARK-20851: --- [~benyuel] [~maropu] Thanks for your comments. Roo

[jira] [Assigned] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18016: --- Assignee: Aleksander Eskilson > Code Generation: Constant Pool Past Limit for Wide/Nested Da

[jira] [Commented] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050031#comment-16050031 ] Dayou Zhou commented on SPARK-21101: Hi [~maropu], >> I'll close this because this s

[jira] [Reopened] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayou Zhou reopened SPARK-21101: > Error running Hive temporary UDTF on latest Spark 2.2 > -

[jira] [Commented] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050041#comment-16050041 ] Sean Owen commented on SPARK-21101: --- [~dyzhou] did you read the link he posted? This do

[jira] [Commented] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050048#comment-16050048 ] Dayou Zhou commented on SPARK-21101: Hi [~sowen], >> did you read the link he posted

[jira] [Comment Edited] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049977#comment-16049977 ] Takeshi Yamamuro edited comment on SPARK-21101 at 6/15/17 6:16 AM:

[jira] [Commented] (SPARK-21076) R dapply doesn't return array or raw columns when array have different length

[ https://issues.apache.org/jira/browse/SPARK-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050052#comment-16050052 ] Felix Cheung commented on SPARK-21076: -- this looks to happen when mapping the Spark

[jira] [Assigned] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21087: Assignee: (was: Apache Spark) > CrossValidator, TrainValidationSplit should preserve a

[jira] [Assigned] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21087: Assignee: Apache Spark > CrossValidator, TrainValidationSplit should preserve all models a

[jira] [Commented] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050053#comment-16050053 ] Apache Spark commented on SPARK-21087: -- User 'hhbyyh' has created a pull request for

[jira] [Updated] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21093: - Shepherd: Felix Cheung > Multiple gapply execution occasionally failed in SparkR > -

[jira] [Created] (SPARK-21092) Wire SQLConf in logical plan and expressions

Reynold Xin created SPARK-21092: --- Summary: Wire SQLConf in logical plan and expressions Key: SPARK-21092 URL: https://issues.apache.org/jira/browse/SPARK-21092 Project: Spark Issue Type: New Fe

[jira] [Commented] (SPARK-21092) Wire SQLConf in logical plan and expressions

[ https://issues.apache.org/jira/browse/SPARK-21092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048788#comment-16048788 ] Apache Spark commented on SPARK-21092: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-21092) Wire SQLConf in logical plan and expressions

[ https://issues.apache.org/jira/browse/SPARK-21092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21092: Assignee: Reynold Xin (was: Apache Spark) > Wire SQLConf in logical plan and expressions

[jira] [Assigned] (SPARK-21092) Wire SQLConf in logical plan and expressions

[ https://issues.apache.org/jira/browse/SPARK-21092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21092: Assignee: Apache Spark (was: Reynold Xin) > Wire SQLConf in logical plan and expressions

[jira] [Commented] (SPARK-21078) JobHistory applications synchronized is invalid

[ https://issues.apache.org/jira/browse/SPARK-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048814#comment-16048814 ] Sean Owen commented on SPARK-21078: --- OK [~ffbin] would you like to work on the change I

[jira] [Commented] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048817#comment-16048817 ] Sean Owen commented on SPARK-21084: --- These are really resource manager concerns. For ex

[jira] [Commented] (SPARK-21043) Add unionByName API to Dataset

[ https://issues.apache.org/jira/browse/SPARK-21043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048818#comment-16048818 ] Apache Spark commented on SPARK-21043: -- User 'maropu' has created a pull request for

[jira] [Assigned] (SPARK-21043) Add unionByName API to Dataset

[ https://issues.apache.org/jira/browse/SPARK-21043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21043: Assignee: Apache Spark > Add unionByName API to Dataset > -- >

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048821#comment-16048821 ] Sean Owen commented on SPARK-21082: --- This doesn't address my question about locality. I

[jira] [Assigned] (SPARK-21043) Add unionByName API to Dataset

[ https://issues.apache.org/jira/browse/SPARK-21043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21043: Assignee: (was: Apache Spark) > Add unionByName API to Dataset > -

[jira] [Resolved] (SPARK-21057) Do not use a PascalDistribution in countApprox

[ https://issues.apache.org/jira/browse/SPARK-21057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21057. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18276 [https://github.co

[jira] [Assigned] (SPARK-21057) Do not use a PascalDistribution in countApprox

[ https://issues.apache.org/jira/browse/SPARK-21057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21057: - Assignee: Sean Owen > Do not use a PascalDistribution in countApprox > -

[jira] [Commented] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048830#comment-16048830 ] Filipp Zhinkin commented on SPARK-21081: I have an application that fires alerts

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048835#comment-16048835 ] DjvuLee commented on SPARK-21082: - Yes, one of the reason why Spark do not balance tasks

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to Exe

[jira] [Commented] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048855#comment-16048855 ] Sean Owen commented on SPARK-21081: --- Just call is stopped to check that. This isn't som

[jira] [Resolved] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21085. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18295 [https://githu

[jira] [Updated] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21021: - Issue Type: Improvement (was: Bug) > Reading partitioned parquet does not respect specif

[jira] [Commented] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048871#comment-16048871 ] Takeshi Yamamuro commented on SPARK-21021: -- This is an expected behaviour, so it

[jira] [Assigned] (SPARK-21052) Add hash map metrics to join

[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21052: Assignee: Apache Spark > Add hash map metrics to join > > >

[jira] [Commented] (SPARK-21052) Add hash map metrics to join

[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048891#comment-16048891 ] Apache Spark commented on SPARK-21052: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-21052) Add hash map metrics to join

[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21052: Assignee: (was: Apache Spark) > Add hash map metrics to join > ---

[jira] [Created] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

Hyukjin Kwon created SPARK-21093: Summary: Multiple gapply execution occasionally failed in SparkR Key: SPARK-21093 URL: https://issues.apache.org/jira/browse/SPARK-21093 Project: Spark Issu

[jira] [Commented] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048934#comment-16048934 ] Hyukjin Kwon commented on SPARK-21093: -- cc [~nick.pentre...@gmail.com] and [~felixch

[jira] [Updated] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21093: - Description: On Centos 7.2.1511 with R 3.4.0/3.3.0, multiple execution of {{gapply}} looks faile

[jira] [Updated] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21093: - Description: On Centos 7.2.1511 with R 3.4.0/3.3.0, multiple execution of {{gapply}} looks faile

[jira] [Commented] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048942#comment-16048942 ] Stavros Kontopoulos commented on SPARK-20812: - Spark-submit should store the

[jira] [Updated] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21093: - Description: On Centos 7.2.1511 with R 3.4.0/3.3.0, multiple execution of {{gapply}} looks faile

[jira] [Updated] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21093: - Description: On Centos 7.2.1511 with R 3.4.0/3.3.0, multiple execution of {{gapply}} looks faile

[jira] [Updated] (SPARK-21045) Spark executor blocked instead of throwing exception because exception occur when python worker send exception info to PythonRDD in Python 2+

[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Summary: Spark executor blocked instead of throwing exception because exception occur when python

[jira] [Resolved] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20211. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.2.0 2.1.2

[jira] [Commented] (SPARK-21021) Reading partitioned parquet does not respect specified schema column order

[ https://issues.apache.org/jira/browse/SPARK-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049100#comment-16049100 ] Michel Lemay commented on SPARK-21021: -- Indeed, the code is not broken so it's not a

[jira] [Commented] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049105#comment-16049105 ] Marco Gaido commented on SPARK-19909: - [~rvoyer] there is a workaround and it is easy

[jira] [Created] (SPARK-21094) Allow stdout/stderr pipes in pyspark.java_gateway.launch_gateway

Peter Parente created SPARK-21094: - Summary: Allow stdout/stderr pipes in pyspark.java_gateway.launch_gateway Key: SPARK-21094 URL: https://issues.apache.org/jira/browse/SPARK-21094 Project: Spark

[jira] [Updated] (SPARK-21094) Allow stdout/stderr pipes in pyspark.java_gateway.launch_gateway

[ https://issues.apache.org/jira/browse/SPARK-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Parente updated SPARK-21094: -- Description: The Popen call to launch the py4j gateway specifies no stdout and stderr options,

[jira] [Commented] (SPARK-20839) Incorrect Dynamic PageRank calculation

[ https://issues.apache.org/jira/browse/SPARK-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049214#comment-16049214 ] Andrew Ray commented on SPARK-20839: 1 & 2 work together to do the algorithm properly

[jira] [Resolved] (SPARK-20839) Incorrect Dynamic PageRank calculation

[ https://issues.apache.org/jira/browse/SPARK-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ray resolved SPARK-20839. Resolution: Not A Problem > Incorrect Dynamic PageRank calculation > --

[jira] [Updated] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Ricard updated SPARK-21067: --- Description: After upgrading our Thrift cluster to 2.1.1, we ran into an issue where CTAS wo

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049228#comment-16049228 ] Thomas Graves commented on SPARK-20589: --- [~Robin Shao] can you please clarify your

[jira] [Commented] (SPARK-17237) DataFrame fill after pivot causing org.apache.spark.sql.AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049230#comment-16049230 ] Apache Spark commented on SPARK-17237: -- User 'maropu' has created a pull request for

[jira] [Commented] (SPARK-19824) Standalone master JSON not showing cores for running applications

[ https://issues.apache.org/jira/browse/SPARK-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049268#comment-16049268 ] Apache Spark commented on SPARK-19824: -- User 'jiangxb1987' has created a pull reques

[jira] [Created] (SPARK-21095) Support for Category type in Spark

Saatvik created SPARK-21095: --- Summary: Support for Category type in Spark Key: SPARK-21095 URL: https://issues.apache.org/jira/browse/SPARK-21095 Project: Spark Issue Type: Bug Components

[jira] [Commented] (SPARK-21093) Multiple gapply execution occasionally failed in SparkR

[ https://issues.apache.org/jira/browse/SPARK-21093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049361#comment-16049361 ] Shivaram Venkataraman commented on SPARK-21093: --- So it looks like the R wor

[jira] [Resolved] (SPARK-21095) Support for Category type in Spark

[ https://issues.apache.org/jira/browse/SPARK-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21095. --- Resolution: Invalid Questions should go to the user@ mailing list > Support for Category type in Spa

[jira] [Commented] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049380#comment-16049380 ] Michael Gummelt commented on SPARK-20812: - The dispatcher won't know about the se

[jira] [Created] (SPARK-21096) Pickle error when passing a member variable to Spark executors

Irina Truong created SPARK-21096: Summary: Pickle error when passing a member variable to Spark executors Key: SPARK-21096 URL: https://issues.apache.org/jira/browse/SPARK-21096 Project: Spark

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Created] (SPARK-21097) Dynamic allocation will preserve cached data

Brad created SPARK-21097: Summary: Dynamic allocation will preserve cached data Key: SPARK-21097 URL: https://issues.apache.org/jira/browse/SPARK-21097 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049443#comment-16049443 ] Brad commented on SPARK-21097: -- I am working on this now and will be posting a more detailed

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049442#comment-16049442 ] Sean Owen commented on SPARK-21097: --- This seems to add a fair bit of complexity when Sp

[jira] [Updated] (SPARK-21096) Pickle error when passing a member variable to Spark executors

[ https://issues.apache.org/jira/browse/SPARK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Truong updated SPARK-21096: - Description: There is a pickle error when submitting a spark job that references a member variab

[jira] [Commented] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049447#comment-16049447 ] Ajay Saini commented on SPARK-21088: I'll work on this one. > CrossValidator, TrainV

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049485#comment-16049485 ] Brad commented on SPARK-21097: -- Hey Sean, thanks for your input. I would definitely like to

[jira] [Comment Edited] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049485#comment-16049485 ] Brad edited comment on SPARK-21097 at 6/14/17 6:29 PM: --- Hey [~srowe

[jira] [Updated] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Ricard updated SPARK-21067: --- Description: After upgrading our Thrift cluster to 2.1.1, we ran into an issue where CTAS wo

[jira] [Created] (SPARK-21098) Add line separator option to csv

Daniel van der Ende created SPARK-21098: --- Summary: Add line separator option to csv Key: SPARK-21098 URL: https://issues.apache.org/jira/browse/SPARK-21098 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-21098) Add line separator option to csv read/write

[ https://issues.apache.org/jira/browse/SPARK-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel van der Ende updated SPARK-21098: Summary: Add line separator option to csv read/write (was: Add line separator opti

[jira] [Closed] (SPARK-16669) Partition pruning for metastore relation size estimates for better join selection.

[ https://issues.apache.org/jira/browse/SPARK-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang closed SPARK-16669. Resolution: Duplicate > Partition pruning for metastore relation size estimates for better join >

[jira] [Assigned] (SPARK-21098) Add line separator option to csv read/write

[ https://issues.apache.org/jira/browse/SPARK-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21098: Assignee: Apache Spark > Add line separator option to csv read/write > ---

[jira] [Commented] (SPARK-21098) Add line separator option to csv read/write

[ https://issues.apache.org/jira/browse/SPARK-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049509#comment-16049509 ] Apache Spark commented on SPARK-21098: -- User 'danielvdende' has created a pull reque

[jira] [Assigned] (SPARK-21098) Add line separator option to csv read/write

[ https://issues.apache.org/jira/browse/SPARK-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21098: Assignee: (was: Apache Spark) > Add line separator option to csv read/write >

[jira] [Assigned] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20988: Assignee: Apache Spark > Convert logistic regression to new aggregator framework > ---

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049519#comment-16049519 ] Apache Spark commented on SPARK-20988: -- User 'sethah' has created a pull request for

[jira] [Assigned] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20988: Assignee: (was: Apache Spark) > Convert logistic regression to new aggregator framewor

[jira] [Commented] (SPARK-21029) All StreamingQuery should be stopped when the SparkSession is stopped

[ https://issues.apache.org/jira/browse/SPARK-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049626#comment-16049626 ] Apache Spark commented on SPARK-21029: -- User 'aray' has created a pull request for t

  1   2   >