[jira] [Updated] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Zhong updated SPARK-17289: --- Description: For the following query: {code} val df2 = (0 to 1000).map(x => (x % 2, x.toString)).toD

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445873#comment-15445873 ] Vincent commented on SPARK-17219: - Cool. I will refine the patch. thanks [~srowen] :) >

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445862#comment-15445862 ] Sean Owen commented on SPARK-17219: --- Agree, and that's a reasonable requirement for any

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445858#comment-15445858 ] Vincent commented on SPARK-17219: - yes, discretizer can do it easily, especially if only

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445820#comment-15445820 ] Sean Owen commented on SPARK-17219: --- No, the discretizer can do this easily, right? The

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445808#comment-15445808 ] Vincent commented on SPARK-17219: - then we have to shift this work to user, who needs to

[jira] [Commented] (SPARK-17292) Setup app on Spark(Hadoop-Yarn) in Standby mode

[ https://issues.apache.org/jira/browse/SPARK-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445800#comment-15445800 ] Patryk commented on SPARK-17292: Thanks, I will write there. > Setup app on Spark(Hadoop

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445779#comment-15445779 ] Sean Owen commented on SPARK-17219: --- No, there's no meaning to a split bounded by NaN.

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445768#comment-15445768 ] Vincent commented on SPARK-17219: - [~srowen] Hi all, per discussion, I thought we are goi

[jira] [Resolved] (SPARK-17292) Setup app on Spark(Hadoop-Yarn) in Standby mode

[ https://issues.apache.org/jira/browse/SPARK-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17292. --- Resolution: Invalid Questions should go to u...@spark.apache.org > Setup app on Spark(Hadoop-Yarn) i

[jira] [Updated] (SPARK-17292) Setup app on Spark(Hadoop-Yarn) in Standby mode

[ https://issues.apache.org/jira/browse/SPARK-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patryk updated SPARK-17292: --- Summary: Setup app on Spark(Hadoop-Yarn) in Standby mode (was: Setup app on Spark(Hadoop-Yarn) in Stanby mod

[jira] [Updated] (SPARK-17292) Setup app on Spark(Hadoop-Yarn) in Stanby mode

[ https://issues.apache.org/jira/browse/SPARK-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patryk updated SPARK-17292: --- Description: Is there any way to run Apache Spark in Standby mode, so I can only give it new data and new ar

[jira] [Created] (SPARK-17292) Setup app on Spark(Hadoop-Yarn) in Stanby mode

Patryk created SPARK-17292: -- Summary: Setup app on Spark(Hadoop-Yarn) in Stanby mode Key: SPARK-17292 URL: https://issues.apache.org/jira/browse/SPARK-17292 Project: Spark Issue Type: Question

[jira] [Comment Edited] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445686#comment-15445686 ] Arihanth Jain edited comment on SPARK-13525 at 8/29/16 12:21 PM: --

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445686#comment-15445686 ] Arihanth Jain commented on SPARK-13525: --- I followed this and it fails to create the

[jira] [Resolved] (SPARK-17291) The shuffle data fetched based on netty were directly stored in off-memoryr?

[ https://issues.apache.org/jira/browse/SPARK-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17291. --- Resolution: Invalid Target Version/s: (was: 1.6.1) Same as in https://issues.apache.org/j

[jira] [Closed] (SPARK-17286) The fetched data of Shuffle read stored in the ManagedBuffer, so its underlying data stored in off-memory or file?

[ https://issues.apache.org/jira/browse/SPARK-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] song fengfei closed SPARK-17286. > The fetched data of Shuffle read stored in the ManagedBuffer, so its > underlying data stored in of

[jira] [Created] (SPARK-17291) The shuffle data fetched based on netty were directly stored in off-memoryr?

song fengfei created SPARK-17291: Summary: The shuffle data fetched based on netty were directly stored in off-memoryr? Key: SPARK-17291 URL: https://issues.apache.org/jira/browse/SPARK-17291 Project:

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445591#comment-15445591 ] Gen TANG commented on SPARK-17110: -- Hi, I tried the similar code in python, scala, java

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445571#comment-15445571 ] song fengfei commented on SPARK-13510: -- Hi, Hong Shen You mean that the shuffle da

[jira] [Created] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

Teng Yutong created SPARK-17290: --- Summary: Spark CSVInferSchema does not always respect nullValue settings Key: SPARK-17290 URL: https://issues.apache.org/jira/browse/SPARK-17290 Project: Spark

[jira] [Created] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

Sean Zhong created SPARK-17289: -- Summary: Sort based partial aggregation breaks due to SPARK-12978 Key: SPARK-17289 URL: https://issues.apache.org/jira/browse/SPARK-17289 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-17287) PySpark sc.AddFile method does not support the recursive keyword argument

[ https://issues.apache.org/jira/browse/SPARK-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17287: Assignee: (was: Apache Spark) > PySpark sc.AddFile method does not support the recursi

[jira] [Assigned] (SPARK-17287) PySpark sc.AddFile method does not support the recursive keyword argument

[ https://issues.apache.org/jira/browse/SPARK-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17287: Assignee: Apache Spark > PySpark sc.AddFile method does not support the recursive keyword

[jira] [Commented] (SPARK-17287) PySpark sc.AddFile method does not support the recursive keyword argument

[ https://issues.apache.org/jira/browse/SPARK-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445456#comment-15445456 ] Apache Spark commented on SPARK-17287: -- User 'jpiper' has created a pull request for

[jira] [Commented] (SPARK-17264) DataStreamWriter should document that it only supports Parquet for now

[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445420#comment-15445420 ] Apache Spark commented on SPARK-17264: -- User 'srowen' has created a pull request for

[jira] [Assigned] (SPARK-17264) DataStreamWriter should document that it only supports Parquet for now

[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17264: Assignee: Sean Owen (was: Apache Spark) > DataStreamWriter should document that it only s

[jira] [Assigned] (SPARK-17264) DataStreamWriter should document that it only supports Parquet for now

[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17264: Assignee: Apache Spark (was: Sean Owen) > DataStreamWriter should document that it only s

[jira] [Assigned] (SPARK-17264) DataStreamWriter should document that it only supports Parquet for now

[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-17264: - Assignee: Sean Owen > DataStreamWriter should document that it only supports Parquet for now > -

[jira] [Created] (SPARK-17288) Spark sbin script on windows support

wei wu created SPARK-17288: -- Summary: Spark sbin script on windows support Key: SPARK-17288 URL: https://issues.apache.org/jira/browse/SPARK-17288 Project: Spark Issue Type: New Feature Co

[jira] [Commented] (SPARK-11001) SQLContext doesn't support window function

[ https://issues.apache.org/jira/browse/SPARK-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445384#comment-15445384 ] Dongjoon Hyun commented on SPARK-11001: --- Hi, [~jixing.ji]. This issue seems to be r

[jira] [Resolved] (SPARK-17286) The fetched data of Shuffle read stored in the ManagedBuffer, so its underlying data stored in off-memory or file?

[ https://issues.apache.org/jira/browse/SPARK-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17286. --- Resolution: Invalid Target Version/s: (was: 1.6.1) user@ is the place for questions. > T

[jira] [Updated] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17256: -- Target Version/s: (was: 2.0.0) > spark-submit.cmd cannot work if path has space and cut off double-qu

[jira] [Updated] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17261: -- Target Version/s: (was: 2.0.0) Fix Version/s: (was: 2.0.0) > Using HiveContext after re-cr

[jira] [Resolved] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13117. --- Resolution: Fixed OK, but you seem to be proposing a different change, to make this separately confi

[jira] [Assigned] (SPARK-17200) Automate building and testing on Windows

[ https://issues.apache.org/jira/browse/SPARK-17200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17200: Assignee: (was: Apache Spark) > Automate building and testing on Windows > --

[jira] [Assigned] (SPARK-17200) Automate building and testing on Windows

[ https://issues.apache.org/jira/browse/SPARK-17200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17200: Assignee: Apache Spark > Automate building and testing on Windows > -

[jira] [Commented] (SPARK-17200) Automate building and testing on Windows

[ https://issues.apache.org/jira/browse/SPARK-17200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445320#comment-15445320 ] Apache Spark commented on SPARK-17200: -- User 'HyukjinKwon' has created a pull reques

[jira] [Updated] (SPARK-17264) DataStreamWriter should document that it only supports Parquet for now

[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17264: -- Priority: Trivial (was: Minor) Summary: DataStreamWriter should document that it only supports Par

[jira] [Commented] (SPARK-17264) DataStreamWriter does not support "json" format

[ https://issues.apache.org/jira/browse/SPARK-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445284#comment-15445284 ] Sean Owen commented on SPARK-17264: --- It is. I'm going to consider this JIRA to be about

[jira] [Created] (SPARK-17287) PySpark sc.AddFile method does not support the recursive keyword argument

Jason Piper created SPARK-17287: --- Summary: PySpark sc.AddFile method does not support the recursive keyword argument Key: SPARK-17287 URL: https://issues.apache.org/jira/browse/SPARK-17287 Project: Spar

[jira] [Commented] (SPARK-17278) better error message for NPE during ScalaUDF execution

[ https://issues.apache.org/jira/browse/SPARK-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445220#comment-15445220 ] Wenchen Fan commented on SPARK-17278: - oh it was my mistake, closing it. > better er

[jira] [Closed] (SPARK-17278) better error message for NPE during ScalaUDF execution

[ https://issues.apache.org/jira/browse/SPARK-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-17278. --- Resolution: Duplicate > better error message for NPE during ScalaUDF execution >

[jira] [Commented] (SPARK-17278) better error message for NPE during ScalaUDF execution

[ https://issues.apache.org/jira/browse/SPARK-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445201#comment-15445201 ] Liang-Chi Hsieh commented on SPARK-17278: - Duplicate to SPARK-17279? > better er

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445198#comment-15445198 ] Apache Spark commented on SPARK-17219: -- User 'VinceShieh' has created a pull request

[jira] [Assigned] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17219: Assignee: (was: Apache Spark) > QuantileDiscretizer does strange things with NaN value

[jira] [Assigned] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17219: Assignee: Apache Spark > QuantileDiscretizer does strange things with NaN values > ---

[jira] [Commented] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445095#comment-15445095 ] HackerWilson commented on SPARK-10795: -- Hi All, I am facing the same problem, too. I

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15445029#comment-15445029 ] Miao Wang commented on SPARK-17110: --- Can you post a sample configuration? It could be s

<    1   2