[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546846#comment-14546846 ] Sean Owen commented on SPARK-7670: -- I can't reproduce this on Ubuntu 14 at master either.

[jira] [Comment Edited] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Fernando Ruben Otero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546793#comment-14546793 ] Fernando Ruben Otero edited comment on SPARK-7670 at 5/16/15 4:11 PM:

[jira] [Comment Edited] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Fernando Ruben Otero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546793#comment-14546793 ] Fernando Ruben Otero edited comment on SPARK-7670 at 5/16/15 4:13 PM:

[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546875#comment-14546875 ] Sean Owen commented on SPARK-7670: -- Yeah I see the same thing with your Dockerfile. The

[jira] [Commented] (SPARK-7527) Wrong detection of REPL mode in ClosureCleaner

2015-05-16 Thread Oleksii Kostyliev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546892#comment-14546892 ] Oleksii Kostyliev commented on SPARK-7527: -- In the end, due to a bigger

[jira] [Updated] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Fernando Ruben Otero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fernando Ruben Otero updated SPARK-7670: Attachment: Dockerfile This docker file reproduces the error on my machine

[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Fernando Ruben Otero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546794#comment-14546794 ] Fernando Ruben Otero commented on SPARK-7670: - I just attacked a docker file

[jira] [Updated] (SPARK-6439) Show per-task metrics when you hover over a task in the web UI visualization

2015-05-16 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-6439: -- Assignee: Kousuke Saruta (was: Kay Ousterhout) Show per-task metrics when you hover over a

[jira] [Comment Edited] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Fernando Ruben Otero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546847#comment-14546847 ] Fernando Ruben Otero edited comment on SPARK-7670 at 5/16/15 4:34 PM:

[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Fernando Ruben Otero (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546847#comment-14546847 ] Fernando Ruben Otero commented on SPARK-7670: - I did the docker file because

[jira] [Comment Edited] (SPARK-4412) Parquet logger cannot be configured

2015-05-16 Thread Yana Kadiyska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545677#comment-14545677 ] Yana Kadiyska edited comment on SPARK-4412 at 5/16/15 5:11 PM:

[jira] [Updated] (SPARK-6657) Fix Python doc build warnings

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6657: - Fix Version/s: (was: 1.3.1) Fix Python doc build warnings -

[jira] [Resolved] (SPARK-6197) handle json parse exception for eventlog file not finished writing

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6197. -- Resolution: Fixed Fix Version/s: 1.3.2 Target Version/s: 1.3.2, 1.4.0 (was: 1.4.0) I

[jira] [Updated] (SPARK-6197) handle json parse exception for eventlog file not finished writing

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6197: - Labels: (was: backport-needed) handle json parse exception for eventlog file not finished writing

[jira] [Updated] (SPARK-3928) Support wildcard matches on Parquet files

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3928: - Fix Version/s: (was: 1.3.0) Support wildcard matches on Parquet files

[jira] [Updated] (SPARK-4325) Improve spark-ec2 cluster launch times

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4325: - Fix Version/s: (was: 1.3.0) Improve spark-ec2 cluster launch times

[jira] [Updated] (SPARK-6657) Fix Python doc build warnings

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6657: - Target Version/s: 1.3.2, 1.4.0 (was: 1.3.1, 1.4.0) Fix Python doc build warnings

[jira] [Resolved] (SPARK-3490) Alleviate port collisions during tests

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3490. -- Resolution: Fixed Target Version/s: 1.2.0, 1.1.1, 0.9.3, 1.0.3 (was: 0.9.3, 1.0.3, 1.1.1,

[jira] [Resolved] (SPARK-3987) NNLS generates incorrect result

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3987. -- Resolution: Fixed From the discussion it sounds like the issue that this JIRA concerns was actually

[jira] [Updated] (SPARK-4258) NPE with new Parquet Filters

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4258: - Fix Version/s: (was: 1.2.0) NPE with new Parquet Filters

[jira] [Updated] (SPARK-2973) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2973: - Fix Version/s: (was: 1.2.0) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

[jira] [Updated] (SPARK-4412) Parquet logger cannot be configured

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4412: - Fix Version/s: (was: 1.2.0) Parquet logger cannot be configured ---

[jira] [Updated] (SPARK-2750) Add Https support for Web UI

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2750: - Fix Version/s: (was: 1.0.3) Add Https support for Web UI

[jira] [Created] (SPARK-7685) Handle high imbalanced data or apply weights to different samples in Logistic Regression

2015-05-16 Thread DB Tsai (JIRA)
DB Tsai created SPARK-7685: -- Summary: Handle high imbalanced data or apply weights to different samples in Logistic Regression Key: SPARK-7685 URL: https://issues.apache.org/jira/browse/SPARK-7685 Project:

[jira] [Updated] (SPARK-7685) Handle high imbalanced data and apply weights to different samples in Logistic Regression

2015-05-16 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-7685: --- Summary: Handle high imbalanced data and apply weights to different samples in Logistic Regression (was:

[jira] [Resolved] (SPARK-4556) binary distribution assembly can't run in local mode

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4556. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6186

[jira] [Updated] (SPARK-4556) Document that make-distribution.sh is required to make a runnable distribution

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4556: - Component/s: (was: Spark Shell) Documentation Deploy

[jira] [Updated] (SPARK-7672) Number format exception with spark.kryoserializer.buffer.mb

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7672: - Assignee: Nishkam Ravi Number format exception with spark.kryoserializer.buffer.mb

[jira] [Resolved] (SPARK-7672) Number format exception with spark.kryoserializer.buffer.mb

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7672. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6198

[jira] [Commented] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-16 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546617#comment-14546617 ] Tathagata Das commented on SPARK-7661: -- N+1 is used in the example, but isnt really

[jira] [Commented] (SPARK-7671) Fix wrong URLs in MLlib Data Types Documentation

2015-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546622#comment-14546622 ] Favio Vázquez commented on SPARK-7671: -- Thanks [~josephkb] and [~srowen] for fixing

[jira] [Commented] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-16 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546623#comment-14546623 ] Murtaza Kanchwala commented on SPARK-7661: -- Ok, Let me try your solution as well

[jira] [Commented] (SPARK-7654) DataFrameReader and DataFrameWriter for input/output API

2015-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546632#comment-14546632 ] Apache Spark commented on SPARK-7654: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-7646) Create table support to JDBC Datasource

2015-05-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7646: --- Labels: 1.4.1 (was: ) Create table support to JDBC Datasource

[jira] [Updated] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-16 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Murtaza Kanchwala updated SPARK-7661: - Description: Currently the no. of cores is (N + 1), where N is no. of shards in a Kinesis

[jira] [Resolved] (SPARK-7655) Akka timeout exception

2015-05-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7655. Resolution: Fixed Fix Version/s: 1.4.0 Akka timeout exception --

[jira] [Updated] (SPARK-7655) Akka timeout exception from ask and table broadcast

2015-05-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7655: --- Summary: Akka timeout exception from ask and table broadcast (was: Akka timeout exception) Akka

[jira] [Commented] (SPARK-7654) DataFrameReader and DataFrameWriter for input/output API

2015-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546647#comment-14546647 ] Apache Spark commented on SPARK-7654: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-16 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546598#comment-14546598 ] Murtaza Kanchwala commented on SPARK-7661: -- Ok I'll correct my terms, My case is

[jira] [Resolved] (SPARK-7671) Fix wrong URLs in MLlib Data Types Documentation

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7671. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6196

[jira] [Updated] (SPARK-7671) Fix wrong URLs in MLlib Data Types Documentation

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7671: - Assignee: Favio Vázquez Fix wrong URLs in MLlib Data Types Documentation

[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546628#comment-14546628 ] Sean Owen commented on SPARK-7670: -- I can't reproduce this. Master builds fine for me

[jira] [Commented] (SPARK-7654) DataFrameReader and DataFrameWriter for input/output API

2015-05-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546637#comment-14546637 ] Reynold Xin commented on SPARK-7654: TODOs: - Move insertInto also into write. -

[jira] [Created] (SPARK-7682) Size of distributed grids still limited by cPickle

2015-05-16 Thread Toby Potter (JIRA)
Toby Potter created SPARK-7682: -- Summary: Size of distributed grids still limited by cPickle Key: SPARK-7682 URL: https://issues.apache.org/jira/browse/SPARK-7682 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5948) Support writing to partitioned table for the Parquet data source

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5948: - Assignee: Michael Armbrust Support writing to partitioned table for the Parquet data source

[jira] [Updated] (SPARK-5281) Registering table on RDD is giving MissingRequirementError

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5281: - Assignee: Iulian Dragos Registering table on RDD is giving MissingRequirementError

[jira] [Updated] (SPARK-5632) not able to resolve dot('.') in field name

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5632: - Assignee: Wenchen Fan not able to resolve dot('.') in field name

[jira] [Updated] (SPARK-4699) Make caseSensitive configurable in Analyzer.scala

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4699: - Assignee: Fei Wang Make caseSensitive configurable in Analyzer.scala

[jira] [Updated] (SPARK-5947) First class partitioning support in data sources API

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5947: - Assignee: Michael Armbrust First class partitioning support in data sources API

[jira] [Updated] (SPARK-6734) Support GenericUDTF.close for Generate

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6734: - Assignee: Cheng Hao Support GenericUDTF.close for Generate --

[jira] [Updated] (SPARK-7109) Push down left side filter for left semi join

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7109: - Assignee: Fei Wang Push down left side filter for left semi join

[jira] [Updated] (SPARK-6439) Show per-task metrics when you hover over a task in the web UI visualization

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6439: - Assignee: Kay Ousterhout Show per-task metrics when you hover over a task in the web UI visualization

[jira] [Updated] (SPARK-6418) Add simple per-stage visualization to the UI

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6418: - Assignee: Kousuke Saruta Add simple per-stage visualization to the UI

[jira] [Updated] (SPARK-7437) Fold literal in (item1, item2, ..., literal, ...) into true or false directly

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7437: - Assignee: Zhongshuai Pei Fold literal in (item1, item2, ..., literal, ...) into true or false directly

[jira] [Updated] (SPARK-7504) NullPointerException when initializing SparkContext in YARN-cluster mode

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7504: - Assignee: Zoltán Zvara NullPointerException when initializing SparkContext in YARN-cluster mode

[jira] [Updated] (SPARK-7595) Window will cause resolve failed with self join

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7595: - Assignee: Weizhong Window will cause resolve failed with self join

[jira] [Updated] (SPARK-7598) Add aliveWorkers metrics in Master

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7598: - Assignee: Rex Xiong Add aliveWorkers metrics in Master --

[jira] [Updated] (SPARK-7601) Support Insert into JDBC Datasource

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7601: - Assignee: Venkata Ramana G Support Insert into JDBC Datasource ---

[jira] [Updated] (SPARK-5782) Python Worker / Pyspark Daemon Memory Issue

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5782: - Priority: Major (was: Blocker) Python Worker / Pyspark Daemon Memory Issue

[jira] [Updated] (SPARK-7269) Incorrect aggregation analysis

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7269: - Priority: Major (was: Blocker) Incorrect aggregation analysis --

[jira] [Updated] (SPARK-6680) Be able to specifie IP for spark-shell(spark driver) blocker for Docker integration

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6680: - Priority: Minor (was: Blocker) Be able to specifie IP for spark-shell(spark driver) blocker for Docker

[jira] [Updated] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7119: - Priority: Major (was: Blocker) ScriptTransform doesn't consider the output data type

[jira] [Resolved] (SPARK-7523) ERROR LiveListenerBus: Listener EventLoggingListener threw an exception

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7523. -- Resolution: Invalid I think this should start as a discussion on the mailing list. It's not clear this

[jira] [Updated] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4452: - Priority: Major (was: Critical) Target Version/s: (was: 1.1.2, 1.2.1, 1.3.0) Shuffle data

[jira] [Updated] (SPARK-5205) Inconsistent behaviour between Streaming job and others, when click kill link in WebUI

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5205: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Inconsistent behaviour between Streaming job and others,

[jira] [Updated] (SPARK-4888) Spark EC2 doesn't mount local disks for i2.8xlarge instances

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4888: - Target Version/s: 1.5.0 (was: 1.0.3, 1.1.2, 1.2.1, 1.3.0) Spark EC2 doesn't mount local disks for

[jira] [Updated] (SPARK-6174) Improve doc: Python ALS, MatrixFactorizationModel

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6174: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Improve doc: Python ALS, MatrixFactorizationModel

[jira] [Updated] (SPARK-4227) Document external shuffle service

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4227: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Document external shuffle service

[jira] [Updated] (SPARK-6266) PySpark SparseVector missing doc for size, indices, values

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6266: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) PySpark SparseVector missing doc for size, indices, values

[jira] [Updated] (SPARK-6173) Python doc parity with Scala/Java in MLlib

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6173: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Python doc parity with Scala/Java in MLlib

[jira] [Updated] (SPARK-6270) Standalone Master hangs when streaming job completes

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6270: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Standalone Master hangs when streaming job completes

[jira] [Updated] (SPARK-6265) PySpark GLMs missing doc for intercept, weights

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6265: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) PySpark GLMs missing doc for intercept, weights

[jira] [Updated] (SPARK-6632) Optimize the parquetSchema to metastore schema reconciliation, so that the process is delegated to each map task itself

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6632: - Fix Version/s: (was: 1.4.0) Optimize the parquetSchema to metastore schema reconciliation, so that

[jira] [Updated] (SPARK-7563) OutputCommitCoordinator.stop() should only be executed in driver

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7563: - Fix Version/s: (was: 1.4.0) OutputCommitCoordinator.stop() should only be executed in driver

[jira] [Updated] (SPARK-6378) srcAttr in graph.triplets don't update when the size of graph is huge

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6378: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) srcAttr in graph.triplets don't update when the size of

[jira] [Updated] (SPARK-6701) Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6701: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python

[jira] [Updated] (SPARK-6981) [SQL] SparkPlanner and QueryExecution should be factored out from SQLContext

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6981: - Fix Version/s: (was: 1.4.0) [SQL] SparkPlanner and QueryExecution should be factored out from

[jira] [Updated] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6484: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) Ganglia metrics xml reporter doesn't escape correctly

[jira] [Updated] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7606: - Fix Version/s: (was: 1.4.0) Document all PySpark SQL/DataFrame public methods with @since tag

[jira] [Updated] (SPARK-7444) Eliminate noisy css warn/error logs for UISeleniumSuite

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7444: - Fix Version/s: (was: 1.4.0) Eliminate noisy css warn/error logs for UISeleniumSuite

[jira] [Updated] (SPARK-7097) Partitioned tables should only consider referred partitions in query during size estimation for checking against autoBroadcastJoinThreshold

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7097: - Fix Version/s: (was: 1.4.0) Partitioned tables should only consider referred partitions in query

[jira] [Updated] (SPARK-6828) Spark returns misleading message when client is incompatible with server

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6828: - Fix Version/s: (was: 1.4.0) Spark returns misleading message when client is incompatible with server

[jira] [Updated] (SPARK-7527) Wrong detection of REPL mode in ClosureCleaner

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7527: - Fix Version/s: (was: 1.4.0) Wrong detection of REPL mode in ClosureCleaner

[jira] [Updated] (SPARK-6803) [SparkR] Support SparkR Streaming

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6803: - Fix Version/s: (was: 1.4.0) [SparkR] Support SparkR Streaming -

[jira] [Updated] (SPARK-7316) Add step capability to RDD sliding window

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7316: - Fix Version/s: (was: 1.4.0) Add step capability to RDD sliding window

[jira] [Updated] (SPARK-6828) Spark returns misleading message when client is incompatible with server

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6828: - Target Version/s: (was: 1.4.0) Spark returns misleading message when client is incompatible with

[jira] [Updated] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7670: - Fix Version/s: (was: 1.4.0) Failure when building with scala 2.11 (after 1.3.1

[jira] [Updated] (SPARK-7627) DAG visualization: cached RDDs not shown on job page

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7627: - Fix Version/s: (was: 1.4.0) DAG visualization: cached RDDs not shown on job page

[jira] [Updated] (SPARK-7658) Update the mouse behaviors for the timeline graphs

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7658: - Fix Version/s: (was: 1.4.0) Update the mouse behaviors for the timeline graphs

[jira] [Updated] (SPARK-7224) Mock repositories for testing with --packages

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7224: - Fix Version/s: (was: 1.4.0) Mock repositories for testing with --packages

[jira] [Updated] (SPARK-6197) handle json parse exception for eventlog file not finished writing

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6197: - Target Version/s: 1.4.0 (was: 1.3.1, 1.4.0) handle json parse exception for eventlog file not finished

[jira] [Updated] (SPARK-7245) Spearman correlation for DataFrames

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7245: - Fix Version/s: (was: 1.4.0) Spearman correlation for DataFrames ---

[jira] [Updated] (SPARK-7498) Params.setDefault should not use varargs annotation

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7498: - Fix Version/s: (was: 1.4.0) Params.setDefault should not use varargs annotation

[jira] [Updated] (SPARK-6216) Check Python version in worker before run PySpark job

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6216: - Fix Version/s: (was: 1.4.0) Check Python version in worker before run PySpark job

[jira] [Updated] (SPARK-7287) Flaky test: o.a.s.deploy.SparkSubmitSuite --packages

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7287: - Fix Version/s: (was: 1.4.0) Flaky test: o.a.s.deploy.SparkSubmitSuite --packages

[jira] [Updated] (SPARK-6197) handle json parse exception for eventlog file not finished writing

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6197: - Fix Version/s: 1.4.0 handle json parse exception for eventlog file not finished writing

[jira] [Updated] (SPARK-6197) handle json parse exception for eventlog file not finished writing

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6197: - Fix Version/s: (was: 1.4.0) handle json parse exception for eventlog file not finished writing

[jira] [Updated] (SPARK-2155) Support effectful / non-deterministic key expressions in CASE WHEN statements

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2155: - Assignee: Wenchen Fan Support effectful / non-deterministic key expressions in CASE WHEN statements

[jira] [Updated] (SPARK-7093) Using newPredicate in NestedLoopJoin to enable code generation

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7093: - Assignee: Fei Wang Using newPredicate in NestedLoopJoin to enable code generation

[jira] [Updated] (SPARK-7123) support table.star in sqlcontext

2015-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7123: - Assignee: Fei Wang support table.star in sqlcontext

  1   2   >