[jira] [Commented] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304729#comment-14304729 ] Apache Spark commented on SPARK-5585: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304724#comment-14304724 ] Davies Liu commented on SPARK-5585: --- [~pwendell] I can not reproduce it locally, will ad

[jira] [Commented] (SPARK-5587) Support change database owner

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304721#comment-14304721 ] Apache Spark commented on SPARK-5587: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-5587) Support change database owner

2015-02-03 Thread wangfei (JIRA)
wangfei created SPARK-5587: -- Summary: Support change database owner Key: SPARK-5587 URL: https://issues.apache.org/jira/browse/SPARK-5587 Project: Spark Issue Type: Improvement Components

[jira] [Created] (SPARK-5586) Automatically provide sqlContext in Spark shell

2015-02-03 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5586: -- Summary: Automatically provide sqlContext in Spark shell Key: SPARK-5586 URL: https://issues.apache.org/jira/browse/SPARK-5586 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5586) Automatically provide sqlContext in Spark shell

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5586: --- Fix Version/s: (was: 1.3.0) > Automatically provide sqlContext in Spark shell > --

[jira] [Updated] (SPARK-5586) Automatically provide sqlContext in Spark shell

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5586: --- Priority: Critical (was: Major) > Automatically provide sqlContext in Spark shell > -

[jira] [Commented] (SPARK-5068) When the path not found in the hdfs,we can't get the result

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304710#comment-14304710 ] Apache Spark commented on SPARK-5068: - User 'chenghao-intel' has created a pull reques

[jira] [Resolved] (SPARK-5341) Support maven coordinates in spark-shell and spark-submit

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5341. Resolution: Fixed Fix Version/s: 1.3.0 > Support maven coordinates in spark-shell and

[jira] [Updated] (SPARK-5341) Support maven coordinates in spark-shell and spark-submit

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5341: --- Assignee: Burak Yavuz > Support maven coordinates in spark-shell and spark-submit > --

[jira] [Resolved] (SPARK-4969) Add binaryRecords support to streaming

2015-02-03 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4969. -- Resolution: Fixed Fix Version/s: 1.3.0 > Add binaryRecords support to streaming > ---

[jira] [Updated] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5585: --- Priority: Critical (was: Major) > Flaky test: Python regression > ---

[jira] [Updated] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5585: --- Labels: flaky-test (was: ) > Flaky test: Python regression > - >

[jira] [Created] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5585: -- Summary: Flaky test: Python regression Key: SPARK-5585 URL: https://issues.apache.org/jira/browse/SPARK-5585 Project: Spark Issue Type: Bug Com

[jira] [Updated] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5585: --- Affects Version/s: 1.3.0 > Flaky test: Python regression > - > >

[jira] [Updated] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5529: --- Description: When I run a spark job, one executor is hold, after 120s, blockManager is removed by dri

[jira] [Closed] (SPARK-5583) Support unique join in hive context

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-5583. -- Resolution: Won't Fix Going to close this one as won't fix since it is a weird syntax that only Hive ha

[jira] [Commented] (SPARK-5475) Java 8 tests are like maintenance overhead.

2015-02-03 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304662#comment-14304662 ] Prashant Sharma commented on SPARK-5475: And this is how it looks after running in

[jira] [Updated] (SPARK-5584) Add Maven Enforcer Plugin dependencyConvergence rule (fail false)

2015-02-03 Thread Markus Dale (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Markus Dale updated SPARK-5584: --- Description: The Spark Maven build uses the Maven Enforcer plugin but does not have a rule for depend

[jira] [Resolved] (SPARK-5237) UDTF don't work with multi-alias of multi-columns as output on SparK SQL

2015-02-03 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-5237. Resolution: Duplicate SPARK-5383 should solved this. > UDTF don't work with multi-alias of multi-co

[jira] [Created] (SPARK-5584) Add Maven Enforcer Plugin dependencyConvergence rule (fail false)

2015-02-03 Thread Markus Dale (JIRA)
Markus Dale created SPARK-5584: -- Summary: Add Maven Enforcer Plugin dependencyConvergence rule (fail false) Key: SPARK-5584 URL: https://issues.apache.org/jira/browse/SPARK-5584 Project: Spark

[jira] [Resolved] (SPARK-4795) Redesign the "primitive type => Writable" implicit APIs to make them be activated automatically

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-4795. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Shixiong Zhu > Redesign the "primit

[jira] [Commented] (SPARK-5583) Support unique join in hive context

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304624#comment-14304624 ] Apache Spark commented on SPARK-5583: - User 'scwf' has created a pull request for this

[jira] [Resolved] (SPARK-5578) Provide a convenient way for Scala users to use UDFs

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5578. Resolution: Fixed Fix Version/s: 1.3.0 > Provide a convenient way for Scala users to use UDFs

[jira] [Created] (SPARK-5583) Support unique join in hive context

2015-02-03 Thread wangfei (JIRA)
wangfei created SPARK-5583: -- Summary: Support unique join in hive context Key: SPARK-5583 URL: https://issues.apache.org/jira/browse/SPARK-5583 Project: Spark Issue Type: Improvement Compo

[jira] [Commented] (SPARK-5367) support star expression in udf

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304611#comment-14304611 ] Apache Spark commented on SPARK-5367: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-5582) History server does not list anything if log root contains an empty directory

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304599#comment-14304599 ] Apache Spark commented on SPARK-5582: - User 'vanzin' has created a pull request for th

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Tor Myklebust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304588#comment-14304588 ] Tor Myklebust commented on SPARK-5472: -- If the data in the underlying table changes,

[jira] [Created] (SPARK-5582) History server does not list anything if log root contains an empty directory

2015-02-03 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-5582: - Summary: History server does not list anything if log root contains an empty directory Key: SPARK-5582 URL: https://issues.apache.org/jira/browse/SPARK-5582 Project

[jira] [Resolved] (SPARK-2440) Enable HistoryServer to display lots of Application History

2015-02-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-2440. --- Resolution: Fixed I'll mark this as fixed since the current history server doesn't have that

[jira] [Commented] (SPARK-5260) Expose JsonRDD.allKeysWithValueTypes() in a utility class

2015-02-03 Thread Corey J. Nolet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304577#comment-14304577 ] Corey J. Nolet commented on SPARK-5260: --- I'm thinking all the schema-specific functi

[jira] [Closed] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi closed SPARK-5580. Resolution: Fixed > Grep bug in compute-classpath.sh > > >

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:40 AM: -

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:39 AM: -

[jira] [Created] (SPARK-5581) When writing sorted map output file, avoid open / close between each partition

2015-02-03 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-5581: - Summary: When writing sorted map output file, avoid open / close between each partition Key: SPARK-5581 URL: https://issues.apache.org/jira/browse/SPARK-5581 Project: Spark

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:37 AM: -

[jira] [Updated] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-5580: - Affects Version/s: 1.2.0 > Grep bug in compute-classpath.sh > > >

[jira] [Updated] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-5580: - Fix Version/s: 1.3.0 > Grep bug in compute-classpath.sh > > >

[jira] [Created] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
Yadong Qi created SPARK-5580: Summary: Grep bug in compute-classpath.sh Key: SPARK-5580 URL: https://issues.apache.org/jira/browse/SPARK-5580 Project: Spark Issue Type: Bug Reporter:

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:29 AM: -

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:27 AM: -

[jira] [Commented] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304509#comment-14304509 ] Lianhui Wang commented on SPARK-5529: - the phenomenon is: blockManagerSlave is timeout

[jira] [Commented] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-03 Thread Corey J. Nolet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304510#comment-14304510 ] Corey J. Nolet commented on SPARK-5140: --- I think the problem is that when actions ar

[jira] [Closed] (SPARK-5526) expression [date '2011-01-01' = cast(timestamp('2011-01-01 23:24:25') as date)] return false

2015-02-03 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xukun closed SPARK-5526. Resolution: Fixed the issue is fixed by #4325 > expression [date '2011-01-01' = cast(timestamp('2011-01-01 23:24:25

[jira] [Commented] (SPARK-5577) Create a convenient way for Python users to register SQL UDFs

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304494#comment-14304494 ] Apache Spark commented on SPARK-5577: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-2945) Allow specifying num of executors in the context configuration

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304446#comment-14304446 ] Apache Spark commented on SPARK-2945: - User 'WangTaoTheTonic' has created a pull reque

[jira] [Commented] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304433#comment-14304433 ] Hong Shen commented on SPARK-5529: -- Executor will lost when a akka throw a disassociatedE

[jira] [Resolved] (SPARK-5520) Make FP-Growth implementation take generic item types

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5520. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4340 [https://githu

[jira] [Commented] (SPARK-5153) flaky test of "Reliable Kafka input stream with multiple topics"

2015-02-03 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304423#comment-14304423 ] Saisai Shao commented on SPARK-5153: Hi TD, thanks a lot for your PR, currently I've n

[jira] [Commented] (SPARK-5552) Automated data science AMI creation and data science cluster deployment on EC2

2015-02-03 Thread Florian Verhein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304412#comment-14304412 ] Florian Verhein commented on SPARK-5552: Thanks [~sowen]. So it wouldn't fit in

[jira] [Updated] (SPARK-5574) Utils.createDirectory ignores namePrefix

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5574: --- Assignee: Imran Rashid > Utils.createDirectory ignores namePrefix > --

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Anand Mohan Tumuluri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304383#comment-14304383 ] Anand Mohan Tumuluri commented on SPARK-5472: - Thanks again [~rxin] and [~tmyk

[jira] [Commented] (SPARK-5579) Provide support for project using SQL expression

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304378#comment-14304378 ] Apache Spark commented on SPARK-5579: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-5579) Provide support for project using SQL expression

2015-02-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-5579: -- Summary: Provide support for project using SQL expression Key: SPARK-5579 URL: https://issues.apache.org/jira/browse/SPARK-5579 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-5460) RandomForest should catch exceptions when removing checkpoint files

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304352#comment-14304352 ] Apache Spark commented on SPARK-5460: - User 'x1-' has created a pull request for this

[jira] [Commented] (SPARK-5578) Provide a convenient way for Scala users to use UDFs

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304349#comment-14304349 ] Apache Spark commented on SPARK-5578: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-5578) Provide a convenient way for Scala users to use UDFs

2015-02-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-5578: -- Summary: Provide a convenient way for Scala users to use UDFs Key: SPARK-5578 URL: https://issues.apache.org/jira/browse/SPARK-5578 Project: Spark Issue Type: Su

[jira] [Created] (SPARK-5577) Create a convenient way for Python users to register SQL UDFs

2015-02-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-5577: -- Summary: Create a convenient way for Python users to register SQL UDFs Key: SPARK-5577 URL: https://issues.apache.org/jira/browse/SPARK-5577 Project: Spark Issu

[jira] [Updated] (SPARK-5554) Add more tests and docs for DataFrame Python API

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5554: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-5166 > Add more tests and docs for Data

[jira] [Resolved] (SPARK-5554) Add more tests and docs for DataFrame Python API

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5554. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Davies Liu > Add more tests and doc

[jira] [Comment Edited] (SPARK-5280) Import RDF graphs into GraphX

2015-02-03 Thread lukovnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304070#comment-14304070 ] lukovnikov edited comment on SPARK-5280 at 2/3/15 11:28 PM: st

[jira] [Comment Edited] (SPARK-5280) Import RDF graphs into GraphX

2015-02-03 Thread lukovnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304070#comment-14304070 ] lukovnikov edited comment on SPARK-5280 at 2/3/15 11:27 PM: st

[jira] [Commented] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304298#comment-14304298 ] Joseph K. Bradley commented on SPARK-5021: -- That BLAS implementation is actually

[jira] [Resolved] (SPARK-4986) Graceful shutdown for Spark Streaming does not work in Standalone cluster mode

2015-02-03 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4986. -- Resolution: Fixed Fix Version/s: 1.2.2 > Graceful shutdown for Spark Streaming does not w

[jira] [Commented] (SPARK-5557) spark-shell failed to start

2015-02-03 Thread Ben Mabey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304252#comment-14304252 ] Ben Mabey commented on SPARK-5557: -- I ran a git bisect and this is the first bad commit:

[jira] [Commented] (SPARK-5576) saveAsTable into Hive fails due to duplicate columns

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304229#comment-14304229 ] Apache Spark commented on SPARK-5576: - User 'danosipov' has created a pull request for

[jira] [Created] (SPARK-5576) saveAsTable into Hive fails due to duplicate columns

2015-02-03 Thread Dan Osipov (JIRA)
Dan Osipov created SPARK-5576: - Summary: saveAsTable into Hive fails due to duplicate columns Key: SPARK-5576 URL: https://issues.apache.org/jira/browse/SPARK-5576 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5574) Utils.createDirectory ignores namePrefix

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304190#comment-14304190 ] Apache Spark commented on SPARK-5574: - User 'squito' has created a pull request for th

[jira] [Created] (SPARK-5575) Artificial neural networks for MLlib deep learning

2015-02-03 Thread Alexander Ulanov (JIRA)
Alexander Ulanov created SPARK-5575: --- Summary: Artificial neural networks for MLlib deep learning Key: SPARK-5575 URL: https://issues.apache.org/jira/browse/SPARK-5575 Project: Spark Issue

[jira] [Created] (SPARK-5574) Utils.createDirectory ignores namePrefix

2015-02-03 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-5574: --- Summary: Utils.createDirectory ignores namePrefix Key: SPARK-5574 URL: https://issues.apache.org/jira/browse/SPARK-5574 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5140: --- Fix Version/s: (was: 1.2.1) (was: 1.3.0) > Two RDDs which are sched

[jira] [Commented] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304114#comment-14304114 ] Sean Owen commented on SPARK-5140: -- Is this not substantially answered by just materializ

[jira] [Created] (SPARK-5573) Support explode in DataFrame DSL

2015-02-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-5573: -- Summary: Support explode in DataFrame DSL Key: SPARK-5573 URL: https://issues.apache.org/jira/browse/SPARK-5573 Project: Spark Issue Type: Sub-task Com

[jira] [Comment Edited] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304099#comment-14304099 ] Manoj Kumar edited comment on SPARK-5021 at 2/3/15 10:01 PM: -

[jira] [Comment Edited] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304099#comment-14304099 ] Manoj Kumar edited comment on SPARK-5021 at 2/3/15 10:02 PM: -

[jira] [Commented] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304099#comment-14304099 ] Manoj Kumar commented on SPARK-5021: Hi. I'm almost there. I have one last question.

[jira] [Resolved] (SPARK-5153) flaky test of "Reliable Kafka input stream with multiple topics"

2015-02-03 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-5153. -- Resolution: Fixed Fix Version/s: 1.2.2 1.3.0 > flaky test of "Reliable

[jira] [Commented] (SPARK-5280) Import RDF graphs into GraphX

2015-02-03 Thread lukovnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304070#comment-14304070 ] lukovnikov commented on SPARK-5280: --- started working on it: https://github.com/lukovnik

[jira] [Commented] (SPARK-5388) Provide a stable application submission gateway in standalone cluster mode

2015-02-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304052#comment-14304052 ] Marcelo Vanzin commented on SPARK-5388: --- Hi Patrick, Most of my questions are relat

[jira] [Commented] (SPARK-5388) Provide a stable application submission gateway in standalone cluster mode

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304010#comment-14304010 ] Patrick Wendell commented on SPARK-5388: The intention for this is really just to

[jira] [Commented] (SPARK-5548) Flaky test: org.apache.spark.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303992#comment-14303992 ] Apache Spark commented on SPARK-5548: - User 'jacek-lewandowski' has created a pull req

[jira] [Commented] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303982#comment-14303982 ] Yin Huai commented on SPARK-5420: - h3. End user APIs added to SQLContext (load related) h4

[jira] [Commented] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303967#comment-14303967 ] Yin Huai commented on SPARK-5420: - I am copying the summary of write related interfaces fr

[jira] [Comment Edited] (SPARK-5501) Write support for the data source API

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303760#comment-14303760 ] Yin Huai edited comment on SPARK-5501 at 2/3/15 8:55 PM: - h3. End

[jira] [Comment Edited] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303771#comment-14303771 ] Yin Huai edited comment on SPARK-5420 at 2/3/15 8:54 PM: - This JIR

[jira] [Resolved] (SPARK-4768) Add Support For Impala Encoded Timestamp (INT96)

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4768. - Resolution: Duplicate SPARK-4987 has resolved the issues. I am resolving this one. > Add Support For Impa

[jira] [Updated] (SPARK-4768) Add Support For Impala Encoded Timestamp (INT96)

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4768: Fix Version/s: 1.3.0 > Add Support For Impala Encoded Timestamp (INT96) > --

[jira] [Comment Edited] (SPARK-4768) Add Support For Impala Encoded Timestamp (INT96)

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303960#comment-14303960 ] Yin Huai edited comment on SPARK-4768 at 2/3/15 8:51 PM: - SPARK-49

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Tor Myklebust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303961#comment-14303961 ] Tor Myklebust commented on SPARK-5472: -- It probably does not handle SQL ARRAY types i

[jira] [Commented] (SPARK-5548) Flaky test: org.apache.spark.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-03 Thread Jacek Lewandowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303953#comment-14303953 ] Jacek Lewandowski commented on SPARK-5548: -- You are absolutely right [~joshrosen]

[jira] [Resolved] (SPARK-4987) Parquet support for timestamp type

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4987. - Resolution: Fixed Fix Version/s: 1.3.0 Thank you [~adrian-wang]! > Parquet support for timestamp t

[jira] [Resolved] (SPARK-4709) Spark SQL support error reading Parquet with timestamp type field

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4709. - Resolution: Duplicate Target Version/s: (was: 1.1.0) Seems it duplicates SPARK-4987. I am reso

[jira] [Updated] (SPARK-4987) Parquet support for timestamp type

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4987: Target Version/s: 1.3.0 > Parquet support for timestamp type > -- > >

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Tor Myklebust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303949#comment-14303949 ] Tor Myklebust commented on SPARK-5472: -- This is good feedback. Thanks. You don't ac

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303943#comment-14303943 ] Reynold Xin commented on SPARK-5472: Actually I think it already supports arbitrary qu

[jira] [Comment Edited] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303933#comment-14303933 ] Reynold Xin edited comment on SPARK-5472 at 2/3/15 8:39 PM: Wh

[jira] [Issue Comment Deleted] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5472: --- Comment: was deleted (was: What if we expand the JDBC data source to support arbitrary queries? ) >

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303933#comment-14303933 ] Reynold Xin commented on SPARK-5472: What if we expand the JDBC data source to support

[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303934#comment-14303934 ] Reynold Xin commented on SPARK-5472: What if we expand the JDBC data source to support

[jira] [Commented] (SPARK-5260) Expose JsonRDD.allKeysWithValueTypes() in a utility class

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303927#comment-14303927 ] Reynold Xin commented on SPARK-5260: BTW I've also added you to the contributor list.

  1   2   >