[jira] [Created] (SPARK-7718) Speed up data source partitioning by avoiding cleaning closures

2015-05-19 Thread Andrew Or (JIRA)
Andrew Or created SPARK-7718: Summary: Speed up data source partitioning by avoiding cleaning closures Key: SPARK-7718 URL: https://issues.apache.org/jira/browse/SPARK-7718 Project: Spark Issue

[jira] [Assigned] (SPARK-7585) User guide update for VectorIndexer

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7585: --- Assignee: Apache Spark (was: Joseph K. Bradley) User guide update for VectorIndexer

[jira] [Assigned] (SPARK-7585) User guide update for VectorIndexer

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7585: --- Assignee: Joseph K. Bradley (was: Apache Spark) User guide update for VectorIndexer

[jira] [Closed] (SPARK-7458) Check 1.3- 1.4 MLlib API compliance using java-compliance-checker

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7458. Resolution: Done Fix Version/s: 1.4.0 Check 1.3- 1.4 MLlib API compliance using

[jira] [Assigned] (SPARK-7320) Add rollup and cube support to DataFrame DSL

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7320: --- Assignee: Cheng Hao (was: Apache Spark) Add rollup and cube support to DataFrame DSL

[jira] [Updated] (SPARK-6270) Standalone Master hangs when streaming job completes

2015-05-19 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6270: - Target Version/s: (was: 1.4.0) Standalone Master hangs when streaming job completes

[jira] [Resolved] (SPARK-3323) yarn website's Tracking UI links to the Standby RM

2015-05-19 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang resolved SPARK-3323. - Resolution: Fixed yarn website's Tracking UI links to the Standby RM

[jira] [Updated] (SPARK-6053) Support model save/load in Python's ALS.

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6053: - Target Version/s: 1.3.0 (was: 1.4.0) Support model save/load in Python's ALS.

[jira] [Updated] (SPARK-6685) Use DSYRK to compute AtA in ALS

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6685: - Target Version/s: 1.5.0 (was: 1.4.0) Use DSYRK to compute AtA in ALS

[jira] [Assigned] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7723: --- Assignee: Apache Spark Fix string interpolation in pipeline examples

[jira] [Assigned] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7723: --- Assignee: (was: Apache Spark) Fix string interpolation in pipeline examples

[jira] [Commented] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549968#comment-14549968 ] Apache Spark commented on SPARK-7723: - User 'tuxdna' has created a pull request for

[jira] [Commented] (SPARK-7458) Check 1.3- 1.4 MLlib API compliance using java-compliance-checker

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549861#comment-14549861 ] Xiangrui Meng commented on SPARK-7458: -- [~josephkb] The following changes would break

[jira] [Commented] (SPARK-7320) Add rollup and cube support to DataFrame DSL

2015-05-19 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549916#comment-14549916 ] Cheng Hao commented on SPARK-7320: -- Sorry, it's not depends on SPARK-7235 any more. Add

[jira] [Closed] (SPARK-6997) Convert StringType in LocalTableScan

2015-05-19 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang closed SPARK-6997. --- Resolution: Won't Fix Convert StringType in LocalTableScan -

[jira] [Commented] (SPARK-6997) Convert StringType in LocalTableScan

2015-05-19 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549924#comment-14549924 ] Fei Wang commented on SPARK-6997: - this is not nessarry since we have convert the data

[jira] [Commented] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549956#comment-14549956 ] Sean Owen commented on SPARK-7699: -- Agree, but this logic change only also happens a bit

[jira] [Commented] (SPARK-7718) Speed up data source partitioning by avoiding cleaning closures

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549846#comment-14549846 ] Apache Spark commented on SPARK-7718: - User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-7718) Speed up data source partitioning by avoiding cleaning closures

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7718: --- Assignee: Apache Spark (was: Andrew Or) Speed up data source partitioning by avoiding

[jira] [Assigned] (SPARK-7718) Speed up data source partitioning by avoiding cleaning closures

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7718: --- Assignee: Andrew Or (was: Apache Spark) Speed up data source partitioning by avoiding

[jira] [Updated] (SPARK-7719) Java 6 code in UnsafeShuffleWriterSuite

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7719: --- Description: This was causing a compile failure because emptyIterator() is not exposed in

[jira] [Assigned] (SPARK-7320) Add rollup and cube support to DataFrame DSL

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7320: --- Assignee: Apache Spark (was: Cheng Hao) Add rollup and cube support to DataFrame DSL

[jira] [Commented] (SPARK-7661) Support for dynamic allocation of resources in Kinesis Spark Streaming

2015-05-19 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549911#comment-14549911 ] Tathagata Das commented on SPARK-7661: -- Currently there is not way to scale up the

[jira] [Commented] (SPARK-7320) Add rollup and cube support to DataFrame DSL

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549915#comment-14549915 ] Apache Spark commented on SPARK-7320: - User 'chenghao-intel' has created a pull

[jira] [Closed] (SPARK-7289) Combine Limit and Sort to avoid total ordering

2015-05-19 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang closed SPARK-7289. --- Resolution: Won't Fix use usually not write sql like this Combine Limit and Sort to avoid total ordering

[jira] [Updated] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

2015-05-19 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6270: - Summary: Standalone Master hangs when streaming job completes and event logging is enabled (was:

[jira] [Updated] (SPARK-7722) Style checks do not run for Kinesis on Jenkins

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7722: --- Component/s: Project Infra Style checks do not run for Kinesis on Jenkins

[jira] [Commented] (SPARK-7585) User guide update for VectorIndexer

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549841#comment-14549841 ] Apache Spark commented on SPARK-7585: - User 'jkbradley' has created a pull request for

[jira] [Created] (SPARK-7719) Java 6 code in UnsafeShuffleWriterSuite

2015-05-19 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-7719: -- Summary: Java 6 code in UnsafeShuffleWriterSuite Key: SPARK-7719 URL: https://issues.apache.org/jira/browse/SPARK-7719 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-7720) MLlib 1.3 - 1.4 migration guide

2015-05-19 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7720: Summary: MLlib 1.3 - 1.4 migration guide Key: SPARK-7720 URL: https://issues.apache.org/jira/browse/SPARK-7720 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-7443) MLlib 1.4 QA plan

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7443: - Description: TODO: create JIRAs for each task and assign them accordingly. h2. API * Check API

[jira] [Commented] (SPARK-7721) Generate test coverage report from Python

2015-05-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549873#comment-14549873 ] Reynold Xin commented on SPARK-7721: [~davies] any idea on this? Generate test

[jira] [Created] (SPARK-7721) Generate test coverage report from Python

2015-05-19 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7721: -- Summary: Generate test coverage report from Python Key: SPARK-7721 URL: https://issues.apache.org/jira/browse/SPARK-7721 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-7721) Generate test coverage report from Python

2015-05-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7721: --- Component/s: Tests PySpark Target Version/s: 1.5.0 Generate test

[jira] [Closed] (SPARK-5240) Adding `createDataSourceTable` interface to Catalog

2015-05-19 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang closed SPARK-5240. --- Resolution: Won't Fix Adding `createDataSourceTable` interface to Catalog

[jira] [Commented] (SPARK-7458) Check 1.3- 1.4 MLlib API compliance using java-compliance-checker

2015-05-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549936#comment-14549936 ] Joseph K. Bradley commented on SPARK-7458: -- Thanks for the list! I'll copy these

[jira] [Commented] (SPARK-7720) MLlib 1.3 - 1.4 migration guide

2015-05-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549937#comment-14549937 ] Joseph K. Bradley commented on SPARK-7720: -- [~mengxr] Should this be separate

[jira] [Created] (SPARK-7722) Style checks do not run for Kinesis on Jenkins

2015-05-19 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-7722: -- Summary: Style checks do not run for Kinesis on Jenkins Key: SPARK-7722 URL: https://issues.apache.org/jira/browse/SPARK-7722 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-7581) User guide update for PolynomialExpansion

2015-05-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7581. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6113

[jira] [Created] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Saleem Ansari (JIRA)
Saleem Ansari created SPARK-7723: Summary: Fix string interpolation in pipeline examples Key: SPARK-7723 URL: https://issues.apache.org/jira/browse/SPARK-7723 Project: Spark Issue Type:

[jira] [Updated] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7723: - Priority: Trivial (was: Minor) Target Version/s: 1.4.0 Assignee: Saleem Ansari

[jira] [Created] (SPARK-7724) Add support for Intersect and Except in Catalyst DSL

2015-05-19 Thread Santiago M. Mola (JIRA)
Santiago M. Mola created SPARK-7724: --- Summary: Add support for Intersect and Except in Catalyst DSL Key: SPARK-7724 URL: https://issues.apache.org/jira/browse/SPARK-7724 Project: Spark

[jira] [Commented] (SPARK-7701) UDT not working

2015-05-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550087#comment-14550087 ] Liang-Chi Hsieh commented on SPARK-7701: You should wrap ExamplePoint in Products

[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550102#comment-14550102 ] Sean Owen commented on SPARK-7670: -- So [~pwendell] tracked this down to a failure in the

[jira] [Commented] (SPARK-6533) Allow using wildcard and other file pattern in Parquet DataSource

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550114#comment-14550114 ] Cheng Lian commented on SPARK-6533: --- {{parquetFile(...)}} should also work as it

[jira] [Resolved] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7723. -- Resolution: Fixed Fix Version/s: 1.4.0 Resolved by https://github.com/apache/spark/pull/6258

[jira] [Commented] (SPARK-7684) TestHive.reset complains Database does not exist: default

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550132#comment-14550132 ] Cheng Lian commented on SPARK-7684: --- This looks quite similar to this one

[jira] [Commented] (SPARK-6533) Allow using wildcard and other file pattern in Parquet DataSource

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550018#comment-14550018 ] Cheng Lian commented on SPARK-6533: --- This ticket duplicates SPARK-3928, which has been

[jira] [Created] (SPARK-7725) --py-files doesn't seem to work in YARN cluster mode

2015-05-19 Thread Shay Rojansky (JIRA)
Shay Rojansky created SPARK-7725: Summary: --py-files doesn't seem to work in YARN cluster mode Key: SPARK-7725 URL: https://issues.apache.org/jira/browse/SPARK-7725 Project: Spark Issue

[jira] [Assigned] (SPARK-7637) StructType.merge slow with large nenormalised tables O(N2)

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7637: --- Assignee: Apache Spark StructType.merge slow with large nenormalised tables O(N2)

[jira] [Assigned] (SPARK-7637) StructType.merge slow with large nenormalised tables O(N2)

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7637: --- Assignee: (was: Apache Spark) StructType.merge slow with large nenormalised tables

[jira] [Commented] (SPARK-7637) StructType.merge slow with large nenormalised tables O(N2)

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550080#comment-14550080 ] Apache Spark commented on SPARK-7637: - User 'rowan000' has created a pull request for

[jira] [Comment Edited] (SPARK-7701) UDT not working

2015-05-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550087#comment-14550087 ] Liang-Chi Hsieh edited comment on SPARK-7701 at 5/19/15 9:12 AM:

[jira] [Created] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-7726: -- Summary: Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher] Key: SPARK-7726 URL: https://issues.apache.org/jira/browse/SPARK-7726 Project:

[jira] [Resolved] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7670. Resolution: Duplicate Fix Version/s: SPARK-7726 Failure when building with scala

[jira] [Commented] (SPARK-5777) Completes data source filter types and remove CatalystScan

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550118#comment-14550118 ] Cheng Lian commented on SPARK-5777: --- Yeah, leaving it in {{experimental}} LGTM.

[jira] [Commented] (SPARK-5777) Completes data source filter types and remove CatalystScan

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550123#comment-14550123 ] Cheng Lian commented on SPARK-5777: --- I meant, leaving it for experimentation LGTM.

[jira] [Commented] (SPARK-6533) Allow using wildcard and other file pattern in Parquet DataSource

2015-05-19 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550028#comment-14550028 ] Jianshi Huang commented on SPARK-6533: -- Ah, right. Tested in 1.4.0, sqlc.load works!

[jira] [Resolved] (SPARK-6533) Allow using wildcard and other file pattern in Parquet DataSource

2015-05-19 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang resolved SPARK-6533. -- Resolution: Won't Fix Don't use sqlc.parquetFile(...), use sqlc.load(..., parquet) instead, or

[jira] [Commented] (SPARK-7701) UDT not working

2015-05-19 Thread bogdan baraila (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550098#comment-14550098 ] bogdan baraila commented on SPARK-7701: --- I've already made my ExamplePoint a case

[jira] [Commented] (SPARK-7092) Update spark scala version to 2.11.6

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550124#comment-14550124 ] Patrick Wendell commented on SPARK-7092: This is reopened because it caused

[jira] [Reopened] (SPARK-7092) Update spark scala version to 2.11.6

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-7092: Update spark scala version to 2.11.6

[jira] [Resolved] (SPARK-5777) Completes data source filter types and remove CatalystScan

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5777. --- Resolution: Won't Fix Completes data source filter types and remove CatalystScan

[jira] [Commented] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550223#comment-14550223 ] meiyoula commented on SPARK-7699: - [~srowen] you said I'm just talking about the case

[jira] [Comment Edited] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550223#comment-14550223 ] meiyoula edited comment on SPARK-7699 at 5/19/15 10:55 AM: ---

[jira] [Comment Edited] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550223#comment-14550223 ] meiyoula edited comment on SPARK-7699 at 5/19/15 10:58 AM: ---

[jira] [Comment Edited] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550223#comment-14550223 ] meiyoula edited comment on SPARK-7699 at 5/19/15 10:57 AM: ---

[jira] [Closed] (SPARK-7709) spark-submit option to quit after submitting in cluster mode

2015-05-19 Thread Shay Rojansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Rojansky closed SPARK-7709. Resolution: Duplicate Oops, it seems this was already implemented... spark-submit option to quit

[jira] [Commented] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550249#comment-14550249 ] Sean Owen commented on SPARK-7699: -- Isn't this the normal state of a Spark app, run with

[jira] [Commented] (SPARK-7701) UDT not working

2015-05-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550143#comment-14550143 ] Liang-Chi Hsieh commented on SPARK-7701: I meant wrapping ExamplePoint in

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550142#comment-14550142 ] Iulian Dragos commented on SPARK-7726: -- This fix in 2.11.3 looks like the culprit

[jira] [Commented] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550157#comment-14550157 ] Saisai Shao commented on SPARK-7699: I think for the first time called

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550170#comment-14550170 ] Iulian Dragos commented on SPARK-7726: -- The problem is different visibility rules in

[jira] [Comment Edited] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550170#comment-14550170 ] Iulian Dragos edited comment on SPARK-7726 at 5/19/15 10:11 AM:

[jira] [Assigned] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7726: --- Assignee: Apache Spark Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

[jira] [Assigned] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7726: --- Assignee: (was: Apache Spark) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2--[2.11.3 or higher]

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550193#comment-14550193 ] Apache Spark commented on SPARK-7726: - User 'dragos' has created a pull request for

[jira] [Commented] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550163#comment-14550163 ] Saisai Shao commented on SPARK-7699: The problem is as you mentioned here: {code} val

[jira] [Commented] (SPARK-7699) Config spark.dynamicAllocation.initialExecutors has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550185#comment-14550185 ] Sean Owen commented on SPARK-7699: -- The parameter has meaning if the system isn't idle,

[jira] [Created] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-19 Thread Santiago M. Mola (JIRA)
Santiago M. Mola created SPARK-7727: --- Summary: Avoid inner classes in RuleExecutor Key: SPARK-7727 URL: https://issues.apache.org/jira/browse/SPARK-7727 Project: Spark Issue Type:

[jira] [Updated] (SPARK-4412) Parquet logger cannot be configured

2015-05-19 Thread Yana Kadiyska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yana Kadiyska updated SPARK-4412: - Affects Version/s: 1.3.1 Parquet logger cannot be configured

[jira] [Created] (SPARK-7728) SQL: cannot add timestamp partition as string

2015-05-19 Thread Kanwaljit Singh (JIRA)
Kanwaljit Singh created SPARK-7728: -- Summary: SQL: cannot add timestamp partition as string Key: SPARK-7728 URL: https://issues.apache.org/jira/browse/SPARK-7728 Project: Spark Issue Type:

[jira] [Updated] (SPARK-7728) SQL: cannot add timestamp partition as string

2015-05-19 Thread Kanwaljit Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kanwaljit Singh updated SPARK-7728: --- Description: create table spark_test(i int) partitioned by (p timestamp); alter table

[jira] [Created] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Archit Thakur (JIRA)
Archit Thakur created SPARK-7729: Summary: Executor which has been killed should also be displayed on Executors Tab. Key: SPARK-7729 URL: https://issues.apache.org/jira/browse/SPARK-7729 Project:

[jira] [Assigned] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7729: --- Assignee: Apache Spark Executor which has been killed should also be displayed on Executors

[jira] [Commented] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Archit Thakur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550327#comment-14550327 ] Archit Thakur commented on SPARK-7729: -- pull request

[jira] [Commented] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550325#comment-14550325 ] Apache Spark commented on SPARK-7729: - User 'archit279thakur' has created a pull

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

2015-05-19 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551795#comment-14551795 ] Nicholas Chammas commented on SPARK-7640: - [~brdwrd] - According to [this doc on

[jira] [Commented] (SPARK-4352) Incorporate locality preferences in dynamic allocation requests

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551836#comment-14551836 ] Saisai Shao commented on SPARK-4352: Hi Sandy, I retrieved back the old code which

[jira] [Commented] (SPARK-7537) Audit new public Scala APIs for MLlib 1.4

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551863#comment-14551863 ] Xiangrui Meng commented on SPARK-7537: -- 1. toPMML should be experimental. Shall we

[jira] [Resolved] (SPARK-6246) spark-ec2 can't handle clusters with 100 nodes

2015-05-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-6246. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request

[jira] [Updated] (SPARK-7165) Sort Merge Join for outer joins

2015-05-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7165: --- Target Version/s: 1.5.0 Sort Merge Join for outer joins ---

[jira] [Assigned] (SPARK-7747) Document spark.sql.planner.externalSort option

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7747: --- Assignee: Apache Spark Document spark.sql.planner.externalSort option

[jira] [Commented] (SPARK-7747) Document spark.sql.planner.externalSort option

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551524#comment-14551524 ] Apache Spark commented on SPARK-7747: - User 'lucamartinetti' has created a pull

[jira] [Assigned] (SPARK-7747) Document spark.sql.planner.externalSort option

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7747: --- Assignee: (was: Apache Spark) Document spark.sql.planner.externalSort option

[jira] [Commented] (SPARK-4352) Incorporate locality preferences in dynamic allocation requests

2015-05-19 Thread Kevin Cox (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551667#comment-14551667 ] Kevin Cox commented on SPARK-4352: -- I believe the idea of this issue was to determine the

[jira] [Commented] (SPARK-7711) startTime() is missing

2015-05-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551498#comment-14551498 ] holdenk commented on SPARK-7711: I can add this. startTime() is missing

[jira] [Created] (SPARK-7747) Document spark.sql.planner.externalSort option

2015-05-19 Thread Luca Martinetti (JIRA)
Luca Martinetti created SPARK-7747: -- Summary: Document spark.sql.planner.externalSort option Key: SPARK-7747 URL: https://issues.apache.org/jira/browse/SPARK-7747 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4352) Incorporate locality preferences in dynamic allocation requests

2015-05-19 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551677#comment-14551677 ] Sandy Ryza commented on SPARK-4352: --- Thanks for posting this Saisai. Can you export and

[jira] [Resolved] (SPARK-7744) Distributed matrix section in MLlib Data Types documentation should be reordered.

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7744. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Issue resolved by pull

  1   2   3   >