spark git commit: SPARK-5776 JIRA version not of form x.y.z breaks merge_spark_pr.py

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 99bd50066 - bc57789bb SPARK-5776 JIRA version not of form x.y.z breaks merge_spark_pr.py Consider only x.y.z verisons from JIRA. CC JoshRosen who will probably know this script well. Alternative is to call the version 2.0.0 after all in

spark git commit: [SPARK-5655] Don't chmod700 application files if running in YARN

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.2 53de2378e - 64254eeec [SPARK-5655] Don't chmod700 application files if running in YARN [Was previously PR4507] As per SPARK-5655, recently committed code chmod 700s all application files created on the local fs by a spark executor.

spark git commit: [SPARK-5757][MLLIB] replace SQL JSON usage in model import/export by json4s

2015-02-12 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 e23c8f5c8 - e26c14990 [SPARK-5757][MLLIB] replace SQL JSON usage in model import/export by json4s This PR detaches MLlib model import/export code from SQL's JSON support, and hence unblocks #4544 . yhuai Author: Xiangrui Meng

spark git commit: ignore cache paths for RAT tests

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 9a3ea49f7 - 9a6efbccf ignore cache paths for RAT tests RAT fails on cache paths. add to .rat-excludes Author: Oren Mazor oren.ma...@gmail.com Closes #4569 from orenmazor/apache_master and squashes the following commits: d0c9e7e [Oren

spark git commit: [EC2] Update default Spark version to 1.2.1

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 893d6fd70 - 9c8076502 [EC2] Update default Spark version to 1.2.1 Author: Katsunori Kanda pot...@gmail.com Closes #4566 from potix2/ec2-update-version-1-2-1 and squashes the following commits: 77e7840 [Katsunori Kanda] [EC2] Update

spark git commit: [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain...

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 1d5663e92 - 947b8bd82 [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain... some time since some reasons, it lead to some exception while NMClient start some containers.example:we do not config

spark git commit: [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain...

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 11d108030 - 02d5b32bb [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain... some time since some reasons, it lead to some exception while NMClient start some containers.example:we do not config

spark git commit: [SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 47c73d410 - 1d5663e92 [SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests The changes are summarized in the commit message. Test or test-related code accounts for 90% of the lines changed. Author: Andrew Or

spark git commit: [SPARK-5762] Fix shuffle write time for sort-based shuffle

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 9a1de4b20 - 0040fc509 [SPARK-5762] Fix shuffle write time for sort-based shuffle mateiz was excluding the time to write this final file from the shuffle write time intentional? Author: Kay Ousterhout kayousterh...@gmail.com Closes

spark git commit: [SPARK-5762] Fix shuffle write time for sort-based shuffle

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 629d0143e - 47c73d410 [SPARK-5762] Fix shuffle write time for sort-based shuffle mateiz was excluding the time to write this final file from the shuffle write time intentional? Author: Kay Ousterhout kayousterh...@gmail.com Closes #4559

spark git commit: [SPARK-5645] Added local read bytes/time to task metrics

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master aa4ca8b87 - 893d6fd70 [SPARK-5645] Added local read bytes/time to task metrics ksakellis I stumbled on your JIRA for this yesterday; I know it's assigned to you but I'd already done this for my own uses a while ago so thought I could

spark git commit: [SPARK-5645] Added local read bytes/time to task metrics

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 e3a975d45 - 74f34bb8b [SPARK-5645] Added local read bytes/time to task metrics ksakellis I stumbled on your JIRA for this yesterday; I know it's assigned to you but I'd already done this for my own uses a while ago so thought I could

spark git commit: [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 74f34bb8b - 9a1de4b20 [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath Author: Venkata Ramana G ramana.gollamudihuawei.com Author: Venkata Ramana Gollamudi ramana.gollam...@huawei.com Closes #4561

spark git commit: [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 64254eeec - b78a686eb [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath Author: Venkata Ramana G ramana.gollamudihuawei.com Author: Venkata Ramana Gollamudi ramana.gollam...@huawei.com Closes #4561

spark git commit: [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 9c8076502 - 629d0143e [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath Author: Venkata Ramana G ramana.gollamudihuawei.com Author: Venkata Ramana Gollamudi ramana.gollam...@huawei.com Closes #4561 from

spark git commit: [SPARK-5780] [PySpark] Mute the logging during unit tests

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 11a0d5b6d - bf0d15c52 [SPARK-5780] [PySpark] Mute the logging during unit tests There a bunch of logging coming from driver and worker, it's noisy and scaring, and a lots of exception in it, people are confusing about the tests are

spark git commit: [SPARK-5780] [PySpark] Mute the logging during unit tests

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 9c5454d06 - c7bac577a [SPARK-5780] [PySpark] Mute the logging during unit tests There a bunch of logging coming from driver and worker, it's noisy and scaring, and a lots of exception in it, people are confusing about the tests are

spark git commit: [SPARK-5780] [PySpark] Mute the logging during unit tests

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 26c816e73 - 0bf031582 [SPARK-5780] [PySpark] Mute the logging during unit tests There a bunch of logging coming from driver and worker, it's noisy and scaring, and a lots of exception in it, people are confusing about the tests are

spark git commit: SPARK-5747: Fix wordsplitting bugs in make-distribution.sh

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 947b8bd82 - 26c816e73 SPARK-5747: Fix wordsplitting bugs in make-distribution.sh The `$MVN` command variable may have spaces, so when referring to it, must wrap in quotes. Author: David Y. Ross dyr...@gmail.com Closes #4540 from

spark git commit: SPARK-5747: Fix wordsplitting bugs in make-distribution.sh

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 02d5b32bb - 11a0d5b6d SPARK-5747: Fix wordsplitting bugs in make-distribution.sh The `$MVN` command variable may have spaces, so when referring to it, must wrap in quotes. Author: David Y. Ross dyr...@gmail.com Closes #4540 from

spark git commit: [SPARK-5573][SQL] Add explode to dataframes

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 b0c79daf4 - c7eb9ee2c [SPARK-5573][SQL] Add explode to dataframes Author: Michael Armbrust mich...@databricks.com Closes #4546 from marmbrus/explode and squashes the following commits: eefd33a [Michael Armbrust] whitespace a8d496c

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 f7103b343 - 5c9db4e75 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.2 c7bac577a - d24971a62 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master d5fc51491 - ada993e95 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits are

spark git commit: [SPARK-5755] [SQL] remove unnecessary Add

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ee04a8b19 - d5fc51491 [SPARK-5755] [SQL] remove unnecessary Add explain extended select +key from src; before: == Parsed Logical Plan == 'Project [(0 + 'key) AS _c0#8] 'UnresolvedRelation [src], None == Analyzed Logical Plan ==

spark git commit: [SPARK-5755] [SQL] remove unnecessary Add

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 c7eb9ee2c - f7103b343 [SPARK-5755] [SQL] remove unnecessary Add explain extended select +key from src; before: == Parsed Logical Plan == 'Project [(0 + 'key) AS _c0#8] 'UnresolvedRelation [src], None == Analyzed Logical Plan ==

spark git commit: [SPARK-3299][SQL]Public API in SQLContext to list tables

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 925fd84a1 - edbac178d [SPARK-3299][SQL]Public API in SQLContext to list tables https://issues.apache.org/jira/browse/SPARK-3299 Author: Yin Huai yh...@databricks.com Closes #4547 from yhuai/tables and squashes the following commits:

spark git commit: [SPARK-3299][SQL]Public API in SQLContext to list tables

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master c025a4688 - 1d0596a16 [SPARK-3299][SQL]Public API in SQLContext to list tables https://issues.apache.org/jira/browse/SPARK-3299 Author: Yin Huai yh...@databricks.com Closes #4547 from yhuai/tables and squashes the following commits:

spark git commit: [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference.

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 bf0d15c52 - b0c79daf4 [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference. Author: Yin Huai yh...@databricks.com Closes #4544 from yhuai/jsonUseLongTypeByDefault and squashes the following

spark git commit: [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference.

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 0bf031582 - c352ffbdb [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference. Author: Yin Huai yh...@databricks.com Closes #4544 from yhuai/jsonUseLongTypeByDefault and squashes the following commits:

spark git commit: [SQL] Move SaveMode to SQL package.

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ada993e95 - c025a4688 [SQL] Move SaveMode to SQL package. Author: Yin Huai yh...@databricks.com Closes #4542 from yhuai/moveSaveMode and squashes the following commits: 65a4425 [Yin Huai] Move SaveMode to sql package. Project:

spark git commit: [SQL] Move SaveMode to SQL package.

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 5c9db4e75 - 925fd84a1 [SQL] Move SaveMode to SQL package. Author: Yin Huai yh...@databricks.com Closes #4542 from yhuai/moveSaveMode and squashes the following commits: 65a4425 [Yin Huai] Move SaveMode to sql package. (cherry picked

spark git commit: Revert [SPARK-5762] Fix shuffle write time for sort-based shuffle

2015-02-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 d24971a62 - 0ba065f0a Revert [SPARK-5762] Fix shuffle write time for sort-based shuffle This reverts commit 9c5454d06e56917521a15697c36f76a33a94dd1e. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SQL] Fix docs of SQLContext.tables

2015-02-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 1d0596a16 - 2aea892eb [SQL] Fix docs of SQLContext.tables Author: Yin Huai yh...@databricks.com Closes #4579 from yhuai/tablesDoc and squashes the following commits: 7f8964c [Yin Huai] Fix doc. Project:

spark git commit: [SQL][DOCS] Update sql documentation

2015-02-12 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 e26c14990 - cbd659e5f [SQL][DOCS] Update sql documentation Updated examples using the new api and added DataFrame concept Author: Antonio Navarro Perez ajnava...@users.noreply.github.com Closes #4560 from

spark git commit: [SPARK-3365][SQL]Wrong schema generated for List type

2015-02-12 Thread lian
Repository: spark Updated Branches: refs/heads/master 2aea892eb - 1c8633f3f [SPARK-3365][SQL]Wrong schema generated for List type This PR fix the issue SPARK-3365. The reason is Spark generated wrong schema for the type `List` in `ScalaReflection.scala` for example: the generated schema for

spark git commit: [SPARK-3365][SQL]Wrong schema generated for List type

2015-02-12 Thread lian
Repository: spark Updated Branches: refs/heads/branch-1.3 edbac178d - b9f332ab6 [SPARK-3365][SQL]Wrong schema generated for List type This PR fix the issue SPARK-3365. The reason is Spark generated wrong schema for the type `List` in `ScalaReflection.scala` for example: the generated schema

spark git commit: SPARK-5727 [BUILD] Remove Debian packaging

2015-02-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master a38e23c30 - 9a3ea49f7 SPARK-5727 [BUILD] Remove Debian packaging (for master / 1.4 only) Author: Sean Owen so...@cloudera.com Closes #4526 from srowen/SPARK-5727.2 and squashes the following commits: 83ba49c [Sean Owen] Remove Debian