spark git commit: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-18 Thread meng
Repository: spark Updated Branches: refs/heads/master e32c0f69f - 1ecfac6e3 [SPARK-6657] [PYSPARK] Fix doc warnings Fixed the following warnings in `make clean html` under `python/docs`: ~~~ /Users/meng/src/spark/python/pyspark/mllib/evaluation.py:docstring of

spark git commit: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-18 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 0e7cd8ff8 - 2c94ffe7e [SPARK-6657] [PYSPARK] Fix doc warnings Fixed the following warnings in `make clean html` under `python/docs`: ~~~ /Users/meng/src/spark/python/pyspark/mllib/evaluation.py:docstring of

spark git commit: [SPARK-7299][SQL] Set precision and scale for Decimal according to JDBC metadata instead of returned BigDecimal

2015-05-18 Thread rxin
Repository: spark Updated Branches: refs/heads/master 775e6f990 - e32c0f69f [SPARK-7299][SQL] Set precision and scale for Decimal according to JDBC metadata instead of returned BigDecimal JIRA: https://issues.apache.org/jira/browse/SPARK-7299 When connecting with oracle db through jdbc, the

spark git commit: [SPARK-7627] [SPARK-7472] DAG visualization: style skipped stages

2015-05-18 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 a95d4e18f - a0ae8ce01 [SPARK-7627] [SPARK-7472] DAG visualization: style skipped stages This patch fixes two things: **SPARK-7627.** Cached RDDs no longer light up on the job page. This is a simple fix. **SPARK-7472.** Display

spark git commit: [SPARK-7272] [MLLIB] User guide for PMML model export

2015-05-18 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 2c94ffe7e - a95d4e18f [SPARK-7272] [MLLIB] User guide for PMML model export https://issues.apache.org/jira/browse/SPARK-7272 Author: Vincenzo Selvaggio vselvag...@hotmail.it Closes #6219 from

spark git commit: [SPARK-7272] [MLLIB] User guide for PMML model export

2015-05-18 Thread meng
Repository: spark Updated Branches: refs/heads/master 1ecfac6e3 - 814b3dabd [SPARK-7272] [MLLIB] User guide for PMML model export https://issues.apache.org/jira/browse/SPARK-7272 Author: Vincenzo Selvaggio vselvag...@hotmail.it Closes #6219 from

spark git commit: [SPARK-6888] [SQL] Make the jdbc driver handling user-definable

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.4 a0ae8ce01 - b41301a13 [SPARK-6888] [SQL] Make the jdbc driver handling user-definable Replace the DriverQuirks with JdbcDialect(s) (and MySQLDialect/PostgresDialect) and allow developers to change the dialects on the fly (for new

spark git commit: [SPARK-6888] [SQL] Make the jdbc driver handling user-definable

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 563bfcc1a - e1ac2a955 [SPARK-6888] [SQL] Make the jdbc driver handling user-definable Replace the DriverQuirks with JdbcDialect(s) (and MySQLDialect/PostgresDialect) and allow developers to change the dialects on the fly (for new JDBCRRDs

spark git commit: [SPARK-7631] [SQL] treenode argString should not print children

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master aa31e431f - fc2480ed1 [SPARK-7631] [SQL] treenode argString should not print children spark-sql explain extended select * from ( select key from src union all select key from src) t; now the spark plan will print children in argString

spark git commit: [SPARK-7631] [SQL] treenode argString should not print children

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.4 65d71bd9f - dbd4ec807 [SPARK-7631] [SQL] treenode argString should not print children spark-sql explain extended select * from ( select key from src union all select key from src) t; now the spark plan will print children in

spark git commit: [SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEquals)

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.4 dbd4ec807 - d6f5f3791 [SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEquals) A modified version of https://github.com/apache/spark/pull/6110, use `semanticEquals` to make it more efficient. Author: Wenchen Fan

spark git commit: [SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEquals)

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master fc2480ed1 - 103c863c2 [SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEquals) A modified version of https://github.com/apache/spark/pull/6110, use `semanticEquals` to make it more efficient. Author: Wenchen Fan

spark git commit: [SPARK-7570] [SQL] Ignores _temporary during partition discovery

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master e1ac2a955 - 010a1c278 [SPARK-7570] [SQL] Ignores _temporary during partition discovery !-- Reviewable:start -- [img src=https://reviewable.io/review_button.png; height=40 alt=Review on

spark git commit: [SPARK-2883] [SQL] ORC data source for Spark SQL

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.4 cf4e04a0c - 65d71bd9f [SPARK-2883] [SQL] ORC data source for Spark SQL This PR updates PR #6135 authored by zhzhan from Hortonworks. This PR implements a Spark SQL data source for accessing ORC files. **NOTE** Although ORC

spark git commit: [SPARK-2883] [SQL] ORC data source for Spark SQL

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 9c7e802a5 - aa31e431f [SPARK-2883] [SQL] ORC data source for Spark SQL This PR updates PR #6135 authored by zhzhan from Hortonworks. This PR implements a Spark SQL data source for accessing ORC files. **NOTE** Although ORC is

spark git commit: [SPARK-7567] [SQL] [follow-up] Use a new flag to set output committer based on mapreduce apis

2015-05-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.4 d6f5f3791 - a385f4b8d [SPARK-7567] [SQL] [follow-up] Use a new flag to set output committer based on mapreduce apis cc liancheng marmbrus Author: Yin Huai yh...@databricks.com Closes #6130 from yhuai/directOutput and squashes the

spark git commit: [SPARK-7673] [SQL] WIP: HadoopFsRelation and ParquetRelation2 performance optimizations

2015-05-18 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 530397ba2 - 9dadf019b [SPARK-7673] [SQL] WIP: HadoopFsRelation and ParquetRelation2 performance optimizations This PR introduces several performance optimizations to `HadoopFsRelation` and `ParquetRelation2`: 1. Moving `FileStatus`

spark git commit: [SPARK-6216] [PYSPARK] check python version of worker with driver

2015-05-18 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 9dadf019b - 32fbd297d [SPARK-6216] [PYSPARK] check python version of worker with driver This PR revert #5404, change to pass the version of python in driver into JVM, check it in worker before deserializing closure, then it can works with

spark git commit: [SPARK-7658] [STREAMING] [WEBUI] Update the mouse behaviors for the timeline graphs

2015-05-18 Thread tdas
Repository: spark Updated Branches: refs/heads/master 32fbd297d - 0b6f503d5 [SPARK-7658] [STREAMING] [WEBUI] Update the mouse behaviors for the timeline graphs 1. If the user click one point of a batch, scroll down to the corresponding batch row and highlight it. And recovery the batch row

spark git commit: [HOTFIX] Fix ORC build break

2015-05-18 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 0b6f503d5 - fcf90b75c [HOTFIX] Fix ORC build break Fix break caused by merging #6225 and #6194. Author: Michael Armbrust mich...@databricks.com Closes #6244 from marmbrus/fixOrcBuildBreak and squashes the following commits: b10e47b

[1/3] spark git commit: [SPARK-7501] [STREAMING] DAG visualization: show DStream operations

2015-05-18 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 ba502ab20 - a475cbc97 http://git-wip-us.apache.org/repos/asf/spark/blob/a475cbc9/core/src/main/scala/org/apache/spark/SparkContext.scala -- diff --git

[2/3] spark git commit: [SPARK-7501] [STREAMING] DAG visualization: show DStream operations

2015-05-18 Thread andrewor14
http://git-wip-us.apache.org/repos/asf/spark/blob/a475cbc9/core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js -- diff --git a/core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js

[3/3] spark git commit: [SPARK-7501] [STREAMING] DAG visualization: show DStream operations

2015-05-18 Thread andrewor14
[SPARK-7501] [STREAMING] DAG visualization: show DStream operations This is similar to #5999, but for streaming. Roughly 200 lines are tests. One thing to note here is that we already do some kind of scoping thing for call sites, so this patch adds the new RDD operation scoping logic in the

spark git commit: Version updates for Spark 1.4.0

2015-05-18 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 dd9f873be - 9d0b7fb71 Version updates for Spark 1.4.0 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9d0b7fb7 Tree:

spark git commit: [SPARK-7681] [MLLIB] Add SparseVector support for gemv

2015-05-18 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 9c485484f - dd9f873be [SPARK-7681] [MLLIB] Add SparseVector support for gemv JIRA: https://issues.apache.org/jira/browse/SPARK-7681 Author: Liang-Chi Hsieh vii...@gmail.com Closes #6209 from viirya/sparsevector_gemv and squashes the

spark git commit: [SPARK-7681] [MLLIB] Add SparseVector support for gemv

2015-05-18 Thread meng
Repository: spark Updated Branches: refs/heads/master 3a6003866 - d03638cc2 [SPARK-7681] [MLLIB] Add SparseVector support for gemv JIRA: https://issues.apache.org/jira/browse/SPARK-7681 Author: Liang-Chi Hsieh vii...@gmail.com Closes #6209 from viirya/sparsevector_gemv and squashes the

spark git commit: [SPARK-7692] Updated Kinesis examples

2015-05-18 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.4 9188ad8dd - 9c485484f [SPARK-7692] Updated Kinesis examples - Updated Kinesis examples to use stable API - Cleaned up comments, etc. - Renamed KinesisWordCountProducerASL to KinesisWordProducerASL Author: Tathagata Das