spark git commit: [SPARK-14254][CORE] Add logs to help investigate the network performance

2016-03-29 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master b66b97cd0 -> 7320f9bd1 [SPARK-14254][CORE] Add logs to help investigate the network performance ## What changes were proposed in this pull request? It would be very helpful for network performance investigation if we log the time spent

spark git commit: [SPARK-14124][SQL] Implement Database-related DDL Commands

2016-03-29 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master e1f684539 -> b66b97cd0 [SPARK-14124][SQL] Implement Database-related DDL Commands What changes were proposed in this pull request? This PR is to implement the following four Database-related DDL commands: - `CREATE DATABASE|SCHEMA

spark git commit: [SPARK-12181] Check Cached unaligned-access capability before using Unsafe

2016-03-29 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 366cac6fb -> e1f684539 [SPARK-12181] Check Cached unaligned-access capability before using Unsafe ## What changes were proposed in this pull request? For MemoryMode.OFF_HEAP, Unsafe.getInt etc. are used with no restriction. However, the

spark git commit: [SPARK-14227][SQL] Add method for printing out generated code for debugging

2016-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master 838cb4583 -> e58c4cb3c [SPARK-14227][SQL] Add method for printing out generated code for debugging ## What changes were proposed in this pull request? This adds `debugCodegen` to the debug package for query execution. ## How was this

spark git commit: [MINOR][SQL] Fix exception message to print string-array correctly.

2016-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master d612228ef -> 838cb4583 [MINOR][SQL] Fix exception message to print string-array correctly. ## What changes were proposed in this pull request? This PR is a simple fix for an exception message to print `string[]` content correctly.

spark git commit: [MINOR][SQL] Fix typos by replacing 'much' with 'match'.

2016-03-29 Thread rxin
Repository: spark Updated Branches: refs/heads/master d26c42982 -> d612228ef [MINOR][SQL] Fix typos by replacing 'much' with 'match'. ## What changes were proposed in this pull request? This PR fixes two trivial typos: 'does not **much**' --> 'does not **match**'. ## How was this patch

spark git commit: [SPARK-10570][CORE] Add version info to json api

2016-03-29 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 15c0b0006 -> d26c42982 [SPARK-10570][CORE] Add version info to json api Add a new api endpoint `/api/v1/version` to retrieve various version info. This PR only adds support for finding the current spark version, however other version

spark git commit: [SPARK-14232][WEBUI] Fix event timeline display issue when an executor is removed with a multiple line reason.

2016-03-29 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 c2ce247ea -> 84ad2544f [SPARK-14232][WEBUI] Fix event timeline display issue when an executor is removed with a multiple line reason. ## What changes were proposed in this pull request? The event timeline doesn't show on job page if

spark git commit: [SPARK-14232][WEBUI] Fix event timeline display issue when an executor is removed with a multiple line reason.

2016-03-29 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master d2a819a63 -> 15c0b0006 [SPARK-14232][WEBUI] Fix event timeline display issue when an executor is removed with a multiple line reason. ## What changes were proposed in this pull request? The event timeline doesn't show on job page if an

spark git commit: [SPARK-13845][CORE][BACKPORT-1.6] Using onBlockUpdated to replace onTaskEnd avioding driver OOM

2016-03-29 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 a7579444d -> c2ce247ea [SPARK-13845][CORE][BACKPORT-1.6] Using onBlockUpdated to replace onTaskEnd avioding driver OOM ## What changes were proposed in this pull request? We have a streaming job using `FlumePollInputStream` always

spark git commit: [SPARK-14154][MLLIB] Simplify the implementation for Kolmogorov–Smirnov test

2016-03-29 Thread srowen
Repository: spark Updated Branches: refs/heads/master a632bb56f -> d2a819a63 [SPARK-14154][MLLIB] Simplify the implementation for Kolmogorov–Smirnov test ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-14154 I just read the code for

spark git commit: [SPARK-14208][SQL] Renames spark.sql.parquet.fileScan

2016-03-29 Thread lian
Repository: spark Updated Branches: refs/heads/master 425bcf6d6 -> a632bb56f [SPARK-14208][SQL] Renames spark.sql.parquet.fileScan ## What changes were proposed in this pull request? Renames SQL option `spark.sql.parquet.fileScan` since now all `HadoopFsRelation` based data sources are

spark git commit: [SPARK-13963][ML] Adding binary toggle param to HashingTF

2016-03-29 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 83775bc78 -> 425bcf6d6 [SPARK-13963][ML] Adding binary toggle param to HashingTF ## What changes were proposed in this pull request? Adding binary toggle parameter to ml.feature.HashingTF, as well as mllib.feature.HashingTF since the

spark git commit: [SPARK-14158][SQL] implement buildReader for json data source

2016-03-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 63b200e8d -> 83775bc78 [SPARK-14158][SQL] implement buildReader for json data source ## What changes were proposed in this pull request? This PR implements buildReader for json data source and enable it in the new data source code path.