[3/5] spark git commit: [SPARK-3468] [WEBUI] Timeline-View feature

http://git-wip-us.apache.org/repos/asf/spark/blob/7fe0f3f2/core/src/main/resources/org/apache/spark/ui/static/vis.min.css -- diff --git a/core/src/main/resources/org/apache/spark/ui/static/vis.min.css b/core/src/main/resources/org

[1/5] spark git commit: [SPARK-3468] [WEBUI] Timeline-View feature

Repository: spark Updated Branches: refs/heads/master c24aeb6a3 -> 7fe0f3f2b http://git-wip-us.apache.org/repos/asf/spark/blob/7fe0f3f2/core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala -- diff --git a/core/src/mai

[4/5] spark git commit: [SPARK-3468] [WEBUI] Timeline-View feature

http://git-wip-us.apache.org/repos/asf/spark/blob/7fe0f3f2/core/src/main/resources/org/apache/spark/ui/static/vis.map -- diff --git a/core/src/main/resources/org/apache/spark/ui/static/vis.map b/core/src/main/resources/org/apache/

[5/5] spark git commit: [SPARK-3468] [WEBUI] Timeline-View feature

[SPARK-3468] [WEBUI] Timeline-View feature I sometimes trouble-shoot and analyse the cause of long time spending job. At the time, I find the stages which spends long time or fails, then I find the tasks which spends long time or fails, next I analyse the proportion of each phase in a task. An

[2/5] spark git commit: [SPARK-3468] [WEBUI] Timeline-View feature

http://git-wip-us.apache.org/repos/asf/spark/blob/7fe0f3f2/core/src/main/resources/org/apache/spark/ui/static/vis.min.js -- diff --git a/core/src/main/resources/org/apache/spark/ui/static/vis.min.js b/core/src/main/resources/org/a

spark git commit: [SPARK-4705] Handle multiple app attempts event logs, history server.

Repository: spark Updated Branches: refs/heads/master 7fe0f3f2b -> 3052f4916 [SPARK-4705] Handle multiple app attempts event logs, history server. This change modifies the event logging listener to write the logs for different application attempts to different files. The attempt ID is set by

spark git commit: [SPARK-3066] [MLLIB] Support recommendAll in matrix factorization model

Repository: spark Updated Branches: refs/heads/master 3052f4916 -> 3b514af8a [SPARK-3066] [MLLIB] Support recommendAll in matrix factorization model This is based on #3098 from debasish83. 1. BLAS' GEMM is used to compute inner products. 2. Reverted changes to MovieLensALS. SPARK-4231 should

spark git commit: [SPARK-5891] [ML] Add Binarizer ML Transformer

Repository: spark Updated Branches: refs/heads/master 3b514af8a -> 7630213ca [SPARK-5891] [ML] Add Binarizer ML Transformer JIRA: https://issues.apache.org/jira/browse/SPARK-5891 Author: Liang-Chi Hsieh Closes #5699 from viirya/add_binarizer and squashes the following commits: 1a0b9a4 [Lia

spark git commit: Limit help option regex

Repository: spark Updated Branches: refs/heads/master 7630213ca -> c8c481da1 Limit help option regex Added word-boundary delimiters so that embedded text such as "-h" within command line options and values doesn't trigger the usage script and exit. Author: Chris Biow Closes #5816 from cbio

spark git commit: Limit help option regex

Repository: spark Updated Branches: refs/heads/branch-1.3 f64b99421 -> d7269493d Limit help option regex Added word-boundary delimiters so that embedded text such as "-h" within command line options and values doesn't trigger the usage script and exit. Author: Chris Biow Closes #5816 from

spark git commit: changing persistence engine trait to an abstract class

Repository: spark Updated Branches: refs/heads/master c8c481da1 -> 27de6fef6 changing persistence engine trait to an abstract class Author: niranda Closes #5832 from nirandaperera/PersistanceEngine_abstract_class and squashes the following commits: 67b9d5a [niranda] changing persistence en

spark git commit: [SPARK-5854] personalized page rank

Repository: spark Updated Branches: refs/heads/master 27de6fef6 -> 7d427222d [SPARK-5854] personalized page rank Here's a modification to PageRank which does personalized PageRank. The approach is basically similar to that outlined by Bahmani et al. from 2010 (http://arxiv.org/pdf/1006.2880

spark git commit: [SPARK-6846] [WEBUI] [HOTFIX] return to GET for kill link in UI since YARN AM won't proxy POST

Repository: spark Updated Branches: refs/heads/master 7d427222d -> 1262e310c [SPARK-6846] [WEBUI] [HOTFIX] return to GET for kill link in UI since YARN AM won't proxy POST Partial undoing of SPARK-6846; YARN AM proxy won't forward POSTs, so go back to GET for kill links in Spark UI. Standalo

spark git commit: [SPARK-7183] [NETWORK] Fix memory leak of TransportRequestHandler.streamIds

Repository: spark Updated Branches: refs/heads/master 1262e310c -> 168603272 [SPARK-7183] [NETWORK] Fix memory leak of TransportRequestHandler.streamIds JIRA: https://issues.apache.org/jira/browse/SPARK-7183 Author: Liang-Chi Hsieh Closes #5743 from viirya/fix_requesthandler_memory_leak and

spark git commit: [SPARK-7274] [SQL] Create Column expression for array/struct creation.

Repository: spark Updated Branches: refs/heads/master 168603272 -> 37537760d [SPARK-7274] [SQL] Create Column expression for array/struct creation. Author: Reynold Xin Closes #5802 from rxin/SPARK-7274 and squashes the following commits: 19aecaa [Reynold Xin] Fixed unicode tests. bfc1538 [R

[1/2] spark git commit: Revert "[SPARK-7287] enabled fixed test"

Repository: spark Updated Branches: refs/heads/master 37537760d -> c6d9a4294 Revert "[SPARK-7287] enabled fixed test" This reverts commit 7cf1eb79b1fa290aa1d867a8a1eaaea86d6b2239. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/

[2/2] spark git commit: Revert "[SPARK-7224] added mock repository generator for --packages tests"

Revert "[SPARK-7224] added mock repository generator for --packages tests" This reverts commit 7dacc08ab36188991a001df23880167433844767. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c6d9a429 Tree: http://git-wip-us.apache.

spark git commit: [SPARK-7213] [YARN] Check for read permissions before copying a Hadoop config file

Repository: spark Updated Branches: refs/heads/master c6d9a4294 -> f53a48827 [SPARK-7213] [YARN] Check for read permissions before copying a Hadoop config file Author: Nishkam Ravi Author: nishkamravi2 Author: nravi Closes #5760 from nishkamravi2/master_nravi and squashes the following co

spark git commit: [SPARK-7281] [YARN] Add option to set AM's lib path in client mode.

Repository: spark Updated Branches: refs/heads/master f53a48827 -> 7b5dd3e3c [SPARK-7281] [YARN] Add option to set AM's lib path in client mode. Author: Marcelo Vanzin Closes #5813 from vanzin/SPARK-7281 and squashes the following commits: 1cb6f42 [Marcelo Vanzin] [SPARK-7281] [yarn] Add op

spark git commit: [SPARK-7240][SQL] Single pass covariance calculation for dataframes

Repository: spark Updated Branches: refs/heads/master 7b5dd3e3c -> 4dc8d7449 [SPARK-7240][SQL] Single pass covariance calculation for dataframes Added the calculation of covariance between two columns to DataFrames. cc mengxr rxin Author: Burak Yavuz Closes #5825 from brkyvz/df-cov and squ

spark git commit: [SPARK-5342] [YARN] Allow long running Spark apps to run on secure YARN/HDFS

Repository: spark Updated Branches: refs/heads/master 4dc8d7449 -> b1f4ca82d [SPARK-5342] [YARN] Allow long running Spark apps to run on secure YARN/HDFS Take 2. Does the same thing as #4688, but fixes Hadoop-1 build. Author: Hari Shreedharan Closes #5823 from harishreedharan/kerberos-longr

spark git commit: Ignore flakey test in SparkSubmitUtilsSuite

Repository: spark Updated Branches: refs/heads/master b1f4ca82d -> 5c1fabafa Ignore flakey test in SparkSubmitUtilsSuite Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5c1fabaf Tree: http://git-wip-us.apache.org/repos/asf

spark git commit: [SPARK-7312][SQL] SPARK-6913 broke jdk6 build

Repository: spark Updated Branches: refs/heads/master 5c1fabafa -> 41c6a44b1 [SPARK-7312][SQL] SPARK-6913 broke jdk6 build JIRA: https://issues.apache.org/jira/browse/SPARK-7312 Author: Yin Huai Closes #5847 from yhuai/jdbcJava6 and squashes the following commits: 68433a2 [Yin Huai] compil

spark git commit: [SPARK-7304] [BUILD] Include $@ in call to mvn consistently in make-distribution.sh

Repository: spark Updated Branches: refs/heads/master 41c6a44b1 -> e6fb37712 [SPARK-7304] [BUILD] Include $@ in call to mvn consistently in make-distribution.sh Adding the $ allows the caller of this script to supply additional arguments to the mvn command and is consistent with how mvn is b

spark git commit: [SPARK-6999] [SQL] Remove the infinite recursive method (useless)

Repository: spark Updated Branches: refs/heads/master e6fb37712 -> 98e704580 [SPARK-6999] [SQL] Remove the infinite recursive method (useless) Remove the method, since it causes infinite recursive calls. And seems it's a dummy method, since we have the API: `def createDataFrame(rowRDD: JavaRD

spark git commit: [SPARK-7309] [CORE] [STREAMING] Shutdown the thread pools in ReceivedBlockHandler and DAGScheduler

Repository: spark Updated Branches: refs/heads/master 98e704580 -> ebc25a4dd [SPARK-7309] [CORE] [STREAMING] Shutdown the thread pools in ReceivedBlockHandler and DAGScheduler Shutdown the thread pools in ReceivedBlockHandler and DAGScheduler when stopping them. Author: zsxwing Closes #58

spark git commit: [SPARK-7112][Streaming][WIP] Add a InputInfoTracker to track all the input streams

Repository: spark Updated Branches: refs/heads/master ebc25a4dd -> b88c275e6 [SPARK-7112][Streaming][WIP] Add a InputInfoTracker to track all the input streams Author: jerryshao Author: Saisai Shao Closes #5680 from jerryshao/SPARK-7111 and squashes the following commits: 339f854 [Saisai

spark git commit: [SPARK-2808][Streaming][Kafka] update kafka to 0.8.2

Repository: spark Updated Branches: refs/heads/master b88c275e6 -> 478648407 [SPARK-2808][Streaming][Kafka] update kafka to 0.8.2 i don't think this should be merged until after 1.3.0 is final Author: cody koeninger Author: Helena Edelson Closes #4537 from koeninger/wip-2808-kafka-0.8.2-up

spark git commit: [SPARK-3444] Provide an easy way to change log level

Repository: spark Updated Branches: refs/heads/master 478648407 -> ae98eec73 [SPARK-3444] Provide an easy way to change log level Add support for changing the log level at run time through the SparkContext. Based on an earlier PR, #2433 includes CR feedback from pwendel & davies Author: Hold

spark git commit: [SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a negative n...

Repository: spark Updated Branches: refs/heads/master ae98eec73 -> 099327d53 [SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a negative n... ...umber of executors Author: Sandy Ryza Closes #5704 from sryza/sandy-spark-6954 and squashes the following commits: b7890fb [S

spark git commit: [SPARK-6443] [SPARK SUBMIT] Could not submit app in standalone cluster mode when HA is enabled

Repository: spark Updated Branches: refs/heads/master 202219341 -> b4b43df8a [SPARK-6443] [SPARK SUBMIT] Could not submit app in standalone cluster mode when HA is enabled **3/26 update:** * Akka-based: Use an array of `ActorSelection` to represent multiple master. Add an `activeMasterActo

spark git commit: [SPARK-7216] [MESOS] Add driver details page to Mesos cluster UI.

Repository: spark Updated Branches: refs/heads/master 099327d53 -> 202219341 [SPARK-7216] [MESOS] Add driver details page to Mesos cluster UI. Add a details page that displays Mesos driver in the Mesos cluster UI Author: Timothy Chen Closes #5763 from tnachen/mesos_cluster_page and squashes

spark git commit: [SPARK-2691] [MESOS] Support for Mesos DockerInfo

Repository: spark Updated Branches: refs/heads/master b4b43df8a -> 8f50a07d2 [SPARK-2691] [MESOS] Support for Mesos DockerInfo This patch adds partial support for running spark on mesos inside of a docker container. Only fine-grained mode is presently supported, and there is no checking done

[2/2] spark git commit: [SPARK-6229] Add SASL encryption to network library.

[SPARK-6229] Add SASL encryption to network library. There are two main parts of this change: - Extending the bootstrap mechanism in the network library to add a server-side bootstrap (which works a little bit differently than the client-side bootstrap), and to allow the bootstraps to modif

[1/2] spark git commit: [SPARK-6229] Add SASL encryption to network library.

Repository: spark Updated Branches: refs/heads/master 8f50a07d2 -> 38d4e9e44 http://git-wip-us.apache.org/repos/asf/spark/blob/38d4e9e4/network/shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleSecuritySuite.java --

spark git commit: [SPARK-7317] [Shuffle] Expose shuffle handle

Repository: spark Updated Branches: refs/heads/master 38d4e9e44 -> b79aeb95b [SPARK-7317] [Shuffle] Expose shuffle handle Details in JIRA, in a nut-shell, all machinary for custom RDD's to leverage spark shuffle directly (without exposing impl details of shuffle) exists - except for this sma

spark git commit: [SPARK-7242] added python api for freqItems in DataFrames

Repository: spark Updated Branches: refs/heads/master b79aeb95b -> 2e0f3579f [SPARK-7242] added python api for freqItems in DataFrames The python api for DataFrame's plus addressed your comments from previous PR. rxin Author: Burak Yavuz Closes #5859 from brkyvz/df-freq-py2 and squashes the

spark git commit: [SPARK-7120] [SPARK-7121] Closure cleaner nesting + documentation + tests

Repository: spark Updated Branches: refs/heads/master 2e0f3579f -> 7394e7ade [SPARK-7120] [SPARK-7121] Closure cleaner nesting + documentation + tests Note: ~600 lines of this is test code, and ~100 lines documentation. **[SPARK-7121]** ClosureCleaner does not handle nested closures properly.