spark git commit: [SPARK-10889] [STREAMING] Bump KCL to add MillisBehindLatest metric

2015-10-04 Thread srowen 
Repository: spark
Updated Branches:
  refs/heads/master 82bbc2a5f -> 883bd8fcc


[SPARK-10889] [STREAMING] Bump KCL to add MillisBehindLatest metric

I don't believe the API changed at all.

Author: Avrohom Katz 

Closes #8957 from akatz/kcl-upgrade.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/883bd8fc
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/883bd8fc
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/883bd8fc

Branch: refs/heads/master
Commit: 883bd8fccf83aae7a2a847c9a6ca129fac86e6a3
Parents: 82bbc2a
Author: Avrohom Katz 
Authored: Sun Oct 4 09:36:07 2015 +0100
Committer: Sean Owen 
Committed: Sun Oct 4 09:36:07 2015 +0100

--
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/883bd8fc/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 6535994..d04ed1e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -152,7 +152,7 @@
 hadoop2
 0.7.1
 1.9.16
-1.2.1
+1.3.0
 
 4.3.2
 


-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-10889] [STREAMING] Bump KCL to add MillisBehindLatest metric

2015-10-04 Thread srowen 
Repository: spark
Updated Branches:
  refs/heads/branch-1.5 8836ac3d1 -> d323e5ede


[SPARK-10889] [STREAMING] Bump KCL to add MillisBehindLatest metric

I don't believe the API changed at all.

Author: Avrohom Katz 

Closes #8957 from akatz/kcl-upgrade.

(cherry picked from commit 883bd8fccf83aae7a2a847c9a6ca129fac86e6a3)
Signed-off-by: Sean Owen 


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d323e5ed
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d323e5ed
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d323e5ed

Branch: refs/heads/branch-1.5
Commit: d323e5ede3ad992c105da82c81acea0960f7d1bd
Parents: 8836ac3
Author: Avrohom Katz 
Authored: Sun Oct 4 09:36:07 2015 +0100
Committer: Sean Owen 
Committed: Sun Oct 4 09:36:18 2015 +0100

--
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/d323e5ed/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 9be80a0..834adc4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -152,7 +152,7 @@
 hadoop2
 0.7.1
 1.9.16
-1.2.1
+1.3.0
 
 4.3.2
 


-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-9570] [DOCS] Consistent recommendation for submitting spark apps to YARN, -master yarn --deploy-mode x vs -master yarn-x'.

2015-10-04 Thread srowen 
Repository: spark
Updated Branches:
  refs/heads/master 721e8b5f3 -> 82bbc2a5f


[SPARK-9570] [DOCS] Consistent recommendation for submitting spark apps to 
YARN, -master yarn --deploy-mode x vs -master yarn-x'.

Recommend `--master yarn --deploy-mode {cluster,client}` consistently in docs.
Follow-on to https://github.com/apache/spark/pull/8385
CC nssalian

Author: Sean Owen 

Closes #8968 from srowen/SPARK-9570.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/82bbc2a5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/82bbc2a5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/82bbc2a5

Branch: refs/heads/master
Commit: 82bbc2a5f2c74604db59060cb5e462a057398ddd
Parents: 721e8b5
Author: Sean Owen 
Authored: Sun Oct 4 09:31:52 2015 +0100
Committer: Sean Owen 
Committed: Sun Oct 4 09:31:52 2015 +0100

--
 README.md   |  2 +-
 docs/running-on-yarn.md | 32 +---
 docs/sql-programming-guide.md   |  2 +-
 docs/submitting-applications.md | 25 +++--
 4 files changed, 34 insertions(+), 27 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/82bbc2a5/README.md
--
diff --git a/README.md b/README.md
index 76e29b4..4116ef3 100644
--- a/README.md
+++ b/README.md
@@ -59,7 +59,7 @@ will run the Pi example locally.
 
 You can set the MASTER environment variable when running examples to submit
 examples to a cluster. This can be a mesos:// or spark:// URL,
-"yarn-cluster" or "yarn-client" to run on YARN, and "local" to run
+"yarn" to run on YARN, and "local" to run
 locally with one thread, or "local[N]" to run locally with N threads. You
 can also use an abbreviated class name if the class is in the `examples`
 package. For instance:

http://git-wip-us.apache.org/repos/asf/spark/blob/82bbc2a5/docs/running-on-yarn.md
--
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index 0e25ccf..6d77db6 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -16,18 +16,19 @@ containers used by the application use the same 
configuration. If the configurat
 Java system properties or environment variables not managed by YARN, they 
should also be set in the
 Spark application's configuration (driver, executors, and the AM when running 
in client mode).
 
-There are two deploy modes that can be used to launch Spark applications on 
YARN. In `yarn-cluster` mode, the Spark driver runs inside an application 
master process which is managed by YARN on the cluster, and the client can go 
away after initiating the application. In `yarn-client` mode, the driver runs 
in the client process, and the application master is only used for requesting 
resources from YARN.
+There are two deploy modes that can be used to launch Spark applications on 
YARN. In `cluster` mode, the Spark driver runs inside an application master 
process which is managed by YARN on the cluster, and the client can go away 
after initiating the application. In `client` mode, the driver runs in the 
client process, and the application master is only used for requesting 
resources from YARN.
 
-Unlike [Spark standalone](spark-standalone.html) and 
[Mesos](running-on-mesos.html) modes, in which the master's address is 
specified in the `--master` parameter, in YARN mode the ResourceManager's 
address is picked up from the Hadoop configuration. Thus, the `--master` 
parameter is `yarn-client` or `yarn-cluster`.
+Unlike [Spark standalone](spark-standalone.html) and 
[Mesos](running-on-mesos.html) modes, in which the master's address is 
specified in the `--master` parameter, in YARN mode the ResourceManager's 
address is picked up from the Hadoop configuration. Thus, the `--master` 
parameter is `yarn`.
 
-To launch a Spark application in `yarn-cluster` mode:
+To launch a Spark application in `cluster` mode:
 
-$ ./bin/spark-submit --class path.to.your.Class --master yarn-cluster 
[options]  [app options]
+$ ./bin/spark-submit --class path.to.your.Class --master yarn 
--deploy-mode cluster [options]  [app options]
 
 For example:
 
 $ ./bin/spark-submit --class org.apache.spark.examples.SparkPi \
---master yarn-cluster \
+--master yarn \
+--deploy-mode cluster \
 --driver-memory 4g \
 --executor-memory 2g \
 --executor-cores 1 \
@@ -37,16 +38,17 @@ For example:
 
 The above starts a YARN client program which starts the default Application 
Master. Then SparkPi will be run as a child thread of Application Master. The 
client will periodically poll the Application Master for status updates and 
display