spark git commit: [SPARK-5582] [history] Ignore empty log directories.

Repository: spark Updated Branches: refs/heads/master 24dbc50b9 - 856928979 [SPARK-5582] [history] Ignore empty log directories. Empty log directories are not useful at the moment, but if one ends up showing in the log root, it breaks the code that checks for log directories. Author: Marcelo

spark git commit: [SPARK-5470][Core]use defaultClassLoader to load classes in KryoSerializer

Repository: spark Updated Branches: refs/heads/master 856928979 - ed3aac791 [SPARK-5470][Core]use defaultClassLoader to load classes in KryoSerializer Now KryoSerializer load classes of classesToRegister at the time of its initialization. when we set spark.kryo.classesToRegister=class1, it

spark git commit: [SPARK-5157][YARN] Configure more JVM options properly when we use ConcMarkSweepGC for AM.

Repository: spark Updated Branches: refs/heads/master f6ba813af - 24dbc50b9 [SPARK-5157][YARN] Configure more JVM options properly when we use ConcMarkSweepGC for AM. When we set `SPARK_USE_CONC_INCR_GC`, ConcurrentMarkSweepGC works on the AM. Actually, if ConcurrentMarkSweepGC is set for

spark git commit: [Build] Set all Debian package permissions to 755

Repository: spark Updated Branches: refs/heads/master f827ef4d7 - cf6778e8d [Build] Set all Debian package permissions to 755 755 means the owner can read, write, and execute, and everyone else can just read and execute. I think that's what we want here since without execute permissions

spark git commit: [Minor] Remove permission for execution from spark-shell.cmd

Repository: spark Updated Branches: refs/heads/master 575d2df35 - f6ba813af [Minor] Remove permission for execution from spark-shell.cmd .cmd files in bin is not set permission for execution except for spark-shell.cmd. Let's unify that. Author: Kousuke Saruta saru...@oss.nttdata.co.jp

spark git commit: [SPARK-5416] init Executor.threadPool before ExecutorSource

Repository: spark Updated Branches: refs/heads/master cf6778e8d - 37d35ab53 [SPARK-5416] init Executor.threadPool before ExecutorSource Some ExecutorSource metrics can NPE by attempting to reference the threadpool otherwise. Author: Ryan Williams ryan.blake.willi...@gmail.com Closes #4212

spark git commit: [SPARK-5366][EC2] Check the mode of private key

Repository: spark Updated Branches: refs/heads/master 5de14cc27 - 6fb141e2a [SPARK-5366][EC2] Check the mode of private key Check the mode of private key file. Author: liuchang0812 liuchang0...@gmail.com Closes #4162 from Liuchang0812/ec2-script and squashes the following commits: fc37355

spark git commit: [SPARK-5672][Web UI] Don't return `ERROR 500` when have missing args

Repository: spark Updated Branches: refs/heads/master 487831369 - 23a99dabf [SPARK-5672][Web UI] Don't return `ERROR 500` when have missing args Spark web UI return `HTTP ERROR 500` when GET arguments is missing. Author: Kirill A. Korinskiy ca...@catap.ru Closes #4239 from catap/ui_500 and

spark git commit: [SPARK-5656] Fail gracefully for large values of k and/or n that will ex...

Repository: spark Updated Branches: refs/heads/master 6fb141e2a - 487831369 [SPARK-5656] Fail gracefully for large values of k and/or n that will ex... ...ceed max int. Large values of k and/or n in EigenValueDecomposition.symmetricEigs will result in array initialization to a value larger

spark git commit: [SPARK-5672][Web UI] Don't return `ERROR 500` when have missing args

Repository: spark Updated Branches: refs/heads/branch-1.3 0f9d76599 - 96010faa3 [SPARK-5672][Web UI] Don't return `ERROR 500` when have missing args Spark web UI return `HTTP ERROR 500` when GET arguments is missing. Author: Kirill A. Korinskiy ca...@catap.ru Closes #4239 from catap/ui_500

spark git commit: [SPARK-3039] [BUILD] Spark assembly for new hadoop API (hadoop 2) contai...

Repository: spark Updated Branches: refs/heads/master 23a99dabf - 75fdccca3 [SPARK-3039] [BUILD] Spark assembly for new hadoop API (hadoop 2) contai... ...ns avro-mapred for hadoop 1 API had been marked as resolved but did not work for at least some builds due to version conflicts using

spark git commit: [SPARK-5473] [EC2] Expose SSH failures after status checks pass

Repository: spark Updated Branches: refs/heads/master 855d12ac0 - 4dfe180fc [SPARK-5473] [EC2] Expose SSH failures after status checks pass If there is some fatal problem with launching a cluster, `spark-ec2` just hangs without giving the user useful feedback on what the problem is. This PR

spark git commit: [SPARK-5473] [EC2] Expose SSH failures after status checks pass

Repository: spark Updated Branches: refs/heads/branch-1.3 5782ee29e - f2aa7b757 [SPARK-5473] [EC2] Expose SSH failures after status checks pass If there is some fatal problem with launching a cluster, `spark-ec2` just hangs without giving the user useful feedback on what the problem is.

spark git commit: [SPARK-5733] Error Link in Pagination of HistroyPage when showing Incomplete Applications

Repository: spark Updated Branches: refs/heads/branch-1.3 057ec4f33 - 864dccd70 [SPARK-5733] Error Link in Pagination of HistroyPage when showing Incomplete Applications The links in pagination of HistroyPage is wrong when showing Incomplete Applications. If 2 is click on the following

spark git commit: [SPARK-5733] Error Link in Pagination of HistroyPage when showing Incomplete Applications

Repository: spark Updated Branches: refs/heads/master bd0d6e0cc - 1ac099e3e [SPARK-5733] Error Link in Pagination of HistroyPage when showing Incomplete Applications The links in pagination of HistroyPage is wrong when showing Incomplete Applications. If 2 is click on the following page

spark git commit: [SPARK-1805] [EC2] Validate instance types

Repository: spark Updated Branches: refs/heads/master ba667935f - 50820f152 [SPARK-1805] [EC2] Validate instance types Addresses [SPARK-1805](https://issues.apache.org/jira/browse/SPARK-1805), though doesn't resolve it completely. Error out quickly if the user asks for the master and slaves

spark git commit: Remove outdated remark about take(n).

Repository: spark Updated Branches: refs/heads/master b694eb9c2 - 03bf704bf Remove outdated remark about take(n). Looking at the code, I believe this remark about `take(n)` computing partitions on the driver is no longer correct. Apologies if I'm wrong. This came up in

spark git commit: Remove outdated remark about take(n).

Repository: spark Updated Branches: refs/heads/branch-1.3 d66aae217 - 72adfc595 Remove outdated remark about take(n). Looking at the code, I believe this remark about `take(n)` computing partitions on the driver is no longer correct. Apologies if I'm wrong. This came up in

spark git commit: [Spark-5717] [MLlib] add stop and reorganize import

Repository: spark Updated Branches: refs/heads/master 50820f152 - 6cc96cf0c [Spark-5717] [MLlib] add stop and reorganize import Trivial. add sc stop and reorganize import https://issues.apache.org/jira/browse/SPARK-5717 Author: JqueryFan fir...@126.com Author: Yuhao Yang hhb...@gmail.com

spark git commit: [Spark-5717] [MLlib] add stop and reorganize import

Repository: spark Updated Branches: refs/heads/branch-1.3 d6f31e0c7 - b32f5535d [Spark-5717] [MLlib] add stop and reorganize import Trivial. add sc stop and reorganize import https://issues.apache.org/jira/browse/SPARK-5717 Author: JqueryFan fir...@126.com Author: Yuhao Yang hhb...@gmail.com

spark git commit: SPARK-5815 [MLLIB] Part 2. Deprecate SVDPlusPlus APIs that expose DoubleMatrix from JBLAS

from srowen/SPARK-5815.2 and squashes the following commits: 6fd2ca5 [Sean Owen] Now, deprecated runSVDPlusPlus and update run, for 1.4.0 / master only Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a3afa4a1 Tree: http://git

spark git commit: SPARK-5795 [STREAMING] api.java.JavaPairDStream.saveAsNewAPIHadoopFiles may not friendly to java

tdas for review Author: Sean Owen so...@cloudera.com Closes #4608 from srowen/SPARK-5795 and squashes the following commits: 36f1ead [Sean Owen] Add code that shows compile problem and fix 036bd27 [Sean Owen] Revise JavaPairDStream API declaration on saveAs Hadoop methods, to allow it to be called

spark git commit: SPARK-5795 [STREAMING] api.java.JavaPairDStream.saveAsNewAPIHadoopFiles may not friendly to java

. CC tdas for review Author: Sean Owen so...@cloudera.com Closes #4608 from srowen/SPARK-5795 and squashes the following commits: 36f1ead [Sean Owen] Add code that shows compile problem and fix 036bd27 [Sean Owen] Revise JavaPairDStream API declaration on saveAs Hadoop methods, to allow

spark git commit: SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example

Repository: spark Updated Branches: refs/heads/branch-1.3 aca799159 - 3997e74f6 SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example numClassesForClassification has been renamed to numClasses. Author: MechCoder manojkumarsivaraj...@gmail.com Closes #4672 from

spark git commit: SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example

Repository: spark Updated Branches: refs/heads/master c1b6fa983 - e79a7a626 SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example numClassesForClassification has been renamed to numClasses. Author: MechCoder manojkumarsivaraj...@gmail.com Closes #4672 from

spark git commit: SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example

Repository: spark Updated Branches: refs/heads/branch-1.2 6be36d5a8 - 068ba45cf SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example numClassesForClassification has been renamed to numClasses. Author: MechCoder manojkumarsivaraj...@gmail.com Closes #4672 from

spark git commit: SPARK-5776 JIRA version not of form x.y.z breaks merge_spark_pr.py

in JIRA. Author: Sean Owen so...@cloudera.com Closes #4570 from srowen/SPARK-5776 and squashes the following commits: fffafde [Sean Owen] Consider only x.y.z verisons from JIRA Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit

spark git commit: [SPARK-5655] Don't chmod700 application files if running in YARN

Repository: spark Updated Branches: refs/heads/branch-1.2 53de2378e - 64254eeec [SPARK-5655] Don't chmod700 application files if running in YARN [Was previously PR4507] As per SPARK-5655, recently committed code chmod 700s all application files created on the local fs by a spark executor.

spark git commit: ignore cache paths for RAT tests

Repository: spark Updated Branches: refs/heads/master 9a3ea49f7 - 9a6efbccf ignore cache paths for RAT tests RAT fails on cache paths. add to .rat-excludes Author: Oren Mazor oren.ma...@gmail.com Closes #4569 from orenmazor/apache_master and squashes the following commits: d0c9e7e [Oren

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

Repository: spark Updated Branches: refs/heads/branch-1.3 f7103b343 - 5c9db4e75 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

Repository: spark Updated Branches: refs/heads/branch-1.2 c7bac577a - d24971a62 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

Repository: spark Updated Branches: refs/heads/master d5fc51491 - ada993e95 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits are

spark git commit: [SPARK-4832][Deploy]some other processes might take the daemon pid

Repository: spark Updated Branches: refs/heads/branch-1.2 0ba065f0a - 2b9dbddd3 [SPARK-4832][Deploy]some other processes might take the daemon pid Some other processes might use the pid saved in pid file. In that case we should ignore it and launch daemons. JIRA is down for maintenance. I

spark git commit: SPARK-5728 [STREAMING] MQTTStreamSuite leaves behind ActiveMQ database files

Repository: spark Updated Branches: refs/heads/branch-1.2 a5d72fd50 - 222ce9ffd SPARK-5728 [STREAMING] MQTTStreamSuite leaves behind ActiveMQ database files Use temp dir for ActiveMQ database Author: Sean Owen so...@cloudera.com Closes #4517 from srowen/SPARK-5728 and squashes the following

spark git commit: [SPARK-4832][Deploy]some other processes might take the daemon pid

Repository: spark Updated Branches: refs/heads/master 1c8633f3f - 1768bd514 [SPARK-4832][Deploy]some other processes might take the daemon pid Some other processes might use the pid saved in pid file. In that case we should ignore it and launch daemons. JIRA is down for maintenance. I will

spark git commit: [SPARK-4832][Deploy]some other processes might take the daemon pid

Repository: spark Updated Branches: refs/heads/branch-1.3 a8f560c4e - 1255e83f8 [SPARK-4832][Deploy]some other processes might take the daemon pid Some other processes might use the pid saved in pid file. In that case we should ignore it and launch daemons. JIRA is down for maintenance. I

spark git commit: SPARK-5841 [CORE] [HOTFIX] Memory leak in DiskBlockManager

from srowen/SPARK-5841.2 and squashes the following commits: 51548db [Sean Owen] Avoid call to remove shutdown hook being called from shutdown hook (cherry picked from commit 49c19fdbad57f0609bbcc9278f9eaa8115a73604) Signed-off-by: Sean Owen so...@cloudera.com Project: http://git-wip

spark git commit: [SPARK-5661]function hasShutdownDeleteTachyonDir should use shutdownDeleteTachyonPaths to determine whether contains file

Repository: spark Updated Branches: refs/heads/master d8f69cf78 - b271c265b [SPARK-5661]function hasShutdownDeleteTachyonDir should use shutdownDeleteTachyonPaths to determine whether contains file hasShutdownDeleteTachyonDir(file: TachyonFile) should use shutdownDeleteTachyonPaths(not

spark git commit: SPARK-5357: Update commons-codec version to 1.10 (current)

Repository: spark Updated Branches: refs/heads/master bb05982dd - c01c4ebcf SPARK-5357: Update commons-codec version to 1.10 (current) Resolves https://issues.apache.org/jira/browse/SPARK-5357 In commons-codec 1.5, Base64 instances are not thread safe. That was only true from 1.4-1.6.

spark git commit: SPARK-5841: remove DiskBlockManager shutdown hook on stop

Repository: spark Updated Branches: refs/heads/master c51ab37fa - bb05982dd SPARK-5841: remove DiskBlockManager shutdown hook on stop After a call to stop, the shutdown hook is redundant, and causes a memory leak. Author: Matt Whelan mwhe...@perka.com Closes #4627 from MattWhelan/SPARK-5841

spark git commit: SPARK-5841: remove DiskBlockManager shutdown hook on stop

Repository: spark Updated Branches: refs/heads/branch-1.3 864d77e0d - dd977dfed SPARK-5841: remove DiskBlockManager shutdown hook on stop After a call to stop, the shutdown hook is redundant, and causes a memory leak. Author: Matt Whelan mwhe...@perka.com Closes #4627 from

spark git commit: [SPARK-5826][Streaming] Fix Configuration not serializable problem

Repository: spark Updated Branches: refs/heads/master c06e42f2c - a65766bf0 [SPARK-5826][Streaming] Fix Configuration not serializable problem Author: jerryshao saisai.s...@intel.com Closes #4612 from jerryshao/SPARK-5826 and squashes the following commits: 7ec71db [jerryshao] Remove

spark git commit: [SPARK-5826][Streaming] Fix Configuration not serializable problem

Repository: spark Updated Branches: refs/heads/branch-1.3 e9241fa70 - b8da5c390 [SPARK-5826][Streaming] Fix Configuration not serializable problem Author: jerryshao saisai.s...@intel.com Closes #4612 from jerryshao/SPARK-5826 and squashes the following commits: 7ec71db [jerryshao] Remove

spark git commit: [SPARK-5661]function hasShutdownDeleteTachyonDir should use shutdownDeleteTachyonPaths to determine whether contains file

Repository: spark Updated Branches: refs/heads/branch-1.3 2bf2b56ef - 420bc9b3a [SPARK-5661]function hasShutdownDeleteTachyonDir should use shutdownDeleteTachyonPaths to determine whether contains file hasShutdownDeleteTachyonDir(file: TachyonFile) should use shutdownDeleteTachyonPaths(not

spark git commit: SPARK-5841 [CORE] [HOTFIX] Memory leak in DiskBlockManager

srowen/SPARK-5841.2 and squashes the following commits: 51548db [Sean Owen] Avoid call to remove shutdown hook being called from shutdown hook Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/49c19fdb Tree: http://git-wip

spark git commit: [SPARK-5827][SQL] Add missing import in the example of SqlContext

Repository: spark Updated Branches: refs/heads/branch-1.3 f87f3b755 - 9c1c70d8c [SPARK-5827][SQL] Add missing import in the example of SqlContext If one tries an example by using copypaste, throw an exception. Author: Takeshi Yamamuro linguin@gmail.com Closes #4615 from

spark git commit: [SPARK-5827][SQL] Add missing import in the example of SqlContext

Repository: spark Updated Branches: refs/heads/master ed5f4bb7c - c771e475c [SPARK-5827][SQL] Add missing import in the example of SqlContext If one tries an example by using copypaste, throw an exception. Author: Takeshi Yamamuro linguin@gmail.com Closes #4615 from

spark git commit: SPARK-5848: tear down the ConsoleProgressBar timer

Repository: spark Updated Branches: refs/heads/master e189cbb05 - 1294a6e01 SPARK-5848: tear down the ConsoleProgressBar timer The timer is a GC root, and failing to terminate it leaks SparkContext instances. Author: Matt Whelan mwhe...@perka.com Closes #4635 from MattWhelan/SPARK-5848 and

spark git commit: SPARK-5848: tear down the ConsoleProgressBar timer

Repository: spark Updated Branches: refs/heads/branch-1.3 8a94bf76b - 52994d83b SPARK-5848: tear down the ConsoleProgressBar timer The timer is a GC root, and failing to terminate it leaks SparkContext instances. Author: Matt Whelan mwhe...@perka.com Closes #4635 from MattWhelan/SPARK-5848

spark git commit: SPARK-5744 [CORE] Take 2. RDD.isEmpty / take fails for (empty) RDD of Nothing

[Nothing] and RDD[Null]. Also, fix a marginally related minor issue with histogram() and EmptyRDD. CC rxin since you reviewed the last one although I imagine this is an uncontroversial resolution. Author: Sean Owen so...@cloudera.com Closes #4698 from srowen/SPARK-5744.2 and squashes the following

spark git commit: SPARK-5669 [BUILD] [HOTFIX] Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS

RC1) Author: Sean Owen so...@cloudera.com Closes #4673 from srowen/SPARK-5669.2 and squashes the following commits: e29693c [Sean Owen] Correct exclusion path for JBLAS native libs Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: SPARK-5669 [BUILD] [HOTFIX] Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS

: 1.3.0 RC1) Author: Sean Owen so...@cloudera.com Closes #4673 from srowen/SPARK-5669.2 and squashes the following commits: e29693c [Sean Owen] Correct exclusion path for JBLAS native libs (cherry picked from commit 5aecdcf1f23a826f6236096001de1dd811dbc443) Signed-off-by: Sean Owen so...@cloudera.com

spark git commit: [SPARK-4949]shutdownCallback in SparkDeploySchedulerBackend should be enclosed by synchronized block.

Repository: spark Updated Branches: refs/heads/master e79a7a626 - 82197ed3b [SPARK-4949]shutdownCallback in SparkDeploySchedulerBackend should be enclosed by synchronized block. A variable `shutdownCallback` in SparkDeploySchedulerBackend can be accessed from multiple threads so it should

spark git commit: SPARK-5239 [CORE] JdbcRDD throws java.lang.AbstractMethodError: oracle.jdbc.driver.xxxxxx.isClosed()Z

. Author: Sean Owen so...@cloudera.com Closes #4470 from srowen/SPARK-5239.2 and squashes the following commits: 2398bde [Sean Owen] Avoid use of JDBC4-only isClosed() (cherry picked from commit 2d1e916730492f5d61b97da6c483d3223ca44315) Signed-off-by: Sean Owen so...@cloudera.com Project: http://git

spark git commit: SPARK-5239 [CORE] JdbcRDD throws java.lang.AbstractMethodError: oracle.jdbc.driver.xxxxxx.isClosed()Z

: Sean Owen so...@cloudera.com Closes #4470 from srowen/SPARK-5239.2 and squashes the following commits: 2398bde [Sean Owen] Avoid use of JDBC4-only isClosed() Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2d1e9167 Tree: http

spark git commit: SPARK-4267 [YARN] Backport: Failing to launch jobs on Spark on YARN with Hadoop 2.5.0 or later

Closes #4575 from srowen/SPARK-4267-1.2 and squashes the following commits: 7bfc75b [Sean Owen] Backport of SPARK-4267 for branch-1.2: Before passing to YARN, escape arguments in extraJavaOptions args, in order to correctly handle cases like -Dfoo=one two three. Also standardize how these args

spark git commit: SPARK-5819 Backported the fix described in SPARK-5805

Repository: spark Updated Branches: refs/heads/branch-1.2 26410a2d3 - f9d8c5e3f SPARK-5819 Backported the fix described in SPARK-5805 Fixes SPARK-5819 Author: Emre Sevinç emre.sev...@gmail.com Closes #4605 from emres/SPARK-5819 and squashes the following commits: 33bd4b6 [Emre Sevinç]

spark git commit: SPARK-5822 [BUILD] cannot import src/main/scala src/test/scala into eclipse as source folder

Repository: spark Updated Branches: refs/heads/master 15a2ab5f8 - ed5f4bb7c SPARK-5822 [BUILD] cannot import src/main/scala src/test/scala into eclipse as source folder When import the whole project into eclipse as maven project, found that the src/main/scala src/test/scala can not

[2/2] spark git commit: Revise formatting of previous commit f80e2629bb74bc62960c61ff313f7e7802d61319

Revise formatting of previous commit f80e2629bb74bc62960c61ff313f7e7802d61319 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/15a2ab5f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/15a2ab5f Diff:

[1/2] spark git commit: [SPARK-5800] Streaming Docs. Change linked files according the selected language

Repository: spark Updated Branches: refs/heads/master e98dfe627 - 15a2ab5f8 [SPARK-5800] Streaming Docs. Change linked files according the selected language Currently, Spark Streaming Programming Guide after updateStateByKey explanation links to file stateful_network_wordcount.py and note

[1/2] spark git commit: [SPARK-5800] Streaming Docs. Change linked files according the selected language

Repository: spark Updated Branches: refs/heads/branch-1.3 ba91bf5f4 - 1945fcfd9 [SPARK-5800] Streaming Docs. Change linked files according the selected language Currently, Spark Streaming Programming Guide after updateStateByKey explanation links to file stateful_network_wordcount.py and

[2/2] spark git commit: Revise formatting of previous commit f80e2629bb74bc62960c61ff313f7e7802d61319

Revise formatting of previous commit f80e2629bb74bc62960c61ff313f7e7802d61319 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1945fcfd Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1945fcfd Diff:

spark git commit: SPARK-5728 [STREAMING] MQTTStreamSuite leaves behind ActiveMQ database files

Repository: spark Updated Branches: refs/heads/master 658687b25 - da89720bf SPARK-5728 [STREAMING] MQTTStreamSuite leaves behind ActiveMQ database files Use temp dir for ActiveMQ database Author: Sean Owen so...@cloudera.com Closes #4517 from srowen/SPARK-5728 and squashes the following

spark git commit: SPARK-5728 [STREAMING] MQTTStreamSuite leaves behind ActiveMQ database files

Repository: spark Updated Branches: refs/heads/branch-1.3 811d1798d - 476b6d77b SPARK-5728 [STREAMING] MQTTStreamSuite leaves behind ActiveMQ database files Use temp dir for ActiveMQ database Author: Sean Owen so...@cloudera.com Closes #4517 from srowen/SPARK-5728 and squashes the following

spark git commit: SPARK-5727 [BUILD] Deprecate Debian packaging

. Author: Sean Owen so...@cloudera.com Closes #4516 from srowen/SPARK-5727.1 and squashes the following commits: d48989f [Sean Owen] Refer to Spark 1.4 6c1c8b3 [Sean Owen] Deprecate Debian packaging (cherry picked from commit bd0d6e0cc3a329c4a1c08451a6d8a9281a422958) Signed-off-by: Sean Owen so

spark git commit: SPARK-5727 [BUILD] Deprecate Debian packaging

: Sean Owen so...@cloudera.com Closes #4516 from srowen/SPARK-5727.1 and squashes the following commits: d48989f [Sean Owen] Refer to Spark 1.4 6c1c8b3 [Sean Owen] Deprecate Debian packaging Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf

spark git commit: SPARK-2149. [MLLIB] Univariate kernel density estimation

Repository: spark Updated Branches: refs/heads/master 4dfe180fc - 0793ee1b4 SPARK-2149. [MLLIB] Univariate kernel density estimation Author: Sandy Ryza sa...@cloudera.com Closes #1093 from sryza/sandy-spark-2149 and squashes the following commits: 5f06b33 [Sandy Ryza] More review comments

spark git commit: [SPARK-5611] [EC2] Allow spark-ec2 repo and branch to be set on CLI of spark_ec2.py

Repository: spark Updated Branches: refs/heads/master f48199eb3 - b884daa58 [SPARK-5611] [EC2] Allow spark-ec2 repo and branch to be set on CLI of spark_ec2.py and by extension, the ami-list Useful for using alternate spark-ec2 repos or branches. Author: Florian Verhein

spark git commit: SPARK-5727 [BUILD] Remove Debian packaging

Repository: spark Updated Branches: refs/heads/master a38e23c30 - 9a3ea49f7 SPARK-5727 [BUILD] Remove Debian packaging (for master / 1.4 only) Author: Sean Owen so...@cloudera.com Closes #4526 from srowen/SPARK-5727.2 and squashes the following commits: 83ba49c [Sean Owen] Remove Debian

spark git commit: [SPARK-5644] [Core]Delete tmp dir when sc is stop

Repository: spark Updated Branches: refs/heads/master 582096128 - 52983d7f4 [SPARK-5644] [Core]Delete tmp dir when sc is stop When we run driver as a service, and for each time we run job we only call sc.stop, then will not delete tmp dir create by HttpFileServer and SparkEnv, it will be

spark git commit: [SQL][Minor] correct some comments

Repository: spark Updated Branches: refs/heads/master 52983d7f4 - 91e351254 [SQL][Minor] correct some comments Author: Sheng, Li oopsoutofmem...@users.noreply.github.com Author: OopsOutOfMemory victorshen...@126.com Closes #4508 from OopsOutOfMemory/cmt and squashes the following commits:

spark git commit: [Spark-5889] Remove pid file after stopping service.

Repository: spark Updated Branches: refs/heads/branch-1.2 856fdcb65 - 18fbed5b5 [Spark-5889] Remove pid file after stopping service. Currently the pid file is not deleted, and potentially may cause some problem after service is stopped. The fix remove the pid file after service stopped.

spark git commit: [Spark-5889] Remove pid file after stopping service.

Repository: spark Updated Branches: refs/heads/master a5fed3435 - ad6b169de [Spark-5889] Remove pid file after stopping service. Currently the pid file is not deleted, and potentially may cause some problem after service is stopped. The fix remove the pid file after service stopped. Author:

spark git commit: [Spark-5889] Remove pid file after stopping service.

Repository: spark Updated Branches: refs/heads/branch-1.3 0c494cf9a - ff8976ec7 [Spark-5889] Remove pid file after stopping service. Currently the pid file is not deleted, and potentially may cause some problem after service is stopped. The fix remove the pid file after service stopped.

spark git commit: [SPARK-6372] [core] Propagate --conf to child processes.

Repository: spark Updated Branches: refs/heads/master 357900311 - 6205a255a [SPARK-6372] [core] Propagate --conf to child processes. And add unit test. Author: Marcelo Vanzin van...@cloudera.com Closes #5057 from vanzin/SPARK-6372 and squashes the following commits: b33728b [Marcelo

spark git commit: [SPARK-6325] [core, yarn] Do not change target executor count when killing executors.

Repository: spark Updated Branches: refs/heads/branch-1.3 ff0a7f400 - 1723f0591 [SPARK-6325] [core,yarn] Do not change target executor count when killing executors. The dynamic execution code has two ways to reduce the number of executors: one where it reduces the total number of executors

spark git commit: [SPARK-6286][minor] Handle missing Mesos case TASK_ERROR.

Repository: spark Updated Branches: refs/heads/master e09c852d6 - 9d112a958 [SPARK-6286][minor] Handle missing Mesos case TASK_ERROR. Author: Iulian Dragos jagua...@gmail.com Closes #5000 from dragos/issue/task-error-case and squashes the following commits: e063627 [Iulian Dragos] Handle

spark git commit: SPARK-6389 YARN app diagnostics report doesn't report NPEs

Repository: spark Updated Branches: refs/heads/master 6205a255a - e09c852d6 SPARK-6389 YARN app diagnostics report doesn't report NPEs Trivial patch to implicitly call `Exception.toString()` over `Exception.getMessage()` —this defaults to including the exception class any non-null

spark git commit: [SPARK-6286][minor] Handle missing Mesos case TASK_ERROR.

Repository: spark Updated Branches: refs/heads/branch-1.3 ba8352c76 - ff0a7f400 [SPARK-6286][minor] Handle missing Mesos case TASK_ERROR. Author: Iulian Dragos jagua...@gmail.com Closes #5000 from dragos/issue/task-error-case and squashes the following commits: e063627 [Iulian Dragos]

[2/2] spark git commit: [SPARK-6132] ContextCleaner race condition across SparkContexts

[SPARK-6132] ContextCleaner race condition across SparkContexts The problem is that `ContextCleaner` may clean variables that belong to a different `SparkContext`. This can happen if the `SparkContext` to which the cleaner belongs stops, and a new one is started immediately afterwards in the

[1/2] spark git commit: [SPARK-6132][HOTFIX] ContextCleaner InterruptedException should be quiet

Repository: spark Updated Branches: refs/heads/branch-1.2 a2a94a154 - abdcec673 [SPARK-6132][HOTFIX] ContextCleaner InterruptedException should be quiet If the cleaner is stopped, we shouldn't print a huge stack trace when the cleaner thread is interrupted because we purposefully did this.

spark git commit: [SPARK-6448] Make history server log parse exceptions

Repository: spark Updated Branches: refs/heads/master 9b1e1f20d - b9fe504b4 [SPARK-6448] Make history server log parse exceptions This helped me to debug a parse error that was due to the event log format changing recently. Author: Ryan Williams ryan.blake.willi...@gmail.com Closes #5122

[1/2] spark git commit: [SPARK-6132] ContextCleaner race condition across SparkContexts

Repository: spark Updated Branches: refs/heads/branch-1.1 c5836816f - 39761f515 [SPARK-6132] ContextCleaner race condition across SparkContexts The problem is that `ContextCleaner` may clean variables that belong to a different `SparkContext`. This can happen if the `SparkContext` to which

[2/2] spark git commit: [SPARK-6132][HOTFIX] ContextCleaner InterruptedException should be quiet

[SPARK-6132][HOTFIX] ContextCleaner InterruptedException should be quiet If the cleaner is stopped, we shouldn't print a huge stack trace when the cleaner thread is interrupted because we purposefully did this. Author: Andrew Or and...@databricks.com Closes #4882 from

spark git commit: SPARK-6454 [DOCS] Fix links to pyspark api

Repository: spark Updated Branches: refs/heads/branch-1.3 e60fbf6c4 - 3ba295fa0 SPARK-6454 [DOCS] Fix links to pyspark api Author: Kamil Smuga smugaka...@gmail.com Author: stderr smugaka...@gmail.com Closes #5120 from kamilsmuga/master and squashes the following commits: fee3281 [Kamil

spark git commit: SPARK-6454 [DOCS] Fix links to pyspark api

Repository: spark Updated Branches: refs/heads/master adb2ff752 - 6ef48632f SPARK-6454 [DOCS] Fix links to pyspark api Author: Kamil Smuga smugaka...@gmail.com Author: stderr smugaka...@gmail.com Closes #5120 from kamilsmuga/master and squashes the following commits: fee3281 [Kamil Smuga]

spark git commit: [SPARK-6455] [docs] Correct some mistakes and typos

Repository: spark Updated Branches: refs/heads/master b9fe504b4 - ab4f516fb [SPARK-6455] [docs] Correct some mistakes and typos Correct some typos. Correct a mistake in lib/PageRank.scala. The first PageRank implementation uses standalone Graph interface, but the second uses Pregel

spark git commit: [SPARK-6453][Mesos] Some Mesos*Suite have a different package with their classes

Repository: spark Updated Branches: refs/heads/master ab4f516fb - adb2ff752 [SPARK-6453][Mesos] Some Mesos*Suite have a different package with their classes - Moved Suites from o.a.s.s.mesos to o.a.s.s.cluster.mesos Author: Jongyoul Lee jongy...@gmail.com Closes #5126 from

spark git commit: [SPARK-6337][Documentation, SQL]Spark 1.3 doc fixes

Repository: spark Updated Branches: refs/heads/branch-1.3 3ba295fa0 - 857e8a60e [SPARK-6337][Documentation, SQL]Spark 1.3 doc fixes Author: vinodkc vinod.kc...@gmail.com Closes #5112 from vinodkc/spark_1.3_doc_fixes and squashes the following commits: 2c6aee6 [vinodkc] Spark 1.3 doc fixes

spark git commit: [SPARK-6337][Documentation, SQL]Spark 1.3 doc fixes

Repository: spark Updated Branches: refs/heads/master 7a0da4770 - 2bf40c58e [SPARK-6337][Documentation, SQL]Spark 1.3 doc fixes Author: vinodkc vinod.kc...@gmail.com Closes #5112 from vinodkc/spark_1.3_doc_fixes and squashes the following commits: 2c6aee6 [vinodkc] Spark 1.3 doc fixes

spark git commit: [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to work around MESOS-1688

Repository: spark Updated Branches: refs/heads/branch-1.3 ad4756321 - 43fcab01a [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to work around MESOS-1688 - MESOS_NATIVE_LIBRARY become deprecated - Chagned MESOS_NATIVE_LIBRARY to MESOS_NATIVE_JAVA_LIBRARY Author: Jongyoul Lee jongy...@gmail.com

spark git commit: [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to work around MESOS-1688

Repository: spark Updated Branches: refs/heads/master 62ede5383 - aa6536fa3 [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to work around MESOS-1688 - MESOS_NATIVE_LIBRARY become deprecated - Chagned MESOS_NATIVE_LIBRARY to MESOS_NATIVE_JAVA_LIBRARY Author: Jongyoul Lee jongy...@gmail.com

svn commit: r1666996 - /spark/site/releases/spark-release-1-3-0.html

Author: srowen Date: Mon Mar 16 13:49:54 2015 New Revision: 1666996 URL: http://svn.apache.org/r1666996 Log: Removed extra word 'extended' Modified: spark/site/releases/spark-release-1-3-0.html Modified: spark/site/releases/spark-release-1-3-0.html URL: http://svn.apache.org/viewvc/spark

spark git commit: [SPARK-6300][Spark Core] sc.addFile(path) does not support the relative path.

Repository: spark Updated Branches: refs/heads/master 45f4c6612 - 00e730b94 [SPARK-6300][Spark Core] sc.addFile(path) does not support the relative path. when i run cmd like that sc.addFile(../test.txt), it did not work and throwed an exception: java.lang.IllegalArgumentException:

spark git commit: [SPARK-6300][Spark Core] sc.addFile(path) does not support the relative path.

Repository: spark Updated Branches: refs/heads/branch-1.3 43fcab01a - 724aab4b9 [SPARK-6300][Spark Core] sc.addFile(path) does not support the relative path. when i run cmd like that sc.addFile(../test.txt), it did not work and throwed an exception: java.lang.IllegalArgumentException:

[2/2] spark git commit: [SPARK-4011] tighten the visibility of the members in Master/Worker class

[SPARK-4011] tighten the visibility of the members in Master/Worker class https://issues.apache.org/jira/browse/SPARK-4011 Currently, most of the members in Master/Worker are with public accessibility. We might wish to tighten the accessibility of them a bit more discussion is here:

[1/2] spark git commit: [SPARK-4011] tighten the visibility of the members in Master/Worker class

Repository: spark Updated Branches: refs/heads/master b2d8c0222 - 25f35806e http://git-wip-us.apache.org/repos/asf/spark/blob/25f35806/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala -- diff --git

spark git commit: [SPARK-6269] [CORE] Use ScalaRunTime's array methods instead of java.lang.reflect.Array in size estimation

Repository: spark Updated Branches: refs/heads/master 25f35806e - 005d1c5f2 [SPARK-6269] [CORE] Use ScalaRunTime's array methods instead of java.lang.reflect.Array in size estimation This patch switches the usage of java.lang.reflect.Array in Size estimation to using scala's RunTime

spark git commit: [SPARK-5843] [API] Allowing map-side combine to be specified in Java.

Repository: spark Updated Branches: refs/heads/master 797f8a000 - 3c4e486b9 [SPARK-5843] [API] Allowing map-side combine to be specified in Java. Specifically, when calling JavaPairRDD.combineByKey(), there is a new six-parameter method that exposes the map-side-combine boolean as the fifth

spark git commit: [SPARK-6402][DOC] - Remove some refererences to shark in docs and ec2

Repository: spark Updated Branches: refs/heads/master 2c3f83c34 - 797f8a000 [SPARK-6402][DOC] - Remove some refererences to shark in docs and ec2 EC2 script and job scheduling documentation still refered to Shark. I removed these references. I also removed a remaining `SHARK_VERSION`

  1   2   3   4   5   6   7   8   9   10   >