spark git commit: [MINOR] Fix Java Lint errors introduced by #13286 and #13280

Repository: spark Updated Branches: refs/heads/branch-2.0 a790ac579 -> 5e9a8e715 [MINOR] Fix Java Lint errors introduced by #13286 and #13280 ## What changes were proposed in this pull request? revived #13464 Fix Java Lint errors introduced by #13286 and #13280 Before: ``` Using `mvn` from

spark git commit: [DOCS] fix example code issues in documentation

Repository: spark Updated Branches: refs/heads/master 5728aa558 -> 2d34183b2 [DOCS] fix example code issues in documentation ## What changes were proposed in this pull request? Fixed broken java code examples in streaming documentation Attn: tdas Author: Matthew Wise

spark git commit: [SPARK-15645][STREAMING] Fix some typos of Streaming module

Repository: spark Updated Branches: refs/heads/master 1360a6d63 -> 5728aa558 [SPARK-15645][STREAMING] Fix some typos of Streaming module ## What changes were proposed in this pull request? No code change, just some typo fixing. ## How was this patch tested? Manually run project build with

spark git commit: [SPARK-15645][STREAMING] Fix some typos of Streaming module

Repository: spark Updated Branches: refs/heads/branch-2.0 6dae027a6 -> 7076b3707 [SPARK-15645][STREAMING] Fix some typos of Streaming module ## What changes were proposed in this pull request? No code change, just some typo fixing. ## How was this patch tested? Manually run project build

spark git commit: [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full

Repository: spark Updated Branches: refs/heads/branch-1.6 ea84b3373 -> 714f4d78a [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full 1. The class allocated 4x space than needed as it was using `Int` to store the `Byte` values 2. If

spark git commit: [CORE][MINOR][DOC] Removing incorrect scaladoc

Repository: spark Updated Branches: refs/heads/master 57adb77e6 -> 695470429 [CORE][MINOR][DOC] Removing incorrect scaladoc ## What changes were proposed in this pull request? I don't think the method will ever throw an exception so removing a false comment. Sorry srowen and rxin again -

spark git commit: [CORE][MINOR][DOC] Removing incorrect scaladoc

Repository: spark Updated Branches: refs/heads/branch-2.0 ef8d16ec4 -> 1f4c4def9 [CORE][MINOR][DOC] Removing incorrect scaladoc ## What changes were proposed in this pull request? I don't think the method will ever throw an exception so removing a false comment. Sorry srowen and rxin ag

spark git commit: [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full

Repository: spark Updated Branches: refs/heads/master 04f925ede -> ac38bdc75 [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full ## What changes were proposed in this pull request? 1. The class allocated 4x space than needed as it was

spark git commit: [SPARK-15709][SQL] Prevent `freqItems` from raising `UnsupportedOperationException: empty.min`

Repository: spark Updated Branches: refs/heads/branch-2.0 b78352fc3 -> a5cb7fc6d [SPARK-15709][SQL] Prevent `freqItems` from raising `UnsupportedOperationException: empty.min` ## What changes were proposed in this pull request? Currently, `freqItems` raises `UnsupportedOperationException`

spark git commit: [SPARK-15208][WIP][CORE][STREAMING][DOCS] Update Spark examples with AccumulatorV2

Repository: spark Updated Branches: refs/heads/branch-2.0 22eb08369 -> 8f98eef69 [SPARK-15208][WIP][CORE][STREAMING][DOCS] Update Spark examples with AccumulatorV2 ## What changes were proposed in this pull request? The patch updates the codes & docs in the example module as well as the

spark git commit: [SPARK-15605][ML][EXAMPLES] Fix broken ML JavaDeveloperApiExample.

Repository: spark Updated Branches: refs/heads/branch-2.0 8f98eef69 -> b78352fc3 [SPARK-15605][ML][EXAMPLES] Fix broken ML JavaDeveloperApiExample. ## What changes were proposed in this pull request? See [SPARK-15605](https://issues.apache.org/jira/browse/SPARK-15605) for the detail of this

spark git commit: [SPARK-15605][ML][EXAMPLES] Fix broken ML JavaDeveloperApiExample.

Repository: spark Updated Branches: refs/heads/master a0eec8e8f -> 4fe7c7bd1 [SPARK-15605][ML][EXAMPLES] Fix broken ML JavaDeveloperApiExample. ## What changes were proposed in this pull request? See [SPARK-15605](https://issues.apache.org/jira/browse/SPARK-15605) for the detail of this bug.

spark git commit: [SPARK-15322][SQL][FOLLOWUP] Use the new long accumulator for old int accumulators.

Repository: spark Updated Branches: refs/heads/branch-2.0 a5cb7fc6d -> fe639adea [SPARK-15322][SQL][FOLLOWUP] Use the new long accumulator for old int accumulators. ## What changes were proposed in this pull request? This PR corrects the remaining cases for using old accumulators. This

spark git commit: [SPARK-15709][SQL] Prevent `freqItems` from raising `UnsupportedOperationException: empty.min`

Repository: spark Updated Branches: refs/heads/master 4fe7c7bd1 -> b85d18f3b [SPARK-15709][SQL] Prevent `freqItems` from raising `UnsupportedOperationException: empty.min` ## What changes were proposed in this pull request? Currently, `freqItems` raises `UnsupportedOperationException` on

spark git commit: [SPARK-15322][SQL][FOLLOWUP] Use the new long accumulator for old int accumulators.

Repository: spark Updated Branches: refs/heads/master b85d18f3b -> 252417fa2 [SPARK-15322][SQL][FOLLOWUP] Use the new long accumulator for old int accumulators. ## What changes were proposed in this pull request? This PR corrects the remaining cases for using old accumulators. This does

spark git commit: [SPARK-15208][WIP][CORE][STREAMING][DOCS] Update Spark examples with AccumulatorV2

Repository: spark Updated Branches: refs/heads/master 5eea33230 -> a0eec8e8f [SPARK-15208][WIP][CORE][STREAMING][DOCS] Update Spark examples with AccumulatorV2 ## What changes were proposed in this pull request? The patch updates the codes & docs in the example module as well as the related

spark git commit: [SPARK-14204][SQL] register driverClass rather than user-specified class

Repository: spark Updated Branches: refs/heads/branch-1.6 714f4d78a -> 0a13e4c07 [SPARK-14204][SQL] register driverClass rather than user-specified class This pull request fixes an issue in which cluster-mode executors fail to properly register a JDBC driver when the driver is provided in a

spark git commit: [SPARK-15542][SPARKR] Make error message clear for script './R/install-dev.sh' when R is missing on Mac

Repository: spark Updated Branches: refs/heads/branch-2.0 c99ad9215 -> 702755f92 [SPARK-15542][SPARKR] Make error message clear for script './R/install-dev.sh' when R is missing on Mac https://issues.apache.org/jira/browse/SPARK-15542 ## What changes were proposed in this pull request?

spark git commit: [SPARK-15542][SPARKR] Make error message clear for script './R/install-dev.sh' when R is missing on Mac

Repository: spark Updated Branches: refs/heads/master 008a5377d -> 6ab973ec5 [SPARK-15542][SPARKR] Make error message clear for script './R/install-dev.sh' when R is missing on Mac https://issues.apache.org/jira/browse/SPARK-15542 ## What changes were proposed in this pull request? When

spark git commit: [SPARK-15523][ML][MLLIB] Update JPMML to 1.2.15

Repository: spark Updated Branches: refs/heads/master e451f7f0c -> 6d506c9ae [SPARK-15523][ML][MLLIB] Update JPMML to 1.2.15 ## What changes were proposed in this pull request? See https://issues.apache.org/jira/browse/SPARK-15523 This PR replaces PR #13293. It's isolated to a new branch,

spark git commit: [SPARK-15523][ML][MLLIB] Update JPMML to 1.2.15

Repository: spark Updated Branches: refs/heads/branch-2.0 c54a07348 -> 87374de43 [SPARK-15523][ML][MLLIB] Update JPMML to 1.2.15 ## What changes were proposed in this pull request? See https://issues.apache.org/jira/browse/SPARK-15523 This PR replaces PR #13293. It's isolated to a new

spark git commit: [SPARK-10722] RDDBlockId not found in driver-heartbeater

Repository: spark Updated Branches: refs/heads/branch-1.6 9a18115a8 -> 5cc1e2cec [SPARK-10722] RDDBlockId not found in driver-heartbeater ## What changes were proposed in this pull request? To ensure that the deserialization of TaskMetrics uses a ClassLoader that knows about RDDBlockIds.

spark git commit: [SPARK-15659][SQL] Ensure FileSystem is gotten from path

Repository: spark Updated Branches: refs/heads/master 1dd925644 -> e4ce1bc4f [SPARK-15659][SQL] Ensure FileSystem is gotten from path ## What changes were proposed in this pull request? Currently `spark.sql.warehouse.dir` is pointed to local dir by default, which will throw exception when

spark git commit: [SPARK-15664][MLLIB] Replace FileSystem.get(conf) with path.getFileSystem(conf) when removing CheckpointFile in MLlib

Repository: spark Updated Branches: refs/heads/master e4ce1bc4f -> 6563d72b1 [SPARK-15664][MLLIB] Replace FileSystem.get(conf) with path.getFileSystem(conf) when removing CheckpointFile in MLlib ## What changes were proposed in this pull request? if sparkContext.set CheckpointDir to another

spark git commit: [SPARK-15664][MLLIB] Replace FileSystem.get(conf) with path.getFileSystem(conf) when removing CheckpointFile in MLlib

Repository: spark Updated Branches: refs/heads/branch-2.0 29a1cdfc4 -> 8fb125bdf [SPARK-15664][MLLIB] Replace FileSystem.get(conf) with path.getFileSystem(conf) when removing CheckpointFile in MLlib ## What changes were proposed in this pull request? if sparkContext.set CheckpointDir to

spark git commit: [SPARK-15528][SQL] Fix race condition in NumberConverter

Repository: spark Updated Branches: refs/heads/master 6878f3e2e -> 95db8a44f [SPARK-15528][SQL] Fix race condition in NumberConverter ## What changes were proposed in this pull request? A local variable in NumberConverter is wrongly shared between threads. This pr fixes the race condition.

spark git commit: [SPARK-15528][SQL] Fix race condition in NumberConverter

Repository: spark Updated Branches: refs/heads/branch-2.0 3d00c7015 -> 2e3ead20c [SPARK-15528][SQL] Fix race condition in NumberConverter ## What changes were proposed in this pull request? A local variable in NumberConverter is wrongly shared between threads. This pr fixes the race

spark git commit: [SPARK-15528][SQL] Fix race condition in NumberConverter

Repository: spark Updated Branches: refs/heads/branch-1.6 c53c83ce8 -> ea84b3373 [SPARK-15528][SQL] Fix race condition in NumberConverter ## What changes were proposed in this pull request? A local variable in NumberConverter is wrongly shared between threads. This pr fixes the race

spark git commit: [MINOR] Resolve a number of miscellaneous build warnings

lly related to the Accumulator API. That will happen separately. Jenkins Author: Sean Owen <so...@cloudera.com> Closes #13377 from srowen/BuildWarnings. (cherry picked from commit ce1572d16f03d383071bcc1f30ede551e8ded49f) Signed-off-by: Sean Owen <so...@cloudera.com> Project:

spark git commit: [MINOR] Resolve a number of miscellaneous build warnings

ess a large number of deprecation warnings, especially related to the Accumulator API. That will happen separately. ## How was this patch tested? Jenkins Author: Sean Owen <so...@cloudera.com> Closes #13377 from srowen/BuildWarnings. Project: http://git-wip-us.apache.org/repos/asf/spark/re

spark git commit: [SPARK-6005][TESTS] Fix flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery

Repository: spark Updated Branches: refs/heads/branch-1.6 d98fb19c1 -> 4fdac3c27 [SPARK-6005][TESTS] Fix flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery ## What changes were proposed in this pull request? Because this test extracts data from `DStream.generatedRDDs`

spark git commit: [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config

0.6 in order to make it fit within default JVM old generation size (2/3 heap). See JIRA discussion. This means a full cache doesn't spill into the new gen. CC andrewor14 ## How was this patch tested? Jenkins tests. Author: Sean Owen <so...@cloudera.com> Closes #13618 from srowen/SPARK-15796.

spark git commit: [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config

der to make it fit within default JVM old generation size (2/3 heap). See JIRA discussion. This means a full cache doesn't spill into the new gen. CC andrewor14 ## How was this patch tested? Jenkins tests. Author: Sean Owen <so...@cloudera.com> Closes #13618 from srowen/SPARK-15796.

spark git commit: [SPARK-15922][MLLIB] `toIndexedRowMatrix` should consider the case `cols < offset+colsPerBlock`

Repository: spark Updated Branches: refs/heads/master f9bf15d9b -> 36110a830 [SPARK-15922][MLLIB] `toIndexedRowMatrix` should consider the case `cols < offset+colsPerBlock` ## What changes were proposed in this pull request? SPARK-15922 reports the following scenario throwing an exception

spark git commit: [SPARK-15942][REPL] Unblock `:reset` command in REPL.

Repository: spark Updated Branches: refs/heads/branch-2.0 dc85bd0a0 -> 2c1c337ba [SPARK-15942][REPL] Unblock `:reset` command in REPL. ## What changes were proposed in this pull (Paste from JIRA issue.) As a follow up for SPARK-15697, I have following semantics for `:reset` command. On

spark git commit: [SPARK-15660][CORE] Update RDD `variance/stdev` description and add popVariance/popStdev

Repository: spark Updated Branches: refs/heads/master 4374a46bf -> 5eef1e6c6 [SPARK-15660][CORE] Update RDD `variance/stdev` description and add popVariance/popStdev ## What changes were proposed in this pull request? In Spark-11490, `variance/stdev` are redefined as the **sample**

spark git commit: [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table

Repository: spark Updated Branches: refs/heads/branch-2.0 0499ed961 -> 34a8e23c7 [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table ## What changes were proposed in this pull request? 1. FORMATTED is actually supported, but partition is not supported; 2. Remove parenthesis as it

spark git commit: [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table

Repository: spark Updated Branches: refs/heads/master a58f40239 -> f3a768b7b [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table ## What changes were proposed in this pull request? 1. FORMATTED is actually supported, but partition is not supported; 2. Remove parenthesis as it is

spark git commit: [SPARK-16129][CORE][SQL] Eliminate direct use of commons-lang classes in favor of commons-lang3

ng3` and forbid the former via scalastyle; remove `NotImplementedException` from `comons-lang` in favor of JDK `UnsupportedOperationException` ## How was this patch tested? Jenkins tests Author: Sean Owen <so...@cloudera.com> Closes #13843 from srowen/SPARK-16129. Project: http://git-wip-us.a

spark git commit: [SPARK-16129][CORE][SQL] Eliminate direct use of commons-lang classes in favor of commons-lang3

ng3` and forbid the former via scalastyle; remove `NotImplementedException` from `comons-lang` in favor of JDK `UnsupportedOperationException` ## How was this patch tested? Jenkins tests Author: Sean Owen <so...@cloudera.com> Closes #13843 from srowen/SPARK-16129. (cherry picked fr

spark git commit: [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite

Repository: spark Updated Branches: refs/heads/master 2d2f607bf -> f4fd7432f [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite ## What changes were proposed in this pull request? Since SPARK-13220(Deprecate "yarn-client" and "yarn-cluster"), YarnClusterSuite

spark git commit: [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite

Repository: spark Updated Branches: refs/heads/branch-2.0 3ccdd6b9c -> b6420db9e [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite ## What changes were proposed in this pull request? Since SPARK-13220(Deprecate "yarn-client" and "yarn-cluster"),

spark git commit: [SPARK-12736][CORE][DEPLOY] Standalone Master cannot be started due t…

Repository: spark Updated Branches: refs/heads/master 5b0d54433 -> b78e028e3 [SPARK-12736][CORE][DEPLOY] Standalone Master cannot be started due t… …o NoClassDefFoundError: org/spark-project/guava/collect/Maps /cc srowen rxin Author: Jacek Laskowski <ja...@japila.pl> Closes #1

spark git commit: [SPARK-2930] clarify docs on using webhdfs with spark.yarn.access.nam…

Repository: spark Updated Branches: refs/heads/master d0a5c32bd -> 96fb894d4 [SPARK-2930] clarify docs on using webhdfs with spark.yarn.access.nam… …enodes Author: Tom Graves Closes #10699 from tgravescs/SPARK-2930. Project:

spark git commit: [SPARK-12655][GRAPHX] GraphX does not unpersist RDDs

Repository: spark Updated Branches: refs/heads/master fe7246fea -> d0a5c32bd [SPARK-12655][GRAPHX] GraphX does not unpersist RDDs Some VertexRDD and EdgeRDD are created during the intermediate step of g.connectedComponents() but unnecessarily left cached after the method is done. The fix is

spark git commit: [SPARK-12701][CORE] FileAppender should use join to ensure writing thread completion

Repository: spark Updated Branches: refs/heads/branch-1.6 5a0052839 -> 773366818 [SPARK-12701][CORE] FileAppender should use join to ensure writing thread completion Changed Logging FileAppender to use join in `awaitTermination` to ensure that thread is properly finished before returning.

spark git commit: [SPARK-9844][CORE] File appender race condition during shutdown

Repository: spark Updated Branches: refs/heads/master 8f13cd4cc -> 56cdbd654 [SPARK-9844][CORE] File appender race condition during shutdown When an Executor process is destroyed, the FileAppender that is asynchronously reading the stderr stream of the process can throw an IOException during

spark git commit: [SPARK-9844][CORE] File appender race condition during shutdown

Repository: spark Updated Branches: refs/heads/branch-1.6 a490787da -> 0c67993cf [SPARK-9844][CORE] File appender race condition during shutdown When an Executor process is destroyed, the FileAppender that is asynchronously reading the stderr stream of the process can throw an IOException

spark git commit: [SPARK-10264][DOCUMENTATION] Added @Since to ml.recomendation

Repository: spark Updated Branches: refs/heads/master bc36b0f1a -> 233d6cee9 [SPARK-10264][DOCUMENTATION] Added @Since to ml.recomendation I create new pr since original pr long time no update. Please help to review. srowen Author: Tommy YU <tumm...@163.com> Closes #10756 fr

spark git commit: [SPARK-12722][DOCS] Fixed typo in Pipeline example

Repository: spark Updated Branches: refs/heads/master 2f7d0b68a -> 86972fa52 [SPARK-12722][DOCS] Fixed typo in Pipeline example http://spark.apache.org/docs/latest/ml-guide.html#example-pipeline ``` val sameModel = Pipeline.load("/tmp/spark-logistic-regression-model") ``` should be ``` val

spark git commit: [SPARK-12722][DOCS] Fixed typo in Pipeline example

Repository: spark Updated Branches: refs/heads/branch-1.6 773366818 -> 5803fce90 [SPARK-12722][DOCS] Fixed typo in Pipeline example http://spark.apache.org/docs/latest/ml-guide.html#example-pipeline ``` val sameModel = Pipeline.load("/tmp/spark-logistic-regression-model") ``` should be ```

spark git commit: [SPARK-7683][PYSPARK] Confusing behavior of fold function of RDD in pyspark

sly, this is a potentially breaking change, so can only happen for 2.x CC davies Author: Sean Owen <so...@cloudera.com> Closes #10771 from srowen/SPARK-7683. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d8c4b00a Tree: http:

spark git commit: [MLLIB] Fix CholeskyDecomposition assertion's message

Repository: spark Updated Branches: refs/heads/branch-1.6 30f55e523 -> 962e618ec [MLLIB] Fix CholeskyDecomposition assertion's message Change assertion's message so it's consistent with the code. The old message says that the invoked method was lapack.dports, where in fact it was

spark git commit: [MLLIB] Fix CholeskyDecomposition assertion's message

Repository: spark Updated Branches: refs/heads/master d8c4b00a2 -> ebd9ce0f1 [MLLIB] Fix CholeskyDecomposition assertion's message Change assertion's message so it's consistent with the code. The old message says that the invoked method was lapack.dports, where in fact it was lapack.dppsv

svn commit: r1750410 [2/2] - in /spark: ./ _plugins/ mllib/ releases/_posts/ site/ site/mllib/ site/news/ site/releases/ site/sql/ site/streaming/ sql/ streaming/

Modified: spark/site/releases/spark-release-1-1-0.html URL: http://svn.apache.org/viewvc/spark/site/releases/spark-release-1-1-0.html?rev=1750410=1750409=1750410=diff == --- spark/site/releases/spark-release-1-1-0.html

svn commit: r1750410 [1/2] - in /spark: ./ _plugins/ mllib/ releases/_posts/ site/ site/mllib/ site/news/ site/releases/ site/sql/ site/streaming/ sql/ streaming/

Author: srowen Date: Mon Jun 27 20:31:41 2016 New Revision: 1750410 URL: http://svn.apache.org/viewvc?rev=1750410=rev Log: Remove Spark site plugins (not used/working); fix jekyll build warning and one bad heading tag; remove inactive {% extra %} tag; commit current output of jekyll

spark git commit: [SPARK-15958] Make initial buffer size for the Sorter configurable

Repository: spark Updated Branches: refs/heads/master a3c7b4187 -> bf665a958 [SPARK-15958] Make initial buffer size for the Sorter configurable ## What changes were proposed in this pull request? Currently the initial buffer size in the sorter is hard coded inside the code and is too small

spark git commit: [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.

Repository: spark Updated Branches: refs/heads/branch-2.0 d079b5de7 -> cbfcdcfb6 [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution. ## What changes were proposed in this pull request? Just adjust the

spark git commit: [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.

Repository: spark Updated Branches: refs/heads/branch-1.6 b7acc1b71 -> 24d59fb64 [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution. ## What changes were proposed in this pull request? Just adjust the

spark git commit: [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.

Repository: spark Updated Branches: refs/heads/master a7d29499d -> a3c7b4187 [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution. ## What changes were proposed in this pull request? Just adjust the size

spark git commit: [SPARK-1301][WEB UI] Added anchor links to Accumulators and Tasks on StagePage

Repository: spark Updated Branches: refs/heads/master bf665a958 -> 3ee9695d1 [SPARK-1301][WEB UI] Added anchor links to Accumulators and Tasks on StagePage ## What changes were proposed in this pull request? Sometimes the "Aggregated Metrics by Executor" table on the Stage page can get very

spark git commit: [SPARK-16193][TESTS] Address flaky ExternalAppendOnlyMapSuite spilling tests

ber of stages that spilled ## How was this patch tested? Existing Jenkins tests. Author: Sean Owen <so...@cloudera.com> Closes #13896 from srowen/SPARK-16193. (cherry picked from commit e87741589a24821b5fe73e5d9ee2164247998580) Signed-off-by: Sean Owen <so...@cloudera.com> Project:

spark git commit: [SPARK-16193][TESTS] Address flaky ExternalAppendOnlyMapSuite spilling tests

ges that spilled ## How was this patch tested? Existing Jenkins tests. Author: Sean Owen <so...@cloudera.com> Closes #13896 from srowen/SPARK-16193. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e8774158 Tree: http:

spark git commit: [SPARK-16193][TESTS] Address flaky ExternalAppendOnlyMapSuite spilling tests

ber of stages that spilled ## How was this patch tested? Existing Jenkins tests. Author: Sean Owen <so...@cloudera.com> Closes #13896 from srowen/SPARK-16193. (cherry picked from commit e87741589a24821b5fe73e5d9ee2164247998580) Signed-off-by: Sean Owen <so...@cloudera.com> Project:

spark git commit: [SPARK-5865][API DOC] Add doc warnings for methods that return local data structures

Repository: spark Updated Branches: refs/heads/master 4f28291f8 -> 81da3bee6 [SPARK-5865][API DOC] Add doc warnings for methods that return local data structures rxin srowen I work out note message for rdd.take function, please help to review. If it's fine, I can apply to all other funct

spark git commit: [SPARK-13132][MLLIB] cache standardization param value in LogisticRegression

Repository: spark Updated Branches: refs/heads/master 81da3bee6 -> bc8890b35 [SPARK-13132][MLLIB] cache standardization param value in LogisticRegression cache the value of the standardization Param in LogisticRegression, rather than re-fetching it from the ParamMap for every index and every

spark git commit: [SPARK-10963][STREAMING][KAFKA] make KafkaCluster public

Repository: spark Updated Branches: refs/heads/master bc8890b35 -> 140ddef37 [SPARK-10963][STREAMING][KAFKA] make KafkaCluster public Author: cody koeninger Closes #9007 from koeninger/SPARK-10963. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-13201][SPARK-13200] Deprecation warning cleanups: KMeans & MFDataGenerator

Repository: spark Updated Branches: refs/heads/master 159198eff -> ce83fe975 [SPARK-13201][SPARK-13200] Deprecation warning cleanups: KMeans & MFDataGenerator KMeans: Make a private non-deprecated version of setRuns API so that we can call it from the PythonAPI without deprecation warnings

spark git commit: [SPARK-13177][EXAMPLES] Update ActorWordCount example to not directly use low level linked list as it is deprecated.

Repository: spark Updated Branches: refs/heads/master c882ec57d -> d9ba4d27f [SPARK-13177][EXAMPLES] Update ActorWordCount example to not directly use low level linked list as it is deprecated. Author: sachin aggarwal Closes #3 from agsachin/master.

spark git commit: [SPARK-13086][SHELL] Use the Scala REPL settings, to enable things like `-i file`.

Repository: spark Updated Branches: refs/heads/master d9ba4d27f -> e30121afa [SPARK-13086][SHELL] Use the Scala REPL settings, to enable things like `-i file`. Now: ``` $ bin/spark-shell -i test.scala NOTE: SPARK_PREPEND_CLASSES is set, placing locally compiled Spark classes ahead of

spark git commit: [SPARK-13165][STREAMING] Replace deprecated synchronizedBuffer in streaming

Repository: spark Updated Branches: refs/heads/master f9307d8fc -> 159198eff [SPARK-13165][STREAMING] Replace deprecated synchronizedBuffer in streaming Building with Scala 2.11 results in the warning trait SynchronizedBuffer in package mutable is deprecated: Synchronization via traits is

spark git commit: [SPARK-13176][CORE] Use native file linking instead of external process ln

Repository: spark Updated Branches: refs/heads/master 3708d13f1 -> f9307d8fc [SPARK-13176][CORE] Use native file linking instead of external process ln Since Spark requires at least JRE 1.7, it is safe to use built-in java.nio.Files. Author: Jakob Odersky Closes #11098

spark git commit: [SPARK-13040][DOCS] Update JDBC deprecated SPARK_CLASSPATH documentation

Repository: spark Updated Branches: refs/heads/master ce83fe975 -> c882ec57d [SPARK-13040][DOCS] Update JDBC deprecated SPARK_CLASSPATH documentation Update JDBC documentation based on http://stackoverflow.com/a/30947090/219530 as SPARK_CLASSPATH is deprecated. Also, that's how it worked,

spark git commit: [SPARK-13170][STREAMING] Investigate replacing SynchronizedQueue as it is deprecated

Repository: spark Updated Branches: refs/heads/master e30121afa -> 68ed3632c [SPARK-13170][STREAMING] Investigate replacing SynchronizedQueue as it is deprecated Replace SynchronizeQueue with synchronized access to a Queue Author: Sean Owen <so...@cloudera.com> Closes #11111 fr

spark git commit: [SPARK-13264][DOC] Removed multi-byte characters in spark-env.sh.template

Repository: spark Updated Branches: refs/heads/master 18bcbbdd8 -> c2f21d889 [SPARK-13264][DOC] Removed multi-byte characters in spark-env.sh.template In spark-env.sh.template, there are multi-byte characters, this PR will remove it. Author: Sasaki Toru Closes

spark git commit: [SPARK-13074][CORE] Add JavaSparkContext. getPersistentRDDs method

Repository: spark Updated Branches: refs/heads/master c2f21d889 -> f9ae99fee [SPARK-13074][CORE] Add JavaSparkContext. getPersistentRDDs method The "getPersistentRDDs()" is a useful API of SparkContext to get cached RDDs. However, the JavaSparkContext does not have this API. Add a simple

spark git commit: [SPARK-11518][DEPLOY, WINDOWS] Handle spaces in Windows command scripts

Repository: spark Updated Branches: refs/heads/master 9269036d8 -> 2ba9b6a2d [SPARK-11518][DEPLOY, WINDOWS] Handle spaces in Windows command scripts Author: Jon Maurer Author: Jonathan Maurer Closes #10789 from tritab/cmd_updates.

spark git commit: [SPARK-13203] Add scalastyle rule banning use of mutable.SynchronizedBuffer

Repository: spark Updated Branches: refs/heads/master 2ba9b6a2d -> e834e421d [SPARK-13203] Add scalastyle rule banning use of mutable.SynchronizedBuffer andrewor14 Please take a look Author: tedyu Closes #11134 from tedyu/master. Project:

svn commit: r1730185 - /spark/site/examples.html

Author: srowen Date: Sat Feb 13 11:36:18 2016 New Revision: 1730185 URL: http://svn.apache.org/viewvc?rev=1730185=rev Log: Regenerate examples.html to correct "Liquid error: pygments" Modified: spark/site/examples.html Modified: spark/site/examples.html URL: http://svn.apache.

spark git commit: [SPARK-13142][WEB UI] Problem accessing Web UI /logPage/ on Microsoft Windows

Repository: spark Updated Branches: refs/heads/branch-1.6 18661a2bb -> 93a55f3df [SPARK-13142][WEB UI] Problem accessing Web UI /logPage/ on Microsoft Windows Due to being on a Windows platform I have been unable to run the tests as described in the "Contributing to Spark" instructions. As

spark git commit: [SPARK-13142][WEB UI] Problem accessing Web UI /logPage/ on Microsoft Windows

Repository: spark Updated Branches: refs/heads/master 2228f074e -> 374c4b286 [SPARK-13142][WEB UI] Problem accessing Web UI /logPage/ on Microsoft Windows Due to being on a Windows platform I have been unable to run the tests as described in the "Contributing to Spark" instructions. As the

spark git commit: [SPARK-12961][CORE] Prevent snappy-java memory leak

Repository: spark Updated Branches: refs/heads/branch-1.6 b40e58cf2 -> 572bc3999 [SPARK-12961][CORE] Prevent snappy-java memory leak JIRA: https://issues.apache.org/jira/browse/SPARK-12961 To prevent memory leak in snappy-java, just call the method once and cache the result. After the

spark git commit: [SPARK-12961][CORE] Prevent snappy-java memory leak

Repository: spark Updated Branches: refs/heads/master 6743de3a9 -> 5936bf9fa [SPARK-12961][CORE] Prevent snappy-java memory leak JIRA: https://issues.apache.org/jira/browse/SPARK-12961 To prevent memory leak in snappy-java, just call the method once and cache the result. After the library

spark git commit: [SPARK-3369][CORE][STREAMING] Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

Iterator, not Iterable. Also fix DStream.flatMap to require a function producing TraversableOnce only, not Traversable. CC rxin pwendell for API change; tdas since it also touches streaming. Author: Sean Owen <so...@cloudera.com> Closes #10413 from srowen/SPARK-3369. Project:

spark git commit: Provide same info as in spark-submit --help

Repository: spark Updated Branches: refs/heads/master 4a0912321 -> c2204436a Provide same info as in spark-submit --help this is stated for --packages and --repositories. Without stating it for --jars, people expect a standard java classpath to work, with expansion and using a different

spark git commit: [SPARK-12759][Core][Spark should fail fast if --executor-memory is too small for spark to start]

Repository: spark Updated Branches: refs/heads/master 831429170 -> bcfd55fa9 [SPARK-12759][Core][Spark should fail fast if --executor-memory is too small for spark to start] Added an exception to be thrown in UnifiedMemoryManager.scala if the configuration given for executor memory is too

spark git commit: [SPARK-13390][SQL][BRANCH-1.6] Fix the issue that Iterator.map().toSeq is not Serializable

Repository: spark Updated Branches: refs/heads/branch-1.6 573a2c97e -> 06f4fce29 [SPARK-13390][SQL][BRANCH-1.6] Fix the issue that Iterator.map().toSeq is not Serializable ## What changes were proposed in this pull request? `scala.collection.Iterator`'s methods (e.g., map, filter) will

spark git commit: [SPARK-13220][CORE] deprecate yarn-client and yarn-cluster mode

Repository: spark Updated Branches: refs/heads/master 87250580f -> e99d01709 [SPARK-13220][CORE] deprecate yarn-client and yarn-cluster mode Author: jerryshao Closes #11229 from jerryshao/SPARK-13220. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-13117][WEB UI] WebUI should use the local ip not 0.0.0.0

Repository: spark Updated Branches: refs/heads/master 2b2c8c332 -> 2e44031fa [SPARK-13117][WEB UI] WebUI should use the local ip not 0.0.0.0 Fixed the HTTP Server Host Name/IP issue i.e. HTTP Server to take the configured host name/IP and not '0.0.0.0' always. Author: Devaraj K

spark git commit: [SPARK-13444][MLLIB] QuantileDiscretizer chooses bad splits on large DataFrames

Repository: spark Updated Branches: refs/heads/master 3fa6491be -> 6f8e835c6 [SPARK-13444][MLLIB] QuantileDiscretizer chooses bad splits on large DataFrames ## What changes were proposed in this pull request? Change line 113 of QuantileDiscretizer.scala to `val requiredSamples =

spark git commit: [SPARK-13490][ML] ML LinearRegression should cache standardization param value

L1 regularization should also cache the value of the ```standardization``` rather than re-fetching it from the ```ParamMap``` for every OWLQN iteration. cc srowen ## How was this patch tested? No extra tests are added. It should pass all existing tests. Author: Yanbo Liang <yblia...@gmail.com> Clos

spark git commit: [SPARK-13444][MLLIB] QuantileDiscretizer chooses bad splits on large DataFrames

Repository: spark Updated Branches: refs/heads/branch-1.6 3cc938ac8 -> cb869a143 [SPARK-13444][MLLIB] QuantileDiscretizer chooses bad splits on large DataFrames Change line 113 of QuantileDiscretizer.scala to `val requiredSamples = math.max(numBins * numBins, 1.0)` so that

spark git commit: [SPARK-13441][YARN] Fix NPE in yarn Client.createConfArchive method

Repository: spark Updated Branches: refs/heads/master 6f8e835c6 -> fae88af18 [SPARK-13441][YARN] Fix NPE in yarn Client.createConfArchive method ## What changes were proposed in this pull request? Instead of using result of File.listFiles() directly, which may throw NPE, check for null

spark git commit: [SPARK-13441][YARN] Fix NPE in yarn Client.createConfArchive method

Repository: spark Updated Branches: refs/heads/branch-1.6 cb869a143 -> 1f031635f [SPARK-13441][YARN] Fix NPE in yarn Client.createConfArchive method ## What changes were proposed in this pull request? Instead of using result of File.listFiles() directly, which may throw NPE, check for null

spark git commit: [SPARK-13439][MESOS] Document that spark.mesos.uris is comma-separated

Repository: spark Updated Branches: refs/heads/master fae88af18 -> c98a93ded [SPARK-13439][MESOS] Document that spark.mesos.uris is comma-separated Author: Michael Gummelt Closes #11311 from mgummelt/document_csv. Project:

spark git commit: [SPARK-13439][MESOS] Document that spark.mesos.uris is comma-separated

Repository: spark Updated Branches: refs/heads/branch-1.6 1f031635f -> e3802a752 [SPARK-13439][MESOS] Document that spark.mesos.uris is comma-separated Author: Michael Gummelt Closes #11311 from mgummelt/document_csv. (cherry picked from commit

spark git commit: [SPARK-12994][CORE] It is not necessary to create ExecutorAllocationM…

d4d4cd2cd Parents: d81a713 Author: Jeff Zhang <zjf...@apache.org> Authored: Mon Feb 29 12:08:37 2016 + Committer: Sean Owen <srowen@percolateur.local> Committed: Mon Feb 29 12:08:37 2016 + -- .../s

spark git commit: [SPARK-13506][MLLIB] Fix the wrong parameter in R code comment in AssociationRulesSuite

Repository: spark Updated Branches: refs/heads/master 2f91f5ac0 -> ac5c63528 [SPARK-13506][MLLIB] Fix the wrong parameter in R code comment in AssociationRulesSuite JIRA: https://issues.apache.org/jira/browse/SPARK-13506 ## What changes were proposed in this pull request? just chang R

spark git commit: [SPARK-7483][MLLIB] Upgrade Chill to 0.7.2 to support Kryo with FPGrowth

Repository: spark Updated Branches: refs/heads/master 7a0cb4e58 -> ec0cc75e1 [SPARK-7483][MLLIB] Upgrade Chill to 0.7.2 to support Kryo with FPGrowth It registers more Scala classes, including ListBuffer to support Kryo with FPGrowth. See https://github.com/twitter/chill/releases for

spark git commit: [MINOR][SQL] Fix modifier order.

Repository: spark Updated Branches: refs/heads/master 7af0de076 -> 727e78014 [MINOR][SQL] Fix modifier order. ## What changes were proposed in this pull request? This PR fixes the order of modifier from `abstract public` into `public abstract`. Currently, when we run `./dev/lint-java`, it

<    4   5   6   7   8   9   10   11   12   13   >