git commit: SPARK-2711. Create a ShuffleMemoryManager to track memory for all spilling collections

2014-08-04 Thread matei
Repository: spark Updated Branches: refs/heads/master 066765d60 -> 4fde28c20 SPARK-2711. Create a ShuffleMemoryManager to track memory for all spilling collections This tracks memory properly if there are multiple spilling collections in the same task (which was a problem before), and also i

git commit: SPARK-2711. Create a ShuffleMemoryManager to track memory for all spilling collections

2014-08-04 Thread matei
Repository: spark Updated Branches: refs/heads/branch-1.1 a09228549 -> d13d253fe SPARK-2711. Create a ShuffleMemoryManager to track memory for all spilling collections This tracks memory properly if there are multiple spilling collections in the same task (which was a problem before), and al

git commit: SPARK-2685. Update ExternalAppendOnlyMap to avoid buffer.remove()

2014-08-04 Thread matei
Repository: spark Updated Branches: refs/heads/master 05bf4e4af -> 066765d60 SPARK-2685. Update ExternalAppendOnlyMap to avoid buffer.remove() Replaces this with an O(1) operation that does not have to shift over the whole tail of the array into the gap produced by the element removed. Author

git commit: SPARK-2685. Update ExternalAppendOnlyMap to avoid buffer.remove()

2014-08-04 Thread matei
Repository: spark Updated Branches: refs/heads/branch-1.1 4ed7b5a2f -> a09228549 SPARK-2685. Update ExternalAppendOnlyMap to avoid buffer.remove() Replaces this with an O(1) operation that does not have to shift over the whole tail of the array into the gap produced by the element removed. Au

git commit: [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext

2014-08-04 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 3fbe3447e -> f536caa1b [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext Author: Reynold Xin Closes #1772 from rxin/accumulator-dagscheduler and squashes the following commits: 6a58520 [Reyno

git commit: [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext

2014-08-04 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.1 2225d18a7 -> 4ed7b5a2f [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext Author: Reynold Xin Closes #1772 from rxin/accumulator-dagscheduler and squashes the following commits: 6a58520 [Reyno

git commit: [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext

2014-08-04 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9fd82dbbc -> 05bf4e4af [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext Author: Reynold Xin Closes #1772 from rxin/accumulator-dagscheduler and squashes the following commits: 6a58520 [Reynold X

svn commit: r6047 - /dev/spark/spark-1.0.2-rc1/ /release/spark/spark-1.0.2/

2014-08-04 Thread tdas
Author: tdas Date: Tue Aug 5 01:53:43 2014 New Revision: 6047 Log: Moving Spark 1.0.2 from dev to release. Added: release/spark/spark-1.0.2/ - copied from r6046, dev/spark/spark-1.0.2-rc1/ Removed: dev/spark/spark-1.0.2-rc1/ ---

git commit: Updating version for Spark 1.0.3 development.

2014-08-04 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.0 439cdc581 -> 3fbe3447e Updating version for Spark 1.0.3 development. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3fbe3447 Tree: http://git-wip-us.apache.org/rep

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.2-rc1 [deleted] 913e3de6d - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc1 [deleted] c8d0eb980 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v0.9.1-rc2 [deleted] 95d70498b - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v0.9.1-rc3 [deleted] 931ac8a0e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v0.9.1-rc1 [deleted] 20a2d247a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v0.9.2-rc1 [deleted] 5a3020615 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.1-rc2 [deleted] 9c4493512 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.1-rc1 [deleted] 2a9114542 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc10 [deleted] eafc6f6f4 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc9 [deleted] 093aeb40f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc5 [deleted] 04ce12153 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc8 [deleted] 9abe0192a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc3 [deleted] e7c46933a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc7 [deleted] bc81ecfdc - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc6 [deleted] aab03f5f9 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v1.0.0-rc2 [deleted] 327ab1e24 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2014-08-04 Thread tdas
Repository: spark Updated Tags: refs/tags/v0.7.0-bizo-1 [created] 02b4a0d96 refs/tags/v0.9.1-rc1 [created] 20a2d247a refs/tags/v0.9.1-rc2 [created] 95d70498b refs/tags/v0.9.1-rc3 [created] 931ac8a0e refs/tags/v1.0.0-rc1 [created] c8d0eb980 refs/tags/v1.0.0-rc10 [created] eafc6f6f4 refs

svn commit: r6045 - /dev/spark/spark-1.0.2-rc1/

2014-08-04 Thread tdas
Author: tdas Date: Tue Aug 5 00:19:19 2014 New Revision: 6045 Log: Adding spark-1.0.2-rc1 Added: dev/spark/spark-1.0.2-rc1/ dev/spark/spark-1.0.2-rc1/spark-1.0.2-bin-cdh4.tgz (with props) dev/spark/spark-1.0.2-rc1/spark-1.0.2-bin-cdh4.tgz.asc dev/spark/spark-1.0.2-rc1/spark-1.0

git commit: [SPARK-1687] [PySpark] fix unit tests related to pickable namedtuple

2014-08-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 aa7a48ee9 -> 2225d18a7 [SPARK-1687] [PySpark] fix unit tests related to pickable namedtuple serializer is imported multiple times during doctests, so it's better to make _hijack_namedtuple() safe to be called multiple times. Author: D

git commit: [SPARK-1687] [PySpark] fix unit tests related to pickable namedtuple

2014-08-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 8e7d5ba1a -> 9fd82dbbc [SPARK-1687] [PySpark] fix unit tests related to pickable namedtuple serializer is imported multiple times during doctests, so it's better to make _hijack_namedtuple() safe to be called multiple times. Author: Davie

git commit: SPARK-2792. Fix reading too much or too little data from each stream in ExternalMap / Sorter

2014-08-04 Thread matei
Repository: spark Updated Branches: refs/heads/branch-1.1 bfd2f3958 -> aa7a48ee9 SPARK-2792. Fix reading too much or too little data from each stream in ExternalMap / Sorter All these changes are from mridulm's work in #1609, but extracted here to fix this specific issue and make it easier t

git commit: SPARK-2792. Fix reading too much or too little data from each stream in ExternalMap / Sorter

2014-08-04 Thread matei
Repository: spark Updated Branches: refs/heads/master 59f84a953 -> 8e7d5ba1a SPARK-2792. Fix reading too much or too little data from each stream in ExternalMap / Sorter All these changes are from mridulm's work in #1609, but extracted here to fix this specific issue and make it easier to me

git commit: [SPARK-1687] [PySpark] pickable namedtuple

2014-08-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 3823f6d25 -> bfd2f3958 [SPARK-1687] [PySpark] pickable namedtuple Add an hook to replace original namedtuple with an pickable one, then namedtuple could be used in RDDs. PS: pyspark should be import BEFORE "from collections import nam

git commit: [SPARK-1687] [PySpark] pickable namedtuple

2014-08-04 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master e053c5581 -> 59f84a953 [SPARK-1687] [PySpark] pickable namedtuple Add an hook to replace original namedtuple with an pickable one, then namedtuple could be used in RDDs. PS: pyspark should be import BEFORE "from collections import namedtu