Updated Branches:
refs/heads/master 79c95527a -> 084839ba3
Merge pull request #498 from ScrapCodes/python-api. Closes #498.
Python api additions
Author: Prashant Sharma
== Merge branch commits ==
commit 8b51591f1a7a79a62c13ee66ff8d83040f7eccd8
Author: Prashant Sharma
Date: Fri Jan 24 11
Merge pull request #523 from JoshRosen/SPARK-1043
Switch from MUTF8 to UTF8 in PySpark serializers.
This fixes SPARK-1043, a bug introduced in 0.9.0 where PySpark couldn't
serialize strings > 64kB.
This fix was written by @tyro89 and @bouk in #512. This commit squashes and
rebases th
Updated Branches:
refs/heads/master 84670f271 -> f8c742ce2
Switch from MUTF8 to UTF8 in PySpark serializers.
This fixes SPARK-1043, a bug introduced in 0.9.0
where PySpark couldn't serialize strings > 64kB.
This fix was written by @tyro89 and @bouk in #512.
This commit squashes and rebases th
Modifications as suggested in PR feedback-
- mapPartitions, foreachPartition moved to JavaRDDLike
- call scala rdd's setGenerator instead of setting directly in JavaRDD
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-s
fixed ClassTag in mapPartitions
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/60e74572
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/60e74572
Diff: http://git-wip-us.apache.org/repos/a
Merge pull request #406 from eklavya/master
Extending Java API coverage
Hi,
I have added three new methods to JavaRDD.
Please review and merge.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/fad6aacf
T
Modifications as suggested in PR feedback-
- more variants of mapPartitions added to JavaRDDLike
- move setGenerator to JavaRDDLike
- clean up
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/1442cd5d
Tree:
Updated Branches:
refs/heads/master a2b47dae6 -> fad6aacfb
Added setter method setGenerator to JavaRDD.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/aae8a014
Tree: http://git-wip-us.apache.org/repos/
Remove default param from mapPartitions
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/fa42951e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/fa42951e
Diff: http://git-wip-us.apache.org
Remove classtag from mapPartitions.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/8fe562c0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/8fe562c0
Diff: http://git-wip-us.apache.org/rep
Added mapPartitions method to JavaRDD.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/dbadc6b9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/dbadc6b9
Diff: http://git-wip-us.apache.org/
Added foreachPartition method to JavaRDD.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/6a65feeb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/6a65feeb
Diff: http://git-wip-us.apache.o
Merge pull request #256 from MLnick/master
Fix 'IPYTHON=1 ./pyspark' throwing ValueError
This fixes an annoying issue where running ```IPYTHON=1 ./pyspark``` resulted
in:
```
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__ / .__/\_,_/_/ /_/
Updated Branches:
refs/heads/master c55e69855 -> d2ced6d58
Fix 'IPYTHON=1 ./pyspark' throwing 'ValueError: Cannot run multiple
SparkContexts at once'
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/8cd
Making IPython PySpark compatible across versions <1.0.0. Also cleaned up '-i'
option and made IPYTHON_OPTS work
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/bb5277b1
Tree: http://git-wip-us.apache.org/
Merge remote-tracking branch 'upstream/master'
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/d36ee3b1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/d36ee3b1
Diff: http://git-wip-us.apa
Leave default value of numPartitions to Scala code.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/e85af507
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/e85af507
Diff: http://git-wip-u
Use braces to shorten the line.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/c82d4f07
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/c82d4f07
Diff: http://git-wip-us.apache.org/repos/a
Hook directly to Scala API
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/0b494f7d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/0b494f7d
Diff: http://git-wip-us.apache.org/repos/asf/in
Updated Branches:
refs/heads/master 97ac06018 -> 2fd781d34
Expose numPartitions parameter in JavaPairRDD.sortByKey()
This change make Java and Scala API on sortByKey() the same.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/as
Merge pull request #249 from ngbinh/partitionInJavaSortByKey
Expose numPartitions parameter in JavaPairRDD.sortByKey()
This change makes Java and Scala API on sortByKey() the same.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/as
21 matches
Mail list logo