I have been thinking about this for a little while and I wonder if it makes sense to look at forcing off heap mmap storage what can be shared with python.
The idea would be that java makes a DirectByteBuffer (or similar) with python doing memoryview over that buffer. Then for all except for real objects everything could be written in a numpy friendly format into the mmap ? -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Interested-in-contributing-to-GraphX-in-Python-tp7629p9813.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
