[no subject]

2015-10-15 Thread Lei Wu
Dear all, Like the design doc in SPARK-1 for Spark memory management, is there a design doc for Spark task scheduling details ? I'd really like to dive deep into the task scheduling module of Spark, thanks so much !

Design doc for Spark task scheduling

2015-10-15 Thread Lei Wu
Like the design doc in SPARK-1 for Spark memory management, is there a design doc for Spark task scheduling details ? I'd really like to dive deep into the task scheduling module of Spark, thanks so much ! Forgot the email title in previous mail, sorry for that.

TaskMemoryManager. cleanUpAllAllocatedMemory -> Memory leaks ???

2015-10-12 Thread Lei Wu
Dear all, I'm reading source code of TaskMemoryManager.java and I got stuck in the last function, that is cleanUpAllAllocatedMemory. What confuses me is the comments for this function : "A non-zero return value can be used to detect memory leaks". And from Executor.scala where this function is