Re: Managed memory leak : spark-2.0.2

2016-12-08 Thread Appu K
Hi, I didn’t hit any oom issues. thanks for the pointer. i guess it’ll be safe to ignore since TaskMemoryManager automatically releases just wondering what would have been the cause in this case - couldn’t see any task failures in the log but some reference to ExternalAppendOnlyMap acquiring

Re: Managed memory leak : spark-2.0.2

2016-12-08 Thread Takeshi Yamamuro
Hi, Did you hit some troubles from the memory leak? I think we can ignore the message in most cases because TaskMemoryManager automatically releases the memory. In fact, spark degraded the message in SPARK-18557. https://issues.apache.org/jira/browse/SPARK-18557 // maropu On Thu, Dec 8, 2016 at

Managed memory leak : spark-2.0.2

2016-12-08 Thread Appu K
Hello, I’ve just ran into an issue where the job is giving me "Managed memory leak" with spark version 2.0.2 — 2016-12-08 16:31:25,231 [Executor task launch worker-0] (TaskMemoryManager.java:381) WARN leak 46.2 MB memory from