Do you have history server enabled ? Posting your code snippet would help us understand your use case (and reproduce the leak).
Thanks On Thu, Mar 24, 2016 at 6:40 AM, Michel Hubert <mich...@phact.nl> wrote: > <dependencies> > <dependency> <!-- Spark dependency --> > <groupId>org.apache.spark</groupId> > <artifactId>spark-core_2.10</artifactId> > <version>1.6.1</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming_2.10</artifactId> > <version>1.6.1</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming-kafka_2.10</artifactId> > <version>1.6.1</version> > </dependency> > > <dependency> > <groupId>org.elasticsearch</groupId> > <artifactId>elasticsearch</artifactId> > <version>2.2.0</version> > </dependency> > > <dependency> > <groupId>org.apache.kafka</groupId> > <artifactId>kafka_2.10</artifactId> > <version>0.8.2.2</version> > </dependency> > > > <dependency> > <groupId>org.elasticsearch</groupId> > <artifactId>elasticsearch-spark_2.10</artifactId> > <version>2.2.0</version> > </dependency> > <dependency> > <groupId>redis.clients</groupId> > <artifactId>jedis</artifactId> > <version>2.8.0</version> > <type>jar</type> > <scope>compile</scope> > </dependency> > </dependencies> > > > > > > How can I look at those tasks? > > > > *Van:* Ted Yu [mailto:yuzhih...@gmail.com] > *Verzonden:* donderdag 24 maart 2016 14:33 > *Aan:* Michel Hubert <mich...@phact.nl> > *CC:* user@spark.apache.org > *Onderwerp:* Re: apache spark errors > > > > Which release of Spark are you using ? > > > > Have you looked the tasks whose Ids were printed to see if there was more > clue ? > > > > Thanks > > > > On Thu, Mar 24, 2016 at 6:12 AM, Michel Hubert <mich...@phact.nl> wrote: > > HI, > > > > I constantly get these errors: > > > > 0 [Executor task launch worker-15] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 6564500 bytes, TID = 38969 > > 310002 [Executor task launch worker-12] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5523550 bytes, TID = 43270 > > 318445 [Executor task launch worker-12] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 6879566 bytes, TID = 43408 > > 388893 [Executor task launch worker-12] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5572546 bytes, TID = 44382 > > 418186 [Executor task launch worker-13] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5289222 bytes, TID = 44795 > > 488421 [Executor task launch worker-4] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 8738142 bytes, TID = 45769 > > 619276 [Executor task launch worker-4] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5759312 bytes, TID = 47571 > > 632275 [Executor task launch worker-12] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5602240 bytes, TID = 47709 > > 644989 [Executor task launch worker-13] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5326260 bytes, TID = 47863 > > 720701 [Executor task launch worker-12] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5399578 bytes, TID = 48959 > > 1147961 [Executor task launch worker-16] ERROR > org.apache.spark.executor.Executor - Managed memory leak detected; size = > 5251872 bytes, TID = 54922 > > > > > > How can I fix this? > > > > With kind regard, > > > > Michel > > >
