This is the size of the serialized task closure. Is stage 246 part of ALS iterations, or something before or after it? -Xiangrui
On Tue, Apr 21, 2015 at 10:36 AM, Christian S. Perone <christian.per...@gmail.com> wrote: > Hi Sean, thanks for the answer. I tried to call repartition() on the input > with many different sizes and it still continues to show that warning > message. > > On Tue, Apr 21, 2015 at 7:05 AM, Sean Owen <so...@cloudera.com> wrote: >> >> I think maybe you need more partitions in your input, which might make >> for smaller tasks? >> >> On Tue, Apr 21, 2015 at 2:56 AM, Christian S. Perone >> <christian.per...@gmail.com> wrote: >> > I keep seeing these warnings when using trainImplicit: >> > >> > WARN TaskSetManager: Stage 246 contains a task of very large size (208 >> > KB). >> > The maximum recommended task size is 100 KB. >> > >> > And then the task size starts to increase. Is this a known issue ? >> > >> > Thanks ! >> > >> > -- >> > Blog | Github | Twitter >> > "Forgive, O Lord, my little jokes on Thee, and I'll forgive Thy great >> > big >> > joke on me." > > > > > -- > Blog | Github | Twitter > "Forgive, O Lord, my little jokes on Thee, and I'll forgive Thy great big > joke on me." --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
