Master stores a cache of completed tasks. Currently the cache capacity (MAX_COMPLETED_TASKS_PER_FRAMEWORK) is set to 1000 though we could make it could make it configurable.
On Wed, May 21, 2014 at 2:18 PM, Sharma Podila <spod...@netflix.com> wrote: > I see that master/state.json has state information on frameworks, where > in, it has a list of all completed_tasks. Each task seems to be about 500 > bytes. > > Does the master have a list of all completed tasks for the framework? > Thinking naively about it, does it mean that if I were to run, say, 100K > tasks a day, we have 50MBytes of data in there? In 3 weeks that's a GB. > Which by itself maybe OK, but not if it grows indefinitely. > > Is that a cause for concern? Or, is that an incorrect extrapolation? Is > there some kind of purging that happens on tasks that completed a while ago? > > Sharma > >
