> On Oct. 17, 2013, 5:32 p.m., Ben Mahler wrote: > > src/slave/slave.hpp, lines 265-269 > > <https://reviews.apache.org/r/14663/diff/1/?file=364985#file364985line265> > > > > What if the GarbageCollector always dealt with the mtimes instead? > > > > Asking the GarbageCollector to schedule something after 2 weeks could > > mean: > > 1. If 2 weeks elapse (without restarting the process): GC the path. > > 2. If the mtime becomes 2 weeks ago: GC the path. > > > > We currently do 1 but neither 1 or 2 alone is robust for all types of > > GC operations. Doing both 1 and 2 together ensures we don't make the GC > > behavior dependent on the whether the slave stays up for long enough to GC. > > > > This would require us, before scheduling for GC, to sometimes 'touch' > > the path and then schedule it for GC. > > > > In the current code (without the GarbageCollector performing 2 > > internally), it seems we can have situations (like slaves continually > > getting restarted before the GC period) in which we can induce huge GC > > operations driven by disk usage rather than mtimes. Using mtimes internally > > in the GarbageCollector would help to consistently space out the GC > > operations so that we have less disk I/O impact on executors / tasks that > > use the disk.
I would like to keep the GarbageCollector simple. All it does is schedule something in the future or unschedule it. It is the same reason why we don't let GC deal with disk usage. That said, I'm not sure what the benefits of GC dealing with mtimes is. In this diff, the slave does both 1 and 2! - Vinod ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14663/#review27134 ----------------------------------------------------------- On Oct. 16, 2013, 12:41 a.m., Vinod Kone wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14663/ > ----------------------------------------------------------- > > (Updated Oct. 16, 2013, 12:41 a.m.) > > > Review request for mesos, Benjamin Hindman and Ben Mahler. > > > Bugs: MESOS-742 > https://issues.apache.org/jira/browse/MESOS-742 > > > Repository: mesos-git > > > Description > ------- > > Old recovered directories are now gc'ed based on modification time. > > > Diffs > ----- > > src/slave/slave.hpp 22fb74b71a0f52d9d67b92ecc286fa8d350e41a4 > src/slave/slave.cpp debb2f4ce05fbfec450197e68bc8a0c78f1d0adf > > Diff: https://reviews.apache.org/r/14663/diff/ > > > Testing > ------- > > make check > > > Thanks, > > Vinod Kone > >
