Thanks for the input, folks. Patch is up here: https://reviews.apache.org/r/42896/
On Wed, Jan 27, 2016 at 8:09 PM, Maxim Khutornenko <ma...@apache.org> wrote: > You're right, ignore me. I guess I was mourning the loss of StorageBackfill > too hard :) Obviously, we don't have to force this change over existing > tasks and let them die out naturally. Some user scraping tools may be > broken due to this but we always advised against taking a dependency on our > task ID format. > > +1 to the proposal. > > On Wed, Jan 27, 2016 at 5:46 PM, Bill Farner <wfar...@apache.org> wrote: > > > I don't believe backwards compatibility is an issue here. This would be > an > > alteration to generation of new task IDs. AFAIK we don't do anything > that > > requires comprehension or synthesis of previously-generated task IDs. > > > > On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <ma...@apache.org> > > wrote: > > > > > What's the cluster upgrade story going to look like? Since task IDs are > > > used as unique identifiers for Mesos, I expect this change would > require > > > rebooting the entire cluster to the new format? I am not really sure > how > > > this can be done in a graceful manner without sending the entire > cluster > > to > > > LOST on first reconciliation run. > > > > > > Unless we have answers to the above, I am -1 to this proposal. The > > benefits > > > don't seem significant enough to offset the pain of migrating to the > new > > > format. > > > > > > On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia < > > > mauriciogaravag...@gmail.com> wrote: > > > > > > > +1 to dropping the timestamp. > > > > > > > > Agree that having the jobkey at hand has been helpful for debugging. > > > > > > > > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <zma...@apache.org> > > > wrote: > > > > > > > > > +1 to removing the timestamp. > > > > > > > > > > The timestamp has not provided me with any benefit as an operator. > > The > > > > > mangled jobkey and UUID have been very useful in grepping logs and > > > > > diagnosing failing jobs. > > > > > > > > > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <zhitaoli...@gmail.com> > > > > wrote: > > > > > > > > > > > +1 for dropping the time and keeping the mangled jobkey. Unless > we > > > are > > > > > sure > > > > > > that all internal logging of Mesos master and agent contains an > > > > > identifier > > > > > > with user some user generated data, changing it to UUID will make > > > adhoc > > > > > > debugging through Mesos logging harder. > > > > > > > > > > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan < > > > > > stephan....@blue-yonder.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > +1 for dropping the timestamp > > > > > > > > > > > > > > However, I am not sure regarding the mangled jobkey. It tends > to > > > make > > > > > it > > > > > > > easier to correlate Mesos tasks to Aurora jobs when skimming > log > > > > files, > > > > > > > viewing the Mesos-UI or even when using the Thermos [1]. I > guess > > > the > > > > > > > traceability of all of those usecases could be improved, but > that > > > > would > > > > > > > probably additional work. > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > https://github.com/apache/aurora/blob/master/docs/images/runningtask.png > > > > > > > ________________________________________ > > > > > > > From: Bill Farner <wfar...@apache.org> > > > > > > > Sent: Wednesday, January 27, 2016 12:03 AM > > > > > > > To: dev@aurora.apache.org > > > > > > > Subject: [PROPOSAL] Revisit task ID format > > > > > > > > > > > > > > Context: a task ID is a unique identifier for a task. Aurora > and > > > > Mesos > > > > > > > both require this uniqueness. Within mesos, frameworks are > > > required > > > > to > > > > > > > craft their own task IDs as they see fit. > > > > > > > > > > > > > > Our task ID format is currently [1] > > > > > > > > > > > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID > > > > > > > > > > > > > > > > > > > > > for an example: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533 > > > > > > > > > > > > > > > > > > > > > In addition to guaranteed uniqueness, this format has the > benefit > > > of > > > > > > being > > > > > > > somewhat human-friendly. Within logs, it is somewhat possible > to > > > > > > partially > > > > > > > recognize a task based solely on the text ID. > > > > > > > > > > > > > > *I would like to propose that we remove the TIMESTAMP- prefix > > from > > > > the > > > > > > task > > > > > > > ID.* It was originally included so that task IDs would be > > > temporally > > > > > > > sortable for scheduling prioritization. At present, tasks are > > not > > > > > sorted > > > > > > > using the ID. > > > > > > > > > > > > > > While proposing the above, i think it's also prudent to take > the > > > > > > > opportunity to consider a complete overhaul of the ID contents. > > > *An > > > > > > > alternative approach would be to only use the UUID.* This has > > the > > > > > > benefit > > > > > > > of decoupling arbitrary user input from the various ways task > IDs > > > are > > > > > > used > > > > > > > (as an example - mesos uses them in file names, limiting length > > and > > > > > > allowed > > > > > > > characters). Task IDs also become fixed width, which offers a > > > (very) > > > > > > > marginal memory reduction over the status quo, and makes > console > > > line > > > > > > > wrapping more consistent when perusing logs. Additionally, it > > > > eschews > > > > > > the > > > > > > > potential problem of users parsing task IDs and coupling to its > > > > format. > > > > > > > > > > > > > > Any thoughts on this? > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Cheers, > > > > > > > > > > > > Zhitao Li > > > > > > > > > > > > -- > > > > > > Zameer Manji > > > > > > > > > > > > > > > > > > > > > > > > > > >
