> On Nov. 19, 2019, 6:12 p.m., Andrei Sekretenko wrote: > > src/master/http.cpp > > Lines 2242 (patched) > > <https://reviews.apache.org/r/71750/diff/1/?file=2172061#file2172061line2242> > > > > Did you consider allocating the `agent` temporaries outside of the > > loops? In my experience, protobuf construction used to be much more > > expensive than modification. > > > > (Tried this, observed a ~10% speedup for `v1 'master::call::GetState' > > application/x-protobuf` in my setup.)
Hm.. that's interesting, could certainly look into this for the GetAgents and GetFrameworks code which both use temporaries. The temporaries aren't necessary though, we could eliminate the need to copy entirely, but allocating them outside the loops seems like an easier win. Unfortunately the most expensive part in practice (non-pending tasks) are not using temporaries like this. I'll look into a patch at the end of this chain to allocate outside the loops. > On Nov. 19, 2019, 6:12 p.m., Andrei Sekretenko wrote: > > src/master/http.cpp > > Lines 2257-2263 (patched) > > <https://reviews.apache.org/r/71750/diff/1/?file=2172061#file2172061line2257> > > > > There is a certain similarity between serializeGetAgents and > > jsonifyGetAgents which probably can be factored out... > > > > However, it might make more sense to leave this as is for now, and > > refactor it after we get to getTasks/Executors/Frameworks. Yeah, the same is true for the other serialization paths which I just published. I debated refactoring in front or behind, and I think I'll go with behind, so that these patches just duplicate the common parts for now. - Benjamin ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71750/#review218678 ----------------------------------------------------------- On Nov. 12, 2019, 12:03 a.m., Benjamin Mahler wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71750/ > ----------------------------------------------------------- > > (Updated Nov. 12, 2019, 12:03 a.m.) > > > Review request for mesos, Andrei Sekretenko, Greg Mann, and Meng Zhu. > > > Bugs: MESOS-10026 > https://issues.apache.org/jira/browse/MESOS-10026 > > > Repository: mesos > > > Description > ------- > > This updates the handling to serialize directly to protobuf or json > from the in-memory v0 state, bypassing expensive intermediate > serialization / de-serialization / object construction / object > destruction. > > This initial patch shows the approach that will be used for the > other expensive calls. Note that this type of manual writing is > more brittle and complex, but it can be mostly eliminated if we > keep an up-to-date v1 GetState in memory in the future. > > When this approach is applied fully to GetState, it leads to the > following improvement: > > Before: > v0 '/state' response took 6.55 secs > v1 'GetState' application/x-protobuf response took 24.08 secs > v1 'GetState' application/json response took 22.76 secs > > After: > v0 '/state' response took 8.00 secs > v1 'GetState' application/x-protobuf response took 5.73 secs > v1 'GetState' application/json response took 9.62 secs > > > Diffs > ----- > > src/master/http.cpp 60765c9b9d6903f6ed94fa8c614055698caad0da > src/master/master.hpp dc45028d2ecfb61bf9ea82d90d2393af648a6023 > > > Diff: https://reviews.apache.org/r/71750/diff/1/ > > > Testing > ------- > > make check > > > Thanks, > > Benjamin Mahler > >
