On Donnerstag, 21. Januar 2016 22:03:23 CET Daniel Pfeifer wrote: > On Wed, Jan 20, 2016 at 10:03 PM, Stephen Kelly <steve...@gmail.com> wrote: > > Milian Wolff wrote: > >>> I'm concerned that the memory usage of a daemon implementing the > >>> proposed > >>> capabilities may be too large to be practical (at least without a major > >>> redesign of certain structures that tend to duplicate substrings, or > >>> some kind of out-of-core approach). > >> > >> This sounds like optimizations to that daemon will benefit CMake. > > > > FYI I merged a reduce-allocations branch to next for testing. It came from > > > > Milian here: > > https://github.com/steveire/CMake/pull/1 > > Very nice! It is impressive what kind of an impact a simple 'const&' > can have. Also, nice work identifying the hotspots. > > > Milian also mentioned the possibility of using something like a sqlite > > database (probably what you meant by out-of-core above) for definitions > > for > > querying by the daemon. > > > > I also mentioned some possibile optimization possibilities, such as > > removing the closure of definitions created for all directories. I wrote > > a branch which does that some months ago, but it resulted in a slow down. > > I'll see if I can rebase the commit to master and push it to github, > > together with a patch for avoiding computing the hash multiple times in > > cmDefinitions. > > What do you think about string interning? I started a cmString class > [2] that stores short strings inplace and puts long strings into a > pool. Copies never allocate extra memory and equality comparison is > always O(1). > In addition to `operator<` (which is lexicographical) there is a O(1) > comparison that can be used where lexicographical ordering is not > required (ie. for lookup tables). > > [1] https://en.wikipedia.org/wiki/String_interning > [2] https://github.com/purpleKarrot/CMake/commits/string-pool
Imo, you should replace this custom code by a boost::flyweight of std::string. That said, I think this can have a significant impact on the memory footprint of CMake, considering how heavy it relies on strings internally. But it also seems to mutate strings a lot. I've seen places e.g. where a list of compile- time known identifiers is prepended with "CMAKE_" at runtime. This is slow with normal strings (reallocations), but will also be slow with a flyweight or interning, possibly even leading to the pollution of the internal pool with temporary strings. Did you measure any impact on both, runtime speed and memory footprint yet? -- Milian Wolff m...@milianw.de http://milianw.de
signature.asc
Description: This is a digitally signed message part.
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
