On 13-08-21 07:47 PM, Corey Richardson wrote: > - Metadata is large. It is multiple megabytes of data (uncompressed. > compressed as of now it is 749K) for libstd. I'm not sure whether we > are encoding too much data or if it's exactly what we need, but this > is a very large constant that every inefficiency gets scaled by.
Only because we read all of it. Which is a bug. https://github.com/mozilla/rust/issues/4572 > - Metadata is stored as EBML, always. I'm sure EBML is fine most of > the time, but it really hurts us here. Part of the problem is that it > is a big endian format. The byte swapping shows up very high (top 5) > in a profile. Additionally, *every time we query the stored metadata > for something, we decode the EBML*. This turns EBML into a > multiplicative slowdown, rather than just an additive if we decoded > the entire structure up front and used the native-rust representation. We cache a lot of the results. > - Metadata is stored in a single blob. To access any part of metadata, > we need to load it all, and there is no index. If we could only load > the metadata we need, we'd reduce memory usage and do less wasted > work. There are indexes. But our access patterns are buggy and we read the whole thing even if we only need a little bit. We're going to stop doing this. Patrick is working on it. It's that bug, above. > - Metadata is scary. It's a hairy part of the codebase that I sure > don't understand. I know its purpose, more or less, but not the > specifics of how or why things are encoded. Michael Sullivan could > speak more to this, he is the last one to have touched it. The > compiler can't help you when you make a mistake, either. I contains indexes of particularly salient extracts from the items in the crate, as well as a general module / item tree, their types, and the serialized ASTs of those items that you might want to inline. -Graydon _______________________________________________ Rust-dev mailing list [email protected] https://mail.mozilla.org/listinfo/rust-dev
