[Sorry for delay: head down in linker plus having nice food poisoning bouts]
On 9 Jul 2019, Mike Stump verbalised: > On Jul 5, 2019, at 11:28 AM, Nix <n...@esperi.org.uk> wrote: >> ICTF for the entire Linux kernel is about 6MiB > > Any reason why not add CTF to the next dwarf standard? Then, we just > support the next dwarf standard. If not, have you started talks with > them to add it? A mixture of impostor syndrome, the fact that CTF is really very non-DWARFish in all sorts of ways, and the fact that CTF-the-format is changing quite fast right now means that... well, if it is to be added, now is not the time. I haven't even documented it in texi yet :) (Just suggestions for improvement I've had on the binutils list will lead to a good few changes :) ). Right now, the rule for compatibility is that libctf will always be able to read all earlier versions written by any released binutils or libdtrace-ctf, and rewrite them as the latest version -- and one improvement I have planned is that it will eventually be able to *write* older versions as well, as long as doing so doesn't lose information or run into limitations of the older format (like trying to write >2^16 types to a format v1 container, or add an enum bitfield to a v2 container). I'm doing this in the obvious fashion: every time the format written by binutils libctf changes, it keeps the ability to upgrade all older CTF formats any release of binutils ever accepted to the latest format. Every binutils release after such a change constitutes a boundary: the next format change after that will bump the CTF format version, and the just-released format will be upgraded to be compatible with any new stuff that gets added. If CTF generation support lands in GCC, I'll treat compiler releases the same way, nailing the format any released GCC emits into binutils libctf at release time and ensuring binutils libctf can always accept it (and thus binutils ld can always link it and gdb can always use it). (I do not plan to ever drop support for any older CTF formats: indeed I plan to extend it so that the FreeBSD/Solaris CTF can be read as well, and hopefully eventually written too.) This should suffice to ensure that the CTF emitted by any released compiler and any released binutils can always be accepted by newer releases, and is probably the right approach until format evolution slows and we can start to actually standardize this. > Long term, this is a better solution, as we then get more > interoperability, more support, more tools and more goodness. Agreed! I do hope libctf remains flexible and useful enough that everyone can use it as a "format oracle", but I would welcome other implementations: the more the merrier! (It's just that now might be too early and too annoying for the other implementors, since the format is evolving faster than it ever has, thanks to all the lovely suggestions on the binutils list). If libctf *does* gain the ability to downgrade as well as upgrade formats, we can keep evolving the format even after standardization, with libctf translating the standardized version to newer versions and back down again as needed, restandardizing at intervals so the other tools can catch up: this seems like a fairly strong reason to gain the ability to write out old versions as well as new ones. (But I'm getting way ahead of myself here: the internal intermediate representation inside libctf that will make this sort of format ubiquity possible only exists inside my head right now, after all.)
