Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
clayborg requested changes to this revision. clayborg added a comment. This revision now requires changes to proceed. One minor IsValid() fix in DWARFDIE to avoid crashes and this is good to go. Comment at: source/Plugins/SymbolFile/DWARF/DWARFDIE.cpp:28-35 @@ -27,1 +27,10 @@ +DIERef +DWARFDIE::GetDIERef() const +{ +dw_offset_t cu_offset = m_cu->GetOffset(); +if (m_cu->GetBaseObjOffset() != DW_INVALID_OFFSET) +cu_offset = m_cu->GetBaseObjOffset(); +return DIERef(cu_offset, m_die->GetOffset()); +} + Need to check IsValid() first and place all above code inside the if and have the else return a DIERef(). http://reviews.llvm.org/D12291 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
tberghammer added inline comments. Comment at: source/Plugins/SymbolFile/DWARF/DIERef.cpp:34-44 @@ +33,13 @@ + +DIERef::DIERef(const DWARFFormValue& form_value) : +cu_offset(DW_INVALID_OFFSET), +die_offset(DW_INVALID_OFFSET) +{ +if (form_value.IsValid()) +{ +if (form_value.GetCompileUnit()) +cu_offset = form_value.GetCompileUnit()->GetOffset(); +die_offset = form_value.Reference(); +} +} + clayborg wrote: > For DWO files, won't every compile unit have offset 0x000b? Or do you use > the actual compile unit DIE from the main executable? > > I have doubts this will work for DWARF in .o files. We used to encode the > compile unit index in the upper 32 bits because all of our compile units have > a dw_offset_t that is 0xb.. The concept of DIERef is to contain the offset of the compile unit in the main object file so based on a DIERef and a SymbolFileDWARF fro the main object file we can find the compile unit and the DIE. In case of dsym it changes a bit with storing the symbol file index in the cu_offset filed instead. I fixed the implementation to set it up properly in case of dwo files Comment at: source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp:1025-1036 @@ +1024,14 @@ +{ +if (GetAttributeValue(dwarf2Data, cu, DW_AT_specification, form_value) || +GetAttributeValue(dwarf2Data, cu, DW_AT_abstract_origin, form_value)) +{ +DWARFDIE die = const_cast(cu)->GetDIE(form_value.Reference()); +if (die) +return die.GetDIE()->GetAttributeValue(die.GetDWARF(), + die.GetCU(), + attr, + form_value, + end_attr_offset_ptr, + false); +} +} clayborg wrote: > This will fail if a DIE can have both a DW_AT_specification and a > DW_AT_abstract_origin. Not sure if that can happen. If it can, we will need > to check each one individually. Probably best to just check both individually. We haven't checked both of it previously either and I don't think we will ever have both a DW_AT_specification and a DW_AT_abstract_origin, but changed it to check both for just in case. Comment at: source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp:1039-1055 @@ -985,1 +1038,19 @@ + +if (!dwo_symbol_file) +return 0; + +DWARFCompileUnit* dwo_cu = dwo_symbol_file->GetCompileUnit(); +if (!dwo_cu) +return 0; + +DWARFDIE dwo_cu_die = dwo_cu->GetCompileUnitDIEOnly(); +if (!dwo_cu_die.IsValid()) +return 0; + +return dwo_cu_die.GetDIE()->GetAttributeValue(dwo_symbol_file, + dwo_cu, + attr, + form_value, + end_attr_offset_ptr, + check_specification_or_abstract_origin); } clayborg wrote: > Don't you want to only do all of this code if: > ``` > if (Tag() == DW_TAG_compile_unit) > ``` Yes I do. The beginning of the function (line 987-995) handles the case for non DW_TAG_compile_unit with forwarding the query to the right compile unit and this part of the code is to handle the case when some data about the compile unit is in the main object file and some data is in the dwo file. This can't happen for any other DIE. http://reviews.llvm.org/D12291 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
clayborg requested changes to this revision. clayborg added a comment. This revision now requires changes to proceed. A few changes. I think this might create problems for DWARF in .o files on MacOSX. Are you able to test on MacOSX? We used to store the compile unit index in the upper 32 bits of the type ID and all of our compile units from each .o file have 0xb as their cu_offset, so I would think that DIERef would always cause us to grab the first compile unit... Comment at: source/Plugins/SymbolFile/DWARF/DIERef.cpp:34-44 @@ +33,13 @@ + +DIERef::DIERef(const DWARFFormValue& form_value) : +cu_offset(DW_INVALID_OFFSET), +die_offset(DW_INVALID_OFFSET) +{ +if (form_value.IsValid()) +{ +if (form_value.GetCompileUnit()) +cu_offset = form_value.GetCompileUnit()->GetOffset(); +die_offset = form_value.Reference(); +} +} + For DWO files, won't every compile unit have offset 0x000b? Or do you use the actual compile unit DIE from the main executable? I have doubts this will work for DWARF in .o files. We used to encode the compile unit index in the upper 32 bits because all of our compile units have a dw_offset_t that is 0xb.. Comment at: source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp:603-605 @@ -581,7 +602,5 @@ { -std::vector::const_iterator pos; -std::vector::const_iterator end = die_offsets.end(); -for (pos = die_offsets.begin(); pos != end; ++pos) +for (auto pos = die_refs.begin(), end = die_refs.end(); pos != end; ++pos) { -dw_offset_t die_offset = *pos; -if (die_offset != DW_INVALID_OFFSET) +const DIERef& die_ref = *pos; +if (die_ref.die_offset != DW_INVALID_OFFSET) These 3 lines would be nicer as: ``` for (const auto _ref : die_refs) { ``` Comment at: source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp:1025-1036 @@ +1024,14 @@ +{ +if (GetAttributeValue(dwarf2Data, cu, DW_AT_specification, form_value) || +GetAttributeValue(dwarf2Data, cu, DW_AT_abstract_origin, form_value)) +{ +DWARFDIE die = const_cast(cu)->GetDIE(form_value.Reference()); +if (die) +return die.GetDIE()->GetAttributeValue(die.GetDWARF(), + die.GetCU(), + attr, + form_value, + end_attr_offset_ptr, + false); +} +} This will fail if a DIE can have both a DW_AT_specification and a DW_AT_abstract_origin. Not sure if that can happen. If it can, we will need to check each one individually. Probably best to just check both individually. Comment at: source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp:1039-1055 @@ -985,1 +1038,19 @@ + +if (!dwo_symbol_file) +return 0; + +DWARFCompileUnit* dwo_cu = dwo_symbol_file->GetCompileUnit(); +if (!dwo_cu) +return 0; + +DWARFDIE dwo_cu_die = dwo_cu->GetCompileUnitDIEOnly(); +if (!dwo_cu_die.IsValid()) +return 0; + +return dwo_cu_die.GetDIE()->GetAttributeValue(dwo_symbol_file, + dwo_cu, + attr, + form_value, + end_attr_offset_ptr, + check_specification_or_abstract_origin); } Don't you want to only do all of this code if: ``` if (Tag() == DW_TAG_compile_unit) ``` http://reviews.llvm.org/D12291 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
tberghammer added inline comments. Comment at: source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.cpp:316-328 @@ +315,15 @@ + +const char* dwo_name = cu_die.GetAttributeValueAsString(m_dwarf2Data, +this, +DW_AT_GNU_dwo_name, +nullptr); +if (!dwo_name) +return; + +const char* comp_dir = cu_die.GetAttributeValueAsString(m_dwarf2Data, +this, +DW_AT_comp_dir, +nullptr); +if (!comp_dir) +return; + clayborg wrote: > Is DW_AT_GNU_dwo_name always a relative path? I haven't seen absolute path in it so far, but it can be absolute. Added code to handle it. Comment at: source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.cpp:660 @@ -564,3 +659,3 @@ { -return m_dwarf2Data->DebugInfo()->GetDIE (die_offset); +return m_dwarf2Data->DebugInfo()->GetDIE (DIERef(die_offset)); } clayborg wrote: > Don't we need the compile unit to be encoded into the DIERef here? No, we don't want to specify the compile unit here. We already checked that the specified DIE offset isn't belongs to the current compile unit. In this case we should try to get a DIE based on offset from the current symbol file. This can happen when a DIE refers to a value in a different compile unit inside the same file based on DIE offset Comment at: source/Plugins/SymbolFile/DWARF/DWARFDIE.cpp:161 @@ -145,3 +160,3 @@ else -return DWARFDIE(dwarf->DebugInfo()->GetCompileUnitContainingDIE(block_die->GetOffset()), block_die); +return DWARFDIE(dwarf->DebugInfo()->GetCompileUnitContainingDIE(DIERef(cu->GetOffset(), block_die->GetOffset())), block_die); } clayborg wrote: > We probably should swift m_die->LookupAddress to take a "DWARFDIE > *function_die, DWARFDIE *block_die" instead of "DWARFDebugInfoEntry* > function_die, DWARFDebugInfoEntry* block_die". Then we can avoid doing the > manual combination of CU + offset here. I would prefer to leave it as it is now because we have to do the manual combination at some point (because the returned DIE might be in a different CU then the current DIE) and moving it into LookupAddress will make that function even more complicated. Comment at: source/Plugins/SymbolFile/DWARF/DWARFDIE.cpp:184 @@ +183,3 @@ +assert ((id&0xull) == 0 || m_cu->GetID() == 0); +id |= ((lldb::user_id_t)m_cu->GetID()) << 32; +} clayborg wrote: > The DWARFCompileUnit::GetID() already shifts the cu_offset over by 32, so we > will just lose the compile unit bits here. SymbolFileDWARF does it, but DWARFCompileUnit isn't. With shifting by 32 we lose the SymbolFileDWARF id, but that one isn't considered to be part of the DIE ID. Comment at: source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp:1250 @@ -1180,2 +1249,3 @@ dw_offset_t end_addr_offset = DW_INVALID_OFFSET; -const dw_offset_t attr_offset = GetAttributeValue(dwarf2Data, cu, attr, form_value, _addr_offset); +SymbolFileDWARF* attribute_symbol_file = nullptr; +const dw_offset_t attr_offset = GetAttributeValue(dwarf2Data, clayborg wrote: > Not sure what you were thinking here? You init this with nullptr and then try > to use it below? Did you mean to init this with dwarf2Data? If so then we > don't need this? Otherwise, please fix The function isn't used at all now, so I just removed it completely. (I made the change here to fetch the block data from the correct symbol file, but one of the merge made it broken) http://reviews.llvm.org/D12291 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
clayborg added a comment. I got my DWARFDIE stuff in with: % svn commit Sendinginclude/lldb/Core/dwarf.h Sendinginclude/lldb/Symbol/ClangASTContext.h Sendinginclude/lldb/Symbol/TypeSystem.h Sendinglldb.xcodeproj/project.pbxproj Sending source/Plugins/DynamicLoader/Hexagon-DYLD/DynamicLoaderHexagonDYLD.h Sending source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.h Sendingsource/Plugins/SymbolFile/DWARF/CMakeLists.txt Adding source/Plugins/SymbolFile/DWARF/DWARFAttribute.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFAttribute.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFCompileUnit.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFCompileUnit.h Adding source/Plugins/SymbolFile/DWARF/DWARFDIE.cpp Adding source/Plugins/SymbolFile/DWARF/DWARFDIE.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFDIECollection.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFDIECollection.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugInfo.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugPubnames.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugRanges.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFDebugRanges.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFDeclContext.h Sendingsource/Plugins/SymbolFile/DWARF/DWARFFormValue.cpp Sendingsource/Plugins/SymbolFile/DWARF/DWARFFormValue.h Sendingsource/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp Sendingsource/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h Sendingsource/Plugins/SymbolFile/DWARF/SymbolFileDWARFDebugMap.cpp Sendingsource/Plugins/SymbolFile/DWARF/SymbolFileDWARFDebugMap.h Sendingsource/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.cpp Sendingsource/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.h Sendingsource/Symbol/ClangASTContext.cpp Sendingsource/Symbol/TypeSystem.cpp Sendingsource/Target/Target.cpp Transmitting file data .. Committed revision 246100. http://reviews.llvm.org/D12291 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
tberghammer added a comment. Sorry, first I manage to submit my comments without finishing them. In http://reviews.llvm.org/D12291#232191, @clayborg wrote: I also question why Symbo In http://reviews.llvm.org/D12291#231523, @tberghammer wrote: In the current version of the patch the compile units in the main object file hands out only the compile unit DIE with the information what is available in the main object file. I considered the other approach (hand out all DIEs by the DWARF compile unit in the main object file) but I dropped it for the following reasons: - If we hand out DIEs from a dwo symbol file then each DIE have to store a pointer to the symbol file (or compile unit) it belongs to what is a significant memory overhead (we have more DIEs then Symbols) if we ever want to store all DIE in memory. Even worse is that we have to change all name to DIE index to contain a pointer (compile unit / dwo symbol file) and an offset to find the DIE belongs to (compared to just an offset now) what is more entry then the number DIEs. Can't we just store the SymbolFile inside the compile unit? We always need the compile unit to really do anything with a DIE anyway. We currently store the DWO file inside the compile unit so it seems that we could just store it once in the compile unit and avoid any extra cost. In the name to DIE indexes (in SymbolFileDWARF) currently we store only a DIE offset and we find the compile unit based on the fact that the DIE should be inside the range of the compile unit. The compile units leave in the dwo symbol files all start at address 0 so just a DIE offset isn't enough to find the compile unit. We can store the offset in 4 byte (we already do it, but I am not sure it is a good idea) and the compile unit index in another 4 byte what isn't a major overhead, but it can matter for large inferiors. Storing the symbol file in the DIE might be avoidable but then the DIE have to ask the compile unit for the correct symbol file when somebody queries it for an attribute (we don't want the caller of the GetAttribute* function to know about dwo files). - In an average debug session run from an IDE the user usually sets the breakpoints based on file name + line number, display some stack traces, some variables and do some stepping. If we can index each dwo file separately then we can handle all of these features without parsing the full debug info what can give us some significant speed benefits at debugger startup time. I don't see how we can ever just index one DWO file? If we index one, we must index them all within an executable otherwise the index will be incomplete. If you set a file + line breakpoint, you can't rely on the file matching the compile unit because there could be inlined functions so you would always need to index all of them. Likewise with setting a breakpoint by function name, you will need to index all DWO files. I made a few measurements a few weeks ago and to set a file + line breakpoint we only need to parse the line table what is reasonably fast while parsing all DIEs is significantly slower. Setting a breakpoint based on function name require a full dwarf parsing, but if you use an IDE you almost never want to do it. Maybe we can: - Have a new class that we hand out for a DIE, maybe named DWARFDIE that contains: ``` class DWARFDIE { DWARFCompileUnit *m_cu; DWARFDebugInfoEntry *m_die; }; ``` Then change all of the places we currently use a DWARFCompileUnit *cu, DWARFDebugInfoEntry* die (we always pass them around together) to use a DWARFDIE instead. This allows us to store the DIEs efficiently, yet pass them around in a slightly larger container for usage. This would allow our memory usage to stay very close to where it is (an extra pointer in the compile unit). Then we modify DWARFCompileUnit to store the SymbolFileDWARF * that the compile unit comes from. We can still store the DWO file in the compile unit as well as you are already doing, we would just need to add a SymbolFileDWARF *m_dwarf; member variable for the non DWO case (and also for digging up the DW_TAG_compile_unit attributes that aren't in the DWO DW_TAG_compile_unit). Then we just make sure that all code that hands out DIEs actually hands out DWARFDIE instances instead of returning a DWARFDebugInfoEntry * and also having an out parameter that fills in the compile unit. Thoughts? I like the idea about passing them around together especially as we already do it in a lot of case and it will have only a small overhead, but it don't help on the fact that the name to DIE indexes have to store a compile unit pointer (or a compile unit index). I am not sure how much we want to worry about the memory usage increase because I estimate it to be under 10% increase (without any measurements to prove it), but it will be
Re: [Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
clayborg added a comment. Let me get the DWARFDIE abstraction in and we can see where we are after I get this in as we will see more of what is possible to abstract when this is done. http://reviews.llvm.org/D12291 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
[Lldb-commits] [PATCH] D12291: Add split dwarf support to SymbolFileDWARF
tberghammer created this revision. tberghammer added reviewers: labath, clayborg. tberghammer added a subscriber: lldb-commits. Add split dwarf support to SymbolFileDWARF * Create new dwo symbol file class * Add handling for .dwo sections * Propagate queries from SymbolFileDWARF to the dwo symbol file when necessary After this change and the other split dwarf related changes (D12238, D12239, D12290) most of the tests passes with split dwarf enabled (25 failures, most of them expression evaluation related) http://reviews.llvm.org/D12291 Files: include/lldb/Symbol/ObjectFile.h source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp source/Plugins/SymbolFile/DWARF/CMakeLists.txt source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.cpp source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.h source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.cpp source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.h source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.cpp source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.h source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.cpp source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.h source/Symbol/ObjectFile.cpp Index: source/Symbol/ObjectFile.cpp === --- source/Symbol/ObjectFile.cpp +++ source/Symbol/ObjectFile.cpp @@ -602,15 +602,23 @@ } SectionList * -ObjectFile::GetSectionList() +ObjectFile::GetSectionList(bool update_module_section_list) { if (m_sections_ap.get() == nullptr) { -ModuleSP module_sp(GetModule()); -if (module_sp) +if (update_module_section_list) { -lldb_private::Mutex::Locker locker(module_sp-GetMutex()); -CreateSections(*module_sp-GetUnifiedSectionList()); +ModuleSP module_sp(GetModule()); +if (module_sp) +{ +lldb_private::Mutex::Locker locker(module_sp-GetMutex()); +CreateSections(*module_sp-GetUnifiedSectionList()); +} +} +else +{ +SectionList unified_section_list; +CreateSections(unified_section_list); } } return m_sections_ap.get(); Index: source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.h === --- source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.h +++ source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.h @@ -86,7 +86,7 @@ lldb::TypeSP m_type_sp; SymbolFileDWARF *m_symfile; -const DWARFCompileUnit *m_cu; +DWARFCompileUnit *m_cu; const DWARFDebugInfoEntry *m_die; lldb_private::Declaration m_declaration; int32_t m_byte_size; @@ -118,7 +118,7 @@ bool Find (SymbolFileDWARF *symfile, - const DWARFCompileUnit *cu, + DWARFCompileUnit *cu, const DWARFDebugInfoEntry *die, const lldb_private::Declaration decl, const int32_t byte_size, @@ -151,7 +151,7 @@ bool Find (const lldb_private::ConstString name, SymbolFileDWARF *symfile, - const DWARFCompileUnit *cu, + DWARFCompileUnit *cu, const DWARFDebugInfoEntry *die, const lldb_private::Declaration decl, const int32_t byte_size, Index: source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.cpp === --- source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.cpp +++ source/Plugins/SymbolFile/DWARF/UniqueDWARFASTType.cpp @@ -21,7 +21,7 @@ UniqueDWARFASTTypeList::Find ( SymbolFileDWARF *symfile, -const DWARFCompileUnit *cu, +DWARFCompileUnit *cu, const DWARFDebugInfoEntry *die, const lldb_private::Declaration decl, const int32_t byte_size, Index: source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.h === --- /dev/null +++ source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.h @@ -0,0 +1,45 @@ +//===-- SymbolFileDWARFDwo.h *- C++ -*-===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===--===// + +#ifndef SymbolFileDWARFDwo_SymbolFileDWARFDwo_h_ +#define SymbolFileDWARFDwo_SymbolFileDWARFDwo_h_ + +// C Includes +// C++ Includes +// Other libraries and framework includes +// Project includes +#include SymbolFileDWARF.h + +class SymbolFileDWARFDwo : public SymbolFileDWARF +{ +public: +SymbolFileDWARFDwo(lldb_private::ObjectFile* objfile, DWARFCompileUnit* dwarf_cu); + +virtual +~SymbolFileDWARFDwo() = default; + +const