[llvm-branch-commits] [llvm] [BOLT] Attach pseudo probes to blocks in YAML profile (PR #99554)
https://github.com/rafaelauler approved this pull request. LGTM, though not an expert in pseudo probes https://github.com/llvm/llvm-project/pull/99554 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Support POSSIBLE_PIC_FIXED_BRANCH (PR #91667)
https://github.com/rafaelauler edited https://github.com/llvm/llvm-project/pull/91667 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Support POSSIBLE_PIC_FIXED_BRANCH (PR #91667)
@@ -2523,6 +2523,16 @@ BinaryFunction
*BinaryContext::getBinaryFunctionAtAddress(uint64_t Address) {
return nullptr;
}
+/// Deregister JumpTable registered at a given \p Address and delete it.
+void BinaryContext::deleteJumpTable(uint64_t Address) {
+ JumpTable *JT = getJumpTableContainingAddress(Address);
+ assert(JT && "Must have a jump table at address");
rafaelauler wrote:
Maybe assert that the jump table you're trying to delete is also _exactly_ at
Address? If I understand correctly the logic here, you don't expect the jump
table to start at any other address other than Address. If it does, we are
accidentally deleting something that we shouldn't be, no?
Or do you want to cover cases where the reference is not exactly to Address as
well?
https://github.com/llvm/llvm-project/pull/91667
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Support POSSIBLE_PIC_FIXED_BRANCH (PR #91667)
@@ -1876,6 +1876,13 @@ class X86MCPlusBuilder : public MCPlusBuilder {
//add %r2, %r1
//jmp *%r1
//
+// or a fixed indirect jump template:
+//
+//movslq En(%rip), {%r2|%r1}
+//lea PIC_JUMP_TABLE(%rip), {%r1|%r2} <- MemLocInstr
rafaelauler wrote:
Update this comment and the one above it to reflect the new matching logic:
there is no MemLocInstr anymore
https://github.com/llvm/llvm-project/pull/91667
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Support POSSIBLE_PIC_FIXED_BRANCH (PR #91667)
https://github.com/rafaelauler approved this pull request. Thanks for working on this, Amir https://github.com/llvm/llvm-project/pull/91667 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Eliminate dead jump tables (PR #91666)
https://github.com/rafaelauler requested changes to this pull request. Looks like this is dangerous and crashes binaries in our tests? Do you still want to move forward with this stack? Or are you working on it? If this stack is work in progress, maybe mark it as "changes planned" or the equivalent in github. https://github.com/llvm/llvm-project/pull/91666 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT][NFCI] Fix return type of BC::getSignedValueAtAddress (PR #91664)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/91664 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT][NFC] Define getExprValue helper (PR #91663)
https://github.com/rafaelauler approved this pull request. LGTM https://github.com/llvm/llvm-project/pull/91663 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT][NFC] Simplify analyzeIndirectBranch (PR #91662)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/91662 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT][NFC] Move out PrintProgramStats from Profile into Rewrite (PR #93075)
https://github.com/rafaelauler approved this pull request. LGTM thanks https://github.com/llvm/llvm-project/pull/93075 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [bolt] Revise IDE folder structure (PR #89742)
https://github.com/rafaelauler approved this pull request. LGTM https://github.com/llvm/llvm-project/pull/89742 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add secondary entry points to BAT (PR #86218)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/86218 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add secondary entry points to BAT (PR #86218)
https://github.com/rafaelauler edited https://github.com/llvm/llvm-project/pull/86218 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add secondary entry points to BAT (PR #86218)
https://github.com/rafaelauler requested changes to this pull request. https://github.com/llvm/llvm-project/pull/86218 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add secondary entry points to BAT (PR #86218)
@@ -244,6 +259,17 @@ void BoltAddressTranslation::writeMaps(std::map ,
InOffset >> 1, BBHash, BBIndex));
}
}
+uint32_t PrevOffset = 0;
+if (!Cold && NumSecondaryEntryPoints) {
+ LLVM_DEBUG(dbgs() << "Secondary entry points: ");
+ // Secondary entry point offsets, delta-encoded
+ for (uint32_t Offset : SecondaryEntryPointsMap.at(Address)) {
rafaelauler wrote:
at
https://github.com/llvm/llvm-project/pull/86218
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT] Emit intra-function control flow in YAMLBAT (PR #76911)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/76911 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add CallSiteInfo entries in YAMLBAT (PR #76896)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/76896 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT] Emit intra-function control flow in YAMLBAT (PR #76911)
@@ -128,6 +128,26 @@ class BoltAddressTranslation {
/// Returns BB index by function output address (after BOLT) and basic block
/// input offset.
unsigned getBBIndex(uint64_t FuncOutputAddress, uint32_t BBInputOffset)
const;
+
+ using BBHashMap = std::map>;
+ /// Return a mapping from basic block input offset to hash and block index
for a given function.
+ const BBHashMap (uint64_t OutputAddress) const {
+return FuncHashes.at(OutputAddress).second;
rafaelauler wrote:
I realized you used "at" a number of times now in this stack of diffs. Isn't
this supposed to throw exceptions? Isn't LLVM codebase exception free?
https://github.com/llvm/llvm-project/pull/76911
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT] Add number of basic blocks to BAT (PR #86045)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/86045 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add BB index to BAT (PR #86044)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/86044 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Output basic YAML profile in BAT mode (PR #76910)
https://github.com/rafaelauler approved this pull request. LG https://github.com/llvm/llvm-project/pull/76910 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT] Write and parse BF/BB hashes in BAT (PR #76907)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/76907 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add binary introspection/JIT manager (PR #81346)
@@ -0,0 +1,99 @@
+#include "bolt/Rewrite/JITRewriteInstance.h"
+#include "bolt/Core/BinaryContext.h"
+#include "llvm/BinaryFormat/ELF.h"
+#include "llvm/Support/TargetSelect.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+using namespace llvm::object;
+using namespace llvm::ELF;
+using namespace bolt;
+
+namespace {
+struct JITRewriteInstanceTester
+: public testing::TestWithParam {
+ void SetUp() override {
+initalizeLLVM();
+initializeBOLT();
+ }
+
+protected:
+ void initalizeLLVM() {
+llvm::InitializeAllTargetInfos();
+llvm::InitializeAllTargetMCs();
+llvm::InitializeAllAsmParsers();
+llvm::InitializeAllDisassemblers();
+llvm::InitializeAllTargets();
+llvm::InitializeAllAsmPrinters();
+ }
+
+ void initializeBOLT() {
+BOLTJIT = cantFail(bolt::JITRewriteInstance::createJITRewriteInstance(
+{llvm::outs(), llvm::errs()}, /*IsPIC*/ false));
+ASSERT_FALSE(!BOLTJIT);
+ }
+
+ std::unique_ptr BOLTJIT;
+};
+} // namespace
+
+#ifdef X86_AVAILABLE
+
+// clang-format off
+extern "C" __attribute((naked)) int fib(int n)
+{
+ __asm__ __volatile__(
+"pushq %%r14\n"
+"pushq %%rbx\n"
+"pushq %%rax\n"
+"movl%%edi, %%r14d\n"
+"xorl%%ebx, %%ebx\n"
+"cmpl$0x2, %%edi\n"
+"jge .Ltmp0\n"
+"movl%%r14d, %%ecx\n"
+"jmp .Ltmp1\n"
+".Ltmp0:\n"
+"xorl%%ebx, %%ebx\n"
+"nopw%%cs:(%%rax,%%rax)\n"
+".Ltmp2:\n"
+"leal-0x1(%%r14), %%edi\n"
+"callq fib\n"
+"leal-0x2(%%r14), %%ecx\n"
+"addl%%eax, %%ebx\n"
+"cmpl$0x3, %%r14d\n"
+"movl%%ecx, %%r14d\n"
+"ja .Ltmp2\n"
+".Ltmp1:\n"
+"addl%%ecx, %%ebx\n"
+"movl%%ebx, %%eax\n"
+"addq$0x8, %%rsp\n"
+"popq%%rbx\n"
+"popq%%r14\n"
+"retq\n"
+:::);
+}
+// clang-format on
+
+INSTANTIATE_TEST_SUITE_P(X86, JITRewriteInstanceTester,
+ ::testing::Values(Triple::x86_64));
+
+TEST_P(JITRewriteInstanceTester, DisassembleFib) {
+ EXPECT_EQ(fib(7), 13);
+
+ // BOLT JIT test/example
+ // Analyze fib function in this binary
+ // Disassemble 63 bytes
+ uint64_t Address = reinterpret_cast();
+ StringRef Data = StringRef(reinterpret_cast(), 63);
rafaelauler wrote:
The rewriter object will only get knowledge about the symbol size after a call
to to registerJITFunction. Not sure if I answered the question, but does it
make sense to you? The user calling registerJITSection function will first make
a section visible to the Rewriter, and then the user can partition this section
in different functions using registerJITFunction, so the user passes the size
down to the rewriter instead of the opposite.
https://github.com/llvm/llvm-project/pull/81346
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT] Log through JournalingStreams (PR #81524)
https://github.com/rafaelauler approved this pull request. Thanks, here's what I think is a suitable commit message for this one: Make core BOLT functionality more friendly to being used as a library instead of in our standalone driver llvm-bolt. To accomplish this, we augment BinaryContext with journaling streams that are to be used by most BOLT code whenever something needs to be logged to the screen. Users of the library can decide if logs should be printed to a file, no file or to the screen, as before. To illustrate this, this patch adds a new option --log-file that allows the user to redirect BOLT logging to a file on disk or completely hide it by using --log-file=/dev/null. Future BOLT code should now use BinaryContext::outs() for printing important messages instead of llvm::outs(). A new test log.test enforces this by verifying that no strings are print to screen once the --log-file option is used. In previous patches we also added a new BOLTError class to report common and fatal errors, so code shouldn't call exit(1) now. To easily handle problems as before (by quitting with exit(1)), callers can now use BinaryContext::logBOLTErrorsAndQuitOnFatal(Error) whenever code needs to deal with BOLT errors. To test this, we have fatal.s that checks we are correctly quitting and printing a fatal error to the screen. Because this is a significant change by itself, not all code was yet ported. Code from Profiler libs (DataAggregator and friends) still print errors directly to screen. https://github.com/llvm/llvm-project/pull/81524 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Propagate BOLTErrors from Core, RewriteInstance, and passes (2/2) (PR #81523)
https://github.com/rafaelauler approved this pull request. Thanks, as commit message: As part of the effort to refactor old error handling code that would directly call exit(1), in this patch continue the migration on libCore, libRewrite and libPasses to use the new BOLTError class whenever a failure occurs. https://github.com/llvm/llvm-project/pull/81523 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT][NFC] Add BOLTError and return it from passes (1/2) (PR #81522)
https://github.com/rafaelauler edited https://github.com/llvm/llvm-project/pull/81522 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [BOLT][NFC] Add BOLTError and return it from passes (1/2) (PR #81522)
https://github.com/rafaelauler approved this pull request. Thanks, as commit message we can put something like: As part of the effort to refactor old error handling code that would directly call exit(1), in this patch we add a new class BOLTError and auxiliary functions createFatalBOLTError() and createNonFatalBOLTError() that allow BOLT code to bubble up the problem to the caller by using the Error class as a return type (or Expected). Also change passes to use these. https://github.com/llvm/llvm-project/pull/81522 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add binary introspection/JIT manager (PR #81346)
rafaelauler wrote: Depends on #81082 https://github.com/llvm/llvm-project/pull/81346 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [BOLT] Add binary introspection/JIT manager (PR #81346)
https://github.com/rafaelauler created
https://github.com/llvm/llvm-project/pull/81346
Add a class that allows a process to introspect or investigate itself by
disassembling its memory contents just-in-time with BOLT. An example is shown
in a new unittest binary.
This leverages the new ability to use BOLT as a library instead of as a regular
executable that processes input binaries, demonstrating how to use BOLT as a
library.
>From 1167a59a185cbc113d6ca30f223e09be1abc8494 Mon Sep 17 00:00:00 2001
From: Rafael Auler
Date: Thu, 8 Feb 2024 19:57:14 -0800
Subject: [PATCH] [BOLT] Add binary introspection/JIT manager
Add a class that allows a process to introspect or investigate itself
by disassembling its memory contents just-in-time with BOLT. An
example is shown in a new unittest binary.
This leverages the new ability to use BOLT as a library instead of as
a regular executable that processes input binaries, demonstrating how
to use BOLT as a library.
---
bolt/include/bolt/Core/BinaryContext.h| 9 +-
bolt/include/bolt/Core/BinaryFunction.h | 1 +
bolt/include/bolt/Core/BinarySection.h| 3 +-
.../include/bolt/Rewrite/JITRewriteInstance.h | 105 +
bolt/lib/Core/BinaryContext.cpp | 39 +-
bolt/lib/Rewrite/CMakeLists.txt | 1 +
bolt/lib/Rewrite/DWARFRewriter.cpp| 2 +-
bolt/lib/Rewrite/JITRewriteInstance.cpp | 367 ++
bolt/lib/Rewrite/MachORewriteInstance.cpp | 35 +-
bolt/lib/Rewrite/RewriteInstance.cpp | 19 +-
bolt/unittests/CMakeLists.txt | 1 +
bolt/unittests/Core/BinaryContext.cpp | 4 +-
bolt/unittests/Core/MCPlusBuilder.cpp | 4 +-
bolt/unittests/Rewrite/CMakeLists.txt | 27 ++
bolt/unittests/Rewrite/JITRewriteInstance.cpp | 99 +
15 files changed, 654 insertions(+), 62 deletions(-)
create mode 100644 bolt/include/bolt/Rewrite/JITRewriteInstance.h
create mode 100644 bolt/lib/Rewrite/JITRewriteInstance.cpp
create mode 100644 bolt/unittests/Rewrite/CMakeLists.txt
create mode 100644 bolt/unittests/Rewrite/JITRewriteInstance.cpp
diff --git a/bolt/include/bolt/Core/BinaryContext.h
b/bolt/include/bolt/Core/BinaryContext.h
index 30336c4e3a74fe..e5af8685d9b0d0 100644
--- a/bolt/include/bolt/Core/BinaryContext.h
+++ b/bolt/include/bolt/Core/BinaryContext.h
@@ -212,9 +212,6 @@ class BinaryContext {
/// input file to internal section representation.
DenseMap SectionRefToBinarySection;
- /// Low level section registration.
- BinarySection (BinarySection *Section);
-
/// Store all functions in the binary, sorted by original address.
std::map BinaryFunctions;
@@ -265,7 +262,8 @@ class BinaryContext {
public:
static Expected>
- createBinaryContext(const ObjectFile *File, bool IsPIC,
+ createBinaryContext(Triple TheTriple, StringRef InputFileName,
+ SubtargetFeatures *Features, bool IsPIC,
std::unique_ptr DwCtx,
JournalingStreams Logger);
@@ -1049,6 +1047,9 @@ class BinaryContext {
BinarySection (const Twine ,
const BinarySection );
+ /// Low level section registration.
+ BinarySection (BinarySection *Section);
+
/// Register or update the information for the section with the given
/// /p Name. If the section already exists, the information in the
/// section will be updated with the new data.
diff --git a/bolt/include/bolt/Core/BinaryFunction.h
b/bolt/include/bolt/Core/BinaryFunction.h
index a177178769e456..68c626d9a2f452 100644
--- a/bolt/include/bolt/Core/BinaryFunction.h
+++ b/bolt/include/bolt/Core/BinaryFunction.h
@@ -655,6 +655,7 @@ class BinaryFunction {
BinaryFunction(const BinaryFunction &) = delete;
friend class MachORewriteInstance;
+ friend class JITRewriteInstance;
friend class RewriteInstance;
friend class BinaryContext;
friend class DataReader;
diff --git a/bolt/include/bolt/Core/BinarySection.h
b/bolt/include/bolt/Core/BinarySection.h
index a85dbf28950e31..d4f9b5955b2029 100644
--- a/bolt/include/bolt/Core/BinarySection.h
+++ b/bolt/include/bolt/Core/BinarySection.h
@@ -50,7 +50,7 @@ class BinarySection {
std::string Name;// Section name
const SectionRef Section;// SectionRef for input binary sections.
StringRef Contents; // Input section contents
- const uint64_t Address; // Address of section in input binary (may be 0)
+ uint64_t Address;// Address of section in input binary (may be 0)
const uint64_t Size; // Input section size
uint64_t InputFileOffset{0}; // Offset in the input binary
unsigned Alignment; // alignment in bytes (must be > 0)
@@ -461,6 +461,7 @@ class BinarySection {
uint32_t getIndex() const { return Index; }
// mutation
+ void setAddress(uint64_t Address) { this->Address = Address; }
void setOutputAddress(uint64_t Address) { OutputAddress = Address; }
[llvm-branch-commits] [clang] [openmp] [libc] [lldb] [clang-tools-extra] [flang] [mlir] [libcxx] [llvm] [BOLT] Write and parse BF/BB hashes in BAT (PR #76907)
@@ -36,9 +37,12 @@ void BoltAddressTranslation::writeEntriesForBB(MapTy , if (BBInputOffset == BinaryBasicBlock::INVALID_OFFSET) return; - LLVM_DEBUG(dbgs() << "BB " << BB.getName() << "\n"); - LLVM_DEBUG(dbgs() << " Key: " << Twine::utohexstr(BBOutputOffset) -<< " Val: " << Twine::utohexstr(BBInputOffset) << "\n"); + LLVM_DEBUG(dbgs() << "BB " << BB.getName() << "\n" rafaelauler wrote: So you're not using the basic block hash anywhere, other than printing it to debug output? https://github.com/llvm/llvm-project/pull/76907 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [openmp] [flang] [clang] [mlir] [lldb] [libcxx] [clang-tools-extra] [libc] [BOLT][NFC] Pass BF/BB hashes to BAT (PR #76906)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/76906 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [clang] [libc] [llvm] [libcxx] [openmp] [lldb] [flang] [mlir] [clang-tools-extra] [BOLT] Deduplicate equal offsets in BAT (PR #76905)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/76905 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [lldb] [clang] [BOLT] Fix spurious boltedcollection from merge-fdata (PR #78653)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/78653 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [flang] [clang] [clang-tools-extra] [mlir] [libc] [libcxx] [openmp] [llvm] [lldb] [BOLT] Use continuous output addresses in delta encoding in BAT (PR #76904)
https://github.com/rafaelauler approved this pull request. LGTM https://github.com/llvm/llvm-project/pull/76904 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [mlir] [clang] [flang] [openmp] [lldb] [libc] [libcxx] [clang-tools-extra] [llvm] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
https://github.com/rafaelauler approved this pull request. Thanks, looks good. https://github.com/llvm/llvm-project/pull/76903 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang-tools-extra] [libcxx] [lldb] [libc] [flang] [clang] [mlir] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
@@ -119,6 +130,7 @@ class BoltAddressTranslation {
uint64_t FuncAddress);
std::map Maps;
+ std::map ColdMaps;
rafaelauler wrote:
Kind of weird that this is only used for write(), and not during parsing. There
is a lost symmetry between write() and parse() after this diff. Maybe move
ColdMaps to live inside write() method then, since it is never used outside of
it.
Once we write(), we're not able to call translate(), since we lost cold
functions in Maps. Not that this workflow was ever used, but if this class was
meant to have any sort of "serialize/deserialize" workflow, now they are out of
sync, and design-wise it feels off. So maybe we should use both a private
"Maps" / "ColdMaps" during write(), so the code doesn't read as if "translate"
depends on either write or parse -- it should depends only on parse. And maybe
write() should be a stateless static method.
https://github.com/llvm/llvm-project/pull/76903
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [clang] [mlir] [clang-tools-extra] [libcxx] [flang] [llvm] [libc] [lldb] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
@@ -11,6 +11,7 @@ #include "llvm/ADT/SmallVector.h" #include "llvm/ADT/StringRef.h" +#include "llvm/Support/DataExtractor.h" rafaelauler wrote: Not sure why was this moved here https://github.com/llvm/llvm-project/pull/76903 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [lldb] [llvm] [libcxx] [mlir] [clang-tools-extra] [clang] [libc] [flang] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
@@ -78,10 +79,20 @@ class BoltAddressTranslation {
BoltAddressTranslation() {}
+ /// Write the serialized address translation table for a function.
+ template
rafaelauler wrote:
Move to private: ?
https://github.com/llvm/llvm-project/pull/76903
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [mlir] [clang-tools-extra] [clang] [llvm] [flang] [libc] [lldb] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
https://github.com/rafaelauler edited https://github.com/llvm/llvm-project/pull/76903 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [mlir] [clang] [flang] [lldb] [llvm] [libcxx] [clang-tools-extra] [libc] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
@@ -160,12 +164,31 @@ std::error_code BoltAddressTranslation::parse(StringRef
Buf) {
return make_error_code(llvm::errc::io_error);
Error Err(Error::success());
+ std::vector HotFuncs;
rafaelauler wrote:
Is this HotFuncs vector going to be used in another diff of this stack? It
looks unused to me atm.
https://github.com/llvm/llvm-project/pull/76903
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [mlir] [flang] [lldb] [libc] [libcxx] [clang] [clang-tools-extra] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
@@ -78,10 +79,20 @@ class BoltAddressTranslation {
BoltAddressTranslation() {}
+ /// Write the serialized address translation table for a function.
+ template
+ void writeMaps(std::map , raw_ostream );
+
/// Write the serialized address translation tables for each reordered
/// function
void write(const BinaryContext , raw_ostream );
+ /// Read the serialized address translation table for a function.
+ /// Return a parse error if failed.
+ template
rafaelauler wrote:
private: ?
https://github.com/llvm/llvm-project/pull/76903
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang-tools-extra] [lldb] [clang] [libc] [libcxx] [mlir] [flang] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)
https://github.com/rafaelauler requested changes to this pull request. https://github.com/llvm/llvm-project/pull/76903 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang-tools-extra] [lldb] [clang] [libc] [libcxx] [mlir] [BOLT] Delta-encode function start addresses in BAT (PR #76902)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/76902 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [clang-tools-extra] [clang] [libcxx] [mlir] [llvm] [libc] [lldb] [BOLT] Delta-encode offsets in BAT (PR #76900)
@@ -164,12 +167,16 @@ std::error_code BoltAddressTranslation::parse(StringRef
Buf) {
LLVM_DEBUG(dbgs() << "Parsing " << NumEntries << " entries for 0x"
<< Twine::utohexstr(Address) << "\n");
+uint64_t InputOffset = 0, OutputOffset = 0;
rafaelauler wrote:
Maybe I'm missing something, but I don't understand why are we naming this as
Offset. Even in BAT.md it is listed as 'address' instead of offset.
https://github.com/llvm/llvm-project/pull/76900
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [clang] [mlir] [libc] [lldb] [clang-tools-extra] [llvm] [libcxx] [BOLT] Delta-encode offsets in BAT (PR #76900)
@@ -114,9 +114,12 @@ void BoltAddressTranslation::write(const BinaryContext
, raw_ostream ) {
<< Twine::utohexstr(Address) << ".\n");
encodeULEB128(Address, OS);
encodeULEB128(NumEntries, OS);
+uint64_t InOffset = 0, OutOffset = 0;
rafaelauler wrote:
It's not an offset, right? Maybe a better name would be PrevIn, PrevOut? (as in
"Previous Input Address")
https://github.com/llvm/llvm-project/pull/76900
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [clang-tools-extra] [mlir] [lldb] [libcxx] [llvm] [libc] [clang] [BOLT] Delta-encode offsets in BAT (PR #76900)
https://github.com/rafaelauler edited https://github.com/llvm/llvm-project/pull/76900 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang-tools-extra] [lldb] [clang] [libc] [libcxx] [mlir] [BOLT] Delta-encode offsets in BAT (PR #76900)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/76900 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [mlir] [clang-tools-extra] [libc] [libcxx] [clang] [lldb] [BOLT] Encode BAT using ULEB128 (PR #76899)
https://github.com/rafaelauler approved this pull request. Great! https://github.com/llvm/llvm-project/pull/76899 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc] [clang-tools-extra] [llvm] [lldb] [clang] [mlir] [BOLT] Add BOLT Address Translation documentation (PR #76898)
@@ -0,0 +1,97 @@ +# BOLT Address Translation (BAT) +# Purpose +A regular profile collection for BOLT involves collecting samples from +unoptimized binary. BOLT Address Translation allows collecting profile +from BOLT-optimized binary and using it for optimizing the input (pre-BOLT) +binary. + +# Overview +BOLT Address Translation is an extra section (`.note.bolt_bat`) inserted by BOLT +into the output binary containing translation tables and split functions linkage +information. This information enables mapping the profile back from optimized +binary onto the original binary. + +# Usage +`--enable-bat` flag controls the generation of BAT section. Sampled profile +needs to be passed along with the optimized binary containing BAT section to +`perf2bolt` which reads BAT section and produces fdata profile for the original +binary. Note that YAML profile generation is not supported since BAT doesn't +contain the metadata for input functions. + +# Internals +## Section contents +The section is organized as follows: +- Functions table + - Address translation tables +- Fragment linkage table + +## Construction and parsing +BAT section is created from `BoltAddressTranslation` class which captures +address translation information provided by BOLT linker. It is then encoded as a +note section in the output binary. + +During profile conversion when BAT-enabled binary is passed to perf2bolt, +`BoltAddressTranslation` class is populated from BAT section. The class is then +queried by `DataAggregator` during sample processing to reconstruct addresses/ +offsets in the input binary. + +## Encoding format +The encoding is specified in +[BoltAddressTranslation.h](/bolt/include/bolt/Profile/BoltAddressTranslation.h) +and [BoltAddressTranslation.cpp](/bolt/lib/Profile/BoltAddressTranslation.cpp). + +### Layout +The general layout is as follows: +``` +Functions table header +|--| +| Function entry | +| |--| | +| | OutOff InOff | | +| |--| | + + +Fragment linkage header +|--| +| ColdAddr HotAddr | + +``` + +### Functions table +Header: +| Entry | Width | Description | +| -- | - | --- | +| `NumFuncs` | 4b | Number of functions in the functions table | rafaelauler wrote: Minor suggestion: 4b -> 4B I usually see Bytes capitalized, but that's up to you https://github.com/llvm/llvm-project/pull/76898 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [libc] [mlir] [libcxx] [clang-tools-extra] [lldb] [clang] [BOLT] Add BOLT Address Translation documentation (PR #76898)
https://github.com/rafaelauler approved this pull request. https://github.com/llvm/llvm-project/pull/76898 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [mlir] [lldb] [clang] [libcxx] [clang-tools-extra] [libc] [BOLT] Add BOLT Address Translation documentation (PR #76898)
https://github.com/rafaelauler edited https://github.com/llvm/llvm-project/pull/76898 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
