[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
weliveindetail wrote: @amy-kwan Thanks for your note and sorry for the long delay. There were so many unrelated buildbot failures today, that I didn't catch this one. I think it was fixed meanwhile with https://github.com/llvm/llvm-project/commit/cb994d41c3afb2bd0b25a4c5b2ac48978bf1b23d thanks to @kparzysz! https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
Stefan =?utf-8?q?Gränitz?= Message-ID: In-Reply-To: amy-kwan wrote: Hi @weliveindetail! I just wanted to give a heads up, that I believe 13078cbc3eeb0ae91c370ce0f604f7165b26e0c8 is causing a failure of the clang-ppc64le-rhel bot: https://lab.llvm.org/buildbot/#/builders/57/builds/33764/steps/7/logs/stdio Would you be able to help take a look? https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
Stefan =?utf-8?q?Gr=C3=A4nitz?= Message-ID: In-Reply-To: jplehr wrote: I see. Thanks for the explanation and the fix! Bot is back to green. https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
weliveindetail wrote: We do need the target and not just the symbols. It could be any non-native target, but implementing a selection mechanism isn't worth the effort. We just try ARM and (with my above patch) otherwise skip the test. https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
Stefan =?utf-8?q?Gränitz?= Message-ID: In-Reply-To: jplehr wrote: Thanks! I am quite unfamiliar with that part of the code base and wonder if the symbol needs to just exist somewhere. The other thing used there (`InitializeNativeTargetAsmPrinter`) is declared in `TargetSelect.h`. So, does it need to be a two parts fix: one to declare the symbol and then the magic to do the right thing at runtime. If what you linked does both, even better. :) https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
weliveindetail wrote: I will try this https://github.com/llvm/llvm-project/blob/release/18.x/llvm/unittests/DebugInfo/LogicalView/CodeViewReaderTest.cpp#L482 and push a quick-fix. https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
weliveindetail wrote: Thanks for your note. Looks like the problem is that the ARM target is not registered. It's an uncommon requirement for a unitttest.. Will see how to check that at runtime. If you have an idea, let me know. Thanks https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
Stefan =?utf-8?q?Gränitz?= Message-ID: In-Reply-To: jplehr wrote: Hi, I think this one broke one of our buildbots: https://lab.llvm.org/buildbot/#/builders/259/builds/1769 I'm happy to help looking into it. https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail closed https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail updated
https://github.com/llvm/llvm-project/pull/84461
From fae2f46d25650b8480f9d3135f33a0d6532f43ca Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Stefan=20Gr=C3=A4nitz?=
Date: Thu, 7 Mar 2024 23:04:22 +0100
Subject: [PATCH 1/2] [clang-repl] Add CreateJITBuilder() for specialization in
derived classes
The LLJITBuilder interface provides a very convenient way to configure the JIT.
---
clang/include/clang/Interpreter/Interpreter.h | 9 ++
clang/lib/Interpreter/IncrementalExecutor.cpp | 33 ++---
clang/lib/Interpreter/IncrementalExecutor.h | 9 +-
clang/lib/Interpreter/Interpreter.cpp | 26 +++-
.../Interpreter/InterpreterExtensionsTest.cpp | 117 +-
5 files changed, 173 insertions(+), 21 deletions(-)
diff --git a/clang/include/clang/Interpreter/Interpreter.h
b/clang/include/clang/Interpreter/Interpreter.h
index 1dcba1ef967980..33ce4bbf5bea10 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -29,7 +29,9 @@
namespace llvm {
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -127,6 +129,13 @@ class Interpreter {
// custom runtime.
virtual std::unique_ptr FindRuntimeInterface();
+ // Lazily construct thev ORCv2 JITBuilder. This called when the internal
+ // IncrementalExecutor is created. The default implementation populates an
+ // in-process JIT with debugging support. Override this to configure the JIT
+ // engine used for execution.
+ virtual llvm::Expected>
+ CreateJITBuilder(CompilerInstance );
+
public:
virtual ~Interpreter();
diff --git a/clang/lib/Interpreter/IncrementalExecutor.cpp
b/clang/lib/Interpreter/IncrementalExecutor.cpp
index 40bcef94797d43..6f036107c14a9c 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.cpp
+++ b/clang/lib/Interpreter/IncrementalExecutor.cpp
@@ -20,6 +20,7 @@
#include "llvm/ExecutionEngine/Orc/Debugging/DebuggerSupport.h"
#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
+#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
#include "llvm/ExecutionEngine/Orc/LLJIT.h"
#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
@@ -36,26 +37,28 @@ LLVM_ATTRIBUTE_USED void linkComponents() {
namespace clang {
+llvm::Expected>
+IncrementalExecutor::createDefaultJITBuilder(
+llvm::orc::JITTargetMachineBuilder JTMB) {
+ auto JITBuilder = std::make_unique();
+ JITBuilder->setJITTargetMachineBuilder(std::move(JTMB));
+ JITBuilder->setPrePlatformSetup([](llvm::orc::LLJIT ) {
+// Try to enable debugging of JIT'd code (only works with JITLink for
+// ELF and MachO).
+consumeError(llvm::orc::enableDebuggerSupport(J));
+return llvm::Error::success();
+ });
+ return std::move(JITBuilder);
+}
+
IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext ,
- llvm::Error ,
- const clang::TargetInfo )
+ llvm::orc::LLJITBuilder ,
+ llvm::Error )
: TSCtx(TSC) {
using namespace llvm::orc;
llvm::ErrorAsOutParameter EAO();
- auto JTMB = JITTargetMachineBuilder(TI.getTriple());
- JTMB.addFeatures(TI.getTargetOpts().Features);
- LLJITBuilder Builder;
- Builder.setJITTargetMachineBuilder(JTMB);
- Builder.setPrePlatformSetup(
- [](LLJIT ) {
-// Try to enable debugging of JIT'd code (only works with JITLink for
-// ELF and MachO).
-consumeError(enableDebuggerSupport(J));
-return llvm::Error::success();
- });
-
- if (auto JitOrErr = Builder.create())
+ if (auto JitOrErr = JITBuilder.create())
Jit = std::move(*JitOrErr);
else {
Err = JitOrErr.takeError();
diff --git a/clang/lib/Interpreter/IncrementalExecutor.h
b/clang/lib/Interpreter/IncrementalExecutor.h
index dd0a210a061415..b4347209e14fe3 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.h
+++ b/clang/lib/Interpreter/IncrementalExecutor.h
@@ -23,7 +23,9 @@
namespace llvm {
class Error;
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -44,8 +46,8 @@ class IncrementalExecutor {
public:
enum SymbolNameKind { IRName, LinkerName };
- IncrementalExecutor(llvm::orc::ThreadSafeContext , llvm::Error ,
- const clang::TargetInfo );
+ IncrementalExecutor(llvm::orc::ThreadSafeContext ,
+ llvm::orc::LLJITBuilder , llvm::Error );
~IncrementalExecutor();
llvm::Error addModule(PartialTranslationUnit );
@@ -56,6 +58,9 @@ class IncrementalExecutor {
getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const;
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail updated
https://github.com/llvm/llvm-project/pull/84461
From fae2f46d25650b8480f9d3135f33a0d6532f43ca Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Stefan=20Gr=C3=A4nitz?=
Date: Thu, 7 Mar 2024 23:04:22 +0100
Subject: [PATCH] [clang-repl] Add CreateJITBuilder() for specialization in
derived classes
The LLJITBuilder interface provides a very convenient way to configure the JIT.
---
clang/include/clang/Interpreter/Interpreter.h | 9 ++
clang/lib/Interpreter/IncrementalExecutor.cpp | 33 ++---
clang/lib/Interpreter/IncrementalExecutor.h | 9 +-
clang/lib/Interpreter/Interpreter.cpp | 26 +++-
.../Interpreter/InterpreterExtensionsTest.cpp | 117 +-
5 files changed, 173 insertions(+), 21 deletions(-)
diff --git a/clang/include/clang/Interpreter/Interpreter.h
b/clang/include/clang/Interpreter/Interpreter.h
index 1dcba1ef967980..33ce4bbf5bea10 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -29,7 +29,9 @@
namespace llvm {
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -127,6 +129,13 @@ class Interpreter {
// custom runtime.
virtual std::unique_ptr FindRuntimeInterface();
+ // Lazily construct thev ORCv2 JITBuilder. This called when the internal
+ // IncrementalExecutor is created. The default implementation populates an
+ // in-process JIT with debugging support. Override this to configure the JIT
+ // engine used for execution.
+ virtual llvm::Expected>
+ CreateJITBuilder(CompilerInstance );
+
public:
virtual ~Interpreter();
diff --git a/clang/lib/Interpreter/IncrementalExecutor.cpp
b/clang/lib/Interpreter/IncrementalExecutor.cpp
index 40bcef94797d43..6f036107c14a9c 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.cpp
+++ b/clang/lib/Interpreter/IncrementalExecutor.cpp
@@ -20,6 +20,7 @@
#include "llvm/ExecutionEngine/Orc/Debugging/DebuggerSupport.h"
#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
+#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
#include "llvm/ExecutionEngine/Orc/LLJIT.h"
#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
@@ -36,26 +37,28 @@ LLVM_ATTRIBUTE_USED void linkComponents() {
namespace clang {
+llvm::Expected>
+IncrementalExecutor::createDefaultJITBuilder(
+llvm::orc::JITTargetMachineBuilder JTMB) {
+ auto JITBuilder = std::make_unique();
+ JITBuilder->setJITTargetMachineBuilder(std::move(JTMB));
+ JITBuilder->setPrePlatformSetup([](llvm::orc::LLJIT ) {
+// Try to enable debugging of JIT'd code (only works with JITLink for
+// ELF and MachO).
+consumeError(llvm::orc::enableDebuggerSupport(J));
+return llvm::Error::success();
+ });
+ return std::move(JITBuilder);
+}
+
IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext ,
- llvm::Error ,
- const clang::TargetInfo )
+ llvm::orc::LLJITBuilder ,
+ llvm::Error )
: TSCtx(TSC) {
using namespace llvm::orc;
llvm::ErrorAsOutParameter EAO();
- auto JTMB = JITTargetMachineBuilder(TI.getTriple());
- JTMB.addFeatures(TI.getTargetOpts().Features);
- LLJITBuilder Builder;
- Builder.setJITTargetMachineBuilder(JTMB);
- Builder.setPrePlatformSetup(
- [](LLJIT ) {
-// Try to enable debugging of JIT'd code (only works with JITLink for
-// ELF and MachO).
-consumeError(enableDebuggerSupport(J));
-return llvm::Error::success();
- });
-
- if (auto JitOrErr = Builder.create())
+ if (auto JitOrErr = JITBuilder.create())
Jit = std::move(*JitOrErr);
else {
Err = JitOrErr.takeError();
diff --git a/clang/lib/Interpreter/IncrementalExecutor.h
b/clang/lib/Interpreter/IncrementalExecutor.h
index dd0a210a061415..b4347209e14fe3 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.h
+++ b/clang/lib/Interpreter/IncrementalExecutor.h
@@ -23,7 +23,9 @@
namespace llvm {
class Error;
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -44,8 +46,8 @@ class IncrementalExecutor {
public:
enum SymbolNameKind { IRName, LinkerName };
- IncrementalExecutor(llvm::orc::ThreadSafeContext , llvm::Error ,
- const clang::TargetInfo );
+ IncrementalExecutor(llvm::orc::ThreadSafeContext ,
+ llvm::orc::LLJITBuilder , llvm::Error );
~IncrementalExecutor();
llvm::Error addModule(PartialTranslationUnit );
@@ -56,6 +58,9 @@ class IncrementalExecutor {
getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const;
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/vgvassilev approved this pull request. LGTM! https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail edited https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail updated
https://github.com/llvm/llvm-project/pull/84461
From 88271c39b30b84041b4b7fb8b8f34c211d8190d5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Stefan=20Gr=C3=A4nitz?=
Date: Thu, 7 Mar 2024 23:04:22 +0100
Subject: [PATCH] [clang-repl] Add CreateJITBuilder() for specialization in
derived classes
The LLJITBuilder interface provides a very convenient way to configure the JIT.
---
clang/include/clang/Interpreter/Interpreter.h | 9 ++
clang/lib/Interpreter/IncrementalExecutor.cpp | 33 ++---
clang/lib/Interpreter/IncrementalExecutor.h | 9 +-
clang/lib/Interpreter/Interpreter.cpp | 26 +++-
.../Interpreter/InterpreterExtensionsTest.cpp | 119 +-
5 files changed, 175 insertions(+), 21 deletions(-)
diff --git a/clang/include/clang/Interpreter/Interpreter.h
b/clang/include/clang/Interpreter/Interpreter.h
index 1dcba1ef967980..33ce4bbf5bea10 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -29,7 +29,9 @@
namespace llvm {
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -127,6 +129,13 @@ class Interpreter {
// custom runtime.
virtual std::unique_ptr FindRuntimeInterface();
+ // Lazily construct thev ORCv2 JITBuilder. This called when the internal
+ // IncrementalExecutor is created. The default implementation populates an
+ // in-process JIT with debugging support. Override this to configure the JIT
+ // engine used for execution.
+ virtual llvm::Expected>
+ CreateJITBuilder(CompilerInstance );
+
public:
virtual ~Interpreter();
diff --git a/clang/lib/Interpreter/IncrementalExecutor.cpp
b/clang/lib/Interpreter/IncrementalExecutor.cpp
index 40bcef94797d43..6f036107c14a9c 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.cpp
+++ b/clang/lib/Interpreter/IncrementalExecutor.cpp
@@ -20,6 +20,7 @@
#include "llvm/ExecutionEngine/Orc/Debugging/DebuggerSupport.h"
#include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
+#include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
#include "llvm/ExecutionEngine/Orc/LLJIT.h"
#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"
@@ -36,26 +37,28 @@ LLVM_ATTRIBUTE_USED void linkComponents() {
namespace clang {
+llvm::Expected>
+IncrementalExecutor::createDefaultJITBuilder(
+llvm::orc::JITTargetMachineBuilder JTMB) {
+ auto JITBuilder = std::make_unique();
+ JITBuilder->setJITTargetMachineBuilder(std::move(JTMB));
+ JITBuilder->setPrePlatformSetup([](llvm::orc::LLJIT ) {
+// Try to enable debugging of JIT'd code (only works with JITLink for
+// ELF and MachO).
+consumeError(llvm::orc::enableDebuggerSupport(J));
+return llvm::Error::success();
+ });
+ return std::move(JITBuilder);
+}
+
IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext ,
- llvm::Error ,
- const clang::TargetInfo )
+ llvm::orc::LLJITBuilder ,
+ llvm::Error )
: TSCtx(TSC) {
using namespace llvm::orc;
llvm::ErrorAsOutParameter EAO();
- auto JTMB = JITTargetMachineBuilder(TI.getTriple());
- JTMB.addFeatures(TI.getTargetOpts().Features);
- LLJITBuilder Builder;
- Builder.setJITTargetMachineBuilder(JTMB);
- Builder.setPrePlatformSetup(
- [](LLJIT ) {
-// Try to enable debugging of JIT'd code (only works with JITLink for
-// ELF and MachO).
-consumeError(enableDebuggerSupport(J));
-return llvm::Error::success();
- });
-
- if (auto JitOrErr = Builder.create())
+ if (auto JitOrErr = JITBuilder.create())
Jit = std::move(*JitOrErr);
else {
Err = JitOrErr.takeError();
diff --git a/clang/lib/Interpreter/IncrementalExecutor.h
b/clang/lib/Interpreter/IncrementalExecutor.h
index dd0a210a061415..b4347209e14fe3 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.h
+++ b/clang/lib/Interpreter/IncrementalExecutor.h
@@ -23,7 +23,9 @@
namespace llvm {
class Error;
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -44,8 +46,8 @@ class IncrementalExecutor {
public:
enum SymbolNameKind { IRName, LinkerName };
- IncrementalExecutor(llvm::orc::ThreadSafeContext , llvm::Error ,
- const clang::TargetInfo );
+ IncrementalExecutor(llvm::orc::ThreadSafeContext ,
+ llvm::orc::LLJITBuilder , llvm::Error );
~IncrementalExecutor();
llvm::Error addModule(PartialTranslationUnit );
@@ -56,6 +58,9 @@ class IncrementalExecutor {
getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const;
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
@@ -102,4 +124,97 @@ TEST(InterpreterExtensionsTest, FindRuntimeInterface) {
EXPECT_EQ(1U, Interp.RuntimeIBPtr->TransformerQueries);
}
+class CustomJBInterpreter : public Interpreter {
+ using CustomJITBuilderCreatorFunction =
+
std::function>()>;
+ CustomJITBuilderCreatorFunction JBCreator = nullptr;
+
+public:
+ CustomJBInterpreter(std::unique_ptr CI, llvm::Error
)
+ : Interpreter(std::move(CI), ErrOut) {}
+
+ ~CustomJBInterpreter() override {
+// Skip cleanUp() because it would trigger LLJIT default dtors
+Interpreter::ResetExecutor();
+ }
+
+ void setCustomJITBuilderCreator(CustomJITBuilderCreatorFunction Fn) {
+JBCreator = std::move(Fn);
+ }
+
+ llvm::Expected>
+ CreateJITBuilder(CompilerInstance ) override {
+if (JBCreator)
+ return JBCreator();
+return Interpreter::CreateJITBuilder(CI);
+ }
+
+ llvm::Error CreateExecutor() { return Interpreter::CreateExecutor(); }
+};
+
+static void initArmTarget() {
+ static llvm::once_flag F;
+ llvm::call_once(F, [] {
+LLVMInitializeARMTarget();
+LLVMInitializeARMTargetInfo();
+LLVMInitializeARMTargetMC();
+LLVMInitializeARMAsmPrinter();
+ });
+}
+
+llvm::llvm_shutdown_obj Shutdown;
+
+TEST(InterpreterExtensionsTest, DefaultCrossJIT) {
+ IncrementalCompilerBuilder CB;
+ CB.SetTargetTriple("armv6-none-eabi");
+ auto CI = cantFail(CB.CreateCpp());
+ llvm::Error ErrOut = llvm::Error::success();
+ CustomJBInterpreter Interp(std::move(CI), ErrOut);
+ cantFail(std::move(ErrOut));
+
+ initArmTarget();
+ cantFail(Interp.CreateExecutor());
+}
+
+TEST(InterpreterExtensionsTest, CustomCrossJIT) {
+ std::string TargetTriple = "armv6-none-eabi";
+
+ IncrementalCompilerBuilder CB;
+ CB.SetTargetTriple(TargetTriple);
+ auto CI = cantFail(CB.CreateCpp());
+ llvm::Error ErrOut = llvm::Error::success();
+ CustomJBInterpreter Interp(std::move(CI), ErrOut);
+ cantFail(std::move(ErrOut));
+
+ using namespace llvm::orc;
+ LLJIT *JIT = nullptr;
+ std::vector> Objs;
+ Interp.setCustomJITBuilderCreator([&]() {
+initArmTarget();
+auto JTMB = JITTargetMachineBuilder(llvm::Triple(TargetTriple));
+JTMB.setCPU("cortex-m0plus");
+auto JB = std::make_unique();
+JB->setJITTargetMachineBuilder(JTMB);
+JB->setPlatformSetUp(setUpInactivePlatform);
+JB->setNotifyCreatedCallback([&](LLJIT ) {
+ ObjectLayer = J.getObjLinkingLayer();
+ auto *JITLinkObjLayer = llvm::dyn_cast();
+ JITLinkObjLayer->setReturnObjectBuffer(
+ [](std::unique_ptr MB) {
+Objs.push_back(std::move(MB));
+ });
+ JIT =
+ return llvm::Error::success();
+});
+return JB;
+ });
+
+ EXPECT_EQ(0U, Objs.size());
+ cantFail(Interp.CreateExecutor());
+ cantFail(Interp.ParseAndExecute("int a = 1;"));
weliveindetail wrote:
This works cross-platform, because we don't actually execute code.
InactivePlatform suppresses execution of the respective initializers.
https://github.com/llvm/llvm-project/pull/84461
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
@@ -102,4 +124,97 @@ TEST(InterpreterExtensionsTest, FindRuntimeInterface) {
EXPECT_EQ(1U, Interp.RuntimeIBPtr->TransformerQueries);
}
+class CustomJBInterpreter : public Interpreter {
+ using CustomJITBuilderCreatorFunction =
+
std::function>()>;
+ CustomJITBuilderCreatorFunction JBCreator = nullptr;
+
+public:
+ CustomJBInterpreter(std::unique_ptr CI, llvm::Error
)
+ : Interpreter(std::move(CI), ErrOut) {}
+
+ ~CustomJBInterpreter() override {
+// Skip cleanUp() because it would trigger LLJIT default dtors
+Interpreter::ResetExecutor();
+ }
+
+ void setCustomJITBuilderCreator(CustomJITBuilderCreatorFunction Fn) {
+JBCreator = std::move(Fn);
+ }
+
+ llvm::Expected>
+ CreateJITBuilder(CompilerInstance ) override {
+if (JBCreator)
+ return JBCreator();
+return Interpreter::CreateJITBuilder(CI);
+ }
+
+ llvm::Error CreateExecutor() { return Interpreter::CreateExecutor(); }
+};
+
+static void initArmTarget() {
+ static llvm::once_flag F;
+ llvm::call_once(F, [] {
+LLVMInitializeARMTarget();
+LLVMInitializeARMTargetInfo();
+LLVMInitializeARMTargetMC();
+LLVMInitializeARMAsmPrinter();
+ });
+}
+
+llvm::llvm_shutdown_obj Shutdown;
+
+TEST(InterpreterExtensionsTest, DefaultCrossJIT) {
+ IncrementalCompilerBuilder CB;
+ CB.SetTargetTriple("armv6-none-eabi");
+ auto CI = cantFail(CB.CreateCpp());
+ llvm::Error ErrOut = llvm::Error::success();
+ CustomJBInterpreter Interp(std::move(CI), ErrOut);
+ cantFail(std::move(ErrOut));
+
+ initArmTarget();
+ cantFail(Interp.CreateExecutor());
+}
+
+TEST(InterpreterExtensionsTest, CustomCrossJIT) {
+ std::string TargetTriple = "armv6-none-eabi";
+
+ IncrementalCompilerBuilder CB;
+ CB.SetTargetTriple(TargetTriple);
+ auto CI = cantFail(CB.CreateCpp());
+ llvm::Error ErrOut = llvm::Error::success();
+ CustomJBInterpreter Interp(std::move(CI), ErrOut);
+ cantFail(std::move(ErrOut));
+
+ using namespace llvm::orc;
+ LLJIT *JIT = nullptr;
+ std::vector> Objs;
+ Interp.setCustomJITBuilderCreator([&]() {
+initArmTarget();
+auto JTMB = JITTargetMachineBuilder(llvm::Triple(TargetTriple));
+JTMB.setCPU("cortex-m0plus");
+auto JB = std::make_unique();
+JB->setJITTargetMachineBuilder(JTMB);
+JB->setPlatformSetUp(setUpInactivePlatform);
+JB->setNotifyCreatedCallback([&](LLJIT ) {
+ ObjectLayer = J.getObjLinkingLayer();
+ auto *JITLinkObjLayer = llvm::dyn_cast();
weliveindetail wrote:
We know that the ARM target uses JITLink, but we could also check explicitly.
It seems nice to check that we produce some actual code. Maybe we want checks
on the object code in the future? This might be a good template for such tests.
https://github.com/llvm/llvm-project/pull/84461
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail edited https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail commented: This PR depends on two predecessor patches, but I wanted to share it for review already. For the moment please refer to the single commit in https://github.com/llvm/llvm-project/pull/84461/commits/cc46034e5288ba54dfc8a0ea7d54792cbb99227d for review. https://github.com/llvm/llvm-project/pull/84461 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
@@ -373,21 +373,32 @@ Interpreter::Parse(llvm::StringRef Code) {
static llvm::Expected
createJITTargetMachineBuilder(const std::string ) {
if (TT == llvm::sys::getProcessTriple())
+// This fails immediately if the target backend is not registered
return llvm::orc::JITTargetMachineBuilder::detectHost();
- // FIXME: This can fail as well if the target is not registered! We just
don't
- // catch it yet.
+
+ // If the target backend is not registered, LLJITBuilder::create() will fail
return llvm::orc::JITTargetMachineBuilder(llvm::Triple(TT));
}
+llvm::Expected>
+Interpreter::CreateJITBuilder(CompilerInstance ) {
+ auto JTMB = createJITTargetMachineBuilder(CI.getTargetOpts().Triple);
+ if (!JTMB)
+return JTMB.takeError();
+ return IncrementalExecutor::createDefaultJITBuilder(std::move(*JTMB));
+}
+
llvm::Error Interpreter::CreateExecutor() {
- const clang::TargetInfo =
- getCompilerInstance()->getASTContext().getTargetInfo();
if (IncrExecutor)
return llvm::make_error("Operation failed. "
"Execution engine exists",
std::error_code());
+ llvm::Expected> JB =
+ CreateJITBuilder(*getCompilerInstance());
weliveindetail wrote:
We keep creating a new LLJITBuilder for each new IncrementalExecutor. This is
questionable, but I didn't want to clutter the patch unnecessarily. I am happy
to change that in a follow-up PR.
https://github.com/llvm/llvm-project/pull/84461
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
Stefan =?utf-8?q?Gränitz?= ,
Stefan =?utf-8?q?Gränitz?=
Message-ID:
In-Reply-To:
llvmbot wrote:
@llvm/pr-subscribers-clang
Author: Stefan Gränitz (weliveindetail)
Changes
The LLJITBuilder interface provides a very convenient way to configure the
ORCv2 JIT engine. IncrementalExecutor already used it internally to construct
the JIT, but didn't provide external access. This patch lifts control of the
creation process to the Interpreter and allows injection of a custom instance
through the extended interface. The Interpreter's default behavior remains
unchanged and the IncrementalExecutor remains an implementation detail.
---
Patch is 29.61 KiB, truncated to 20.00 KiB below, full version:
https://github.com/llvm/llvm-project/pull/84461.diff
6 Files Affected:
- (modified) clang/include/clang/Interpreter/Interpreter.h (+47-6)
- (modified) clang/lib/Interpreter/IncrementalExecutor.cpp (+18-15)
- (modified) clang/lib/Interpreter/IncrementalExecutor.h (+7-2)
- (modified) clang/lib/Interpreter/Interpreter.cpp (+174-107)
- (modified) clang/unittests/Interpreter/CMakeLists.txt (+2)
- (added) clang/unittests/Interpreter/InterpreterExtensionsTest.cpp (+220)
``diff
diff --git a/clang/include/clang/Interpreter/Interpreter.h
b/clang/include/clang/Interpreter/Interpreter.h
index c8f932e95c4798..8b47a899873715 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -18,6 +18,7 @@
#include "clang/AST/GlobalDecl.h"
#include "clang/Interpreter/PartialTranslationUnit.h"
#include "clang/Interpreter/Value.h"
+#include "clang/Sema/Ownership.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ExecutionEngine/JITSymbol.h"
@@ -28,7 +29,9 @@
namespace llvm {
namespace orc {
+class JITTargetMachineBuilder;
class LLJIT;
+class LLJITBuilder;
class ThreadSafeContext;
} // namespace orc
} // namespace llvm
@@ -75,18 +78,26 @@ class IncrementalCompilerBuilder {
llvm::StringRef CudaSDKPath;
};
+/// Generate glue code between the Interpreter's built-in runtime and user
code.
+class RuntimeInterfaceBuilder {
+public:
+ virtual ~RuntimeInterfaceBuilder() = default;
+
+ using TransformExprFunction = ExprResult(RuntimeInterfaceBuilder *Builder,
+ Expr *, ArrayRef);
+ virtual TransformExprFunction *getPrintValueTransformer() = 0;
+};
+
/// Provides top-level interfaces for incremental compilation and execution.
class Interpreter {
std::unique_ptr TSCtx;
std::unique_ptr IncrParser;
std::unique_ptr IncrExecutor;
+ std::unique_ptr RuntimeIB;
// An optional parser for CUDA offloading
std::unique_ptr DeviceParser;
- Interpreter(std::unique_ptr CI, llvm::Error );
-
- llvm::Error CreateExecutor();
unsigned InitPTUSize = 0;
// This member holds the last result of the value printing. It's a class
@@ -94,8 +105,40 @@ class Interpreter {
// printing happens, it's in an invalid state.
Value LastValue;
+ // Add a call to an Expr to report its result. We query the function from
+ // RuntimeInterfaceBuilder once and store it as a function pointer to avoid
+ // frequent virtual function calls.
+ RuntimeInterfaceBuilder::TransformExprFunction *AddPrintValueCall = nullptr;
+
+protected:
+ // Derived classes can make use an extended interface of the Interpreter.
+ // That's useful for testing and out-of-tree clients.
+ Interpreter(std::unique_ptr CI, llvm::Error );
+
+ // Create the internal IncrementalExecutor, or re-create it after calling
+ // ResetExecutor().
+ llvm::Error CreateExecutor();
+
+ // Delete the internal IncrementalExecutor. This causes a hard shutdown of
the
+ // JIT engine. In particular, it doesn't run cleanup or destructors.
+ void ResetExecutor();
+
+ // Lazily construct the RuntimeInterfaceBuilder. The provided instance will
be
+ // used for the entire lifetime of the interpreter. The default
implementation
+ // targets the in-process __clang_Interpreter runtime. Override this to use a
+ // custom runtime.
+ virtual std::unique_ptr FindRuntimeInterface();
+
+ // Lazily construct thev ORCv2 JITBuilder. This called when the internal
+ // IncrementalExecutor is created. The default implementation populates an
+ // in-process JIT with debugging support. Override this to configure the JIT
+ // engine used for execution.
+ virtual llvm::Expected>
+ CreateJITBuilder(CompilerInstance );
+
public:
- ~Interpreter();
+ virtual ~Interpreter();
+
static llvm::Expected>
create(std::unique_ptr CI);
static llvm::Expected>
@@ -143,8 +186,6 @@ class Interpreter {
private:
size_t getEffectivePTUSize() const;
- bool FindRuntimeInterface();
-
llvm::DenseMap Dtors;
llvm::SmallVector ValuePrintingInfo;
diff --git a/clang/lib/Interpreter/IncrementalExecutor.cpp
b/clang/lib/Interpreter/IncrementalExecutor.cpp
index 40bcef94797d43..6f036107c14a9c 100644
--- a/clang/lib/Interpreter/IncrementalExecutor.cpp
+++
[clang] [clang-repl] Factor out CreateJITBuilder() and allow specialization in derived classes (PR #84461)
https://github.com/weliveindetail created
https://github.com/llvm/llvm-project/pull/84461
The LLJITBuilder interface provides a very convenient way to configure the
ORCv2 JIT engine. IncrementalExecutor already used it internally to construct
the JIT, but didn't provide external access. This patch lifts control of the
creation process to the Interpreter and allows injection of a custom instance
through the extended interface. The Interpreter's default behavior remains
unchanged and the IncrementalExecutor remains an implementation detail.
From f87c7ccf59ad436087ada4be479102f7317d50ad Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Stefan=20Gr=C3=A4nitz?=
Date: Fri, 8 Mar 2024 00:08:56 +0100
Subject: [PATCH 1/3] [clang-repl] Expose RuntimeInterfaceBuilder for
customizations
---
clang/include/clang/Interpreter/Interpreter.h | 35 ++-
clang/lib/Interpreter/Interpreter.cpp | 247 ++
clang/unittests/Interpreter/CMakeLists.txt| 1 +
.../Interpreter/InterpreterExtensionsTest.cpp | 79 ++
4 files changed, 253 insertions(+), 109 deletions(-)
create mode 100644 clang/unittests/Interpreter/InterpreterExtensionsTest.cpp
diff --git a/clang/include/clang/Interpreter/Interpreter.h
b/clang/include/clang/Interpreter/Interpreter.h
index c8f932e95c4798..d972d960dcb7cd 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -18,6 +18,7 @@
#include "clang/AST/GlobalDecl.h"
#include "clang/Interpreter/PartialTranslationUnit.h"
#include "clang/Interpreter/Value.h"
+#include "clang/Sema/Ownership.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ExecutionEngine/JITSymbol.h"
@@ -75,17 +76,26 @@ class IncrementalCompilerBuilder {
llvm::StringRef CudaSDKPath;
};
+/// Generate glue code between the Interpreter's built-in runtime and user
code.
+class RuntimeInterfaceBuilder {
+public:
+ virtual ~RuntimeInterfaceBuilder() = default;
+
+ using TransformExprFunction = ExprResult(RuntimeInterfaceBuilder *Builder,
+ Expr *, ArrayRef);
+ virtual TransformExprFunction *getPrintValueTransformer() = 0;
+};
+
/// Provides top-level interfaces for incremental compilation and execution.
class Interpreter {
std::unique_ptr TSCtx;
std::unique_ptr IncrParser;
std::unique_ptr IncrExecutor;
+ std::unique_ptr RuntimeIB;
// An optional parser for CUDA offloading
std::unique_ptr DeviceParser;
- Interpreter(std::unique_ptr CI, llvm::Error );
-
llvm::Error CreateExecutor();
unsigned InitPTUSize = 0;
@@ -94,8 +104,25 @@ class Interpreter {
// printing happens, it's in an invalid state.
Value LastValue;
+ // Add a call to an Expr to report its result. We query the function from
+ // RuntimeInterfaceBuilder once and store it as a function pointer to avoid
+ // frequent virtual function calls.
+ RuntimeInterfaceBuilder::TransformExprFunction *AddPrintValueCall = nullptr;
+
+protected:
+ // Derived classes can make use an extended interface of the Interpreter.
+ // That's useful for testing and out-of-tree clients.
+ Interpreter(std::unique_ptr CI, llvm::Error );
+
+ // Lazily construct the RuntimeInterfaceBuilder. The provided instance will
be
+ // used for the entire lifetime of the interpreter. The default
implementation
+ // targets the in-process __clang_Interpreter runtime. Override this to use a
+ // custom runtime.
+ virtual std::unique_ptr FindRuntimeInterface();
+
public:
- ~Interpreter();
+ virtual ~Interpreter();
+
static llvm::Expected>
create(std::unique_ptr CI);
static llvm::Expected>
@@ -143,8 +170,6 @@ class Interpreter {
private:
size_t getEffectivePTUSize() const;
- bool FindRuntimeInterface();
-
llvm::DenseMap Dtors;
llvm::SmallVector ValuePrintingInfo;
diff --git a/clang/lib/Interpreter/Interpreter.cpp
b/clang/lib/Interpreter/Interpreter.cpp
index 37696b28976428..3485da8196683a 100644
--- a/clang/lib/Interpreter/Interpreter.cpp
+++ b/clang/lib/Interpreter/Interpreter.cpp
@@ -507,9 +507,13 @@ static constexpr llvm::StringRef MagicRuntimeInterface[] =
{
"__clang_Interpreter_SetValueWithAlloc",
"__clang_Interpreter_SetValueCopyArr", "__ci_newtag"};
-bool Interpreter::FindRuntimeInterface() {
+static std::unique_ptr
+createInProcessRuntimeInterfaceBuilder(Interpreter , ASTContext ,
+ Sema );
+
+std::unique_ptr Interpreter::FindRuntimeInterface() {
if (llvm::all_of(ValuePrintingInfo, [](Expr *E) { return E != nullptr; }))
-return true;
+return nullptr;
Sema = getCompilerInstance()->getSema();
ASTContext = S.getASTContext();
@@ -528,120 +532,34 @@ bool Interpreter::FindRuntimeInterface() {
if (!LookupInterface(ValuePrintingInfo[NoAlloc],
MagicRuntimeInterface[NoAlloc]))
-return false;
+return nullptr;
if (!LookupInterface(ValuePrintingInfo[WithAlloc],
