[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -1744,6 +1744,273 @@ a call to ``llvm.coro.suspend.retcon`` after resuming abnormally. In a yield-once coroutine, it is undefined behavior if the coroutine executes a call to ``llvm.coro.suspend.retcon`` after resuming in any way. +.. _coro.await.suspend: + +'llvm.coro.await.suspend' Intrinsic +^^ +:: + + declare void @llvm.coro.await.suspend( +ptr , +ptr , +ptr ) + +Overview: +" + +The '``llvm.coro.await.suspend``' intrinsic hides C++ `await-suspend` +block code from optimizations on presplit coroutine body +to avoid miscompilations. This version of intrinsic corresponds to +'``void awaiter.await_suspend(...)``' variant. + +Arguments: +"" + +The first argument is a pointer to `awaiter` object. + +The second argument is a pointer to the current coroutine's frame. + +The third argument is a pointer to the helper function encapsulating +`await-suspend` logic. Its signature must be + +.. code-block:: llvm + +declare void @await_suspend_function(ptr %awaiter, ptr %hdl) + +Semantics: +"" + +The intrinsic must be used between corresponding `coro.save`_ and +`coro.suspend`_ calls. It is lowered to an inlined +`await_suspend_function` call during `CoroSplit`_ pass. + +Example: + + +.. code-block:: llvm + + ; before lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +call void @llvm.coro.await.suspend( +ptr %awaiter, +ptr %hdl, +ptr @await_suspend_function) +%suspend = call i8 @llvm.coro.suspend(token %save, i1 false) +... + + ; after lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +; the call to await_suspend_function is inlined +call void @await_suspend_function( +ptr %awaiter, +ptr %hdl) +%suspend = call i8 @llvm.coro.suspend(token %save, i1 false) +... + + ; helper function example + define void @await_suspend_function(ptr %awaiter, ptr %hdl) +entry: + %hdl.tmp = alloca %"struct.std::coroutine_handle" + %hdl.result.tmp = alloca %"struct.std::coroutine_handle" + %hdl.promise.tmp = alloca %"struct.std::coroutine_handle.0" + %hdl.promise = call ptr @"std::corouine_handle::from_address"(ptr %hdl) + %hdl.promise.tmp.dive = getelementptr inbounds %"struct.std::coroutine_handle.0", +ptr %hdl.promise.tmp, i32 0, i32 0 + %hdl.promise.tmp.dive2 = getelementptr inbounds %"struct.std::coroutine_handle", +ptr %hdl.promise.tmp.dive, i32 0, i32 0 + store ptr %hdl.promise, ptr %hdl.promise.tmp.dive2 + call void @llvm.memcpy.p0.p0.i64(ptr %hdl.tmp, ptr %hdl.promise.tmp, i64 8, i1 false) + %hdl.tmp.dive = getelementptr inbounds %"struct.std::coroutine_handle", +ptr %hdl.tmp, i32 0, i32 0 + %hdl.arg = load ptr, ptr %hdl.tmp.dive + call void @"Awaiter::await_suspend"(ptr %awaiter, ptr %hdl.arg) + ret void + +.. _coro.await.suspend.bool: + +'llvm.coro.await.suspend.bool' Intrinsic +^^ +:: + + declare i1 @llvm.coro.await.suspend.bool( +ptr , +ptr , +ptr ) + +Overview: +" + +The '``llvm.coro.await.suspend.bool``' intrinsic hides C++ `await-suspend` +block code from optimizations on presplit coroutine body +to avoid miscompilations. This version of intrinsic corresponds to +'``bool awaiter.await_suspend(...)``' variant. + +Arguments: +"" + +The first argument is a pointer to `awaiter` object. + +The second argument is a pointer to the current coroutine's frame. + +The third argument is a pointer to the helper function encapsulating +`await-suspend` logic. Its signature must be + +.. code-block:: llvm + +declare i1 @await_suspend_function(ptr %awaiter, ptr %hdl) + +Semantics: +"" + +The intrinsic must be used between corresponding `coro.save`_ and +`coro.suspend`_ calls. It is lowered to an inlined +`await_suspend_function` call during `CoroSplit`_ pass. + +If `await_suspend_function` call returns `true`, the current coroutine is +immediately resumed. + +Example: + + +.. code-block:: llvm + + ; before lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +%resume = call i1 @llvm.coro.await.suspend( +ptr %awaiter, +ptr %hdl, +ptr @await_suspend_function) +br i1 %resume, %await.suspend.bool, %await.ready + await.suspend.bool: +%suspend = call i8 @llvm.coro.suspend(token %save, i1 false) +... + await.ready: +call void @"Awaiter::await_ready"(ptr %awaiter) +... + + ; after lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +; the call to await_suspend_function is inlined +%resume = call i1 @await_suspend_function( +ptr %awaiter, +ptr %hdl) +
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -79,6 +79,73 @@ using namespace llvm;
namespace {
+// Created on demand if the coro-early pass has work to do.
+class Lowerer : public coro::LowererBase {
+ IRBuilder<> Builder;
+ void lowerAwaitSuspend(CoroAwaitSuspendInst *CB);
+
+public:
+ Lowerer(Module ) : LowererBase(M), Builder(Context) {}
+
+ void lowerAwaitSuspends(Function );
+};
+
+void Lowerer::lowerAwaitSuspend(CoroAwaitSuspendInst *CB) {
+ auto Helper = CB->getHelperFunction();
+ auto Awaiter = CB->getAwaiter();
+ auto FramePtr = CB->getFrame();
+
+ Builder.SetInsertPoint(CB);
+
+ CallBase *NewCall = nullptr;
+ if (auto Invoke = dyn_cast(CB)) {
+auto HelperInvoke =
+Builder.CreateInvoke(Helper, Invoke->getNormalDest(),
+ Invoke->getUnwindDest(), {Awaiter, FramePtr});
+
+HelperInvoke->setCallingConv(Invoke->getCallingConv());
+std::copy(Invoke->bundle_op_info_begin(), Invoke->bundle_op_info_end(),
+ HelperInvoke->bundle_op_info_begin());
+AttributeList NewAttributes =
+Invoke->getAttributes().removeParamAttributes(Context, 2);
+HelperInvoke->setAttributes(NewAttributes);
+HelperInvoke->setDebugLoc(Invoke->getDebugLoc());
+NewCall = HelperInvoke;
+ } else if (auto Call = dyn_cast(CB)) {
+auto HelperCall = Builder.CreateCall(Helper, {Awaiter, FramePtr});
+
+AttributeList NewAttributes =
+Call->getAttributes().removeParamAttributes(Context, 2);
+HelperCall->setAttributes(NewAttributes);
+HelperCall->setDebugLoc(Call->getDebugLoc());
+NewCall = HelperCall;
+ }
+
+ CB->replaceAllUsesWith(NewCall);
+ CB->eraseFromParent();
+
+ InlineFunctionInfo FnInfo;
+ auto InlineRes = InlineFunction(*NewCall, FnInfo);
+ assert(InlineRes.isSuccess() && "Expected inlining to succeed");
+ (void)InlineRes;
+}
+
+void Lowerer::lowerAwaitSuspends(Function ) {
fpasserby wrote:
Moved collection to the construction of Shape
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -79,6 +79,73 @@ using namespace llvm;
namespace {
+// Created on demand if the coro-early pass has work to do.
+class Lowerer : public coro::LowererBase {
+ IRBuilder<> Builder;
+ void lowerAwaitSuspend(CoroAwaitSuspendInst *CB);
+
+public:
+ Lowerer(Module ) : LowererBase(M), Builder(Context) {}
+
+ void lowerAwaitSuspends(Function );
+};
+
+void Lowerer::lowerAwaitSuspend(CoroAwaitSuspendInst *CB) {
+ auto Helper = CB->getHelperFunction();
+ auto Awaiter = CB->getAwaiter();
+ auto FramePtr = CB->getFrame();
+
+ Builder.SetInsertPoint(CB);
+
+ CallBase *NewCall = nullptr;
+ if (auto Invoke = dyn_cast(CB)) {
+auto HelperInvoke =
+Builder.CreateInvoke(Helper, Invoke->getNormalDest(),
+ Invoke->getUnwindDest(), {Awaiter, FramePtr});
+
+HelperInvoke->setCallingConv(Invoke->getCallingConv());
+std::copy(Invoke->bundle_op_info_begin(), Invoke->bundle_op_info_end(),
+ HelperInvoke->bundle_op_info_begin());
+AttributeList NewAttributes =
+Invoke->getAttributes().removeParamAttributes(Context, 2);
+HelperInvoke->setAttributes(NewAttributes);
+HelperInvoke->setDebugLoc(Invoke->getDebugLoc());
+NewCall = HelperInvoke;
+ } else if (auto Call = dyn_cast(CB)) {
+auto HelperCall = Builder.CreateCall(Helper, {Awaiter, FramePtr});
+
+AttributeList NewAttributes =
+Call->getAttributes().removeParamAttributes(Context, 2);
+HelperCall->setAttributes(NewAttributes);
+HelperCall->setDebugLoc(Call->getDebugLoc());
+NewCall = HelperCall;
+ }
+
+ CB->replaceAllUsesWith(NewCall);
+ CB->eraseFromParent();
+
+ InlineFunctionInfo FnInfo;
+ auto InlineRes = InlineFunction(*NewCall, FnInfo);
+ assert(InlineRes.isSuccess() && "Expected inlining to succeed");
+ (void)InlineRes;
fpasserby wrote:
Removed manual inlining
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -5036,14 +5036,17 @@ class CoroutineSuspendExpr : public Expr {
Stmt *SubExprs[SubExpr::Count];
OpaqueValueExpr *OpaqueValue = nullptr;
+ OpaqueValueExpr *OpaqueFramePtr = nullptr;
fpasserby wrote:
Now `__builtin_coro_frame` is used.
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
https://github.com/fpasserby edited https://github.com/llvm/llvm-project/pull/79712 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
ChuanqiXu9 wrote: BTW, in my original comment, I said: > Then in the middle end, we can perform analysis the awaitSuspendFn to see if > the coroutine handle escapes or not. If it won't escape, we can replace above > intrinsic to the direct call. Otherwise, we will only convert them in the > CoroSplit phase. But this patch only implements the otherwise part. This is not a blocking issue. But we need a TODO or a FIXME here. https://github.com/llvm/llvm-project/pull/79712 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -338,6 +386,85 @@ static QualType getCoroutineSuspendExprReturnType(const
ASTContext ,
}
#endif
+llvm::Function *CodeGenFunction::generateAwaitSuspendHelper(
+Twine const , Twine const ,
+CoroutineSuspendExpr const , bool CanThrow) {
+ std::string FuncName = "__await_suspend_helper_";
+ FuncName += CoroName.str();
+ FuncName += '_';
+ FuncName += SuspendPointName.str();
+
+ ASTContext = getContext();
+
+ FunctionArgList args;
+
+ ImplicitParamDecl AwaiterDecl(C, C.VoidPtrTy, ImplicitParamKind::Other);
+ ImplicitParamDecl FrameDecl(C, C.VoidPtrTy, ImplicitParamKind::Other);
+ QualType ReturnTy = S.getSuspendExpr()->getType();
+
+ if (ReturnTy->isBooleanType()) {
+ReturnTy = C.BoolTy;
+ } else if (ReturnTy->isVoidPointerType()) {
+ReturnTy = C.VoidPtrTy;
+ } else {
+ReturnTy = C.VoidTy;
+ }
+
+ args.push_back();
+ args.push_back();
+
+ const CGFunctionInfo =
+ CGM.getTypes().arrangeBuiltinFunctionDeclaration(ReturnTy, args);
+
+ llvm::FunctionType *LTy = CGM.getTypes().GetFunctionType(FI);
+
+ llvm::Function *Fn = llvm::Function::Create(
+ LTy, llvm::GlobalValue::PrivateLinkage, FuncName, ());
+
+ Fn->addParamAttr(0, llvm::Attribute::AttrKind::NonNull);
+ Fn->addParamAttr(0, llvm::Attribute::AttrKind::NoUndef);
+
+ Fn->addParamAttr(1, llvm::Attribute::AttrKind::NoUndef);
+
+ Fn->setMustProgress();
+ Fn->addFnAttr(llvm::Attribute::AttrKind::AlwaysInline);
+
+ if (!CanThrow) {
+Fn->addFnAttr(llvm::Attribute::AttrKind::NoUnwind);
+ }
ChuanqiXu9 wrote:
We don't need this. If I remember correctly, LLVM can add it automatically as
long as the await_suspend is marked noexcept.
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
https://github.com/ChuanqiXu9 edited https://github.com/llvm/llvm-project/pull/79712 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -1553,6 +1625,9 @@ static bool hasCallsInBlocksBetween(BasicBlock *SaveBB,
BasicBlock *ResDesBB) {
}
static bool hasCallsBetween(Instruction *Save, Instruction *ResumeOrDestroy) {
+ if (Save == ResumeOrDestroy)
+return false;
ChuanqiXu9 wrote:
How can this be true?
Oh, I see you use it in `isSimpleHelper`.
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -79,6 +79,73 @@ using namespace llvm;
namespace {
+// Created on demand if the coro-early pass has work to do.
+class Lowerer : public coro::LowererBase {
+ IRBuilder<> Builder;
+ void lowerAwaitSuspend(CoroAwaitSuspendInst *CB);
+
+public:
+ Lowerer(Module ) : LowererBase(M), Builder(Context) {}
+
+ void lowerAwaitSuspends(Function );
+};
+
+void Lowerer::lowerAwaitSuspend(CoroAwaitSuspendInst *CB) {
+ auto Helper = CB->getHelperFunction();
+ auto Awaiter = CB->getAwaiter();
+ auto FramePtr = CB->getFrame();
+
+ Builder.SetInsertPoint(CB);
+
+ CallBase *NewCall = nullptr;
+ if (auto Invoke = dyn_cast(CB)) {
+auto HelperInvoke =
+Builder.CreateInvoke(Helper, Invoke->getNormalDest(),
+ Invoke->getUnwindDest(), {Awaiter, FramePtr});
+
+HelperInvoke->setCallingConv(Invoke->getCallingConv());
+std::copy(Invoke->bundle_op_info_begin(), Invoke->bundle_op_info_end(),
+ HelperInvoke->bundle_op_info_begin());
+AttributeList NewAttributes =
+Invoke->getAttributes().removeParamAttributes(Context, 2);
+HelperInvoke->setAttributes(NewAttributes);
+HelperInvoke->setDebugLoc(Invoke->getDebugLoc());
+NewCall = HelperInvoke;
+ } else if (auto Call = dyn_cast(CB)) {
+auto HelperCall = Builder.CreateCall(Helper, {Awaiter, FramePtr});
+
+AttributeList NewAttributes =
+Call->getAttributes().removeParamAttributes(Context, 2);
+HelperCall->setAttributes(NewAttributes);
+HelperCall->setDebugLoc(Call->getDebugLoc());
+NewCall = HelperCall;
+ }
+
+ CB->replaceAllUsesWith(NewCall);
+ CB->eraseFromParent();
+
+ InlineFunctionInfo FnInfo;
+ auto InlineRes = InlineFunction(*NewCall, FnInfo);
+ assert(InlineRes.isSuccess() && "Expected inlining to succeed");
+ (void)InlineRes;
+}
+
+void Lowerer::lowerAwaitSuspends(Function ) {
ChuanqiXu9 wrote:
We can collect CoroAwaitSuspendInst during the construction of Shape. So that
we can save one iterations of the function.
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -1744,6 +1744,273 @@ a call to ``llvm.coro.suspend.retcon`` after resuming abnormally. In a yield-once coroutine, it is undefined behavior if the coroutine executes a call to ``llvm.coro.suspend.retcon`` after resuming in any way. +.. _coro.await.suspend: + +'llvm.coro.await.suspend' Intrinsic +^^ +:: + + declare void @llvm.coro.await.suspend( +ptr , +ptr , +ptr ) + +Overview: +" + +The '``llvm.coro.await.suspend``' intrinsic hides C++ `await-suspend` +block code from optimizations on presplit coroutine body +to avoid miscompilations. This version of intrinsic corresponds to +'``void awaiter.await_suspend(...)``' variant. + +Arguments: +"" + +The first argument is a pointer to `awaiter` object. + +The second argument is a pointer to the current coroutine's frame. + +The third argument is a pointer to the helper function encapsulating +`await-suspend` logic. Its signature must be + +.. code-block:: llvm + +declare void @await_suspend_function(ptr %awaiter, ptr %hdl) + +Semantics: +"" + +The intrinsic must be used between corresponding `coro.save`_ and +`coro.suspend`_ calls. It is lowered to an inlined +`await_suspend_function` call during `CoroSplit`_ pass. + +Example: + + +.. code-block:: llvm + + ; before lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +call void @llvm.coro.await.suspend( +ptr %awaiter, +ptr %hdl, +ptr @await_suspend_function) +%suspend = call i8 @llvm.coro.suspend(token %save, i1 false) +... + + ; after lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +; the call to await_suspend_function is inlined +call void @await_suspend_function( +ptr %awaiter, +ptr %hdl) +%suspend = call i8 @llvm.coro.suspend(token %save, i1 false) +... + + ; helper function example + define void @await_suspend_function(ptr %awaiter, ptr %hdl) +entry: + %hdl.tmp = alloca %"struct.std::coroutine_handle" + %hdl.result.tmp = alloca %"struct.std::coroutine_handle" + %hdl.promise.tmp = alloca %"struct.std::coroutine_handle.0" + %hdl.promise = call ptr @"std::corouine_handle::from_address"(ptr %hdl) + %hdl.promise.tmp.dive = getelementptr inbounds %"struct.std::coroutine_handle.0", +ptr %hdl.promise.tmp, i32 0, i32 0 + %hdl.promise.tmp.dive2 = getelementptr inbounds %"struct.std::coroutine_handle", +ptr %hdl.promise.tmp.dive, i32 0, i32 0 + store ptr %hdl.promise, ptr %hdl.promise.tmp.dive2 + call void @llvm.memcpy.p0.p0.i64(ptr %hdl.tmp, ptr %hdl.promise.tmp, i64 8, i1 false) + %hdl.tmp.dive = getelementptr inbounds %"struct.std::coroutine_handle", +ptr %hdl.tmp, i32 0, i32 0 + %hdl.arg = load ptr, ptr %hdl.tmp.dive + call void @"Awaiter::await_suspend"(ptr %awaiter, ptr %hdl.arg) + ret void + +.. _coro.await.suspend.bool: + +'llvm.coro.await.suspend.bool' Intrinsic +^^ +:: + + declare i1 @llvm.coro.await.suspend.bool( +ptr , +ptr , +ptr ) + +Overview: +" + +The '``llvm.coro.await.suspend.bool``' intrinsic hides C++ `await-suspend` +block code from optimizations on presplit coroutine body +to avoid miscompilations. This version of intrinsic corresponds to +'``bool awaiter.await_suspend(...)``' variant. + +Arguments: +"" + +The first argument is a pointer to `awaiter` object. + +The second argument is a pointer to the current coroutine's frame. + +The third argument is a pointer to the helper function encapsulating +`await-suspend` logic. Its signature must be + +.. code-block:: llvm + +declare i1 @await_suspend_function(ptr %awaiter, ptr %hdl) + +Semantics: +"" + +The intrinsic must be used between corresponding `coro.save`_ and +`coro.suspend`_ calls. It is lowered to an inlined +`await_suspend_function` call during `CoroSplit`_ pass. + +If `await_suspend_function` call returns `true`, the current coroutine is +immediately resumed. + +Example: + + +.. code-block:: llvm + + ; before lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +%resume = call i1 @llvm.coro.await.suspend( +ptr %awaiter, +ptr %hdl, +ptr @await_suspend_function) +br i1 %resume, %await.suspend.bool, %await.ready + await.suspend.bool: +%suspend = call i8 @llvm.coro.suspend(token %save, i1 false) +... + await.ready: +call void @"Awaiter::await_ready"(ptr %awaiter) +... + + ; after lowering + await.suspend: +%save = call token @llvm.coro.save(ptr %hdl) +; the call to await_suspend_function is inlined +%resume = call i1 @await_suspend_function( +ptr %awaiter, +ptr %hdl) +
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -5036,14 +5036,17 @@ class CoroutineSuspendExpr : public Expr {
Stmt *SubExprs[SubExpr::Count];
OpaqueValueExpr *OpaqueValue = nullptr;
+ OpaqueValueExpr *OpaqueFramePtr = nullptr;
ChuanqiXu9 wrote:
I still think we can get rid of storing `OpaqueFramePtr` in
`CoroutineSuspendExpr`. For example, we can call ` @llvm.coro.frame()` directly
to get it.
https://llvm.org/docs/Coroutines.html#llvm-coro-frame-intrinsic
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
@@ -232,16 +237,59 @@ static LValueOrRValue
emitSuspendExpression(CodeGenFunction , CGCoroData
auto *NullPtr = llvm::ConstantPointerNull::get(CGF.CGM.Int8PtrTy);
auto *SaveCall = Builder.CreateCall(CoroSave, {NullPtr});
- CGF.CurCoro.InSuspendBlock = true;
- auto *SuspendRet = CGF.EmitScalarExpr(S.getSuspendExpr());
- CGF.CurCoro.InSuspendBlock = false;
+ const auto AwaitSuspendCanThrow =
+ AwaitSuspendStmtCanThrow(S.getSuspendExpr());
+
+ auto SuspendHelper = CodeGenFunction(CGF.CGM).generateAwaitSuspendHelper(
+ CGF.CurFn->getName(), Prefix, S, AwaitSuspendCanThrow);
- if (SuspendRet != nullptr && SuspendRet->getType()->isIntegerTy(1)) {
-// Veto suspension if requested by bool returning await_suspend.
-BasicBlock *RealSuspendBlock =
-CGF.createBasicBlock(Prefix + Twine(".suspend.bool"));
-CGF.Builder.CreateCondBr(SuspendRet, RealSuspendBlock, ReadyBlock);
-CGF.EmitBlock(RealSuspendBlock);
+ llvm::CallBase *SuspendRet = nullptr;
+
+ {
+CGF.CurCoro.InSuspendBlock = true;
+
+auto FramePtrBinder = CodeGenFunction::OpaqueValueMappingData::bind(
+CGF, S.getOpaqueFramePtr(), S.getOpaqueFramePtr()->getSourceExpr());
+auto UnbindFramePtrOnExit =
+llvm::make_scope_exit([&] { FramePtrBinder.unbind(CGF); });
+
+SmallVector SuspendHelperCallArgs;
+SuspendHelperCallArgs.push_back(
+
CGF.getOrCreateOpaqueLValueMapping(S.getOpaqueValue()).getPointer(CGF));
+SuspendHelperCallArgs.push_back(
+CGF.getOrCreateOpaqueRValueMapping(S.getOpaqueFramePtr())
+.getScalarVal());
+SuspendHelperCallArgs.push_back(SuspendHelper);
+
+auto IID = llvm::Intrinsic::coro_await_suspend;
+if (S.getSuspendExpr()->getType()->isBooleanType())
+ IID = llvm::Intrinsic::coro_await_suspend_bool;
+else if (S.getSuspendExpr()->getType()->isVoidPointerType())
+ IID = llvm::Intrinsic::coro_await_suspend_handle;
+
+llvm::Function *AwaitSuspendIntrinsic = CGF.CGM.getIntrinsic(IID);
+// FIXME: add call attributes?
+if (AwaitSuspendCanThrow)
+ SuspendRet =
+ CGF.EmitCallOrInvoke(AwaitSuspendIntrinsic, SuspendHelperCallArgs);
+else
+ SuspendRet = CGF.EmitNounwindRuntimeCall(AwaitSuspendIntrinsic,
+ SuspendHelperCallArgs);
+
+CGF.CurCoro.InSuspendBlock = false;
+ }
+
+ if (SuspendRet != nullptr) {
+if (SuspendRet->getType()->isIntegerTy(1)) {
+ // Veto suspension if requested by bool returning await_suspend.
+ BasicBlock *RealSuspendBlock =
+ CGF.createBasicBlock(Prefix + Twine(".suspend.bool"));
+ CGF.Builder.CreateCondBr(SuspendRet, RealSuspendBlock, ReadyBlock);
+ CGF.EmitBlock(RealSuspendBlock);
+} else if (SuspendRet->getType()->isPointerTy()) {
+ auto ResumeIntrinsic =
CGF.CGM.getIntrinsic(llvm::Intrinsic::coro_resume);
+ Builder.CreateCall(ResumeIntrinsic, SuspendRet);
+}
ChuanqiXu9 wrote:
```suggestion
} else
llvm_unreachable(...);
```
nit: personal interest.
https://github.com/llvm/llvm-project/pull/79712
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
https://github.com/ChuanqiXu9 edited https://github.com/llvm/llvm-project/pull/79712 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [coroutine] Implement llvm.coro.await.suspend intrinsic (PR #79712)
https://github.com/ChuanqiXu9 commented: Thanks for looking into this. I haven't looked it in details. Given this is complex, it should take a relative longer time. Here is some quick feedbacks: - Every time we change the non-static data members of AST nodes, we need to update the serializations. Otherwise, modules won't work. - Every time we change the LLVM intrinsics, we need to change the documentation. - Every time we change the LLVM with a functional change, we need a test in the LLVM part. And there is something I haven't fully understand: - I feel odd about `helperFunction`. I don't understand the necessity and I feel we should make it in the middle end. - I feel odd about `IsSuspendNoThrow`. Can't we make it simply during the CodeGen time by the attribute of `await_suspend`?. - I am wondering if we can get rid of the introduction of `OpaqueFramePtr` from `CoroutineSuspendExpr` either in CodeGen part or in Sema part. But I am not sure if my feelings are correct. I need more time looking into it. https://github.com/llvm/llvm-project/pull/79712 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
