Hello Dylan and Mark!
On 5/12/2026 5:00 AM, Dylan Hatch wrote:
> On Fri, May 1, 2026 at 9:46 AM Mark Rutland <[email protected]>
> wrote:
>> More generally, there are a few things that aren't addressed by
>> this series that we will also need to address. Importantly:
>>
>> (1) For correctness, we'll need to address a latent issue with
>> unwinding across an fgraph return trampoline, where the return
>> address is transiently unrecoverable.
>>
>> Before this series, that doesn't matter for livepatching because
>> the livepatching code isn't called synchronously within the fgraph
>> handler, and unwinds which cross an exception boundary are marked
>> as unreliable.
>>
>> After this series, that does matter as we can unwind across an
>> exception boundary, and might happen to interrupt that transient
>> window.
>>
>> I think we can solve that with some restructuring of that code,
>> restoring the original address *before* removing that from the
>> fgraph return stack, and ensuring that the unwinder can find it.
>
> If my understanding is correct, the issue arrises in
> return_to_handler as the return address is recovered:
>
> mov x0, sp bl ftrace_return_to_handler // addr =
> ftrace_return_to_hander(fregs); mov x30, x0 // restore the original
> return address
>
> Because ftrace_return_to_handler pops the return address from the
> return stack before it can be restored into the LR, it cannot be
> recovered.
Based on reliable-stacktrace.rst section "4.4 Rewriting of return
addresses" I wonder whether the following might work:
- If an unwound RA points at return_to_handler the actual RA needs to
be obtained using ftrace_graph_ret_addr(). This might already be
taken into account if ftrace_graph_ret_addr() is used unconditionally.
- If an unwound RA points into return_to_handler() mark the stack trace
as unreliable. This could be accomplished by marking LR in
return_to_handler() as undefined (i.e. .cfi_undefined 30) to use
SFrame's outermost frame indication to stop and mark the stack trace
as unreliable:
diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
@@ -329,8 +329,12 @@ SYM_FUNC_END(ftrace_stub_graph)
* @fp is checked against the value passed by ftrace_graph_caller().
*/
SYM_CODE_START(return_to_handler)
+ .cfi_startproc
+ /* Mark unwinding of LR as unreliable */
+ .cfi_undefined 30
/* Make room for ftrace_regs */
sub sp, sp, #FREGS_SIZE
+ .cfi_adjust_cfa_offset -FREGS_SIZE
/* Save return value regs */
stp x0, x1, [sp, #FREGS_X0]
@@ -344,6 +348,8 @@ SYM_CODE_START(return_to_handler)
mov x0, sp
bl ftrace_return_to_handler // addr =
ftrace_return_to_hander(fregs);
mov x30, x0 // restore the original return
address
+ /* Mark unwinding of LR as reliable */
+ .cfi_restore 30
/* Restore return value regs */
ldp x0, x1, [sp, #FREGS_X0]
@@ -351,7 +357,9 @@ SYM_CODE_START(return_to_handler)
ldp x4, x5, [sp, #FREGS_X4]
ldp x6, x7, [sp, #FREGS_X6]
add sp, sp, #FREGS_SIZE
+ .cfi_adjust_cfa_offset FREGS_SIZE
ret
+ .cfi_endproc
SYM_CODE_END(return_to_handler)
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
Notes regarding above:
- return_to_handler() saves the caller's FP in ftrace_regs but never
restores it. I suppose this is because ftrace_regs is an input to
ftrace_return_to_handler(). The DWARF CFI above assumes SP and FP
can be restored at all times as SP=CFA and FP=FP.
- One might be tempted to add a .cfi_register 30, 0 after the call to
ftrace_return_to_handler. This would be wrong, because if unwinding
comes from ftrace_return_to_handler() the unwound RA will point there
and the unwinding logic would erroneously assume x0 to contain the RA.
- The DWARF CFI could be simplified as follows to just convey that
unwinding through return_to_handler is impossible at all times:
diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
@@ -329,6 +329,9 @@ SYM_FUNC_END(ftrace_stub_graph)
* @fp is checked against the value passed by ftrace_graph_caller().
*/
SYM_CODE_START(return_to_handler)
+ .cfi_startproc simple
+ /* Mark unwinding of LR as unreliable */
+ .cfi_undefined 30
/* Make room for ftrace_regs */
sub sp, sp, #FREGS_SIZE
@@ -353,5 +356,6 @@ SYM_CODE_START(return_to_handler)
add sp, sp, #FREGS_SIZE
ret
+ .cfi_endproc
SYM_CODE_END(return_to_handler)
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>
> Based on this, I believe you are suggesting to restructure this code
> path such that the return address is removed from the return stack
> only after it has been restored to LR. But since kernel/trace/
> fgraph.c is core kernel code, will this end up requiring either (1)
> a similar restructuring of other arches supporting ftrace, or (2) an
> arm64-specific implementation of this recovery logic?
>
> It looks to me like there is essentially the same recovery pattern
> on other arches; is there a reason this transient unrecoverability
> isn't an issue for reliable unwind on other platforms?
>
>>
>> I'm not immediately sure whether kretprobes has a similar issue.
Regards,
Jens
--
Jens Remus
Linux on Z Development (D3303)
[email protected] / [email protected]
IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats:
Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft:
Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/
