On 1/15/19 11:34 AM, Sean Christopherson wrote:
> On Tue, Jan 15, 2019 at 08:13:22AM +0100, Paolo Bonzini wrote:
>> On 15/01/19 08:04, Qian Cai wrote:
>>>
>>>
>>> On 1/15/19 1:44 AM, Qian Cai wrote:
>>>> compilation warning since v5.0-rc1,
>>>>
>>>> arch/x86/kvm/vmx/vmx.o: warning: objtool: vmx_vcpu_run.part.17()+0x3171:
>>>> call without frame pointer save/setup
>
> The warning is complaining about vmx_vcpu_run() in vmx.c, not vmenter.S.
> The rule being "broken" is that a call is made without creating a stack
> frame, and vmx_vmenter() obviously makes no calls.
>
> E.g., manually running objtool check:
>
> $ tools/objtool/objtool check arch/x86/kvm/vmx/vmenter.o
> $ tools/objtool/objtool check arch/x86/kvm/vmx/vmx.o
> arch/x86/kvm/vmx/vmx.o: warning: objtool: vmx_vcpu_run.part.19()+0x83e:
> call without frame pointer save/setup
>
> I put "broken" in quotes because AFAICT we're not actually violating the
> rule. From tools/objtool/Documentation/stack-validation.txt:
>
> If it's a GCC-compiled .c file, the error may be because the function
> uses an inline asm() statement which has a "call" instruction. An
> asm() statement with a call instruction must declare the use of the
> stack pointer in its output operand. On x86_64, this means adding
> the ASM_CALL_CONSTRAINT as an output constraint:
>
> asm volatile("call func" : ASM_CALL_CONSTRAINT);
>
> Otherwise the stack frame may not get created before the call.
>
>
> The asm() blob that calls vmx_vmenter() uses ASM_CALL_CONSTRAINT, and
> the resulting asm output generates a frame pointer, e.g. this is from
> the vmx.o that objtool warns on:
>
> Dump of assembler code for function vmx_vcpu_run:
> 0x0000000000007440 <+0>: e8 00 00 00 00 callq 0x7445 <vmx_vcpu_run+5>
> 0x0000000000007445 <+5>: 55 push %rbp
> 0x0000000000007446 <+6>: 48 89 e5 mov %rsp,%rbp
>
>
> The warning only shows up in certain configs, e.g. I was only able to
> reproduce this using the .config provided by lkp. Even explicitly
> enabling CONFIG_FRAME_POINTERS and CONFIG_STACK_VALIDATION didn't
> trigger the warning using my usual config.
>
> And all that being said, I'm pretty sure this isn't related to the call
> to vmx_vmenter() at all, but rather is something that was exposed by
> removing __noclone from vmx_vcpu_run().
>
> E.g. I still get the warning if I comment out the call to vmx_vmenter,
> it just shifts to something else (and continues to shift I comment out
> more calls). The warning goes away if I re-add __noclone, regardless
> of whether or not commit 2bcbd406715d ("Revert "compiler-gcc: disable
> -ftracer for __noclone functions"") is applied.
It complained the call right here at the end of vmx_vcpu_run().
"callq 19eb6" in __read_once_size() via atomic_read()
and then jump back to vmx_vcpu_run() again.
/root/linux-debug/arch/x86/kvm/vmx/vmx.c:6650
}
19e94: e8 00 00 00 00 callq 19e99
<vmx_vcpu_run.part.21+0x3159>
__read_once_size():
/root/linux-debug/./include/linux/compiler.h:191
19e99: 48 c7 c7 00 00 00 00 mov $0x0,%rdi
19ea0: e8 00 00 00 00 callq 19ea5
<vmx_vcpu_run.part.21+0x3165>
19ea5: e9 8b dd ff ff jmpq 17c35
<vmx_vcpu_run.part.21+0xef5>
19eaa: 48 8b bd 48 ff ff ff mov -0xb8(%rbp),%rdi
19eb1: e8 00 00 00 00 callq 19eb6
<vmx_vcpu_run.part.21+0x3176>
19eb6: e9 b8 df ff ff jmpq 17e73
<vmx_vcpu_run.part.21+0x1133>
vmx_vcpu_run():
/root/linux-debug/arch/x86/kvm/vmx/vmx.c:6621
vcpu->arch.regs_dirty = 0;
19ebb: 48 89 f7 mov %rsi,%rdi
19ebe: e8 00 00 00 00 callq 19ec3
<vmx_vcpu_run.part.21+0x3183>
19ec3: e9 f1 e0 ff ff jmpq 17fb9
<vmx_vcpu_run.part.21+0x1279>
