On Wed, Mar 22, 2017 at 9:38 AM, Thomas Garnier <thgar...@google.com> wrote:
> On Wed, Mar 22, 2017 at 9:33 AM, Andy Lutomirski <l...@amacapital.net> wrote:
>> On Wed, Mar 22, 2017 at 12:36 AM, Ingo Molnar <mi...@kernel.org> wrote:
>>>
>>> * Thomas Garnier <thgar...@google.com> wrote:
>>>
>>>> > static inline void setup_fixmap_gdt(int cpu)
>>>> > {
>>>> > __set_fixmap(get_cpu_gdt_ro_index(cpu),
>>>> > - __pa(get_cpu_gdt_rw(cpu)), pg_fixmap_gdt_flags);
>>>> > + slow_virt_to_phys(get_cpu_gdt_rw(cpu)),
>>>> > + pg_fixmap_gdt_flags);
>>>> > }
>>>> >
>>>> > /* Load the original GDT from the per-cpu structure */
>>>> >
>>>> > This makes UP boot for me, but SMP (2 cpus) is still busted.
>>>>
>>>> This change fixed boot for me:
>>>>
>>>> diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
>>>> index b65155cc3760..4e30707d9f9a 100644
>>>> --- a/arch/x86/include/asm/fixmap.h
>>>> +++ b/arch/x86/include/asm/fixmap.h
>>>> @@ -104,7 +104,9 @@ enum fixed_addresses {
>>>> FIX_GDT_REMAP_BEGIN,
>>>> FIX_GDT_REMAP_END = FIX_GDT_REMAP_BEGIN + NR_CPUS - 1,
>>>>
>>>> - __end_of_permanent_fixed_addresses,
>>>> + __end_of_permanent_fixed_addresses =
>>>> + (FIX_GDT_REMAP_END + PTRS_PER_PTE - 1) &
>>>> + -PTRS_PER_PTE,
>>>>
>>>> Just ensure PKMAP_BASE & FIX_WP_TEST are on a different PMD.
>>>>
>>>> I don't think that the right fix but it might help understand the
>>>> exact root cause.
>>>
>>> Could this be related to the permission bits in the PMD itself getting out
>>> of sync
>>> with the PTEs? WP test marks a page writable/unwritable, and maybe we mess
>>> up the
>>> restoration. If they are on separate PMDs then this is worked around
>>> because the
>>> fixmap GDT is on a separate PMD.
>>>
>>
>> I don't think so. I think it's a pair of bugs related to the way that
>> percpu areas are virtually mapped.
>>
>> Bug 1: __pa is totally bogus on percpu pointers. Oddly, we have one
>> older instance of exactly the same bug (on the same GDT address) in
>> the kernel. I'll send a patch.
>>
>> Bug 2: Nothing syncs a freshly-set-up CPU's percpu area into
>> initial_page_table. This makes access to the gdt fail in
>> startup_32_smp. This looks like a longstanding bug, and I don't see
>> what it has to do with Thomas' series. I'm still mulling over what to
>> do about it.
>
> Why do you think padding the fixmap also fix the problem? That's the
> thing I don't get.
>
> With the padding the PA is now correct and the memcmp check also
> succeed. That's odd.
>
Not sure. There are some complicated heuristics in the percpu code
that determine how it's allocated, and the padding might be affecting
those heuristics.
--Andy
