During initialization, U-Boot may rely on clean caches for the code and data in
the memory area where its binary is located. Cleaning that cache area could be
a responsibility of the loader of U-Boot (e.g., BL2 from ARM Trusted Firmware).
Otherwise, it may be hard for U-Boot to avoid using corrupted data. The
situation may be different for memory areas outside of the U-Boot binary.
That's probably why we can see the call to "invalidate_dcache_all" in the
function "dcache_enable" in "u-boot/arch/arm/cpu/armv8/cache_v8.c". However,
this function invalidates all cache, and U-Boot initializes its data in the
binary before invoking "dcache_enable". This makes it inconsistent with any of
the 3 following possibilities to make assumptions about the cache state:
1. It is assumed that all data cache may be dirty when U-Boot receives
control. In this case, all data cache should be invalidated before any data is
initialized. Otherwise, the stale cache entries may overwrite the initialized
data in physical memory at any time, even before the data cache is enabled [3].
Besides, data cache hits are also permitted by the architecture when the cache
is disabled [2].
2. It is assumed that the responsibility of the loader of U-Boot is to clean
the cache for the area of the U-Boot binary, but the other memory areas may
correspond to dirty caches. This would match the assumptions made by the Linux
kernel (see "Caches, MMUs" in "Documentation/arm64/booting.txt" in Linux). In
this case, U-Boot should only invalidate caches for the memory areas outside
the U-Boot binary, and it should do so before initializing these memory areas.
3. It is assumed that the responsibility of the previously running firmware
is to clean the complete data cache before passing control to U-Boot. In this
case, it is unclear why U-Boot invalidates the data cache itself.
Because the current implementation in U-Boot is not consistent with any of the
above assumptions, we come across memory corruption in our use cases.
Specifically, U-Boot initializes the Global Data structure (pointed to by the
variable 'gd') as zero before using it. The member variable of the Global Data
structure, 'gd->arch.tlb_fillptr', is set after page table installation and is
used to determine whether the page tables are set up already or not [1]. If the
variable 'gd->arch.tlb_fillptr' value is zero, u-boot will set up a page table.
Otherwise, u-boot will skip setting up the page table to avoid duplicated page
table settings. However, we experience a stale D-cache entry where the variable
'gd->arch.tlb_fillptr' is located. In our use case, this stale entry overwrites
the variable in physical memory when U-Boot invalidates the data cache because
we run U-Boot in EL1 with stage 2 translation enabled, which promotes the cache
invalidation into clean and invalidate [4]. In our virtualized use case, the
pointer corruption is guaranteed this way, but in non-virtualized use cases,
such corruption is also possible because the cache is not guaranteed to stay
dirty [3].
I would like to propose U-Boot to follow the same set of assumptions as in
Linux. That is, to assume that the cache is only clean for the memory area
where the U-Boot binary is loaded (see 2. above).
[1] Based on latest master branch when writing this report: commit hash:
e7fb67df319cec410c20906bbf33936a6f7479b2
(File path: u-boot/arch/arm/cpu/armv8/cache_v8.c)
/* to activate the MMU we need to set up virtual memory */
__weak void mmu_setup(void)
{
int el;
/* Set up page tables only once */
if (!gd->arch.tlb_fillptr)
setup_all_pgtables();
el = current_el();
set_ttbr_tcr_mair(el, gd->arch.tlb_addr, get_tcr(el, NULL, NULL),
MEMORY_ATTRIBUTES);
/* enable the mmu */
set_sctlr(get_sctlr() | CR_M);
}
[2] ARM DDI 0487D.a ID103018; D4.4.5 Behavior of caches at reset
If an implementation permits cache hits when the Cacheability control fields
force all memory locations to be treated as Non-cacheable then the cache
initialization routine must:
— Provide a mechanism to ensure the correct initialization of the caches.
— Be documented clearly as part of the documentation of the device.
In particular, if an implementation permits cache hits when the Cacheability
controls force all memory locations to be treated as Non-cacheable, and the
cache contents are not invalidated at reset, the initialization routine must
avoid any possibility of running from an uninitialized cache. It is acceptable
for an initialization routine to require a fixed instruction sequence to be
placed in a restricted range of memory.
[3] ARM DDI 0487D.a ID103018; D4.4.1 General behavior of the caches
An unlocked entry in a cache might not remain in that cache. The architecture
does not guarantee that an unlocked cache entry remains in the cache or remains
incoherent with the rest of memory. Software must not assume that an unlocked
item that remains in the cache remains dirty.
[4] ARM DDI 0487D.a ID103018; D4.4 Cache support; The data cache maintenance
instruction (DC)
When executed at EL1, a DC ISW instruction performs a clean and invalidate,
meaning it performs the same maintenance as a DC CISW instruction, if all of
the following apply:
* EL2 is implemented and enabled in the current Security state.
* Either:
* The value of HCR_EL2.SWIO is 1, forcing a cache clean to perform a
clean and invalidate.
* The value of HCR_EL2.VM is 1, meaning EL1&0 stage two address
translation is enabled.
Best Regards,
Jeungwoo Yoo
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Telefon: +49 (30) 60 98 54 0 - 37
Fax: +49 (30) 60 98 54 0 - 99
E-MAIL: jeungwoo....@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
Please mind our privacy
notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/>
pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13
DSGVO finden Sie
hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>
