Re: [edk2-devel] [PATCH 1/2] ArmPkg: implement EFI_MP_SERVICES_PROTOCOL based on PSCI calls
Thanks for the update, Rebecca. I will also keep an eye on the cache issue when I have a chance to run this on FVP models. Regards, Kun On 11/28/2022 4:04 PM, Rebecca Cran wrote: On 11/28/22 15:59, Kun Qin wrote: Hi Rebecca/Ard, I was trying to reach out regarding the original patches earlier (see below) but it might fell off your stacks due to high traffic on the mailing list. Could you please kindly review the questions when you have a chance? In addition, I found another edge case of the MP service: when the AP routine hits a timeout, the metadata will be left in the unfinished states. If the AP routine eventually completes and return, this AP will stay in "finished" but never become "ready" during this boot. I tried to add below change, which seems to work. But please let me know if you have other concerns: https://github.com/kuqin12/mu_silicon_arm_tiano/commit/c76072b37018276f2fec2582d0c540be5b40d0f2 Thanks, I'll take a look and integrate the fix into the next revision of the patch series. Lastly, do you plan to merge these patches in the near future? This will be a great add-on for ARM platforms. The issue that's currently preventing them from being merged is a failure I noticed on the Neoverse N2 FVP: for some reason despite enabling the MMU and caches, manual cache flushes are still required for the data to be seen between CPUs. I don't know if that's a bug in the code or in the FVP model. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96718): https://edk2.groups.io/g/devel/message/96718 Mute This Topic: https://groups.io/mt/93329492/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 1/2] ArmPkg: implement EFI_MP_SERVICES_PROTOCOL based on PSCI calls
On 11/28/22 15:59, Kun Qin wrote: Hi Rebecca/Ard, I was trying to reach out regarding the original patches earlier (see below) but it might fell off your stacks due to high traffic on the mailing list. Could you please kindly review the questions when you have a chance? In addition, I found another edge case of the MP service: when the AP routine hits a timeout, the metadata will be left in the unfinished states. If the AP routine eventually completes and return, this AP will stay in "finished" but never become "ready" during this boot. I tried to add below change, which seems to work. But please let me know if you have other concerns: https://github.com/kuqin12/mu_silicon_arm_tiano/commit/c76072b37018276f2fec2582d0c540be5b40d0f2 Thanks, I'll take a look and integrate the fix into the next revision of the patch series. Lastly, do you plan to merge these patches in the near future? This will be a great add-on for ARM platforms. The issue that's currently preventing them from being merged is a failure I noticed on the Neoverse N2 FVP: for some reason despite enabling the MMU and caches, manual cache flushes are still required for the data to be seen between CPUs. I don't know if that's a bug in the code or in the FVP model. -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96663): https://edk2.groups.io/g/devel/message/96663 Mute This Topic: https://groups.io/mt/93329492/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 1/2] ArmPkg: implement EFI_MP_SERVICES_PROTOCOL based on PSCI calls
Sorry, I missed this originally. I've added my replies inline.][ On 9/29/22 12:45, Kun Qin wrote: Hi Rebecca, Thanks for sending this patch. I have a few questions inline "[KQ]". Could you please help me to understand the patch better? Thanks in advance. Regards, Kun On 8/29/2022 8:59 AM, Rebecca Cran wrote: +#define GET_MPIDR_AFFINITY_BITS(x) ((x) & 0xFF00FF) + +#define MPIDR_MT_BIT BIT24 [KQ]: Is there any reason why this is not defined in a more formal place? I think that will allow the platform to use as well. That's a good point. I can certainly move it somewhere more proper. + BOOLEAN IsBsp; [KQ]: Can we add a check to see if BSP is found during this routine? I think failing to identify a BSP might cause other issues in the following services? We could certainly log an error and abort if IsBsp is never set to TRUE. + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + if (GET_MPIDR_AFFINITY_BITS (ArmReadMpidr ()) == CoreInfo[Index].Mpidr) { [KQ]: Does this mean BSP should never set the `MPIDR_MT_BIT` in the gArmMpCoreInfoGuid HOB data? Otherwise, this logic will never be able to find the BSP? No, here we assume we're running on the BSP, and the code is looking for the entry in the CoreInfo array that matches the BSP's affinity fields. + switch (GetApState (CpuData)) { + case CpuStateReady: + SetApProcedure (CpuData, Procedure, ProcedureArgument); + Status = DispatchCpu (Index); + if (EFI_ERROR (Status)) { + CpuData->State = CpuStateIdle; + Status = EFI_NOT_READY; + goto Done; [KQ]: Is it really necessary to use "goto Done"? This might cause all the CPUs after this failing index to not reset the CPU state. That way, all the subsequent "StartupAllAps" will fail due to "CheckAllCpusReady" returning FALSE on those "dirty cores". Please let me know if that is by design or not. Good point. I'll fix it. -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96662): https://edk2.groups.io/g/devel/message/96662 Mute This Topic: https://groups.io/mt/93329492/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 1/2] ArmPkg: implement EFI_MP_SERVICES_PROTOCOL based on PSCI calls
Hi Rebecca/Ard, I was trying to reach out regarding the original patches earlier (see below) but it might fell off your stacks due to high traffic on the mailing list. Could you please kindly review the questions when you have a chance? In addition, I found another edge case of the MP service: when the AP routine hits a timeout, the metadata will be left in the unfinished states. If the AP routine eventually completes and return, this AP will stay in "finished" but never become "ready" during this boot. I tried to add below change, which seems to work. But please let me know if you have other concerns: https://github.com/kuqin12/mu_silicon_arm_tiano/commit/c76072b37018276f2fec2582d0c540be5b40d0f2 Lastly, do you plan to merge these patches in the near future? This will be a great add-on for ARM platforms. Thanks, Kun On 9/29/2022 11:45 AM, Kun Qin wrote: Hi Rebecca, Thanks for sending this patch. I have a few questions inline "[KQ]". Could you please help me to understand the patch better? Thanks in advance. Regards, Kun On 8/29/2022 8:59 AM, Rebecca Cran wrote: Add support for EFI_MP_SERVICES_PROTOCOL during the DXE phase under AArch64. PSCI_CPU_ON is called to power on the core, the supplied procedure is executed and PSCI_CPU_OFF is called to power off the core. Fixes contributed by Ard Biesheuvel. Signed-off-by: Rebecca Cran --- ArmPkg/ArmPkg.dsc | 1 + ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf | 55 + ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h | 351 ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.c | 1774 ArmPkg/Drivers/ArmPsciMpServicesDxe/MpFuncs.S | 57 + 5 files changed, 2238 insertions(+) diff --git a/ArmPkg/ArmPkg.dsc b/ArmPkg/ArmPkg.dsc index 59fd8f295d4f..4716789402fc 100644 --- a/ArmPkg/ArmPkg.dsc +++ b/ArmPkg/ArmPkg.dsc @@ -125,6 +125,7 @@ [Components.common] ArmPkg/Drivers/CpuPei/CpuPei.inf ArmPkg/Drivers/ArmGic/ArmGicDxe.inf ArmPkg/Drivers/ArmGic/ArmGicLib.inf + ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf ArmPkg/Drivers/GenericWatchdogDxe/GenericWatchdogDxe.inf ArmPkg/Drivers/TimerDxe/TimerDxe.inf diff --git a/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf b/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf new file mode 100644 index ..2b109c72e0a0 --- /dev/null +++ b/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf @@ -0,0 +1,55 @@ +## @file +# ARM MP services protocol driver +# +# Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION = 1.27 + BASE_NAME = ArmPsciMpServicesDxe + FILE_GUID = 007ab472-dc4a-4df8-a5c2-abb4a327278c + MODULE_TYPE = DXE_DRIVER + VERSION_STRING = 1.0 + + ENTRY_POINT = ArmPsciMpServicesDxeInitialize + +[Sources.Common] + ArmPsciMpServicesDxe.c + MpFuncs.S + MpServicesInternal.h + +[Packages] + ArmPkg/ArmPkg.dec + ArmPlatformPkg/ArmPlatformPkg.dec + EmbeddedPkg/EmbeddedPkg.dec + MdePkg/MdePkg.dec + MdeModulePkg/MdeModulePkg.dec + +[LibraryClasses] + ArmLib + ArmMmuLib + ArmSmcLib + BaseMemoryLib + CacheMaintenanceLib + DebugLib + HobLib + MemoryAllocationLib + UefiBootServicesTableLib + UefiDriverEntryPoint + UefiLib + +[Protocols] + gEfiMpServiceProtocolGuid ## PRODUCES + gEfiLoadedImageProtocolGuid ## CONSUMES + +[Guids] + gArmMpCoreInfoGuid + +[Depex] + TRUE + +[BuildOptions] + GCC:*_*_*_CC_FLAGS = -mstrict-align diff --git a/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h b/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h new file mode 100644 index ..ee13816ef64c --- /dev/null +++ b/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h @@ -0,0 +1,351 @@ +/** @file + +Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. +Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved. +Portions copyright (c) 2011, Apple Inc. All rights reserved. + +SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef MP_SERVICES_INTERNAL_H_ +#define MP_SERVICES_INTERNAL_H_ + +#include +#include + +#include +#include + +#define AP_STACK_SIZE 0x1000 + +// +// Internal Data Structures +// + +// +// AP state +// +// The state transitions for an AP when it process a procedure are: +// Idle > Ready > Busy > Idle +// [BSP] [AP] [AP] +// +typedef enum { + CpuStateIdle, + CpuStateReady, + CpuStateBlocked, + CpuStateBusy, + CpuStateFinished, + CpuStateDisabled +} CPU_STATE; + +// +// Define Individual Processor Data block. +// +typedef struct { + EFI_PROCESSOR_INFORMATION Info; + EFI_AP_PROCEDURE Procedure; + VOID *Parameter; + CPU_STATE State; +
Re: [edk2-devel] [PATCH 1/2] ArmPkg: implement EFI_MP_SERVICES_PROTOCOL based on PSCI calls
Hi Rebecca, Thanks for sending this patch. I have a few questions inline "[KQ]". Could you please help me to understand the patch better? Thanks in advance. Regards, Kun On 8/29/2022 8:59 AM, Rebecca Cran wrote: Add support for EFI_MP_SERVICES_PROTOCOL during the DXE phase under AArch64. PSCI_CPU_ON is called to power on the core, the supplied procedure is executed and PSCI_CPU_OFF is called to power off the core. Fixes contributed by Ard Biesheuvel. Signed-off-by: Rebecca Cran --- ArmPkg/ArmPkg.dsc|1 + ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf | 55 + ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h | 351 ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.c | 1774 ArmPkg/Drivers/ArmPsciMpServicesDxe/MpFuncs.S| 57 + 5 files changed, 2238 insertions(+) diff --git a/ArmPkg/ArmPkg.dsc b/ArmPkg/ArmPkg.dsc index 59fd8f295d4f..4716789402fc 100644 --- a/ArmPkg/ArmPkg.dsc +++ b/ArmPkg/ArmPkg.dsc @@ -125,6 +125,7 @@ [Components.common] ArmPkg/Drivers/CpuPei/CpuPei.inf ArmPkg/Drivers/ArmGic/ArmGicDxe.inf ArmPkg/Drivers/ArmGic/ArmGicLib.inf + ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf ArmPkg/Drivers/GenericWatchdogDxe/GenericWatchdogDxe.inf ArmPkg/Drivers/TimerDxe/TimerDxe.inf diff --git a/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf b/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf new file mode 100644 index ..2b109c72e0a0 --- /dev/null +++ b/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf @@ -0,0 +1,55 @@ +## @file +# ARM MP services protocol driver +# +# Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION= 1.27 + BASE_NAME = ArmPsciMpServicesDxe + FILE_GUID = 007ab472-dc4a-4df8-a5c2-abb4a327278c + MODULE_TYPE= DXE_DRIVER + VERSION_STRING = 1.0 + + ENTRY_POINT= ArmPsciMpServicesDxeInitialize + +[Sources.Common] + ArmPsciMpServicesDxe.c + MpFuncs.S + MpServicesInternal.h + +[Packages] + ArmPkg/ArmPkg.dec + ArmPlatformPkg/ArmPlatformPkg.dec + EmbeddedPkg/EmbeddedPkg.dec + MdePkg/MdePkg.dec + MdeModulePkg/MdeModulePkg.dec + +[LibraryClasses] + ArmLib + ArmMmuLib + ArmSmcLib + BaseMemoryLib + CacheMaintenanceLib + DebugLib + HobLib + MemoryAllocationLib + UefiBootServicesTableLib + UefiDriverEntryPoint + UefiLib + +[Protocols] + gEfiMpServiceProtocolGuid## PRODUCES + gEfiLoadedImageProtocolGuid ## CONSUMES + +[Guids] + gArmMpCoreInfoGuid + +[Depex] + TRUE + +[BuildOptions] + GCC:*_*_*_CC_FLAGS = -mstrict-align diff --git a/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h b/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h new file mode 100644 index ..ee13816ef64c --- /dev/null +++ b/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h @@ -0,0 +1,351 @@ +/** @file + +Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. +Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved. +Portions copyright (c) 2011, Apple Inc. All rights reserved. + +SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef MP_SERVICES_INTERNAL_H_ +#define MP_SERVICES_INTERNAL_H_ + +#include +#include + +#include +#include + +#define AP_STACK_SIZE 0x1000 + +// +// Internal Data Structures +// + +// +// AP state +// +// The state transitions for an AP when it process a procedure are: +// Idle > Ready > Busy > Idle +// [BSP] [AP] [AP] +// +typedef enum { + CpuStateIdle, + CpuStateReady, + CpuStateBlocked, + CpuStateBusy, + CpuStateFinished, + CpuStateDisabled +} CPU_STATE; + +// +// Define Individual Processor Data block. +// +typedef struct { + EFI_PROCESSOR_INFORMATIONInfo; + EFI_AP_PROCEDURE Procedure; + VOID *Parameter; + CPU_STATEState; + EFI_EVENTCheckThisAPEvent; + UINTNTimeout; + UINTNTimeTaken; + VOID *Ttbr0; + UINTNTcr; + UINTNMair; +} CPU_AP_DATA; + +// +// Define MP data block which consumes individual processor block. +// +typedef struct { + UINTN NumberOfProcessors; + UINTN NumberOfEnabledProcessors; + EFI_EVENT CheckAllAPsEvent; + EFI_EVENT WaitEvent; + UINTN FinishCount; + UINTN StartCount; + EFI_AP_PROCEDUREProcedure; + VOID*ProcedureArgument; + BOOLEAN SingleThread; + UINTN StartedNumber; + CPU_AP_DATA *CpuData; + UINTN Timeout; + UINTN
[edk2-devel] [PATCH 1/2] ArmPkg: implement EFI_MP_SERVICES_PROTOCOL based on PSCI calls
Add support for EFI_MP_SERVICES_PROTOCOL during the DXE phase under AArch64. PSCI_CPU_ON is called to power on the core, the supplied procedure is executed and PSCI_CPU_OFF is called to power off the core. Fixes contributed by Ard Biesheuvel. Signed-off-by: Rebecca Cran --- ArmPkg/ArmPkg.dsc|1 + ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf | 55 + ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h | 351 ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.c | 1774 ArmPkg/Drivers/ArmPsciMpServicesDxe/MpFuncs.S| 57 + 5 files changed, 2238 insertions(+) diff --git a/ArmPkg/ArmPkg.dsc b/ArmPkg/ArmPkg.dsc index 59fd8f295d4f..4716789402fc 100644 --- a/ArmPkg/ArmPkg.dsc +++ b/ArmPkg/ArmPkg.dsc @@ -125,6 +125,7 @@ [Components.common] ArmPkg/Drivers/CpuPei/CpuPei.inf ArmPkg/Drivers/ArmGic/ArmGicDxe.inf ArmPkg/Drivers/ArmGic/ArmGicLib.inf + ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf ArmPkg/Drivers/GenericWatchdogDxe/GenericWatchdogDxe.inf ArmPkg/Drivers/TimerDxe/TimerDxe.inf diff --git a/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf b/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf new file mode 100644 index ..2b109c72e0a0 --- /dev/null +++ b/ArmPkg/Drivers/ArmPsciMpServicesDxe/ArmPsciMpServicesDxe.inf @@ -0,0 +1,55 @@ +## @file +# ARM MP services protocol driver +# +# Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION= 1.27 + BASE_NAME = ArmPsciMpServicesDxe + FILE_GUID = 007ab472-dc4a-4df8-a5c2-abb4a327278c + MODULE_TYPE= DXE_DRIVER + VERSION_STRING = 1.0 + + ENTRY_POINT= ArmPsciMpServicesDxeInitialize + +[Sources.Common] + ArmPsciMpServicesDxe.c + MpFuncs.S + MpServicesInternal.h + +[Packages] + ArmPkg/ArmPkg.dec + ArmPlatformPkg/ArmPlatformPkg.dec + EmbeddedPkg/EmbeddedPkg.dec + MdePkg/MdePkg.dec + MdeModulePkg/MdeModulePkg.dec + +[LibraryClasses] + ArmLib + ArmMmuLib + ArmSmcLib + BaseMemoryLib + CacheMaintenanceLib + DebugLib + HobLib + MemoryAllocationLib + UefiBootServicesTableLib + UefiDriverEntryPoint + UefiLib + +[Protocols] + gEfiMpServiceProtocolGuid## PRODUCES + gEfiLoadedImageProtocolGuid ## CONSUMES + +[Guids] + gArmMpCoreInfoGuid + +[Depex] + TRUE + +[BuildOptions] + GCC:*_*_*_CC_FLAGS = -mstrict-align diff --git a/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h b/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h new file mode 100644 index ..ee13816ef64c --- /dev/null +++ b/ArmPkg/Drivers/ArmPsciMpServicesDxe/MpServicesInternal.h @@ -0,0 +1,351 @@ +/** @file + +Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. +Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved. +Portions copyright (c) 2011, Apple Inc. All rights reserved. + +SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef MP_SERVICES_INTERNAL_H_ +#define MP_SERVICES_INTERNAL_H_ + +#include +#include + +#include +#include + +#define AP_STACK_SIZE 0x1000 + +// +// Internal Data Structures +// + +// +// AP state +// +// The state transitions for an AP when it process a procedure are: +// Idle > Ready > Busy > Idle +// [BSP] [AP] [AP] +// +typedef enum { + CpuStateIdle, + CpuStateReady, + CpuStateBlocked, + CpuStateBusy, + CpuStateFinished, + CpuStateDisabled +} CPU_STATE; + +// +// Define Individual Processor Data block. +// +typedef struct { + EFI_PROCESSOR_INFORMATIONInfo; + EFI_AP_PROCEDURE Procedure; + VOID *Parameter; + CPU_STATEState; + EFI_EVENTCheckThisAPEvent; + UINTNTimeout; + UINTNTimeTaken; + VOID *Ttbr0; + UINTNTcr; + UINTNMair; +} CPU_AP_DATA; + +// +// Define MP data block which consumes individual processor block. +// +typedef struct { + UINTN NumberOfProcessors; + UINTN NumberOfEnabledProcessors; + EFI_EVENT CheckAllAPsEvent; + EFI_EVENT WaitEvent; + UINTN FinishCount; + UINTN StartCount; + EFI_AP_PROCEDUREProcedure; + VOID*ProcedureArgument; + BOOLEAN SingleThread; + UINTN StartedNumber; + CPU_AP_DATA *CpuData; + UINTN Timeout; + UINTN *FailedList; + UINTN FailedListIndex; + BOOLEAN TimeoutActive; +} CPU_MP_DATA; + +/** Secondary core entry point. + +**/ +VOID +ApEntryPoint ( + VOID + ); + +/** C entry-point for the AP. +