Laszlo, We have the PCD that specifies the max CPUs.
## Specifies max supported number of Logical Processors. # @Prompt Configure max supported number of Logical Processors gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber|64|UINT32|0x00000002 We could exit the wait loop as soon as the number of detected CPUs matches PcdCpuMaxLogicalProcessorNumber. Mike >-----Original Message----- >From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of >Laszlo Ersek >Sent: Wednesday, October 21, 2015 1:40 AM >To: Justen, Jordan L >Cc: Xiao Guangrong; Eduardo Habkost; edk2-de...@ml01.01.org; Alex >Williamson; Kinney, Michael D >Subject: Re: [edk2] [PATCH] OvmfPkg: increase MP services startup timeout > >CC'ing Eduardo > >On 10/20/15 23:39, Jordan Justen wrote: >> On 2015-10-20 12:27:42, Laszlo Ersek wrote: >>> Due to Linux kernel commit b18d5431acc7 ("KVM: x86: fix CR0.CD >>> virtualization"), vCPUs need more time to start up, because: >>> >>> - KVM now zaps all the mappings for the guest memory in EPT or shadow >page >>> table, hence more VM exits are required to rebuild the mappings for all >>> memory accesses. >>> >>> - If a physical device has been assigned to the guest, and the IOMMU lacks >>> the snoop control feature, guest memory will become uncacheable after >>> CR0.CD is set to 1. >>> >>> UefiCpuPkg/UefiCpuPkg.dec sets the timeout to 50ms; startup failures with >>> 100ms have been reported. Xiao Guangrong suggested 1s, and helped >word the >>> commit message. >> >> We will wait 1 second each boot for APs to start up? Meaning we will >> basically add 1 second to the OVMF boot time? > >Yes, in StartApsStackless() that's the case. > >> I wonder if we can get the actual max from QEMU and use it to tell >> CpuDxe how many processors to wait for. This would allow it to bail >> out earlier if all processors startup faster. > >QEMU exports two fw_cfg keys (not files -- directly keys) that are >related to VCPU counts: > > QemuFwCfgItemSmpCpuCount = 0x0005, > > QemuFwCfgItemMaximumCpuCount = 0x000f, > >(called FW_CFG_NB_CPUS and FW_CFG_MAX_CPUS in the QEMU source, >respectively). > >FW_CFG_MAX_CPUS is not really good for this, because it carries the >maximum possible APIC ID value, plus one, not the maximum number of >CPUs. Please see the big comment in bochs_bios_init() in QEMU >[hw/i386/pc.c]. > >FW_CFG_NB_CPUS exposes the "smp_cpus" variable of QEMU, which is the >count of VCPUs present at QEMU startup (more can be hotplugged later). >See the -smp option: > > -smp [cpus=]n[,cores=cores][,threads=threads][,sockets=sockets] > [,maxcpus=maxcpus] > >"smp_cpus" corresponds to "n" in the above (where "cpus" is enforced to >equal cores * threads * sockets ==> smp_parse()), whereas "maxcpus" >means "maximum number of CPUs, including future hotplug" (variable >"max_cpus", which is not exposed via fw_cfg at all). > >In addition, (smp_cpus - 1) is also visible in the CMOS, at offset 0x5f. > >Confused yet? Good; it is very confusing. Here's a summary: > > quantity |QEMU command line |QEMU variable or |guest >interface > | |function | > > ---------|--------------------------------------|------------------|--------------- > initially|-smp cpus= |smp_cpus > |FW_CFG_NB_CPUS > plugged |and/or | |and > CPUs |-smp cores=,threads=,sockets= | > |cmos[0x5f] + 1 > > ---------+--------------------------------------+------------------+--------------- > maximum |-smp maxcpus= |max_cpus |n/a > possible | | | > CPUs | | | > > ---------+--------------------------------------+------------------+--------------- > maximum |-smp >cores=,threads=,sockets=,maxcpus=|pc_apic_id_limit()|FW_CFG_MAX_CPUS > APIC ID |(i.e., derived from topology and | |-1 > |maximum possible count) | | > >SeaBIOS too counts the CPUs at startup (see smp_setup() in >"src/fw/smp.c"). The quantity that it uses to terminate the loop is >(cmos[0x5f] + 1) -- i.e., number of initially plugged CPUs. > >If we want to do something similar, then UefiCpuPkg/CpuDxe needs to >accept and consider such a limit (from a dynamic PCD), and: > >(a) in OvmfPkg/PlatformPei, we have to set that PCD from (cmos[0x5f]+1) > or (equivalently) QemuFwCfgItemSmpCpuCount, or >(b) set the PCD (from one of the same sources) in a NULL library that we > hook into CpuDxe in our platform DSC files. > >If you can write a patch for this, I'll thank you. :) > >Thanks, >Laszlo > >> >> -Jordan >> >>> Cc: Xiao Guangrong <guangrong.x...@linux.intel.com> >>> Cc: Jordan Justen <jordan.l.jus...@intel.com> >>> Cc: Janusz Mocek <janusz...@gmail.com> >>> Cc: Alex Williamson <alex.william...@redhat.com> >>> Reported-by: Janusz Mocek <janusz...@gmail.com> >>> Suggested-by: Xiao Guangrong <guangrong.x...@linux.intel.com> >>> Contributed-under: TianoCore Contribution Agreement 1.0 >>> Signed-off-by: Laszlo Ersek <ler...@redhat.com> >>> --- >>> OvmfPkg/OvmfPkgIa32.dsc | 6 ++++++ >>> OvmfPkg/OvmfPkgIa32X64.dsc | 8 +++++++- >>> OvmfPkg/OvmfPkgX64.dsc | 6 ++++++ >>> 3 files changed, 19 insertions(+), 1 deletion(-) >>> >>> diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc >>> index 0d044c2..670a80f 100644 >>> --- a/OvmfPkg/OvmfPkgIa32.dsc >>> +++ b/OvmfPkg/OvmfPkgIa32.dsc >>> @@ -358,6 +358,12 @@ [PcdsFixedAtBuild] >>> >gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2 >>> !endif >>> >>> + # Initial AP detection may take a long time on KVM; dependent on >device >>> + # assignment, IOMMU features, VCPU count, and more. Allow a >generous interval >>> + # for the MP services initialization in UefiCpuPkg/CpuDxe to count the >APs, >>> + # and to wait until they become idle again. >>> + >gUefiCpuPkgTokenSpaceGuid.PcdCpuApInitTimeOutInMicroSeconds|1000000 >>> + >>> !ifndef $(USE_OLD_SHELL) >>> gEfiIntelFrameworkModulePkgTokenSpaceGuid.PcdShellFile|{ 0x83, 0xA5, >0x04, 0x7C, 0x3E, 0x9E, 0x1C, 0x4F, 0xAD, 0x65, 0xE0, 0x52, 0x68, 0xD0, 0xB4, >0xD1 } >>> !endif >>> diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc >>> index 19d2221..8c00096 100644 >>> --- a/OvmfPkg/OvmfPkgIa32X64.dsc >>> +++ b/OvmfPkg/OvmfPkgIa32X64.dsc >>> @@ -51,7 +51,7 @@ [BuildOptions] >>> >>> [BuildOptions.common.EDKII.DXE_RUNTIME_DRIVER] >>> GCC:*_*_*_DLINK_FLAGS = -z common-page-size=0x1000 >>> - >>> + >>> >################################################################# >############### >>> # >>> # SKU Identification section - list of all SKU IDs supported by this >>> Platform. >>> @@ -363,6 +363,12 @@ [PcdsFixedAtBuild] >>> >gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2 >>> !endif >>> >>> + # Initial AP detection may take a long time on KVM; dependent on >device >>> + # assignment, IOMMU features, VCPU count, and more. Allow a >generous interval >>> + # for the MP services initialization in UefiCpuPkg/CpuDxe to count the >APs, >>> + # and to wait until they become idle again. >>> + >gUefiCpuPkgTokenSpaceGuid.PcdCpuApInitTimeOutInMicroSeconds|1000000 >>> + >>> !ifndef $(USE_OLD_SHELL) >>> gEfiIntelFrameworkModulePkgTokenSpaceGuid.PcdShellFile|{ 0x83, 0xA5, >0x04, 0x7C, 0x3E, 0x9E, 0x1C, 0x4F, 0xAD, 0x65, 0xE0, 0x52, 0x68, 0xD0, 0xB4, >0xD1 } >>> !endif >>> diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc >>> index b8df1dc..eb4da4f 100644 >>> --- a/OvmfPkg/OvmfPkgX64.dsc >>> +++ b/OvmfPkg/OvmfPkgX64.dsc >>> @@ -363,6 +363,12 @@ [PcdsFixedAtBuild] >>> >gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2 >>> !endif >>> >>> + # Initial AP detection may take a long time on KVM; dependent on >device >>> + # assignment, IOMMU features, VCPU count, and more. Allow a >generous interval >>> + # for the MP services initialization in UefiCpuPkg/CpuDxe to count the >APs, >>> + # and to wait until they become idle again. >>> + >gUefiCpuPkgTokenSpaceGuid.PcdCpuApInitTimeOutInMicroSeconds|1000000 >>> + >>> !ifndef $(USE_OLD_SHELL) >>> gEfiIntelFrameworkModulePkgTokenSpaceGuid.PcdShellFile|{ 0x83, 0xA5, >0x04, 0x7C, 0x3E, 0x9E, 0x1C, 0x4F, 0xAD, 0x65, 0xE0, 0x52, 0x68, 0xD0, 0xB4, >0xD1 } >>> !endif >>> -- >>> 1.8.3.1 >>> > >_______________________________________________ >edk2-devel mailing list >edk2-devel@lists.01.org >https://lists.01.org/mailman/listinfo/edk2-devel _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
