[PATCH V3 0/9] powerpc: Support for ibm,dynamic-memory-v2
This patch set provides a series of updates to de-couple the LMB information provided in the device tree property from the device tree property format. This eases the ability to support a new format for the dynamic memory property, ibm,dynamic-memory-v2. This series of patches consolidates the routines for parsing the LMB device tree properties into a new file, powerpc/mm/drmem.c, and provides the ability to retrieve LMB information without having to know the backing device tree format. To do this, a set of routines are introduced that will walk the device tree property and make a call back for each LMB represented in the device tree. These are to be used by init routines during boot, currently in prom.c and numa.c. A late_initcall is used to allocate and initialize a LMB array to provide a common data structure of per-LMB data. This array provides a data structure to retrieve LMB information without knowing the backing tree format. This is used in numa.c and pseries/hotplug-memory.c. This is a big design change from the first two version of the patch set that attempted to allocate the LMB array with bootmem very early. Testing showed that this allocation can fail which pushed the change to the current design. To support memory hotplug needing to update the device tree, a common routine is introduced to create a new copy of the device tree property in the proper format. The first three patches update the of_get_assoc_arrays(), of_get_usable_memory(), and of_drconf_to_nid_single() routines to do device tree lookups for information they need instead of having the nodes/properties passed in. These are updates needed for later changes. The fourth patch adds the walk_drmem_lmbs_early() routine to provide parsing of the flattened device tree and make a per-LMB call back, used in prom.c The fifth patch provides a walk_drmem_lmbs() routine to parse the device tree and provide a per-LMB call back, used in numa.c. This also allocates and initializes the LMB array, and updates numa.c to use the array. The sixth patch updates pseries hotplug code new LMB array data instead of parsing the device tree directly and introduces the common routine to create a new device tree property. The seventh patch moves the of_drconf_cell struct to drmem.h where it fits better than prom.h The eighth patch introduces support for the ibm,dynamic-memory-v2 property format by updating the new drmem.c code to be able to parse and create this new device tree format. The last patch in the series updates the architecture vector to indicate support for ibm,dynamic-memory-v2. -Nathan --- Nathan Fontenot (9): powerpc/numa: Look up device node in of_get_assoc_arrays() powerpc/numa: Look up device node in of_get_usable_memory() powerpc/numa: Look up associativity array in of_drconf_to_nid_single powerpc/mm: Separate ibm,dynamic-memory data from DT format powerpc/numa: Update numa code use walk_drmem_lmbs powerpc/pseries: Update memory hotplug code to use drmem LMB array powerpc: Move of_drconf_cell struct to asm/drmem.h powerpc/drmem: Add support for ibm,dynamic-memory-v2 property powerpc: Enable support of ibm,dynamic-memory-v2 arch/powerpc/include/asm/drmem.h| 100 arch/powerpc/include/asm/firmware.h |3 arch/powerpc/include/asm/prom.h | 17 - arch/powerpc/kernel/prom.c | 114 ++--- arch/powerpc/kernel/prom_init.c |1 arch/powerpc/mm/Makefile|2 arch/powerpc/mm/drmem.c | 438 +++ arch/powerpc/mm/numa.c | 252 +++ arch/powerpc/platforms/pseries/firmware.c |1 arch/powerpc/platforms/pseries/hotplug-memory.c | 522 +-- 10 files changed, 864 insertions(+), 586 deletions(-) create mode 100644 arch/powerpc/include/asm/drmem.h create mode 100644 arch/powerpc/mm/drmem.c
Re: [PATCH V3 0/9] powerpc: Support for ibm,dynamic-memory-v2
Nathan Fontenot writes: > This patch set provides a series of updates to de-couple the LMB > information provided in the device tree property from the device > tree property format. This eases the ability to support a new > format for the dynamic memory property, ibm,dynamic-memory-v2. Something in here is still blowing up for me in a KVM guest: OF stdout device is: /vdevice/vty@7100 Preparing to boot Linux version 4.14.0-rc2-gcc6x-g9e1fc7e (kerkins@alpine1-p1) (gcc version 6.4.1 20171202 (Custom 6328ca9eaa476138)) #1 SMP Sun Dec 3 21:45:32 AEDT 2017 Detected machine type: 0101 command line: Max number of cores passed to firmware: 256 (NR_CPUS = 2048) Calling ibm,client-architecture-support... done memory layout at init: memory_limit : (16 MB aligned) alloc_bottom : 015c alloc_top: 3000 alloc_top_hi : 0001 rmo_top : 3000 ram_top : 0001 instantiating rtas at 0x2fff... done prom_hold_cpus: skipped copying OF device tree... Building dt strings... Building dt structure... Device tree strings 0x017d -> 0x017d09d8 Device tree struct 0x017e -> 0x017f Quiescing Open Firmware ... Booting Linux via __start() @ 0x0040 ... [0.00] bootconsole [udbg0] enabled [0.00] Allocated 2883584 bytes for 2048 pacas at cfd4 [0.00] hash-mmu: Page sizes from device-tree: [0.00] hash-mmu: base_shift=12: shift=12, sllp=0x, avpnm=0x, tlbiel=1, penc=0 [0.00] hash-mmu: base_shift=16: shift=16, sllp=0x0110, avpnm=0x, tlbiel=1, penc=1 [0.00] -> fw_vec5_feature_init() [0.00] <- fw_vec5_feature_init() [0.00] -> fw_hypertas_feature_init() [0.00] <- fw_hypertas_feature_init() [0.00] Page orders: linear mapping = 16, virtual = 16, io = 16, vmemmap = 16 [0.00] Using 1TB segments [0.00] hash-mmu: Initializing hash mmu with SLB [0.00] Linux version 4.14.0-rc2-gcc6x-g9e1fc7e (kerkins@alpine1-p1) (gcc version 6.4.1 20171202 (Custom 6328ca9eaa476138)) #1 SMP Sun Dec 3 21:45:32 AEDT 2017 [0.00] Found initrd at 0xc15c:0xc178d70b [0.00] Machine is LPAR ! [0.00] -> pseries_init() [0.00] -> fw_cmo_feature_init() [0.00] CMO not available [0.00] <- fw_cmo_feature_init() [0.00] <- pseries_init() [0.00] Using pSeries machine description [0.00] Partition configured for 16 cpus. [0.00] CPU maps initialized for 8 threads per core [0.00] (thread shift is 3) [0.00] Freed 2818048 bytes for unused pacas [0.00] - [0.00] ppc64_pft_size= 0x19 [0.00] phys_mem_size = 0x1 [0.00] dcache_bsize = 0x80 [0.00] icache_bsize = 0x80 [0.00] cpu_features = 0x17dc7aec18500249 [0.00] possible= 0xdfdf18500649 [0.00] always = 0x18100040 [0.00] cpu_user_features = 0xdc0065c2 0xef00 [0.00] mmu_features = 0x78006001 [0.00] firmware_features = 0x0001405a440b [0.00] htab_hash_mask= 0x3 [0.00] - [0.00] numa: NODE_DATA [mem 0xfff6a300-0xfff73fff] [0.00] -> smp_init_pSeries() [0.00] <- smp_init_pSeries() [0.00] PCI host bridge /pci@8002000 ranges: [0.00] IO 0x01008000..0x01008000 -> 0x [0.00] MEM 0x0100a000..0x01101fff -> 0x8000 [0.00] PPC64 nvram contains 65536 bytes [0.00] Top of RAM: 0x1, Total RAM: 0x1 [0.00] Memory hole size: 0MB [0.00] Zone ranges: [0.00] DMA [mem 0x-0x] [0.00] DMA32empty [0.00] Normal empty [0.00] Movable zone start for each node [0.00] Early memory node ranges [0.00] node 0: [mem 0x-0x] [0.00] Initmem setup node 0 [mem 0x-0x] [0.00] On node 0 totalpages: 65536 [0.00] DMA zone: 64 pages used for memmap [0.00] DMA zone: 0 pages reserved [0.00] DMA zone: 65536 pages, LIFO batch:1 [0.00] percpu: Embedded 4 pages/cpu @c000ffb0 s167064 r0 d95080 u262144 [0.00] pcpu-alloc: s167064 r0 d95080 u262144 alloc=1*1
Re: [PATCH V3 0/9] powerpc: Support for ibm,dynamic-memory-v2
On 12/03/2017 09:13 PM, Michael Ellerman wrote: > Nathan Fontenot writes: > >> This patch set provides a series of updates to de-couple the LMB >> information provided in the device tree property from the device >> tree property format. This eases the ability to support a new >> format for the dynamic memory property, ibm,dynamic-memory-v2. > > Something in here is still blowing up for me in a KVM guest: So, it looks like this series was applied despite observing this KVM guest crash. Cyril posted yesterday to the list about hitting this same issue with 4.16-rc1. -Tyrel > > OF stdout device is: /vdevice/vty@7100 > Preparing to boot Linux version 4.14.0-rc2-gcc6x-g9e1fc7e > (kerkins@alpine1-p1) (gcc version 6.4.1 20171202 (Custom 6328ca9eaa476138)) > #1 SMP Sun Dec 3 21:45:32 AEDT 2017 > Detected machine type: 0101 > command line: > Max number of cores passed to firmware: 256 (NR_CPUS = 2048) > Calling ibm,client-architecture-support... done > memory layout at init: > memory_limit : (16 MB aligned) > alloc_bottom : 015c > alloc_top: 3000 > alloc_top_hi : 0001 > rmo_top : 3000 > ram_top : 0001 > instantiating rtas at 0x2fff... done > prom_hold_cpus: skipped > copying OF device tree... > Building dt strings... > Building dt structure... > Device tree strings 0x017d -> 0x017d09d8 > Device tree struct 0x017e -> 0x017f > Quiescing Open Firmware ... > Booting Linux via __start() @ 0x0040 ... > [0.00] bootconsole [udbg0] enabled > [0.00] Allocated 2883584 bytes for 2048 pacas at cfd4 > [0.00] hash-mmu: Page sizes from device-tree: > [0.00] hash-mmu: base_shift=12: shift=12, sllp=0x, > avpnm=0x, tlbiel=1, penc=0 > [0.00] hash-mmu: base_shift=16: shift=16, sllp=0x0110, > avpnm=0x, tlbiel=1, penc=1 > [0.00] -> fw_vec5_feature_init() > [0.00] <- fw_vec5_feature_init() > [0.00] -> fw_hypertas_feature_init() > [0.00] <- fw_hypertas_feature_init() > [0.00] Page orders: linear mapping = 16, virtual = 16, io = 16, > vmemmap = 16 > [0.00] Using 1TB segments > [0.00] hash-mmu: Initializing hash mmu with SLB > [0.00] Linux version 4.14.0-rc2-gcc6x-g9e1fc7e > (kerkins@alpine1-p1) (gcc version 6.4.1 20171202 (Custom 6328ca9eaa476138)) > #1 SMP Sun Dec 3 21:45:32 AEDT 2017 > [0.00] Found initrd at 0xc15c:0xc178d70b > [0.00] Machine is LPAR ! > [0.00] -> pseries_init() > [0.00] -> fw_cmo_feature_init() > [0.00] CMO not available > [0.00] <- fw_cmo_feature_init() > [0.00] <- pseries_init() > [0.00] Using pSeries machine description > [0.00] Partition configured for 16 cpus. > [0.00] CPU maps initialized for 8 threads per core > [0.00] (thread shift is 3) > [0.00] Freed 2818048 bytes for unused pacas > [0.00] - > [0.00] ppc64_pft_size= 0x19 > [0.00] phys_mem_size = 0x1 > [0.00] dcache_bsize = 0x80 > [0.00] icache_bsize = 0x80 > [0.00] cpu_features = 0x17dc7aec18500249 > [0.00] possible= 0xdfdf18500649 > [0.00] always = 0x18100040 > [0.00] cpu_user_features = 0xdc0065c2 0xef00 > [0.00] mmu_features = 0x78006001 > [0.00] firmware_features = 0x0001405a440b > [0.00] htab_hash_mask= 0x3 > [0.00] - > [0.00] numa: NODE_DATA [mem 0xfff6a300-0xfff73fff] > [0.00] -> smp_init_pSeries() > [0.00] <- smp_init_pSeries() > [0.00] PCI host bridge /pci@8002000 ranges: > [0.00] IO 0x01008000..0x01008000 -> > 0x > [0.00] MEM 0x0100a000..0x01101fff -> > 0x8000 > [0.00] PPC64 nvram contains 65536 bytes > [0.00] Top of RAM: 0x1, Total RAM: 0x1 > [0.00] Memory hole size: 0MB > [0.00] Zone ranges: > [0.00] DMA [mem 0x-0x] > [0.00] DMA32empty > [0.00] Normal empty > [0.00] Movable zone start for each node > [0.00] Early memory node ranges > [0.00] node 0: [mem 0x-0x] > [0.00] Initmem setup node 0 [mem
Re: [PATCH V3 0/9] powerpc: Support for ibm,dynamic-memory-v2
On 02/14/2018 03:30 PM, Tyrel Datwyler wrote: > On 12/03/2017 09:13 PM, Michael Ellerman wrote: >> Nathan Fontenot writes: >> >>> This patch set provides a series of updates to de-couple the LMB >>> information provided in the device tree property from the device >>> tree property format. This eases the ability to support a new >>> format for the dynamic memory property, ibm,dynamic-memory-v2. >> >> Something in here is still blowing up for me in a KVM guest: > > So, it looks like this series was applied despite observing this KVM guest > crash. Cyril posted yesterday to the list about hitting this same issue with > 4.16-rc1. > > -Tyrel > Yes, Michael pointed out that he hit this on his system but I have never been able to replicate this error. Now that others are seeing it any help I could get on re-creating the failure would be appreciated. -Nathan >> >> OF stdout device is: /vdevice/vty@7100 >> Preparing to boot Linux version 4.14.0-rc2-gcc6x-g9e1fc7e >> (kerkins@alpine1-p1) (gcc version 6.4.1 20171202 (Custom 6328ca9eaa476138)) >> #1 SMP Sun Dec 3 21:45:32 AEDT 2017 >> Detected machine type: 0101 >> command line: >> Max number of cores passed to firmware: 256 (NR_CPUS = 2048) >> Calling ibm,client-architecture-support... done >> memory layout at init: >> memory_limit : (16 MB aligned) >> alloc_bottom : 015c >> alloc_top: 3000 >> alloc_top_hi : 0001 >> rmo_top : 3000 >> ram_top : 0001 >> instantiating rtas at 0x2fff... done >> prom_hold_cpus: skipped >> copying OF device tree... >> Building dt strings... >> Building dt structure... >> Device tree strings 0x017d -> 0x017d09d8 >> Device tree struct 0x017e -> 0x017f >> Quiescing Open Firmware ... >> Booting Linux via __start() @ 0x0040 ... >> [0.00] bootconsole [udbg0] enabled >> [0.00] Allocated 2883584 bytes for 2048 pacas at cfd4 >> [0.00] hash-mmu: Page sizes from device-tree: >> [0.00] hash-mmu: base_shift=12: shift=12, sllp=0x, >> avpnm=0x, tlbiel=1, penc=0 >> [0.00] hash-mmu: base_shift=16: shift=16, sllp=0x0110, >> avpnm=0x, tlbiel=1, penc=1 >> [0.00] -> fw_vec5_feature_init() >> [0.00] <- fw_vec5_feature_init() >> [0.00] -> fw_hypertas_feature_init() >> [0.00] <- fw_hypertas_feature_init() >> [0.00] Page orders: linear mapping = 16, virtual = 16, io = 16, >> vmemmap = 16 >> [0.00] Using 1TB segments >> [0.00] hash-mmu: Initializing hash mmu with SLB >> [0.00] Linux version 4.14.0-rc2-gcc6x-g9e1fc7e >> (kerkins@alpine1-p1) (gcc version 6.4.1 20171202 (Custom 6328ca9eaa476138)) >> #1 SMP Sun Dec 3 21:45:32 AEDT 2017 >> [0.00] Found initrd at 0xc15c:0xc178d70b >> [0.00] Machine is LPAR ! >> [0.00] -> pseries_init() >> [0.00] -> fw_cmo_feature_init() >> [0.00] CMO not available >> [0.00] <- fw_cmo_feature_init() >> [0.00] <- pseries_init() >> [0.00] Using pSeries machine description >> [0.00] Partition configured for 16 cpus. >> [0.00] CPU maps initialized for 8 threads per core >> [0.00] (thread shift is 3) >> [0.00] Freed 2818048 bytes for unused pacas >> [0.00] - >> [0.00] ppc64_pft_size= 0x19 >> [0.00] phys_mem_size = 0x1 >> [0.00] dcache_bsize = 0x80 >> [0.00] icache_bsize = 0x80 >> [0.00] cpu_features = 0x17dc7aec18500249 >> [0.00] possible= 0xdfdf18500649 >> [0.00] always = 0x18100040 >> [0.00] cpu_user_features = 0xdc0065c2 0xef00 >> [0.00] mmu_features = 0x78006001 >> [0.00] firmware_features = 0x0001405a440b >> [0.00] htab_hash_mask= 0x3 >> [0.00] - >> [0.00] numa: NODE_DATA [mem 0xfff6a300-0xfff73fff] >> [0.00] -> smp_init_pSeries() >> [0.00] <- smp_init_pSeries() >> [0.00] PCI host bridge /pci@8002000 ranges: >> [0.00] IO 0x01008000..0x01008000 -> >> 0x >> [0.00] MEM 0x0100a000..0x01101fff -> >> 0x8000 >> [0.00] PPC64 nvram contains 65536 bytes >> [0.00] Top of RAM: 0x1, Total RAM: 0x1 >> [0.00] Memory hole size: 0MB >> [0.00] Zone ranges: >> [
Re: [PATCH V3 0/9] powerpc: Support for ibm,dynamic-memory-v2
Tyrel Datwyler writes: > On 12/03/2017 09:13 PM, Michael Ellerman wrote: >> Nathan Fontenot writes: >> >>> This patch set provides a series of updates to de-couple the LMB >>> information provided in the device tree property from the device >>> tree property format. This eases the ability to support a new >>> format for the dynamic memory property, ibm,dynamic-memory-v2. >> >> Something in here is still blowing up for me in a KVM guest: > > So, it looks like this series was applied despite observing this KVM > guest crash. The crash has gone away for me, I do that KVM guest test on every push. So I applied the series, but yeah you're right that was probably silly of me. cheers