[Xen-devel] Odroid XU3 support
Hello, i'm trying to boot dom0 linux on exynos5422 platform on A15 (big cpu cluster). From the past mailing lists on this link: http://lists.xen.org/archives/html/xen-devel/2016-02/msg02275.html) I'm not sure that is well defined how to change A7's for A15's. Should change be in DTS or should one change cpu power up routines in linux? So far every time it boots from A7's. Additionally, I found this in xen boot.ini file: # use the A7s for dom0 and pin them to it setenv xen_bootargs sync_console console=dtuart dtuart=/serial@12C2 \ dom0_mem=800M dom0_max_vcpus=4 dom0_vcpus_pin How this defines A7s for dom0? How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. Regards, Ivan Pavic. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
(CC the author of the Odroid support in Xen) On 25/04/2016 11:01, Ivan Pavić2 wrote: Hello, Hello, I have CCed Suriyan who added the support of Odroid in Xen. He might be able to help you here. i'm trying to boot dom0 linux on exynos5422 platform on A15 (big cpu cluster). From the past mailing lists on this link: http://lists.xen.org/archives/html/xen-devel/2016-02/msg02275.html) I'm not sure that is well defined how to change A7's for A15's. Should change be in DTS or should one change cpu power up routines in linux? So far every time it boots from A7's. Additionally, I found this in xen boot.ini file: # use the A7s for dom0 and pin them to it setenv xen_bootargs sync_console console=dtuart dtuart=/serial@12C2 \ dom0_mem=800M dom0_max_vcpus=4 dom0_vcpus_pin How this defines A7s for dom0? How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. Regards, Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
On Tue, Apr 26, 2016 at 3:31 AM, Julien Grall wrote: > (CC the author of the Odroid support in Xen) > > On 25/04/2016 11:01, Ivan Pavić2 wrote: > >> Hello, >> > > Hello, > > I have CCed Suriyan who added the support of Odroid in Xen. He might be > able to help you here. > > >> i'm trying to boot dom0 linux on exynos5422 platform on A15 (big cpu >> cluster). >> From the past mailing lists on this link: >> http://lists.xen.org/archives/html/xen-devel/2016-02/msg02275.html) >> I'm not sure that is well defined how to change A7's for A15's. >> Should change be in DTS or should one change cpu power up routines in >> linux? >> So far every time it boots from A7's. >> > The Odroid XU3/XU4 afaict has a hardware pin which dictates if a A7 or an A15 core should boot up. It is currently set to boot up from the A7s. > Additionally, I found this in xen boot.ini file: >> >> # use the A7s for dom0 and pin them to it >> setenv xen_bootargs sync_console console=dtuart dtuart=/serial@12C2 \ >> dom0_mem=800M dom0_max_vcpus=4 dom0_vcpus_pin >> >> How this defines A7s for dom0? >> >> As mentioned above, the hardware by design boots up from the A7 cluster. Hence, the first 4 cpus are A7s and the next 4 are the A15s. > How can I check which core is currently active? >> Judging by this link on big.LITTLE architecture: >> http://forum.odroid.com/viewtopic.php?f=65&t=2580 >> >> result of: cat /proc/cpuinfo | grep "CPU part" is >> CPU part: 0xc07 >> >> which stands for A7. >> >> If you do this in dom0, it will show all of them to be 0xc07. They are vCPUs after all. I am not sure if I answered your question. Please feel free to probe more. Thanks! -Suriyan > Regards, >> > > Regards, > > -- > Julien Grall > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
Hello, On 27/04/16 23:53, Suriyan Ramasami wrote: How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. If you do this in dom0, it will show all of them to be 0xc07. They are vCPUs after all. Which is not a good idea. This means that Linux is not able to detect potential errata for the underlying cores (in this case the cortex-A15). Also some userspace may do some runtime optimization based on the kind of CPUs available in the guest. Xen is not ready for big.LITTLE, so I would highly recommend you to disable either all the Cortex-A15 or Cortex-A7. For your information, I am planning to send a patch to park any CPUs whose MIDR is not matching the boot CPU one. Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
> Xen is not ready for big.LITTLE, so I would highly recommend you to > disable either all the Cortex-A15 or Cortex-A7. > For your information, I am planning to send a patch to park any CPUs > whose MIDR is not matching the boot CPU one. > Julien Grall Ok, I decided to use A15s... How can I disable A7s in software and use only A15s... Additionally how can I be sure that I'm using A15 cluster? This maybe more ODROID related: Suriyan Ramasami wrote: > The Odroid XU3/XU4 afaict has a hardware pin which dictates if a A7 > or an A15 core should boot up. It is currently set to boot up from the A7s. This may sound silly but where is that PIN on XU3 board.? Can this be modfied(is it accessible)? If not, how can I do it in software... Where to change order? Where to disable A7s? Probably in uBoot? Regards, Ivan Pavic. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
Hello Julien, Thank you for the advice. I do have a follow up question. On Thu, Apr 28, 2016 at 2:50 AM, Julien Grall wrote: > Hello, > > On 27/04/16 23:53, Suriyan Ramasami wrote: > > How can I check which core is currently active? >> Judging by this link on big.LITTLE architecture: >> http://forum.odroid.com/viewtopic.php?f=65&t=2580 >> >> result of: cat /proc/cpuinfo | grep "CPU part" is >> CPU part: 0xc07 >> >> which stands for A7. >> >> If you do this in dom0, it will show all of them to be 0xc07. They are >> vCPUs after all. >> > > Which is not a good idea. This means that Linux is not able to detect > potential errata for the underlying cores (in this case the cortex-A15). > Also some userspace may do some runtime optimization based on the kind of > CPUs available in the guest. > > Xen is not ready for big.LITTLE, so I would highly recommend you to > disable either all the Cortex-A15 or Cortex-A7. > > Ian did recommend that if they were in their own cpu pools it would avoid mixing them in a guest. I was researching that angle. What is your take on that? If Linux is not recognizing it, that is a dom0/domU issue, is it not? Nonetheless, to start with, to add support, I think there would be less resistance if the boot cluster (a7) cpus are enabled and the other cluster disabled (a15) For your information, I am planning to send a patch to park any CPUs whose > MIDR is not matching the boot CPU one. > > Regards, > > -- > Julien Grall > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
On 28/04/16 14:26, Suriyan Ramasami wrote: Hello Julien, Hello Suriyan, Thank you for the advice. I do have a follow up question. On Thu, Apr 28, 2016 at 2:50 AM, Julien Grall mailto:julien.gr...@arm.com>> wrote: Hello, On 27/04/16 23:53, Suriyan Ramasami wrote: How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. If you do this in dom0, it will show all of them to be 0xc07. They are vCPUs after all. Which is not a good idea. This means that Linux is not able to detect potential errata for the underlying cores (in this case the cortex-A15). Also some userspace may do some runtime optimization based on the kind of CPUs available in the guest. Xen is not ready for big.LITTLE, so I would highly recommend you to disable either all the Cortex-A15 or Cortex-A7. Ian did recommend that if they were in their own cpu pools it would avoid mixing them in a guest. I was researching that angle. What is your take on that? I have the same idea in mind. The pools are created at boot time by Xen, physical CPUs will be assigned to a pool depending on the CPU ID. However this is not enough to get support of big.LITTLE. You will at least need to modify the domain creation code to set the vMIDR based on the based where the vCPU will run. Furthermore Xen is expecting all the CPUs to have the same set of features, hence the boot CPU data is often used to know what could be run. This would need some work to get a system wide safe value (see arch/arm64/kernel/cpufeature.c in Linux). If Linux is not recognizing it, that is a dom0/domU issue, is it not? This is a Xen issue. Xen is exposing the wrong CPU ID to the domain and therefore the kernel may not apply properly errata or apply wrong optimization. Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
(CC Stefano) I forgot the CC stefano on this mail. On 05/05/16 12:39, Julien Grall wrote: On 28/04/16 14:26, Suriyan Ramasami wrote: Hello Julien, Hello Suriyan, Thank you for the advice. I do have a follow up question. On Thu, Apr 28, 2016 at 2:50 AM, Julien Grall mailto:julien.gr...@arm.com>> wrote: Hello, On 27/04/16 23:53, Suriyan Ramasami wrote: How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. If you do this in dom0, it will show all of them to be 0xc07. They are vCPUs after all. Which is not a good idea. This means that Linux is not able to detect potential errata for the underlying cores (in this case the cortex-A15). Also some userspace may do some runtime optimization based on the kind of CPUs available in the guest. Xen is not ready for big.LITTLE, so I would highly recommend you to disable either all the Cortex-A15 or Cortex-A7. Ian did recommend that if they were in their own cpu pools it would avoid mixing them in a guest. I was researching that angle. What is your take on that? I have the same idea in mind. The pools are created at boot time by Xen, physical CPUs will be assigned to a pool depending on the CPU ID. However this is not enough to get support of big.LITTLE. You will at least need to modify the domain creation code to set the vMIDR based on the based where the vCPU will run. Furthermore Xen is expecting all the CPUs to have the same set of features, hence the boot CPU data is often used to know what could be run. This would need some work to get a system wide safe value (see arch/arm64/kernel/cpufeature.c in Linux). If Linux is not recognizing it, that is a dom0/domU issue, is it not? This is a Xen issue. Xen is exposing the wrong CPU ID to the domain and therefore the kernel may not apply properly errata or apply wrong optimization. Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
Hi Julien, On Thu, Apr 28, 2016 at 10:50:33AM +0100, Julien Grall wrote: >Hello, > >On 27/04/16 23:53, Suriyan Ramasami wrote: > >>How can I check which core is currently active? >>Judging by this link on big.LITTLE architecture: >>http://forum.odroid.com/viewtopic.php?f=65&t=2580 >> >>result of: cat /proc/cpuinfo | grep "CPU part" is >>CPU part: 0xc07 >> >>which stands for A7. >> >>If you do this in dom0, it will show all of them to be 0xc07. They are >>vCPUs after all. > >Which is not a good idea. This means that Linux is not able to detect >potential errata for the underlying cores (in this case the cortex-A15). Also >some userspace may do some runtime optimization based on the kind of CPUs >available in the guest. > >Xen is not ready for big.LITTLE, so I would highly recommend you to disable >either all the Cortex-A15 or Cortex-A7. For Cortex A53 + A72 or A53 + A57, xen also not ready? If need to let xen support A53 + A72 or A53 + A57, what work may need to be done? Thanks, Peng > >For your information, I am planning to send a patch to park any CPUs whose >MIDR is not matching the boot CPU one. > >Regards, > >-- >Julien Grall > >___ >Xen-devel mailing list >Xen-devel@lists.xen.org >http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
On 08/05/2016 12:59, Peng Fan wrote: Hi Julien, Hello Peng, On Thu, Apr 28, 2016 at 10:50:33AM +0100, Julien Grall wrote: Hello, On 27/04/16 23:53, Suriyan Ramasami wrote: How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. If you do this in dom0, it will show all of them to be 0xc07. They are vCPUs after all. Which is not a good idea. This means that Linux is not able to detect potential errata for the underlying cores (in this case the cortex-A15). Also some userspace may do some runtime optimization based on the kind of CPUs available in the guest. Xen is not ready for big.LITTLE, so I would highly recommend you to disable either all the Cortex-A15 or Cortex-A7. For Cortex A53 + A72 or A53 + A57, xen also not ready? Xen is not ready at all for big.LITTLE, no matter the combinations of the CPUs. Some of the cores you mentioned (especially A53 and A57) may contain errata that requires workaround in the Linux kernel to avoid potential data corruption or deadlock. As the vPIDR will always be equal to the boot CPU vMIDR, Linux won't apply the necessary workaround to itself. So using big and little cores at the same time with Xen is not safe at all, even if you create CPU pool with current version of Xen. If need to let xen support A53 + A72 or A53 + A57, what work may need to be done? I gave some insights later this thread. See: http://lists.xen.org/archives/html/xen-devel/2016-05/msg00466.html Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
Hi Julien, On Mon, May 09, 2016 at 10:49:58AM +0100, Julien Grall wrote: > > >On 08/05/2016 12:59, Peng Fan wrote: >>Hi Julien, > >Hello Peng, > >>On Thu, Apr 28, 2016 at 10:50:33AM +0100, Julien Grall wrote: >>>Hello, >>> >>>On 27/04/16 23:53, Suriyan Ramasami wrote: >>> How can I check which core is currently active? Judging by this link on big.LITTLE architecture: http://forum.odroid.com/viewtopic.php?f=65&t=2580 result of: cat /proc/cpuinfo | grep "CPU part" is CPU part: 0xc07 which stands for A7. If you do this in dom0, it will show all of them to be 0xc07. They are vCPUs after all. >>> >>>Which is not a good idea. This means that Linux is not able to detect >>>potential errata for the underlying cores (in this case the cortex-A15). Also >>>some userspace may do some runtime optimization based on the kind of CPUs >>>available in the guest. >>> >>>Xen is not ready for big.LITTLE, so I would highly recommend you to disable >>>either all the Cortex-A15 or Cortex-A7. >> >>For Cortex A53 + A72 or A53 + A57, xen also not ready? > >Xen is not ready at all for big.LITTLE, no matter the combinations of the >CPUs. > >Some of the cores you mentioned (especially A53 and A57) may contain errata >that requires workaround in the Linux kernel to avoid potential data >corruption or deadlock. As the vPIDR will always be equal to the boot CPU >vMIDR, Linux won't apply the necessary workaround to itself. > >So using big and little cores at the same time with Xen is not safe at all, >even if you create CPU pool with current version of Xen. > >> >>If need to let xen support A53 + A72 or A53 + A57, what work may need to be >>done? > >I gave some insights later this thread. See: > >http://lists.xen.org/archives/html/xen-devel/2016-05/msg00466.html Thanks for this info. Do you have plan to add bit.LITTLE support for xen? I would like to use this on AArch64 platform-:) Thanks, Peng. > >Regards, > >-- >Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Odroid XU3 support
On 10/05/16 08:53, Peng Fan wrote: Hi Julien, Hello Peng, Thanks for this info. Do you have plan to add bit.LITTLE support for xen? It is not on my top priority for Xen. Feel free to send a patch series to add support for big.LITTLE. Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel