Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
Hi Mohamed,
just a small drive-by comment:
On Wed, Jan 20, 2021 at 2:32 PM Mohamed Mediouni
wrote:
> + for_each_cpu (cpu, mask) {
> + smp_mb__before_atomic();
> + atomic_or(1u << irqnr, per_cpu_ptr(&aic_ipi_mask, cpu));
Use:
#include
atomic_or(BIT(irqnr), per_cpu_ptr(&aic_ipi_mask, cpu));
Yours,
Linus Walleij
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Thu, Jan 21, 2021 at 12:09 PM Mohamed Mediouni wrote: > > > > > On 21 Jan 2021, at 18:37, Rob Herring wrote: > > > > On Thu, Jan 21, 2021 at 10:43 AM Mohamed Mediouni > > wrote: > >>> On 21 Jan 2021, at 17:40, Rob Herring wrote: > >>> On Thu, Jan 21, 2021 at 6:52 AM Mohamed Mediouni > >>> wrote: > > On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: > > On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni > > wrote: > > > > [...] > > > >> @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct > >> device_node *node, > >> if (WARN(!aic.base, "unable to map aic registers\n")) > >> return -EINVAL; > >> > >> + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); > > > > Where is this property documented, and what decides which one to use? > It’s getting documented in the next patch set. > > This property is there to enable support for older iPhone processors > later on, some of which do not have fast IPI support. > > On Apple M1, fast-ipi is always on. > >>> > >>> This should be implied by the compatible string which needs to be more > >>> specific and include the SoC name. > >>> > >>> Rob > >> > >> Then we’ll eventually have two aic compatible strings, aic which is > >> compatible > >> with Apple A7 onwards and aicv2 which is a superset with fast IPI > >> (introduced > >> on the Apple A11, 3 years ago, with no further programmer-visible changes > >> since > >> then). > >> > >> Does that look right? > > > > If we did this from the start, it would evolve like this: > > > > A7: "AAPL,a7-aic" > > A8: "AAPL,a8-aic", "AAPL,a7-aic" # Read this as A8 AIC is backwards > > compatible with A7 AIC > > A9: "AAPL,a9-aic", "AAPL,a7-aic" > > > > A11: "AAPL,a11-aic", "AAPL,a7-aic" > > > > If the A11 version could work on an OS that only supported the > > original model (sounds like this is the case) Or if it's not backwards > > compatible: > > > > The A11 AIC indeed can be used by older drivers that aren’t aware > of the fast IPI path introduced on A11 just fine. > > > A11: "AAPL,a11-aic" > > > > If the A11 is different and not backwards compatible. > > > > Then M1 could be: > > > > M1: "AAPL,m1-aic", "AAPL,a11-aic" > > > > Or to even support an OS with only v1 support: > > > > M1: "AAPL,m1-aic", "AAPL,a11-aic", "AAPL,a7-aic" > > > > You don't really need the fallback here because there isn't any > > existing OS support and the baseline is the M1. > > > > If you want to have generic fallback compatible strings with versions, > > that's fine too. I'm not really a fan of version numbers that are just > > made up by the binding author though. Most SoC vendors don't have > > rigorous versioning of their IP and those that do seem to have a new > > version on every SoC. > > > > The important part is *always* having an SoC specific compatible so > > you can deal with any quirk or feature without having to change the > > DTB. Everyone says blocks are 'the same' until they aren’t. > > > Is it fine if such a SoC-specific compatible is present but with having > the driver only know about AAPL,a11-aic for example? > (To just have it when it’d be needed if ever in the future, but not uselessly > add entries to the driver that will not be currently used) Yes, that's expected. You add the more specific compatible when you add the feature or quirk work-around. > > On a tangent: > > The internal naming scheme used by Apple is off-by-one: > > Apple A14 for example is Apple H13P (H-series 13th gen processor, Phone) > Apple M1 is Apple H13G (H-series 13th gen, G series) > (And Apple A12X is Apple H11G for example, with A12 being H11P) > > Should we bother with those or use the marketing names? Especially because > the beefier SoCs might not be of the H series anyway… as the internal scheme > reveals that M1 could as well have been an A14X. > > And there’s also the other internal naming scheme: > Apple A12 being t8020, Apple A12X being t8027 > Apple A14 being t8101 > Apple M1 being t8103 > > T there means the foundry at which the chip was manufactured, in the cases > above TSMC. > > Of course Apple itself uses both… with the marketing name being nowhere in > their device > trees. I'd probably lean toward the marketing names, but don't really care as long as you're consistent both for a given SoC and across generations. Rob
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
> On 21 Jan 2021, at 18:37, Rob Herring wrote: > > On Thu, Jan 21, 2021 at 10:43 AM Mohamed Mediouni > wrote: >>> On 21 Jan 2021, at 17:40, Rob Herring wrote: >>> On Thu, Jan 21, 2021 at 6:52 AM Mohamed Mediouni >>> wrote: > On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: > On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni > wrote: > > [...] > >> @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node >> *node, >> if (WARN(!aic.base, "unable to map aic registers\n")) >> return -EINVAL; >> >> + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); > > Where is this property documented, and what decides which one to use? It’s getting documented in the next patch set. This property is there to enable support for older iPhone processors later on, some of which do not have fast IPI support. On Apple M1, fast-ipi is always on. >>> >>> This should be implied by the compatible string which needs to be more >>> specific and include the SoC name. >>> >>> Rob >> >> Then we’ll eventually have two aic compatible strings, aic which is >> compatible >> with Apple A7 onwards and aicv2 which is a superset with fast IPI (introduced >> on the Apple A11, 3 years ago, with no further programmer-visible changes >> since >> then). >> >> Does that look right? > > If we did this from the start, it would evolve like this: > > A7: "AAPL,a7-aic" > A8: "AAPL,a8-aic", "AAPL,a7-aic" # Read this as A8 AIC is backwards > compatible with A7 AIC > A9: "AAPL,a9-aic", "AAPL,a7-aic" > > A11: "AAPL,a11-aic", "AAPL,a7-aic" > > If the A11 version could work on an OS that only supported the > original model (sounds like this is the case) Or if it's not backwards > compatible: > The A11 AIC indeed can be used by older drivers that aren’t aware of the fast IPI path introduced on A11 just fine. > A11: "AAPL,a11-aic" > > If the A11 is different and not backwards compatible. > > Then M1 could be: > > M1: "AAPL,m1-aic", "AAPL,a11-aic" > > Or to even support an OS with only v1 support: > > M1: "AAPL,m1-aic", "AAPL,a11-aic", "AAPL,a7-aic" > > You don't really need the fallback here because there isn't any > existing OS support and the baseline is the M1. > > If you want to have generic fallback compatible strings with versions, > that's fine too. I'm not really a fan of version numbers that are just > made up by the binding author though. Most SoC vendors don't have > rigorous versioning of their IP and those that do seem to have a new > version on every SoC. > > The important part is *always* having an SoC specific compatible so > you can deal with any quirk or feature without having to change the > DTB. Everyone says blocks are 'the same' until they aren’t. > Is it fine if such a SoC-specific compatible is present but with having the driver only know about AAPL,a11-aic for example? (To just have it when it’d be needed if ever in the future, but not uselessly add entries to the driver that will not be currently used) On a tangent: The internal naming scheme used by Apple is off-by-one: Apple A14 for example is Apple H13P (H-series 13th gen processor, Phone) Apple M1 is Apple H13G (H-series 13th gen, G series) (And Apple A12X is Apple H11G for example, with A12 being H11P) Should we bother with those or use the marketing names? Especially because the beefier SoCs might not be of the H series anyway… as the internal scheme reveals that M1 could as well have been an A14X. And there’s also the other internal naming scheme: Apple A12 being t8020, Apple A12X being t8027 Apple A14 being t8101 Apple M1 being t8103 T there means the foundry at which the chip was manufactured, in the cases above TSMC. Of course Apple itself uses both… with the marketing name being nowhere in their device trees. Thank you, > Rob
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Thu, Jan 21, 2021 at 10:43 AM Mohamed Mediouni wrote: > > On 21 Jan 2021, at 17:40, Rob Herring wrote: > > On Thu, Jan 21, 2021 at 6:52 AM Mohamed Mediouni > > wrote: > >>> On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: > >>> On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni > >>> wrote: [...] > @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node > *node, > if (WARN(!aic.base, "unable to map aic registers\n")) > return -EINVAL; > > + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); > >>> > >>> Where is this property documented, and what decides which one to use? > >> It’s getting documented in the next patch set. > >> > >> This property is there to enable support for older iPhone processors > >> later on, some of which do not have fast IPI support. > >> > >> On Apple M1, fast-ipi is always on. > > > > This should be implied by the compatible string which needs to be more > > specific and include the SoC name. > > > > Rob > > Then we’ll eventually have two aic compatible strings, aic which is compatible > with Apple A7 onwards and aicv2 which is a superset with fast IPI (introduced > on the Apple A11, 3 years ago, with no further programmer-visible changes > since > then). > > Does that look right? If we did this from the start, it would evolve like this: A7: "AAPL,a7-aic" A8: "AAPL,a8-aic", "AAPL,a7-aic" # Read this as A8 AIC is backwards compatible with A7 AIC A9: "AAPL,a9-aic", "AAPL,a7-aic" A11: "AAPL,a11-aic", "AAPL,a7-aic" If the A11 version could work on an OS that only supported the original model (sounds like this is the case) Or if it's not backwards compatible: A11: "AAPL,a11-aic" If the A11 is different and not backwards compatible. Then M1 could be: M1: "AAPL,m1-aic", "AAPL,a11-aic" Or to even support an OS with only v1 support: M1: "AAPL,m1-aic", "AAPL,a11-aic", "AAPL,a7-aic" You don't really need the fallback here because there isn't any existing OS support and the baseline is the M1. If you want to have generic fallback compatible strings with versions, that's fine too. I'm not really a fan of version numbers that are just made up by the binding author though. Most SoC vendors don't have rigorous versioning of their IP and those that do seem to have a new version on every SoC. The important part is *always* having an SoC specific compatible so you can deal with any quirk or feature without having to change the DTB. Everyone says blocks are 'the same' until they aren't. Rob
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
> On 21 Jan 2021, at 17:40, Rob Herring wrote:
>
> On Thu, Jan 21, 2021 at 6:52 AM Mohamed Mediouni
> wrote:
>>
>>
>>
>>> On 21 Jan 2021, at 13:44, Arnd Bergmann wrote:
>>>
>>> On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni
>>> wrote:
>>>
+#ifdef CONFIG_SMP
+static void apple_aic_ipi_send_mask(struct irq_data *d,
+ const struct cpumask *mask)
>>>
>>> Not sure we care about the #ifdef here, given that arch/arm64 does not
>>> allow building a kernel without CONFIG_SMP.
>>>
+ /*
+ * Ensure that stores to Normal memory are visible to the
+ * other CPUs before issuing the IPI.
+ */
+ wmb();
+
+ for_each_cpu (cpu, mask) {
+ smp_mb__before_atomic();
+ atomic_or(1u << irqnr, per_cpu_ptr(&aic_ipi_mask, cpu));
+ smp_mb__after_atomic();
+ lcpu = get_cpu();
+ if (aic.fast_ipi) {
+ if ((lcpu >> 2) == (cpu >> 2))
+ write_sysreg(cpu & 3, SR_APPLE_IPI_LOCAL);
+ else
+ write_sysreg((cpu & 3) | ((cpu >> 2) <<
16),
+SR_APPLE_IPI_REMOTE);
+ } else
+ writel(lcpu == cpu ? REG_IPI_FLAG_SELF :
+ (REG_IPI_FLAG_OTHER <<
cpu),
+ aic.base + REG_IPI_SET);
+ put_cpu();
+ }
+
+ /* Force the above writes to be executed */
+ if (aic.fast_ipi)
+ isb();
+}
>>>
>>> Since this just loops over all CPUs, I'd probably just turn it into
>>> an ipi_send_single() callback and have the caller do the
>>> loop for simplicity.
>>>
>>> I also have the feeling that splitting one hardware IPI into multiple
>>> logical interrupts, which are then all registered by the same irq
>>> handler adds a little more complexity than necessary.
>>>
>>> Changing this would of course require modifications to
>>> arch/arm64/kernel/smp.c, which is hardwired to use
>>> CONFIG_GENERIC_IRQ_IPI in smp_cross_call(), and allowing
>>> a different code path there may be worse than emulating an
>>> irqchip.
>>>
@@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node
*node,
if (WARN(!aic.base, "unable to map aic registers\n"))
return -EINVAL;
+ aic.fast_ipi = of_property_read_bool(node, "fast-ipi");
>>>
>>> Where is this property documented, and what decides which one to use?
>> It’s getting documented in the next patch set.
>>
>> This property is there to enable support for older iPhone processors
>> later on, some of which do not have fast IPI support.
>>
>> On Apple M1, fast-ipi is always on.
>
> This should be implied by the compatible string which needs to be more
> specific and include the SoC name.
>
> Rob
Then we’ll eventually have two aic compatible strings, aic which is compatible
with Apple A7 onwards and aicv2 which is a superset with fast IPI (introduced
on the Apple A11, 3 years ago, with no further programmer-visible changes since
then).
Does that look right?
Thank you,
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Thu, Jan 21, 2021 at 6:52 AM Mohamed Mediouni
wrote:
>
>
>
> > On 21 Jan 2021, at 13:44, Arnd Bergmann wrote:
> >
> > On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni
> > wrote:
> >
> >> +#ifdef CONFIG_SMP
> >> +static void apple_aic_ipi_send_mask(struct irq_data *d,
> >> + const struct cpumask *mask)
> >
> > Not sure we care about the #ifdef here, given that arch/arm64 does not
> > allow building a kernel without CONFIG_SMP.
> >
> >> + /*
> >> + * Ensure that stores to Normal memory are visible to the
> >> + * other CPUs before issuing the IPI.
> >> + */
> >> + wmb();
> >> +
> >> + for_each_cpu (cpu, mask) {
> >> + smp_mb__before_atomic();
> >> + atomic_or(1u << irqnr, per_cpu_ptr(&aic_ipi_mask, cpu));
> >> + smp_mb__after_atomic();
> >> + lcpu = get_cpu();
> >> + if (aic.fast_ipi) {
> >> + if ((lcpu >> 2) == (cpu >> 2))
> >> + write_sysreg(cpu & 3, SR_APPLE_IPI_LOCAL);
> >> + else
> >> + write_sysreg((cpu & 3) | ((cpu >> 2) <<
> >> 16),
> >> +SR_APPLE_IPI_REMOTE);
> >> + } else
> >> + writel(lcpu == cpu ? REG_IPI_FLAG_SELF :
> >> + (REG_IPI_FLAG_OTHER <<
> >> cpu),
> >> + aic.base + REG_IPI_SET);
> >> + put_cpu();
> >> + }
> >> +
> >> + /* Force the above writes to be executed */
> >> + if (aic.fast_ipi)
> >> + isb();
> >> +}
> >
> > Since this just loops over all CPUs, I'd probably just turn it into
> > an ipi_send_single() callback and have the caller do the
> > loop for simplicity.
> >
> > I also have the feeling that splitting one hardware IPI into multiple
> > logical interrupts, which are then all registered by the same irq
> > handler adds a little more complexity than necessary.
> >
> > Changing this would of course require modifications to
> > arch/arm64/kernel/smp.c, which is hardwired to use
> > CONFIG_GENERIC_IRQ_IPI in smp_cross_call(), and allowing
> > a different code path there may be worse than emulating an
> > irqchip.
> >
> >> @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node
> >> *node,
> >> if (WARN(!aic.base, "unable to map aic registers\n"))
> >> return -EINVAL;
> >>
> >> + aic.fast_ipi = of_property_read_bool(node, "fast-ipi");
> >
> > Where is this property documented, and what decides which one to use?
> It’s getting documented in the next patch set.
>
> This property is there to enable support for older iPhone processors
> later on, some of which do not have fast IPI support.
>
> On Apple M1, fast-ipi is always on.
This should be implied by the compatible string which needs to be more
specific and include the SoC name.
Rob
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
> On 21 Jan 2021, at 16:09, Arnd Bergmann wrote: > > On Thu, Jan 21, 2021 at 3:10 PM Marc Zyngier wrote: >> On 2021-01-21 13:34, Mohamed Mediouni wrote: On 21 Jan 2021, at 14:22, Marc Zyngier wrote: >>> Because UART access adapters for the new M1 Macs aren’t plentiful >>> at all, I actually use this for development, with iPhones which have >>> much more easy to buy Lightning-to-UART adapters. >>> >>> (That’s why the old implementation is there too) >>> >>> Might be worth splitting the new one to a new commit though... >> >> This series is supposed to cover M1 only, and adding extra support >> as part of it is only likely to make the code harder to review. >> >> I'd rather you focus on a single IPI interface (fast or slow, >> I don't really care). Extra features can come in later. > > Agreed. The slow interface is probably easier to start with, > because it avoids hooking into the FIQ, so the FIQ can be > completely decoupled from AIC and just used for the timer. > > Maybe there is even a way to use more than one hardware IPI in > the AIC? > > Arnd Hello, Decided to only have only the slow interface in the second patch series. Thank you,
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Thu, Jan 21, 2021 at 3:10 PM Marc Zyngier wrote: > On 2021-01-21 13:34, Mohamed Mediouni wrote: > >> On 21 Jan 2021, at 14:22, Marc Zyngier wrote: > > Because UART access adapters for the new M1 Macs aren’t plentiful > > at all, I actually use this for development, with iPhones which have > > much more easy to buy Lightning-to-UART adapters. > > > > (That’s why the old implementation is there too) > > > > Might be worth splitting the new one to a new commit though... > > This series is supposed to cover M1 only, and adding extra support > as part of it is only likely to make the code harder to review. > > I'd rather you focus on a single IPI interface (fast or slow, > I don't really care). Extra features can come in later. Agreed. The slow interface is probably easier to start with, because it avoids hooking into the FIQ, so the FIQ can be completely decoupled from AIC and just used for the timer. Maybe there is even a way to use more than one hardware IPI in the AIC? Arnd
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On 2021-01-21 13:34, Mohamed Mediouni wrote: On 21 Jan 2021, at 14:22, Marc Zyngier wrote: On 2021-01-21 12:50, Mohamed Mediouni wrote: On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni [...] + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); Where is this property documented, and what decides which one to use? It’s getting documented in the next patch set. This property is there to enable support for older iPhone processors later on, some of which do not have fast IPI support. On Apple M1, fast-ipi is always on. Then please focus on a single implementation. Additional features can always be merged later once something is up and running. Also, there sysregs can be detected by matching the MIDR, so I don't think we need a DT property for that. Thanks, Because UART access adapters for the new M1 Macs aren’t plentiful at all, I actually use this for development, with iPhones which have much more easy to buy Lightning-to-UART adapters. (That’s why the old implementation is there too) Might be worth splitting the new one to a new commit though... This series is supposed to cover M1 only, and adding extra support as part of it is only likely to make the code harder to review. I'd rather you focus on a single IPI interface (fast or slow, I don't really care). Extra features can come in later. Thanks, M. -- Jazz is not dead. It just smells funny...
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On 2021-01-21 13:32, Mark Rutland wrote: On Thu, Jan 21, 2021 at 01:22:37PM +, Marc Zyngier wrote: On 2021-01-21 12:50, Mohamed Mediouni wrote: > > On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: > > > > On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni [...] > > > + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); > > > > Where is this property documented, and what decides which one to use? > It’s getting documented in the next patch set. > > This property is there to enable support for older iPhone processors > later on, some of which do not have fast IPI support. > > On Apple M1, fast-ipi is always on. Then please focus on a single implementation. Additional features can always be merged later once something is up and running. Also, there sysregs can be detected by matching the MIDR, so I don't think we need a DT property for that. Generally we do not detect IMP-DEF sysregs based on MIDR because they won't necessarily be exposed to a VM, so I suspect that we do need DT properties to describe that IMP-DEF sysregs are accessible, and should not rely on the MIDR alone. Maybe that's implicit in another property, but worth bearing in mind. Hmm. That's a good point. I think this could be keyed off the compatible property, which should accurately reflect the version of the interrupt controller. Thanks, M. -- Jazz is not dead. It just smells funny...
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
> On 21 Jan 2021, at 14:22, Marc Zyngier wrote: > > On 2021-01-21 12:50, Mohamed Mediouni wrote: >>> On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: >>> On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni > > [...] > + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); >>> Where is this property documented, and what decides which one to use? >> It’s getting documented in the next patch set. >> This property is there to enable support for older iPhone processors >> later on, some of which do not have fast IPI support. >> On Apple M1, fast-ipi is always on. > > Then please focus on a single implementation. Additional features can > always be merged later once something is up and running. > > Also, there sysregs can be detected by matching the MIDR, so I don't > think we need a DT property for that. > > Thanks, > Because UART access adapters for the new M1 Macs aren’t plentiful at all, I actually use this for development, with iPhones which have much more easy to buy Lightning-to-UART adapters. (That’s why the old implementation is there too) Might be worth splitting the new one to a new commit though... Thank you, >M. > -- > Jazz is not dead. It just smells funny...
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Thu, Jan 21, 2021 at 01:22:37PM +, Marc Zyngier wrote: > On 2021-01-21 12:50, Mohamed Mediouni wrote: > > > On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: > > > > > > On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni > > [...] > > > > > + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); > > > > > > Where is this property documented, and what decides which one to use? > > It’s getting documented in the next patch set. > > > > This property is there to enable support for older iPhone processors > > later on, some of which do not have fast IPI support. > > > > On Apple M1, fast-ipi is always on. > > Then please focus on a single implementation. Additional features can > always be merged later once something is up and running. > > Also, there sysregs can be detected by matching the MIDR, so I don't > think we need a DT property for that. Generally we do not detect IMP-DEF sysregs based on MIDR because they won't necessarily be exposed to a VM, so I suspect that we do need DT properties to describe that IMP-DEF sysregs are accessible, and should not rely on the MIDR alone. Maybe that's implicit in another property, but worth bearing in mind. Thanks, Mark.
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On 2021-01-21 12:50, Mohamed Mediouni wrote: On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni [...] + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); Where is this property documented, and what decides which one to use? It’s getting documented in the next patch set. This property is there to enable support for older iPhone processors later on, some of which do not have fast IPI support. On Apple M1, fast-ipi is always on. Then please focus on a single implementation. Additional features can always be merged later once something is up and running. Also, there sysregs can be detected by matching the MIDR, so I don't think we need a DT property for that. Thanks, M. -- Jazz is not dead. It just smells funny...
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On 21/01/2021 22.00, Arnd Bergmann wrote: Ok, makes sense. Does fast-ipi mean you cannot use the other mode at all, or is it just faster as implied by the name? If so, how much faster? The other mode still works, as the AIC IPI registers are still there on M1 and work properly. I don't have performance numbers though. -- Hector Martin "marcan" (mar...@marcan.st) Public Key: https://mrcn.st/pub
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Thu, Jan 21, 2021 at 1:50 PM Mohamed Mediouni wrote: > > On 21 Jan 2021, at 13:44, Arnd Bergmann wrote: > >> @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node > >> *node, > >> if (WARN(!aic.base, "unable to map aic registers\n")) > >> return -EINVAL; > >> > >> + aic.fast_ipi = of_property_read_bool(node, "fast-ipi"); > > > > Where is this property documented, and what decides which one to use? > It’s getting documented in the next patch set. > > This property is there to enable support for older iPhone processors > later on, some of which do not have fast IPI support. > > On Apple M1, fast-ipi is always on. Ok, makes sense. Does fast-ipi mean you cannot use the other mode at all, or is it just faster as implied by the name? If so, how much faster? Arnd
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
> On 21 Jan 2021, at 13:44, Arnd Bergmann wrote:
>
> On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni
> wrote:
>
>> +#ifdef CONFIG_SMP
>> +static void apple_aic_ipi_send_mask(struct irq_data *d,
>> + const struct cpumask *mask)
>
> Not sure we care about the #ifdef here, given that arch/arm64 does not
> allow building a kernel without CONFIG_SMP.
>
>> + /*
>> + * Ensure that stores to Normal memory are visible to the
>> + * other CPUs before issuing the IPI.
>> + */
>> + wmb();
>> +
>> + for_each_cpu (cpu, mask) {
>> + smp_mb__before_atomic();
>> + atomic_or(1u << irqnr, per_cpu_ptr(&aic_ipi_mask, cpu));
>> + smp_mb__after_atomic();
>> + lcpu = get_cpu();
>> + if (aic.fast_ipi) {
>> + if ((lcpu >> 2) == (cpu >> 2))
>> + write_sysreg(cpu & 3, SR_APPLE_IPI_LOCAL);
>> + else
>> + write_sysreg((cpu & 3) | ((cpu >> 2) << 16),
>> +SR_APPLE_IPI_REMOTE);
>> + } else
>> + writel(lcpu == cpu ? REG_IPI_FLAG_SELF :
>> + (REG_IPI_FLAG_OTHER <<
>> cpu),
>> + aic.base + REG_IPI_SET);
>> + put_cpu();
>> + }
>> +
>> + /* Force the above writes to be executed */
>> + if (aic.fast_ipi)
>> + isb();
>> +}
>
> Since this just loops over all CPUs, I'd probably just turn it into
> an ipi_send_single() callback and have the caller do the
> loop for simplicity.
>
> I also have the feeling that splitting one hardware IPI into multiple
> logical interrupts, which are then all registered by the same irq
> handler adds a little more complexity than necessary.
>
> Changing this would of course require modifications to
> arch/arm64/kernel/smp.c, which is hardwired to use
> CONFIG_GENERIC_IRQ_IPI in smp_cross_call(), and allowing
> a different code path there may be worse than emulating an
> irqchip.
>
>> @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node
>> *node,
>> if (WARN(!aic.base, "unable to map aic registers\n"))
>> return -EINVAL;
>>
>> + aic.fast_ipi = of_property_read_bool(node, "fast-ipi");
>
> Where is this property documented, and what decides which one to use?
It’s getting documented in the next patch set.
This property is there to enable support for older iPhone processors
later on, some of which do not have fast IPI support.
On Apple M1, fast-ipi is always on.
Thank you,
> Arnd
Re: [RFC PATCH 7/7] irqchip/apple-aic: add SMP support to the Apple AIC driver.
On Wed, Jan 20, 2021 at 2:27 PM Mohamed Mediouni
wrote:
> +#ifdef CONFIG_SMP
> +static void apple_aic_ipi_send_mask(struct irq_data *d,
> + const struct cpumask *mask)
Not sure we care about the #ifdef here, given that arch/arm64 does not
allow building a kernel without CONFIG_SMP.
> + /*
> + * Ensure that stores to Normal memory are visible to the
> + * other CPUs before issuing the IPI.
> + */
> + wmb();
> +
> + for_each_cpu (cpu, mask) {
> + smp_mb__before_atomic();
> + atomic_or(1u << irqnr, per_cpu_ptr(&aic_ipi_mask, cpu));
> + smp_mb__after_atomic();
> + lcpu = get_cpu();
> + if (aic.fast_ipi) {
> + if ((lcpu >> 2) == (cpu >> 2))
> + write_sysreg(cpu & 3, SR_APPLE_IPI_LOCAL);
> + else
> + write_sysreg((cpu & 3) | ((cpu >> 2) << 16),
> +SR_APPLE_IPI_REMOTE);
> + } else
> + writel(lcpu == cpu ? REG_IPI_FLAG_SELF :
> + (REG_IPI_FLAG_OTHER <<
> cpu),
> + aic.base + REG_IPI_SET);
> + put_cpu();
> + }
> +
> + /* Force the above writes to be executed */
> + if (aic.fast_ipi)
> + isb();
> +}
Since this just loops over all CPUs, I'd probably just turn it into
an ipi_send_single() callback and have the caller do the
loop for simplicity.
I also have the feeling that splitting one hardware IPI into multiple
logical interrupts, which are then all registered by the same irq
handler adds a little more complexity than necessary.
Changing this would of course require modifications to
arch/arm64/kernel/smp.c, which is hardwired to use
CONFIG_GENERIC_IRQ_IPI in smp_cross_call(), and allowing
a different code path there may be worse than emulating an
irqchip.
> @@ -186,8 +325,11 @@ static int __init apple_aic_init(struct device_node
> *node,
> if (WARN(!aic.base, "unable to map aic registers\n"))
> return -EINVAL;
>
> + aic.fast_ipi = of_property_read_bool(node, "fast-ipi");
Where is this property documented, and what decides which one to use?
Arnd
