Re: [PATCH] fsl/smp: add low power boot support to replace spin boot
On Thu, 2015-01-15 at 14:05 +0800, Dongsheng Wang wrote: > From: Wang Dongsheng > > U-boot put non-boot cpus into an low power state(PW10/PW20 or DOZE) when cpu > powered up. To exit low power state kernel will send DOORBELL or MPIC-IPI > signal to all those CPUs. U-Boot does not do this yet. A patch was just posted to propose such a change. > @@ -292,11 +315,58 @@ static int smp_85xx_kick_cpu(int nr) > __secondary_hold_acknowledge = -1; > } > #endif > + > flush_spin_table(spin_table); > - out_be32(&spin_table->pir, hw_cpu); > + /* > + * U-boot will wait kernel send eoi to MPIC, after EOI has send > + * kernel will set PIR for uboot, let uboot know EOI has send. > + */ > + out_be32(&spin_table->pir, 0); Why is the kernel sending EOI for an interrupt U-Boot receives? Why are you abusing PIR for anything other than setting the desired PIR value? What happens when you enter the kernel with secondaries having PIR = 0? > + /* Let cpu exit low power state, and from u-boot jump to kernel */ > + arch_send_call_function_single_ipi(kick_cpus); > + > + /* > + * Let we ACK interrput and Send EOI signal to finish INT server > + * U-boot has read EPR to ACK interrput when MPIC work in external > + * proxy mode. Without the external proxy facility, we need to read > + * MPIC ACK register. > + * > + * There just ACK interrput, we don't need to get the interrupt vector > + * and to handle it. Because there just IPI or DOORBELL interrupt to > + * make u-boot exit low power state and jump to kernel. > + */ > + mpic_cpu_ack(nr); How do you know the interrupt is ready to be acked at this point? -Scott ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: [PATCH] fsl/smp: add low power boot support to replace spin boot
Thanks for your review. > -Original Message- > From: Sun York-R58495 > Sent: Friday, January 16, 2015 1:09 AM > To: Wang Dongsheng-B40534; Wood Scott-B07421; Li Yang-Leo-R58472 > Cc: linuxppc-dev@lists.ozlabs.org > Subject: Re: [PATCH] fsl/smp: add low power boot support to replace spin boot > > > > On 01/14/2015 10:05 PM, Dongsheng Wang wrote: > > From: Wang Dongsheng > > > > U-boot put non-boot cpus into an low power state(PW10/PW20 or DOZE) > > when cpu powered up. To exit low power state kernel will send DOORBELL > > or MPIC-IPI signal to all those CPUs. > > > > e500/e500v2 use mpic to send IPI signal. > > e500mc and later use doorbell to send IPI signal. > > > > This feature tested on: > > POWER UP TEST: > > P1022DS(e500v2),96k times. > > P4080(e500mc), 110k times. > > T1024(e5500), 83k times. > > T4240(e6500), 150k times. > > > > CPU HOTPLUG TEST: > > P1022DS(e500v2),1.4 million times. > > P4080(e500mc), 1.8 million times. > > T1024(e5500), 1.3 million times. > > T4240(e6500), 1.1 million times. > > > > Signed-off-by: Wang Dongsheng > > > > This is a very good move. Can you explain what has been tested for 100K times, > or over a million times? > Thanks, I tested linux boot process and cpu hotplug. For Linux boot process test: Each CPU can normally be started and random run an application, the application works well. For cpu hotplug test: Just plug out and plug in for each cpu. Cpu hotplug test works well. If I missed some case please let me know. > Have you verified on older e500v2 platforms, such as MPC8572? > I haven't tested MPC8572 and Just test P1022DS. Because is e500 and e500v2 use the same Low power boot codes. I will test this feature on MPC8572 platform. Regards, -Dongsheng ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH] fsl/smp: add low power boot support to replace spin boot
On 01/14/2015 10:05 PM, Dongsheng Wang wrote: > From: Wang Dongsheng > > U-boot put non-boot cpus into an low power state(PW10/PW20 or DOZE) when cpu > powered up. To exit low power state kernel will send DOORBELL or MPIC-IPI > signal to all those CPUs. > > e500/e500v2 use mpic to send IPI signal. > e500mc and later use doorbell to send IPI signal. > > This feature tested on: > POWER UP TEST: > P1022DS(e500v2),96k times. > P4080(e500mc), 110k times. > T1024(e5500), 83k times. > T4240(e6500), 150k times. > > CPU HOTPLUG TEST: > P1022DS(e500v2),1.4 million times. > P4080(e500mc), 1.8 million times. > T1024(e5500), 1.3 million times. > T4240(e6500), 1.1 million times. > > Signed-off-by: Wang Dongsheng > This is a very good move. Can you explain what has been tested for 100K times, or over a million times? Have you verified on older e500v2 platforms, such as MPC8572? York ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: [PATCH] fsl/smp: add low power boot support to replace spin boot
Hi all,
U-boot patch link:
http://patchwork.ozlabs.org/patch/429265/
Regards,
-Dongsheng
> -Original Message-
> From: Dongsheng Wang [mailto:dongsheng.w...@freescale.com]
> Sent: Thursday, January 15, 2015 2:06 PM
> To: Wood Scott-B07421; Sun York-R58495; Li Yang-Leo-R58472
> Cc: linuxppc-dev@lists.ozlabs.org; Wang Dongsheng-B40534
> Subject: [PATCH] fsl/smp: add low power boot support to replace spin boot
>
> From: Wang Dongsheng
>
> U-boot put non-boot cpus into an low power state(PW10/PW20 or DOZE) when cpu
> powered up. To exit low power state kernel will send DOORBELL or MPIC-IPI
> signal to all those CPUs.
>
> e500/e500v2 use mpic to send IPI signal.
> e500mc and later use doorbell to send IPI signal.
>
> This feature tested on:
> POWER UP TEST:
> P1022DS(e500v2),96k times.
> P4080(e500mc), 110k times.
> T1024(e5500), 83k times.
> T4240(e6500), 150k times.
>
> CPU HOTPLUG TEST:
> P1022DS(e500v2),1.4 million times.
> P4080(e500mc), 1.8 million times.
> T1024(e5500), 1.3 million times.
> T4240(e6500), 1.1 million times.
>
> Signed-off-by: Wang Dongsheng
>
> diff --git a/arch/powerpc/include/asm/mpic.h b/arch/powerpc/include/asm/mpic.h
> index 754f93d..8af6a25 100644
> --- a/arch/powerpc/include/asm/mpic.h
> +++ b/arch/powerpc/include/asm/mpic.h
> @@ -474,6 +474,15 @@ extern int mpic_cpu_get_priority(void);
> /* Set the current cpu priority for this cpu */
> extern void mpic_cpu_set_priority(int prio);
>
> +/* Set cpu priority */
> +void mpic_set_cpu_priority(int nr, int prio);
> +
> +/* Set cpu EOI */
> +void mpic_cpu_eoi_write(int cpu);
> +
> +/* CPU ACK interrupt */
> +void mpic_cpu_ack(int cpu);
> +
> /* Request IPIs on primary mpic */
> extern void mpic_request_ipis(void);
>
> diff --git a/arch/powerpc/platforms/85xx/smp.c
> b/arch/powerpc/platforms/85xx/smp.c
> index d7c1e69..6c54632 100644
> --- a/arch/powerpc/platforms/85xx/smp.c
> +++ b/arch/powerpc/platforms/85xx/smp.c
> @@ -193,6 +193,30 @@ static int smp_85xx_kick_cpu(int nr)
> const u64 *cpu_rel_addr;
> __iomem struct epapr_spin_table *spin_table;
> struct device_node *np;
> +
> + /*
> + * DOORBELL:
> + * When kernel kick one of cpus, all cpus will be wakenup. To make
> + * sure that only the target cpu is effected, other cpus (by checking
> + * spin_table->addr_l) should go back to low power state.
> + *
> + * U-boot has renumber the cpu PIR Why we need to set all of PIR to
> + * the same value?
> + * A: Before kernel kicking cpu, the doorbell message was not configured
> + * for target cpu(cpu_messages->data). If we try to send a
> + * non-configured message to target cpu, it cannot correctly receive
> + * doorbell interrput. So SET ALL OF CPU'S PIR to the same value to
> + * let all cpus catch the interrupt.
> + *
> + * Why set PIR to zero?
> + * A: U-boot cannot know how many cpus will be kicked up(Kernel allow us
> + * to configure NR_CPUS) and IPI is a per_cpu variable, u-boot cannot
> + * set a appropriate PIR for every cpu, but the boot cpu(CPU0) always be
> + * there. U-boot set PIR to zero as a default PIR ID for each CPU, so
> + * initialize the kick_cpus to 0.
> + */
> + u32 kick_cpus = 0;
> +
> int hw_cpu = get_hard_smp_processor_id(nr);
> int ioremappable;
> int ret = 0;
> @@ -251,8 +275,7 @@ static int smp_85xx_kick_cpu(int nr)
> spin_table = phys_to_virt(*cpu_rel_addr);
>
> local_irq_save(flags);
> -#ifdef CONFIG_PPC32
> -#ifdef CONFIG_HOTPLUG_CPU
> +#if defined(CONFIG_PPC32) && defined(CONFIG_HOTPLUG_CPU)
> /* Corresponding to generic_set_cpu_dead() */
> generic_set_cpu_up(nr);
>
> @@ -292,11 +315,58 @@ static int smp_85xx_kick_cpu(int nr)
> __secondary_hold_acknowledge = -1;
> }
> #endif
> +
> flush_spin_table(spin_table);
> - out_be32(&spin_table->pir, hw_cpu);
> + /*
> + * U-boot will wait kernel send eoi to MPIC, after EOI has send
> + * kernel will set PIR for uboot, let uboot know EOI has send.
> + */
> + out_be32(&spin_table->pir, 0);
> +
> +#ifdef CONFIG_PPC32
> out_be32(&spin_table->addr_l, __pa(__early_start));
> +#else
> + out_be64((u64 *)(&spin_table->addr_h),
> + __pa(ppc_function_entry(generic_secondary_smp_init)));
> +#endif
> flush_spin_table(spin_table);
>
> + /*
> + * e500, e500v2 need to use MPIC to send IPI signal, so we need to
> + * open IPI firstly.
> + */
> + if (!cpu_has_feature(CPU_FTR_DBELL)) {
> + mpic_set_cpu_priority(nr, 0);
> + kick_cpus = nr;
> + }
> +
> + /* Let cpu exit low power state, and from u-boot jump to kernel */
> + arch_send_call_function_single_ipi(kick_cpus);
> +
> + /*
> + * Let we ACK interrput and Send EOI signal to finish INT server
> + * U-boot has read EPR to ACK interrput when MPIC work in external
> + *
