RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Hi All,

After further debugging we know the place it hangs.

In function:
static int ath_reset_internal (struct ath_softc *sc, struct ath9k_channel 
*hchan)
{
disable_irq(sc->irq);
tasklet_disable(>intr_tq);
tasklet_disable(>bcon_tasklet);
spin_lock_bh(>sc_pcu_lock);



if (!ath_complete_reset(sc, true))  -> This function enables 
hardware interrupts
r = -EIO;

out:
enable_irq(sc->irq);-> Here IRQ line state is 
changed to enable state
spin_unlock_bh(>sc_pcu_lock);
tasklet_enable(>bcon_tasklet);
tasklet_enable(>intr_tq);

}

static bool ath_complete_reset(struct ath_softc *sc, bool start)
{
struct ath_hw *ah = sc->sc_ah;
struct ath_common *common = ath9k_hw_common(ah);
unsigned long flags;

ath9k_calculate_summary_state(sc, sc->cur_chan);
ath_startrecv(sc);


  
sc->gtt_cnt = 0;

ath9k_hw_set_interrupts(ah);-> Here hardware interrupts are 
being enabled
ath9k_hw_enable_interrupts(ah); -> We see hang after this line
ieee80211_wake_queues(sc->hw);
ath9k_p2p_ps_timer(sc);

return true;
}

Before changing IRQ line to to enabled state, hardware interrupts are being 
enabled. 
Wont this cause a race condition where within this period of hardware raises an 
interrupt, but IRQ line state is disabled state, this will 
reach the following condition making EP handler not being invoked.

void handle_simple_irq(struct irq_desc *desc)
{
raw_spin_lock(>lock);
   ... 
if (unlikely(!desc->action || irqd_irq_disabled(>irq_data))) {
// This condition is reaching and becoming true.
desc->istate |= IRQS_PENDING;
goto out_unlock;
}

kstat_incr_irqs_this_cpu(desc);
handle_irq_event(desc);

out_unlock:
raw_spin_unlock(>lock);
}

We see hang at that statement, without reaching back enable_irq, looks like by 
this time CPU is in stall.

Can any tell why hardware interrupts are being enabled before kernel changing 
IRQ line state?


Regards,
Bharat

> 
> > On Sat, Dec 10, 2016 at 02:40:48PM +, Bharat Kumar Gogada wrote:
> > > Hi,
> > >
> > > After taking some more lecroy traces, we see that after 2nd ASSERT
> > > from EP
> > on ARM64 we see continuous data movement of 32 dwords or 12 dwords and
> > never sign of DEASSERT.
> > > Comparatively on working traces (x86) after 2nd assert there are
> > > only BAR
> > register reads and writes and then DEASSERT, for almost most of the
> > interrupts and we haven't seen 12 or 32 dwords data movement on this trace.
> > >
> > > I did not work on EP wifi/network drivers, any help why EP needs
> > > those many
> > number of data at scan time ?
> >
> > The device doesn't know whether it's in an x86 or an arm64 system.  If
> > it works differently, it must be because the PCI core or the driver is
> > programming the device differently.
> >
> > You should be able to match up Memory transactions from the host in
> > the trace with things the driver does.  For example, if you see an
> > Assert_INTx message from the device, you should eventually see a
> > Memory Read from the host to get the ISR, i.e., some read done in the bowels
> of ath9k_hw_getisr().
> >
> > I don't know how the ath9k device works, but there must be some Memory
> > Read or Write done by the driver that tells the device "we've handled this
> interrupt".
> > The device should then send a Deassert_INTx; of course, if the device
> > still requires service, e.g., because it has received more packets, it
> > might leave the INTx asserted.
> >
> > I doubt you'd see exactly the same traces on x86 and arm64 because
> > they aren't seeing the same network packets and the driver is executing at
> different rates.
> > But you should at least be able to identify interrupt assertion and
> > the actions of the driver's interrupt service routine.
> 
> 
> Thanks Bjorn.
> 
> As you mentioned we did try to debug in that path. After we start scan after 
> 2nd
> ASSERT we see lots of 32 and 12 dword data, and in function void
> ath9k_hw_enable_interrupts(struct ath_hw *ah) {
>   ...
>   ..
>   REG_WRITE(ah, AR_IER, AR_IER_ENABLE);
>   // EP driver hangs at this
> position after 2nd ASSERT
>   // The following writes are not
> happening
> if (!AR_SREV_9100(ah)) {
> REG_WRITE(ah, AR_INTR_ASYNC_ENABLE, async_mask);
> REG_WRITE(ah, AR_INTR_ASYNC_MASK, async_mask);
> 
> REG_WRITE(ah, AR_INTR_SYNC_ENABLE, sync_default);
> REG_WRITE(ah, AR_INTR_SYNC_MASK, sync_default);
> }
> ath_dbg(common, INTERRUPT, "AR_IMR 0x%x IER 0x%x\n",
> REG_READ(ah, AR_IMR), REG_READ(ah, AR_IER)); } 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Hi All,

After further debugging we know the place it hangs.

In function:
static int ath_reset_internal (struct ath_softc *sc, struct ath9k_channel 
*hchan)
{
disable_irq(sc->irq);
tasklet_disable(>intr_tq);
tasklet_disable(>bcon_tasklet);
spin_lock_bh(>sc_pcu_lock);



if (!ath_complete_reset(sc, true))  -> This function enables 
hardware interrupts
r = -EIO;

out:
enable_irq(sc->irq);-> Here IRQ line state is 
changed to enable state
spin_unlock_bh(>sc_pcu_lock);
tasklet_enable(>bcon_tasklet);
tasklet_enable(>intr_tq);

}

static bool ath_complete_reset(struct ath_softc *sc, bool start)
{
struct ath_hw *ah = sc->sc_ah;
struct ath_common *common = ath9k_hw_common(ah);
unsigned long flags;

ath9k_calculate_summary_state(sc, sc->cur_chan);
ath_startrecv(sc);


  
sc->gtt_cnt = 0;

ath9k_hw_set_interrupts(ah);-> Here hardware interrupts are 
being enabled
ath9k_hw_enable_interrupts(ah); -> We see hang after this line
ieee80211_wake_queues(sc->hw);
ath9k_p2p_ps_timer(sc);

return true;
}

Before changing IRQ line to to enabled state, hardware interrupts are being 
enabled. 
Wont this cause a race condition where within this period of hardware raises an 
interrupt, but IRQ line state is disabled state, this will 
reach the following condition making EP handler not being invoked.

void handle_simple_irq(struct irq_desc *desc)
{
raw_spin_lock(>lock);
   ... 
if (unlikely(!desc->action || irqd_irq_disabled(>irq_data))) {
// This condition is reaching and becoming true.
desc->istate |= IRQS_PENDING;
goto out_unlock;
}

kstat_incr_irqs_this_cpu(desc);
handle_irq_event(desc);

out_unlock:
raw_spin_unlock(>lock);
}

We see hang at that statement, without reaching back enable_irq, looks like by 
this time CPU is in stall.

Can any tell why hardware interrupts are being enabled before kernel changing 
IRQ line state?


Regards,
Bharat

> 
> > On Sat, Dec 10, 2016 at 02:40:48PM +, Bharat Kumar Gogada wrote:
> > > Hi,
> > >
> > > After taking some more lecroy traces, we see that after 2nd ASSERT
> > > from EP
> > on ARM64 we see continuous data movement of 32 dwords or 12 dwords and
> > never sign of DEASSERT.
> > > Comparatively on working traces (x86) after 2nd assert there are
> > > only BAR
> > register reads and writes and then DEASSERT, for almost most of the
> > interrupts and we haven't seen 12 or 32 dwords data movement on this trace.
> > >
> > > I did not work on EP wifi/network drivers, any help why EP needs
> > > those many
> > number of data at scan time ?
> >
> > The device doesn't know whether it's in an x86 or an arm64 system.  If
> > it works differently, it must be because the PCI core or the driver is
> > programming the device differently.
> >
> > You should be able to match up Memory transactions from the host in
> > the trace with things the driver does.  For example, if you see an
> > Assert_INTx message from the device, you should eventually see a
> > Memory Read from the host to get the ISR, i.e., some read done in the bowels
> of ath9k_hw_getisr().
> >
> > I don't know how the ath9k device works, but there must be some Memory
> > Read or Write done by the driver that tells the device "we've handled this
> interrupt".
> > The device should then send a Deassert_INTx; of course, if the device
> > still requires service, e.g., because it has received more packets, it
> > might leave the INTx asserted.
> >
> > I doubt you'd see exactly the same traces on x86 and arm64 because
> > they aren't seeing the same network packets and the driver is executing at
> different rates.
> > But you should at least be able to identify interrupt assertion and
> > the actions of the driver's interrupt service routine.
> 
> 
> Thanks Bjorn.
> 
> As you mentioned we did try to debug in that path. After we start scan after 
> 2nd
> ASSERT we see lots of 32 and 12 dword data, and in function void
> ath9k_hw_enable_interrupts(struct ath_hw *ah) {
>   ...
>   ..
>   REG_WRITE(ah, AR_IER, AR_IER_ENABLE);
>   // EP driver hangs at this
> position after 2nd ASSERT
>   // The following writes are not
> happening
> if (!AR_SREV_9100(ah)) {
> REG_WRITE(ah, AR_INTR_ASYNC_ENABLE, async_mask);
> REG_WRITE(ah, AR_INTR_ASYNC_MASK, async_mask);
> 
> REG_WRITE(ah, AR_INTR_SYNC_ENABLE, sync_default);
> REG_WRITE(ah, AR_INTR_SYNC_MASK, sync_default);
> }
> ath_dbg(common, INTERRUPT, "AR_IMR 0x%x IER 0x%x\n",
> REG_READ(ah, AR_IMR), REG_READ(ah, AR_IER)); } 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On Sat, Dec 10, 2016 at 02:40:48PM +, Bharat Kumar Gogada wrote:
> > Hi,
> >
> > After taking some more lecroy traces, we see that after 2nd ASSERT from EP
> on ARM64 we see continuous data movement of 32 dwords or 12 dwords and
> never sign of DEASSERT.
> > Comparatively on working traces (x86) after 2nd assert there are only BAR
> register reads and writes and then DEASSERT, for almost most of the interrupts
> and we haven't seen 12 or 32 dwords data movement on this trace.
> >
> > I did not work on EP wifi/network drivers, any help why EP needs those many
> number of data at scan time ?
> 
> The device doesn't know whether it's in an x86 or an arm64 system.  If it 
> works
> differently, it must be because the PCI core or the driver is programming the
> device differently.
> 
> You should be able to match up Memory transactions from the host in the trace
> with things the driver does.  For example, if you see an Assert_INTx message
> from the device, you should eventually see a Memory Read from the host to get
> the ISR, i.e., some read done in the bowels of ath9k_hw_getisr().
> 
> I don't know how the ath9k device works, but there must be some Memory Read
> or Write done by the driver that tells the device "we've handled this 
> interrupt".
> The device should then send a Deassert_INTx; of course, if the device still
> requires service, e.g., because it has received more packets, it might leave 
> the
> INTx asserted.
> 
> I doubt you'd see exactly the same traces on x86 and arm64 because they aren't
> seeing the same network packets and the driver is executing at different 
> rates.
> But you should at least be able to identify interrupt assertion and the 
> actions of
> the driver's interrupt service routine.


Thanks Bjorn.

As you mentioned we did try to debug in that path. After we start scan after 
2nd ASSERT we see lots of 32 and 12 dword
data, and in function
void ath9k_hw_enable_interrupts(struct ath_hw *ah) 
{
...
..
REG_WRITE(ah, AR_IER, AR_IER_ENABLE);
// EP driver hangs at this 
position after 2nd ASSERT
// The following writes are not 
happening
if (!AR_SREV_9100(ah)) {
REG_WRITE(ah, AR_INTR_ASYNC_ENABLE, async_mask);
REG_WRITE(ah, AR_INTR_ASYNC_MASK, async_mask);

REG_WRITE(ah, AR_INTR_SYNC_ENABLE, sync_default);
REG_WRITE(ah, AR_INTR_SYNC_MASK, sync_default);
}   
ath_dbg(common, INTERRUPT, "AR_IMR 0x%x IER 0x%x\n",
REG_READ(ah, AR_IMR), REG_READ(ah, AR_IER));
}
The above funtion is invoked from tasklet.
I tried several boots every it stops here. The condition (!AR_SREV_9100(ah)) is 
true as per before 1st ASSERT handling.

Regards,
Bharat

> 
> > > Hello there,
> > >
> > > as this is a thread about ath9k and ARM64, i'm not sure if i should
> > > answer here or not, but i have similar "stalls" with ath9k on x86_64
> > > (starting with 4.9rc), stack trace is posted down below where the original
> ARM64 stall traces are.
> > >
> > > Greetings,
> > >
> > > Tobias
> > >
> > >
> > > On 08.12.2016 18:36, Kalle Valo wrote:
> > > > Bharat Kumar Gogada  writes:
> > > >
> > > >>   > [+cc Kalle, ath9k list]
> > > > Thanks, but please also CC linux-wireless. Full thread below for
> > > > the folks there.
> > > >
> > > >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada
> wrote:
> > >  Hi,
> > > 
> > >  Did anyone test Atheros ATH9
> > >  driver(drivers/net/wireless/ath/ath9k/)
> > >  on ARM64.  The end point is TP link wifi card with which
> > >  supports only legacy interrupts.
> > > >>> If it works on other arches and the arm64 PCI enumeration works,
> > > >>> my first guess would be an INTx issue, e.g., maybe the driver is
> > > >>> waiting for an interrupt that never arrives.
> > > >> We are not sure for now.
> > >  We are trying to test it on ARM64 with
> > >  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> > > 
> > >  EP is getting enumerated and able to link up.
> > > 
> > >  But when we start scan system gets hanged.
> > > >>> When you say the system hangs when you start a scan, I assume
> > > >>> you mean a wifi scan, not the PCI enumeration.  A problem with a
> > > >>> wifi scan might cause a *process* to hang, but it shouldn't hang
> > > >>> the entire system.
> > > >>>
> > > >> Yes wifi scan.
> > >  When we took trace we see that after we start scan assert
> > >  message is sent but there is no de assert from end point.
> > > >>> Are you talking about a trace from a PCIe analyzer?  Do you see
> > > >>> an Assert_INTx PCIe message on the link?
> > > >>>
> > > >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx
> > > >> happening
> > > when we do interface link up.
> > > >> When we have less debug prints in Atheros 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On Sat, Dec 10, 2016 at 02:40:48PM +, Bharat Kumar Gogada wrote:
> > Hi,
> >
> > After taking some more lecroy traces, we see that after 2nd ASSERT from EP
> on ARM64 we see continuous data movement of 32 dwords or 12 dwords and
> never sign of DEASSERT.
> > Comparatively on working traces (x86) after 2nd assert there are only BAR
> register reads and writes and then DEASSERT, for almost most of the interrupts
> and we haven't seen 12 or 32 dwords data movement on this trace.
> >
> > I did not work on EP wifi/network drivers, any help why EP needs those many
> number of data at scan time ?
> 
> The device doesn't know whether it's in an x86 or an arm64 system.  If it 
> works
> differently, it must be because the PCI core or the driver is programming the
> device differently.
> 
> You should be able to match up Memory transactions from the host in the trace
> with things the driver does.  For example, if you see an Assert_INTx message
> from the device, you should eventually see a Memory Read from the host to get
> the ISR, i.e., some read done in the bowels of ath9k_hw_getisr().
> 
> I don't know how the ath9k device works, but there must be some Memory Read
> or Write done by the driver that tells the device "we've handled this 
> interrupt".
> The device should then send a Deassert_INTx; of course, if the device still
> requires service, e.g., because it has received more packets, it might leave 
> the
> INTx asserted.
> 
> I doubt you'd see exactly the same traces on x86 and arm64 because they aren't
> seeing the same network packets and the driver is executing at different 
> rates.
> But you should at least be able to identify interrupt assertion and the 
> actions of
> the driver's interrupt service routine.


Thanks Bjorn.

As you mentioned we did try to debug in that path. After we start scan after 
2nd ASSERT we see lots of 32 and 12 dword
data, and in function
void ath9k_hw_enable_interrupts(struct ath_hw *ah) 
{
...
..
REG_WRITE(ah, AR_IER, AR_IER_ENABLE);
// EP driver hangs at this 
position after 2nd ASSERT
// The following writes are not 
happening
if (!AR_SREV_9100(ah)) {
REG_WRITE(ah, AR_INTR_ASYNC_ENABLE, async_mask);
REG_WRITE(ah, AR_INTR_ASYNC_MASK, async_mask);

REG_WRITE(ah, AR_INTR_SYNC_ENABLE, sync_default);
REG_WRITE(ah, AR_INTR_SYNC_MASK, sync_default);
}   
ath_dbg(common, INTERRUPT, "AR_IMR 0x%x IER 0x%x\n",
REG_READ(ah, AR_IMR), REG_READ(ah, AR_IER));
}
The above funtion is invoked from tasklet.
I tried several boots every it stops here. The condition (!AR_SREV_9100(ah)) is 
true as per before 1st ASSERT handling.

Regards,
Bharat

> 
> > > Hello there,
> > >
> > > as this is a thread about ath9k and ARM64, i'm not sure if i should
> > > answer here or not, but i have similar "stalls" with ath9k on x86_64
> > > (starting with 4.9rc), stack trace is posted down below where the original
> ARM64 stall traces are.
> > >
> > > Greetings,
> > >
> > > Tobias
> > >
> > >
> > > On 08.12.2016 18:36, Kalle Valo wrote:
> > > > Bharat Kumar Gogada  writes:
> > > >
> > > >>   > [+cc Kalle, ath9k list]
> > > > Thanks, but please also CC linux-wireless. Full thread below for
> > > > the folks there.
> > > >
> > > >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada
> wrote:
> > >  Hi,
> > > 
> > >  Did anyone test Atheros ATH9
> > >  driver(drivers/net/wireless/ath/ath9k/)
> > >  on ARM64.  The end point is TP link wifi card with which
> > >  supports only legacy interrupts.
> > > >>> If it works on other arches and the arm64 PCI enumeration works,
> > > >>> my first guess would be an INTx issue, e.g., maybe the driver is
> > > >>> waiting for an interrupt that never arrives.
> > > >> We are not sure for now.
> > >  We are trying to test it on ARM64 with
> > >  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> > > 
> > >  EP is getting enumerated and able to link up.
> > > 
> > >  But when we start scan system gets hanged.
> > > >>> When you say the system hangs when you start a scan, I assume
> > > >>> you mean a wifi scan, not the PCI enumeration.  A problem with a
> > > >>> wifi scan might cause a *process* to hang, but it shouldn't hang
> > > >>> the entire system.
> > > >>>
> > > >> Yes wifi scan.
> > >  When we took trace we see that after we start scan assert
> > >  message is sent but there is no de assert from end point.
> > > >>> Are you talking about a trace from a PCIe analyzer?  Do you see
> > > >>> an Assert_INTx PCIe message on the link?
> > > >>>
> > > >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx
> > > >> happening
> > > when we do interface link up.
> > > >> When we have less debug prints in Atheros driver, and do wifi
> > > >> scan we 

Re: ATH9 driver issues on ARM64

[Bjorn Helgaas]

On Sat, Dec 10, 2016 at 02:40:48PM +, Bharat Kumar Gogada wrote:
> Hi,
> 
> After taking some more lecroy traces, we see that after 2nd ASSERT from EP on 
> ARM64 we see continuous data movement of 32 dwords or 12 dwords and never 
> sign of DEASSERT.
> Comparatively on working traces (x86) after 2nd assert there are only BAR 
> register reads and writes and then DEASSERT, for almost most of the 
> interrupts and we haven't seen 12 or 32 dwords data movement on this trace.
> 
> I did not work on EP wifi/network drivers, any help why EP needs those many 
> number of data at scan time ?

The device doesn't know whether it's in an x86 or an arm64 system.  If
it works differently, it must be because the PCI core or the driver is
programming the device differently.

You should be able to match up Memory transactions from the host in
the trace with things the driver does.  For example, if you see an
Assert_INTx message from the device, you should eventually see a
Memory Read from the host to get the ISR, i.e., some read done in the
bowels of ath9k_hw_getisr().

I don't know how the ath9k device works, but there must be some Memory
Read or Write done by the driver that tells the device "we've handled
this interrupt".  The device should then send a Deassert_INTx; of
course, if the device still requires service, e.g., because it has
received more packets, it might leave the INTx asserted.

I doubt you'd see exactly the same traces on x86 and arm64 because
they aren't seeing the same network packets and the driver is
executing at different rates.  But you should at least be able to
identify interrupt assertion and the actions of the driver's interrupt
service routine.

> > Hello there,
> > 
> > as this is a thread about ath9k and ARM64, i'm not sure if i should answer 
> > here
> > or not, but i have similar "stalls" with ath9k on x86_64 (starting with 
> > 4.9rc), stack
> > trace is posted down below where the original ARM64 stall traces are.
> > 
> > Greetings,
> > 
> > Tobias
> > 
> > 
> > On 08.12.2016 18:36, Kalle Valo wrote:
> > > Bharat Kumar Gogada  writes:
> > >
> > >>   > [+cc Kalle, ath9k list]
> > > Thanks, but please also CC linux-wireless. Full thread below for the
> > > folks there.
> > >
> > >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> >  Hi,
> > 
> >  Did anyone test Atheros ATH9
> >  driver(drivers/net/wireless/ath/ath9k/)
> >  on ARM64.  The end point is TP link wifi card with which supports
> >  only legacy interrupts.
> > >>> If it works on other arches and the arm64 PCI enumeration works, my
> > >>> first guess would be an INTx issue, e.g., maybe the driver is
> > >>> waiting for an interrupt that never arrives.
> > >> We are not sure for now.
> >  We are trying to test it on ARM64 with
> >  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> > 
> >  EP is getting enumerated and able to link up.
> > 
> >  But when we start scan system gets hanged.
> > >>> When you say the system hangs when you start a scan, I assume you
> > >>> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> > >>> scan might cause a *process* to hang, but it shouldn't hang the
> > >>> entire system.
> > >>>
> > >> Yes wifi scan.
> >  When we took trace we see that after we start scan assert message
> >  is sent but there is no de assert from end point.
> > >>> Are you talking about a trace from a PCIe analyzer?  Do you see an
> > >>> Assert_INTx PCIe message on the link?
> > >>>
> > >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> > when we do interface link up.
> > >> When we have less debug prints in Atheros driver, and do wifi scan we
> > >> see Assert_INTx but never Deassert_INTx,
> >  What might cause end point not sending de assert ?
> > >>> If the endpoint doesn't send a Deassert_INTx message, I expect that
> > >>> would mean the driver didn't service the interrupt and remove the
> > >>> condition that caused the device to assert the interrupt in the
> > >>> first place.
> > >>>
> > >>> If the driver didn't receive the interrupt, it couldn't service it,
> > >>> of course.  You could add a printk in the ath9k interrupt service
> > >>> routine to see if you ever get there.
> > >>>
> > >> The interrupt behavior is changing w.r.t amount of debug prints we
> > >> add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~# iw 
> > >> dev
> > wlan0 scan
> > >> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> > >> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> > >> [   83.074257] ath9k_hw_kill_interrupts   793
> > >> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> > >> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> > >> [   83.087882] ath9k_hw_kill_interrupts   793
> > >> [   83.095450] ath9k_hw_enable_interrupts 821
> > >> [   83.099557] ath9k_hw_enable_interrupts 825
> > >> [   83.103721] 

Re: ATH9 driver issues on ARM64

[Bjorn Helgaas]

On Sat, Dec 10, 2016 at 02:40:48PM +, Bharat Kumar Gogada wrote:
> Hi,
> 
> After taking some more lecroy traces, we see that after 2nd ASSERT from EP on 
> ARM64 we see continuous data movement of 32 dwords or 12 dwords and never 
> sign of DEASSERT.
> Comparatively on working traces (x86) after 2nd assert there are only BAR 
> register reads and writes and then DEASSERT, for almost most of the 
> interrupts and we haven't seen 12 or 32 dwords data movement on this trace.
> 
> I did not work on EP wifi/network drivers, any help why EP needs those many 
> number of data at scan time ?

The device doesn't know whether it's in an x86 or an arm64 system.  If
it works differently, it must be because the PCI core or the driver is
programming the device differently.

You should be able to match up Memory transactions from the host in
the trace with things the driver does.  For example, if you see an
Assert_INTx message from the device, you should eventually see a
Memory Read from the host to get the ISR, i.e., some read done in the
bowels of ath9k_hw_getisr().

I don't know how the ath9k device works, but there must be some Memory
Read or Write done by the driver that tells the device "we've handled
this interrupt".  The device should then send a Deassert_INTx; of
course, if the device still requires service, e.g., because it has
received more packets, it might leave the INTx asserted.

I doubt you'd see exactly the same traces on x86 and arm64 because
they aren't seeing the same network packets and the driver is
executing at different rates.  But you should at least be able to
identify interrupt assertion and the actions of the driver's interrupt
service routine.

> > Hello there,
> > 
> > as this is a thread about ath9k and ARM64, i'm not sure if i should answer 
> > here
> > or not, but i have similar "stalls" with ath9k on x86_64 (starting with 
> > 4.9rc), stack
> > trace is posted down below where the original ARM64 stall traces are.
> > 
> > Greetings,
> > 
> > Tobias
> > 
> > 
> > On 08.12.2016 18:36, Kalle Valo wrote:
> > > Bharat Kumar Gogada  writes:
> > >
> > >>   > [+cc Kalle, ath9k list]
> > > Thanks, but please also CC linux-wireless. Full thread below for the
> > > folks there.
> > >
> > >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> >  Hi,
> > 
> >  Did anyone test Atheros ATH9
> >  driver(drivers/net/wireless/ath/ath9k/)
> >  on ARM64.  The end point is TP link wifi card with which supports
> >  only legacy interrupts.
> > >>> If it works on other arches and the arm64 PCI enumeration works, my
> > >>> first guess would be an INTx issue, e.g., maybe the driver is
> > >>> waiting for an interrupt that never arrives.
> > >> We are not sure for now.
> >  We are trying to test it on ARM64 with
> >  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> > 
> >  EP is getting enumerated and able to link up.
> > 
> >  But when we start scan system gets hanged.
> > >>> When you say the system hangs when you start a scan, I assume you
> > >>> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> > >>> scan might cause a *process* to hang, but it shouldn't hang the
> > >>> entire system.
> > >>>
> > >> Yes wifi scan.
> >  When we took trace we see that after we start scan assert message
> >  is sent but there is no de assert from end point.
> > >>> Are you talking about a trace from a PCIe analyzer?  Do you see an
> > >>> Assert_INTx PCIe message on the link?
> > >>>
> > >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> > when we do interface link up.
> > >> When we have less debug prints in Atheros driver, and do wifi scan we
> > >> see Assert_INTx but never Deassert_INTx,
> >  What might cause end point not sending de assert ?
> > >>> If the endpoint doesn't send a Deassert_INTx message, I expect that
> > >>> would mean the driver didn't service the interrupt and remove the
> > >>> condition that caused the device to assert the interrupt in the
> > >>> first place.
> > >>>
> > >>> If the driver didn't receive the interrupt, it couldn't service it,
> > >>> of course.  You could add a printk in the ath9k interrupt service
> > >>> routine to see if you ever get there.
> > >>>
> > >> The interrupt behavior is changing w.r.t amount of debug prints we
> > >> add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~# iw 
> > >> dev
> > wlan0 scan
> > >> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> > >> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> > >> [   83.074257] ath9k_hw_kill_interrupts   793
> > >> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> > >> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> > >> [   83.087882] ath9k_hw_kill_interrupts   793
> > >> [   83.095450] ath9k_hw_enable_interrupts 821
> > >> [   83.099557] ath9k_hw_enable_interrupts 825
> > >> [   83.103721] ath9k_hw_enable_interrupts 832

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Hi,

After taking some more lecroy traces, we see that after 2nd ASSERT from EP on 
ARM64 we see continuous data movement of 32 dwords or 12 dwords and never sign 
of DEASSERT.
Comparatively on working traces (x86) after 2nd assert there are only BAR 
register reads and writes and then DEASSERT, for almost most of the interrupts 
and we haven't seen 12 or 32 dwords data movement on this trace.

I did not work on EP wifi/network drivers, any help why EP needs those many 
number of data at scan time ?

Regards,
Bharat

 
> Hello there,
> 
> as this is a thread about ath9k and ARM64, i'm not sure if i should answer 
> here
> or not, but i have similar "stalls" with ath9k on x86_64 (starting with 
> 4.9rc), stack
> trace is posted down below where the original ARM64 stall traces are.
> 
> Greetings,
> 
> Tobias
> 
> 
> On 08.12.2016 18:36, Kalle Valo wrote:
> > Bharat Kumar Gogada  writes:
> >
> >>   > [+cc Kalle, ath9k list]
> > Thanks, but please also CC linux-wireless. Full thread below for the
> > folks there.
> >
> >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
>  Hi,
> 
>  Did anyone test Atheros ATH9
>  driver(drivers/net/wireless/ath/ath9k/)
>  on ARM64.  The end point is TP link wifi card with which supports
>  only legacy interrupts.
> >>> If it works on other arches and the arm64 PCI enumeration works, my
> >>> first guess would be an INTx issue, e.g., maybe the driver is
> >>> waiting for an interrupt that never arrives.
> >> We are not sure for now.
>  We are trying to test it on ARM64 with
>  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> 
>  EP is getting enumerated and able to link up.
> 
>  But when we start scan system gets hanged.
> >>> When you say the system hangs when you start a scan, I assume you
> >>> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> >>> scan might cause a *process* to hang, but it shouldn't hang the
> >>> entire system.
> >>>
> >> Yes wifi scan.
>  When we took trace we see that after we start scan assert message
>  is sent but there is no de assert from end point.
> >>> Are you talking about a trace from a PCIe analyzer?  Do you see an
> >>> Assert_INTx PCIe message on the link?
> >>>
> >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> when we do interface link up.
> >> When we have less debug prints in Atheros driver, and do wifi scan we
> >> see Assert_INTx but never Deassert_INTx,
>  What might cause end point not sending de assert ?
> >>> If the endpoint doesn't send a Deassert_INTx message, I expect that
> >>> would mean the driver didn't service the interrupt and remove the
> >>> condition that caused the device to assert the interrupt in the
> >>> first place.
> >>>
> >>> If the driver didn't receive the interrupt, it couldn't service it,
> >>> of course.  You could add a printk in the ath9k interrupt service
> >>> routine to see if you ever get there.
> >>>
> >> The interrupt behavior is changing w.r.t amount of debug prints we
> >> add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~# iw dev
> wlan0 scan
> >> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.074257] ath9k_hw_kill_interrupts 793
> >> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.087882] ath9k_hw_kill_interrupts 793
> >> [   83.095450] ath9k_hw_enable_interrupts   821
> >> [   83.099557] ath9k_hw_enable_interrupts   825
> >> [   83.103721] ath9k_hw_enable_interrupts   832
> >> [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.112748] AR_SREV_9100 0
> >> [   83.115438] ath9k_hw_enable_interrupts   848
> >> [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.124389] ath9k_hw_intrpend762
> >> [   83.127761] (AR_SREV_9340(ah) val 0
> >> [   83.131234] ath9k_hw_intrpend767
> >> [   83.134628] ath_isr  603
> >> [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.146771] ath9k_hw_kill_interrupts 793
> >> [   83.150864] ath9k_hw_enable_interrupts   821
> >> [   83.154971] ath9k_hw_enable_interrupts   825
> >> [   83.159135] ath9k_hw_enable_interrupts   832
> >> [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.168161] AR_SREV_9100 0
> >> [   83.170852] ath9k_hw_enable_interrupts   848
> >> [   83.170855] ath9k_hw_intrpend762
> >> [   83.178398] (AR_SREV_9340(ah) val 0
> >> [   83.181873] ath9k_hw_intrpend767
> >> [   83.185265] ath_isr  603
> >> [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.197411] ath9k_hw_kill_interrupts 793
> >> [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.206258] ath9k_hw_enable_interrupts 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Hi,

After taking some more lecroy traces, we see that after 2nd ASSERT from EP on 
ARM64 we see continuous data movement of 32 dwords or 12 dwords and never sign 
of DEASSERT.
Comparatively on working traces (x86) after 2nd assert there are only BAR 
register reads and writes and then DEASSERT, for almost most of the interrupts 
and we haven't seen 12 or 32 dwords data movement on this trace.

I did not work on EP wifi/network drivers, any help why EP needs those many 
number of data at scan time ?

Regards,
Bharat

 
> Hello there,
> 
> as this is a thread about ath9k and ARM64, i'm not sure if i should answer 
> here
> or not, but i have similar "stalls" with ath9k on x86_64 (starting with 
> 4.9rc), stack
> trace is posted down below where the original ARM64 stall traces are.
> 
> Greetings,
> 
> Tobias
> 
> 
> On 08.12.2016 18:36, Kalle Valo wrote:
> > Bharat Kumar Gogada  writes:
> >
> >>   > [+cc Kalle, ath9k list]
> > Thanks, but please also CC linux-wireless. Full thread below for the
> > folks there.
> >
> >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
>  Hi,
> 
>  Did anyone test Atheros ATH9
>  driver(drivers/net/wireless/ath/ath9k/)
>  on ARM64.  The end point is TP link wifi card with which supports
>  only legacy interrupts.
> >>> If it works on other arches and the arm64 PCI enumeration works, my
> >>> first guess would be an INTx issue, e.g., maybe the driver is
> >>> waiting for an interrupt that never arrives.
> >> We are not sure for now.
>  We are trying to test it on ARM64 with
>  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> 
>  EP is getting enumerated and able to link up.
> 
>  But when we start scan system gets hanged.
> >>> When you say the system hangs when you start a scan, I assume you
> >>> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> >>> scan might cause a *process* to hang, but it shouldn't hang the
> >>> entire system.
> >>>
> >> Yes wifi scan.
>  When we took trace we see that after we start scan assert message
>  is sent but there is no de assert from end point.
> >>> Are you talking about a trace from a PCIe analyzer?  Do you see an
> >>> Assert_INTx PCIe message on the link?
> >>>
> >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> when we do interface link up.
> >> When we have less debug prints in Atheros driver, and do wifi scan we
> >> see Assert_INTx but never Deassert_INTx,
>  What might cause end point not sending de assert ?
> >>> If the endpoint doesn't send a Deassert_INTx message, I expect that
> >>> would mean the driver didn't service the interrupt and remove the
> >>> condition that caused the device to assert the interrupt in the
> >>> first place.
> >>>
> >>> If the driver didn't receive the interrupt, it couldn't service it,
> >>> of course.  You could add a printk in the ath9k interrupt service
> >>> routine to see if you ever get there.
> >>>
> >> The interrupt behavior is changing w.r.t amount of debug prints we
> >> add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~# iw dev
> wlan0 scan
> >> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.074257] ath9k_hw_kill_interrupts 793
> >> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.087882] ath9k_hw_kill_interrupts 793
> >> [   83.095450] ath9k_hw_enable_interrupts   821
> >> [   83.099557] ath9k_hw_enable_interrupts   825
> >> [   83.103721] ath9k_hw_enable_interrupts   832
> >> [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.112748] AR_SREV_9100 0
> >> [   83.115438] ath9k_hw_enable_interrupts   848
> >> [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.124389] ath9k_hw_intrpend762
> >> [   83.127761] (AR_SREV_9340(ah) val 0
> >> [   83.131234] ath9k_hw_intrpend767
> >> [   83.134628] ath_isr  603
> >> [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.146771] ath9k_hw_kill_interrupts 793
> >> [   83.150864] ath9k_hw_enable_interrupts   821
> >> [   83.154971] ath9k_hw_enable_interrupts   825
> >> [   83.159135] ath9k_hw_enable_interrupts   832
> >> [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.168161] AR_SREV_9100 0
> >> [   83.170852] ath9k_hw_enable_interrupts   848
> >> [   83.170855] ath9k_hw_intrpend762
> >> [   83.178398] (AR_SREV_9340(ah) val 0
> >> [   83.181873] ath9k_hw_intrpend767
> >> [   83.185265] ath_isr  603
> >> [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.197411] ath9k_hw_kill_interrupts 793
> >> [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.206258] ath9k_hw_enable_interrupts   821
> >> [   83.210368] 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

 Correcting Manohar Mail ID.

> Hello there,
> 
> as this is a thread about ath9k and ARM64, i'm not sure if i should
> answer here or not, but i have similar "stalls" with ath9k on x86_64
> (starting with 4.9rc), stack trace is posted down below where the
> original ARM64 stall traces are.
> 
> Greetings,
> 
> Tobias
> 
> 
> On 08.12.2016 18:36, Kalle Valo wrote:
> > Bharat Kumar Gogada  writes:
> >
> >>   > [+cc Kalle, ath9k list]
> > Thanks, but please also CC linux-wireless. Full thread below for the
> > folks there.
> >
> >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
>  Hi,
> 
>  Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
>  on ARM64.  The end point is TP link wifi card with which supports
>  only legacy interrupts.
> >>> If it works on other arches and the arm64 PCI enumeration works, my
> >>> first guess would be an INTx issue, e.g., maybe the driver is waiting
> >>> for an interrupt that never arrives.
> >> We are not sure for now.
>  We are trying to test it on ARM64 with
>  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> 
>  EP is getting enumerated and able to link up.
> 
>  But when we start scan system gets hanged.
> >>> When you say the system hangs when you start a scan, I assume you mean
> >>> a wifi scan, not the PCI enumeration.  A problem with a wifi scan
> >>> might cause a *process* to hang, but it shouldn't hang the entire
> >>> system.
> >>>
> >> Yes wifi scan.
>  When we took trace we see that after we start scan assert message is
>  sent but there is no de assert from end point.
> >>> Are you talking about a trace from a PCIe analyzer?  Do you see an
> >>> Assert_INTx PCIe message on the link?
> >>>
> >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> when we do interface link up.
> >> When we have less debug prints in Atheros driver, and do wifi scan we see
> Assert_INTx but never Deassert_INTx,
>  What might cause end point not sending de assert ?
> >>> If the endpoint doesn't send a Deassert_INTx message, I expect that
> >>> would mean the driver didn't service the interrupt and remove the
> >>> condition that caused the device to assert the interrupt in the first
> >>> place.
> >>>
> >>> If the driver didn't receive the interrupt, it couldn't service it, of
> >>> course.  You could add a printk in the ath9k interrupt service
> >>> routine to see if you ever get there.
> >>>
> >> The interrupt behavior is changing w.r.t amount of debug prints we add. (I
> kept many prints to aid debug)
> >> root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
> >> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.074257] ath9k_hw_kill_interrupts 793
> >> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.087882] ath9k_hw_kill_interrupts 793
> >> [   83.095450] ath9k_hw_enable_interrupts   821
> >> [   83.099557] ath9k_hw_enable_interrupts   825
> >> [   83.103721] ath9k_hw_enable_interrupts   832
> >> [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.112748] AR_SREV_9100 0
> >> [   83.115438] ath9k_hw_enable_interrupts   848
> >> [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.124389] ath9k_hw_intrpend762
> >> [   83.127761] (AR_SREV_9340(ah) val 0
> >> [   83.131234] ath9k_hw_intrpend767
> >> [   83.134628] ath_isr  603
> >> [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.146771] ath9k_hw_kill_interrupts 793
> >> [   83.150864] ath9k_hw_enable_interrupts   821
> >> [   83.154971] ath9k_hw_enable_interrupts   825
> >> [   83.159135] ath9k_hw_enable_interrupts   832
> >> [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.168161] AR_SREV_9100 0
> >> [   83.170852] ath9k_hw_enable_interrupts   848
> >> [   83.170855] ath9k_hw_intrpend762
> >> [   83.178398] (AR_SREV_9340(ah) val 0
> >> [   83.181873] ath9k_hw_intrpend767
> >> [   83.185265] ath_isr  603
> >> [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.197411] ath9k_hw_kill_interrupts 793
> >> [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.206258] ath9k_hw_enable_interrupts   821
> >> [   83.210368] ath9k_hw_enable_interrupts   825
> >> [   83.214531] ath9k_hw_enable_interrupts   832
> >> [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.223558] AR_SREV_9100 0
> >> [   83.226243] ath9k_hw_enable_interrupts   848
> >> [   83.226246] ath9k_hw_intrpend762
> >> [   83.233794] (AR_SREV_9340(ah) val 0
> >> [   83.237268] ath9k_hw_intrpend767
> >> [   83.240661] ath_isr  603
> >> [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> >> [   

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

 Correcting Manohar Mail ID.

> Hello there,
> 
> as this is a thread about ath9k and ARM64, i'm not sure if i should
> answer here or not, but i have similar "stalls" with ath9k on x86_64
> (starting with 4.9rc), stack trace is posted down below where the
> original ARM64 stall traces are.
> 
> Greetings,
> 
> Tobias
> 
> 
> On 08.12.2016 18:36, Kalle Valo wrote:
> > Bharat Kumar Gogada  writes:
> >
> >>   > [+cc Kalle, ath9k list]
> > Thanks, but please also CC linux-wireless. Full thread below for the
> > folks there.
> >
> >>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
>  Hi,
> 
>  Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
>  on ARM64.  The end point is TP link wifi card with which supports
>  only legacy interrupts.
> >>> If it works on other arches and the arm64 PCI enumeration works, my
> >>> first guess would be an INTx issue, e.g., maybe the driver is waiting
> >>> for an interrupt that never arrives.
> >> We are not sure for now.
>  We are trying to test it on ARM64 with
>  (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> 
>  EP is getting enumerated and able to link up.
> 
>  But when we start scan system gets hanged.
> >>> When you say the system hangs when you start a scan, I assume you mean
> >>> a wifi scan, not the PCI enumeration.  A problem with a wifi scan
> >>> might cause a *process* to hang, but it shouldn't hang the entire
> >>> system.
> >>>
> >> Yes wifi scan.
>  When we took trace we see that after we start scan assert message is
>  sent but there is no de assert from end point.
> >>> Are you talking about a trace from a PCIe analyzer?  Do you see an
> >>> Assert_INTx PCIe message on the link?
> >>>
> >> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> when we do interface link up.
> >> When we have less debug prints in Atheros driver, and do wifi scan we see
> Assert_INTx but never Deassert_INTx,
>  What might cause end point not sending de assert ?
> >>> If the endpoint doesn't send a Deassert_INTx message, I expect that
> >>> would mean the driver didn't service the interrupt and remove the
> >>> condition that caused the device to assert the interrupt in the first
> >>> place.
> >>>
> >>> If the driver didn't receive the interrupt, it couldn't service it, of
> >>> course.  You could add a printk in the ath9k interrupt service
> >>> routine to see if you ever get there.
> >>>
> >> The interrupt behavior is changing w.r.t amount of debug prints we add. (I
> kept many prints to aid debug)
> >> root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
> >> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.074257] ath9k_hw_kill_interrupts 793
> >> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.087882] ath9k_hw_kill_interrupts 793
> >> [   83.095450] ath9k_hw_enable_interrupts   821
> >> [   83.099557] ath9k_hw_enable_interrupts   825
> >> [   83.103721] ath9k_hw_enable_interrupts   832
> >> [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.112748] AR_SREV_9100 0
> >> [   83.115438] ath9k_hw_enable_interrupts   848
> >> [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.124389] ath9k_hw_intrpend762
> >> [   83.127761] (AR_SREV_9340(ah) val 0
> >> [   83.131234] ath9k_hw_intrpend767
> >> [   83.134628] ath_isr  603
> >> [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.146771] ath9k_hw_kill_interrupts 793
> >> [   83.150864] ath9k_hw_enable_interrupts   821
> >> [   83.154971] ath9k_hw_enable_interrupts   825
> >> [   83.159135] ath9k_hw_enable_interrupts   832
> >> [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.168161] AR_SREV_9100 0
> >> [   83.170852] ath9k_hw_enable_interrupts   848
> >> [   83.170855] ath9k_hw_intrpend762
> >> [   83.178398] (AR_SREV_9340(ah) val 0
> >> [   83.181873] ath9k_hw_intrpend767
> >> [   83.185265] ath_isr  603
> >> [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.197411] ath9k_hw_kill_interrupts 793
> >> [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> >> [   83.206258] ath9k_hw_enable_interrupts   821
> >> [   83.210368] ath9k_hw_enable_interrupts   825
> >> [   83.214531] ath9k_hw_enable_interrupts   832
> >> [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.223558] AR_SREV_9100 0
> >> [   83.226243] ath9k_hw_enable_interrupts   848
> >> [   83.226246] ath9k_hw_intrpend762
> >> [   83.233794] (AR_SREV_9340(ah) val 0
> >> [   83.237268] ath9k_hw_intrpend767
> >> [   83.240661] ath_isr  603
> >> [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> >> [   83.248030] ath9k: ath9k_ioread32 

Re: ATH9 driver issues on ARM64

[Tobias Klausmann]

Hello there,

as this is a thread about ath9k and ARM64, i'm not sure if i should 
answer here or not, but i have similar "stalls" with ath9k on x86_64 
(starting with 4.9rc), stack trace is posted down below where the 
original ARM64 stall traces are.


Greetings,

Tobias


On 08.12.2016 18:36, Kalle Valo wrote:

Bharat Kumar Gogada  writes:


  > [+cc Kalle, ath9k list]

Thanks, but please also CC linux-wireless. Full thread below for the
folks there.


On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:

Hi,

Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
on ARM64.  The end point is TP link wifi card with which supports
only legacy interrupts.

If it works on other arches and the arm64 PCI enumeration works, my
first guess would be an INTx issue, e.g., maybe the driver is waiting
for an interrupt that never arrives.

We are not sure for now.

We are trying to test it on ARM64 with
(drivers/pci/host/pcie-xilinx-nwl.c) as root port.

EP is getting enumerated and able to link up.

But when we start scan system gets hanged.

When you say the system hangs when you start a scan, I assume you mean
a wifi scan, not the PCI enumeration.  A problem with a wifi scan
might cause a *process* to hang, but it shouldn't hang the entire
system.


Yes wifi scan.

When we took trace we see that after we start scan assert message is
sent but there is no de assert from end point.

Are you talking about a trace from a PCIe analyzer?  Do you see an
Assert_INTx PCIe message on the link?


Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening when we 
do interface link up.
When we have less debug prints in Atheros driver, and do wifi scan we see 
Assert_INTx but never Deassert_INTx,

What might cause end point not sending de assert ?

If the endpoint doesn't send a Deassert_INTx message, I expect that
would mean the driver didn't service the interrupt and remove the
condition that caused the device to assert the interrupt in the first
place.

If the driver didn't receive the interrupt, it couldn't service it, of
course.  You could add a printk in the ath9k interrupt service
routine to see if you ever get there.


The interrupt behavior is changing w.r.t amount of debug prints we add. (I kept 
many prints to aid debug)
root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
[   83.064675] ath9k: ath9k_iowrite32 ff800a400024
[   83.069486] ath9k: ath9k_ioread32 ff800a400024
[   83.074257] ath9k_hw_kill_interrupts  793
[   83.078260] ath9k: ath9k_iowrite32 ff800a400024
[   83.083107] ath9k: ath9k_ioread32 ff800a400024
[   83.087882] ath9k_hw_kill_interrupts  793
[   83.095450] ath9k_hw_enable_interrupts821
[   83.099557] ath9k_hw_enable_interrupts825
[   83.103721] ath9k_hw_enable_interrupts832
[   83.107887] ath9k: ath9k_iowrite32 ff800a400024
[   83.112748] AR_SREV_9100 0
[   83.115438] ath9k_hw_enable_interrupts848
[   83.119607] ath9k: ath9k_ioread32 ff800a400024
[   83.124389] ath9k_hw_intrpend 762
[   83.127761] (AR_SREV_9340(ah) val 0
[   83.131234] ath9k_hw_intrpend 767
[   83.134628] ath_isr   603
[   83.137134] ath9k: ath9k_iowrite32 ff800a400024
[   83.141995] ath9k: ath9k_ioread32 ff800a400024
[   83.146771] ath9k_hw_kill_interrupts  793
[   83.150864] ath9k_hw_enable_interrupts821
[   83.154971] ath9k_hw_enable_interrupts825
[   83.159135] ath9k_hw_enable_interrupts832
[   83.163300] ath9k: ath9k_iowrite32 ff800a400024
[   83.168161] AR_SREV_9100 0
[   83.170852] ath9k_hw_enable_interrupts848
[   83.170855] ath9k_hw_intrpend 762
[   83.178398] (AR_SREV_9340(ah) val 0
[   83.181873] ath9k_hw_intrpend 767
[   83.185265] ath_isr   603
[   83.187773] ath9k: ath9k_iowrite32 ff800a400024
[   83.192635] ath9k: ath9k_ioread32 ff800a400024
[   83.197411] ath9k_hw_kill_interrupts  793
[   83.201414] ath9k: ath9k_ioread32 ff800a400024
[   83.206258] ath9k_hw_enable_interrupts821
[   83.210368] ath9k_hw_enable_interrupts825
[   83.214531] ath9k_hw_enable_interrupts832
[   83.218698] ath9k: ath9k_iowrite32 ff800a400024
[   83.223558] AR_SREV_9100 0
[   83.226243] ath9k_hw_enable_interrupts848
[   83.226246] ath9k_hw_intrpend 762
[   83.233794] (AR_SREV_9340(ah) val 0
[   83.237268] ath9k_hw_intrpend 767
[   83.240661] ath_isr   603
[   83.243169] ath9k: ath9k_iowrite32 ff800a400024
[   83.248030] ath9k: ath9k_ioread32 ff800a400024
[   83.252806] ath9k_hw_kill_interrupts  793
[   83.256811] ath9k: ath9k_ioread32 ff800a400024
[   83.261651] ath9k_hw_enable_interrupts821
[   83.265753] ath9k_hw_enable_interrupts825
[   83.269919] ath9k_hw_enable_interrupts832
[   83.274083] ath9k: ath9k_iowrite32 ff800a400024
[   83.278945] AR_SREV_9100 0
[   83.281630] ath9k_hw_enable_interrupts848
[   83.281633] 

Re: ATH9 driver issues on ARM64

[Tobias Klausmann]

Hello there,

as this is a thread about ath9k and ARM64, i'm not sure if i should 
answer here or not, but i have similar "stalls" with ath9k on x86_64 
(starting with 4.9rc), stack trace is posted down below where the 
original ARM64 stall traces are.


Greetings,

Tobias


On 08.12.2016 18:36, Kalle Valo wrote:

Bharat Kumar Gogada  writes:


  > [+cc Kalle, ath9k list]

Thanks, but please also CC linux-wireless. Full thread below for the
folks there.


On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:

Hi,

Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
on ARM64.  The end point is TP link wifi card with which supports
only legacy interrupts.

If it works on other arches and the arm64 PCI enumeration works, my
first guess would be an INTx issue, e.g., maybe the driver is waiting
for an interrupt that never arrives.

We are not sure for now.

We are trying to test it on ARM64 with
(drivers/pci/host/pcie-xilinx-nwl.c) as root port.

EP is getting enumerated and able to link up.

But when we start scan system gets hanged.

When you say the system hangs when you start a scan, I assume you mean
a wifi scan, not the PCI enumeration.  A problem with a wifi scan
might cause a *process* to hang, but it shouldn't hang the entire
system.


Yes wifi scan.

When we took trace we see that after we start scan assert message is
sent but there is no de assert from end point.

Are you talking about a trace from a PCIe analyzer?  Do you see an
Assert_INTx PCIe message on the link?


Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening when we 
do interface link up.
When we have less debug prints in Atheros driver, and do wifi scan we see 
Assert_INTx but never Deassert_INTx,

What might cause end point not sending de assert ?

If the endpoint doesn't send a Deassert_INTx message, I expect that
would mean the driver didn't service the interrupt and remove the
condition that caused the device to assert the interrupt in the first
place.

If the driver didn't receive the interrupt, it couldn't service it, of
course.  You could add a printk in the ath9k interrupt service
routine to see if you ever get there.


The interrupt behavior is changing w.r.t amount of debug prints we add. (I kept 
many prints to aid debug)
root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
[   83.064675] ath9k: ath9k_iowrite32 ff800a400024
[   83.069486] ath9k: ath9k_ioread32 ff800a400024
[   83.074257] ath9k_hw_kill_interrupts  793
[   83.078260] ath9k: ath9k_iowrite32 ff800a400024
[   83.083107] ath9k: ath9k_ioread32 ff800a400024
[   83.087882] ath9k_hw_kill_interrupts  793
[   83.095450] ath9k_hw_enable_interrupts821
[   83.099557] ath9k_hw_enable_interrupts825
[   83.103721] ath9k_hw_enable_interrupts832
[   83.107887] ath9k: ath9k_iowrite32 ff800a400024
[   83.112748] AR_SREV_9100 0
[   83.115438] ath9k_hw_enable_interrupts848
[   83.119607] ath9k: ath9k_ioread32 ff800a400024
[   83.124389] ath9k_hw_intrpend 762
[   83.127761] (AR_SREV_9340(ah) val 0
[   83.131234] ath9k_hw_intrpend 767
[   83.134628] ath_isr   603
[   83.137134] ath9k: ath9k_iowrite32 ff800a400024
[   83.141995] ath9k: ath9k_ioread32 ff800a400024
[   83.146771] ath9k_hw_kill_interrupts  793
[   83.150864] ath9k_hw_enable_interrupts821
[   83.154971] ath9k_hw_enable_interrupts825
[   83.159135] ath9k_hw_enable_interrupts832
[   83.163300] ath9k: ath9k_iowrite32 ff800a400024
[   83.168161] AR_SREV_9100 0
[   83.170852] ath9k_hw_enable_interrupts848
[   83.170855] ath9k_hw_intrpend 762
[   83.178398] (AR_SREV_9340(ah) val 0
[   83.181873] ath9k_hw_intrpend 767
[   83.185265] ath_isr   603
[   83.187773] ath9k: ath9k_iowrite32 ff800a400024
[   83.192635] ath9k: ath9k_ioread32 ff800a400024
[   83.197411] ath9k_hw_kill_interrupts  793
[   83.201414] ath9k: ath9k_ioread32 ff800a400024
[   83.206258] ath9k_hw_enable_interrupts821
[   83.210368] ath9k_hw_enable_interrupts825
[   83.214531] ath9k_hw_enable_interrupts832
[   83.218698] ath9k: ath9k_iowrite32 ff800a400024
[   83.223558] AR_SREV_9100 0
[   83.226243] ath9k_hw_enable_interrupts848
[   83.226246] ath9k_hw_intrpend 762
[   83.233794] (AR_SREV_9340(ah) val 0
[   83.237268] ath9k_hw_intrpend 767
[   83.240661] ath_isr   603
[   83.243169] ath9k: ath9k_iowrite32 ff800a400024
[   83.248030] ath9k: ath9k_ioread32 ff800a400024
[   83.252806] ath9k_hw_kill_interrupts  793
[   83.256811] ath9k: ath9k_ioread32 ff800a400024
[   83.261651] ath9k_hw_enable_interrupts821
[   83.265753] ath9k_hw_enable_interrupts825
[   83.269919] ath9k_hw_enable_interrupts832
[   83.274083] ath9k: ath9k_iowrite32 ff800a400024
[   83.278945] AR_SREV_9100 0
[   83.281630] ath9k_hw_enable_interrupts848
[   83.281633] ath9k_hw_intrpend 762
[   83.281634] 

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 09/12/16 11:04, Bharat Kumar Gogada wrote:
>> On 09/12/16 02:07, Bharat Kumar Gogada wrote:
 On 08/12/16 18:33, Bharat Kumar Gogada wrote:
>> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
>>> 218: 61  0  0  0 GICv2  81 Level
>>>  mmc0
>>> 219:  0  0  0  0 GICv2 187 Level
>>>  arm-smmu global
>> fault
>>> 220:471  0  0  0 GICv2  53 Level
>>>  xuartps
>>> 223:  0  0  0  0 GICv2 154 Level
>>>  fd4c.dma
>>> 224:  3  0  0  0 dummy   1 Edge 
>>>  ath9k
>>
>> What is this "dummy" controller? And if that's supposed to be a
>> legacy interrupt from the PCI device, it has the wrong trigger.
>
> Yes it is for legacy interrupt, wrong trigger means ?

 Aren't legacy interrupts supposed to be *level* triggered, and not edge?

>>> Yes agreed.
>>> For legacy interrupts im using irq_set_chained_handler_and_data so the
>>> irq line between bridge and GIC Will not be shown here. The above how
>>> is virq for legacy, which is given by kernel, not sure why its state is set 
>>> to edge.
>>
>> Well, you should try and find out. Edge triggering for legacy interrupts is 
>> a real
>> bug, and I don't think it has anything to do with arm64 (despite what the 
>> subject
>> says).
>>
> Thanks Marc. Here is the ARM32 bit log for cat /proc/interrupts, 
> Here also it shows edge but still scan works successfully.

Because it works doesn't mean it is right.

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 09/12/16 11:04, Bharat Kumar Gogada wrote:
>> On 09/12/16 02:07, Bharat Kumar Gogada wrote:
 On 08/12/16 18:33, Bharat Kumar Gogada wrote:
>> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
>>> 218: 61  0  0  0 GICv2  81 Level
>>>  mmc0
>>> 219:  0  0  0  0 GICv2 187 Level
>>>  arm-smmu global
>> fault
>>> 220:471  0  0  0 GICv2  53 Level
>>>  xuartps
>>> 223:  0  0  0  0 GICv2 154 Level
>>>  fd4c.dma
>>> 224:  3  0  0  0 dummy   1 Edge 
>>>  ath9k
>>
>> What is this "dummy" controller? And if that's supposed to be a
>> legacy interrupt from the PCI device, it has the wrong trigger.
>
> Yes it is for legacy interrupt, wrong trigger means ?

 Aren't legacy interrupts supposed to be *level* triggered, and not edge?

>>> Yes agreed.
>>> For legacy interrupts im using irq_set_chained_handler_and_data so the
>>> irq line between bridge and GIC Will not be shown here. The above how
>>> is virq for legacy, which is given by kernel, not sure why its state is set 
>>> to edge.
>>
>> Well, you should try and find out. Edge triggering for legacy interrupts is 
>> a real
>> bug, and I don't think it has anything to do with arm64 (despite what the 
>> subject
>> says).
>>
> Thanks Marc. Here is the ARM32 bit log for cat /proc/interrupts, 
> Here also it shows edge but still scan works successfully.

Because it works doesn't mean it is right.

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On 09/12/16 02:07, Bharat Kumar Gogada wrote:
> >> On 08/12/16 18:33, Bharat Kumar Gogada wrote:
>  On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> > 218: 61  0  0  0 GICv2  81 Level
> >  mmc0
> > 219:  0  0  0  0 GICv2 187 Level
> >  arm-smmu global
> fault
> > 220:471  0  0  0 GICv2  53 Level
> >  xuartps
> > 223:  0  0  0  0 GICv2 154 Level
> >  fd4c.dma
> > 224:  3  0  0  0 dummy   1 Edge 
> >  ath9k
> 
>  What is this "dummy" controller? And if that's supposed to be a
>  legacy interrupt from the PCI device, it has the wrong trigger.
> >>>
> >>> Yes it is for legacy interrupt, wrong trigger means ?
> >>
> >> Aren't legacy interrupts supposed to be *level* triggered, and not edge?
> >>
> > Yes agreed.
> > For legacy interrupts im using irq_set_chained_handler_and_data so the
> > irq line between bridge and GIC Will not be shown here. The above how
> > is virq for legacy, which is given by kernel, not sure why its state is set 
> > to edge.
> 
> Well, you should try and find out. Edge triggering for legacy interrupts is a 
> real
> bug, and I don't think it has anything to do with arm64 (despite what the 
> subject
> says).
> 
Thanks Marc. Here is the ARM32 bit log for cat /proc/interrupts, 
Here also it shows edge but still scan works successfully.
root@:~#cat /proc/interrupts 
   CPU0   CPU1   
 16:  1  0 GIC-0  27 Edge  gt
 17:  0  0 GIC-0  43 Level ttc_clockevent
 18:   3049   1170 GIC-0  29 Edge  twd
 21: 43  0 GIC-0  39 Level f8007100.adc
141:462  0 GIC-0  57 Level cdns-i2c
143:  0  0 GIC-0  35 Level f800c000.ocmc
144:   1259  0 GIC-0  82 Level xuartps
145:  3  0 GIC-0  51 Level e000d000.spi
146:  0  0 GIC-0  54 Level eth0
147: 54  0 GIC-0  56 Level mmc0
148:  0  0 GIC-0  45 Level f8003000.dmac
149:  0  0 GIC-0  46 Level f8003000.dmac
150:  0  0 GIC-0  47 Level f8003000.dmac
151:  0  0 GIC-0  48 Level f8003000.dmac
152:  0  0 GIC-0  49 Level f8003000.dmac
153:  0  0 GIC-0  72 Level f8003000.dmac
154:  0  0 GIC-0  73 Level f8003000.dmac
155:  0  0 GIC-0  74 Level f8003000.dmac
156:  0  0 GIC-0  75 Level f8003000.dmac
157:  0  0 GIC-0  40 Level f8007000.devcfg
163:  0  0 GIC-0  53 Level e0002000.usb
164:  0  0 GIC-0  41 Edge  f8005000.watchdog
165:158  0 GIC-0  61 Level xilinx-pcie
166:122 18 dummy   1 Edge  ath9k
IPI1:  0  0  Timer broadcast interrupts
IPI2:   1101   2349  Rescheduling interrupts
IPI3: 34 20  Function call interrupts
IPI4:  0  0  CPU stop interrupts
IPI5:  0  0  IRQ work interrupts
IPI6:  0  0  completion interrupts

root@Xilinx-ZC706-2016_3:~# iw dev wlnan0 scan
BSS d8:c7:c8:26:6a:72(on wlan0)
TSF: 349496494967 usec (4d, 01:04:56)
freq: 2412
beacon interval: 100 TUs
capability: ESS ShortPreamble ShortSlotTime (0x0421)
signal: -47.00 dBm
last seen: 2170 ms ago
.


Thanks & Regards,
Bharat

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On 09/12/16 02:07, Bharat Kumar Gogada wrote:
> >> On 08/12/16 18:33, Bharat Kumar Gogada wrote:
>  On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> > 218: 61  0  0  0 GICv2  81 Level
> >  mmc0
> > 219:  0  0  0  0 GICv2 187 Level
> >  arm-smmu global
> fault
> > 220:471  0  0  0 GICv2  53 Level
> >  xuartps
> > 223:  0  0  0  0 GICv2 154 Level
> >  fd4c.dma
> > 224:  3  0  0  0 dummy   1 Edge 
> >  ath9k
> 
>  What is this "dummy" controller? And if that's supposed to be a
>  legacy interrupt from the PCI device, it has the wrong trigger.
> >>>
> >>> Yes it is for legacy interrupt, wrong trigger means ?
> >>
> >> Aren't legacy interrupts supposed to be *level* triggered, and not edge?
> >>
> > Yes agreed.
> > For legacy interrupts im using irq_set_chained_handler_and_data so the
> > irq line between bridge and GIC Will not be shown here. The above how
> > is virq for legacy, which is given by kernel, not sure why its state is set 
> > to edge.
> 
> Well, you should try and find out. Edge triggering for legacy interrupts is a 
> real
> bug, and I don't think it has anything to do with arm64 (despite what the 
> subject
> says).
> 
Thanks Marc. Here is the ARM32 bit log for cat /proc/interrupts, 
Here also it shows edge but still scan works successfully.
root@:~#cat /proc/interrupts 
   CPU0   CPU1   
 16:  1  0 GIC-0  27 Edge  gt
 17:  0  0 GIC-0  43 Level ttc_clockevent
 18:   3049   1170 GIC-0  29 Edge  twd
 21: 43  0 GIC-0  39 Level f8007100.adc
141:462  0 GIC-0  57 Level cdns-i2c
143:  0  0 GIC-0  35 Level f800c000.ocmc
144:   1259  0 GIC-0  82 Level xuartps
145:  3  0 GIC-0  51 Level e000d000.spi
146:  0  0 GIC-0  54 Level eth0
147: 54  0 GIC-0  56 Level mmc0
148:  0  0 GIC-0  45 Level f8003000.dmac
149:  0  0 GIC-0  46 Level f8003000.dmac
150:  0  0 GIC-0  47 Level f8003000.dmac
151:  0  0 GIC-0  48 Level f8003000.dmac
152:  0  0 GIC-0  49 Level f8003000.dmac
153:  0  0 GIC-0  72 Level f8003000.dmac
154:  0  0 GIC-0  73 Level f8003000.dmac
155:  0  0 GIC-0  74 Level f8003000.dmac
156:  0  0 GIC-0  75 Level f8003000.dmac
157:  0  0 GIC-0  40 Level f8007000.devcfg
163:  0  0 GIC-0  53 Level e0002000.usb
164:  0  0 GIC-0  41 Edge  f8005000.watchdog
165:158  0 GIC-0  61 Level xilinx-pcie
166:122 18 dummy   1 Edge  ath9k
IPI1:  0  0  Timer broadcast interrupts
IPI2:   1101   2349  Rescheduling interrupts
IPI3: 34 20  Function call interrupts
IPI4:  0  0  CPU stop interrupts
IPI5:  0  0  IRQ work interrupts
IPI6:  0  0  completion interrupts

root@Xilinx-ZC706-2016_3:~# iw dev wlnan0 scan
BSS d8:c7:c8:26:6a:72(on wlan0)
TSF: 349496494967 usec (4d, 01:04:56)
freq: 2412
beacon interval: 100 TUs
capability: ESS ShortPreamble ShortSlotTime (0x0421)
signal: -47.00 dBm
last seen: 2170 ms ago
.


Thanks & Regards,
Bharat

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 09/12/16 02:07, Bharat Kumar Gogada wrote:
>> On 08/12/16 18:33, Bharat Kumar Gogada wrote:
 On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> 218: 61  0  0  0 GICv2  81 Level 
> mmc0
> 219:  0  0  0  0 GICv2 187 Level 
> arm-smmu global fault
> 220:471  0  0  0 GICv2  53 Level 
> xuartps
> 223:  0  0  0  0 GICv2 154 Level 
> fd4c.dma
> 224:  3  0  0  0 dummy   1 Edge  
> ath9k

 What is this "dummy" controller? And if that's supposed to be a
 legacy interrupt from the PCI device, it has the wrong trigger.
>>>
>>> Yes it is for legacy interrupt, wrong trigger means ?
>>
>> Aren't legacy interrupts supposed to be *level* triggered, and not edge?
>>
> Yes agreed.
> For legacy interrupts im using irq_set_chained_handler_and_data so the irq 
> line between bridge and GIC
> Will not be shown here. The above how is virq for legacy, which is given by 
> kernel, not sure why its state is set
> to edge.

Well, you should try and find out. Edge triggering for legacy interrupts
is a real bug, and I don't think it has anything to do with arm64
(despite what the subject says).

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 09/12/16 02:07, Bharat Kumar Gogada wrote:
>> On 08/12/16 18:33, Bharat Kumar Gogada wrote:
 On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> 218: 61  0  0  0 GICv2  81 Level 
> mmc0
> 219:  0  0  0  0 GICv2 187 Level 
> arm-smmu global fault
> 220:471  0  0  0 GICv2  53 Level 
> xuartps
> 223:  0  0  0  0 GICv2 154 Level 
> fd4c.dma
> 224:  3  0  0  0 dummy   1 Edge  
> ath9k

 What is this "dummy" controller? And if that's supposed to be a
 legacy interrupt from the PCI device, it has the wrong trigger.
>>>
>>> Yes it is for legacy interrupt, wrong trigger means ?
>>
>> Aren't legacy interrupts supposed to be *level* triggered, and not edge?
>>
> Yes agreed.
> For legacy interrupts im using irq_set_chained_handler_and_data so the irq 
> line between bridge and GIC
> Will not be shown here. The above how is virq for legacy, which is given by 
> kernel, not sure why its state is set
> to edge.

Well, you should try and find out. Edge triggering for legacy interrupts
is a real bug, and I don't think it has anything to do with arm64
(despite what the subject says).

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Sorry, Forgot to add kernel version, we are using 4.6 kernel. 

> Hi,
> Can any one tell, when exactly the chip sends ASSERT & DEASSERT in driver.
> It might help us to debug issue further.
> 
> Thanks & Regards,
> Bharat
> 
> > >  > [+cc Kalle, ath9k list]
> >
> > Thanks, but please also CC linux-wireless. Full thread below for the folks 
> > there.
> >
> > >> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> > >> > Hi,
> > >> >
> > >> > Did anyone test Atheros ATH9
> > >> > driver(drivers/net/wireless/ath/ath9k/)
> > >> > on ARM64.  The end point is TP link wifi card with which supports
> > >> > only legacy interrupts.
> > >>
> > >> If it works on other arches and the arm64 PCI enumeration works, my
> > >> first guess would be an INTx issue, e.g., maybe the driver is
> > >> waiting for an interrupt that never arrives.
> > > We are not sure for now.
> > >>
> > >> > We are trying to test it on ARM64 with
> > >> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> > >> >
> > >> > EP is getting enumerated and able to link up.
> > >> >
> > >> > But when we start scan system gets hanged.
> > >>
> > >> When you say the system hangs when you start a scan, I assume you
> > >> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> > >> scan might cause a *process* to hang, but it shouldn't hang the
> > >> entire system.
> > >>
> > > Yes wifi scan.
> > >> > When we took trace we see that after we start scan assert message
> > >> > is sent but there is no de assert from end point.
> > >>
> > >> Are you talking about a trace from a PCIe analyzer?  Do you see an
> > >> Assert_INTx PCIe message on the link?
> > >>
> > > Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx
> > > happening
> > when we do interface link up.
> > > When we have less debug prints in Atheros driver, and do wifi scan
> > > we see Assert_INTx but never Deassert_INTx,
> > >> > What might cause end point not sending de assert ?
> > >>
> > >> If the endpoint doesn't send a Deassert_INTx message, I expect that
> > >> would mean the driver didn't service the interrupt and remove the
> > >> condition that caused the device to assert the interrupt in the
> > >> first place.
> > >>
> > >> If the driver didn't receive the interrupt, it couldn't service it,
> > >> of course.  You could add a printk in the ath9k interrupt service
> > >> routine to see if you ever get there.
> > >>
> > > The interrupt behavior is changing w.r.t amount of debug prints we
> > > add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~#
> > > iw dev
> > wlan0 scan
> > > [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.074257] ath9k_hw_kill_interrupts793
> > > [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.087882] ath9k_hw_kill_interrupts793
> > > [   83.095450] ath9k_hw_enable_interrupts  821
> > > [   83.099557] ath9k_hw_enable_interrupts  825
> > > [   83.103721] ath9k_hw_enable_interrupts  832
> > > [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.112748] AR_SREV_9100 0
> > > [   83.115438] ath9k_hw_enable_interrupts  848
> > > [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.124389] ath9k_hw_intrpend   762
> > > [   83.127761] (AR_SREV_9340(ah) val 0
> > > [   83.131234] ath9k_hw_intrpend   767
> > > [   83.134628] ath_isr 603
> > > [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.146771] ath9k_hw_kill_interrupts793
> > > [   83.150864] ath9k_hw_enable_interrupts  821
> > > [   83.154971] ath9k_hw_enable_interrupts  825
> > > [   83.159135] ath9k_hw_enable_interrupts  832
> > > [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.168161] AR_SREV_9100 0
> > > [   83.170852] ath9k_hw_enable_interrupts  848
> > > [   83.170855] ath9k_hw_intrpend   762
> > > [   83.178398] (AR_SREV_9340(ah) val 0
> > > [   83.181873] ath9k_hw_intrpend   767
> > > [   83.185265] ath_isr 603
> > > [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.197411] ath9k_hw_kill_interrupts793
> > > [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.206258] ath9k_hw_enable_interrupts  821
> > > [   83.210368] ath9k_hw_enable_interrupts  825
> > > [   83.214531] ath9k_hw_enable_interrupts  832
> > > [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.223558] AR_SREV_9100 0
> > > [   83.226243] ath9k_hw_enable_interrupts  848
> > > [   83.226246] ath9k_hw_intrpend   762
> > > [   83.233794] (AR_SREV_9340(ah) val 0
> > > [   83.237268] ath9k_hw_intrpend   767
> > > [   83.240661] ath_isr 603
> > > [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.248030] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.252806] 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Sorry, Forgot to add kernel version, we are using 4.6 kernel. 

> Hi,
> Can any one tell, when exactly the chip sends ASSERT & DEASSERT in driver.
> It might help us to debug issue further.
> 
> Thanks & Regards,
> Bharat
> 
> > >  > [+cc Kalle, ath9k list]
> >
> > Thanks, but please also CC linux-wireless. Full thread below for the folks 
> > there.
> >
> > >> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> > >> > Hi,
> > >> >
> > >> > Did anyone test Atheros ATH9
> > >> > driver(drivers/net/wireless/ath/ath9k/)
> > >> > on ARM64.  The end point is TP link wifi card with which supports
> > >> > only legacy interrupts.
> > >>
> > >> If it works on other arches and the arm64 PCI enumeration works, my
> > >> first guess would be an INTx issue, e.g., maybe the driver is
> > >> waiting for an interrupt that never arrives.
> > > We are not sure for now.
> > >>
> > >> > We are trying to test it on ARM64 with
> > >> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> > >> >
> > >> > EP is getting enumerated and able to link up.
> > >> >
> > >> > But when we start scan system gets hanged.
> > >>
> > >> When you say the system hangs when you start a scan, I assume you
> > >> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> > >> scan might cause a *process* to hang, but it shouldn't hang the
> > >> entire system.
> > >>
> > > Yes wifi scan.
> > >> > When we took trace we see that after we start scan assert message
> > >> > is sent but there is no de assert from end point.
> > >>
> > >> Are you talking about a trace from a PCIe analyzer?  Do you see an
> > >> Assert_INTx PCIe message on the link?
> > >>
> > > Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx
> > > happening
> > when we do interface link up.
> > > When we have less debug prints in Atheros driver, and do wifi scan
> > > we see Assert_INTx but never Deassert_INTx,
> > >> > What might cause end point not sending de assert ?
> > >>
> > >> If the endpoint doesn't send a Deassert_INTx message, I expect that
> > >> would mean the driver didn't service the interrupt and remove the
> > >> condition that caused the device to assert the interrupt in the
> > >> first place.
> > >>
> > >> If the driver didn't receive the interrupt, it couldn't service it,
> > >> of course.  You could add a printk in the ath9k interrupt service
> > >> routine to see if you ever get there.
> > >>
> > > The interrupt behavior is changing w.r.t amount of debug prints we
> > > add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~#
> > > iw dev
> > wlan0 scan
> > > [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.074257] ath9k_hw_kill_interrupts793
> > > [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.087882] ath9k_hw_kill_interrupts793
> > > [   83.095450] ath9k_hw_enable_interrupts  821
> > > [   83.099557] ath9k_hw_enable_interrupts  825
> > > [   83.103721] ath9k_hw_enable_interrupts  832
> > > [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.112748] AR_SREV_9100 0
> > > [   83.115438] ath9k_hw_enable_interrupts  848
> > > [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.124389] ath9k_hw_intrpend   762
> > > [   83.127761] (AR_SREV_9340(ah) val 0
> > > [   83.131234] ath9k_hw_intrpend   767
> > > [   83.134628] ath_isr 603
> > > [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.146771] ath9k_hw_kill_interrupts793
> > > [   83.150864] ath9k_hw_enable_interrupts  821
> > > [   83.154971] ath9k_hw_enable_interrupts  825
> > > [   83.159135] ath9k_hw_enable_interrupts  832
> > > [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.168161] AR_SREV_9100 0
> > > [   83.170852] ath9k_hw_enable_interrupts  848
> > > [   83.170855] ath9k_hw_intrpend   762
> > > [   83.178398] (AR_SREV_9340(ah) val 0
> > > [   83.181873] ath9k_hw_intrpend   767
> > > [   83.185265] ath_isr 603
> > > [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.197411] ath9k_hw_kill_interrupts793
> > > [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.206258] ath9k_hw_enable_interrupts  821
> > > [   83.210368] ath9k_hw_enable_interrupts  825
> > > [   83.214531] ath9k_hw_enable_interrupts  832
> > > [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.223558] AR_SREV_9100 0
> > > [   83.226243] ath9k_hw_enable_interrupts  848
> > > [   83.226246] ath9k_hw_intrpend   762
> > > [   83.233794] (AR_SREV_9340(ah) val 0
> > > [   83.237268] ath9k_hw_intrpend   767
> > > [   83.240661] ath_isr 603
> > > [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> > > [   83.248030] ath9k: ath9k_ioread32 ff800a400024
> > > [   83.252806] 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Hi,
Can any one tell, when exactly the chip sends ASSERT & DEASSERT in driver.
It might help us to debug issue further.

Thanks & Regards,
Bharat 

> >  > [+cc Kalle, ath9k list]
> 
> Thanks, but please also CC linux-wireless. Full thread below for the folks 
> there.
> 
> >> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> >> > Hi,
> >> >
> >> > Did anyone test Atheros ATH9
> >> > driver(drivers/net/wireless/ath/ath9k/)
> >> > on ARM64.  The end point is TP link wifi card with which supports
> >> > only legacy interrupts.
> >>
> >> If it works on other arches and the arm64 PCI enumeration works, my
> >> first guess would be an INTx issue, e.g., maybe the driver is waiting
> >> for an interrupt that never arrives.
> > We are not sure for now.
> >>
> >> > We are trying to test it on ARM64 with
> >> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> >> >
> >> > EP is getting enumerated and able to link up.
> >> >
> >> > But when we start scan system gets hanged.
> >>
> >> When you say the system hangs when you start a scan, I assume you
> >> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> >> scan might cause a *process* to hang, but it shouldn't hang the
> >> entire system.
> >>
> > Yes wifi scan.
> >> > When we took trace we see that after we start scan assert message
> >> > is sent but there is no de assert from end point.
> >>
> >> Are you talking about a trace from a PCIe analyzer?  Do you see an
> >> Assert_INTx PCIe message on the link?
> >>
> > Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> when we do interface link up.
> > When we have less debug prints in Atheros driver, and do wifi scan we
> > see Assert_INTx but never Deassert_INTx,
> >> > What might cause end point not sending de assert ?
> >>
> >> If the endpoint doesn't send a Deassert_INTx message, I expect that
> >> would mean the driver didn't service the interrupt and remove the
> >> condition that caused the device to assert the interrupt in the first
> >> place.
> >>
> >> If the driver didn't receive the interrupt, it couldn't service it,
> >> of course.  You could add a printk in the ath9k interrupt service
> >> routine to see if you ever get there.
> >>
> > The interrupt behavior is changing w.r.t amount of debug prints we
> > add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~# iw dev
> wlan0 scan
> > [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> > [   83.074257] ath9k_hw_kill_interrupts  793
> > [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> > [   83.087882] ath9k_hw_kill_interrupts  793
> > [   83.095450] ath9k_hw_enable_interrupts821
> > [   83.099557] ath9k_hw_enable_interrupts825
> > [   83.103721] ath9k_hw_enable_interrupts832
> > [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.112748] AR_SREV_9100 0
> > [   83.115438] ath9k_hw_enable_interrupts848
> > [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> > [   83.124389] ath9k_hw_intrpend 762
> > [   83.127761] (AR_SREV_9340(ah) val 0
> > [   83.131234] ath9k_hw_intrpend 767
> > [   83.134628] ath_isr   603
> > [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> > [   83.146771] ath9k_hw_kill_interrupts  793
> > [   83.150864] ath9k_hw_enable_interrupts821
> > [   83.154971] ath9k_hw_enable_interrupts825
> > [   83.159135] ath9k_hw_enable_interrupts832
> > [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.168161] AR_SREV_9100 0
> > [   83.170852] ath9k_hw_enable_interrupts848
> > [   83.170855] ath9k_hw_intrpend 762
> > [   83.178398] (AR_SREV_9340(ah) val 0
> > [   83.181873] ath9k_hw_intrpend 767
> > [   83.185265] ath_isr   603
> > [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> > [   83.197411] ath9k_hw_kill_interrupts  793
> > [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> > [   83.206258] ath9k_hw_enable_interrupts821
> > [   83.210368] ath9k_hw_enable_interrupts825
> > [   83.214531] ath9k_hw_enable_interrupts832
> > [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.223558] AR_SREV_9100 0
> > [   83.226243] ath9k_hw_enable_interrupts848
> > [   83.226246] ath9k_hw_intrpend 762
> > [   83.233794] (AR_SREV_9340(ah) val 0
> > [   83.237268] ath9k_hw_intrpend 767
> > [   83.240661] ath_isr   603
> > [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.248030] ath9k: ath9k_ioread32 ff800a400024
> > [   83.252806] ath9k_hw_kill_interrupts  793
> > [   83.256811] ath9k: ath9k_ioread32 ff800a400024
> > [   83.261651] ath9k_hw_enable_interrupts821
> > [   83.265753] ath9k_hw_enable_interrupts825
> > [   83.269919] ath9k_hw_enable_interrupts  

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

Hi,
Can any one tell, when exactly the chip sends ASSERT & DEASSERT in driver.
It might help us to debug issue further.

Thanks & Regards,
Bharat 

> >  > [+cc Kalle, ath9k list]
> 
> Thanks, but please also CC linux-wireless. Full thread below for the folks 
> there.
> 
> >> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> >> > Hi,
> >> >
> >> > Did anyone test Atheros ATH9
> >> > driver(drivers/net/wireless/ath/ath9k/)
> >> > on ARM64.  The end point is TP link wifi card with which supports
> >> > only legacy interrupts.
> >>
> >> If it works on other arches and the arm64 PCI enumeration works, my
> >> first guess would be an INTx issue, e.g., maybe the driver is waiting
> >> for an interrupt that never arrives.
> > We are not sure for now.
> >>
> >> > We are trying to test it on ARM64 with
> >> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> >> >
> >> > EP is getting enumerated and able to link up.
> >> >
> >> > But when we start scan system gets hanged.
> >>
> >> When you say the system hangs when you start a scan, I assume you
> >> mean a wifi scan, not the PCI enumeration.  A problem with a wifi
> >> scan might cause a *process* to hang, but it shouldn't hang the
> >> entire system.
> >>
> > Yes wifi scan.
> >> > When we took trace we see that after we start scan assert message
> >> > is sent but there is no de assert from end point.
> >>
> >> Are you talking about a trace from a PCIe analyzer?  Do you see an
> >> Assert_INTx PCIe message on the link?
> >>
> > Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening
> when we do interface link up.
> > When we have less debug prints in Atheros driver, and do wifi scan we
> > see Assert_INTx but never Deassert_INTx,
> >> > What might cause end point not sending de assert ?
> >>
> >> If the endpoint doesn't send a Deassert_INTx message, I expect that
> >> would mean the driver didn't service the interrupt and remove the
> >> condition that caused the device to assert the interrupt in the first
> >> place.
> >>
> >> If the driver didn't receive the interrupt, it couldn't service it,
> >> of course.  You could add a printk in the ath9k interrupt service
> >> routine to see if you ever get there.
> >>
> > The interrupt behavior is changing w.r.t amount of debug prints we
> > add. (I kept many prints to aid debug) root@Xilinx-ZCU102-2016_3:~# iw dev
> wlan0 scan
> > [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> > [   83.074257] ath9k_hw_kill_interrupts  793
> > [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> > [   83.087882] ath9k_hw_kill_interrupts  793
> > [   83.095450] ath9k_hw_enable_interrupts821
> > [   83.099557] ath9k_hw_enable_interrupts825
> > [   83.103721] ath9k_hw_enable_interrupts832
> > [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.112748] AR_SREV_9100 0
> > [   83.115438] ath9k_hw_enable_interrupts848
> > [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> > [   83.124389] ath9k_hw_intrpend 762
> > [   83.127761] (AR_SREV_9340(ah) val 0
> > [   83.131234] ath9k_hw_intrpend 767
> > [   83.134628] ath_isr   603
> > [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> > [   83.146771] ath9k_hw_kill_interrupts  793
> > [   83.150864] ath9k_hw_enable_interrupts821
> > [   83.154971] ath9k_hw_enable_interrupts825
> > [   83.159135] ath9k_hw_enable_interrupts832
> > [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.168161] AR_SREV_9100 0
> > [   83.170852] ath9k_hw_enable_interrupts848
> > [   83.170855] ath9k_hw_intrpend 762
> > [   83.178398] (AR_SREV_9340(ah) val 0
> > [   83.181873] ath9k_hw_intrpend 767
> > [   83.185265] ath_isr   603
> > [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> > [   83.197411] ath9k_hw_kill_interrupts  793
> > [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> > [   83.206258] ath9k_hw_enable_interrupts821
> > [   83.210368] ath9k_hw_enable_interrupts825
> > [   83.214531] ath9k_hw_enable_interrupts832
> > [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.223558] AR_SREV_9100 0
> > [   83.226243] ath9k_hw_enable_interrupts848
> > [   83.226246] ath9k_hw_intrpend 762
> > [   83.233794] (AR_SREV_9340(ah) val 0
> > [   83.237268] ath9k_hw_intrpend 767
> > [   83.240661] ath_isr   603
> > [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> > [   83.248030] ath9k: ath9k_ioread32 ff800a400024
> > [   83.252806] ath9k_hw_kill_interrupts  793
> > [   83.256811] ath9k: ath9k_ioread32 ff800a400024
> > [   83.261651] ath9k_hw_enable_interrupts821
> > [   83.265753] ath9k_hw_enable_interrupts825
> > [   83.269919] ath9k_hw_enable_interrupts  

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> > >> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> > >>
> > >> Two things:
> > >>
> > >>> Here is the cat /proc/interrupts (after we do interface up):
> > >>>
> > >>> root@:~# ifconfig wlan0 up
> > >>> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> > >>> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
> > >>>CPU0   CPU1   CPU2   CPU3
> > >>>   1:  0  0  0  0 GICv2  29 Edge 
> > >>>  arch_timer
> > >>>   2:  19873  20058  19089  17435 GICv2  30 Edge 
> > >>>  arch_timer
> >
> > By the way, please use a recent kernel. Seeing edge here means you're
> > running with something that is a bit old (and broken). And since you
> > haven't even said what revision of the kernel you're using, hslping
> > you is not an easy task. tglx told you something similar about a week ago.
> >
> > >>>  12:  0  0  0  0 GICv2 156 Level
> > >>>  zynqmp-dma
> > >>>  13:  0  0  0  0 GICv2 157 Level
> > >>>  zynqmp-dma
> > >>>  14:  0  0  0  0 GICv2 158 Level
> > >>>  zynqmp-dma
> > >>>  15:  0  0  0  0 GICv2 159 Level
> > >>>  zynqmp-dma
> > >>>  16:  0  0  0  0 GICv2 160 Level
> > >>>  zynqmp-dma
> > >>>  17:  0  0  0  0 GICv2 161 Level
> > >>>  zynqmp-dma
> > >>>  18:  0  0  0  0 GICv2 162 Level
> > >>>  zynqmp-dma
> > >>>  19:  0  0  0  0 GICv2 163 Level
> > >>>  zynqmp-dma
> > >>>  20:  0  0  0  0 GICv2 164 Level
> > >>>  Mali_GP_MMU,
> > Mali_GP,
> > >> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
> > >>
> > >> I'm not even going to consider looking at something that is running
> > >> out of tree code. So please start things with a fresh kernel that
> > >> doesn't contain stuff we can't debug.
> > >>
> > > Ok will test with fresh kernel.
> > >
> > >>>  30:  0  0  0  0 GICv2  95 Level
> > >>>  eth0, eth0
> > >>> 206:314  0  0  0 GICv2  49 Level
> > >>>  cdns-i2c
> > >>> 207: 40  0  0  0 GICv2  50 Level
> > >>>  cdns-i2c
> > >>> 209:  0  0  0  0 GICv2 150 Level
> > >>>  nwl_pcie:misc
> This irq line is handling miscellaneous interrupts this shows level triggered.
> > >>> 214: 12  0  0  0 GICv2  47 Level
> > >>>  ff0f.spi
> > >>> 215:  0  0  0  0 GICv2  58 Level
> > >>>  ffa6.rtc
> > >>> 216:  0  0  0  0 GICv2  59 Level
> > >>>  ffa6.rtc
> > >>> 217:  0  0  0  0 GICv2 165 Level
> > >>>  ahci-
> > ceva[fd0c.ahci]
> > >>> 218: 61  0  0  0 GICv2  81 Level
> > >>>  mmc0
> > >>> 219:  0  0  0  0 GICv2 187 Level
> > >>>  arm-smmu global
> fault
> > >>> 220:471  0  0  0 GICv2  53 Level
> > >>>  xuartps
> > >>> 223:  0  0  0  0 GICv2 154 Level
> > >>>  fd4c.dma
> > >>> 224:  3  0  0  0 dummy   1 Edge 
> > >>>  ath9k
> > >>
> > >> What is this "dummy" controller? And if that's supposed to be a
> > >> legacy interrupt from the PCI device, it has the wrong trigger.
> > >
> > > Yes it is for legacy interrupt, wrong trigger means ?
> >
> > Aren't legacy interrupts supposed to be *level* triggered, and not edge?
> >
> Yes agreed.
> For legacy interrupts im using irq_set_chained_handler_and_data so the irq 
> line
> between bridge and GIC Will not be shown here. The above how is virq for
> legacy, which is given by kernel, not sure why its state is set to edge.
> 
> 

Here I'm adding fresh kernel log: (using 4.6.0 kernel version)
root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts 
   CPU0   CPU1   CPU2   CPU3   
  1:  0  0  0  0 GICv2  29 Edge  
arch_timer
  2:   1368   1294   1655   2657 GICv2  30 Edge  
arch_timer
 30:  0  0  0  0 GICv2  95 Level eth0, 
eth0
206:311  0  0  0 GICv2  49 Level 
cdns-i2c
207: 40  0  0  0 GICv2  50 Level 
cdns-i2c
209:  0  0  0  0 GICv2 150 Level 
nwl_pcie:misc
214:  7  0  0  0 GICv2  47 Level 
ff0f.spi
215:  0  0  0  0 GICv2  58 Level 
ffa6.rtc
216:  0  0  0 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> > >> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> > >>
> > >> Two things:
> > >>
> > >>> Here is the cat /proc/interrupts (after we do interface up):
> > >>>
> > >>> root@:~# ifconfig wlan0 up
> > >>> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> > >>> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
> > >>>CPU0   CPU1   CPU2   CPU3
> > >>>   1:  0  0  0  0 GICv2  29 Edge 
> > >>>  arch_timer
> > >>>   2:  19873  20058  19089  17435 GICv2  30 Edge 
> > >>>  arch_timer
> >
> > By the way, please use a recent kernel. Seeing edge here means you're
> > running with something that is a bit old (and broken). And since you
> > haven't even said what revision of the kernel you're using, hslping
> > you is not an easy task. tglx told you something similar about a week ago.
> >
> > >>>  12:  0  0  0  0 GICv2 156 Level
> > >>>  zynqmp-dma
> > >>>  13:  0  0  0  0 GICv2 157 Level
> > >>>  zynqmp-dma
> > >>>  14:  0  0  0  0 GICv2 158 Level
> > >>>  zynqmp-dma
> > >>>  15:  0  0  0  0 GICv2 159 Level
> > >>>  zynqmp-dma
> > >>>  16:  0  0  0  0 GICv2 160 Level
> > >>>  zynqmp-dma
> > >>>  17:  0  0  0  0 GICv2 161 Level
> > >>>  zynqmp-dma
> > >>>  18:  0  0  0  0 GICv2 162 Level
> > >>>  zynqmp-dma
> > >>>  19:  0  0  0  0 GICv2 163 Level
> > >>>  zynqmp-dma
> > >>>  20:  0  0  0  0 GICv2 164 Level
> > >>>  Mali_GP_MMU,
> > Mali_GP,
> > >> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
> > >>
> > >> I'm not even going to consider looking at something that is running
> > >> out of tree code. So please start things with a fresh kernel that
> > >> doesn't contain stuff we can't debug.
> > >>
> > > Ok will test with fresh kernel.
> > >
> > >>>  30:  0  0  0  0 GICv2  95 Level
> > >>>  eth0, eth0
> > >>> 206:314  0  0  0 GICv2  49 Level
> > >>>  cdns-i2c
> > >>> 207: 40  0  0  0 GICv2  50 Level
> > >>>  cdns-i2c
> > >>> 209:  0  0  0  0 GICv2 150 Level
> > >>>  nwl_pcie:misc
> This irq line is handling miscellaneous interrupts this shows level triggered.
> > >>> 214: 12  0  0  0 GICv2  47 Level
> > >>>  ff0f.spi
> > >>> 215:  0  0  0  0 GICv2  58 Level
> > >>>  ffa6.rtc
> > >>> 216:  0  0  0  0 GICv2  59 Level
> > >>>  ffa6.rtc
> > >>> 217:  0  0  0  0 GICv2 165 Level
> > >>>  ahci-
> > ceva[fd0c.ahci]
> > >>> 218: 61  0  0  0 GICv2  81 Level
> > >>>  mmc0
> > >>> 219:  0  0  0  0 GICv2 187 Level
> > >>>  arm-smmu global
> fault
> > >>> 220:471  0  0  0 GICv2  53 Level
> > >>>  xuartps
> > >>> 223:  0  0  0  0 GICv2 154 Level
> > >>>  fd4c.dma
> > >>> 224:  3  0  0  0 dummy   1 Edge 
> > >>>  ath9k
> > >>
> > >> What is this "dummy" controller? And if that's supposed to be a
> > >> legacy interrupt from the PCI device, it has the wrong trigger.
> > >
> > > Yes it is for legacy interrupt, wrong trigger means ?
> >
> > Aren't legacy interrupts supposed to be *level* triggered, and not edge?
> >
> Yes agreed.
> For legacy interrupts im using irq_set_chained_handler_and_data so the irq 
> line
> between bridge and GIC Will not be shown here. The above how is virq for
> legacy, which is given by kernel, not sure why its state is set to edge.
> 
> 

Here I'm adding fresh kernel log: (using 4.6.0 kernel version)
root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts 
   CPU0   CPU1   CPU2   CPU3   
  1:  0  0  0  0 GICv2  29 Edge  
arch_timer
  2:   1368   1294   1655   2657 GICv2  30 Edge  
arch_timer
 30:  0  0  0  0 GICv2  95 Level eth0, 
eth0
206:311  0  0  0 GICv2  49 Level 
cdns-i2c
207: 40  0  0  0 GICv2  50 Level 
cdns-i2c
209:  0  0  0  0 GICv2 150 Level 
nwl_pcie:misc
214:  7  0  0  0 GICv2  47 Level 
ff0f.spi
215:  0  0  0  0 GICv2  58 Level 
ffa6.rtc
216:  0  0  0 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On 08/12/16 18:33, Bharat Kumar Gogada wrote:
> >> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> >>
> >> Two things:
> >>
> >>> Here is the cat /proc/interrupts (after we do interface up):
> >>>
> >>> root@:~# ifconfig wlan0 up
> >>> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> >>> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
> >>>CPU0   CPU1   CPU2   CPU3
> >>>   1:  0  0  0  0 GICv2  29 Edge  
> >>> arch_timer
> >>>   2:  19873  20058  19089  17435 GICv2  30 Edge  
> >>> arch_timer
> 
> By the way, please use a recent kernel. Seeing edge here means you're running
> with something that is a bit old (and broken). And since you haven't even said
> what revision of the kernel you're using, hslping you is not an easy task. 
> tglx told
> you something similar about a week ago.
> 
> >>>  12:  0  0  0  0 GICv2 156 Level 
> >>> zynqmp-dma
> >>>  13:  0  0  0  0 GICv2 157 Level 
> >>> zynqmp-dma
> >>>  14:  0  0  0  0 GICv2 158 Level 
> >>> zynqmp-dma
> >>>  15:  0  0  0  0 GICv2 159 Level 
> >>> zynqmp-dma
> >>>  16:  0  0  0  0 GICv2 160 Level 
> >>> zynqmp-dma
> >>>  17:  0  0  0  0 GICv2 161 Level 
> >>> zynqmp-dma
> >>>  18:  0  0  0  0 GICv2 162 Level 
> >>> zynqmp-dma
> >>>  19:  0  0  0  0 GICv2 163 Level 
> >>> zynqmp-dma
> >>>  20:  0  0  0  0 GICv2 164 Level 
> >>> Mali_GP_MMU,
> Mali_GP,
> >> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
> >>
> >> I'm not even going to consider looking at something that is running
> >> out of tree code. So please start things with a fresh kernel that
> >> doesn't contain stuff we can't debug.
> >>
> > Ok will test with fresh kernel.
> >
> >>>  30:  0  0  0  0 GICv2  95 Level 
> >>> eth0, eth0
> >>> 206:314  0  0  0 GICv2  49 Level 
> >>> cdns-i2c
> >>> 207: 40  0  0  0 GICv2  50 Level 
> >>> cdns-i2c
> >>> 209:  0  0  0  0 GICv2 150 Level 
> >>> nwl_pcie:misc
This irq line is handling miscellaneous interrupts this shows level triggered. 
> >>> 214: 12  0  0  0 GICv2  47 Level 
> >>> ff0f.spi
> >>> 215:  0  0  0  0 GICv2  58 Level 
> >>> ffa6.rtc
> >>> 216:  0  0  0  0 GICv2  59 Level 
> >>> ffa6.rtc
> >>> 217:  0  0  0  0 GICv2 165 Level 
> >>> ahci-
> ceva[fd0c.ahci]
> >>> 218: 61  0  0  0 GICv2  81 Level 
> >>> mmc0
> >>> 219:  0  0  0  0 GICv2 187 Level 
> >>> arm-smmu global fault
> >>> 220:471  0  0  0 GICv2  53 Level 
> >>> xuartps
> >>> 223:  0  0  0  0 GICv2 154 Level 
> >>> fd4c.dma
> >>> 224:  3  0  0  0 dummy   1 Edge  
> >>> ath9k
> >>
> >> What is this "dummy" controller? And if that's supposed to be a
> >> legacy interrupt from the PCI device, it has the wrong trigger.
> >
> > Yes it is for legacy interrupt, wrong trigger means ?
> 
> Aren't legacy interrupts supposed to be *level* triggered, and not edge?
> 
Yes agreed.
For legacy interrupts im using irq_set_chained_handler_and_data so the irq line 
between bridge and GIC
Will not be shown here. The above how is virq for legacy, which is given by 
kernel, not sure why its state is set
to edge.


Thanks & Regards,
Bharat 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On 08/12/16 18:33, Bharat Kumar Gogada wrote:
> >> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> >>
> >> Two things:
> >>
> >>> Here is the cat /proc/interrupts (after we do interface up):
> >>>
> >>> root@:~# ifconfig wlan0 up
> >>> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> >>> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
> >>>CPU0   CPU1   CPU2   CPU3
> >>>   1:  0  0  0  0 GICv2  29 Edge  
> >>> arch_timer
> >>>   2:  19873  20058  19089  17435 GICv2  30 Edge  
> >>> arch_timer
> 
> By the way, please use a recent kernel. Seeing edge here means you're running
> with something that is a bit old (and broken). And since you haven't even said
> what revision of the kernel you're using, hslping you is not an easy task. 
> tglx told
> you something similar about a week ago.
> 
> >>>  12:  0  0  0  0 GICv2 156 Level 
> >>> zynqmp-dma
> >>>  13:  0  0  0  0 GICv2 157 Level 
> >>> zynqmp-dma
> >>>  14:  0  0  0  0 GICv2 158 Level 
> >>> zynqmp-dma
> >>>  15:  0  0  0  0 GICv2 159 Level 
> >>> zynqmp-dma
> >>>  16:  0  0  0  0 GICv2 160 Level 
> >>> zynqmp-dma
> >>>  17:  0  0  0  0 GICv2 161 Level 
> >>> zynqmp-dma
> >>>  18:  0  0  0  0 GICv2 162 Level 
> >>> zynqmp-dma
> >>>  19:  0  0  0  0 GICv2 163 Level 
> >>> zynqmp-dma
> >>>  20:  0  0  0  0 GICv2 164 Level 
> >>> Mali_GP_MMU,
> Mali_GP,
> >> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
> >>
> >> I'm not even going to consider looking at something that is running
> >> out of tree code. So please start things with a fresh kernel that
> >> doesn't contain stuff we can't debug.
> >>
> > Ok will test with fresh kernel.
> >
> >>>  30:  0  0  0  0 GICv2  95 Level 
> >>> eth0, eth0
> >>> 206:314  0  0  0 GICv2  49 Level 
> >>> cdns-i2c
> >>> 207: 40  0  0  0 GICv2  50 Level 
> >>> cdns-i2c
> >>> 209:  0  0  0  0 GICv2 150 Level 
> >>> nwl_pcie:misc
This irq line is handling miscellaneous interrupts this shows level triggered. 
> >>> 214: 12  0  0  0 GICv2  47 Level 
> >>> ff0f.spi
> >>> 215:  0  0  0  0 GICv2  58 Level 
> >>> ffa6.rtc
> >>> 216:  0  0  0  0 GICv2  59 Level 
> >>> ffa6.rtc
> >>> 217:  0  0  0  0 GICv2 165 Level 
> >>> ahci-
> ceva[fd0c.ahci]
> >>> 218: 61  0  0  0 GICv2  81 Level 
> >>> mmc0
> >>> 219:  0  0  0  0 GICv2 187 Level 
> >>> arm-smmu global fault
> >>> 220:471  0  0  0 GICv2  53 Level 
> >>> xuartps
> >>> 223:  0  0  0  0 GICv2 154 Level 
> >>> fd4c.dma
> >>> 224:  3  0  0  0 dummy   1 Edge  
> >>> ath9k
> >>
> >> What is this "dummy" controller? And if that's supposed to be a
> >> legacy interrupt from the PCI device, it has the wrong trigger.
> >
> > Yes it is for legacy interrupt, wrong trigger means ?
> 
> Aren't legacy interrupts supposed to be *level* triggered, and not edge?
> 
Yes agreed.
For legacy interrupts im using irq_set_chained_handler_and_data so the irq line 
between bridge and GIC
Will not be shown here. The above how is virq for legacy, which is given by 
kernel, not sure why its state is set
to edge.


Thanks & Regards,
Bharat 

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 08/12/16 18:33, Bharat Kumar Gogada wrote:
>> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
>>
>> Two things:
>>
>>> Here is the cat /proc/interrupts (after we do interface up):
>>>
>>> root@:~# ifconfig wlan0 up
>>> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
>>> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
>>>CPU0   CPU1   CPU2   CPU3
>>>   1:  0  0  0  0 GICv2  29 Edge  
>>> arch_timer
>>>   2:  19873  20058  19089  17435 GICv2  30 Edge  
>>> arch_timer

By the way, please use a recent kernel. Seeing edge here means you're
running with something that is a bit old (and broken). And since you
haven't even said what revision of the kernel you're using, hslping you
is not an easy task. tglx told you something similar about a week ago.

>>>  12:  0  0  0  0 GICv2 156 Level 
>>> zynqmp-dma
>>>  13:  0  0  0  0 GICv2 157 Level 
>>> zynqmp-dma
>>>  14:  0  0  0  0 GICv2 158 Level 
>>> zynqmp-dma
>>>  15:  0  0  0  0 GICv2 159 Level 
>>> zynqmp-dma
>>>  16:  0  0  0  0 GICv2 160 Level 
>>> zynqmp-dma
>>>  17:  0  0  0  0 GICv2 161 Level 
>>> zynqmp-dma
>>>  18:  0  0  0  0 GICv2 162 Level 
>>> zynqmp-dma
>>>  19:  0  0  0  0 GICv2 163 Level 
>>> zynqmp-dma
>>>  20:  0  0  0  0 GICv2 164 Level 
>>> Mali_GP_MMU, Mali_GP,
>> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
>>
>> I'm not even going to consider looking at something that is running out of 
>> tree
>> code. So please start things with a fresh kernel that doesn't contain stuff 
>> we
>> can't debug.
>>
> Ok will test with fresh kernel.
> 
>>>  30:  0  0  0  0 GICv2  95 Level 
>>> eth0, eth0
>>> 206:314  0  0  0 GICv2  49 Level 
>>> cdns-i2c
>>> 207: 40  0  0  0 GICv2  50 Level 
>>> cdns-i2c
>>> 209:  0  0  0  0 GICv2 150 Level 
>>> nwl_pcie:misc
>>> 214: 12  0  0  0 GICv2  47 Level 
>>> ff0f.spi
>>> 215:  0  0  0  0 GICv2  58 Level 
>>> ffa6.rtc
>>> 216:  0  0  0  0 GICv2  59 Level 
>>> ffa6.rtc
>>> 217:  0  0  0  0 GICv2 165 Level 
>>> ahci-ceva[fd0c.ahci]
>>> 218: 61  0  0  0 GICv2  81 Level 
>>> mmc0
>>> 219:  0  0  0  0 GICv2 187 Level 
>>> arm-smmu global fault
>>> 220:471  0  0  0 GICv2  53 Level 
>>> xuartps
>>> 223:  0  0  0  0 GICv2 154 Level 
>>> fd4c.dma
>>> 224:  3  0  0  0 dummy   1 Edge  
>>> ath9k
>>
>> What is this "dummy" controller? And if that's supposed to be a legacy 
>> interrupt
>> from the PCI device, it has the wrong trigger.
> 
> Yes it is for legacy interrupt, wrong trigger means ? 

Aren't legacy interrupts supposed to be *level* triggered, and not edge?

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 08/12/16 18:33, Bharat Kumar Gogada wrote:
>> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
>>
>> Two things:
>>
>>> Here is the cat /proc/interrupts (after we do interface up):
>>>
>>> root@:~# ifconfig wlan0 up
>>> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
>>> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
>>>CPU0   CPU1   CPU2   CPU3
>>>   1:  0  0  0  0 GICv2  29 Edge  
>>> arch_timer
>>>   2:  19873  20058  19089  17435 GICv2  30 Edge  
>>> arch_timer

By the way, please use a recent kernel. Seeing edge here means you're
running with something that is a bit old (and broken). And since you
haven't even said what revision of the kernel you're using, hslping you
is not an easy task. tglx told you something similar about a week ago.

>>>  12:  0  0  0  0 GICv2 156 Level 
>>> zynqmp-dma
>>>  13:  0  0  0  0 GICv2 157 Level 
>>> zynqmp-dma
>>>  14:  0  0  0  0 GICv2 158 Level 
>>> zynqmp-dma
>>>  15:  0  0  0  0 GICv2 159 Level 
>>> zynqmp-dma
>>>  16:  0  0  0  0 GICv2 160 Level 
>>> zynqmp-dma
>>>  17:  0  0  0  0 GICv2 161 Level 
>>> zynqmp-dma
>>>  18:  0  0  0  0 GICv2 162 Level 
>>> zynqmp-dma
>>>  19:  0  0  0  0 GICv2 163 Level 
>>> zynqmp-dma
>>>  20:  0  0  0  0 GICv2 164 Level 
>>> Mali_GP_MMU, Mali_GP,
>> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
>>
>> I'm not even going to consider looking at something that is running out of 
>> tree
>> code. So please start things with a fresh kernel that doesn't contain stuff 
>> we
>> can't debug.
>>
> Ok will test with fresh kernel.
> 
>>>  30:  0  0  0  0 GICv2  95 Level 
>>> eth0, eth0
>>> 206:314  0  0  0 GICv2  49 Level 
>>> cdns-i2c
>>> 207: 40  0  0  0 GICv2  50 Level 
>>> cdns-i2c
>>> 209:  0  0  0  0 GICv2 150 Level 
>>> nwl_pcie:misc
>>> 214: 12  0  0  0 GICv2  47 Level 
>>> ff0f.spi
>>> 215:  0  0  0  0 GICv2  58 Level 
>>> ffa6.rtc
>>> 216:  0  0  0  0 GICv2  59 Level 
>>> ffa6.rtc
>>> 217:  0  0  0  0 GICv2 165 Level 
>>> ahci-ceva[fd0c.ahci]
>>> 218: 61  0  0  0 GICv2  81 Level 
>>> mmc0
>>> 219:  0  0  0  0 GICv2 187 Level 
>>> arm-smmu global fault
>>> 220:471  0  0  0 GICv2  53 Level 
>>> xuartps
>>> 223:  0  0  0  0 GICv2 154 Level 
>>> fd4c.dma
>>> 224:  3  0  0  0 dummy   1 Edge  
>>> ath9k
>>
>> What is this "dummy" controller? And if that's supposed to be a legacy 
>> interrupt
>> from the PCI device, it has the wrong trigger.
> 
> Yes it is for legacy interrupt, wrong trigger means ? 

Aren't legacy interrupts supposed to be *level* triggered, and not edge?

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> 
> Two things:
> 
> > Here is the cat /proc/interrupts (after we do interface up):
> >
> > root@:~# ifconfig wlan0 up
> > [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> > root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
> >CPU0   CPU1   CPU2   CPU3
> >   1:  0  0  0  0 GICv2  29 Edge  
> > arch_timer
> >   2:  19873  20058  19089  17435 GICv2  30 Edge  
> > arch_timer
> >  12:  0  0  0  0 GICv2 156 Level 
> > zynqmp-dma
> >  13:  0  0  0  0 GICv2 157 Level 
> > zynqmp-dma
> >  14:  0  0  0  0 GICv2 158 Level 
> > zynqmp-dma
> >  15:  0  0  0  0 GICv2 159 Level 
> > zynqmp-dma
> >  16:  0  0  0  0 GICv2 160 Level 
> > zynqmp-dma
> >  17:  0  0  0  0 GICv2 161 Level 
> > zynqmp-dma
> >  18:  0  0  0  0 GICv2 162 Level 
> > zynqmp-dma
> >  19:  0  0  0  0 GICv2 163 Level 
> > zynqmp-dma
> >  20:  0  0  0  0 GICv2 164 Level 
> > Mali_GP_MMU, Mali_GP,
> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
> 
> I'm not even going to consider looking at something that is running out of 
> tree
> code. So please start things with a fresh kernel that doesn't contain stuff we
> can't debug.
> 
Ok will test with fresh kernel.

> >  30:  0  0  0  0 GICv2  95 Level 
> > eth0, eth0
> > 206:314  0  0  0 GICv2  49 Level 
> > cdns-i2c
> > 207: 40  0  0  0 GICv2  50 Level 
> > cdns-i2c
> > 209:  0  0  0  0 GICv2 150 Level 
> > nwl_pcie:misc
> > 214: 12  0  0  0 GICv2  47 Level 
> > ff0f.spi
> > 215:  0  0  0  0 GICv2  58 Level 
> > ffa6.rtc
> > 216:  0  0  0  0 GICv2  59 Level 
> > ffa6.rtc
> > 217:  0  0  0  0 GICv2 165 Level 
> > ahci-ceva[fd0c.ahci]
> > 218: 61  0  0  0 GICv2  81 Level 
> > mmc0
> > 219:  0  0  0  0 GICv2 187 Level 
> > arm-smmu global fault
> > 220:471  0  0  0 GICv2  53 Level 
> > xuartps
> > 223:  0  0  0  0 GICv2 154 Level 
> > fd4c.dma
> > 224:  3  0  0  0 dummy   1 Edge  
> > ath9k
> 
> What is this "dummy" controller? And if that's supposed to be a legacy 
> interrupt
> from the PCI device, it has the wrong trigger.

Yes it is for legacy interrupt, wrong trigger means ? 

Thanks & Regards,
Bharat

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

> On 08/12/16 15:29, Bharat Kumar Gogada wrote:
> 
> Two things:
> 
> > Here is the cat /proc/interrupts (after we do interface up):
> >
> > root@:~# ifconfig wlan0 up
> > [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> > root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts
> >CPU0   CPU1   CPU2   CPU3
> >   1:  0  0  0  0 GICv2  29 Edge  
> > arch_timer
> >   2:  19873  20058  19089  17435 GICv2  30 Edge  
> > arch_timer
> >  12:  0  0  0  0 GICv2 156 Level 
> > zynqmp-dma
> >  13:  0  0  0  0 GICv2 157 Level 
> > zynqmp-dma
> >  14:  0  0  0  0 GICv2 158 Level 
> > zynqmp-dma
> >  15:  0  0  0  0 GICv2 159 Level 
> > zynqmp-dma
> >  16:  0  0  0  0 GICv2 160 Level 
> > zynqmp-dma
> >  17:  0  0  0  0 GICv2 161 Level 
> > zynqmp-dma
> >  18:  0  0  0  0 GICv2 162 Level 
> > zynqmp-dma
> >  19:  0  0  0  0 GICv2 163 Level 
> > zynqmp-dma
> >  20:  0  0  0  0 GICv2 164 Level 
> > Mali_GP_MMU, Mali_GP,
> Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1
> 
> I'm not even going to consider looking at something that is running out of 
> tree
> code. So please start things with a fresh kernel that doesn't contain stuff we
> can't debug.
> 
Ok will test with fresh kernel.

> >  30:  0  0  0  0 GICv2  95 Level 
> > eth0, eth0
> > 206:314  0  0  0 GICv2  49 Level 
> > cdns-i2c
> > 207: 40  0  0  0 GICv2  50 Level 
> > cdns-i2c
> > 209:  0  0  0  0 GICv2 150 Level 
> > nwl_pcie:misc
> > 214: 12  0  0  0 GICv2  47 Level 
> > ff0f.spi
> > 215:  0  0  0  0 GICv2  58 Level 
> > ffa6.rtc
> > 216:  0  0  0  0 GICv2  59 Level 
> > ffa6.rtc
> > 217:  0  0  0  0 GICv2 165 Level 
> > ahci-ceva[fd0c.ahci]
> > 218: 61  0  0  0 GICv2  81 Level 
> > mmc0
> > 219:  0  0  0  0 GICv2 187 Level 
> > arm-smmu global fault
> > 220:471  0  0  0 GICv2  53 Level 
> > xuartps
> > 223:  0  0  0  0 GICv2 154 Level 
> > fd4c.dma
> > 224:  3  0  0  0 dummy   1 Edge  
> > ath9k
> 
> What is this "dummy" controller? And if that's supposed to be a legacy 
> interrupt
> from the PCI device, it has the wrong trigger.

Yes it is for legacy interrupt, wrong trigger means ? 

Thanks & Regards,
Bharat

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 08/12/16 15:29, Bharat Kumar Gogada wrote:

Two things:

> Here is the cat /proc/interrupts (after we do interface up):
> 
> root@:~# ifconfig wlan0 up
> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts 
>CPU0   CPU1   CPU2   CPU3   
>   1:  0  0  0  0 GICv2  29 Edge  
> arch_timer
>   2:  19873  20058  19089  17435 GICv2  30 Edge  
> arch_timer
>  12:  0  0  0  0 GICv2 156 Level 
> zynqmp-dma
>  13:  0  0  0  0 GICv2 157 Level 
> zynqmp-dma
>  14:  0  0  0  0 GICv2 158 Level 
> zynqmp-dma
>  15:  0  0  0  0 GICv2 159 Level 
> zynqmp-dma
>  16:  0  0  0  0 GICv2 160 Level 
> zynqmp-dma
>  17:  0  0  0  0 GICv2 161 Level 
> zynqmp-dma
>  18:  0  0  0  0 GICv2 162 Level 
> zynqmp-dma
>  19:  0  0  0  0 GICv2 163 Level 
> zynqmp-dma
>  20:  0  0  0  0 GICv2 164 Level 
> Mali_GP_MMU, Mali_GP, Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1

I'm not even going to consider looking at something that is running out
of tree code. So please start things with a fresh kernel that doesn't
contain stuff we can't debug.

>  30:  0  0  0  0 GICv2  95 Level 
> eth0, eth0
> 206:314  0  0  0 GICv2  49 Level 
> cdns-i2c
> 207: 40  0  0  0 GICv2  50 Level 
> cdns-i2c
> 209:  0  0  0  0 GICv2 150 Level 
> nwl_pcie:misc
> 214: 12  0  0  0 GICv2  47 Level 
> ff0f.spi
> 215:  0  0  0  0 GICv2  58 Level 
> ffa6.rtc
> 216:  0  0  0  0 GICv2  59 Level 
> ffa6.rtc
> 217:  0  0  0  0 GICv2 165 Level 
> ahci-ceva[fd0c.ahci]
> 218: 61  0  0  0 GICv2  81 Level mmc0
> 219:  0  0  0  0 GICv2 187 Level 
> arm-smmu global fault
> 220:471  0  0  0 GICv2  53 Level 
> xuartps
> 223:  0  0  0  0 GICv2 154 Level 
> fd4c.dma
> 224:  3  0  0  0 dummy   1 Edge  ath9k

What is this "dummy" controller? And if that's supposed to be a legacy
interrupt from the PCI device, it has the wrong trigger.

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

Re: ATH9 driver issues on ARM64

[Marc Zyngier]

On 08/12/16 15:29, Bharat Kumar Gogada wrote:

Two things:

> Here is the cat /proc/interrupts (after we do interface up):
> 
> root@:~# ifconfig wlan0 up
> [ 1548.926601] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
> root@Xilinx-ZCU102-2016_3:~# cat /proc/interrupts 
>CPU0   CPU1   CPU2   CPU3   
>   1:  0  0  0  0 GICv2  29 Edge  
> arch_timer
>   2:  19873  20058  19089  17435 GICv2  30 Edge  
> arch_timer
>  12:  0  0  0  0 GICv2 156 Level 
> zynqmp-dma
>  13:  0  0  0  0 GICv2 157 Level 
> zynqmp-dma
>  14:  0  0  0  0 GICv2 158 Level 
> zynqmp-dma
>  15:  0  0  0  0 GICv2 159 Level 
> zynqmp-dma
>  16:  0  0  0  0 GICv2 160 Level 
> zynqmp-dma
>  17:  0  0  0  0 GICv2 161 Level 
> zynqmp-dma
>  18:  0  0  0  0 GICv2 162 Level 
> zynqmp-dma
>  19:  0  0  0  0 GICv2 163 Level 
> zynqmp-dma
>  20:  0  0  0  0 GICv2 164 Level 
> Mali_GP_MMU, Mali_GP, Mali_PP0_MMU, Mali_PP0, Mali_PP1_MMU, Mali_PP1

I'm not even going to consider looking at something that is running out
of tree code. So please start things with a fresh kernel that doesn't
contain stuff we can't debug.

>  30:  0  0  0  0 GICv2  95 Level 
> eth0, eth0
> 206:314  0  0  0 GICv2  49 Level 
> cdns-i2c
> 207: 40  0  0  0 GICv2  50 Level 
> cdns-i2c
> 209:  0  0  0  0 GICv2 150 Level 
> nwl_pcie:misc
> 214: 12  0  0  0 GICv2  47 Level 
> ff0f.spi
> 215:  0  0  0  0 GICv2  58 Level 
> ffa6.rtc
> 216:  0  0  0  0 GICv2  59 Level 
> ffa6.rtc
> 217:  0  0  0  0 GICv2 165 Level 
> ahci-ceva[fd0c.ahci]
> 218: 61  0  0  0 GICv2  81 Level mmc0
> 219:  0  0  0  0 GICv2 187 Level 
> arm-smmu global fault
> 220:471  0  0  0 GICv2  53 Level 
> xuartps
> 223:  0  0  0  0 GICv2 154 Level 
> fd4c.dma
> 224:  3  0  0  0 dummy   1 Edge  ath9k

What is this "dummy" controller? And if that's supposed to be a legacy
interrupt from the PCI device, it has the wrong trigger.

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...

Re: ATH9 driver issues on ARM64

[Kalle Valo]

Bharat Kumar Gogada  writes:

>  > [+cc Kalle, ath9k list]

Thanks, but please also CC linux-wireless. Full thread below for the
folks there.

>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
>> > Hi,
>> >
>> > Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
>> > on ARM64.  The end point is TP link wifi card with which supports
>> > only legacy interrupts.
>> 
>> If it works on other arches and the arm64 PCI enumeration works, my
>> first guess would be an INTx issue, e.g., maybe the driver is waiting
>> for an interrupt that never arrives.
> We are not sure for now.
>> 
>> > We are trying to test it on ARM64 with
>> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
>> >
>> > EP is getting enumerated and able to link up.
>> >
>> > But when we start scan system gets hanged.
>> 
>> When you say the system hangs when you start a scan, I assume you mean
>> a wifi scan, not the PCI enumeration.  A problem with a wifi scan
>> might cause a *process* to hang, but it shouldn't hang the entire
>> system.
>> 
> Yes wifi scan.
>> > When we took trace we see that after we start scan assert message is
>> > sent but there is no de assert from end point.
>> 
>> Are you talking about a trace from a PCIe analyzer?  Do you see an
>> Assert_INTx PCIe message on the link?
>> 
> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening when 
> we do interface link up.
> When we have less debug prints in Atheros driver, and do wifi scan we see 
> Assert_INTx but never Deassert_INTx, 
>> > What might cause end point not sending de assert ?
>> 
>> If the endpoint doesn't send a Deassert_INTx message, I expect that
>> would mean the driver didn't service the interrupt and remove the
>> condition that caused the device to assert the interrupt in the first
>> place.
>> 
>> If the driver didn't receive the interrupt, it couldn't service it, of
>> course.  You could add a printk in the ath9k interrupt service
>> routine to see if you ever get there.
>>
> The interrupt behavior is changing w.r.t amount of debug prints we add. (I 
> kept many prints to aid debug)
> root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> [   83.074257] ath9k_hw_kill_interrupts793
> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> [   83.087882] ath9k_hw_kill_interrupts793
> [   83.095450] ath9k_hw_enable_interrupts  821
> [   83.099557] ath9k_hw_enable_interrupts  825
> [   83.103721] ath9k_hw_enable_interrupts  832
> [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> [   83.112748] AR_SREV_9100 0
> [   83.115438] ath9k_hw_enable_interrupts  848
> [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> [   83.124389] ath9k_hw_intrpend   762
> [   83.127761] (AR_SREV_9340(ah) val 0
> [   83.131234] ath9k_hw_intrpend   767
> [   83.134628] ath_isr 603
> [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> [   83.146771] ath9k_hw_kill_interrupts793
> [   83.150864] ath9k_hw_enable_interrupts  821
> [   83.154971] ath9k_hw_enable_interrupts  825
> [   83.159135] ath9k_hw_enable_interrupts  832
> [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> [   83.168161] AR_SREV_9100 0
> [   83.170852] ath9k_hw_enable_interrupts  848
> [   83.170855] ath9k_hw_intrpend   762
> [   83.178398] (AR_SREV_9340(ah) val 0
> [   83.181873] ath9k_hw_intrpend   767
> [   83.185265] ath_isr 603
> [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> [   83.197411] ath9k_hw_kill_interrupts793
> [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> [   83.206258] ath9k_hw_enable_interrupts  821
> [   83.210368] ath9k_hw_enable_interrupts  825
> [   83.214531] ath9k_hw_enable_interrupts  832
> [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> [   83.223558] AR_SREV_9100 0
> [   83.226243] ath9k_hw_enable_interrupts  848
> [   83.226246] ath9k_hw_intrpend   762
> [   83.233794] (AR_SREV_9340(ah) val 0
> [   83.237268] ath9k_hw_intrpend   767
> [   83.240661] ath_isr 603
> [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> [   83.248030] ath9k: ath9k_ioread32 ff800a400024
> [   83.252806] ath9k_hw_kill_interrupts793
> [   83.256811] ath9k: ath9k_ioread32 ff800a400024
> [   83.261651] ath9k_hw_enable_interrupts  821
> [   83.265753] ath9k_hw_enable_interrupts  825
> [   83.269919] ath9k_hw_enable_interrupts  832
> [   83.274083] ath9k: ath9k_iowrite32 ff800a400024
> [   83.278945] AR_SREV_9100 0
> [   83.281630] ath9k_hw_enable_interrupts  848
> [   83.281633] ath9k_hw_intrpend   762
> [   83.281634] (AR_SREV_9340(ah) 

Re: ATH9 driver issues on ARM64

[Kalle Valo]

Bharat Kumar Gogada  writes:

>  > [+cc Kalle, ath9k list]

Thanks, but please also CC linux-wireless. Full thread below for the
folks there.

>> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
>> > Hi,
>> >
>> > Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
>> > on ARM64.  The end point is TP link wifi card with which supports
>> > only legacy interrupts.
>> 
>> If it works on other arches and the arm64 PCI enumeration works, my
>> first guess would be an INTx issue, e.g., maybe the driver is waiting
>> for an interrupt that never arrives.
> We are not sure for now.
>> 
>> > We are trying to test it on ARM64 with
>> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
>> >
>> > EP is getting enumerated and able to link up.
>> >
>> > But when we start scan system gets hanged.
>> 
>> When you say the system hangs when you start a scan, I assume you mean
>> a wifi scan, not the PCI enumeration.  A problem with a wifi scan
>> might cause a *process* to hang, but it shouldn't hang the entire
>> system.
>> 
> Yes wifi scan.
>> > When we took trace we see that after we start scan assert message is
>> > sent but there is no de assert from end point.
>> 
>> Are you talking about a trace from a PCIe analyzer?  Do you see an
>> Assert_INTx PCIe message on the link?
>> 
> Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening when 
> we do interface link up.
> When we have less debug prints in Atheros driver, and do wifi scan we see 
> Assert_INTx but never Deassert_INTx, 
>> > What might cause end point not sending de assert ?
>> 
>> If the endpoint doesn't send a Deassert_INTx message, I expect that
>> would mean the driver didn't service the interrupt and remove the
>> condition that caused the device to assert the interrupt in the first
>> place.
>> 
>> If the driver didn't receive the interrupt, it couldn't service it, of
>> course.  You could add a printk in the ath9k interrupt service
>> routine to see if you ever get there.
>>
> The interrupt behavior is changing w.r.t amount of debug prints we add. (I 
> kept many prints to aid debug)
> root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
> [   83.064675] ath9k: ath9k_iowrite32 ff800a400024
> [   83.069486] ath9k: ath9k_ioread32 ff800a400024
> [   83.074257] ath9k_hw_kill_interrupts793
> [   83.078260] ath9k: ath9k_iowrite32 ff800a400024
> [   83.083107] ath9k: ath9k_ioread32 ff800a400024
> [   83.087882] ath9k_hw_kill_interrupts793
> [   83.095450] ath9k_hw_enable_interrupts  821
> [   83.099557] ath9k_hw_enable_interrupts  825
> [   83.103721] ath9k_hw_enable_interrupts  832
> [   83.107887] ath9k: ath9k_iowrite32 ff800a400024
> [   83.112748] AR_SREV_9100 0
> [   83.115438] ath9k_hw_enable_interrupts  848
> [   83.119607] ath9k: ath9k_ioread32 ff800a400024
> [   83.124389] ath9k_hw_intrpend   762
> [   83.127761] (AR_SREV_9340(ah) val 0
> [   83.131234] ath9k_hw_intrpend   767
> [   83.134628] ath_isr 603
> [   83.137134] ath9k: ath9k_iowrite32 ff800a400024
> [   83.141995] ath9k: ath9k_ioread32 ff800a400024
> [   83.146771] ath9k_hw_kill_interrupts793
> [   83.150864] ath9k_hw_enable_interrupts  821
> [   83.154971] ath9k_hw_enable_interrupts  825
> [   83.159135] ath9k_hw_enable_interrupts  832
> [   83.163300] ath9k: ath9k_iowrite32 ff800a400024
> [   83.168161] AR_SREV_9100 0
> [   83.170852] ath9k_hw_enable_interrupts  848
> [   83.170855] ath9k_hw_intrpend   762
> [   83.178398] (AR_SREV_9340(ah) val 0
> [   83.181873] ath9k_hw_intrpend   767
> [   83.185265] ath_isr 603
> [   83.187773] ath9k: ath9k_iowrite32 ff800a400024
> [   83.192635] ath9k: ath9k_ioread32 ff800a400024
> [   83.197411] ath9k_hw_kill_interrupts793
> [   83.201414] ath9k: ath9k_ioread32 ff800a400024
> [   83.206258] ath9k_hw_enable_interrupts  821
> [   83.210368] ath9k_hw_enable_interrupts  825
> [   83.214531] ath9k_hw_enable_interrupts  832
> [   83.218698] ath9k: ath9k_iowrite32 ff800a400024
> [   83.223558] AR_SREV_9100 0
> [   83.226243] ath9k_hw_enable_interrupts  848
> [   83.226246] ath9k_hw_intrpend   762
> [   83.233794] (AR_SREV_9340(ah) val 0
> [   83.237268] ath9k_hw_intrpend   767
> [   83.240661] ath_isr 603
> [   83.243169] ath9k: ath9k_iowrite32 ff800a400024
> [   83.248030] ath9k: ath9k_ioread32 ff800a400024
> [   83.252806] ath9k_hw_kill_interrupts793
> [   83.256811] ath9k: ath9k_ioread32 ff800a400024
> [   83.261651] ath9k_hw_enable_interrupts  821
> [   83.265753] ath9k_hw_enable_interrupts  825
> [   83.269919] ath9k_hw_enable_interrupts  832
> [   83.274083] ath9k: ath9k_iowrite32 ff800a400024
> [   83.278945] AR_SREV_9100 0
> [   83.281630] ath9k_hw_enable_interrupts  848
> [   83.281633] ath9k_hw_intrpend   762
> [   83.281634] (AR_SREV_9340(ah) val 0
> [   83.281637] 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

 > [+cc Kalle, ath9k list]
> 
> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> > Hi,
> >
> > Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
> > on ARM64.  The end point is TP link wifi card with which supports
> > only legacy interrupts.
> 
> If it works on other arches and the arm64 PCI enumeration works, my
> first guess would be an INTx issue, e.g., maybe the driver is waiting
> for an interrupt that never arrives.
We are not sure for now.
> 
> > We are trying to test it on ARM64 with
> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> >
> > EP is getting enumerated and able to link up.
> >
> > But when we start scan system gets hanged.
> 
> When you say the system hangs when you start a scan, I assume you mean
> a wifi scan, not the PCI enumeration.  A problem with a wifi scan
> might cause a *process* to hang, but it shouldn't hang the entire
> system.
> 
Yes wifi scan.
> > When we took trace we see that after we start scan assert message is
> > sent but there is no de assert from end point.
> 
> Are you talking about a trace from a PCIe analyzer?  Do you see an
> Assert_INTx PCIe message on the link?
> 
Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening when we 
do interface link up.
When we have less debug prints in Atheros driver, and do wifi scan we see 
Assert_INTx but never Deassert_INTx, 
> > What might cause end point not sending de assert ?
> 
> If the endpoint doesn't send a Deassert_INTx message, I expect that
> would mean the driver didn't service the interrupt and remove the
> condition that caused the device to assert the interrupt in the first
> place.
> 
> If the driver didn't receive the interrupt, it couldn't service it, of
> course.  You could add a printk in the ath9k interrupt service
> routine to see if you ever get there.
>
The interrupt behavior is changing w.r.t amount of debug prints we add. (I kept 
many prints to aid debug)
root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
[   83.064675] ath9k: ath9k_iowrite32 ff800a400024
[   83.069486] ath9k: ath9k_ioread32 ff800a400024
[   83.074257] ath9k_hw_kill_interrupts  793
[   83.078260] ath9k: ath9k_iowrite32 ff800a400024
[   83.083107] ath9k: ath9k_ioread32 ff800a400024
[   83.087882] ath9k_hw_kill_interrupts  793
[   83.095450] ath9k_hw_enable_interrupts821
[   83.099557] ath9k_hw_enable_interrupts825
[   83.103721] ath9k_hw_enable_interrupts832
[   83.107887] ath9k: ath9k_iowrite32 ff800a400024
[   83.112748] AR_SREV_9100 0
[   83.115438] ath9k_hw_enable_interrupts848
[   83.119607] ath9k: ath9k_ioread32 ff800a400024
[   83.124389] ath9k_hw_intrpend 762
[   83.127761] (AR_SREV_9340(ah) val 0
[   83.131234] ath9k_hw_intrpend 767
[   83.134628] ath_isr   603
[   83.137134] ath9k: ath9k_iowrite32 ff800a400024
[   83.141995] ath9k: ath9k_ioread32 ff800a400024
[   83.146771] ath9k_hw_kill_interrupts  793
[   83.150864] ath9k_hw_enable_interrupts821
[   83.154971] ath9k_hw_enable_interrupts825
[   83.159135] ath9k_hw_enable_interrupts832
[   83.163300] ath9k: ath9k_iowrite32 ff800a400024
[   83.168161] AR_SREV_9100 0
[   83.170852] ath9k_hw_enable_interrupts848
[   83.170855] ath9k_hw_intrpend 762
[   83.178398] (AR_SREV_9340(ah) val 0
[   83.181873] ath9k_hw_intrpend 767
[   83.185265] ath_isr   603
[   83.187773] ath9k: ath9k_iowrite32 ff800a400024
[   83.192635] ath9k: ath9k_ioread32 ff800a400024
[   83.197411] ath9k_hw_kill_interrupts  793
[   83.201414] ath9k: ath9k_ioread32 ff800a400024
[   83.206258] ath9k_hw_enable_interrupts821
[   83.210368] ath9k_hw_enable_interrupts825
[   83.214531] ath9k_hw_enable_interrupts832
[   83.218698] ath9k: ath9k_iowrite32 ff800a400024
[   83.223558] AR_SREV_9100 0
[   83.226243] ath9k_hw_enable_interrupts848
[   83.226246] ath9k_hw_intrpend 762
[   83.233794] (AR_SREV_9340(ah) val 0
[   83.237268] ath9k_hw_intrpend 767
[   83.240661] ath_isr   603
[   83.243169] ath9k: ath9k_iowrite32 ff800a400024
[   83.248030] ath9k: ath9k_ioread32 ff800a400024
[   83.252806] ath9k_hw_kill_interrupts  793
[   83.256811] ath9k: ath9k_ioread32 ff800a400024
[   83.261651] ath9k_hw_enable_interrupts821
[   83.265753] ath9k_hw_enable_interrupts825
[   83.269919] ath9k_hw_enable_interrupts832
[   83.274083] ath9k: ath9k_iowrite32 ff800a400024
[   83.278945] AR_SREV_9100 0
[   83.281630] ath9k_hw_enable_interrupts848
[   83.281633] ath9k_hw_intrpend 762
[   83.281634] (AR_SREV_9340(ah) val 0
[   83.281637] ath9k_hw_intrpend 767
[   83.281648] ath_isr   603
[   83.281649] ath9k: ath9k_iowrite32 ff800a400024
[   83.281651] ath9k: ath9k_ioread32 ff800a400024
[   83.281654] ath9k_hw_kill_interrupts  793
[   83.312192] ath9k: ath9k_ioread32 ff800a400024
[   83.317030] 

RE: ATH9 driver issues on ARM64

[Bharat Kumar Gogada]

 > [+cc Kalle, ath9k list]
> 
> On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> > Hi,
> >
> > Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
> > on ARM64.  The end point is TP link wifi card with which supports
> > only legacy interrupts.
> 
> If it works on other arches and the arm64 PCI enumeration works, my
> first guess would be an INTx issue, e.g., maybe the driver is waiting
> for an interrupt that never arrives.
We are not sure for now.
> 
> > We are trying to test it on ARM64 with
> > (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> >
> > EP is getting enumerated and able to link up.
> >
> > But when we start scan system gets hanged.
> 
> When you say the system hangs when you start a scan, I assume you mean
> a wifi scan, not the PCI enumeration.  A problem with a wifi scan
> might cause a *process* to hang, but it shouldn't hang the entire
> system.
> 
Yes wifi scan.
> > When we took trace we see that after we start scan assert message is
> > sent but there is no de assert from end point.
> 
> Are you talking about a trace from a PCIe analyzer?  Do you see an
> Assert_INTx PCIe message on the link?
> 
Yes lecroy trace, yes we do see Assert_INTx and Deassert_INTx happening when we 
do interface link up.
When we have less debug prints in Atheros driver, and do wifi scan we see 
Assert_INTx but never Deassert_INTx, 
> > What might cause end point not sending de assert ?
> 
> If the endpoint doesn't send a Deassert_INTx message, I expect that
> would mean the driver didn't service the interrupt and remove the
> condition that caused the device to assert the interrupt in the first
> place.
> 
> If the driver didn't receive the interrupt, it couldn't service it, of
> course.  You could add a printk in the ath9k interrupt service
> routine to see if you ever get there.
>
The interrupt behavior is changing w.r.t amount of debug prints we add. (I kept 
many prints to aid debug)
root@Xilinx-ZCU102-2016_3:~# iw dev wlan0 scan
[   83.064675] ath9k: ath9k_iowrite32 ff800a400024
[   83.069486] ath9k: ath9k_ioread32 ff800a400024
[   83.074257] ath9k_hw_kill_interrupts  793
[   83.078260] ath9k: ath9k_iowrite32 ff800a400024
[   83.083107] ath9k: ath9k_ioread32 ff800a400024
[   83.087882] ath9k_hw_kill_interrupts  793
[   83.095450] ath9k_hw_enable_interrupts821
[   83.099557] ath9k_hw_enable_interrupts825
[   83.103721] ath9k_hw_enable_interrupts832
[   83.107887] ath9k: ath9k_iowrite32 ff800a400024
[   83.112748] AR_SREV_9100 0
[   83.115438] ath9k_hw_enable_interrupts848
[   83.119607] ath9k: ath9k_ioread32 ff800a400024
[   83.124389] ath9k_hw_intrpend 762
[   83.127761] (AR_SREV_9340(ah) val 0
[   83.131234] ath9k_hw_intrpend 767
[   83.134628] ath_isr   603
[   83.137134] ath9k: ath9k_iowrite32 ff800a400024
[   83.141995] ath9k: ath9k_ioread32 ff800a400024
[   83.146771] ath9k_hw_kill_interrupts  793
[   83.150864] ath9k_hw_enable_interrupts821
[   83.154971] ath9k_hw_enable_interrupts825
[   83.159135] ath9k_hw_enable_interrupts832
[   83.163300] ath9k: ath9k_iowrite32 ff800a400024
[   83.168161] AR_SREV_9100 0
[   83.170852] ath9k_hw_enable_interrupts848
[   83.170855] ath9k_hw_intrpend 762
[   83.178398] (AR_SREV_9340(ah) val 0
[   83.181873] ath9k_hw_intrpend 767
[   83.185265] ath_isr   603
[   83.187773] ath9k: ath9k_iowrite32 ff800a400024
[   83.192635] ath9k: ath9k_ioread32 ff800a400024
[   83.197411] ath9k_hw_kill_interrupts  793
[   83.201414] ath9k: ath9k_ioread32 ff800a400024
[   83.206258] ath9k_hw_enable_interrupts821
[   83.210368] ath9k_hw_enable_interrupts825
[   83.214531] ath9k_hw_enable_interrupts832
[   83.218698] ath9k: ath9k_iowrite32 ff800a400024
[   83.223558] AR_SREV_9100 0
[   83.226243] ath9k_hw_enable_interrupts848
[   83.226246] ath9k_hw_intrpend 762
[   83.233794] (AR_SREV_9340(ah) val 0
[   83.237268] ath9k_hw_intrpend 767
[   83.240661] ath_isr   603
[   83.243169] ath9k: ath9k_iowrite32 ff800a400024
[   83.248030] ath9k: ath9k_ioread32 ff800a400024
[   83.252806] ath9k_hw_kill_interrupts  793
[   83.256811] ath9k: ath9k_ioread32 ff800a400024
[   83.261651] ath9k_hw_enable_interrupts821
[   83.265753] ath9k_hw_enable_interrupts825
[   83.269919] ath9k_hw_enable_interrupts832
[   83.274083] ath9k: ath9k_iowrite32 ff800a400024
[   83.278945] AR_SREV_9100 0
[   83.281630] ath9k_hw_enable_interrupts848
[   83.281633] ath9k_hw_intrpend 762
[   83.281634] (AR_SREV_9340(ah) val 0
[   83.281637] ath9k_hw_intrpend 767
[   83.281648] ath_isr   603
[   83.281649] ath9k: ath9k_iowrite32 ff800a400024
[   83.281651] ath9k: ath9k_ioread32 ff800a400024
[   83.281654] ath9k_hw_kill_interrupts  793
[   83.312192] ath9k: ath9k_ioread32 ff800a400024
[   83.317030] 

Re: ATH9 driver issues on ARM64

[Bjorn Helgaas]

[+cc Kalle, ath9k list]

On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> Hi,
> 
> Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
> on ARM64.  The end point is TP link wifi card with which supports
> only legacy interrupts.

If it works on other arches and the arm64 PCI enumeration works, my
first guess would be an INTx issue, e.g., maybe the driver is waiting
for an interrupt that never arrives.

> We are trying to test it on ARM64 with
> (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> 
> EP is getting enumerated and able to link up. 
> 
> But when we start scan system gets hanged.

When you say the system hangs when you start a scan, I assume you mean
a wifi scan, not the PCI enumeration.  A problem with a wifi scan
might cause a *process* to hang, but it shouldn't hang the entire
system.

> When we took trace we see that after we start scan assert message is
> sent but there is no de assert from end point.

Are you talking about a trace from a PCIe analyzer?  Do you see an
Assert_INTx PCIe message on the link?

> What might cause end point not sending de assert ?

If the endpoint doesn't send a Deassert_INTx message, I expect that
would mean the driver didn't service the interrupt and remove the
condition that caused the device to assert the interrupt in the first
place.

If the driver didn't receive the interrupt, it couldn't service it, of
course.  You could add a printk in the ath9k interrupt service
routine to see if you ever get there.

> We are not seeing any issues on 32-bit ARM platform and X86
> platform. 

Can you collect a dmesg log (or, if the system hang means you can't
collect that, a console log with "ignore_loglevel"), and "lspci -vv"
output as root?  That should have clues about whether the INTx got
routed correctly.  /proc/interrupts should also show whether we're
receiving interrupts from the device.

Bjorn

Re: ATH9 driver issues on ARM64

[Bjorn Helgaas]

[+cc Kalle, ath9k list]

On Thu, Dec 08, 2016 at 01:49:42PM +, Bharat Kumar Gogada wrote:
> Hi,
> 
> Did anyone test Atheros ATH9 driver(drivers/net/wireless/ath/ath9k/)
> on ARM64.  The end point is TP link wifi card with which supports
> only legacy interrupts.

If it works on other arches and the arm64 PCI enumeration works, my
first guess would be an INTx issue, e.g., maybe the driver is waiting
for an interrupt that never arrives.

> We are trying to test it on ARM64 with
> (drivers/pci/host/pcie-xilinx-nwl.c) as root port.
> 
> EP is getting enumerated and able to link up. 
> 
> But when we start scan system gets hanged.

When you say the system hangs when you start a scan, I assume you mean
a wifi scan, not the PCI enumeration.  A problem with a wifi scan
might cause a *process* to hang, but it shouldn't hang the entire
system.

> When we took trace we see that after we start scan assert message is
> sent but there is no de assert from end point.

Are you talking about a trace from a PCIe analyzer?  Do you see an
Assert_INTx PCIe message on the link?

> What might cause end point not sending de assert ?

If the endpoint doesn't send a Deassert_INTx message, I expect that
would mean the driver didn't service the interrupt and remove the
condition that caused the device to assert the interrupt in the first
place.

If the driver didn't receive the interrupt, it couldn't service it, of
course.  You could add a printk in the ath9k interrupt service
routine to see if you ever get there.

> We are not seeing any issues on 32-bit ARM platform and X86
> platform. 

Can you collect a dmesg log (or, if the system hang means you can't
collect that, a console log with "ignore_loglevel"), and "lspci -vv"
output as root?  That should have clues about whether the INTx got
routed correctly.  /proc/interrupts should also show whether we're
receiving interrupts from the device.

Bjorn