Re: STM32F4 Ethernet Issues

2022-07-19 Thread Fotis Panagiotopoulos
Hello,

> We have deployed hundreds of boards with stm32f427 and ethernet, they
> have all been working reliably for months without stopping, we know it
> because they critically depend on network functionality and we have
> reports if a card becomes unreachable. None has so far outside of
> dedicated tests.

> So I believe that there is no obvious hard bug in these drivers.

Good to hear that!
Although, I may be using a feature or protocol that you are not.
Of course, I don't believe that NuttX is broken per se, but a minor bug may
lurk somewhere...


> I have seen that when I enable the network debugging features, it seems to
> hit an assertion failure before getting to nsh prompt at startup. This was
> on a quite recent master. I haven't had a chance to diagnose this further.
> Have you tried enabling these and if so, do they work?

If you refer to CONFIG_DEBUG_NET, then yes I have enabled it and it works.
I have some devices under test, waiting to reproduce the issue to see if
this option provides any useful information.


> Also, out of curiosity, have you tried running ostest on your board?

I just tried.
It passed all the tests.

On Tue, Jul 19, 2022 at 4:44 PM Sebastien Lorquet 
wrote:

> Hi,
>
> We have deployed hundreds of boards with stm32f427 and ethernet, they
> have all been working reliably for months without stopping, we know it
> because they critically depend on network functionality and we have
> reports if a card becomes unreachable. None has so far outside of
> dedicated tests.
>
> So I believe that there is no obvious hard bug in these drivers.
>
> Most certainly a build option on your particular config. debug is a
> possible issue, thread problems is another possibility.
>
> Sebastien
>
>
> On 7/19/22 13:47, Fotis Panagiotopoulos wrote:
> > Hello!
> >
> > I am using Ethernet on an STM32F427 target, but I am facing some issues.
> >
> > Initially the device works correctly. After some hours of continuous
> > operation I completely lose all network communications.
> > Trying to troubleshoot the issue, I enabled assertions and various other
> > debug features.
> >
> > Again the device works correctly for some hours, and then I get a failed
> > assertion at stm32_eth.c, line 1372:
> >
> > DEBUGASSERT(dev->d_len == 0 && dev->d_buf == NULL);
> >
> > No other errors are reported (e.g. stack overflows etc).
> >
> >
> > I have observed that this issue usually manifests itself when there is
> > insufficient stack on a task.
> > But in my case, all tasks have oversized stacks. Typically they do not
> > exceed 50% utilization.
> > I have plenty of room available in the heap too (> 100kB).
> >
> > Regarding the rest of the firmware, I cannot see any other misbehaviour
> or
> > problem.
> > I haven't ever seen any other unexplained problem, assertion fail,
> > hard-fault etc.
> > The application code passes all of our tests.
> > In fact, even when this issue happens, although I lose network
> > connectivity, the rest of the system works perfectly.
> >
> > Please note that I have checked the contents of dev->d_len and
> dev->d_buf,
> > and they seem to contain valid data.
> > The address lies within the normal address space of the MCU, and the size
> > is sane.
> > So it doesn't look like any kind of memory corruption.
> >
> >
> > At this point I believe that this is an actual bug either on the STM32
> MAC
> > driver, or at the TCP/IP stack itself.
> > I had a look at the driver code, but I didn't see anything suspicious.
> >
> >
> > Has anyone observed the same issue before?
> > Can it be affected in any way with my configuration?
> > Or maybe, do you have any recommendations on what to test next?
> >
> >
> > Thank you!
> >
>


Re: STM32F4 Ethernet Issues

2022-07-19 Thread Sebastien Lorquet

Hi,

We have deployed hundreds of boards with stm32f427 and ethernet, they 
have all been working reliably for months without stopping, we know it 
because they critically depend on network functionality and we have 
reports if a card becomes unreachable. None has so far outside of 
dedicated tests.


So I believe that there is no obvious hard bug in these drivers.

Most certainly a build option on your particular config. debug is a 
possible issue, thread problems is another possibility.


Sebastien


On 7/19/22 13:47, Fotis Panagiotopoulos wrote:

Hello!

I am using Ethernet on an STM32F427 target, but I am facing some issues.

Initially the device works correctly. After some hours of continuous
operation I completely lose all network communications.
Trying to troubleshoot the issue, I enabled assertions and various other
debug features.

Again the device works correctly for some hours, and then I get a failed
assertion at stm32_eth.c, line 1372:

DEBUGASSERT(dev->d_len == 0 && dev->d_buf == NULL);

No other errors are reported (e.g. stack overflows etc).


I have observed that this issue usually manifests itself when there is
insufficient stack on a task.
But in my case, all tasks have oversized stacks. Typically they do not
exceed 50% utilization.
I have plenty of room available in the heap too (> 100kB).

Regarding the rest of the firmware, I cannot see any other misbehaviour or
problem.
I haven't ever seen any other unexplained problem, assertion fail,
hard-fault etc.
The application code passes all of our tests.
In fact, even when this issue happens, although I lose network
connectivity, the rest of the system works perfectly.

Please note that I have checked the contents of dev->d_len and dev->d_buf,
and they seem to contain valid data.
The address lies within the normal address space of the MCU, and the size
is sane.
So it doesn't look like any kind of memory corruption.


At this point I believe that this is an actual bug either on the STM32 MAC
driver, or at the TCP/IP stack itself.
I had a look at the driver code, but I didn't see anything suspicious.


Has anyone observed the same issue before?
Can it be affected in any way with my configuration?
Or maybe, do you have any recommendations on what to test next?


Thank you!



Re: STM32F4 Ethernet Issues

2022-07-19 Thread Nathan Hartman
On Tue, Jul 19, 2022 at 7:47 AM Fotis Panagiotopoulos 
wrote:

> Hello!
>
> I am using Ethernet on an STM32F427 target, but I am facing some issues.
>
> Initially the device works correctly. After some hours of continuous
> operation I completely lose all network communications.
> Trying to troubleshoot the issue, I enabled assertions and various other
> debug features.
>
> Again the device works correctly for some hours, and then I get a failed
> assertion at stm32_eth.c, line 1372:
>
> DEBUGASSERT(dev->d_len == 0 && dev->d_buf == NULL);
>
> No other errors are reported (e.g. stack overflows etc).
>
>
> I have observed that this issue usually manifests itself when there is
> insufficient stack on a task.
> But in my case, all tasks have oversized stacks. Typically they do not
> exceed 50% utilization.
> I have plenty of room available in the heap too (> 100kB).
>
> Regarding the rest of the firmware, I cannot see any other misbehaviour or
> problem.
> I haven't ever seen any other unexplained problem, assertion fail,
> hard-fault etc.
> The application code passes all of our tests.
> In fact, even when this issue happens, although I lose network
> connectivity, the rest of the system works perfectly.
>
> Please note that I have checked the contents of dev->d_len and dev->d_buf,
> and they seem to contain valid data.
> The address lies within the normal address space of the MCU, and the size
> is sane.
> So it doesn't look like any kind of memory corruption.
>
>
> At this point I believe that this is an actual bug either on the STM32 MAC
> driver, or at the TCP/IP stack itself.
> I had a look at the driver code, but I didn't see anything suspicious.
>
>
> Has anyone observed the same issue before?
> Can it be affected in any way with my configuration?
> Or maybe, do you have any recommendations on what to test next?
>
>
> Thank you!



I am currently working on a networking device and trying to get it working
more robustly. My device is using a different ARM- based micro.

I have seen that when I enable the network debugging features, it seems to
hit an assertion failure before getting to nsh prompt at startup. This was
on a quite recent master. I haven't had a chance to diagnose this further.
Have you tried enabling these and if so, do they work?

Also, out of curiosity, have you tried running ostest on your board?

Thanks,
Nathan


STM32F4 Ethernet Issues

2022-07-19 Thread Fotis Panagiotopoulos
Hello!

I am using Ethernet on an STM32F427 target, but I am facing some issues.

Initially the device works correctly. After some hours of continuous
operation I completely lose all network communications.
Trying to troubleshoot the issue, I enabled assertions and various other
debug features.

Again the device works correctly for some hours, and then I get a failed
assertion at stm32_eth.c, line 1372:

DEBUGASSERT(dev->d_len == 0 && dev->d_buf == NULL);

No other errors are reported (e.g. stack overflows etc).


I have observed that this issue usually manifests itself when there is
insufficient stack on a task.
But in my case, all tasks have oversized stacks. Typically they do not
exceed 50% utilization.
I have plenty of room available in the heap too (> 100kB).

Regarding the rest of the firmware, I cannot see any other misbehaviour or
problem.
I haven't ever seen any other unexplained problem, assertion fail,
hard-fault etc.
The application code passes all of our tests.
In fact, even when this issue happens, although I lose network
connectivity, the rest of the system works perfectly.

Please note that I have checked the contents of dev->d_len and dev->d_buf,
and they seem to contain valid data.
The address lies within the normal address space of the MCU, and the size
is sane.
So it doesn't look like any kind of memory corruption.


At this point I believe that this is an actual bug either on the STM32 MAC
driver, or at the TCP/IP stack itself.
I had a look at the driver code, but I didn't see anything suspicious.


Has anyone observed the same issue before?
Can it be affected in any way with my configuration?
Or maybe, do you have any recommendations on what to test next?


Thank you!