Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-21 Thread Julien Jemine 
Hi Jens, hi Sergio,

Yes that's exactly my problem. Here's the patch I've came up with on my
driver :

int packetCount = 0;

static void eth_notify (ARM_ETH_MAC_EVENT event) {
  /* Send notification on RX event */
  if (event == ARM_ETH_MAC_EVENT_RX_FRAME) {
packetCount++;
  }
}

void ethernetif_poll (struct netif *netif) {
  struct ethernetif *eth = netif-state;

  if (!eth-phy_ok || eth-link == ARM_ETH_LINK_DOWN) {
return;
  }
  while (packetCount)
  {
ethernetif_input (netif);
atomicDecrement(packetCount);
  }
}

It works perfectly !

Thanks a lot for your help !

Julien.



2014-03-20 19:46 GMT+01:00 Jens Nielsen d...@telia.com:

  Alright, I also use the ST drivers (the ones from STM32Cube 1.0.0) on an
 STM32F4

 I haven't verified that my fix is 100% accurate but I think it looks safe
 so I'm sending it to ST anyway so they can figure it out if they want to.

 My fix goes something like this:


 At the bottom of HAL_ETH_GetReceivedFrame_IT() I return an error code if I
 don't find a complete packet:

 ---
 a/clients/embedded/source/STM32F4XX/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_eth.c
 +++
 b/clients/embedded/source/STM32F4XX/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_eth.c
 @@ -868,7 +868,7 @@ HAL_StatusTypeDef
 HAL_ETH_GetReceivedFrame_IT(ETH_HandleTypeDef *heth)
__HAL_UNLOCK(heth);

/* Return function status */
 -  return HAL_OK;
 +  return HAL_ERROR;
  }

  /**



 And then return NULL from low_level_input() in my ethernetif:

 ---
 a/clients/embedded/source/lwip-1.4.1/port/STM32F4x7/FreeRTOS/ethernetif.c
 +++
 b/clients/embedded/source/lwip-1.4.1/port/STM32F4x7/FreeRTOS/ethernetif.c
 @@ -225,7 +224,10 @@ static struct pbuf * low_level_input(struct netif
 *netif)
uint32_t i=0;

/* get received frame */
 -  HAL_ETH_GetReceivedFrame_IT(EthHandle);
 +  if ( HAL_ETH_GetReceivedFrame_IT(EthHandle) != HAL_OK )
 +  {
 +  return NULL;
 +  }

/* Obtain the size of the packet and put it into the len variable. */
len = EthHandle.RxFrameInfos.length;



 And in my receive task I have this loop (ok yeah this could be prettier...)

 +do
 +{
 +p = low_level_input(netif);
 +if (p != NULL)
 +{
 +if (netif-input(p, netif) != ERR_OK)
 +{
 +pbuf_free(p);
 +}
 +}
 +} while (p != NULL);


 Hope it helps!

 BR /Jens



 On 2014-03-20 19:13, Sergio R. Caprile wrote:

 Yes Jens, looks like you are absolutely right, missed that, there is no
 frame loss here, the receiving process must be getting only the first
 frame in the chip cue.

 Julien is using some ST microcontroller, I guess it has a built-in eth
 controller, probably with DMA.

 I've studied the Linux drivers for the DM9000 to write my driver, and
 I've seen they read all the packets out of the chip every time; then I
 setup a simple test to see if interrupts would trigger on every frame or
 only on empty buffer, and guess what... empty buffer only (there is no
 int ack here)

 Looks like we've got a different situation here, the chip seems to be
 issuing an interrupt after the first one is ACKed, as you Jens described
 for your case.

 Julien, can you confirm this from the chip docs ?
 Pitty one has to fix the drivers for a manufacturer, maybe someone in ST
 can lend a hand ?




 ___
 lwip-users mailing list
 lwip-users@nongnu.org
 https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-20 Thread Sergio R. Caprile 
Hi Julien.
Yes, there is a TCP retransmission from your host.
I bet that is because when the two packets come in such a close
proximity your Ethernet driver is losing the second one. This, in fact,
can be because there is a problem with the driver itself, or you have a
really low-memory condition and the driver can't find enough memory to
store the packet.
If I would have to hunt this witch, I would start by setting a
breakpoint on the Ethernet driver low_level_input() routine, somewhere
around here:


static struct pbuf *
low_level_input(struct netif *netif)
{
  [...]
  /* We allocate a pbuf chain of pbufs from the pool. */
  p = pbuf_alloc(PBUF_RAW, len, PBUF_POOL);

  if (p != NULL) {
[...]
  read data into(q-payload, q-len);
}
acknowledge that packet has been read();
  [...]
  } else {
drop packet();  -

It actually depends on how the one who actually wrote the driver wanted
to make it difficult for you.
Let me know how it goes.
Cheers

-- 


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-20 Thread Julien Jemine 
Hi Sergio,

I've found this drop packet thing in my driver, however it doesn't seem to
break there.
I've had a look at the driver code but I can't find anything...

I am a bit surprised that the ethernet interrupt just raises a flag :

static void eth_notify (ARM_ETH_MAC_EVENT event) {
  /* Send notification on RX event */
  if (event == ARM_ETH_MAC_EVENT_RX_FRAME) {
rx_event = true;
  }
}

It seems to imply that calling ethernetif_poll in an IDLE thread will only
process on packet even if multiple interrupts have been raised :

void ethernetif_poll (struct netif *netif) {
  struct ethernetif *eth = netif-state;

  if (!eth-phy_ok || eth-link == ARM_ETH_LINK_DOWN) {
return;
  }
  if (rx_event) {
rx_event = false;
/* process received ethernet packet */
ethernetif_input (netif);
  }
}

I think that an atomic counter and a while loop instead of a flag and a
if would be more appropriate. Am I wrong ?
Could that have anything to do with my problem ?

Thanks for your help,

Julien.




2014-03-20 15:10 GMT+01:00 Sergio R. Caprile scapr...@gmail.com:

 Hi Julien.
 Yes, there is a TCP retransmission from your host.
 I bet that is because when the two packets come in such a close
 proximity your Ethernet driver is losing the second one. This, in fact,
 can be because there is a problem with the driver itself, or you have a
 really low-memory condition and the driver can't find enough memory to
 store the packet.
 If I would have to hunt this witch, I would start by setting a
 breakpoint on the Ethernet driver low_level_input() routine, somewhere
 around here:


 static struct pbuf *
 low_level_input(struct netif *netif)
 {
   [...]
   /* We allocate a pbuf chain of pbufs from the pool. */
   p = pbuf_alloc(PBUF_RAW, len, PBUF_POOL);

   if (p != NULL) {
 [...]
   read data into(q-payload, q-len);
 }
 acknowledge that packet has been read();
   [...]
   } else {
 drop packet();  -

 It actually depends on how the one who actually wrote the driver wanted
 to make it difficult for you.
 Let me know how it goes.
 Cheers

 --


 ___
 lwip-users mailing list
 lwip-users@nongnu.org
 https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-20 Thread Sergio R. Caprile 
 I am a bit surprised that the ethernet interrupt just raises a flag :

Don't be surprised, lwIP is single threaded, meaning you shouldn't call
an lwIP memory allocation routine from inside an interrupt, since it
might have interrupted another lwIP memory allocation routine... (I
might be wrong since I didn't write the code, but that is what I
understand by single-threaded and what I did on my driver)

As you are using an RTOS, it surely dispatches a task to get the
frame(s) later. The frames should happily rest inside the eth chip (or
equivalent) until them, providing there is enough memory for them. The
chip I use has 13K devoted to that.

 It seems to imply that calling ethernetif_poll in an IDLE thread will
 only process on packet even if multiple interrupts have been raised :

Many eth chips issue only one interrupt for the first frame, the driver
then checks for how many outstanding frames there are.

check here --  ethernetif_input (netif);
it should have some form of a do while loop

 Could that have anything to do with my problem ?

IMHO, yes, quite probably, if that function does not perform a loop to
get all the outstanding frames.
Sorry but you'll have to learn your chip to check how it issues
interrupts and buffers frames, and then check the driver.
One quick try: send ping floods and see if you get the same behaviour.
Pings are answered at the very first part of the IP processing, so if
pings are OK, the issue is not probably inside the driver (do check
inter-ping time with wireshark)

Regards



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-20 Thread Jens Nielsen 

Hi

Yes you're probably right, that would be a better solution. What looks 
like happens next in the wireshark capture is that the interrupt 
triggered by the retransmission signals your code to read and ack the 
queued old packet. Then there's an ACK in packet 26 triggering another 
interrupt that signals your application to read the queued 
retransmission packet, hence the dup ack in packet 27. And so on, your 
application is constantly one packet behind the interrupt.


One comment though: I don't know which MCU you're on but I wouldn't 
assume one interrupt equals one packet. I recently fixed a bug in my 
application that had a similar setup (interrupt signals with counting 
semaphore to receive thread, which processed a number of packets equal 
to the counting semaphore) which didn't cover the case where a second 
packet is received before the ethernet interrupt has serviced the first 
packet. My interrupt would only hit once so the second packet was left 
until the next interrupt.


My solution was to simply try to fetch packets from DMA until it was 
empty each time my receive thread was woken.


BR /Jens


On 2014-03-20 16:21, Julien Jemine wrote:

Hi Sergio,

I've found this drop packet thing in my driver, however it doesn't 
seem to break there.

I've had a look at the driver code but I can't find anything...

I am a bit surprised that the ethernet interrupt just raises a flag :

static void eth_notify (ARM_ETH_MAC_EVENT event) {
  /* Send notification on RX event */
  if (event == ARM_ETH_MAC_EVENT_RX_FRAME) {
rx_event = true;
  }
}

It seems to imply that calling ethernetif_poll in an IDLE thread will 
only process on packet even if multiple interrupts have been raised :


void ethernetif_poll (struct netif *netif) {
  struct ethernetif *eth = netif-state;

  if (!eth-phy_ok || eth-link == ARM_ETH_LINK_DOWN) {
return;
  }
  if (rx_event) {
rx_event = false;
/* process received ethernet packet */
ethernetif_input (netif);
  }
}

I think that an atomic counter and a while loop instead of a flag and 
a if would be more appropriate. Am I wrong ?

Could that have anything to do with my problem ?

Thanks for your help,

Julien.




2014-03-20 15:10 GMT+01:00 Sergio R. Caprile scapr...@gmail.com 
mailto:scapr...@gmail.com:


Hi Julien.
Yes, there is a TCP retransmission from your host.
I bet that is because when the two packets come in such a close
proximity your Ethernet driver is losing the second one. This, in
fact,
can be because there is a problem with the driver itself, or you
have a
really low-memory condition and the driver can't find enough memory to
store the packet.
If I would have to hunt this witch, I would start by setting a
breakpoint on the Ethernet driver low_level_input() routine, somewhere
around here:


static struct pbuf *
low_level_input(struct netif *netif)
{
  [...]
  /* We allocate a pbuf chain of pbufs from the pool. */
  p = pbuf_alloc(PBUF_RAW, len, PBUF_POOL);

  if (p != NULL) {
[...]
  read data into(q-payload, q-len);
}
acknowledge that packet has been read();
  [...]
  } else {
drop packet();  -

It actually depends on how the one who actually wrote the driver
wanted
to make it difficult for you.
Let me know how it goes.
Cheers

--


___
lwip-users mailing list
lwip-users@nongnu.org mailto:lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users




___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-20 Thread Sergio R. Caprile 
Yes Jens, looks like you are absolutely right, missed that, there is no
frame loss here, the receiving process must be getting only the first
frame in the chip cue.

Julien is using some ST microcontroller, I guess it has a built-in eth
controller, probably with DMA.

I've studied the Linux drivers for the DM9000 to write my driver, and
I've seen they read all the packets out of the chip every time; then I
setup a simple test to see if interrupts would trigger on every frame or
only on empty buffer, and guess what... empty buffer only (there is no
int ack here)

Looks like we've got a different situation here, the chip seems to be
issuing an interrupt after the first one is ACKed, as you Jens described
for your case.

Julien, can you confirm this from the chip docs ?
Pitty one has to fix the drivers for a manufacturer, maybe someone in ST
can lend a hand ?

-- 


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Retransmission when receiving consecutive packets

2014-03-20 Thread Jens Nielsen 
Alright, I also use the ST drivers (the ones from STM32Cube 1.0.0) on an 
STM32F4


I haven't verified that my fix is 100% accurate but I think it looks 
safe so I'm sending it to ST anyway so they can figure it out if they 
want to.


My fix goes something like this:


At the bottom of HAL_ETH_GetReceivedFrame_IT() I return an error code if 
I don't find a complete packet:


   ---
   
a/clients/embedded/source/STM32F4XX/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_eth.c
   +++
   
b/clients/embedded/source/STM32F4XX/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_eth.c
   @@ -868,7 +868,7 @@ HAL_StatusTypeDef
   HAL_ETH_GetReceivedFrame_IT(ETH_HandleTypeDef *heth)
   __HAL_UNLOCK(heth);

   /* Return function status */
   -  return HAL_OK;
   +  return HAL_ERROR;
 }

 /**



And then return NULL from low_level_input() in my ethernetif:

   ---
   a/clients/embedded/source/lwip-1.4.1/port/STM32F4x7/FreeRTOS/ethernetif.c
   +++
   b/clients/embedded/source/lwip-1.4.1/port/STM32F4x7/FreeRTOS/ethernetif.c
   @@ -225,7 +224,10 @@ static struct pbuf * low_level_input(struct
   netif *netif)
   uint32_t i=0;

   /* get received frame */
   -  HAL_ETH_GetReceivedFrame_IT(EthHandle);
   +  if ( HAL_ETH_GetReceivedFrame_IT(EthHandle) != HAL_OK )
   +  {
   +  return NULL;
   +  }

   /* Obtain the size of the packet and put it into the len
   variable. */
   len = EthHandle.RxFrameInfos.length;



And in my receive task I have this loop (ok yeah this could be prettier...)

   +do
   +{
   +p = low_level_input(netif);
   +if (p != NULL)
   +{
   +if (netif-input(p, netif) != ERR_OK)
   +{
   +pbuf_free(p);
   +}
   +}
   +} while (p != NULL);


Hope it helps!

BR /Jens


On 2014-03-20 19:13, Sergio R. Caprile wrote:

Yes Jens, looks like you are absolutely right, missed that, there is no
frame loss here, the receiving process must be getting only the first
frame in the chip cue.

Julien is using some ST microcontroller, I guess it has a built-in eth
controller, probably with DMA.

I've studied the Linux drivers for the DM9000 to write my driver, and
I've seen they read all the packets out of the chip every time; then I
setup a simple test to see if interrupts would trigger on every frame or
only on empty buffer, and guess what... empty buffer only (there is no
int ack here)

Looks like we've got a different situation here, the chip seems to be
issuing an interrupt after the first one is ACKed, as you Jens described
for your case.

Julien, can you confirm this from the chip docs ?
Pitty one has to fix the drivers for a manufacturer, maybe someone in ST
can lend a hand ?



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users