from:"Joel Cunningham"

Re: [lwip-users] TCP keep-alive not working on half-closed connections

2018-06-06 Thread Joel Cunningham




On 06/06/2018 02:13 PM, goldsi...@gmx.de wrote:


https://stackoverflow.com/questions/18110239/does-tcp-endpoint-that-has-sent-a-fin-still-send-keepalive
https://social.msdn.microsoft.com/Forums/en-US/d20e5e67-d23c-42a5-95c5-ba80f4263546/keepalive-packets-continue-being-sent-after-client-closes-server-socket-is-closewait-client

Here's some findings on a FIN_WAIT_2 timeout:

https://serverfault.com/questions/738300/why-are-connections-in-fin-wait2-state-not-closed-by-the-linux-kernel
https://www.akronohio.gov/manual/misc/fin_wait_2.html

I don't believe a FIN_WAIT_2 timeout is implemented in LwIP.


No, we don't have that. We always said it's OK for a connection to 
stay in FIN_WAIT_2. However, it seems we must handle this differently 
depending on wether the user has called "shutdown" or "close". Then we 
can implement a timeout.


Agreed, if the application used shutdown() on only the TX pathway, we 
would not want to start a timeout until the RX pathway is also 
shutdown.  Maybe even further delay the timeout until the app is done 
with the socket as indicated by close().




I'm also thinking this issue may not be that common since tcp_alloc 
would reap the half-open connection eventually to prevent pcb exhaustion


I don't think tcp_alloc will kill pcbs in FIN_WAIT_2 unless they have 
a lower priority. A timeout on FIN_WAIT_2 *does* make sense!
You're right, I forgot the behavior was changed somewhat recently to not 
kill same priority anymore


Joel



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Incorrect ACK number for the TCP Server with PPPoS, NetConn and FreeRTOS

2018-06-06 Thread Joel Cunningham




On 06/06/2018 09:29 AM, sarp wrote:

I am implementing a TCP Server with PPPoS where NO_SYS = 0(FreeRTOS).

Everything is OK, one thread feeds ppp and the other opens a netconn socket
for server and starts listening.

However, when a client is connected, connection is reset because of
incorrect ACK number with

   the switch case SYN_RCVD of tcp_process() by the function tcp_rst() in
tcp_in.c

What could be the reason for that?


Are TCP checksums enabled and passing?  First guess is usually data 
corruption in the link layer due to buggy driver


Joel


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP keep-alive not working on half-closed connections

2018-06-06 Thread Joel Cunningham

On 06/06/2018 07:45 AM, goldsi...@gmx.de wrote:

On 06.06.2018 11:42, R. Diez wrote:

[..]
The trouble is, this changes the connection state (pcb->state) from
ESTABLISHED to FIN_WAIT_1, which then turns quickly into FIN_WAIT_2.
Afterwards, TCP keep-alive does not work anymore, because of this logic
in tcp_slowtmr():

/* Check if KEEPALIVE should be sent */
if (ip_get_option(pcb, SOF_KEEPALIVE) &&
((pcb->state == ESTABLISHED) ||
(pcb->state == CLOSE_WAIT))) {

If I then pull the server's Ethernet cable while the connection is
transferring data, the connection never times out on the client.

I am no TCP/IP expert. Is there an error in my reasoning regarding
half-closed connections? Can TCP keep-alive packets, whatever they are,
still be sent and/or received on half-closed TCP connections?
Half-opened connections should be usable like you did it, yes. I'm not
sure about keepalives though.

Technically, keepalives are just empty ACKs, so that should work.

However, I'm not sure wether tcp keepalive is meant to work on
half-open connections.

In order to move from ESTABLISHED to FIN_WAIT_1 and ultimately
FIN_WAIT_2, your application would have had to shutdown the TX pathway
(either a call to shutdown() or close()). In this scenario, TCP can no
longer send data packets, so you couldn't detect a half-open connection
by sending data. Since the connection is in a receive only state, either
a FIN_WAIT_2 timeout or TCP keepalives with no data could be used.

I believe keepalives are used in this case, I found some evidence with a
quick google. A more exhaustive study of other stacks would be a good
to confirm this behavior.

https://stackoverflow.com/questions/18110239/does-tcp-endpoint-that-has-sent-a-fin-still-send-keepalive
https://social.msdn.microsoft.com/Forums/en-US/d20e5e67-d23c-42a5-95c5-ba80f4263546/keepalive-packets-continue-being-sent-after-client-closes-server-socket-is-closewait-client

Here's some findings on a FIN_WAIT_2 timeout:

https://serverfault.com/questions/738300/why-are-connections-in-fin-wait2-state-not-closed-by-the-linux-kernel
https://www.akronohio.gov/manual/misc/fin_wait_2.html

I don't believe a FIN_WAIT_2 timeout is implemented in LwIP. I'm also
thinking this issue may not be that common since tcp_alloc would reap
the half-open connection eventually to prevent pcb exhaustion

Joel

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Getting Raw ARP Packets

2018-05-29 Thread Joel Cunningham



> On May 22, 2018, at 06:13, David M. Zar  wrote:
> 
> As a side note, we have a software version of our proposed system running on 
> a PC... it opens a BSD RAW_SOCKET and does everything at that level. 
> Essentially, that's what I'm trying to emulate, here, but being as efficient 
> as possible in an embedded environment. I do have an Ethernet driver working 
> very well, it seems, so maybe I'm 90% there.)

From the use case you’ve described, I’m guessing you’re using an AF_PACKET 
socket on your PC, which allows sending/receiving raw Ethernet packets instead 
of IP packets you get with AF_INET/SOCK_RAW

LwIP supports SOCK_RAW, but not AF_PACKET sockets. There have been some 
requests on the mailing lists for AF_PACKET support, but no implementation yet.

Joel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Help debugging IPv6 SLAAC

2018-04-19 Thread Joel Cunningham



> On Apr 19, 2018, at 2:49 PM, goldsi...@gmx.de wrote:
> 
> On 19.04.2018 14:45, thomasfogh wrote:
>> I've changed my IPv4 only application to be IPv6 only.
>> Right now it doesn't do anything but try to get an IPv6 address.
> 
> What's your setting for LWIP_IPV6_MLD? I think I remember that you must 
> explicitly join the "allnodes" group for this to work correctly...? This 
> seems like it could need a rework or explicit documentation.
> 
> 

Also, you’ll need to set netif->flags to enable MLD.  I found the same issue 
with the pcap netif a while back, see 
http://git.savannah.gnu.org/cgit/lwip/lwip-contrib.git/commit/?id=2de26f1b92fcfccade7123e54f22f04902e08551

Joel


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LWIP Socket Interface + Raw Ethernet Data

2018-03-29 Thread Joel Cunningham



> On Mar 26, 2018, at 6:34 AM, Kashyap Gada  
> wrote:
> 
> Dear all,
>  
> We have  a project of a custom radio working as a point to point device 
> working as a bridge between two LAN Networks. We have two requirements – 
> 1.   Sending and receiving all Ethernet packets to the opposite radio to 
> & fro. 
> 2.   Also run a web server on the radio for managing the radio.
>  
> This requires me to communicate all the Ethernet packets to the opposite 
> radio and also consume requests on the local web server. 
> How can I bifurcate or this traffic and what locations or c functions should 
> I tap the data from. Need simple guidance to achieve this. 
> 

For anyone else on the list wondering about bridge functionality, this was 
double-posted on StackOverflow and funny enough, I ran across that instance 
first.  See my answer:

https://stackoverflow.com/questions/49490523/lwip-read-raw-ethernet-packets-also-run-a-web-server

Joel

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] access to tcp_pcb data from socket+

2018-03-07 Thread Joel Cunningham




On 03/07/2018 04:05 AM, Mattia Settin wrote:

Hi
For example I need to check the send buffer size (e.i. SO_SNDBUF) 
which is not implemented options.

Regards

Setting the send buffer size at run-time is not supported. We have a 
compile time setting TCP_SND_BUF for that.  Are you wanting to only get 
the current send buffer size?  I'm curious what the use case is.  You 
can easily use non-blocking sockets and an application specific write 
buffer. TCP will then copy as much as it can to fill the send buffer and 
return a partial write.
On Wed, Mar 7, 2018 at 10:55 AM, Mattia Settin 
> wrote:


Hi
Yes you are right but I don't need to touch the pcb, just some
check on the pcb.
getpeername  stores the address of the peer that is connected
socket (it dosen't return the pcb structure).
Which is the api for retrive the connection tcp data for a certain
socket ?
Thanks
reagards
m


On Wed, Mar 7, 2018 at 10:26 AM, Jens Nielsen > wrote:

Hi

The short answer is: don't. If you're using the socket api
you're not touching the pcb.

What do you really want to do?  There are basically two
options here

a) Just like your question about getpeername, there's an api
for that. Google is your friend.
or
b) Since the beginning of time people have managed without it,
in which case you're probably doing something weird and you're
probably on your own

BR /Jens


On 2018-03-07 10:07, Mattia Settin wrote:

Dear all
I developing an application wih lwip 2.0.3 using socket bsd.
The tcp server performs the following operation:
s = socket()
bind
accept //wait a new connection
read  //packet received

I need to perform some check on the pcb data of my socket.
I note the get_socket is defined as static and the
get_sockopt do not return all data I need.
I solve adding following function in socket.c:
struct tcp_pcb * lwip_getsockpcb(int s)
{
  struct lwip_sock *sock = get_socket(s);
  if (!sock) {
    return NULL;
  }
  return sock->conn->pcb.tcp;
}

My question is:
how can acces to the tcp_pcb of my socket with out modify the
library ?
Which is the smart way ?
Thank all
regard


-- 
Mattia Settin

Software and System Engineer




___
lwip-users mailing list
lwip-users@nongnu.org 
https://lists.nongnu.org/mailman/listinfo/lwip-users




___
lwip-users mailing list
lwip-users@nongnu.org 
https://lists.nongnu.org/mailman/listinfo/lwip-users





-- 
Mattia Settin

Software and System Engineer





--
Mattia Settin
Software and System Engineer


Joel


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP Sessions Scale Limit

2018-02-27 Thread Joel Cunningham

If you have 128GB of RAM (wow!!!), it might be best to turn off LwIP's 
heap and pool and just use stdc malloc.  Then you won't be limited by 
any pool or heap defines in LwIP.  See the below options:


/**
 * MEM_LIBC_MALLOC==1: Use malloc/free/realloc provided by your C-library
 * instead of the lwip internal allocator. Can save code size if you
 * already use it.
 */
#if !defined MEM_LIBC_MALLOC || defined __DOXYGEN__
#define MEM_LIBC_MALLOC 0
#endif

/**
 * MEMP_MEM_MALLOC==1: Use mem_malloc/mem_free instead of the lwip pool 
allocator.
 * Especially useful with MEM_LIBC_MALLOC but handle with care 
regarding execution
 * speed (heap alloc can be much slower than pool alloc) and usage from 
interrupts
 * (especially if your netif driver allocates PBUF_POOL pbufs for 
received frames

 * from interrupt)!
 * ATTENTION: Currently, this uses the heap for ALL pools (also for 
private pools,

 * not only for internal pools defined in memp_std.h)!
 */
#if !defined MEMP_MEM_MALLOC || defined __DOXYGEN__
#define MEMP_MEM_MALLOC 0
#endif

Joel

On 02/27/2018 08:33 AM, Jan Menzel wrote:

The error is cased by a failed call to memp_malloc(MEMP_TCP_SEG). So
you're running out of "simultaneously queued TCP segments". Try
increasing MEMP_NUM_TCP_SEG. Default is 16. Also check the statistics
more closely. After the protocol related block use pasted the TCP part
from a memory and memory pool related block follows which contains
numbers about "TCP_SEG". There you probably have errors as well.

Jan

On 27.02.2018 15:09, Anil kumar wrote:

I have 128 gig RAM , I feel i have been restricted by some macro.

tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)
tcp_create_segment:192 tcp_create_segment: no memory.
tcp_write:779 tcp_write: 0 (with mem err)


anil@anil:~$ cat /proc/meminfo
MemTotal:       132024188 kB
MemFree:        101949284 kB
MemAvailable:   119384708 kB
Buffers:         1616608 kB
Cached:         15111964 kB
SwapCached:            0 kB
Active:         4268 kB
Inactive:        8722364 kB
Active(anon):    3108632 kB
Inactive(anon):     1368 kB
Active(file):    8005636 kB
Inactive(file):  8720996 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      134205436 kB
SwapFree:       134205436 kB
Dirty:               548 kB
Writeback:             0 kB
AnonPages:       3108060 kB
Mapped:          2152924 kB
Shmem:              1936 kB
Slab:            1435644 kB
SReclaimable:    1329316 kB
SUnreclaim:       106328 kB
KernelStack:       10192 kB
PageTables:        31252 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    196023224 kB
Committed_AS:   34203972 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2727936 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:    4096
HugePages_Free:     4032
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      272572 kB
DirectMap2M:    18591744 kB
DirectMap1G:    115343360 kB
anil@KAMET:~$


On 27 February 2018 at 19:29, Jan Menzel > wrote:

 On 27.02.2018 14:54, Anil kumar wrote:
 [...]
 >         memerr: 20829
 [...]

 You have memory (RAM) problems.

         Jan

 ___
 lwip-users mailing list
 lwip-users@nongnu.org 
 https://lists.nongnu.org/mailman/listinfo/lwip-users
 




--
With Regards
Anil Kumar S N


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users




___

Re: [lwip-users] [OT] Minimize .html, .js and .css files

2018-02-20 Thread Joel Cunningham




On 02/20/2018 09:35 AM, Giuseppe Modugno wrote:

Another OT.

In order to reduce Flash space for the filesystem used by httpd, I'm 
thinking to reduce the size of .html, .css and .js files. This means 
removing comments, spaces (when not needed) and maybe changing 
variable names (from "my_long_variable" in "xtr").


Do you know of some tools that makes those things?


The term I've heard for this is "minify" or "minification".  You can 
find a number of tools online that do this, this one seems useful: 
https://github.com/matthiasmullie/minify


Joel



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users




___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] missing definition for mem_perf_init()

2018-02-12 Thread Joel Cunningham




On 02/07/2018 01:41 PM, goldsi...@gmx.de wrote:

On 07.02.2018 17:48, vinay s wrote:
My clone is a week old, I tried enabling MEM_PERF (unix port) and 
realized there is no def for mem_perf_init(). Not sure if I saw 
something to the effect in wiki either.


Any ideas?


My idea would be that that's orphaned code. There's no such thing as 
MEM_PERF in lwIP, only in the unix port. Don't know what that was 
though, sorry.


I went back and looked in git blame and when the code was added in 
edd18aad it was in the same non-working state.  I have removed it for 
now, thanks for the report.


Joel


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwIP in mainline linux kernel

2018-02-06 Thread Joel Cunningham

I haven't heard of LwIP running in the Linux kernel, but LwIP has been 
integrated into a number of other operating systems:


ReactOS

GNU Hurd

Minix 3


I haven't studied the ReactOS integration, but GNU Hurd/Minix 3 happened 
more recently and their respective developers communicated on the 
mailing list how LwIP was integrated :). Both are micro kernels and the 
integration point with LwIP is different.  GNU Hurd integrated at the 
sockets level where as Minix 3, integrated at the callback API level 
(providing their own socket interface).



I would imagine for Linux, you'd do something similar, integrating 
LwIP's callback API level to the kernel side sockets layer.  At the 
bottom end, you could possibly do something like map LwIP's netif to a 
netdev so you can use the existing Linux drivers.


Joel


On 02/05/2018 06:31 PM, vinay s wrote:

Hello All,

I am doing some source code evaluation (cycles spend in various 
blocks) of TCP/IP stack between linux network stack and lwip. I am 
able to run lwip in userspace, however for my requirement I would like 
to put lwip in mainline kernel (either as module or intergrated 
alternate stack), as running it in userspace fudges the numbers 
(preemption by high priority processes/interrupts etc).  I am curious 
as to if this has been already done (any pointers would be helpful).


Please note, I am not looking into lwip in rtos kernels (e.g, 
RTLinux/RTAI etc).


Thanks,
vks


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Best way for determining if a TCP connection is dead?

2018-01-24 Thread Joel Cunningham




On 01/24/2018 10:35 AM, Chris Seto wrote:

I have some code which works great, based on the echo client example.

I'm working on hardening it, and I'm wondering with how best to deal 
with a dead TCP connection as a result of the network link being down 
(ie, Ethernet cable disconnected).


This is a great source for dealing with half-open TCP connections due to 
all kinds of conditions:


https://www.codeproject.com/Articles/37490/Detection-of-Half-Open-Dropped-TCP-IP-Socket-Conne



The echo client contains conditions for a software termination of the 
socket (ie, server sends am empty frame), but I don't see anything 
dealing with a more blunt loss of the connection.


Any advice?

Thanks,
Chris


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Joel

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] issue adding static arp entries.

2017-12-19 Thread Joel Cunningham




On 12/19/2017 1:59 PM, Kent Pluntze wrote:

Hello,

I am trying to create a network with limited traffic so I am 
attempting to initialize all ARP tables to static. Currently I am 
calling a function in the application which does the following.


makeArpTableStatic(void){

/*inits*/
int i;
u8_t count = 0;
u8_t entry_ret = 0;
s8_t ret =0;
ip_addr_t **ip_addr_ret = NULL;
struct netif **netif = NULL;
struct eth_addr **eth_ret = NULL;

for(i=0; i

Re: [lwip-users] 2 Devices

2017-11-13 Thread Joel Cunningham




On 11/10/2017 03:51 AM, Fabian Cenedese wrote:

struct sockaddr_in addr;
s=socket(AF_INET, SOCK_DGRAM, 0);
if (s >= 0) {
 // set up port from any address to bind to
 memset(, 0, sizeof(addr));
 addr.sin_len=sizeof(addr);
 addr.sin_family=AF_INET;
 addr.sin_port=PP_HTONS(port);
 addr.sin_addr.s_addr=PP_HTONL(INADDR_ANY);
 // connect
 int ret = bind(s, (struct sockaddr*), sizeof(addr));
 // should succeed
 if (ret < 0) {
 // error
 }
 else {
 // limit to one interface
 struct ifreq ifr;
 memset(, 0, sizeof(struct ifreq));
 ifr.ifr_name[0] = 'e';// IFNAME0;
 ifr.ifr_name[1] = '0';// IFNAME1


Unless you've made modifications to LwIP's interface naming/numbering, 
the name is at least least 3 characters: two chars and a number (which 
is also the index).  See netif_find(), which is invoked from 
SO_BINDTODEVICE.


 int rc = setsockopt(s, SOL_SOCKET, SO_BINDTODEVICE,
 (void*), sizeof(struct ifreq));
 }
}

Joel


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] win32 porting contrib: double IP address

2017-11-09 Thread Joel Cunningham




On 11/09/2017 03:41 AM, Giuseppe Modugno wrote:

Il 08/11/2017 16:34, Joel Cunningham ha scritto:
The ping works well if I launch "ping 192.168.1.156" command from 
*another* computer on the same network.
You're most likely running into a checksum offload problem since you 
can't contact the LwIP stack from the Windows host, but you CAN from 
another machine.  The Window's TCP/IP stack is going to use checksum 
offloading for IPv4/UDP/TCP and during the transmit path, Winpcap 
will capture them before they go to hardware (where checksum is 
added).  Then they will fail checksum validation in LwIP and be 
discarded.


You can verify this easily by opening up wireshark and capturing on 
the interface.  Wireshark (also using winpcap) will capture the 
packets at the same point and you can verify the checksum is missing.

You're right.

If this is the case, you can disable checksum offloading in Windows 
for TX.

What is the procedure to disable checksum offloading in Windows for TX?

It varies depending on your network card.  Easiest way is if the adapter 
has an option to disable it.  I have one like this.  For Win 7, Control 
Panel -> Network and Internet -> Network and Sharing Center -> Change 
Adapter Settings -> Right click adapter -> Properties -> Configure -> 
Advanced.  Then I have options like 'TCP Checksum Offload (IPv4)' which 
I can set to Disabled, RX or TX Disabled/Enabled


If you're card doesn't have those options, you'll have to set some 
registry values (Google for some how-to's).  I've had varying success 
with using registry values with different cards.


You can also use the command: 'netsh interface ipv4 show offload' to 
view the state of your adapters


Joel



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] win32 porting contrib: double IP address

2017-11-08 Thread Joel Cunningham




On 11/08/2017 04:58 AM, Giuseppe Modugno wrote:
Hello lwip users, this is my first post. I hope this is the right 
mailing list for my question.


I started with lwip, so I downloaded both lwip and lwip-contrib. I was 
able to compile lwip test application for win32 (mingw compiler). When 
I launch test application on third adapter interface (in my computer 
it's a normal Ethernet interface integrated in the motherboard) with 
0.0.0.0 ip address, lwip automatically requests and obtains a new IP 
address (192.168.1.156). Of course, I have a DHCP server running on 
the network.


The Ethernet interface is already configured in Windows to use DHCP 
and its "Windows IP address" is 192.168.1.102 (DHCP static assignment).


So I have an Ethernet interface with two IP and MAC addresses. It 
seems it works, except for one thing.
I tested connectivity by running "ping 192.168.1.156" command from a 
shell on the same computer. No reply!
The ARP table is ok (192.168.1.156 is associated to 01.02.03.04.05.06 
MAC address). It seems no packets are detected from lwip.
Actually the LwIP winpcap adapter provides its own MAC address, so it's 
not going to collide with th card's actual address. Grep for 
LWIP_MAC_ADDR_BASE in lwip-contrib.  You can think of LwIP with winpcap 
as running a virtual network stack on top of the adapter. The Windows 
host and physically remote machines on the same local network just think 
it's another device.




The ping works well if I launch "ping 192.168.1.156" command from 
*another* computer on the same network.
You're most likely running into a checksum offload problem since you 
can't contact the LwIP stack from the Windows host, but you CAN from 
another machine.  The Window's TCP/IP stack is going to use checksum 
offloading for IPv4/UDP/TCP and during the transmit path, Winpcap will 
capture them before they go to hardware (where checksum is added).  Then 
they will fail checksum validation in LwIP and be discarded.


You can verify this easily by opening up wireshark and capturing on the 
interface.  Wireshark (also using winpcap) will capture the packets at 
the same point and you can verify the checksum is missing.


If this is the case, you can disable checksum offloading in Windows for TX.



Is there a solution to use my development computer to run lwip TCP/IP 
stack *and* to simulate an Internet node that wants to connect to it?


I think a solution is to use a virtual machine with a virtual network 
adapter, but this means to open VMWare Player or Virtual Box... I'd 
like to avoid.



Joel



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip 1.4.1 send a sporadic RST ACK

2017-10-25 Thread Joel Cunningham

I've updated the remaining comments to mention the same priority on git 
master

Another thing you could do is have your application track number of 
active connections, make sure enough TCP pcbs are configured and then 
close the listener pcb once the maximum is reach.

Joel

On 10/24/2017 10:38 AM, Mattia Settin wrote:

Yes
The main problem was related that the tcp_kill_prio() kills the active 
pcb with a priority equals to the pcb passaed in input.
For my specific web server application, this lead to send a (sporadic) 
RST ACK to the client, when the tcp_alloc fails to allocate new 
connection.

br

On Tue, Oct 24, 2017 at 4:34 PM, Axel Lin > wrote:

2017-10-24 16:37 GMT+08:00 Axel Lin >:
> 2017-10-24 1:13 GMT+08:00 Mattia Settin >:
>> Dear all
>> After a week of debugging I finally find the issue. The problem
is related to priority in pcb.
>> Taking a deep look I saw that the priority in the stack 1.4.1
is change, in particular:
>> Tcp_alloc return e pcb with a prior equal to the prior passed
in input.
>> Tcp_listen_with_backlog return e pcb with a prior equal to the
pcb.prior passed in input.
>> This two changes lead to a sporadic RST ACK call from the
kill_prio routine.
>> Finally I fix this issue removing the equal condition in the
kill_prio function.
>
> The comment on the code seems confusion for "the same priority
case":
> (I check the latest git tree)
>
> Th comment on the caller:
>           /* Try killing active connections with lower priority than
> the new one. */
>           LWIP_DEBUGF(TCP_DEBUG, ("tcp_alloc: killing connection
with
> prio lower than %d\n", prio));
>           tcp_kill_prio(prio);
>
> But the comment on tcp_kill_prio() function:
> /**
>  * Kills the oldest active connection that has the same or lower
priority than
>  * 'prio'.

The comment for tcp_kill_prio() is updated by:
commit 6a4c30fe5d65 fixed bug #31723 (tcp_kill_prio() kills pcbs with
the same prio) by updating its documentation only

And take a look at bug #31723: tcp_kill_prio() kills pcbs with the
same prio

According to its documentation, tcp_kill_prio() kills "the oldest
active connection that has lower priority than prio". However, the
current code also kills connections with a priority equal to 'prio'.
We should either change the documentation or the implementation.
The downside of the current implementation is that every call to
tcp_new() can lead to aborting an old (active) connection.
This has been in there since revision 1.1 of tcp.c, so changing it
might involve changing applications relying on this (like I have one
:).

I'm wondering if it should change the implementation rather than the
documentation.

___
lwip-users mailing list
lwip-users@nongnu.org 
https://lists.nongnu.org/mailman/listinfo/lwip-users

--
Mattia Settin
System engineer

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LWIP_MPU_COMPATIBLE set to 1, but still get memory management fault in lwip_select.

2017-09-08 Thread Joel Cunningham

David,

Would you mind opening a bug report at 
https://savannah.nongnu.org/bugs/?func=additem=lwip 
 and attaching the 
patches so we can do the review there?

Also, it would be preferable to have the changes in a git patchset that can be 
applied to the repo (easier to review than looking at individual patch files)

Thanks

Joel


> On Sep 8, 2017, at 9:50 AM, David Lockyer  wrote:
> 
> So something like attached?
> 
> I removed my raise/reset macro's.
> It appears to work for me, I added a new private pool, not sure if this was 
> what you intended or not.
> 
> Let me know what you think.
> 
> BR
> David
> 
> On 07/09/17 19:40, goldsi...@gmx.de  wrote:
>> David Lockyer wrote:
>>> Okay, thank you for the suggestion. Just to be clear are you suggesting 
>>> modifying lwip_select() to allocate select_cb from a pool & free prior to 
>>> return?
>> 
>> Via a define, like Joel wrote, yes. This might need a new memp pool though...
>> We don't have an abstraction for your priviledge raising macro yet, and I'm 
>> not sure it's the best solution for everyone using this mode, either.
>> 
>>> I will have to investigate the speed impact of this, as I have 
>>> MEM_LIBC_MALLOC and MAEP_MEM_MALLOC both defined as 1.
>>> 
>> 
>> Well, you have this pool allocation at some other places already when socket 
>> threads communicate with the tcpip thread for asynchronous things.
>> Having those 2 defines combined with a possibly slow libc malloc() (heap) is 
>> probabably not the fastest solution, anyway.
>> 
>> The best thing to get this fixed would be 
>> 
>> Simon 
>> __
>> This email has been scanned by the Symantec Email Security.cloud service.
>> For more information please visit http://www.symanteccloud.com 
>> 
>> __
>> 
>> 
>> ___
>> lwip-users mailing list
>> lwip-users@nongnu.org 
>> https://lists.nongnu.org/mailman/listinfo/lwip-users 
>> 
> 
> __
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> __
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LWIP_MPU_COMPATIBLE set to 1, but still get memory management fault in lwip_select.

2017-09-07 Thread Joel Cunningham


> On Sep 7, 2017, at 9:14 AM, David Lockyer  wrote:
> 
> Hi Simon,
> 
> Okay, thank you for the suggestion. Just to be clear are you suggesting 
> modifying lwip_select() to allocate select_cb from a pool & free prior to 
> return?
> 
> I will have to investigate the speed impact of this, as I have 
> MEM_LIBC_MALLOC and MAEP_MEM_MALLOC both defined as 1.
> 
> David
> 
> 
> 

Normally the MPU allocations that use dynamic memory are abstracted behind 
macros that use static memory in the !LWIP_MPU_COMPATIBLE case.  Take a look at 
API_VAR_ALLOC for both LWIP_MPU_COMPATBILE and !LWIP_MPU_COMPATBILE cases

Typically each allocation type has its own memory pool.  Not sure if Simon is 
suggesting that as well.  I know there’s been a hesitancy to add new memory 
pools

Joel
> 
> On 07/09/17 13:12, goldsimon wrote:
>> Looks correct. I guess mpu mode is not used too often. My suggestion would 
>> probably be to get select_cb from a Memo pool...
>> 
>> Simon
>> 
>> 
>> Am 6. September 2017 16:12:47 MESZ schrieb David Lockyer 
>>  :
>> Hi,
>> 
>> I have a project that uses an STM32F MCU running FreeRTOS (cortex mpu port) 
>> & lwip, with the MPU enabled.
>> I'm upgrading to lwip 2.0.2 from lwip 1.4.1, that I had to customize to be 
>> compatible with the MPU, in particular in lwip_select().
>> 
>> The motivation was to use the LWIP_MPU_COMPATIBLE define, so a direct 
>> modification of the stack source was not required.  However for me it still 
>> appears to try to access another threads memory, triggering an exception.
>> 
>> The changes I had to make were:
>> Index: lib/lwip-2.0.2/src/api/sockets.c
>> ===
>> --- lib/lwip-2.0.2/src/api/sockets.c(revision x)
>> +++ lib/lwip-2.0.2/src/api/sockets.c(working copy)
>> @@ -1428,7 +1428,9 @@
>>  /* Put this select_cb on top of list */ 
>>  select_cb.next = select_cb_list; 
>>  if (select_cb_list != NULL) { 
>> +  RAISE_PRIVILEGE(); 
>>select_cb_list->prev = _cb; 
>> +  RESET_PRIVILEGE(); 
>>  } 
>>  select_cb_list = _cb; 
>>  /* Increasing this counter tells event_callback that the list has 
>> changed. */ 
>> @@ -1508,7 +1510,9 @@
>>  /* Take us off the list */ 
>>  SYS_ARCH_PROTECT(lev); 
>>  if (select_cb.next != NULL) { 
>> +  RAISE_PRIVILEGE(); 
>>select_cb.next->prev = select_cb.prev; 
>> +  RESET_PRIVILEGE(); 
>>  } 
>>  if (select_cb_list == _cb) { 
>>LWIP_ASSERT("select_cb.prev == NULL", select_cb.prev == NULL); 
>> @@ -1515,7 +1519,9 @@
>>select_cb_list = select_cb.next; 
>>  } else { 
>>LWIP_ASSERT("select_cb.prev != NULL", select_cb.prev != NULL); 
>> +  RAISE_PRIVILEGE(); 
>>select_cb.prev->next = select_cb.next; 
>> +  RESET_PRIVILEGE(); 
>>  } 
>>  /* Increasing this counter tells event_callback that the list has 
>> changed. */ 
>>  select_cb_ctr++;
>> 
>> Is there something else I missed here that would remove the need for this 
>> modification?
>> 
>> Kind regards,
>> 
>> David Lockyer
>> 
>> __
>> This email has been scanned by the Symantec Email Security.cloud service.
>> For more information please visit http://www.symanteccloud.com 
>> 
>> __
>> 
>> -- 
>> Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
>> __
>> This email has been scanned by the Symantec Email Security.cloud service.
>> For more information please visit http://www.symanteccloud.com 
>> 
>> __
>> 
>> 
>> ___
>> lwip-users mailing list
>> lwip-users@nongnu.org 
>> https://lists.nongnu.org/mailman/listinfo/lwip-users 
>> 
> 
> __
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> __
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] IPv6 MTU vs IPv4 MTU

2017-09-05 Thread Joel Cunningham

Sent from my iPad
> On Sep 4, 2017, at 13:48, "goldsi...@gmx.de"  wrote:
> 
> Raphael Zulliger wrote:
>> What do you think? Have I found a bug and shall I open a bug report or
>> am I wrong?
> 
> I'm not really sure. After all, the MTU is what a network can send. Having a 
> different MTU for IPv4 and IPv6 doesn't make much sense to me?

Looks like one of the differences is minimum supported MTU of 1280 for IPv6 
versus 576 for IPv4.

Also, the change in fragmentation (routers can't fragment with IPv6) could 
result in the host device lowering it's MTU to accommodate a link with lower 
MTU. IPv4 could continue with the higher MTU since the link with smaller MTU 
could just fragment

http://tcpipguide.com/free/t_IPv6DatagramSizeMaximumTransmissionUnitMTUFragment.htm

Joel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP 2.0.2 pcb->snd_nxt is not updated somewhere?

2017-08-07 Thread Joel Cunningham

> 
> On Aug 7, 2017, at 4:43 AM, Simonas Kazlauskas  
> wrote:
> 
> So it seems like for some reason the pcb->snd_nxt is one-off.

This seems really strange.  For an active open, sending of the SYN packet 
advances pcb->snd_nxt by 1.  See the following code in tcp_output():

snd_nxt = lwip_ntohl(seg->tcphdr->seqno) + TCP_TCPLEN(seg);
if (TCP_SEQ_LT(pcb->snd_nxt, snd_nxt)) {
  pcb->snd_nxt = snd_nxt;
}

In tcp_priv.h:

#define TCP_TCPLEN(seg) ((seg)->len + (((TCPH_FLAGS((seg)->tcphdr) & (TCP_FIN | 
TCP_SYN)) != 0) ? 1U : 0U))

Joel___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Bind fails after previous server-initiated connection close

2017-06-09 Thread Joel Cunningham

Hi,

This is normal BSD semantics of a TCP listener, use SO_REUSEADDR before calling 
bind to avoid the issue


Joel

> On Jun 8, 2017, at 4:18 PM, Ignacio García Pérez  wrote:
> 
> Hi All,
> 
> I'm writing a simple FTP server that needs to be started up and shut down 
> with relatively high frequency (say once per minute), so each time I must 
> create the listening PCB, bind, listen, accept, etc. The problem is that when 
> I close a client connection the PCB goes into the TIME-WAIT state and the 
> next bind operation will fail with ERR_USE.
> 
> Looking at the code I see that all TIME-WAIT PCB are kept in a list and this 
> list is checked for the requested port in the tcp_bind() function.
> 
> Of course eventually the PCB is purged (after 2 * TCP_MSL = 120s by default), 
> but  that does not solve the problem in my scenario (which I admit is 
> somewhat exotic).
> 
> Question:
> 
> 1- Is this behaviour expected?
> 
> 2- Is there any workaround?
> 
> If the purpose of keeping the PCB in TIME-WAIT is to prevent new connections 
> to/from the same host:port to be created, it doesn't make sense to check when 
> calling tcp_bind(), it would make more sense to check calling tcp_connect() 
> or when an incoming connection is initiated.
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Raw api + multiple TCP Connections

2017-05-27 Thread Joel Cunningham

You're seeing the send stop from the server because the client's receive window 
is smaller than the next segment to send. At packet 28091, the receiver's 
window drops to 4 bytes and the sender's next segment is 8 bytes

LwIP (on the server) is treating this as a zero window rather than splitting 
the next segment to be 4 bytes.

You may want to check your client application to make sure it's updating the 
window when processing received data. See tcp_received()

Joel

> On May 26, 2017, at 06:28, Werner Motz  wrote:
> 
> Thank you very much for your answers. I am able to receive my data from both
> clients now at the same time.
> One strange issue or at least something I do not understand still remains :
> 
> I can receive all incoming data but when I try to send dummy data back in
> the receive callback, after 
> about 20 seconds lwip sends out only ACK Frames.
> 
> In my example two clients are connected, but only one client is sending data
> every 5ms. As you can see in the
> Link  https://www.pic-upload.de/view-33230789/wireshark.png.html my client
> with ip 192.168.0.8 sends data
> periodically to the server (192.168.0.2) In reply I send back dummy data.
> But suddenly the server does not send
> back dummy data any longer. The client does not get dummy data + ack  and
> because his timeout still did not 
> occur, he sends new data in greater frames. 
> In Frame 28094 the server only sends back an ACK without any data.
> 
> I debugged this issue and found out, that although tcp_write() does not
> return an error, the send_wnd int tcp_output()
> suddenly becomes too small. 
> 
> In tcp_output() I found the reason for the ACK Frame.
> 
> wnd = LWIP_MIN(pcb->snd_wnd, pcb->cwnd);
> 
> if (pcb->flags & TF_ACK_NOW && (seg == NULL ||
> lwip_ntohl(seg->tcphdr->seqno) - pcb->lastack + seg->len > wnd)) 
> {
> return tcp_send_empty_ack(pcb);
> }
> 
> By setting a breakpoint to "return tcp_send_empty_ack(pcb)" I got the
> arguments for the if case and because 
> of lwip_ntohl(389349632) - 79127 + 16 > 4 ) I go there. 
> 
> Of course I watched the lwip stats but I did not get any errors. Neither
> link, tcp nor  mem / memp.
> What could be the reason for that behavior? I increased my TCP_SND_BUF and
> TCP_SND_QUEULEN
> but without any difference.
> 
> #define TCP_WND16 * TCP_MSS
> #define TCP_MSS1460
> #define TCP_SND_BUF   16 * TCP_MSS   
> #define TCP_SND_QUEUELEN   48
> 
> Thank you very much. 
> 
> 
> 
> 
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Listen/Accept lwip 2.0.0

2017-05-04 Thread Joel Cunningham


Do you have TCP_LISTEN_BACKLOG enabled in your port?

Also keep in mind, the backlog only limits connections that have not been 
accepted.  Once accepted by your application, the connections are not part of 
the backlog.  If your goal is to support 4 simultaneous connections at once 
(with no pending connections), you need to close the listener once the server 
has 4 connections.

Joel

> On May 4, 2017, at 9:44 AM, Marcelo V  wrote:
> 
> Hello,
> 
> I am implementing a small web-server using lwip 2.0.0 and it seems that the 
> TCP stack is accepting more connections than set by my listen call.
> In my use-case I have an webpage with HTML requesting many external resources 
> to be loaded. However, I want to only serve max of 4 requests simultaneously 
> (4 sessions) and “keep-alive” is disable (replies 200 OK with "connection: 
> close”).
> 
> However, as soon as my code replies the very first accepted connection then I 
> see my web browser issuing 8~10 GET’s even before I call accept again. 
> Wireshark shows those 8~10 connections each with their port number and the 
> entire GET had flown to the server.
> 
> It seems that the number of connections provided by the listen were ignored 
> and TCP is accepting and buffering all requests.
> 
> I browsed the bug track and I did not find anything being reported.
> 
> How can I prevent this?
> 
> Thanks in advance,
> Marcelo
> 
> 
> 
> 
> 
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] zero window probe causes duplicated byte to be received

2017-04-02 Thread Joel Cunningham

Bug report: https://savannah.nongnu.org/bugs/?func=detailitem_id=49128
Fix: 
http://git.savannah.nongnu.org/cgit/lwip.git/commit/?id=9ba9dee2aa809624eda0e96c96abd8abff29539e

Joel

> On Apr 1, 2017, at 22:41, Yao Lin  wrote:
> 
> Thanks Joel. In case you find that bug, could you inform me.
> 
> 
> 
> --
> View this message in context: 
> http://lwip.100.n7.nabble.com/zero-window-probe-causes-duplicated-byte-to-be-received-tp29221p29223.html
> Sent from the lwip-users mailing list archive at Nabble.com.
> 
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] zero window probe causes duplicated byte to be received

2017-04-01 Thread Joel Cunningham

There was a bug where send_nxt was not advanced during the zero window probe 
and thus the zero window probe ACK caused LwIP to send back another ACK, 
creating an ACK storm. This has been fixed on master. I can dig up the bug 
report if you need it (away from git history at the moment)

Even with this fixed, you'll still see some of the behavior you described about 
repeating the same probe byte even if the remote side ACKs it. This is because 
LwIP can't send or ACK a partial segment, only the full segment. Once the 
window opens back up, the probe byte will be sent again as part of the full 
segment. Wireshark typically marks it as a retransmission 

Joel 

> On Apr 1, 2017, at 17:33, Yao Lin  wrote:
> 
> Hi,
> 
> I am using Stable 1.4.1 in my device. This device sends 4MB of TCP data to a
> Windows laptop. I am using netcomm API.
> 
> 1. When Windows is about to use up its TCP buffer, it sends a packet (P1)
> with rwnd = 0 and ACK to, say, 2M.
> 2. Within 0.12s after receiving P1, lwip sends a packet (P2) with seq = 2M
> and len = 1.
> 3. Windows sends P3 with ACK = 2M + 1 and rwnd = 0.
> 4. Now we go through many iterations of 4.1 and 4.2:
>4.1. lwip sends P4 with seq = 2M and len = 0.
>4.2. Windows sends P5 with ACK = 2M + 1 and rwnd = 0.
> 5. Eventually, Windows opens up its rwnd. So it sends P6 with ACK = 2M + 1
> and rwnd = 4096 (or something similar).
> 6. Lwip sends P7 with seq = 2M and len = 1460.
> 
> Now you can see Windows receives the duplicated byte 2M.
> 
> So my question is, why does lwip send P4 and P7 with seq = 2M instead of 2M
> + 1? Note that Windows ACK = 2M + 1 in P3, P5, and P6.
> 
> Thanks.
> 
> 
> 
> 
> --
> View this message in context: 
> http://lwip.100.n7.nabble.com/zero-window-probe-causes-duplicated-byte-to-be-received-tp29221.html
> Sent from the lwip-users mailing list archive at Nabble.com.
> 
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip FreeRTOS tcp window

2017-04-01 Thread Joel Cunningham

> On Apr 1, 2017, at 7:25 AM, Mohamed Hnezli  wrote:
> 
> Thank you for support,  
> Are there any way to change this, such as 1 ack for 5 TCP segments sent from 
> client (lwip)

What you referring to is a stretch ACK (where the ACK covers more than 1 full 
sized segment).  This is not configurable and stretching more than the 2 
maximum sized segments allowed by delayed ACKs algorithm is a violation of RFC 
1122  (see 4.2.3.2 When to Send an ACK 
Segment).

If you’re trying to tune the TCP performance from the sender’s side, stretching 
ACKs (on the receiver) will not improve performance.  Have you looked into 
tuning your send buffer (TCP_SND_BUF) given your bandwidth product delay?

Joel

> 
> On 31 March 2017 at 19:10, goldsi...@gmx.de  
> > wrote:
> Mohamed Hnezli wrote:
> 
> I am using lwip ontop of FreeRTOS to send data to a server. I am trying to 
> maximize the data flow. I've tuned several paramerters in "lwipopts.h" but I 
> am always receiving an ACK after emitting only two packets.
> 
> 
> That's just TCP. The window size does not have any influence about the number 
> of ACKs and normally, an ACK for every 2nd data segment is the default.
> 
> Simon
> 
> ___
> lwip-users mailing list
> lwip-users@nongnu.org 
> https://lists.nongnu.org/mailman/listinfo/lwip-users 
> 
> 
> 
> 
> -- 
> Mohamed HNEZLI
> Computer Science Engineer Student
>   NSCS_Tunisia
> ___
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Question regarding mdns responder

2017-03-21 Thread Joel Cunningham


> On Mar 21, 2017, at 9:28 AM, Dirk Ziegelmeier  wrote:
> 
> from mdns.c:
> 
>  * Things left to implement:
>  * -
>  *
>  * - Probing/conflict resolution
> […]

Given the list of limitations, should the current mDNS support only be used for 
querying?

Joel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Question regarding mdns responder

2017-03-21 Thread Joel Cunningham

> On Mar 21, 2017, at 6:30 AM, Dirk Ziegelmeier  wrote:
> 
> The problem is that you have two hosts with the same name "BGW" (= 
> "BGW.local" MDNS name) in the same network. They should have different names.
> 
> 

Is this a known limitation of LwIP’s mDNS implementation?  I haven’t done an 
audit, but from my experience with mDNS, the “Probing” step of claiming records 
should ensure the hostname is unique. See 
https://tools.ietf.org/html/rfc6762#section-8 

I've worked with Apple’s mDNSResponder and if two devices have the same 
hostname, the one that detects the collision ends up adding “(2)" the hostname 
to make it unique.

Joel

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip_sendto hanging with lwip 2.0

2017-01-25 Thread Joel Cunningham

If you want to send and receive on the same socket from two different threads, 
you’ll need to enable LWIP_NETCONN_FULLDUPLEX in opt.h (by default it’s not 
supported and feature is in alpha state):

/** LWIP_NETCONN_FULLDUPLEX==1: Enable code that allows reading from one thread,
 * writing from a 2nd thread and closing from a 3rd thread at the same time.
 * ATTENTION: This is currently really alpha! Some requirements:
 * - LWIP_NETCONN_SEM_PER_THREAD==1 is required to use one socket/netconn from
 *   multiple threads at once
 * - sys_mbox_free() has to unblock receive tasks waiting on recvmbox/acceptmbox
 *   and prevent a task pending on this during/after deletion
 */
#if !defined LWIP_NETCONN_FULLDUPLEX || defined __DOXYGEN__
#define LWIP_NETCONN_FULLDUPLEX 0
#endif

Joel

> On Jan 25, 2017, at 1:02 AM, Neerav Patel  wrote:
> 
> Hi,
> 
> I read that same statement after I had posted and so I created another system 
> thread (been debugging the whole day):
> 
> sys_thread_new("handle_thread", handle_thread, NULL, 400, 3);
> 
> and tried to send from it, and it also hung in the same way, seems to not 
> return from a semaphore call deep in the bowls of lwip, still digging further 
> into it.
> 
> Maybe this will do the same and call back from the tcpipthread.  
> 
> How would you propose structuring the code, I want one thread to receive 
> packets and another to send out?  Is creating a system thread the right way 
> to go?  Why would I be hanging with the code below?
> 
> Thanks for all the help!
> 
> void
> handle_thread( )
> {
>   while ( 1 )
>   {
> if ( send_actual_pkt ) 
> {
>   send_actual_pkt = 0;
>   uint8_t pkt[576];
>   // ... populate the pkt
>  uint16_t len = generate_pkt();
>  if ( lwip_sendto( s, pkt, len, 0, (struct sockaddr *), 
> sizeof(from) ) != len )   < hanging here
>  {
>uart_printf( "Didnt send correctly %x\n", len );
>  }
> }
>}
> }
> 
> 
> while( 1 )
>{
>   if ( (lwip_recvfrom(s, raw_buf, sizeof(raw_buf), 0, (struct 
> sockaddr*), (socklen_t*))) > 0 )
>   {
>   send_actual_pkt = 1;
>   }
>}
> 
> 
> 
> From: lwip-users  > on behalf of 
> Dirk Ziegelmeier >
> Sent: January 25, 2017 6:48 AM
> To: Mailing list for lwIP users
> Subject: Re: [lwip-users] lwip_sendto hanging with lwip 2.0
>  
> Calling sys_timeout() results in send_actual_pkt being called back from TCPIP 
> thread. You must not call sequencial style functions from TCPIP thread.
> 
> http://www.nongnu.org/lwip/2_0_0/group__sequential__api.html 
>  says: "More 
> overhead, but can be called from any thread except TCPIP thread."
> lwIP: Sequential-style APIs - nongnu.org 
> 
> www.nongnu.org 
> Detailed Description. Sequential-style APIs, blocking functions. More 
> overhead, but can be called from any thread except TCPIP thread.
> 
> 
> 
> Dirk
> 
> ___
> lwip-users mailing list
> lwip-users@nongnu.org 
> https://lists.nongnu.org/mailman/listinfo/lwip-users 
> 
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP send() fails when other sockets perform retransmissions

2017-01-05 Thread Joel Cunningham


> On Jan 3, 2017, at 9:34 AM, Daniel Pauli  wrote:
> 
> If you have LwIP stats enabled, you can check the memory pools for errors to 
> figure out which one is failing.  You should be able to resolve this by 
> sizing your memory pools to handle the number of supported connections.  For 
> example if you only support 5 simultaneous TCP connections, then your pools 
> should be big enough to allocate 5 send buffers worth of segments.  This is 
> how I configure my products, which typically have plenty of RAM.  Not sure 
> what the recommendation is for very constrained RAM products.
> 
> Using LwIP stats I found that memory allocations from RAW_PCB failed. By 
> increasing MEM_SIZE I was able to avoid the issue. I'm still unsure about a 
> reasonable value here, given that we want to support up to 64 simultaneous 
> TCP connections (MEMP_NUM_TCP_PCB==64). It is not a constrained RAM product, 
> though.
> 

RAW_PCB exhaustion is most likely unrelated to the problem you’re seeing.  When 
TCP segments are allocated in tcp_write() (with TCP_WRITE_FLAG_COPY specified) 
there are two allocations: PBUF_RAM (coming from LwIP heap, controlled by 
MEM_SIZE) and struct tcp_seg (coming from MEMP_NUM_TCP_SEG static memory pool)

In order to size these two pools according to what your connections can handle, 
you can use a worst case calculation where you assume all connections have a 
full send buffer worth of MSS segments.

MEM_SIZE: should be at least MEMP_NUM_TCP_PCB * TCP_SND_BUF with some extra 
space for miscellaneous heap allocations

MEMP_NUM_TCP_SEG: should be at least MEMP_NUM_TCP_PCB * (TCP_SND_BUF / TCP_MSS)

> Yes there is, with SO_LINGER you can perform an abortive closure rather than 
> graceful by setting the timeout to 0.  Typically this is a bad idea.  There’s 
> a decent discussion here on stackoverflow:
> 
> http://stackoverflow.com/questions/3757289/tcp-option-so-linger-zero-when-its-required
>  
> 
> 
>  We are using the default setting of LWIP_SO_LINGER==0. If I understand 
> correctly, this already completely disables linger processing and should 
> correspond to an abortive closure by setting the timeout to 0 as you 
> suggested. Is there another way to tell LWIP to release any resources 
> associated with a socket immediately? 
> 

I haven’t used LWIP_SO_LINGER in my port, but when not enabled, you get the 
default TCP closure behavior which is a graceful close with a 20 second timeout 
(see LWIP_TCP_CLOSE_TIMEOUT_MS_DEFAULT)

> Even when increasing MEM_SIZE, I feel uneasy about misbehaving clients 
> (ignoring retransmissions, reconnecting frequently) eating up all server 
> resources over time. In the case of unanswered retransmissions, I observed 
> that the TCP_PCB_LISTEN allocation counter is not decremented after close() 
> for a long time (probably 25 minutes?).

What state is the PCB in after closing the listener?  That sounds strange that 
this would pcb would hang around

Joel___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP send() fails when other sockets perform retransmissions

2016-12-30 Thread Joel Cunningham


> On Dec 30, 2016, at 12:59 PM, Daniel Pauli <dapau...@gmail.com> wrote:
> 
> Many thanks for your advice! I'll try to check the memory pools when I 
> reproduce the issue next time.
> 
> Further, since it sounds like you initially had sockets configured in 
> blocking mode, when the new socket tries to transmit, it will block trying to 
> allocate TCP segments due to the exhausted memory pool.  The blocking will 
> continue until SO_SNDTIMEOUT is reached or the memory exhaustion is resolved
> 
> To clarify, I ran two tests: In the first, all sockets used the MSG_DONTWAIT 
> flag for send() (non-blocking), in the second no socket used the flag 
> (blocking), so there should be no mixing of blocking/non-blocking from my 
> point of view. I'm not sure if I understand what you mean with "initially 
> configured in blocking mode". Does this mean that send() may still block 
> under certain circumstances (exhausted memory pool) even with MSG_DONTWAIT 
> flag set, so I should initially set the O_NONBLOCK option on the socket to 
> ensure that send() never blocks?
> 

I was referring to your original posting where you described seeing blocking 
when sending on a new socket after you had a stale socket in a half-open state 
with submitted data.  I was attempting to explain why that was happening so we 
know there is not erroneous behavior in LwIP

My understanding of BSD socket semantics is that using MSG_DONTWAIT should be 
equivalent to setting the O_NONBLOCK, though you’ll need to include the flag 
for each call rather than set the mode once.

> 
> On Fri, Dec 30, 2016 at 6:30 PM, Joel Cunningham <joel.cunning...@me.com 
> <mailto:joel.cunning...@me.com>> wrote:
> 
>> On Dec 30, 2016, at 10:43 AM, Daniel Pauli <dapau...@gmail.com 
>> <mailto:dapau...@gmail.com>> wrote:
>> 
>> I'm a little confused about the use of select in your application.  Are you 
>> using it with blocking sockets?
>> 
>> I tested with both blocking and non-blocking send. I observed that 
>> non-blocking send (MSG_DONTWAIT flag set) on sockets determined as 
>> write-ready by select() sometimes returned ENOMEM when "stale sockets" are 
>> around. After applying the patch from 
>> http://lwip.100.n7.nabble.com/bug-49684-lwip-netconn-do-writemore-non-blocking-ERR-MEM-treated-as-failure-td27860.html
>>  
>> <http://lwip.100.n7.nabble.com/bug-49684-lwip-netconn-do-writemore-non-blocking-ERR-MEM-treated-as-failure-td27860.html>,
>>  I got EWOULDBLOCK errors instead.
>> 
> 
> Thanks for including this information. The ENOMEM gives me a good clue of 
> what’s most likely going on.  My guess is that you’re experiencing a memory 
> pool exhaustion and the stale socket has claimed memory from a pool for the 
> segments which are queued for transmit.  Since those segments are not being 
> ACKed in the half open state, the claimed memory won’t be available until the 
> segments are freed (happens during transmission timeout or when socket is 
> aborted)
> 
> Further, since it sounds like you initially had sockets configured in 
> blocking mode, when the new socket tries to transmit, it will block trying to 
> allocate TCP segments due to the exhausted memory pool.  The blocking will 
> continue until SO_SNDTIMEOUT is reached or the memory exhaustion is resolved
> 
> If you have LwIP stats enabled, you can check the memory pools for errors to 
> figure out which one is failing.  You should be able to resolve this by 
> sizing your memory pools to handle the number of supported connections.  For 
> example if you only support 5 simultaneous TCP connections, then your pools 
> should be big enough to allocate 5 send buffers worth of segments.  This is 
> how I configure my products, which typically have plenty of RAM.  Not sure 
> what the recommendation is for very constrained RAM products.
> 
>> 
>> Calling close() will initiate a graceful synchronized closure of the 
>> connection.  This means continuing to send any queued data until it is 
>> ACKed, the send times out, or we received a RST.  Then a FIN is sent 
>> indicating the sending pathway is closed.
>> 
>> So there's no direct way for the application to tell LWIP to just give up on 
>> one socket without further trying to send data? Can the application specify 
>> a send timeout?\
> 
> Yes there is, with SO_LINGER you can perform an abortive closure rather than 
> graceful by setting the timeout to 0.  Typically this is a bad idea.  There’s 
> a decent discussion here on stackoverflow:
> 
> http://stackoverflow.com/questions/3757289/tcp-option-so-linger-zero-when-its-required
>  
> <http://stackoverflow.com/questions/3757289/tcp-optio

Re: [lwip-users] TCP send() fails when other sockets perform retransmissions

2016-12-30 Thread Joel Cunningham

> On Dec 30, 2016, at 10:43 AM, Daniel Pauli <dapau...@gmail.com> wrote:
> 
> I'm a little confused about the use of select in your application.  Are you 
> using it with blocking sockets?
> 
> I tested with both blocking and non-blocking send. I observed that 
> non-blocking send (MSG_DONTWAIT flag set) on sockets determined as 
> write-ready by select() sometimes returned ENOMEM when "stale sockets" are 
> around. After applying the patch from 
> http://lwip.100.n7.nabble.com/bug-49684-lwip-netconn-do-writemore-non-blocking-ERR-MEM-treated-as-failure-td27860.html
>  
> <http://lwip.100.n7.nabble.com/bug-49684-lwip-netconn-do-writemore-non-blocking-ERR-MEM-treated-as-failure-td27860.html>,
>  I got EWOULDBLOCK errors instead.
> 

Thanks for including this information. The ENOMEM gives me a good clue of 
what’s most likely going on.  My guess is that you’re experiencing a memory 
pool exhaustion and the stale socket has claimed memory from a pool for the 
segments which are queued for transmit.  Since those segments are not being 
ACKed in the half open state, the claimed memory won’t be available until the 
segments are freed (happens during transmission timeout or when socket is 
aborted)

Further, since it sounds like you initially had sockets configured in blocking 
mode, when the new socket tries to transmit, it will block trying to allocate 
TCP segments due to the exhausted memory pool.  The blocking will continue 
until SO_SNDTIMEOUT is reached or the memory exhaustion is resolved

If you have LwIP stats enabled, you can check the memory pools for errors to 
figure out which one is failing.  You should be able to resolve this by sizing 
your memory pools to handle the number of supported connections.  For example 
if you only support 5 simultaneous TCP connections, then your pools should be 
big enough to allocate 5 send buffers worth of segments.  This is how I 
configure my products, which typically have plenty of RAM.  Not sure what the 
recommendation is for very constrained RAM products.

> 
> Calling close() will initiate a graceful synchronized closure of the 
> connection.  This means continuing to send any queued data until it is ACKed, 
> the send times out, or we received a RST.  Then a FIN is sent indicating the 
> sending pathway is closed.
> 
> So there's no direct way for the application to tell LWIP to just give up on 
> one socket without further trying to send data? Can the application specify a 
> send timeout?\

Yes there is, with SO_LINGER you can perform an abortive closure rather than 
graceful by setting the timeout to 0.  Typically this is a bad idea.  There’s a 
decent discussion here on stackoverflow:

http://stackoverflow.com/questions/3757289/tcp-option-so-linger-zero-when-its-required

<http://stackoverflow.com/questions/3757289/tcp-option-so-linger-zero-when-its-required>

> 
> Lastly, what version of LwIP are you using?
> 
> I'm using 2.0.0 RC1

Joel

> On Wed, Dec 28, 2016 at 4:23 PM, Joel Cunningham <joel.cunning...@me.com 
> <mailto:joel.cunning...@me.com>> wrote:
> 
> 
> On Dec 28, 2016, at 06:45 AM, Daniel Pauli <dapau...@gmail.com 
> <mailto:dapau...@gmail.com>> wrote:
> 
>> Am I understanding the description correctly that sending on the stale 
>> connection eventually blocks once the remote side has crashed and this 
>> prevents sending on the new socket (only because the thread is blocked)?
>> 
>> If so, then the socket buffer on the stale socket has filled up (most 
>> likely) and is now blocking.  This is blocking I/O operating as expected 
>> when data is not being acknowledged.  You should use non-blocking sockets 
>> and select if your server is servicing multiple sockets on a single thread.
>> 
>> Joel
>> 
>> Attempting to send on the stale socket blocks, which is okay on its own. But 
>> I'm already using select() and observed that
> 
>  
>> 
>> these stale sockets still somehow seem to block communication over new 
>> sockets,
> 
> 
> If this is actually happening as described, that would be unexpected/faulty 
> behavior.  One TCP socket in the half-open state should not have any effect 
> on the other TCP connections.
>  
>> 
>> even when no stale sockets are included in the write set of select().
> 
> 
> I'm a little confused about the use of select in your application.  Are you 
> using it with blocking sockets?  Select returning write-ability doesn't 
> guarantee the send call won't block.  If you have a blocking socket and the 
> size in the send call can't fit in the amount of available buffer space, the 
> call will block
>  
>> 
>> I even close() (successfully, according to the return value) those stale 
>> sockets a

Re: [lwip-users] TCP send() fails when other sockets perform retransmissions

2016-12-28 Thread Joel Cunningham





On Dec 28, 2016, at 06:45 AM, Daniel Pauli  wrote:


Am I understanding the description correctly that sending on the stale 
connection eventually blocks once the remote side has crashed and this prevents 
sending on the new socket (only because the thread is blocked)?



If so, then the socket buffer on the stale socket has filled up (most likely) 
and is now blocking.  This is blocking I/O operating as expected when data is 
not being acknowledged.  You should use non-blocking sockets and select if your 
server is servicing multiple sockets on a single thread.



Joel


Attempting to send on the stale socket blocks, which is okay on its own. But 
I'm already using select() and observed that
 
these stale sockets still somehow seem to block communication over new sockets,


If this is actually happening as described, that would be unexpected/faulty 
behavior.  One TCP socket in the half-open state should not have any effect on 
the other TCP connections.

 
even when no stale sockets are included in the write set of select().


I'm a little confused about the use of select in your application.  Are you 
using it with blocking sockets?  Select returning write-ability doesn't 
guarantee the send call won't block.  If you have a blocking socket and the 
size in the send call can't fit in the amount of available buffer space, the 
call will block

 
I even close() (successfully, according to the return value) those stale 
sockets after they failed to be write-ready after 10 seconds, but I can see in 
Wireshark that LWIP still sends retransmissions from the port number of the 
closed socket. 


Could it be that close() cannot send FIN because the output buffer is full, so 
the socket still remains active? Is there a way from the API to just drop the 
connection without involving any more communication?


Calling close() will initiate a graceful synchronized closure of the 
connection.  This means continuing to send any queued data until it is ACKed, 
the send times out, or we received a RST.  Then a FIN is sent indicating the 
sending pathway is closed.



Lastly, what version of LwIP are you using?


Joel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP send() fails when other sockets perform retransmissions

2016-12-27 Thread Joel Cunningham


Am I understanding the description correctly that sending on the stale 
connection eventually blocks once the remote side has crashed and this prevents 
sending on the new socket (only because the thread is blocked)?



If so, then the socket buffer on the stale socket has filled up (most likely) 
and is now blocking.  This is blocking I/O operating as expected when data is 
not being acknowledged.  You should use non-blocking sockets and select if your 
server is servicing multiple sockets on a single thread.



Joel


On Dec 23, 2016, at 04:15 PM, Daniel Pauli  wrote:


Hey there

I have a TCP client running on a Windows PC that communicates with a TCP
server on a LWIP box. The client sends requests to the server in high
frequency (polling) and receives responses of approx. 16 KB.

The problem: When a clients crashes (without being able to send FIN/RST
because of e.g. temporary link down) and gets restarted, the server is
unable to send responses over the new connection because send() blocks.
This happens even though I'm using select() to determine if sockets are
writable. I'm using a SO_SNDTIMEO of 3 seconds for tracking the problem
down. Strangely, send() already returns after 1 second with EWOULDBLOCK.

I observed that LWIP performs TCP retransmissions on the socket from the
crashed client. In my setting, these retransmissions are ignored by the
PC (due to the Windows firewall).
The problem resolves after approx 25 min when LWIP stops performing
these retransmissions. After that, send() works again for new connections.

Any hints are greatly appreciated :)

Regards
Daniel

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Zero window and refused data problem

2016-12-09 Thread Joel Cunningham


Oleg,



Would you mind testing the patch I've posted to 
https://savannah.nongnu.org/bugs/?49631



Thanks,



Joel

On Nov 09, 2016, at 01:26 AM, Oleg Gladyshev  wrote:




And since we don't want to create even more traffic in an overload situation, 
we don't even send
and ACK (with an old ACKNO), since there's nothing the remote host can improve: 
we can't accept new
segments! By doing that, we give the application time to overcome resource 
shortage and only can
hope the connection survives this.
Simon, let's look at the issue from another side. If the refused_data is
full LwIP doesn't send any ACK for probe packet but if the receiving
window is really zero it WILL ACK with previous ACKNO. From sender's
point of view these two cases are identical! In both cases receiver
announces zero window and sender "asks peer if it has a liitle more
space". I think it is wrong when received under LwIP control has
different behavior depending on it's internal state (real window size
and refused_data variable). For sender both cases are exactly the same
so receiver must ACK with old ACKNO if it can't receive more data no
matter why it can't.

--
Best regards, Oleg


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] UDP bind error due to address and port reuse

2016-12-05 Thread Joel Cunningham





On Dec 03, 2016, at 01:05 PM, Michael Steinberg  wrote:



Am 03.12.2016 um 17:37 schrieb Surya Chaitanya:

Hi,


The scenario is something like this. I have created an array of say, 5 UDP 
PCB's.
I'd like to bind all these 5 UDP PCB's to the same local IP address and port 
number
combination. Isn't this posible? Like you said, if we cannot bind an IP:Port 
more
than one time, what is the purpose of SO_REUSE and SOF_REUSEADDR flags? Aren't 
they
for reusing a particular port:IP combination for several PCBs at the same time?
Thank you.


Regards,
Surya

Hello Surya,

Have a look at this nice explanation

http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t

which shows in detail why those options are not meant to do what you
seem to be thinking (are you expecting some kind of load balancing?) on
standard bsd socket behaviour. I cannot tell if the intended behaviour
is different on lwip though.


LwIP implements SO_REUSEADDR and last I examined the behavior, it actually 
behaved like SO_REUSEPORT according to the BSD semantics (not the Linux load 
balancing behavior).   This should allow multiple UDP binds on the same IP+ 
port, so I'm surprised to hear there are problems



See discussion here: 
http://lists.nongnu.org/archive/html/lwip-devel/2015-07/msg00017.html



Joel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Zero window and refused data problem

2016-11-08 Thread Joel Cunningham


Follow up on my response, see below


On Nov 08, 2016, at 05:58 PM, Joel Cunningham <joel.cunning...@me.com> wrote:


Simon,


On Nov 08, 2016, at 02:54 AM, Simon Goldschmidt <goldsi...@gmx.de> wrote:


Hi Oleg,
 
first of all, I think you are "misusing" TCP as a queue here (and at least our 
implementation is not really meant for this). It might work (or not, as you see), but in 
my opinion, the various timeouts implemented by various stacks impose the risk that your 
setup won't work if you change the client's system (e.g. update windows).
 
If I read correctly, by now you reported 2 possible issues:
1) segment is accepted but old window size is sent
 
I'm not sure what's best here. Of course, we must prevent silly window updates 
during normal operation. In your case, it would probably have been OK to send 
the actual/real window size, but we would haveto find a way to decide when it's 
OK and when not...


As we saw in https://savannah.nongnu.org/bugs/?49128 (and I've also seen with 
Windows 7 as a receiver) a stack can ACK (increase ACK by 1) zero window probes 
which contain 1 byte from the next unsent segment, after the window is closed. 
In the case I've seen with Windows 7, the reported window size in the ACK is 
still 0.


We could separate the silly window avoidance from the update threshold setting 
because it should be safe to report the window once 1 MSS is available regardless 
of if the update threshold is 1/4 of the window.  This issue would still exist for 
when the application has read < 1 MSS of data though.  My understanding of the 
update threshold is that it reduces the number of window updates by giving a 
chance to combine the window update with an outgoing data segment/delayed ACK



From TCP/IP Illustraved Vol: 1, section 22.3:



The receiver must not advertise small windows. The normal algorithm is for the 
receiver not to advertise
a larger window than it is currently advertising (which can be 0) until the 
window can be increased by
either one full-sized segment (i.e„ the MSS being received) or by one-half the 
receiver's buffer space,
whichever is smaller


Just below this section in TCP/IP Illustrated, it gives an excellent example 
that walks through the exact same issue we are discussing:


When the persist timer expires, 1 byte of data is sent (segment 6). The 
receiving application has read 256 bytes
from the receive buffer (at time 3.99), so the byte is accepted and 
acknowledged (segment 7). But the
advertised window is still 0, since the receiver does not have room for either 
one full-sized segment or one-half
of its buffer. This is silly window avoidance by the receiver.



The sender's persist timer is reset and goes off again 5 seconds later (at time 
10.151). One byte is again sent and
acknowledged (segments 8 and 9). Again the amount of room in the receiver's 
buffer (1022 bytes) forces it to
advertise a window of 0.



When the sender's persist timer expires next, at time 15.151, another byte is 
sent and acknowledged (segments
10 and 11). This time the receiver has 1533 bytes available in its buffer, so a 
nonzero window is advertised. The
sender immediately takes advantage of the window and sends 1024 bytes (segment 
12). The acknowledgment
of these 1024 bytes (segment 13) advertises a window of 509 bytes. This appears 
to contradict what we've seen
earlier with small window advertisements.


So LwIP is  behaving correctly for when the window is < 1 MSS.  For wnd > 1 MSS 
(regardless of update threshold) we should be using the current window value in the 
ACK





 
2) after storing a segment in "refused_data", no more ACKs are sent
 
The whole purpose of "refused_data" was to let the stack behave like a buffer 
overflowed: if your device cannot handle incoming data in the speed it is sent by the 
remote host, the remote host should throttle its sender. This is achieved by not 
handling/not answering a packet at all, just like it was dropped due to congestion. This 
should bring the remote host's TCP to send less. ACKing an old seqno instead might work 
for you, but I don't know what will be the result for all remote stacks, so I'm very 
reluctant to change this...
 
As you can see from this, TCP is meant to achieve the highest possible 
throughput possible for the combination of remote host, network and local host. 
What you want instead to make it a queue that keeps up a connection as long as 
possible without data being exchanged. I'm not fully convinced one can coexist 
with the other, but please come up with suggestions of how to fix this ;-)
 
 


Is the intent that an application would use the refused_data feature as part of 
it's normal workflow?  Or is it expected that once this condition happens, the 
developer becomes aware of it and either increases resources in the mbox 
receive buffer implementation (to match the configured window size) or reduce 
the configured window size since the sys

Re: [lwip-users] Zero window and refused data problem

2016-11-08 Thread Joel Cunningham


Simon,


On Nov 08, 2016, at 02:54 AM, Simon Goldschmidt  wrote:


Hi Oleg,
 
first of all, I think you are "misusing" TCP as a queue here (and at least our 
implementation is not really meant for this). It might work (or not, as you see), but in 
my opinion, the various timeouts implemented by various stacks impose the risk that your 
setup won't work if you change the client's system (e.g. update windows).
 
If I read correctly, by now you reported 2 possible issues:
1) segment is accepted but old window size is sent
 
I'm not sure what's best here. Of course, we must prevent silly window updates 
during normal operation. In your case, it would probably have been OK to send 
the actual/real window size, but we would haveto find a way to decide when it's 
OK and when not...


As we saw in https://savannah.nongnu.org/bugs/?49128 (and I've also seen with 
Windows 7 as a receiver) a stack can ACK (increase ACK by 1) zero window probes 
which contain 1 byte from the next unsent segment, after the window is closed. 
In the case I've seen with Windows 7, the reported window size in the ACK is 
still 0.


We could separate the silly window avoidance from the update threshold setting 
because it should be safe to report the window once 1 MSS is available regardless 
of if the update threshold is 1/4 of the window.  This issue would still exist for 
when the application has read < 1 MSS of data though.  My understanding of the 
update threshold is that it reduces the number of window updates by giving a 
chance to combine the window update with an outgoing data segment/delayed ACK



From TCP/IP Illustraved Vol: 1, section 22.3:



The receiver must not advertise small windows. The normal algorithm is for the 
receiver not to advertise
a larger window than it is currently advertising (which can be 0) until the 
window can be increased by
either one full-sized segment (i.e„ the MSS being received) or by one-half the 
receiver's buffer space,
whichever is smaller


 
2) after storing a segment in "refused_data", no more ACKs are sent
 
The whole purpose of "refused_data" was to let the stack behave like a buffer 
overflowed: if your device cannot handle incoming data in the speed it is sent by the 
remote host, the remote host should throttle its sender. This is achieved by not 
handling/not answering a packet at all, just like it was dropped due to congestion. This 
should bring the remote host's TCP to send less. ACKing an old seqno instead might work 
for you, but I don't know what will be the result for all remote stacks, so I'm very 
reluctant to change this...
 
As you can see from this, TCP is meant to achieve the highest possible 
throughput possible for the combination of remote host, network and local host. 
What you want instead to make it a queue that keeps up a connection as long as 
possible without data being exchanged. I'm not fully convinced one can coexist 
with the other, but please come up with suggestions of how to fix this ;-)
 
 


Is the intent that an application would use the refused_data feature as part of 
it's normal workflow?  Or is it expected that once this condition happens, the 
developer becomes aware of it and either increases resources in the mbox 
receive buffer implementation (to match the configured window size) or reduce 
the configured window size since the system can't handle the data segment 
pattern?

 
Joel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Zero window and refused data problem

2016-11-07 Thread Joel Cunningham


Oleg,



I agree that setting the window update threshold below 1 MSS would be bad.  I 
didn't realize your window was being updated in such small amounts.  Is there a 
reason why your application isn't continuing to drain the buffer?



In terms of receiving a zero-window probe.  Is LwIP ACKing the probe because 
you technically do have window space available because your application read 
some data, but not enough to trigger the threshold?  If that's the case, then 
it sounds like LwIP is behaving correctly.  If LwIP is continuing to accept 
data even though the receive buffer is full and real window is 0, then that's a 
problem.


A zero window probe that is borrowing the probe byte of the next unsent segment 
is normal and in this case, the it sounds like the window has space, but we 
aren't advertising that space.  Eventually the window should fully fill up and 
LwIP would (hopefully) stop ACKing zero window probe.



Joel


On Nov 07, 2016, at 01:16 PM, Oleg Gladyshev  wrote:


Joel, thank you for your reply. 




Oleg,



In your original post, you mentioned that your application reads some data out 
of the buffer, but this doesn't result in a window update. This is because LwIP 
waits until 1/4 of the window has been opened up.  There is a setting that 
controls the update threshold in opt.h.  You can adjust this to match your use 
case.

Yes, it is normal behavior to aviod silly window if I right. I know that I can 
reduce threshold, but IMHO it is not good solution. I can change threshold from 
N to M bytes, but for M bytes the problem still here. In worst case I need 
exactly M-1 mbox capacity. Obviosly I can't set threshold to few bytes because 
it will disable silly window avoidance.



I'm also surprised that when LwIP ACKs the zero window probe, it doesn't 
contain the updated window value. 
Yes, it's true for 1.4.1 version. And it looks right for me because this time 
real (internal) window size still less than the threshold.

What top-level API is your application using, sockets or Raw?  With the Raw 
API, I believe you have to call tcp_recved() after your application receives 
data out of the buffer to update the window.
No, I use netconn API which calls tcp_recved() internally.





Joel


On Nov 07, 2016, at 10:03 AM, Oleg Gladyshev  wrote:


Hi Noam, 

Thank you very much for your reply but I think you misunderstood me. 
Look at the article: https://wiki.wireshark.org/TCP%20ZeroWindow
Of course my host (controlled by Windows now) sends data using quite large 
packets. But when device's buffer becomes full
host starts periodical checking of the buffer. Windows performs it using 1 byte packets 
called "Zero Window Probe packet" and we can't affect this behavior.




Hi,

First of all if you have control over the host machine avoid sending 1 byte. 
This is not efficient.
I suggest adding a dual mechanism. Defining size and/or timeout.

I mean adding some mechanism that counts bytes ready to be sent. If you have 
more than X
Bytes you send them out. Also add a timeout. Meaning if you have collected less 
then X bytes
but a time period has elapsed since the last send, send it anyway. This way you 
force the sender
to send a minimum sized buffer and not send out 1 byte at a time with a large 
overhead...

At your device side you probably have a DMA collecting data. You may modify the 
code and check
How many bytes have you received. Similar to the above mechanism. If you got 
less than some limit
you do not allocate a pBuf ... only if you have a minimum data size or a time 
difference from last
data handling you allocate a pBuf, copy data to it and let the TCP stack handle 
it.

BR,
Noam.

-Original Message-
From: lwip-users [mailto:lwip-users-bounces+noam=silrd@nongnu.org] On 
Behalf Of Oleg Gladyshev
Sent: Monday, November 07, 2016 3:23 PM
To: lwip-users@nongnu.org
Subject: [lwip-users] Zero window and refused data problem

Hi all,
I develop a device which receives data from host machine using TCP (LwIP 
1.4.1). Host sends data continuously queuing it to the controller.
Controller process data as fast as possible, but data process time is not 
determined. it can be from milliseconds to hours. And I wish to use TCP/IP 
incoming buffer as buffer for the data.

In my case TCP window on the device becomes full very often. Sometimes device's 
application receives from the TCP stack portion of data less than TCP windows 
size. This case LwIP doesn't update receive window leaving it zero. But the 
device able to receive some bytes now. And when the host sends 1 byte ZeroProbe 
packet device receives it.
For this one byte LwIP allocates new pbuf structure and sends it to application 
using sys_mbox_xxx call.
I use fixed-size mbox queues so after some probe packets from the host mbox 
becomes full and refuses new data. tcp_input notices this and tries to deliver 
refused data to the application every time it called.
But if application still refuses

Re: [lwip-users] Zero window and refused data problem

2016-11-07 Thread Joel Cunningham


Oleg,



In your original post, you mentioned that your application reads some data out 
of the buffer, but this doesn't result in a window update.  This is because 
LwIP waits until 1/4 of the window has been opened up.  There is a setting that 
controls the update threshold in opt.h.  You can adjust this to match your use 
case.



/**
 * TCP_WND_UPDATE_THRESHOLD: difference in window to trigger an
 * explicit window update
 */
#ifndef TCP_WND_UPDATE_THRESHOLD
#define TCP_WND_UPDATE_THRESHOLD   LWIP_MIN((TCP_WND / 4), (TCP_MSS * 4))
#endif



I'm also surprised that when LwIP ACKs the zero window probe, it doesn't 
contain the updated window value.  What top-level API is your application 
using, sockets or Raw?  With the Raw API, I believe you have to call 
tcp_recved() after your application receives data out of the buffer to update 
the window.



Joel


On Nov 07, 2016, at 10:03 AM, Oleg Gladyshev  wrote:


Hi Noam, 

Thank you very much for your reply but I think you misunderstood me. 
Look at the article: https://wiki.wireshark.org/TCP%20ZeroWindow 
Of course my host (controlled by Windows now) sends data using quite large 
packets. But when device's buffer becomes full
host starts periodical checking of the buffer. Windows performs it using 1 byte packets 
called "Zero Window Probe packet" and we can't affect this behavior.




Hi,

First of all if you have control over the host machine avoid sending 1 byte. 
This is not efficient.
I suggest adding a dual mechanism. Defining size and/or timeout.

I mean adding some mechanism that counts bytes ready to be sent. If you have 
more than X
Bytes you send them out. Also add a timeout. Meaning if you have collected less 
then X bytes
but a time period has elapsed since the last send, send it anyway. This way you 
force the sender
to send a minimum sized buffer and not send out 1 byte at a time with a large 
overhead...

At your device side you probably have a DMA collecting data. You may modify the 
code and check
How many bytes have you received. Similar to the above mechanism. If you got 
less than some limit
you do not allocate a pBuf ... only if you have a minimum data size or a time 
difference from last
data handling you allocate a pBuf, copy data to it and let the TCP stack handle 
it.

BR,
Noam.

-Original Message-
From: lwip-users [mailto:lwip-users-bounces+noam=silrd@nongnu.org] On 
Behalf Of Oleg Gladyshev
Sent: Monday, November 07, 2016 3:23 PM
To: lwip-users@nongnu.org
Subject: [lwip-users] Zero window and refused data problem

Hi all,
I develop a device which receives data from host machine using TCP (LwIP 
1.4.1). Host sends data continuously queuing it to the controller.
Controller process data as fast as possible, but data process time is not 
determined. it can be from milliseconds to hours. And I wish to use TCP/IP 
incoming buffer as buffer for the data.

In my case TCP window on the device becomes full very often. Sometimes device's 
application receives from the TCP stack portion of data less than TCP windows 
size. This case LwIP doesn't update receive window leaving it zero. But the 
device able to receive some bytes now. And when the host sends 1 byte ZeroProbe 
packet device receives it.
For this one byte LwIP allocates new pbuf structure and sends it to application 
using sys_mbox_xxx call.
I use fixed-size mbox queues so after some probe packets from the host mbox 
becomes full and refuses new data. tcp_input notices this and tries to deliver 
refused data to the application every time it called.
But if application still refuses new data tcp_input() doesn't send ACK to the 
host! And host disconnects after some tries.

Making my mboxes dynamically doesn't solve the problem because in general we 
can waste all RAM with (TCP_WND-1) pbuf structures for every incoming byte from 
window probe packets.

I think it would be better to ACK with previous value to the host instead of 
silently dropping new data. What do you think about the issue? What is the best 
way to avoid disconnects in my project?

--
Best regards, Oleg


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users




---
С уважением, Олег (gla...@mail.ru)


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Is there a way to cancel a call to lwip_select()?

2016-09-27 Thread Joel Cunningham


You can achieve the same thing with LwIP by using a UDP socket bound to a port 
on the loopback adapter.  Sending to the loopback address and port (from 
another thread) will be received on your UDP socket and will return the select 
call



Joel


On Sep 24, 2016, at 03:56 PM, doragasu  wrote:


I think this is a typical problem. I have lwip_select() blocked waiting
events on several sockets. But now I want to add another socket to the FD
set. When using select() on Linux/BSD, the usual way to solve this problem
is to add another descriptor (e.g. a pipe) to the FD set, so you can do
dummy writes to this descriptor to "cancel" the select() call.

Is it possible to do anything similar when using lwip_select()?



--
View this message in context: 
http://lwip.100.n7.nabble.com/Is-there-a-way-to-cancel-a-call-to-lwip-select-tp27390.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] ethernet driver overloaded?

2016-08-29 Thread Joel Cunningham


I've ran into this problem on my products as well, but the situation is a 
little different.  We use LwIP on top of a WiFi subsystem and it's easy to fill 
up the driver's transmit queue when the device (operating as a station) moves 
out of range of the AP.



We've handled it in two manners, both have pros/cons:



1) Block the calling thread until space opens up in the driver's transmit 
queue.  This became problematic because the sending thread was blocked for a 
long time in the out of range case.  We've mainly seen this with UDP sockets 
because there is no socket buffering and each UDP sendto() calls right into the 
driver.  I believe it can happen with TCP too because LwIP does tcp_write() and 
then tcp_output() (which would block in this scenario).

2) Drop the packet when the transmit queue is full.  We've been using this 
behavior when the long blocking is unacceptable.  Cons here are mainly for UDP 
where the packet is dropped (TCP will at least retransmit).  This isn't too 
much of a concern for us because in the out of range case, you get a lot of 
dropped data anyways.



I think the ideal system for non-blocking sockets is to propagate the out of 
resource message.  For TCP, that means leave the packet in the send buffer (if 
send buffer is full, packet is rejected and return EWOULDBLOCK).  For UDP, 
reject the send since there is no buffering and return EWOULDBLOCK.  This lets 
the application know it can't send right now and gives the application the 
option to stop generating data until the socket is marked as writable again.  
Blocking sockets is much simpler as we just remained blocked until the driver 
lets us transmit :)



The TCP pathway is probably closer to supporting this now as we do have some 
handling for out of memory errors.  The UDP pathway would need work and to 
leverage a callback as described by Simon in order to mark the socket as 
writable again (once callback is triggered)



Joel


On Aug 29, 2016, at 02:01 PM, "goldsi...@gmx.de"  wrote:


Hi,

originally, this hasn't been covered by lwIP: the systems were so slow that you 
basically
never got into that mode (the assumption was that the network hardaware is 
faster in
transmitting than the CPU is in sending more packets).

Of course, this doesn't hold any more, so today it's assumed that a driver that 
doesn't
send a frame blocking (easy to implement but bad performance)
a) enqueues a packet into hardware buffers for sending or
b) enqueues the pbuf to transmit on a software list of pbufs that is sent later 
if
enough harder buffers are free

If it returns "out of memory", TCP will stop sending until for some time, so 
that's not
a good idea. If it just stops sending, the result is mostly the same. So 
indeed, a new
return code for "hw full, try again on callback" plus an additional callback 
might be
best. It is *not* required to achieve good performance, though. It could improve
overall memory usage however.

Simon


Am 29.08.2016 um 12:40 schrieb Alexander Schulz:


hi,

 

in a current project I am trying to send out a relatively large consecutive 
data stream

using tcp/ip.

 

I can see that data is handed over to the ethernet driver via the registered 
function

at “linkoutput”.

what I am missing is the evaluation of a return value indicating that the 
underlying ethernet

driver is still busy sending. I see that “linkoutput”’s retval goes up to 
“tcp_output_segment()”

but here, the retval of “ip_output()” is just discarded.

how is the overload at the ethernet driver prevented?

 

maybe relying on tcp’s recover mechanism could cover this up but I’d like to 
prefer

avoiding stream errors right in the first place.

maybe I just overlook something…

 

b.-regards.

 





Renesas Electronics Europe GmbH,Geschaeftsfuehrer/President : Michael 
Hannawald, Sitz der Gesellschaft/Registered office: Duesseldorf, Arcadiastrasse 
10, 40472 Duesseldorf, Germany,Handelsregister/Commercial Register: 
Duesseldorf, HRB 3708 USt-IDNr./Tax identification no.: DE 119353406 
WEEE-Reg.-Nr./WEEE reg. no.: DE 14978647





___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] [lwip-devel] lwIP documentation update

2016-08-15 Thread Joel Cunningham


On the Overview page, there is a minor typo under the documentation section:



There is now a constantly growin wiki about lwIP at
 http://lwip.wikia.com/wiki/LwIP_Wiki

It should say "growing wiki"


Joel

On Aug 08, 2016, at 02:14 AM, Dirk Ziegelmeier  wrote:


Hello all,


the lwIP documentation at

http://www.nongnu.org/lwip/2_0_0/index.html


was greatly improved. Have a look, and please tell me if you have any comments, 
corrections or improvements!


Dirk


___
lwip-devel mailing list
lwip-de...@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-devel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwIP delays outgoing TCP packets by up to 500ms

2016-07-13 Thread Joel Cunningham


It's worth comparing the behavior of the netconn API.  Even though the netconn 
API is only available with NO_SYS = 0, it is still a user of the raw API like 
the applications mentioned in this email thread.



lwip_netconn_do_writemore() calls tcp_write() until all segments have been put 
into the TCP send buffer, then calls tcp_output().



Joel


On Jul 13, 2016, at 10:36 AM, Pîrvu Mihai  wrote:


Again, it's probably not be the wrong approach, it's just what I've been told 
on this thread:


https://lists.gnu.org/archive/html/lwip-users/2016-06/msg00055.html



Since it worked for me at that moment, I just let it like that, and reading the 
responses here confirms that it's the correct approach if you don't want to 
wait for the delay.


Mihai


On Wed, Jul 13, 2016 at 5:59 PM, Jakub Schmidtke  wrote:


I think you might have missed reading the documentation: tcp_write enqueue 
data, tcp_outout tries to send it. You always have to call both.




I actually found the recommended call flow on wiki (Here: 
http://lwip.wikia.com/wiki/Raw/TCP ) after Mihai mentioned using tcp_output().

And yes, I haven't fully read the docs/wiki yet, since I am trying to fix 
existing code written by someone else - who must have missed that...


However, Mihai mentions that calling tcp_output() might be the wrong 
approach... Is it really?


Thanks!

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LwIP multithread select mode problems

2016-06-28 Thread Joel Cunningham


See responses in-line



Joel


On Jun 24, 2016, at 09:31 PM, lampo  wrote:


thanks a lot.

now I use *recv*, *send*, *close* apis in the same thread per one
socket, that means each client socket occupy a new thread, and each of
them calles *select*, *recv*, *send*, *close* .

I'm wondering is this usage still so called multi-threaded environment
? can I just neglect SYS_ARCH_PROTECT here? for using
SYS_ARCH_PROTECT , there's really some bad thing happening inside the whole
schedule system. And it seems OK with 3 days of running without
SYS_ARCH_PROTECT .


I would say no.  The SYS_ARCH_PROTECT isn't just protecting against your 
application threads, but also the LwIP core thread and your application


If your port can't support SYS_ARCH_PROTECT in any of its forms (disable/enable 
interrupts, semaphore, mutex) then I would really question how safe that 
OS/platform is to use with LwIP in multi-threading mode.  How are the other SYS 
arch constructs (mbox, mutex, sem) working?



I just want to confirm the usage of SYS_ARCH_PROTECT, or where may
lead to data consistency problems due to not using SYS_ARCH_PROTECT
mechanism, when programing in this *specific multi-threaded environment *



If you want more confidence that your usage is safe, I think you're going to 
have to do analysis of the code to confirm no sections (that you are using) are 
unprotected.  I wouldn't want to perform that kind of analysis and would 
correctly implement SYS_ARCH_PROTECT


Appreciate for your advice!



--
View this message in context: 
http://lwip.100.n7.nabble.com/LwIP-multithread-select-mode-problems-tp26561p26584.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LwIP multithread select mode problems

2016-06-24 Thread Joel Cunningham


Yes you need SYS_ARCH_PROTECT in a multi-threaded environment.  Whether or not 
you use SYS_LIGHTWEIGHT_PROT just controls whether the function prototypes are 
produced and used in SYS_ARCH_PROTECT:



#define SYS_ARCH_DECL_PROTECT(lev) sys_prot_t lev
#define SYS_ARCH_PROTECT(lev) lev = sys_arch_protect()
#define SYS_ARCH_UNPROTECT(lev) sys_arch_unprotect(lev)


sys_prot_t sys_arch_protect(void);
void sys_arch_unprotect(sys_prot_t pval);


See the comments for how these should be implemented, you can use either 
interrupt disable/enable, semaphore or mutex



You can also define SYS_ARCH_PROTECT directly in sys_arch.h rather than going 
through a layer of sys_arch functions



Example:



sys_arch.h:

#define SYS_ARCH_DECL_PROTECT(lev) cpu_flags

#define SYS_ARCH_PROTECT(lev) level = disable_interrupts()

#define SYS_ARCH_UNPROTECT(lev) enable_interrupts(lev)



Joel


On Jun 23, 2016, at 09:33 PM, lampo  wrote:


thank you ! Joel

I still want to know if /SYS_ARCH_PROTECT/ is a must in multiple
threads environment?
I see a lot of /SYS_ARCH_PROTECT/ in select() and other sockets apis.

But when I set /SYS_LIGHATWEIGHT_PROT/ to 1 to use *SYS_ARCH_PROTECT*, the
applicaton run into hardfault or stackoverflow(due to RTOS thread scheduler,
or deadlock something ), I tried disabling interrupts and mutex both , in
implementing SYS_ARCH_PROTECT, both not ok.

I wonder if should use SYS_ARCH_PROTECT plus LWIP_NETCONN_SEM_PER_THREAD ?

Really appreciate!



--
View this message in context: 
http://lwip.100.n7.nabble.com/LwIP-multithread-select-mode-problems-tp26561p26573.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LwIP multithread select mode problems

2016-06-23 Thread Joel Cunningham


Hi,



select() can be used by multiple threads at the same time and you can even have 
the same sockets in multiple calls and it will be safe.  The limitation comes 
from trying to use the same sockets from multiple simultaneous threads in other 
socket APIs (select is the exception, simultaneous send, recv, or close on the 
same socket is not supported, especially in 1.4.1).



In regards to the maxfd question, I think there is a small misunderstanding.  
The maxfd parameter in select() is the largest FD in your input FD_SETs + 1.  
This is because maxfd is used to iterate through the FD_SETs provided.  It has 
nothing to do with other threads calling select() or how many sockets are in 
the system, but the maximum value of any FDs in the current select() call.



Joel


On Jun 23, 2016, at 03:59 AM, lampo  wrote:


hello,
can someone help me with multithread problem ?

we use lwip 1.4.1 in PLC products, and if it live through long time(1 month
for example) testing in industry use,
I will back here to report.

*our problem*
we use multithread select mode in our system, and it "seems" it's ok till
now(1 days passed),
but I see the doc in rawapi.txt,"/Netconn or Socket Api are not reentrant at
the control bolck granularity level/" .
so I really cannot figure out if im doing right thing.

*our application background/restriction*

a, our system is RTOS based , we need 2 TCP clients(separately connecting to
2 TCP servers, i.e., 2 lasers),
1 TCP server(deal with max of 4 remote clients,i.e.,4 modbus clients ), and
1 web server.

b, lwip related thread cannot be in polling mode, for here is already a
polling thread in RTOS and it has "soft real time" requirements.

c, current lwip version is 1.4.1(we are evaluating 2.0.0)

d, we need to send data to tcp servers periodically, 10 ms

*our design/implementation *

3 thread has used 'select' , but the same socket is only closed by the
thread who created it. here is details,

we use 'select' in "user tcp client send thread" ,we first call
'connect',and then call 'select' to determine whether the socket is
connected correctly or not; if some errors detected in connecting or data
sending, we call 'close' to close the socket.
we use 'select' in "user tcp client receive thread", blocked forever
waiting for data, and call 'recvfrom' if data received, if some errors
detected
in this thread,we don't just clost the socket ,but send message to "user tcp
client send thread" and tell it to close.
we use 'select' in "user tcp server thread",listening for incoming requests,
receiving and responding(sending) data.

*our doubt*
1、 ‘select’ can be used in multithread or not? if yes, the first param
maxfdp1 of lwip_select() must be set to the total number of sockets in each
thread?
for example,in our situation, maxfdp1 = 2(clients) + 4(remote clients) +
1(listen socket) in each select?

2、is SYS_ARCH_PROTECT must be used in multithread?

3、if lwip 1.4.1 does not support the multithread select mode, does lwip
2.0.0 supports it?

Thanks a lot !



--
View this message in context: 
http://lwip.100.n7.nabble.com/LwIP-multithread-select-mode-problems-tp26561.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] [lwip-devel] 2.0.0 Beta2?

2016-06-20 Thread Joel Cunningham

On the timers.c issue, I haven't ran into that one yet, but I have ran into 
source file name collisions with mem.c

My preferred solution is to fix it at the build system level.  Hopefully the 
project can use something smarter than a flattened global list of source files 
to compile. But if not, all-aboard the rename boat :(

I'd prefer not to add lwip_ to the file name because none of the other files 
are named that.  I like being consistent :)  Here's a couple:

timer.c

timr(s).c

sys_timer(s).c

combine with sys.c (which only contains sys_msleep)

Joel

On Jun 20, 2016, at 05:49 PM, Greg Smith  wrote:

Hi Simon & Sylvain.

Sent: Monday, 20 June 2016 04:27

> - work around the often found link-time issue of the fact that the file name
> 'timers.c' is used by FreeRTOS, too (anyone have a good idea?)

timeouts.c, timetriggers.c ?

Just a couple thoughts:
Easier: Rename to lwip_timers.c/.h

Harder: Or maybe refactor and put the TCP timers in tcp.c/.h (or new files 
called tcp_timers.c/.h) and put the others into sys.c/.h (or new files 
sys_timers.c/.h).
(I'm not intimately familiar with these files, so I don't know if this is a 
reasonable suggestion or not.)

> I'm open for votes on what else to include into 2.0.0 final, but I'd rather
> throw out 2.0.1 soon instead of delaying 2.0.0 further...

I started working on a PPP rework to allow user custom configuration (as
per Greg request), which is almost finished, the hardest part is done. A
couple of macros and removal of useless function arguments replaced with
config set and we should be done. If everything went I expected it
should be finished in a couple of days, is that ok for you ?

I'd prefer to get the PPP(oS) stuff in the 2.0.0 release, of course. However, I 
don't want to be the hold-up for the whole community and I can make do with 
what I have right now. So if I get outvoted, I won't hold it personally against 
anyone. :-)

-- Greg

This email has been scanned for email related threats and delivered safely by 
Mimecast.
For more information please visit http://www.mimecast.com
___
lwip-devel mailing list
lwip-de...@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-devel
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP keep alive info to the application

2016-06-02 Thread Joel Cunningham


Agreed, trying to leverage TCP keepalive information in the application level 
is incorrect.  The keepalives are used internally by TCP and will abort the 
connection when communication is lost.  This event is then typically 
communicated through socket APIs returning an error on next use.



You most likely need an application level heartbeat/keepalive. See this article 
for information about detecting half-open (stale/dead) connections: 
http://www.codeproject.com/Articles/37490/Detection-of-Half-Open-Dropped-TCP-IP-Socket-Conne



Joel


On May 30, 2016, at 07:49 AM, "Sergio R. Caprile"  wrote:


Playing the role of the pedantic teacher here, what you want to do
violates the allmighty layer independence of the OSI model. Before the
ISO police comes after you in the form of many unexpected problems, I
suggest you implement your own in-application keep-alive mechanism. Been
there.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Detect DHCP enabled for netif

2016-02-25 Thread Joel Cunningham

 ___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] LWIP raw TCP socket disconnection

2016-02-08 Thread Joel Cunningham

Here's a great article on everything you'd ever want to know about handling half-open TCP connections:http://www.codeproject.com/Articles/37490/Detection-of-Half-Open-Dropped-TCP-IP-Socket-ConneI make sure to keep this book marked :D JoelOn Feb 08, 2016, at 04:06 PM, Sylvain Rochet  wrote:Hi Frédéric,On Mon, Feb 08, 2016 at 12:52:43PM +, Frédéric Grandjean wrote:Hi,I'm using LWIP 1.4.1 and I'm facing an issue with "brutal" raw socket disconnection.Imagine the following scenario:- Create a TCP_PCB listener (server)- On connection from a remote host (a Windows PC), accept it, with all callbacks- Now, I remove the Ethernet cable from my application- I also exit (brutally) the remote client application- Now, I wait 1 minutes and connect back the Ethernet cable- LWIP never signals the connection is lost, not the state of the PCB tells me (Still ESTABLISHED)!How to detect this kind of disconnection ?TCP keepalive is your friend here (LWIP_TCP_KEEPALIVE).Other solutions usually chosen are server side timeout (e.g. HTTP) or an application level ping/pong protocol (e.g. IRC).Sylvain___lwip-users mailing listlwip-users@nongnu.orghttps://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc
Description: PGP signature
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] TCP_SNDBUF limit

2016-02-04 Thread Joel Cunningham

 ___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] The issue about gethostbyname() function

2016-01-12 Thread Joel Cunningham

 ___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip_close() doesn't work when lwip_write() hangs

2015-10-12 Thread Joel Cunningham


LwIP doesn't support this kind of threading model.  Multiple threads can not 
perform simultaneous operations (read+write, write+close, etc.) on the same 
socket.  The main limitation is that the netconn only has a single semaphore 
for blocking the calling thread when entering the core context.

On the master branch there is support for this model but the feature is in 
alpha state (see LWIP_NETCONN_FULLDUPLEX). In LwIP 1.4.1, this is not 
supported.

Joel

On Oct 10, 2015, at 12:29 AM, alhadpalkar  wrote:


I am using branch 1.4.1 of lwip. I have a thread that connects to a remote
server and writes data to it using lwip_write(). Sometimes this hangs
indefinitely. Looks like its waiting on the op->completed semaphore which
never gets signaled.

I tried using the SO_SNDTIMEO socket option, but that just causes panics in
my system. So I tried another approach where I use a watchdog that detects
this hang and calls lwip_close(). But it looks like LWIP doesn't like it. I
hit this assert

netconn_free(struct netconn *conn)
{
LWIP_ASSERT("PCB must be deallocated outside this function", conn->pcb.tcp
== NULL);
...
}

On debugging further it looks like we end up getting the ERR_INPROGRESS in
do_delconn().

do_delconn(struct api_msg_msg *msg)
{
/* @todo TCP: abort running write/connect? */
if ((msg->conn->state != NETCONN_NONE) &&
(msg->conn->state != NETCONN_LISTEN) &&
(msg->conn->state != NETCONN_CONNECT)) {
/* this only happens for TCP netconns */
LWIP_ASSERT("msg->conn->type == NETCONN_TCP", msg->conn->type ==
NETCONN_TCP);
km_printf("err in progress\n");
msg->err = ERR_INPROGRESS;
...
}

so we never end up cleanup up the pcbs which leads to this assert.

Is there a way around this?






--
View this message in context: 
http://lwip.100.n7.nabble.com/lwip-close-doesn-t-work-when-lwip-write-hangs-tp25191.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip_close() doesn't work when lwip_write() hangs

2015-10-12 Thread Joel Cunningham


You can use SO_SNDTIMEOUT, which should work on LwIP 1.4.1.  I have used it in 
my port with LwIP 1.4.1, so possibly there's a problem with your port?

I've also written applications that used non-blocking sockets and select to 
achieve a similar behavior of having blocking I/O that can be canceled.  The 
trick here is adding a second socket to the read FD set in select and then set 
select to block until your write or read is ready.  This second socket is bound 
to the loopback address.  When you want to cancel the blocking select from 
another thread, simply send a datagram to the additional socket, which will 
return the select call.  Then you can detect that a cancel/wakeup happened 
because the second socket is marked as ready.

Joel

On Oct 12, 2015, at 12:45 PM, Alhad Palkar <alhadpal...@gmail.com> wrote:


Thanks. Is there anyway around lwip_write blocking forever?

On Mon, Oct 12, 2015 at 7:44 AM, Joel Cunningham <joel.cunning...@me.com> wrote:

LwIP doesn't support this kind of threading model.  Multiple threads can 
not perform simultaneous operations (read+write, write+close, etc.) on the same 
socket.  The main limitation is that the netconn only has a single semaphore 
for blocking the calling thread when entering the core context.

On the master branch there is support for this model but the feature is in 
alpha state (see LWIP_NETCONN_FULLDUPLEX). In LwIP 1.4.1, this is not supported.

Joel

On Oct 10, 2015, at 12:29 AM, alhadpalkar <alhadpal...@gmail.com> wrote:


I am using branch 1.4.1 of lwip. I have a thread that connects to a remote
server and writes data to it using lwip_write(). Sometimes this hangs
indefinitely. Looks like its waiting on the op->completed semaphore which
never gets signaled.

I tried using the SO_SNDTIMEO socket option, but that just causes panics in
my system. So I tried another approach where I use a watchdog that detects
this hang and calls lwip_close(). But it looks like LWIP doesn't like it. I
hit this assert

netconn_free(struct netconn *conn)
{
LWIP_ASSERT("PCB must be deallocated outside this function", conn->pcb.tcp
== NULL);
...
}

On debugging further it looks like we end up getting the ERR_INPROGRESS in
do_delconn().

do_delconn(struct api_msg_msg *msg)
{
/* @todo TCP: abort running write/connect? */
if ((msg->conn->state != NETCONN_NONE) &&
(msg->conn->state != NETCONN_LISTEN) &&
(msg->conn->state != NETCONN_CONNECT)) {
/* this only happens for TCP netconns */
LWIP_ASSERT("msg->conn->type == NETCONN_TCP", msg->conn->type ==
NETCONN_TCP);
km_printf("err in progress\n");
msg->err = ERR_INPROGRESS;
...
}

so we never end up cleanup up the pcbs which leads to this assert.

Is there a way around this?






--
View this message in context: 
http://lwip.100.n7.nabble.com/lwip-close-doesn-t-work-when-lwip-write-hangs-tp25191.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip_close() doesn't work when lwip_write() hangs

2015-10-12 Thread Joel Cunningham

It's a great trick, hopefully others can leverage it as well :)I'm not sure what I'd do without it.  Having select() and non-blocking sockets operate as the blocking construct of a server's event loop is essential for managing multiple connections in a high performance manner.Joel  On Oct 12, 2015, at 02:19 PM, Sylvain Rochet <grada...@gradator.net> wrote:Hi Joel,On Mon, Oct 12, 2015 at 07:10:39PM +, Joel Cunningham wrote:You can use SO_SNDTIMEOUT, which should work on LwIP 1.4.1. I have used it in my port with LwIP 1.4.1, so possibly there's a problem with your port?I've also written applications that used non-blocking sockets andselect to achieve a similar behavior of having blocking I/O that canbe canceled. The trick here is adding a second socket to the read FDset in select and then set select to block until your write or read isready. This second socket is bound to the loopback address. When youwant to cancel the blocking select from another thread, simply send adatagram to the additional socket, which will return the select call.Then you can detect that a cancel/wakeup happened because the secondsocket is marked as ready.I really like this trick. It remembers myself of the well known wake up pipe I explained here[1], but using the loopback to do so in lwIP is very very clever :-)Sylvain[1] http://lists.gnu.org/archive/html/lwip-devel/2015-09/msg00028.html___lwip-users mailing listlwip-users@nongnu.orghttps://lists.nongnu.org/mailman/listinfo/lwip-users

signature.asc
Description: PGP signature
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip full duplex?

2015-08-11 Thread Joel Cunningham


LwIP has not had supported for full duplex sockets in any release versions.  On 
the master branch, there is some initial support under the flag 
LWIP_NETCONN_FULLDUPLEX, but I think the feature is pretty early in 
development.  Here is the comment from the opt.h, noting the alpha state of 
this:

/** LWIP_NETCONN_FULLDUPLEX==1: Enable code that allows reading from one thread,
* writing from a 2nd thread and closing from a 3rd thread at the same time.
* ATTENTION: This is currently really alpha! Some requirements:
* - LWIP_NETCONN_SEM_PER_THREAD==1 is required to use one socket/netconn from
*   multiple threads at once
* - sys_mbox_free() has to unblock receive tasks waiting on recvmbox/acceptmbox
*   and prevent a task pending on this during/after deletion
*/
#ifndef LWIP_NETCONN_FULLDUPLEX
#define LWIP_NETCONN_FULLDUPLEX 0
#endif

Joel

On Aug 10, 2015, at 05:08 PM, Michael Steinberg m...@tu-clausthal.de wrote:


Hello,

you make it sound like this was a limitation of lwip... but if at all,
it would only be one of the berkeley socket API emulation layer...
That being said, the socket API uses the netconn API, which in turn uses
mailboxes for receiving packets from the lwip core/driver. I cannot see
any additional locking operation on the path from recv to
sys_mailbox_fetch, so I don't think a send would block during a
receive... Socket state is touched though, one would have to see, if
there's conflicting state overlap on the send and receive paths...

In constrained environments, I would argue that berkeley socket API is
not the weapon of choice anyways (actually I would argue that for any
environment, hehe)

Kind Regards,
Michael


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip full duplex?

2015-08-11 Thread Joel Cunningham


Michael,

Historically there hasn't been support for multiple threads to issue operations 
within the LwIP core context at the same time for the same netconn.  Each 
netconn only has a single op_completed semaphore.  This prevents a simultaneous 
read and write from occurring.  This limitation also prevents one thread 
blocking on send/recv and another thread issuing a close.

The new option LWIP_NETCONN_SEM_PER_THREAD enables each thread operating on the 
netconn to have its own semaphore

Joel

On Aug 11, 2015, at 09:54 AM, Michael Steinberg m...@tu-clausthal.de wrote:


Hi Joel,
well I had not anticipated that it's common to have three threads on a socket 
running, each doing their own task on it. The problems stem from the third 
deleting thread, right? Reading + Writing seem to be unproblematic as it stands?

Kind Regards,
Michael


Am 11.08.2015 um 16:38 schrieb Joel Cunningham:

LwIP has not had supported for full duplex sockets in any release versions.  On 
the master branch, there is some initial support under the flag 
LWIP_NETCONN_FULLDUPLEX, but I think the feature is pretty early in 
development.  Here is the comment from the opt.h, noting the alpha state of 
this:

/** LWIP_NETCONN_FULLDUPLEX==1: Enable code that allows reading from one thread,
 * writing from a 2nd thread and closing from a 3rd thread at the same time.
 * ATTENTION: This is currently really alpha! Some requirements:
 * - LWIP_NETCONN_SEM_PER_THREAD==1 is required to use one socket/netconn from
 *   multiple threads at once
 * - sys_mbox_free() has to unblock receive tasks waiting on recvmbox/acceptmbox
 *   and prevent a task pending on this during/after deletion
 */
#ifndef LWIP_NETCONN_FULLDUPLEX
#define LWIP_NETCONN_FULLDUPLEX 0
#endif

Joel

On Aug 10, 2015, at 05:08 PM, Michael Steinberg m...@tu-clausthal.de wrote:


Hello,

you make it sound like this was a limitation of lwip... but if at all,
it would only be one of the berkeley socket API emulation layer...
That being said, the socket API uses the netconn API, which in turn uses
mailboxes for receiving packets from the lwip core/driver. I cannot see
any additional locking operation on the path from recv to
sys_mailbox_fetch, so I don't think a send would block during a
receive... Socket state is touched though, one would have to see, if
there's conflicting state overlap on the send and receive paths...

In constrained environments, I would argue that berkeley socket API is
not the weapon of choice anyways (actually I would argue that for any
environment, hehe)

Kind Regards,
Michael


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
 


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] lwip full duplex?

2015-08-11 Thread Joel Cunningham


If you look further down the call past sys_arch_mbox_fetch() in 
netconn_recv_data(), you can see for the TCP case it calls into the core to 
execute do_recv() with TCPIP_APIMSG().  This uses the op_completed semaphore as 
I mentioned.

It just happens to be that UDP doesn't need to do this additional function 
call that may allow a UDP recv() and UDP send() on the same netconn/socket to 
work, but the calls weren't designed to support this use case.

Joel

On Aug 11, 2015, at 11:14 AM, Michael Steinberg m...@tu-clausthal.de wrote:


Hi Joel,

though I did see that semaphore in the structure definition, I could not see a 
reference to it on the path from recv to sys_arch_mailbox_fetch... I'll dig 
deeper... Probably it's eventually used in tcp-ip-thread context? ChibiOS 
allows to close a mailbox while somebody is waiting on it, notifying the 
waiter. Is that generally possible on other platforms?

What I'm currently also thinking about... is it really the most elegant 
solution to put all the guards into the connection stuff, instead of making the 
lwip-core itself threadsafe (conditionally)? For now I can't really compare the 
both approaches... I'll dig deeper once again...
Would it prove helpful to bring all the touched shared resources into a 
diagram? I could create such a diagram while digging... Perhaps it makes reasoning 
easier...

Kind Regards,
Michael


Am 11.08.2015 um 17:09 schrieb Joel Cunningham:

Michael,

Historically there hasn't been support for multiple threads to issue operations 
within the LwIP core context at the same time for the same netconn.  Each 
netconn only has a single op_completed semaphore.  This prevents a simultaneous 
read and write from occurring.  This limitation also prevents one thread 
blocking on send/recv and another thread issuing a close.

The new option LWIP_NETCONN_SEM_PER_THREAD enables each thread operating on the 
netconn to have its own semaphore

Joel

On Aug 11, 2015, at 09:54 AM, Michael Steinberg m...@tu-clausthal.de wrote:


Hi Joel,
well I had not anticipated that it's common to have three threads on a socket 
running, each doing their own task on it. The problems stem from the third 
deleting thread, right? Reading + Writing seem to be unproblematic as it stands?

Kind Regards,
Michael


Am 11.08.2015 um 16:38 schrieb Joel Cunningham:

LwIP has not had supported for full duplex sockets in any release versions.  On 
the master branch, there is some initial support under the flag 
LWIP_NETCONN_FULLDUPLEX, but I think the feature is pretty early in 
development.  Here is the comment from the opt.h, noting the alpha state of 
this:

/** LWIP_NETCONN_FULLDUPLEX==1: Enable code that allows reading from one thread,
 * writing from a 2nd thread and closing from a 3rd thread at the same time.
 * ATTENTION: This is currently really alpha! Some requirements:
 * - LWIP_NETCONN_SEM_PER_THREAD==1 is required to use one socket/netconn from
 *   multiple threads at once
 * - sys_mbox_free() has to unblock receive tasks waiting on recvmbox/acceptmbox
 *   and prevent a task pending on this during/after deletion
 */
#ifndef LWIP_NETCONN_FULLDUPLEX
#define LWIP_NETCONN_FULLDUPLEX 0
#endif

Joel

On Aug 10, 2015, at 05:08 PM, Michael Steinberg m...@tu-clausthal.de wrote:


Hello,

you make it sound like this was a limitation of lwip... but if at all,
it would only be one of the berkeley socket API emulation layer...
That being said, the socket API uses the netconn API, which in turn uses
mailboxes for receiving packets from the lwip core/driver. I cannot see
any additional locking operation on the path from recv to
sys_mailbox_fetch, so I don't think a send would block during a
receive... Socket state is touched though, one would have to see, if
there's conflicting state overlap on the send and receive paths...

In constrained environments, I would argue that berkeley socket API is
not the weapon of choice anyways (actually I would argue that for any
environment, hehe)

Kind Regards,
Michael


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
 
 


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
 


___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] SO_REUSEPORT is supported in LWIP 1.4.1

2015-07-01 Thread Joel Cunningham


I'm using it on a project with LwIP 1.4.1.  You'll need to enable it with 
#define SO_REUSE 1 in your options header.  The implementation is in sockets.c 
and calls ip_set_option()/ip_get_option()

Joel

On Jul 01, 2015, at 01:39 AM, jkp jithin...@in.abb.com wrote:


Hi,

I'm using LWIP Version 1.4.1 as shown below and target ARM Cortex 4 proc.
I'm using the socket option
SO_REUSEPORT for multiple threads binding to same port and it seems like I
can't find the implementation for the same.

I used lwip_setsockopt() and it did returned me , not supported protocol.
But, in the lwip form I came to know that its already part of the LWIP stack
for quite long time. Is that I'm missing something here ?

/** X.x.x: Major version of the stack */
#define LWIP_VERSION_MAJOR 1U
/** x.X.x: Minor version of the stack */
#define LWIP_VERSION_MINOR 4U
/** x.x.X: Revision of the stack */
#define LWIP_VERSION_REVISION 1U




--
View this message in context: 
http://lwip.100.n7.nabble.com/SO-REUSEPORT-is-supported-in-LWIP-1-4-1-tp24649.html
Sent from the lwip-users mailing list archive at Nabble.com.

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Sending to a non-local network without default netif set

2014-11-18 Thread Joel Cunningham


we do have a setup with two netifs and we cannot determine what the customer will 
do with the device so we need to also check for „same subnet“ scenarios.

I would like to make a clear distinction between two possible same subnet 
scenarios

1) A multi-homed product is connected to the same logical network via two (or 
more) different links.  One example is a laptop connected to the same LAN via a 
wired Ethernet connection and a wireless connection.  This is a valid 
configuration and both links will have the same subnet of the LAN. Stacks like 
Windows support this and assign an interface metric to the out going 
interfaces.  The metric is used in picking an outgoing interface.

2) A multi-homed product is connected to two (or more) separate physical 
networks that have the same subnet.  The important differences is that the 
links are to different separate physical networks.  Since the subnets are the 
same, different networks can no longer be identified at the IP address 
abstraction level.  This can be considered an invalid network configuration 
since different IP networks are identified by their subnet/address range.

I'm all in favor of supporting 1) in LwIP as long as the implementation meets the 
light-weight requirements, but I'm not a fan of supporting 2).  This case is a 
huge pain because it permits invalid network configurations.  Not to mention, from the 
application layer you have to introduce some other identifier for addressing the different 
networks (can't use subnet). You end up having to forcing sockets onto the appropriate 
network with SO_BINDTODEVICE.  If we want LwIP to detect a case like 2) that would be fine, 
but then how to do you tell the difference between 1  2?

Joel

On Nov 17, 2014, at 06:34 AM, Fabian Koch fabian.k...@de.abb.com wrote:


Hey Simon, Hey Erik,

 


we do have a setup with two netifs and we cannot determine what the customer 
will do with the device so we need to also check for „same subnet“ scenarios.


I already wrote and email to this (or devel?) list about that a while ago.

 


We also attacked the problem by including the src IP into the ip_route() 
function and I can provide a patch, but it just doesn’t feel right.

 


The reason this is needed is actually because LwIP uses ip_route() to find a 
netif when the local address is INADDR_ANY.

 


In udp_connect(), udp_sendto(), tcp_eff_send_mss(), snmp_send_trap() and 
etharp_add_static_entry().

 


In all those cases, ip_route() is actually not really a good function to find a 
matching netif. If we were to replace those instances with another function 
that is a bit more complicated and finds a fitting netif with more aspects 
including gateway settings and network reachability (if that’s even a word…) 
then it would make the final routing a bit easier.

 


Also, the default_netif construct is just too simple to make all this work 
correctly.

 


If you need a quick solution, you can make ip_route include the src, but then 
you also have to add that parameter to tcp_eff_send_mss().

 


Our middle part of ip_route() looks like this at the moment:

 


  /* iterate through netifs */

  for (netif = netif_list; netif != NULL ; netif = netif- next ) {

if ( netif_is_up (netif)  netif_is_link_up (netif)) {   //only 
consider interfaces which are up and have a link

  if ( ip_addr_isany (src)) {//when the source IP is INADDR_ANY, the 
socket is not bound to an interface = find the first match (netmask or is 
broadcast)

 if ( ip_addr_netcmp (dest, (netif-ip_addr), (netif- netmask )) || 
ip_addr_cmp ( IP_ADDR_BROADCAST , dest)) {

  /* return netif on which to forward IP packet (first matching netif 
when socket is not bound) */

  return netif;

}

  } else { // socket is bound to a specific IP so only match the right 
netif (matching IPs and either fits netmask or is broadcast)

if ( ip_addr_cmp ((netif- ip_addr ), src)  ( ip_addr_netcmp (dest, 
(netif-ip_addr), (netif- netmask )) || ip_addr_cmp ( IP_ADDR_BROADCAST , dest))) {

  /* this socket is bound to a specific interface, so look for that */

  return netif;

}

  }

}

  }

 


This essentially makes sending on specific interfaces possible when they are 
bound to the IP. Unbound sockets just send on the best matching netif they can 
find.

 


For this to work, you also need to netif_set_link_up(loop_netif) in netif.c if 
you’re using a loopback netif. (I wonder if it can be considered a bug that the 
loopif never gets a link up…)

 


I would still not like this in the main lwIP source tree, since it just doesn’t 
feel right to include the src IP here.

 

 


Cheers,

Fabian

 


From: lwip-users-bounces+fabian.koch=de.abb@nongnu.org 
[mailto:lwip-users-bounces+fabian.koch=de.abb@nongnu.org] On Behalf Of 
goldsi...@gmx.de
Sent: Freitag, 14. November 2014 21:29
To: Mailing list for lwIP users
Subject: Re: [lwip-users] Sending to a

[lwip-users] event_callback() context switch when calling sys_sem_signal()

2014-09-19 Thread Joel Cunningham


I'm running LwIP 1.4.1 and have some questions about the event_callback() in 
sockets.c

In my project, I am experiencing a crash related to synchronization in 
event_callback() and an application thread calling select().  My project is a 
uniprocessor system running an RTOS that implements a static priority 
scheduler.  SYS_ARCH_PROTECT() is implemented by disabling interrupts.  
sys_sem_signal() is implemented using a counting semaphore.  TCPIP thread is 
higher priority than application threads.

The crash happens when the application thread is waiting in select() and the 
TCPIP thread is calling event_callback() to process an event.  What's happen is 
in the below loop, calling sys_sem_signal() results in a context switch on my 
project's RTOS even though application thread is lower priority.  The RTOS's 
semaphore construct doesn't support priority inheritance/elevation.  The 
application thread wakes up and finishes the select call, modifying the 
select_cb_list.  When the context switches back to TCPIP thread, it finishes 
the loop iteration and crashes because the select_cb_list has been modified.

What I've done to mitigate the context switch is move the line 
last_select_cb_ctr = select_cb_ctr; to the top of the for loop.  To me the loop 
already had handling for a context switch per iteration, but it only saved the 
counter at the end. So now it can handle a switch in the call to 
sys_sem_signal() as well.

My questions to whether this is a bug depend on:

1) Should SYS_ARCH_PROTECT() do more than just disable interrupts?  Something 
that would act as a critical section in the case where a context switch happens?
2) Is it assumed that calling sys_sem_signal() will not cause a voluntary 
context switch?

SYS_ARCH_PROTECT(lev);
...
again:
  for (scb = select_cb_list; scb != NULL; scb = scb-next) {
    if (scb-sem_signalled == 0) {
      /* semaphore not signalled yet */
      int do_signal = 0;
      /* Test this select call for our socket */
      if (sock-rcvevent  0) {
        if (scb-readset  FD_ISSET(s, scb-readset)) {
          do_signal = 1;
        }
      }
      if (sock-sendevent != 0) {
        if (!do_signal  scb-writeset  FD_ISSET(s, scb-writeset)) {
          do_signal = 1;
        }
      }
      if (sock-errevent != 0) {
        if (!do_signal  scb-exceptset  FD_ISSET(s, scb-exceptset)) {
          do_signal = 1;
        }
      }
      if (do_signal) {
        scb-sem_signalled = 1;
        /* Don't call SYS_ARCH_UNPROTECT() before signaling the semaphore, as 
this might
           lead to the select thread taking itself off the list, invalidagin 
the semaphore. */
        sys_sem_signal(scb-sem);
      }
    }
    last_select_cb_ctr = select_cb_ctr;
    /* unlock interrupts with each step */
    SYS_ARCH_UNPROTECT(lev);
    /* this makes sure interrupt protection time is short */
    SYS_ARCH_PROTECT(lev);
    if (last_select_cb_ctr != select_cb_ctr) {
      /* someone has changed select_cb_list, restart at the beginning */
      goto again;
    }

Thanks,

Joel

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Re: [lwip-users] Multi-threaded socket access once again

2014-07-21 Thread Joel Cunningham


Fabian,

You're definitely correct.  I just thought wanted to remind people of the 
behavior/issue that are using lwIP and sharing sockets across multiple threads.

Joel

On Jul 21, 2014, at 03:57 AM, Fabian Koch fabian.k...@de.abb.com wrote:

Hey Joel,
 
well that’s not LwIP specific. The usual BSD socket API works in the same way.
If you want to design your application in that way, you need some 
application-wide resource management that can take care of that.
 
Kind regards,
Fabian
 
From: lwip-users-bounces+fabian.koch=de.abb@nongnu.org 
[mailto:lwip-users-bounces+fabian.koch=de.abb@nongnu.org] On Behalf Of Joel 
Cunningham
Sent: Freitag, 18. Juli 2014 19:21
To: Mailing list for lwIP users
Subject: Re: [lwip-users] Multi-threaded socket access once again
 
Another problem beside the lwIP API not being designed for multiple thread 
access on the same socket, is an inability to detect stale socket (file) 
handles.
 
Say you have an application 1 with two thread A for sending, B for receiving.  Both 
threads share a socket allocated as 1 and hold references to that socket.  
During the execution of thread B, it closes socket 1.  Thread A still has a reference to 
socket 1.
 
If an application 2, thread C executes next (before thread A) and opens a 
socket, the lwIP stack may reuse socket 1 because it is now free and hands out 
a reference to application 2.  Now when thread A gets around to running, any 
operations that it performs will be using application 2's socket
 
Joel

On Jul 18, 2014, at 10:12 AM, Steffen Wolfer wol...@weiss-robotics.de wrote:

Hi Fabian,

thanks for your reply. The submitting threads should always be stopped
before the application closes the socket. The receiver thread does all
the socket management, like open, close, bind, accept etc. and also
controls the other submitting threads. But I'm not sure what happens if
the remote site closes the connection unexpectedly, so I'll have to
check that.

Regards,
Steffen

Am 18.07.2014 14:55, schrieb Fabian Koch:
        Hey Steffen,
        
        insert the usual not supported answer here first  
        
        Having said that, it seems like an okay setup if there are no other unexpected accesses.

        For example if another task closes the socket while your receive task 
is in the select() you'll also be in trouble.
        For our adaptation we modified LwIP to silently exit the select() call 
if that happens but in the official release, is runs into assertions.
        
        The whole ordeal of access sockets from multiple tasks is a big topic in my opinion and not something that LwIP can hide from.

        I know the construct of the single op_completed semaphore and the 
mboxes is one of the main problems behind not being able to support multiple 
tasks, but the argument of no longer being light-weight is not true if we make all 
the necessary changes optional and flexible.
        
        Cheers,

        Fabian
        
        ___

        lwip-users mailing list
        lwip-users@nongnu.org
        https://lists.nongnu.org/mailman/listinfo/lwip-users
        
        



___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users___
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

64 matches

Mail list logo