Wow Eero! Thanks for taking the time to determine what the problem was and report it back to the group.
So in summary (to make sure I understand this correctly) 1) If you send data too rapidly for the device to download from the GPRS (exceed it's bandwidth), then all data will queue up there instead of it trickling down until caught up. If the remote end sends any message the GPRS send queue is unblocked and data is sent to the remote device. Is this correct? 2) If you have a delay of more than 50 seconds between messages sent to the remote device they will queue up at the GPRS until the remote end sends a message and the GPRS send queue is unblocked. Is this correct? Which cellular network were your tests done on? Cingular/AT&T, Sprint/Nextel, Verizon, T-Mobile, something else? Do you have the ability to repeat these tests with another cellular network? It would be interesting to see if these numbers and particular problem exist only with one network or if it is a common one. In addition to the tests you've already run, could you do a few more? Some thing's I'd be interested in knowing is: In scenario #1 above (assuming what I've stated is correct), If you exceed the remote devices bandwith and the queue blocks, but you continue to send data what happens? Will the block eventually be cleared once some threshold of backed up data is reached? What happens if you continue to send data at a high rate, and the remote end sends a message unblocking the queue. Will the GPRS send queue ever block again, (assume continuous stream of new data) or will it simply stream properly from that point on? In scenario #2, does the size of the message or the cumulative size of queued data make any difference? I.E. Normally ping messages are small. If you sent a ping message of approximately 1500 bytes (or larger) would the ping message go through even after a 50 second delay? What if you sent several small ping messages in rapid succession after the 50 second delay? (So the backlog adds up to at least 1500 bytes, perhaps even more.) I'm wondering if there is something Naggle-ish at work in the GPRS. Thanks for all your help!! Rob ----- Original Message ---- From: Eero Nevalainen <[EMAIL PROTECTED]> To: dev@mina.apache.org Cc: Rob Butler <[EMAIL PROTECTED]> Sent: Monday, May 28, 2007 4:05:53 AM Subject: Re: Session close problem I've now finished my tests which revealed the problem. Took a while because I had to redo the last test due to lack of materials..:) A description of the problem scenario is on my earlier message included further below. Tests, their results and conclusions (immediately) below: 1. Test - Replicate the problem I simulated the gateway by listening with netcat. At first I sent 'PING' messages rapidly and the connection worked fine. Then I waited for three minutes and tried again. The connection wasn't working. - The problem can be replicated with other tools and is related to the activity of communication. 2. Test - Measure delay threshold On the second setup I wrote a fake GW which sends PINGs with increasing delays: 5s, 6s, 7s, ... At a certain threshold the remote device no longer replied PONG. Before the threshold all PINGs received a response. On three measurements the thresholds were: 48s 49s 48s 3. Test - Figure characteristic behaviour This time I used a remote device on our lab, which had an ethernet connection in addition to the GPRS. I used netcat on both ends to send chosen traffic activity over the GPRS. The following behavioural rules were obtained: - messages from the remote device always arrive properly - if the gateway sends messages thicker than the ~50s threshold, the messages are received and continue to be received normally. - if the gateway exceeds the threshold in it's communication, the messages are queued. A message sent by the remote device flushes the queue. 4. Test - Test radio blackout scenario This test was like the 3rd test, except I yanked out the GPRS antenna. I had to redo this one because the tiny antenna connector was actually providing enough of antenna for GPRS to work! Wrapping the device in aluminum foil proved effective though :) The following was observed: - if the remote device has lost it's connection for more than 50s the connection will become 'locked' even if the gateway sends messages during that time. A message from the remote device flushes the queue as usual. A coworker has heard this is a known problem with the Gateway GPRS Support Node (GGSN). It will probably be fixed at some point, but currently the only safe workaround is to send messages from the remote end. Hope this is useful. -Eero Nevalainen Rob Butler wrote: > Eero, > > I came across this (http://www.cl.cam.ac.uk/~rc277/globe02.pdf). > > It would appear that a cellular device could have issue with receiving data > because it temporarily goes out of range or has some other issue which causes > packets to "back up" at the GPRS device. From the above document it looks > like it could take up to a minute before that backlog is cleared up. Have > you tried waiting around for a while to see if the ping packet eventually > makes it to the device? If it does make it "eventually" you could decrease > your ping send interval while simultaneously increasing the time you wait for > a ping response. Unfortunately this means your connection could go down for > a little while before you notice it, but at least you'd have something that > lets you know the connection is bad eventually. > > Another consideration may be the infamous naggle algorithm. You can of > course turn it off on your server application and on your client as well. > Unfortunately you have no control on whether the GPRS is using naggle for > it's connection to the cellular device. Have you tried sending a lot of ping > packets? This is of course wasteful, but if you send approximately 1500 > bytes worth of ping data and miraculously they all arrive at the device at > once naggle, or something equally nefarious, is probably the cause. > > Be aware that 1500 bytes or so is the usual Maximum Transmission Unit on most > LAN's by default, but this can be changed easily and may very likely be > different on the cellular network. I'd try sending multiple ping packets > slowly (like 1 every 30 seconds to one minute) and see how many you have sent > before the first one arrives. When the first one arrives you'll probably get > a bunch of them together at once. Try to count how many arrive at once (or > in very rapid succession) and then try sending that many ping packets + 1 > more all at once from your server. If your rapid fire of multiple pings > works more reliably one solution may simply be to send artifically large ping > packets. By fine tuning the packet size of your ping message you may be able > to more reliably circumvent the "grouping" of naggle and similar approaches. > Whatever you find is the "sweet spot" for packet size you may want to > increase it slightly so on other networks with potentially larger MTU sizes > your > application still works. Again, this is all wasteful from a bandwidth > perspective - especially when end-users are charged per MB of data transfer > on some cell networks, but at least it would be a working solution. > > Please report your findings back to the list, or at least to me. I'd be very > interested in the results. > > Thanks! > Rob > > ----- Original Message ---- > From: Eero Nevalainen <[EMAIL PROTECTED]> > To: dev@mina.apache.org > Sent: Friday, May 11, 2007 7:30:32 AM > Subject: Re: Session close problem > > Hi all, > > I've recently deployed a gateway server to mediate communication between > clients and remote devices. The gateway works fine in a 'regular' > network, but now we're experiencing problems when the remote devices are > behind a GPRS network. > > The gateway sends an application level (TCP) 'PING' message periodically > to the remote devices. For some reason the PING messages seem to queue > up in the GPRS connection, without the devices receiving them. > > Meanwhile, sending an ICMP request to the same devices gives a much > smaller roundtrip time (~700ms) and no message losses. Also the messages > sent by the remote devices to the gateway always arrive properly. > > I can't reproduce this problem with an ethernet connection so any help > with the GPRS network would be most welcome! > > -Eero Nevalainen > > Dawie Malan wrote: >> You're quite right - there is something in the middle, and it does cause the >> symptoms you are reporting. >> >> A GPRS network has two essential elements: >> >> *Serving GPRS Support Node (SGSN)-Sends data to and receives data from mobile >> stations, and maintains information about the location of a mobile station >> (MS). >> The SGSN communicates between the MS and the GGSN. >> >> *Gateway GPRS Support Node (GGSN)-A wireless gateway that allows mobile cell >> phone users to access the public data network (PDN) or specified private IP >> networks. >> >> The GGSN acts like a proxy and is the device that sends the ACKs to the phone >> even though the session is already closed on the server side. >> >> I hope this answers your question. >> >> Regards, >> >> Dawie Malan >> >> >> -----Original Message----- >> From: Michael Bauroth [mailto:[EMAIL PROTECTED] >> Sent: 22 March 2007 04:40 PM >> To: dev@mina.apache.org >> Subject: Session close problem >> >> Hi, >> >> I'm using currently the trunk version of Mina. I use it to communicate >> with GPRS devices on the other side. In most cases this works very well! >> >> But sometimes the things become strange. I've collected a bunch of >> problems where I would need help / advices / ideas. >> >> 1. I call session.close() from inside my server. It seems, that the >> session doesn't close as required. >> >> 2. I close the session. The GPRS device on the other side continues to >> send data and seems to get ACKs too. It doesn't recognize, that the >> other side of the connection (the server) has closed the session a while >> before. It seems that there is something in the middle which responds >> the ACKs even with not opened server session. >> >> Knows anybody one of the two (most painful) problems and has an answer >> for me? >> >> Best Regards >> Michael >> >> >> > > > > > > > > ____________________________________________________________________________________ > Never miss an email again! > Yahoo! Toolbar alerts you the instant new Mail arrives. > http://tools.search.yahoo.com/toolbar/features/mail/ > ____________________________________________________________________________________Get the free Yahoo! toolbar and rest assured with the added security of spyware protection. http://new.toolbar.yahoo.com/toolbar/features/norton/index.php
