Response inline.
Rob Butler wrote:
Wow Eero!
Thanks for taking the time to determine what the problem was and
> report it back to the group.
So in summary (to make sure I understand this correctly)
1) If you send data too rapidly for the device to download from the
GPRS (exceed it's bandwidth), then all data will queue up there
> instead of it trickling down until caught up. If the remote end
> sends any message the GPRS send queue is unblocked and data is
> sent to the remote device. Is this correct?
Seems I should have stated my tests a bit more clearly.
I have NOT tested exceeding the bandwidth. I've only tested how the
network behaves when the connection is idle for a long time. All the
messages I sent were very small. For some reason our network "blocks"
one-directionally, when there's no communication.
2) If you have a delay of more than 50 seconds between messages sent to the
remote device they will queue up at the GPRS until the remote end sends a
message and the GPRS send queue is unblocked. Is this correct?
Correct.
Which cellular network were your tests done on? Cingular/AT&T, Sprint/Nextel,
Verizon, T-Mobile, something else?
Do you have the ability to repeat these tests with another cellular network?
It would be interesting to see if these numbers and particular problem exist
only with one network or if it is a common one.
In addition to the tests you've already run, could you do a few more? Some
thing's I'd be interested in knowing is:
<clip>
I don't want to name our network provider because we don't want to look
like we are blaming them(We're actually quite happy with them). We also
don't have SIMs from other providers so I can't repeat the tests on
another network.
As for the suggested additional tests, I'm afraid I have to direct my
time to more urgent matters. Maybe someone else can pickup from here.
-Eero Nevalainen
----- Original Message ----
From: Eero Nevalainen <[EMAIL PROTECTED]>
To: dev@mina.apache.org
Cc: Rob Butler <[EMAIL PROTECTED]>
Sent: Monday, May 28, 2007 4:05:53 AM
Subject: Re: Session close problem
I've now finished my tests which revealed the problem. Took a while
because I had to redo the last test due to lack of materials..:)
A description of the problem scenario is on my earlier message included
further below.
Tests, their results and conclusions (immediately) below:
1. Test - Replicate the problem
I simulated the gateway by listening with netcat. At first I sent 'PING'
messages rapidly and the connection worked fine. Then I waited for three
minutes and tried again. The connection wasn't working.
- The problem can be replicated with other tools and is related to the
activity of communication.
2. Test - Measure delay threshold
On the second setup I wrote a fake GW which sends PINGs with increasing
delays: 5s, 6s, 7s, ...
At a certain threshold the remote device no longer replied PONG. Before
the threshold all PINGs received a response. On three measurements the
thresholds were:
48s
49s
48s
3. Test - Figure characteristic behaviour
This time I used a remote device on our lab, which had an ethernet
connection in addition to the GPRS. I used netcat on both ends to send
chosen traffic activity over the GPRS. The following behavioural rules
were obtained:
- messages from the remote device always arrive properly
- if the gateway sends messages thicker than the ~50s threshold, the
messages are received and continue to be received normally.
- if the gateway exceeds the threshold in it's communication, the
messages are queued. A message sent by the remote device flushes the queue.
4. Test - Test radio blackout scenario
This test was like the 3rd test, except I yanked out the GPRS antenna. I
had to redo this one because the tiny antenna connector was actually
providing enough of antenna for GPRS to work! Wrapping the device in
aluminum foil proved effective though :) The following was observed:
- if the remote device has lost it's connection for more than 50s the
connection will become 'locked' even if the gateway sends messages
during that time. A message from the remote device flushes the queue as
usual.
A coworker has heard this is a known problem with the Gateway GPRS
Support Node (GGSN). It will probably be fixed at some point, but
currently the only safe workaround is to send messages from the remote end.
Hope this is useful.
-Eero Nevalainen
Rob Butler wrote:
Eero,
I came across this (http://www.cl.cam.ac.uk/~rc277/globe02.pdf).
It would appear that a cellular device could have issue with receiving data because it temporarily
goes out of range or has some other issue which causes packets to "back up" at the GPRS
device. From the above document it looks like it could take up to a minute before that backlog is
cleared up. Have you tried waiting around for a while to see if the ping packet eventually makes
it to the device? If it does make it "eventually" you could decrease your ping send
interval while simultaneously increasing the time you wait for a ping response. Unfortunately this
means your connection could go down for a little while before you notice it, but at least you'd
have something that lets you know the connection is bad eventually.
Another consideration may be the infamous naggle algorithm. You can of course
turn it off on your server application and on your client as well.
Unfortunately you have no control on whether the GPRS is using naggle for it's
connection to the cellular device. Have you tried sending a lot of ping
packets? This is of course wasteful, but if you send approximately 1500 bytes
worth of ping data and miraculously they all arrive at the device at once
naggle, or something equally nefarious, is probably the cause.
Be aware that 1500 bytes or so is the usual Maximum Transmission Unit on most LAN's by default, but
this can be changed easily and may very likely be different on the cellular network. I'd try
sending multiple ping packets slowly (like 1 every 30 seconds to one minute) and see how many you
have sent before the first one arrives. When the first one arrives you'll probably get a bunch of
them together at once. Try to count how many arrive at once (or in very rapid succession) and then
try sending that many ping packets + 1 more all at once from your server. If your rapid fire of
multiple pings works more reliably one solution may simply be to send artifically large ping
packets. By fine tuning the packet size of your ping message you may be able to more reliably
circumvent the "grouping" of naggle and similar approaches. Whatever you find is the
"sweet spot" for packet size you may want to increase it slightly so on other networks
with potentially larger MTU sizes your
application still works. Again, this is all wasteful from a bandwidth
perspective - especially when end-users are charged per MB of data transfer on
some cell networks, but at least it would be a working solution.
Please report your findings back to the list, or at least to me. I'd be very
interested in the results.
Thanks!
Rob
----- Original Message ----
From: Eero Nevalainen <[EMAIL PROTECTED]>
To: dev@mina.apache.org
Sent: Friday, May 11, 2007 7:30:32 AM
Subject: Re: Session close problem
Hi all,
I've recently deployed a gateway server to mediate communication between
clients and remote devices. The gateway works fine in a 'regular'
network, but now we're experiencing problems when the remote devices are
behind a GPRS network.
The gateway sends an application level (TCP) 'PING' message periodically
to the remote devices. For some reason the PING messages seem to queue
up in the GPRS connection, without the devices receiving them.
Meanwhile, sending an ICMP request to the same devices gives a much
smaller roundtrip time (~700ms) and no message losses. Also the messages
sent by the remote devices to the gateway always arrive properly.
I can't reproduce this problem with an ethernet connection so any help
with the GPRS network would be most welcome!
-Eero Nevalainen
Dawie Malan wrote:
You're quite right - there is something in the middle, and it does cause the
symptoms you are reporting.
A GPRS network has two essential elements:
*Serving GPRS Support Node (SGSN)-Sends data to and receives data from mobile
stations, and maintains information about the location of a mobile station (MS).
The SGSN communicates between the MS and the GGSN.
*Gateway GPRS Support Node (GGSN)-A wireless gateway that allows mobile cell
phone users to access the public data network (PDN) or specified private IP
networks.
The GGSN acts like a proxy and is the device that sends the ACKs to the phone
even though the session is already closed on the server side.
I hope this answers your question.
Regards,
Dawie Malan
-----Original Message-----
From: Michael Bauroth [mailto:[EMAIL PROTECTED]
Sent: 22 March 2007 04:40 PM
To: dev@mina.apache.org
Subject: Session close problem
Hi,
I'm using currently the trunk version of Mina. I use it to communicate
with GPRS devices on the other side. In most cases this works very well!
But sometimes the things become strange. I've collected a bunch of
problems where I would need help / advices / ideas.
1. I call session.close() from inside my server. It seems, that the
session doesn't close as required.
2. I close the session. The GPRS device on the other side continues to
send data and seems to get ACKs too. It doesn't recognize, that the
other side of the connection (the server) has closed the session a while
before. It seems that there is something in the middle which responds
the ACKs even with not opened server session.
Knows anybody one of the two (most painful) problems and has an answer
for me?
Best Regards
Michael
____________________________________________________________________________________
Never miss an email again!
Yahoo! Toolbar alerts you the instant new Mail arrives.
http://tools.search.yahoo.com/toolbar/features/mail/
____________________________________________________________________________________Get the free Yahoo! toolbar and rest assured with the added security of spyware protection.
http://new.toolbar.yahoo.com/toolbar/features/norton/index.php
