On 1/18/12 2:57 AM, Chad Beaulac wrote:
Ok. I'll test the throughput on it with the tests I have.
Let me know if you have any isues with the throughput you obtain.
Be sure to set the default read buffer size to the maximum size, AFAIR,
it's set to 2048 bytes in MINA 2 (and can't be higher than 65536),
otherwise you'll may get atrocious throughput...
Thanks
Chad
Sent from my iPad
On Jan 17, 2012, at 11:33 AM, Emmanuel Lecharny<[email protected]> wrote:
IFAICT, on MINA 2, you should not have any issue.
The loop where we write buffers into the channel is :
...
final int maxWrittenBytes = session.getConfig().getMaxReadBufferSize()
+ (session.getConfig().getMaxReadBufferSize()>>> 1);
int writtenBytes = 0;
WriteRequest req = null;
try {
// Clear OP_WRITE
setInterestedInWrite(session, false);
do {
// Check for pending writes.
req = session.getCurrentWriteRequest();
if (req == null) {
req = writeRequestQueue.poll(session);
if (req == null) {
break;
}
session.setCurrentWriteRequest(req);
}
int localWrittenBytes = 0;
Object message = req.getMessage();
if (message instanceof IoBuffer) {
localWrittenBytes = writeBuffer(session, req,
hasFragmentation, maxWrittenBytes - writtenBytes,
currentTime);
if (( localWrittenBytes> 0 )
&& ((IoBuffer) message).hasRemaining()) {
// the buffer isn't empty, we re-interest it in writing
writtenBytes += localWrittenBytes;
setInterestedInWrite(session, true);
return false;
}
} else { // Blahhh }
if (localWrittenBytes == 0) {
// Kernel buffer is full.
setInterestedInWrite(session, true);
return false;
}
writtenBytes += localWrittenBytes;
if (writtenBytes>= maxWrittenBytes) {
// Wrote too much
scheduleFlush(session);
return false;
}
} while (writtenBytes< maxWrittenBytes);
...
with :
private int writeBuffer(S session, WriteRequest req,
boolean hasFragmentation, int maxLength, long currentTime)
throws Exception {
IoBuffer buf = (IoBuffer) req.getMessage();
int localWrittenBytes = 0;
if (buf.hasRemaining()) {
int length;
if (hasFragmentation) {
length = Math.min(buf.remaining(), maxLength);
} else {
length = buf.remaining();
}
localWrittenBytes = write(session, buf, length);
}
and :
protected int write(NioSession session, IoBuffer buf, int length)
throws Exception {
if (buf.remaining()<= length) {
return session.getChannel().write(buf.buf());
}
int oldLimit = buf.limit();
buf.limit(buf.position() + length);
try {
return session.getChannel().write(buf.buf());
} finally {
buf.limit(oldLimit);
}
}
So we try our best to stuff the channel with as many bytes as possible, before
giving up (either because we don't have anything to write, to because the
channel is full...)
I don't see if we can do any better.
On 1/17/12 3:57 PM, Chad Beaulac wrote:
On Jan 17, 2012, at 9:32 AM, Emmanuel Lécharny<[email protected]> wrote:
On 1/17/12 3:02 PM, Chad Beaulac wrote:
On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny<[email protected]>wrote:
On 1/16/12 2:56 PM, Chad Beaulac wrote:
Emmanuel, (all)
I'm working on this Camel ticket:
https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
I finished the initial cut of
https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to
create a mina2
component in Camel.
CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
Would it be possible to backport the IoBuffer reading and writing
discussed
in this email thread from Mina3 to Mina2?
Following the depth of the stack trace through
AbstractIoSession.write(...), I'm a little concerned about the throughput.
My current code (mina-less) is supporting single TCP channels with 320+
Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
Buffers codec I wrote. I know if this is a real problem soon when I finish
CAMEL-2624 and setup some throughput tests.
Another option would be to port this to MINA3, and make it work with
Camel. Right now, MINA 3 is pretty rough, but we have made it works well
with Http and LDAP. I'd rather spend some time making it a bit more solid
and working well in your case instead of trying to inject the code in MINA2.
Now, it's your call. We can discuss the pros and cons of both approach if
you like.
Hi Emmanuel,
One of my pros/cons trade-offs is time-to-market. I can have a solution in
Camel with Mina2 fairly quickly. Although I might have issues with high
data rate streams.
With that said, my approach would be the following:
1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
endpoints I need and a quick time-to-market. I'll put off issues concerning
high throughput.
2) Work on Mina3 to ensure it has low latency with small data rate streams
and high throughput with large data pipes.
3) Upgrade my Google protocol buffers codec for Mina3.
4) When Mina3 is ready, open a new Camel ticket and create a new mina3
Camel Component.
What do you think?
I'll try to squeeze 2 hours to backport the patch to MINA 2 today or tomorrow.
Feel free to ping me on mail or on #mina if I don't send any feedback in the
next 2 days (I'm pretty busy and may slip)
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Wow. That is nice! Look forward to checking it out. I'll move forward with my
plan in the meantime.
Chad
Sent from my iPhone
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
