[jira] Created: (FTPSERVER-212) Full Javadocs for the public API
Full Javadocs for the public API Key: FTPSERVER-212 URL: https://issues.apache.org/jira/browse/FTPSERVER-212 Project: FtpServer Issue Type: Improvement Components: Core, Ftplets Affects Versions: 1.0-M3, 1.0-M2, 1.0-M1 Reporter: Niklas Gustavsson Assignee: Niklas Gustavsson Fix For: 1.0-M4 The public API should be fully documented in Javadocs -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: ExecutorFilter quick poll: How do you use it ?
On Tue, 04 Nov 2008 13:40:09 +0100 Emmanuel Lecharny <[EMAIL PROTECTED]> wrote: > Hi guys, > > while thinking about the best possible implementation for the new > chain, I had a look at the ExecutorFilter which is quite an > interesting filter, wrt the way we handle the chain. > > Remember that a session may handle more than one event at the same > time, especially when using the executor filter, as you may have read > and write event being processed (if I'm not totally wrong). > > Also, as the executor filter will multiplex the processing, this may > occur that the chain will potentially be shared by many threads. > Currently, there is no protection against such potential concurrent > modification if we use an ExecutorFilter. > > So I would be interested to know if you are using this filter, and if > so, how. Here is a list of questions I have : > > 1) Are you using the ExecutorFilter ? Yes > 2) Where in the chain do you put this filter ? Just after ProtocolCodec, I use it mainly for demuxing IoHandling > 3) what if the IoHandler itself implement an executor instead of > using a specific filter ? Could fit my basic needs. > 4) Is this a problem if the chain is not protected when using an > ExecutorFilter ? protected for ? synchronization ? well yes it could be problem if you try to modify the chain in reception of a message in a filter or IoHandler after the ExecutorFilter no ? > > I may miss some other points, so feel free to share your ideas about > how to use this filter correctly ! > > Many thanks ! > signature.asc Description: PGP signature
AW: ExecutorFilter quick poll: How do you use it ? no---signature
> Emmanuel Lecharny [mailto:[EMAIL PROTECTED] wrote > > Hi guys, Hi! [snip] > > 1) Are you using the ExecutorFilter ? Yes > 2) Where in the chain do you put this filter ? Behind the codec and (SLF4J) logging filters, just before the long running tasks and database stuff begins. > 3) what if the IoHandler itself implement an executor instead of using > a specific filter ? We have Filters that will do database stuff, like logging things, checking for some preconditions, etc. So it wouldn't help very much. > 4) Is this a problem if the chain is not protected when using an > ExecutorFilter ? The one method/event at a time per session should be ok for most cases. Maybe it would be good for beginners to have a single threaded chain. regards Steve
Re: ExecutorFilter quick poll: How do you use it ?
On Tue, Nov 4, 2008 at 6:10 PM, Emmanuel Lecharny <[EMAIL PROTECTED]> wrote: > Hi guys, > > while thinking about the best possible implementation for the new chain, I > had a look at the ExecutorFilter which is quite an interesting filter, wrt > the way we handle the chain. > > Remember that a session may handle more than one event at the same time, > especially when using the executor filter, as you may have read and write > event being processed (if I'm not totally wrong). > > Also, as the executor filter will multiplex the processing, this may occur > that the chain will potentially be shared by many threads. Currently, there > is no protection against such potential concurrent modification if we use an > ExecutorFilter. > > So I would be interested to know if you are using this filter, and if so, > how. Here is a list of questions I have : > > 1) Are you using the ExecutorFilter ? Yes > 2) Where in the chain do you put this filter ? Multiple places to implement SEDA (before ProtocolDecoder, before IoHandler) > 3) what if the IoHandler itself implement an executor instead of using a > specific filter ? Not much of help in my architecture. Hope we will still be able to use Executor filter as we do today. > 4) Is this a problem if the chain is not protected when using an > ExecutorFilter ? > Not sure if I can comment on this, but does this mean same byte chunk (IoBuffer) can be processed by more than one thread? If it can be, I have flow in my implementation :-( > I may miss some other points, so feel free to share your ideas about how to > use this filter correctly ! > > Many thanks ! -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: [About the Filter Chain] Events
Hello, I also use messageSent but I guess I can also replace it with an IoFutureListener. What do you mean when you say that SESSION_IDLE is questionable ? What would you replace it with ? My guts feeling is that exceptionCaught should also be propagated to the outgoing chain... but right now I have no examples to back up that feeling. Maarten On Mon, Nov 3, 2008 at 4:29 PM, Mark Webb <[EMAIL PROTECTED]> wrote: > I had logic in my IoHandler that was quite complex and there were 3 > different parts of the code that could send data back to the client. > Instead of placing a WriteFuture in all 3 places, I put the logic in > messageSent. > > I could just define an IoFutureListener that is passed into all 3 > WriteFuture's though. If it makes our lives (as API developers) > easier, then I can change my code (as an end user). > > --Mark > > > On Mon, Nov 3, 2008 at 10:13 AM, Emmanuel Lecharny <[EMAIL PROTECTED]> > wrote: > > Mark Webb wrote: > >> > >> I have used MESSAGE_SENT, not sure I agree that it should disappear. > >> > > > > May I ask you in which context you used it ? I'm wondering if we can > cover > > your case with WriteFuture ? > > > >> What about situations where MINA is used in a client library? Won't > >> much of your list be reversed? > >> > > > > That's a very good question ! I don't think it makes any difference. From > > the client side, the incoming events are generated by an outgoing event > on > > the server side. Ie, the server write a response, then the client get a > > MESSAGE_RECEIVED event to deal with. In fact, it's really like if you > have > > nothing in between the client and the server. The client write a message, > > the server read it, then the server write the response, the client read > it. > > > > If you take the protocolCodecFilter, it works the same for the server and > > the client. A message will have to be encoded in both case when writing > the > > message (and it's always on the outgoing chain), and decoded when read > > (incoming chain). > > > > I may miss something, but AFAICS, it's absolutely symetric. > > > > -- > > -- > > cordialement, regards, > > Emmanuel Lécharny > > www.iktek.com > > directory.apache.org > > > > > > >
setTrafficMask poll again
Hi, There is something in MINA who has hook everywhere in the core, it's traffic mask. As far I understand the concept, the idea is to be able to block read and/or writes using session.setTrafficMask(...), I never needed it, and I wonder who use it and for what exactly ? Julien signature.asc Description: PGP signature
Re: [About the Filter Chain] Events
Maarten Bosteels wrote: Hello, Hi Maarten (eh eh, I typed it correctly this time !), I also use messageSent but I guess I can also replace it with an IoFutureListener. What do you mean when you say that SESSION_IDLE is questionable ? What would you replace it with ? There is a thread which looks for sessions in idle state, in order to kill them after a certain period of time (a timeout). So my question is why do we have to deal with a sessionIdle message ? And when is it generated ? I'm missing something in this area. My guts feeling is that exceptionCaught should also be propagated to the outgoing chain... but right now I have no examples to back up that feeling. I have to go deeper into it, too. Thanks Maarten ! -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
Re: setTrafficMask poll again
Wasn't it an attempt to implement throttling ? When requests are coming in faster than they're being processed => set TrafficMask to block reading => TCP buffers will fill up (OS level) => TCP will tell sender to slow down => OOM prevented when queue of incoming messages gets smaller => resume reading I haven't tried this yet, so I could be totally wrong. Maarten On Tue, Nov 4, 2008 at 5:50 PM, Julien Vermillard <[EMAIL PROTECTED]>wrote: > Hi, > > There is something in MINA who has hook everywhere in the core, it's > traffic mask. As far I understand the concept, the idea is to be able > to block read and/or writes using session.setTrafficMask(...), I never > needed it, and I wonder who use it and for what exactly ? > > Julien >
Re: ExecutorFilter quick poll: How do you use it ?
Ashish wrote: 2) Where in the chain do you put this filter ? Multiple places to implement SEDA (before ProtocolDecoder, before IoHandler) I would like to know if you are not ask risk to have a hell lot of thread if you do so. And I'm not sure that it has any added value, as the idea is to use a thread to handle a costly operation, no need to spawn a new thread when it's already done... Am I missing something ? 4) Is this a problem if the chain is not protected when using an ExecutorFilter ? Not sure if I can comment on this, but does this mean same byte chunk (IoBuffer) can be processed by more than one thread? If it can be, I have flow in my implementation :-( That's for sure, unless you have already gathered all the bytes before processing the created message in its own thread (via the Executor). The idea is pretty much to collect the bytes until you can build a new Object, and then spawn a thread passing this object to it. Otherwise, assuming that the bytes arrive by chunk, which thread will swallow the next incoming bytes ? -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
Re: setTrafficMask poll again
Maarten Bosteels wrote: Wasn't it an attempt to implement throttling ? I have no idea. The documentation is pretty scarse, it's not used anywhere but as a parameter passed from method to method, the event type is never used to create an event ... When requests are coming in faster than they're being processed => set TrafficMask to block reading Which part of MINA is setting this flag ? => TCP buffers will fill up (OS level) => TCP will tell sender to slow down => OOM prevented when queue of incoming messages gets smaller => resume reading May be it's a part of a big DoS protection heuristic, but it seems to be lagging seriously. Even the throttling filter is a bit incomplete ... Anyone has a better picture than me on this matter ? -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
Re: setTrafficMask poll again
It was used by Read/WriteThrottlingFilter wich was removed of 2.0 : http://www.nabble.com/Dropping-traffic-throttling-from-2.0-td16092085.html as said by Emm look like it's used nowhere is MINA codebase. As said by Trustin in this mail the remplacement is supposed to be o.a.m.f.executor.* and no references to setTrafficMask(); Frankly I don't understand how you can throttle read, without using setTrafficMask and disabling OP_READ on the low level socket. Julien On Tue, 4 Nov 2008 18:01:58 +0100 "Maarten Bosteels" <[EMAIL PROTECTED]> wrote: > Wasn't it an attempt to implement throttling ? > > When requests are coming in faster than they're being processed > => set TrafficMask to block reading > => TCP buffers will fill up (OS level) > => TCP will tell sender to slow down > => OOM prevented > > when queue of incoming messages gets smaller => resume reading > > I haven't tried this yet, so I could be totally wrong. > > Maarten > > On Tue, Nov 4, 2008 at 5:50 PM, Julien Vermillard > <[EMAIL PROTECTED]>wrote: > > > Hi, > > > > There is something in MINA who has hook everywhere in the core, it's > > traffic mask. As far I understand the concept, the idea is to be > > able to block read and/or writes using session.setTrafficMask(...), > > I never needed it, and I wonder who use it and for what exactly ? > > > > Julien > > signature.asc Description: PGP signature
Re: ExecutorFilter quick poll: How do you use it ?
On Tue, Nov 4, 2008 at 10:33 PM, Emmanuel Lecharny <[EMAIL PROTECTED]> wrote: > Ashish wrote: >> >>> 2) Where in the chain do you put this filter ? >>> >> >> Multiple places to implement SEDA (before ProtocolDecoder, before >> IoHandler) >> > > I would like to know if you are not ask risk to have a hell lot of thread if > you do so. And I'm not sure that it has any added value, as the idea is to > use a thread to handle a costly operation, no need to spawn a new thread > when it's already done... Am I missing something ? Well we needed SEDA to attain a very high processing rate. Since its one way, no response to be sent back. That's why we choose this approach. Again at each stage, we do something meaningful. Say in protocolcodec, we convert raw bytes into objects, next these objects get converted into OSSJ object and then handler dumps them into DB. Each of these stages are decoupled, onces passed no need to look back. Processing in one thread was like blocking it, let the messages be in queue and then use them. This is quite helpful for a high burst or a sustained load for a while. I hope I have used MINA correctly, though I haven't benchmarked my application for sustained load, for a peak load it did fairly well, taking care of 5000 packets/sec, though had to set slightly higher value of receive buffer. >>> 4) Is this a problem if the chain is not protected when using an >>> ExecutorFilter ? >>> >>> >> >> Not sure if I can comment on this, but does this mean same byte chunk >> (IoBuffer) can be processed by more than one thread? >> If it can be, I have flow in my implementation :-( >> > > That's for sure, unless you have already gathered all the bytes before > processing the created message in its own thread (via the Executor). The > idea is pretty much to collect the bytes until you can build a new Object, > and then spawn a thread passing this object to it. Otherwise, assuming that > the bytes arrive by chunk, which thread will swallow the next incoming bytes > ? I guess I am safe here :-) I wait until my message is complete, before putting it into next queue. > -- > cordialement, regards, > Emmanuel Lécharny > www.iktek.com > directory.apache.org > > > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
ApacheCon live video streaming available; keynotes and Apache 101 are free
Can't make ApacheCon this week in New Orleans? You can still watch all
the keynotes, Apache 101 sessions, and system administration track in
live video streams:
http://streaming.linux-magazin.de/en/program_apacheconus08.htm?ann
Keynotes and the Apache 101 lunchtime sessions are free; the full
sysadmin track, including httpd performance, security, and server stack
administration talks are available for a fee.
Keynotes include:
- David Recordon, Six Apart (Wednesday 09:30)
"Learning from Apache to create Open Specifications"
- Shahani Markus Weerawarana, Ph.D. (Thursday 11:30)
"Standing on the Shoulders of Giants"
- Sam Ramji, Microsoft (Friday 11:30)
"struct.new("future", :open, :microsoft)"
Reminder: New Orleans is CST or UTC/GMT -6 hours.
Advance notice: ApacheCon EU 2009 returns to Amsterdam, 23-27 March. We
had a great response to our CFP and look forward to announcing the
schedule in the next month.
Regards
--
Lars Eilebrecht - V.P., Conference Planning
[EMAIL PROTECTED] - http://www.us.apachecon.com
Q about the SessionCreated event
Hi,
I have a question regarding the way this event is handled in MINA. When
you connect to a remote server, the client will call the
NioSocketConnector, which will create a connection, then a Processor
will be spawn, its add() method will be called, the session will be
initialized (the chain is injected into it), and at the end, we will
fire the CreateSessionEvent :
// DefaultIoFilterChain.CONNECT_FUTURE is cleared inside here
// in AbstractIoFilterChain.fireSessionOpened().
((AbstractIoService)
session.getService()).getListeners().fireSessionCreated(session);
This will in return notify the waiting ConnectFuture object which is
used by the client, when the event has been processed by the Tail filter.
So far, so good. Now, my question : why do we notify the future in the
tail filter, instead of doing so when we come back from the filter ? I
understand that we have to traverse the full chain in order to be sure
that no exception is thrown, but I don't get why we can't simply wait
for the return to notify.
What we have is :
fireEvent -> filter1.handle() -> filter2.handle() -> ... ->
TailFilter.handle() -> notify(future)
why not :
try {
fireEvent -> filter1.handle() -> filter2.handle() -> ... ->
TailFilter.handle() (do nothing)
notify(future)
} catch( error ) {
handle exception
}
The problem I have with the first approach is that you _have_ to deal
with this TailFilter, even if you don't need it, which means that the
Handler must be distinct from a filter, _or_ that the last filter (if we
consider that an handler should be a filter) _have to_ call the
TailFilter handler for this event ?
Seems to me pretty complicated, by maybe it's just me...
wdyt ?
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
where should I close the session?
While using mina, I didn't know where to call session.close() after I write the message to the session. Shall I just set the idletime and leave the session close itself?
[jira] Created: (FTPSERVER-213) Mark internal classes as internal
Mark internal classes as internal - Key: FTPSERVER-213 URL: https://issues.apache.org/jira/browse/FTPSERVER-213 Project: FtpServer Issue Type: Improvement Components: Core Affects Versions: 1.0-M3, 1.0-M2, 1.0-M1 Reporter: Niklas Gustavsson Assignee: Niklas Gustavsson Fix For: 1.0-M4 All classes of the internal API should be documented as internal in the class Javadoc -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: where should I close the session?
Hi Erix, You can do like this: ioSession.write(someMessage).addListener(IoFutureListener.CLOSE); Bogdan On Tue, Nov 4, 2008 at 10:38 AM, Erix Yao <[EMAIL PROTECTED]> wrote: > While using mina, I didn't know where to call session.close() after I write > the message to the session. > Shall I just set the idletime and leave the session close itself? >
ExecutorFilter quick poll: How do you use it ?
Hi guys, while thinking about the best possible implementation for the new chain, I had a look at the ExecutorFilter which is quite an interesting filter, wrt the way we handle the chain. Remember that a session may handle more than one event at the same time, especially when using the executor filter, as you may have read and write event being processed (if I'm not totally wrong). Also, as the executor filter will multiplex the processing, this may occur that the chain will potentially be shared by many threads. Currently, there is no protection against such potential concurrent modification if we use an ExecutorFilter. So I would be interested to know if you are using this filter, and if so, how. Here is a list of questions I have : 1) Are you using the ExecutorFilter ? 2) Where in the chain do you put this filter ? 3) what if the IoHandler itself implement an executor instead of using a specific filter ? 4) Is this a problem if the chain is not protected when using an ExecutorFilter ? I may miss some other points, so feel free to share your ideas about how to use this filter correctly ! Many thanks ! -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
Re: [About the Filter Chain] Proposals
On Mon, 03 Nov 2008 17:49:48 +0100 Emmanuel Lecharny <[EMAIL PROTECTED]> wrote: > Niklas Gustavsson wrote: > > On Mon, Nov 3, 2008 at 4:13 PM, Mark Webb <[EMAIL PROTECTED]> > > wrote: > >> I think we should focus on getting 2.0 out the door. We have been > >> working on it long enough and I think there are many people using > >> it in production or near-production systems. Once we release, we > >> will probably get alot more feedback and can use that feedback to > >> enhance/fix the next version. > >> > > > > Big +1. We will find areas that we would like to improve during the > > foreseeable future (this change and ByteBuffer comes to mind). > > > yop. And I don't see how we can include that in 2 weeks... > > Including all such changes will delay 2.0 for a long time, long > > enough for MINA to get behind other frameworks. Having a real > > release out will mean getting further feedback from users, so far I > > haven't seen a lot of users requesting this change nor the > > ByteBuffer change. I think we're too critical, the code is great. > Well, IMHO, the code works. Saying that it's great is another > story :) (but this might just be a matter of taste ...) > > Anyway, I agree with what you say. We don't release fast enough. Atm, > regardless to the current code quality, and performance, I think MINA > 2 is usable, even if there are still some issues to fix. I will do > some quick perf tests on ADS with MINA 2 and give you some feedback > soon. > > > Release early, release often. > > We do neither. > > > eh ;) > > > >> I would think that we should move right > >> towards 3.0. > >> > > > > I say go work on a branch (as already suggested) and see where that > > leads. > There is a new branche for such experiment. Branching is certainly > the way to go, whatever we do regarding the release ! > > I would like to let this thread go for a little bit (let's say a > couple of days), and then, I think we will have to vote : going for > 2.0-RC or modify the code massively. > I think it's a good idea. Let's wait few days for see where the branch is going. If it's looking like a viable and an interesting simplification then let's vote for choosing between integration or post-pone. Julien signature.asc Description: PGP signature
Re: [About the Filter Chain] Proposals
Hi All, I'm not a direct Mina user but rather someone who integrates with ApacheDS, and had to delve deep into the Mina 1.x code to fix a bug. Hence my opinions, for what they are worth, are those of an outsider who expects to be able to debug pretty much any code when I have a) the source and b) a running program. I can't agree that the MINA code I looked at as "great" - far from it. Rather - it took me two whole days to work out how to hack around a problem that was causing unintended buffering of search results (the intention was to have the results sent to the client as soon as they became available). I'd say the code I looked at was bordering on impenetrable, for a lot of the reasons Emmanuel has been describing (particularly the immensely deep call stack and associated difficulty when stepping in the debugger). Hence I think the API is definitely in urgent need of simplification, and my usual expectation is that APIs are tidied up at major release boundaries. Personally I'd be much happier with a Mina 2.0 with a much simplified API - although I'm not using pre-release versions in production and therefore don't have some of the logistical concerns that others apparently do. I don't know how much I represent the "possible users of MINA" and therefore whether my opinion is of any interest, but to summarize my thoughts: 1. I would definitely not use the Mina API directly without the proposed simplifications 2. I would definitely not wait a year for such an API if I had a need for one. I'm not familiar enough with the Mina nuts and bolts to speak to how much has changed between Mina 1.1.7 and the current code, but would it makes sense to to release the current 2.0 code as a 1.2 (or 1.5) minor release, or: 1. has the API already changed enough to break backward compatibility even without Emmanuel's proposed simplifications? 2. is an extra release too much additional work. My 2c as well... Thanks On Tue, Nov 4, 2008 at 9:57 PM, Julien Vermillard <[EMAIL PROTECTED]> wrote: > On Mon, 03 Nov 2008 17:49:48 +0100 > Emmanuel Lecharny <[EMAIL PROTECTED]> wrote: > >> Niklas Gustavsson wrote: >> > On Mon, Nov 3, 2008 at 4:13 PM, Mark Webb <[EMAIL PROTECTED]> >> > wrote: >> >> I think we should focus on getting 2.0 out the door. We have been >> >> working on it long enough and I think there are many people using >> >> it in production or near-production systems. Once we release, we >> >> will probably get alot more feedback and can use that feedback to >> >> enhance/fix the next version. >> >> >> > >> > Big +1. We will find areas that we would like to improve during the >> > foreseeable future (this change and ByteBuffer comes to mind). >> > >> yop. And I don't see how we can include that in 2 weeks... >> > Including all such changes will delay 2.0 for a long time, long >> > enough for MINA to get behind other frameworks. Having a real >> > release out will mean getting further feedback from users, so far I >> > haven't seen a lot of users requesting this change nor the >> > ByteBuffer change. I think we're too critical, the code is great. >> Well, IMHO, the code works. Saying that it's great is another >> story :) (but this might just be a matter of taste ...) >> >> Anyway, I agree with what you say. We don't release fast enough. Atm, >> regardless to the current code quality, and performance, I think MINA >> 2 is usable, even if there are still some issues to fix. I will do >> some quick perf tests on ADS with MINA 2 and give you some feedback >> soon. >> >> > Release early, release often. >> > We do neither. >> > >> eh ;) >> > >> >> I would think that we should move right >> >> towards 3.0. >> >> >> > >> > I say go work on a branch (as already suggested) and see where that >> > leads. >> There is a new branche for such experiment. Branching is certainly >> the way to go, whatever we do regarding the release ! >> >> I would like to let this thread go for a little bit (let's say a >> couple of days), and then, I think we will have to vote : going for >> 2.0-RC or modify the code massively. >> > > I think it's a good idea. > Let's wait few days for see where the branch is going. If it's looking > like a viable and an interesting simplification then let's vote for > choosing between integration or post-pone. > > Julien >
