I have been hacking on FTP for the last two weeks (prior to a short but well spent vacation). The mission was to remove the threadpool and make FTP asynchronous. I am about a week away from landing assuming that I do not run into any major problems. During this time, I hope to find as may problems as possible. If you are so inclined, feel free to help me out and pull these changes.
A quick look at the performance delta is about 7% for directory listing download (not tree/rdf building) favoring the new implementation. I hope to gather some more in-depth data later.
How to update:
cvs up -rDOUGT_FTP_12122000_BRANCH mozilla/netwerk/protocol/ftp
Tags:
FTP Base tag: DOUGT_FTP_12122000_BASE
FTP Branch tag: DOUGT_FTP_12122000_BRANCH
Diffs:
cvs diff -u -rDOUGT_FTP_12122000_BASE -rDOUGT_FTP_12122000_BRANCH
mozilla/netwerk/protocol/ftp
Socket Transport Patch:
Youll, probably want to apply the patches in this bug too:
http://bugzilla.mozilla.org/show_bug.cgi?id=63565
Without these changes, you will leak socket transports. I should be checking in this fix in the next day or so.
FTP Changes include:
- Changes to make FTP asynchronous (eg. be driven by the socket transport).
- Moved status codes from public idl to ftpCore.h
- Reworked connection cache to store the ftp channel instead of the underlying socket. This was done so that we could maintain an open pipe to the server. The previous implementation did not use Async calls so there was no tie between the channel and transport. AsyncRead holds an AddRef to the stream listener. There is not a way to break this without closing the pipe.
- Remove all the AsyncEvent and nsISupport Proxy code. This is obsolete since we are asynchronous.
- Removed lock from nsFTPHandler which protected the connection hashtables. Instead, I created the hashtables thread safe.
- Removed ThreadPools
- Remove the ConnectionCache interface. Static methods are better for this kind of thing because we are in the same component and we do not have to hold references to the protocol handler.
- Create a new object nsFtpControlConnection which will wrap the control pipe so that we can (a) cache it and (b) be able to replace the observer without closing the socket.
- Renamed mConnThread to mFtpState
- Removed Available() call in OnDataAvailable. We should just the byte count that was passed to us.
- Implemented nsFtpState::OnStopRequest so that it cancels us. This is to avoid leaks when a problem occurs before the control connection is established.
- More PR_LOG's so that we can understand what the heck is going on! :-)
- A whole bunch more... :-)
Known Things To Do:
- Need some perf verification
- Need some mac love
- Better caching and management of control connections
- Check for regressions
