Thank you so much toad. Its very detailed and I ll need to go through it thoroughly.
Also I am not sure this is relevant, but I did get a mail from a "stranger" who happened to be going through the archives and decided to send me an opinion. I thought it's best if we discussed here. I requested the person to participate here, and will hopefully do so. On Wed, Apr 4, 2012 at 1:16 PM, cagsm <cumandgets0mem00f at gmail.com> wrote: Hello GSOC participant, I am not on the Freenet mailing list, but I read the archvies and I have seen an interesting discussion on Freenet (insinde Freenet, within the FMS communications tool, in the Freenet forum there). Some fellow FMS/Freenet user proposed to look at the open source project "udt" (Sourceforge), that is essentially an intelligent and very robust and proven way to do udp communications. If you are already re-implementing the old raw udp way of Freenet transport layers and modularizing it, which is needed due to compatibility reasons as another fellow FMS/Freenet participant (bombe I think) pointed out, then it would be probably better to migrate away from the raw udp way to the udt way in the long run, before attempting the next real transport plugin (tcp). Please consider into looking at the udt project and making use of it. Would be very glad if you could include it in Freenet. http://udt.sourceforge.net/ On Thu, Apr 5, 2012 at 6:21 PM, Matthew Toseland <toad at amphibian.dyndns.org> wrote: > On Wednesday 04 Apr 2012 07:15:26 Chetan Hosmani wrote: >> Hello, >> >> With only three days left for the deadline of submission of proposals >> I was hoping I could get some suggestions on my proposal. >> This is my only proposal. I am also working on another proposal to >> Freenet itself (implementing JCA). >> Any suggestions will be really helpful. >> >> The proposal can viewed publicly here- >> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/chetanhosmani/1 >> >> Apart from adding references and links, this is the final version. >> Please leave comments. > > The introduction is good. > > IMHO it's not grouped *that* tightly: exactly what does NodeCrypto do that is > low level? The key classes afaics are NewPacketFormat, UdpSocketHandler, > FNPPacketMangler. IMHO UdpSocketHandler and SocketHandler in general already > effectively separate the actual socket implementation. Of course pretty much > all of it assumes a packet transport, e.g. PacketSender constantly loops over > peers to send packets to them. > > We have a structure for messages that are ready to send, PeerMessageQueue on > PeerNode. > > However I agree that the messages-in-flight structure is too tightly > integrated with NewPacketFormat, in NewPacketFormat itself and > NewPacketFormatKeyContext. I don't know whether this structure will be > adequate for *all* transports; it's likely to be overkill for TCP-based > transports, but for some packet based formats with bigger packets and longer > latencies it might be too limited? A single messages in flight queue makes > sense though for robustness, in fact given we assume reliable delivery (or > throw an exception), it's essential. But I'm not sure how you would manage > resends for multiple transports... Wouldn't they have separate timers etc? > Clearly we would want to send a message over another transport if one dies, > or persistently can't send it ... > > NewPacketFormat contains the queue of messages in flight. > NewPacketFormatKeyContext contains the data on *packets* in flight. TCP > presumably won't have packets, but will need to group messages and HMAC them, > and the two streams aren't necessarily perfectly synchronised so we'd > probably want some sort of sequence number acknowledgements ... > > The OldPacketFormat/PacketTracker code can probably be got rid of now, people > have had plenty of time to upgrade. > > If it's acceptable for TCP, then I'm quite happy for you to factor it out > into another class. Rewriting it yet again would be a lot of work for no > obvious advantage, but if you have a clear set of reasons for doing so that > would be interesting. > > Re multiple connections to a single peer, this is all in PeerNode. It can > receive messages from any transport (handle*), it has a queue which is read > e.g. when PacketSender asks it to prepare a packet, but a significant part of > the complexity of the UDP transport results from hacks related to various > failure modes in session setup (I believe we spoof a restart in some cases). > As does code elsewhere, e.g. there is a single throttle used by PacketSender > for a PeerNode. > > To implement UDP as a transport plugin, you will need to implement the > transport plugin API *first*, because it is an unreliable packet transport. > You will need to refactor crypto support for the plugin to be where it needs > to be. But it's the same crypto; the main difference is we can handle > multiple connections to the same peer over different IP addresses more > cleanly. > > Then you will need to implement connection setup for stream-based transports, > as well as the basic plugin API, and the driver code (I assume a stream-based > transport will present a pair of InputStream/OutputStream?). Connection setup > should be fairly similar to the existing code. Whether you want to support > sending parts of messages like we do with UDP is unclear; see above. > > IMHO multiple connections between the same two nodes are an essential part of > the GSoC, if only because we will want nodes to expose more than one > transport, so that they can connect to the maximum possible range of other > nodes. So it will be possible that two connection setups complete > simultaneously. "Just close one" is not anywhere near as simple as you might > think, from my experience with UDP ... > > I agree that HTTP etc is optional though. > > Obviously NodeCrypto should have a set of transport addresses, not just the > one port number for UDP. > > I agree that you will need a different SocketHandler base class for > stream-based transports. > > Ideas on where the boundaries lie: > - Clearly the packet format is specific to a particular transport type. It > will be different for packets vs streams. > - Crypto setup for streams could be the same algorithm, but even if it is, it > may not share the same code. The current code is designed for lossy packets, > whereas a much simpler control flow is possible for a stream. Authentication > is obviously different: A packet has an HMAC, whereas for a stream you can > use a self-authenticating cipher or you can group messages in a cluster (with > a sequence number for ack's) and then give an HMAC. IMHO the latter is better. > - Retransmission logic will need to be different for each transport, but the > messages in flight must be shared with failover. This is going to be tricky > as I explained above. > - The queue of messages that have not yet been sent at all is common. > > IMHO it should be possible to keep more or less the same crypto, packet > format, connection setup and message queueing for UDP-based connections, so > back compatibility shouldn't be a problem - but if there are reasons for a > totally new message queueing system, please post them! > > Obviously you need to ensure that only the code that is absolutely essential > is in the actual socket handler class. Stuff that is related to decoding > messages etc should be elsewhere - UdpSocketHandler should simply connect a > UDP socket to the standard crypto/etc code. It should be really simple. > Ideally this would be a separate class (rather than inheriting), so it can > more easily be passed in by a plugin. > > We don't need an actual plugin at this point, just a clean API. So TCP can be > built-in IMHO, although you could build it as a plugin if you want to > demonstrate the whole thing. > > Regular updates are essential. You will need to work closely with a competent > mentor (probably nextgens), and he will regularly review your commits; saving > them up for a month and then "dropping a bomb" is very anti-social. > > Re documentation, try to javadoc stuff when you are creating new big classes. > Not that we all do all the time. This will help in review and in not getting > confused later. > > Generally looks good, though ultimately whether you are accepted is up to > your mentor (and google). I will try to help out where I can but I don't have > the time to be a mentor.
