On Thursday 05 Apr 2012 14:29:12 Chetan Hosmani wrote: > Thank you so much toad. Its very detailed and I ll need to go through > it thoroughly. > > Also I am not sure this is relevant, but I did get a mail from a > "stranger" who happened to be going through the archives and decided > to send me an opinion. I thought it's best if we discussed here. I > requested the person to participate here, and will hopefully do so. > > On Wed, Apr 4, 2012 at 1:16 PM, cagsm <cumandgets0mem...@gmail.com> wrote: > Hello GSOC participant, > > I am not on the Freenet mailing list, but I read the archvies and I > have seen an interesting discussion on Freenet (insinde Freenet, > within the FMS communications tool, in the Freenet forum there). Some > fellow FMS/Freenet user proposed to look at the open source project > "udt" (Sourceforge), that is essentially an intelligent and very > robust and proven way to do udp communications. > > If you are already re-implementing the old raw udp way of Freenet > transport layers and modularizing it, which is needed due to > compatibility reasons as another fellow FMS/Freenet participant (bombe > I think) pointed out, then it would be probably better to migrate away > from the raw udp way to the udt way in the long run, before attempting > the next real transport plugin (tcp). > > Please consider into looking at the udt project and making use of it. > Would be very glad if you could include it in Freenet. > http://udt.sourceforge.net/
I don't think the transport layer is a big performance issue for Freenet, which is the real reason why people want utp. Maybe it's a minor one. In any case, we can't use utp directly. Whatever we use, the basic UDP-based FNP must be hard to identify. Using utp, or any other off the shelf transport, without wrapping it in an outer layer of obfuscation-encryption, would violate this goal, as all such protocols are easily identified. An obscure one like utp that nobody else uses would be particularly easily identified. A big part of the purpose of transport plugins is so Freenet can send traffic that looks like other, common, things, such as VoIP traffic - this is not possible if we use utp. > > On Thu, Apr 5, 2012 at 6:21 PM, Matthew Toseland > <t...@amphibian.dyndns.org> wrote: > > On Wednesday 04 Apr 2012 07:15:26 Chetan Hosmani wrote: > >> Hello, > >> > >> With only three days left for the deadline of submission of proposals > >> I was hoping I could get some suggestions on my proposal. > >> This is my only proposal. I am also working on another proposal to > >> Freenet itself (implementing JCA). > >> Any suggestions will be really helpful. > >> > >> The proposal can viewed publicly here- > >> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/chetanhosmani/1 > >> > >> Apart from adding references and links, this is the final version. > >> Please leave comments. > > > > The introduction is good. > > > > IMHO it's not grouped *that* tightly: exactly what does NodeCrypto do that > > is low level? The key classes afaics are NewPacketFormat, UdpSocketHandler, > > FNPPacketMangler. IMHO UdpSocketHandler and SocketHandler in general > > already effectively separate the actual socket implementation. Of course > > pretty much all of it assumes a packet transport, e.g. PacketSender > > constantly loops over peers to send packets to them. > > > > We have a structure for messages that are ready to send, PeerMessageQueue > > on PeerNode. > > > > However I agree that the messages-in-flight structure is too tightly > > integrated with NewPacketFormat, in NewPacketFormat itself and > > NewPacketFormatKeyContext. I don't know whether this structure will be > > adequate for *all* transports; it's likely to be overkill for TCP-based > > transports, but for some packet based formats with bigger packets and > > longer latencies it might be too limited? A single messages in flight queue > > makes sense though for robustness, in fact given we assume reliable > > delivery (or throw an exception), it's essential. But I'm not sure how you > > would manage resends for multiple transports... Wouldn't they have separate > > timers etc? Clearly we would want to send a message over another transport > > if one dies, or persistently can't send it ... > > > > NewPacketFormat contains the queue of messages in flight. > > NewPacketFormatKeyContext contains the data on *packets* in flight. TCP > > presumably won't have packets, but will need to group messages and HMAC > > them, and the two streams aren't necessarily perfectly synchronised so we'd > > probably want some sort of sequence number acknowledgements ... > > > > The OldPacketFormat/PacketTracker code can probably be got rid of now, > > people have had plenty of time to upgrade. > > > > If it's acceptable for TCP, then I'm quite happy for you to factor it out > > into another class. Rewriting it yet again would be a lot of work for no > > obvious advantage, but if you have a clear set of reasons for doing so that > > would be interesting. > > > > Re multiple connections to a single peer, this is all in PeerNode. It can > > receive messages from any transport (handle*), it has a queue which is read > > e.g. when PacketSender asks it to prepare a packet, but a significant part > > of the complexity of the UDP transport results from hacks related to > > various failure modes in session setup (I believe we spoof a restart in > > some cases). As does code elsewhere, e.g. there is a single throttle used > > by PacketSender for a PeerNode. > > > > To implement UDP as a transport plugin, you will need to implement the > > transport plugin API *first*, because it is an unreliable packet transport. > > You will need to refactor crypto support for the plugin to be where it > > needs to be. But it's the same crypto; the main difference is we can handle > > multiple connections to the same peer over different IP addresses more > > cleanly. > > > > Then you will need to implement connection setup for stream-based > > transports, as well as the basic plugin API, and the driver code (I assume > > a stream-based transport will present a pair of InputStream/OutputStream?). > > Connection setup should be fairly similar to the existing code. Whether you > > want to support sending parts of messages like we do with UDP is unclear; > > see above. > > > > IMHO multiple connections between the same two nodes are an essential part > > of the GSoC, if only because we will want nodes to expose more than one > > transport, so that they can connect to the maximum possible range of other > > nodes. So it will be possible that two connection setups complete > > simultaneously. "Just close one" is not anywhere near as simple as you > > might think, from my experience with UDP ... > > > > I agree that HTTP etc is optional though. > > > > Obviously NodeCrypto should have a set of transport addresses, not just the > > one port number for UDP. > > > > I agree that you will need a different SocketHandler base class for > > stream-based transports. > > > > Ideas on where the boundaries lie: > > - Clearly the packet format is specific to a particular transport type. It > > will be different for packets vs streams. > > - Crypto setup for streams could be the same algorithm, but even if it is, > > it may not share the same code. The current code is designed for lossy > > packets, whereas a much simpler control flow is possible for a stream. > > Authentication is obviously different: A packet has an HMAC, whereas for a > > stream you can use a self-authenticating cipher or you can group messages > > in a cluster (with a sequence number for ack's) and then give an HMAC. IMHO > > the latter is better. > > - Retransmission logic will need to be different for each transport, but > > the messages in flight must be shared with failover. This is going to be > > tricky as I explained above. > > - The queue of messages that have not yet been sent at all is common. > > > > IMHO it should be possible to keep more or less the same crypto, packet > > format, connection setup and message queueing for UDP-based connections, so > > back compatibility shouldn't be a problem - but if there are reasons for a > > totally new message queueing system, please post them! > > > > Obviously you need to ensure that only the code that is absolutely > > essential is in the actual socket handler class. Stuff that is related to > > decoding messages etc should be elsewhere - UdpSocketHandler should simply > > connect a UDP socket to the standard crypto/etc code. It should be really > > simple. Ideally this would be a separate class (rather than inheriting), so > > it can more easily be passed in by a plugin. > > > > We don't need an actual plugin at this point, just a clean API. So TCP can > > be built-in IMHO, although you could build it as a plugin if you want to > > demonstrate the whole thing. > > > > Regular updates are essential. You will need to work closely with a > > competent mentor (probably nextgens), and he will regularly review your > > commits; saving them up for a month and then "dropping a bomb" is very > > anti-social. > > > > Re documentation, try to javadoc stuff when you are creating new big > > classes. Not that we all do all the time. This will help in review and in > > not getting confused later. > > > > Generally looks good, though ultimately whether you are accepted is up to > > your mentor (and google). I will try to help out where I can but I don't > > have the time to be a mentor. > >
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
