On Mon, 7 Nov 2005, Matthew Toseland wrote:
> On Sun, Nov 06, 2005 at 10:33:25AM +0100, Roman Bednarek wrote:
>>
>> Hello.
>> Probably it is too late for changes and you probably already discussed
>> it, but I would like to say few words about the design.
>> Building monolithic solutions - that is doing everything in one place -
>> is often not the best solution. The better approach may be some smaller
>> cooperating layers. For example it may look like this:
>
> Everyone uses layers; this is perfectly normal.
>
>> Layer 0 - the freenet core/server - it manages datastore and network
>> connections. The api to it can be extremaly simple: Data getData(key) and
>> Key putData(data), where size of data is limited by the desing(currently
>> 1M), the metadata can also be handled here or on the next layer.
>
> Each layer can use the layers below, but will generally depend most on
> the layer immediately below:
>
> Layer 0 - transport plugins (very simple, interface depends on class)
> Layer 1 - transport classes, inc. packet retransmission, crypto
> Layer 2 - messages (taken from dijjer)
> Layer 3 - node locations, swapping node locations
> Layer 4 - requests, inserts, datastore
> Layer 5 - trivial interface to above - get/put CHK's
This is the internal structure I was referring to what is visible and
usable to the user. So I see that my proposition basically goes down to
including these two calls (get/put CHK) into FCP2.
>
>> Layer 1 - client services - splitfile handling, download queue,
>> containers, calculate keys and similar utilities. It may be run on the
>> same computer, as part of the node(started on the same javaVM, as layer 0)
>> or it may run elswere. It also may be available as a C library (such as
>> fcptools)
>
> Layer 6 - metadata handling, splitfiles, updatable keys, redirects, etc.
> Will also include the download manager and so on eventually.
>
> The stuff I'm building now.
>
> The interface to layer 7 is either FCPv2, or an internal interface
> equivalent to FCPv2 (to allow running fproxy in the same JVM).
>
> This layer will be part of the node. 0.7 metadata is binary, and quite
> complex (it can itself be split!), and I do not want clients to have to
> deal with it. FCPv2 will therefore not (normally) expose it to clients;
> FCPv2 is fairly high level.
It is good that clients will not have to deal with all the complex
details, but it is really bad that they cannot do it. I never liked
situations when something is possible inside the library, but is hidden
from the user who cannot use it.
>
> There is no reason for layer 7 clients to access anything below layer 6.
> If you want to reimplement layer 6 in C, or run it on another machine,
> that is quite possible, however it would mean providing a low-level FCP
> as well as a high-level FCP, which I consider to be ugly and
> unnecessary.
>
As I written before I think that hiding something is unnecessary.
> Note that you can have a fixed darknet link between the node on your
> fast workstation which gets switched off regularly, and the node on your
> slow firewall which is on 24/7. In which case you could run the fetches
> on the workstation (at the cost of them not running while it is switched
> off), even though the firewall node is on the open-net.
In the meantime my setup has changed, I use my small firewall machine
for downloads too, using solution based on fcp tools. As a C
program/library it has very low memory/CPU usage and can run on PII
without problems.
>
>> Layer 2 - applications - FPProxy, frost, fuqid. They usually talk to
>> the server through layer 1, but some more advanced(like fuqid or maybe
>> frost) may choose to talk directly to the core server at layer 0.
>
> Layer 7 in my model. FUQID, Frost and FIW should talk to layer 6
> exclusively, via the exposed APIs, which will include a download manager
> backend. The main advantages are:
> - We can take control of download policy, and optimize it however we
> want. The code is maintained.
It is not necessary a good thing, maybe a download queue in a client
will perform better? and client can optimize it in its own way.
> - Clients can be very simple, because they don't need to worry about
> e.g. multi-layer splitfiles.
> - We only need one HTML anonymity filter implementation, and we can
> include I18N (charset detection on insert) regardless of the client.
>
So the bottom line is that I think clients should be able to work with
lower level of the node. Today clients are working that way and are
doing good job with it, there is no need to throw away that code and rely
only on the node.
>> It is clearly visible that core server/layer 0 is the heart of freenet,
>> where all the anonimity and encryption takes place, and main developers
>> could concentrate work there. The client part is independent and can be
>> developed in pararell.
>
> We can separate layer 7 apps (e.g. fproxy, which is likely to be a very
> thin shell over the layer 6 API) into a separate code module, if there
> is demand for this. We can provide a java interface to layer 6 which can
> be either satisfied by FCP or internally in the node's VM.
>
> Layer 6 is fairly core stuff IMHO, but people are welcome to hack on it.
> Most of it is in freenet.client.
>>
> Matthew J Toseland - toad at amphibian.dyndns.org
Roman Bednarek
