Wow, I really didn't mean to make a reply this long, but there's a lot
of content to cover (and when I post messages on Freenet and link
pointers to them here on the list, they're too easy to ignore ;)
You bring up a lot of good questions and answer a lot of mine, but before
addressing them, let me just pull out and explain an assertion I made -
"Once FRED supports JMS, it is likely that no other transports will be
introduced down the line (other than possibly IPC), as JMS should fill
all of those transport needs without need for modifying FRED"
You're right to be warry of such a statement, but let me explain why I
stand behind it. JMS is just an API to an arbitrary message oriented
transport mechanism. It takes what Fred has done so gracefully with
the Address, Transport, Connection, ListenerAddress, and Listener classes
and essentially factors out their implementation into the MOM. The MOM
then has all of the responsability to deliver a message sent through a
connection.getOut() to whatever is supposed to be on the other side. The
MOM has the responsability to deliver that message - those bytes - to the
intended recipient through a connection.getIn(). Thats all Fred needs to
know - that it wrote those bytes to the OutputStream, and that those bytes
are recieved by the intended target node via the InputStream. At that
point, Fred then jumps in and does its FnpLink stuff.
How the MOM implements that is up to the MOM. For instance, it could be
set up with three routers (more info on what a router is shortly):
Fred1 <---> Router1 ---> Router2 <---> Router3 <---> Fred2
^local tcp ^ polling ^ email via ^ local tcp
HTTP UUCP
This toplogoy maps a user in a secure organization with extremely
limited/costly bandwidth (HTTP proxy to the intranet and UUCP email
to the outside world).
How that particular network could be set up would be the end user
(Fred1) had Router1 installed on their own machine (bundled install,
whatever). Router2 is set up (covertly?) for all of the users within
that organization outside the HTTP proxy. Router3 is set up on the
user of Fred2's machine. Messages are passed bidirectionally (both
Fred1 and Fred2 just treat their connection.getIn() and .getOut() like
normal TCP/IP Socket connections), but behind the scenes, the messages
are passed through an HTTP proxy and over a dialup UUCP modem link before
being delivered.
In a less exotic scenario:
Fred1 <---> Router1 ---> Router2 <---> Fred2
^ local tcp ^ polling ^ local tcp
HTTP
This has two users (Fred1 and Fred2) with the bundled MOM router installed
on their local machines, but Fred1 is behind a corporate firewall and
HTTP proxy. While normally, Fred2 would not be able to contact Fred1,
Router1 could send an HTTP POST to Router2 every 30 seconds or so,
"uploading" any messages Fred1 wants to send to Fred2, and "downloading"
any messages Fred2 wants to send to Fred2. As always, Fred1 doesn't need
to know more than its connected to Fred2, and visa versa.
And, as the baseline scenario:
Fred1 <---> Router1 <---> Router2 <---> Fred2
^local tcp ^ tcp ^ local tcp
This is unnecessary for Fred, as it already iplements the tcp transport, but
this provides another implementation of it.
Why am I confident that this lays the infrastructure for any possible network
topology and factors out the implementation of that topology from Fred?
Because I can't think of a network setup that this couldn't be implemented
over (but I'd love to hear of one!). MOMs (and Java's API to MOMS - JMS)
don't solve world hunger, nor do they provide the anonymity or data management
that Freenet has, but they do implement message delivery really really well,
and are used in huge diverse networks.
In my opinion, we need Freenet to run in hugely diverse networks. For speech
to be free, we need to be able to publish and read when we want, where we
want. Unfortunately, for security reasons, the corporate and university
world has shut down transparent TCP/IP access in favor of firewalled, proxied,
and monitored networks, and ISPs are following suit.
>From an OOAD perspective, is the core concept behind Fred the ability to
deliver messages, or is its primary goal to provide anonymity for distributed
communication? If possible, why not use an open API like JMS (or its
equivilant in other languages for accessing MOMs), and leave all of the
transport implementation details up to applications whose only goal in life
is to deliver messages?
You do raise some important questions, so I'll get to those now:
> * Do JMS providers already provide the transports you list?
Short answer: some of them provide some of them. None of the free ones
provide message delivery over email, ftp, or UUCP (though I've used
commercial ones that do). None of them provide stenography or use Freenet's
Silent Bob defense. However, with jbossMQ or openJMS, we can pierce firewalls
and get through NATs by using either bidirectional or polling HTTP as a
transport as well as jbossMQ's UIL.
Most importantly, both jbossMQ and openJMS are open source (JBoss uses GPL and
openJMS uses a BSD style license). To address other toplogies, we merely
(understatement of the year) need to go to one of those JMS implementations
and figure out how to deliver the message across that particular scenario,
rather than think about its implications for Fred's encryption, routing tables,
or what have you.
> * "Polling HTTP" is insufficient. We need arbitrary other who have been
> passed our node's address to be able to connect to us.
I disagree. Sure, it'd be great if Machine X could always talk to Machine Y,
but sometimes there just isn't a bridge to pass messages across. And sometimes,
that bridge is obscure, requiring messages to pass through several gateways in
a very peculiar manner. Yes, polling HTTP isn't sufficient for all network
topologies, but it does address some of them - Machine X behind an HTTP proxy
and firewall, for instance. As long as the MOM promises to do its best to
deliver the message, Fred can treat aggressively secure network like simple
point to point Input- and OutputStreams.
I guess now would be a good time to explain that I don't forsee a giant mesh
network of MOM routers handling the delivery of messages. Instead, they would
be used in a point to point fashion, with intermediary routers placed only
for those really gnarly networks. As in, the normal case would be my router
connecting to your router.
> * We do not on the whole give a damn about J2EE etc. We use java because
> we have used java from the beginning, and because it is a reasonable
> compromize.
Thats the right attitude to take. J2EE is bloated and and almost always
unnecessary. However, JMS is an open API providing transport independent
asynchronous message delivery, which seems like a good way to seperate
Fred's low level message passing code from its anonymnity, routing and
data caching functionality. I'm not suggesting JMS because "its neat",
I'm suggesting JMS because I've used it to deal with situations that
I think Freenet needs to deal with.
> Is OpenJVM available to other languages that fred might be
> reimplemented in?
Thats a good question. OpenJMS specifically doesn't have a C or Python
API, as its a Java MOM implementation (though it can be contacted through
its wire level protocol from Python, C, Perl, etc). JMS in general is
only Java - its the Java API to MOMs. MOMs however have a hearty history
dealing with other languages - JMS was only added to MOMs in the last 5 years
or so. IBM's MQseries has at least a C API, and I'm sure Microsoft's
MSMQ h
as a .NET API.
The key here is that with Fred supporting JMS, it could plug in whatever
MOM implementation it wanted without touching a line of code (except
for the configuration files, and as long as that MOM provided a JMS
API).
> * "More storage capacity due to more nodes being able to participate in
> the network" is probably a negligible effect.
Having thousands of nodes running on computer labs in universities with
all of their disk space wouldn't help? I know those nodes wouldn't
"pull" much data, but they'd store plenty, and would usually have
reasonably fast connections.
> * What are these "JMS routers"? Are they a big fat semi-central point of
> failure, can they be used to detect and connect to all nodes in a
> given network, do they need to be set up independantly of fred?
The JMS spec doesn't require the JMS implementation to sit in seperate JVMs,
so in theory Fred could have its own in memory router, but in practice,
all of them do (at least the ones I've seen). JMS routers, or bridges,
are called that so that you think of them exactly as you think of normal
network routers and bridges - they pass data from one network to another.
Some are more intelligent than others and do complex message routing, others
simply bridge whatever data they recieve to the other side.
They could be set up to create a huge WAN for passing messages across all
of Freenet, but thats not what we want - we merely want lots of small, wholey
independent point to point links to be created between nodes. JMS leaves
how messages are delivered completely up to the JMS implementation.
> * We also intend to use mixmastered first two hops to reduce the amount
> that a hostile node can learn about a node from its messages and their
> Hops-To-Live values, amongst other things. This may be before 1.0.
Awesome. This is another reason why I think it'd be worthwhile to put this
kind of functionality into a JMS implementation rather than have it be part
of Fred - OTHER applications should would be able to use this really awesome
feature (sending out mixes for message delivery). I don't know the details
of how you envision this working, and it may be a truly Freenet specific
thing, in which case it shouldn't go into a JMS implementation, but if its
possibly relevent for general purpose message deliver, keeping it outside
of Fred would allow for more rapid reuse (application reuse instead of
component or code reuse).
> * I very much doubt that it will generate many more users.
I know its a small pool, but lots of people on IIP say they'd love to use
Freenet, but they can't as they're behind a firewall or NAT. I'm not
at Uni now, but I remember that nearly every computer lab and dorm was
at least one way firewalled (outbound only). Wouldn't the students doing
their time in school be interested in such a useful project, either for
political reasons (college students generally being more politically active
than people "in the workforce"), or for research reasons (as Ian points out
in his talk, Freenet is ripe with unanswered questions)?
Beyond that too, from a more blackhat vantage point, wouldn't placing
Freenet into hostile networks (either with or without authorization) allow
Freenet to deliver more content more quickly in a more distributed manner?
> * Additional transports are a post-1.0 thing, but you are welcome to
> experiment. You will need to maintain a CVS branch for them however
> since we will not incorporate other transports in 1.0.
Thats fine with me. I'm sure there's a lot scheduled and plenty of things
that are really essential to the core "just get it to work" 1.0. Getting
Fred working over JMS and making sure a JMS implementation can provide the
necessary network bridging functionality won't be done tomorrow either.
> * Safely jumping over NATs that cannot be reconfigured is definitely a
> post-1.0 issue and may be very difficult, for various reasons, the
> main one being that we absolutely must be able to accept connections
> from any node that has obtained our reference.
I may be beating a dead horse here, but while we absolutely must be able to
receive messages, we don't necessarily have to receive inbound TCP/IP
socket connections.
If you've made it this far, especially without shouting too many
expletives at me, thanks. This isn't a short term kludge, and I don't
want to rush it. There is meat to what I've proposed, but it may or may
not be what Freenet's looking for. Or there may be better ways to go about
it. There are implementation details I've glossed over for brevity (maybe
I should have just pasted the code instead), and I'm completely open to
discussing all of this further. I'll keep the list apprised as things
progress as well, but only when its relevent.
-jrandom
!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+
CryptoMail provides free end-to-end message encryption.
http://www.cryptomail.org/ Ensure your right to privacy.
Traditional email messages are not secure. They are sent as
clear-text and thus are readable by anyone with the motivation
to acquire a copy.
!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+!+
_______________________________________________
devl mailing list
[EMAIL PROTECTED]
http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
