http://127.0.0.1:8888/freenet:USK at
1dpEDA89y9cCqLtNuVY9B~NdGSQHL2crtpRvqSTZtDs,Vwka6Ah7dUUYi85gmxR64zqMFjtIQVi4wsQE0ujlmm0,AQACAAE/nowwhat/18/04-01-2010_04-30-2010.html
Archive for April 2010
? March 2010
Plug It In
Posted on Thursday, April 22, 2010
There comes a point with Freenet where if you have a developer mindset and you
start poking around, you'll eventually start overdosing on ideas. There's just
so much potential for what could be done with Freenet that there just isn't
enough time to work on it all. I've found myself tinkering with so many things
lately that eventually I'm going to end up with a hard drive full of half
finished ideas. Freenet needs a good email system, a good forum system, a near
realtime chat/IM system, a good file sharing interface, freesite creation
software, a decentralised wiki and that's just the basics that people are used
to. Beyond that there's the unique stuff that uses Freenet to do things that
simply can't happen on the conventional web.
When I returned to Freenet after some time away I think what encouraged me the
most was some of the plugins that were being developed. In theory it should be
easy to expand Freenet to do all the above mentioned things in a nicely
integrated front end, without worrying about different UIs and runtime
environments. Unfortunately, like a lot of things in Freenet the plugin
interface isn't particularly well documented, but I was pleased to see another
recently returning FreenetUser detailing his adventures with creating a Freenet
plugin. It's all very interesting and encouraging. Now all I need to do is
relearn Java so I can spend some quality tinkering time with the plugin
interface. It's surprising how much you forget after spending some years away
using languages substantially less fiddly than the sprawling beast that is Java.
As I understand it, Google summer of code students have recently started work
on Freenet related projects. I couldn't find out what exactly they are working
on, but hopefully some kind of finished product will pop out at the end of it
all. I believe GSoC students have made significant contributions to Freenet in
the past in terms of less visible, but nonetheless essential, code
improvements, but I think Freenet needs some killer apps rather more than a
2.65% improved routing algorithm at the moment.
A new build of Freenet is due soon and maybe even FreeTalk will make an
appearance and the final battle between Frost, FMS and Freetalk can get
underway. Here's to hoping for an interesting summer of Freenet.
Posted in Misc
Living in Realtime
Posted on Saturday, April 17, 2010
The lack of updates recently has been mostly due to me spending all my time
tinkering with Freenet. In my quest to get the Linkageddon updater working as
smoothly as possible I've been trying to see how much I can squeeze out of
Freenet. Along the way I came across something which although I'd previously
been aware of them, I'd never really grasped the significance of them before.
The things I discovered are ULPRs or Ultra-Lghtweight Passive Requests.
If you create any kind of app for Freenet when you are downloading data you are
either trying to fetch keys you know should be there, or you are trying to keep
track of key slots where you know data should be inserted by somebody else at
some later time. Waiting on keys to be inserted is how messaging apps work.
Frost, FMS, Freetalk etc will spend most of their time waiting for keys to
appear in prearranged slots. It used to be the case that for an app to know
when those slots were filled it would have to periodically launch a fresh
request in those slots, often polling sequentially over a wide range of slots.
This is most evident when you observe how Frost works in automatic update mode.
Now however, with ULPRs this is no longer required as all an app has to do is
tell the node it wants to know when a key is inserted in a particular slot and
the node will simply tell it when something pops up there. That's clever and
useful enough but the really interesting thing about it and the thing that took
me by surprise is just how well it works.
I threw together some test scripts to see how fast an ULPR would detect some
small keys being inserted from another node and was shocked when in most cases
it would detect the inserts in under one minute, sometimes in as little at 10
seconds. I was used to keys taking a long time to propagate through Freenet so
this was very surprising. I figured it must be some fluke of the way my nodes
were set up so I looked for some other way to see what was going on.
Eventually I realised Frost would make a good test for this as the Frost
messages are easy to process (just ask the spammers), include time stamps and
some of the public boards are busy enough to generate a fairly regular stream
of inserts. So I set my script running and watched the results from tracking 11
of the busiest boards.
I hadn't been running the tests for long before it became apparent my initial
results were not a fluke after all and ULPRs really are fast. Even allowing for
clock skews and Frost insert holdups most messages seemed to be detected within
2-5 minutes of their time stamps. Some came in even faster.
Until now I'd just got used to the idea that if I insert a message into Freenet
it's going to take a while for it to reach its destination, now I know the only
reason communication on Freenet is slow is because the apps haven't been
written to take full advantage of Freenet's features. I had a quick look at the
Frost source to see if it could be modified to run in permanent update mode and
hence update in near realtime, but from what I saw it would take some serious
hacking to do it properly.
I've read of Freenet developers mentioning the possibility of realtime Freenet
communication before, with services equivalent to IRC,Twitter and perhaps even
VOIP, but thought that seemed unlikely. It now seems like all it needs is
somebody to put the pieces together. There are a few freesites that have early
apps designed on this basis, but they seem to have either been abandoned (like
a lot of really promising Freenet projects unfortunately) or have some serious
design flaw.
I can only hope some talented developer can put together some decent realtime
communication apps. As I understand it most of the main Freenet developers do
their work via IRC. If we had something similar in Freenet perhaps they could
be coaxed down to our level. Unfortunately, the last time I tried to use
Freetalk, which seems to be the great hope of Toad and co, it seemed to be more
interested in downloading messages from 9 months ago than what was happening
now. Now that I know what is possible, if Freetalk does not have a fairly fast
message turnaround I'm not even going to waste my time on it again.
I look forward to the day when a group of users can have realtime and lively
discussions in Freenet about things that are happening right now and not 12-24
hours ago. The use of Freenet as a repository of forbidden data is fine, but
realtime communication and discussion will give it the beating heart that is
more likely to attract the kind of people to Freenet who will invigorate the
network and bring real innovation.
Posted in Freetalk, Frost, Misc
Definition of Frustration and Other Stories
Posted on Tuesday, April 13, 2010
My last entry gained a response via FMS. The Seeker explained (direct FMS link)
how large splitfiles work and why it's common to get yourself in situations
where you may be short only a few parts on a large download. I was always under
the impression you only needed a certain percentage of all the thousands of
chunks inserted to reconstruct the original file, but apparently it's much more
complicated than that. A large file is actually broken up into 4MB chunks. The
chunks themselves use forward error correction to recover them, but if one
chunk fails the whole download fails.
There are apparently ways around this, but they are complex. I only hope a good
solution is found and implemented directly in Freenet rather than requiring
somebody to come up with an ad-hoc solution that requires a separate Freenet
app or plug-in to use. On the bright side though large files that have been
recently inserted or are in high demand still seem to be very reliable so
Freenet is still suitable for handling 0-day style content.
In other news, a war of words has broken out between Freetalk and FMS
developers and supporters. With Freetalk, which was supposed to be the
officially supported Freenet forum system, being long delayed many feel FMS
should be bundled into Freenet in its place. Toad seems concerned about the
issues involved in bundling the C/C++ FMS client and the time factor involved
in reviewing the code. The FMS supporters are concerned about some of the
secrecy surrounding the specification and development of Freetalk.
Personally I think Freetalk looks to be a long way from being ready for
primetime, but I can understand the concerns about FMS. It's not so much that I
distrust the author of FMS, but I think any project can benefit from at least a
few fresh pairs of eyes looking through it's code for anonymity leaks. From
what I understand, there have been problems in the past with HTML filtering.
Thankfully a dialog is developing between the FMS and Freetalk groups which I
hope will lead to some positive outcomes.
In response to SomeDude's FMS post Toad posted this to a mailing list:
p0s is of the view that completing FMS to the point where it could be
bundled is much less work than either reviewing FMS (which I would have
considerable difficulty with on past experience, it's a lot of code and some of
it is old hard to review low level C code that has had e.g. format string
vulnerabilities), or cloning it in a new plugin.
Hopefully Freetalk will be done soon. If it is not we should reconsider.
Bundling FMS would be problematic on a practical level - in that we'd have to
bundle binaries for multiple platforms (requiring building them), figure out
how to update them, and have them run as daemons as Fred does - but the main
issue is that the code would need to be reviewed.
p0s has agreed to complete writing a spec after Freetalk works reasonably
well. Currently there is this:
http://wiki.freenetproject.org/Freetalk
Also, evand chimes in on his flog on this issue and I think we can clearly see
a them and us communication problem that just doesn't seem to want to go away.
Those of us trying to stay anonymous and use Freenet in the way it is intended
are struggling to communicate with many key Freenet developers who often appear
either unable or unwilling to actually use Freenet to communicate with people.
I understand that communication outside of Freenet is a hell of a lot easier,
but the fact it's so hard in Freenet is not something we can just keep trying
to route around. If something this important is so difficult it means it needs
to be solved very soon and as a high priority.
Edited on: Tuesday, 13 Apr 2010
Posted in Misc, News
Definition of Frustration
Posted on Sunday, April 11, 2010
Not too long ago, at the end of last month, I mentioned I was trying to
retrieve a 700MB file from Freenet. All had been going well and I was at 90%.
Slowly but surely it crept closer to the magic 100% and then it all went
horribly wrong.
Currently, no new chunk has been retrieved for 4 days and of the 22,124 chunks
I need to put it all together I am short by just 5. I have no idea whether
Freenet has now given up or if it's actually trying all those chunks it
couldn't find the first time around. Will the download ever complete? Stay
tuned and maybe I'll have it all by this time next month. Just as well I wasn't
actually interested in the file. I just wanted to see if I could get it all.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL:
<https://emu.freenetproject.org/pipermail/devl/attachments/20100423/0687f6b6/attachment.pgp>
