Send Devl mailing list submissions to
devl at freenetproject.org
To subscribe or unsubscribe via the World Wide Web, visit
http://www.uprizer.com/mailman/listinfo/devl
or, via email, send a message with subject or body 'help' to
devl-request at freenetproject.org
You can reach the person managing the list at
devl-admin at freenetproject.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Devl digest..."
Today's Topics:
1. Re: Please Examine 0.3.7 (Mr.Bad)
2. Re: Please Examine 0.3.7 (Steven Hazel)
3. Re: Please Examine 0.3.7 (Steven Hazel)
4. Re: [Freenet-dev] Intermittent booster leaf (Peter Todd)
5. Re: Killing Freenet (Re: [freenet-devl] Aardvark) (Peter Todd)
6. Re: HAR! (Peter Todd)
7. Re: 0.3.7 (Sebastian Spaeth)
8. Re: Announcement Protocol (Theodore Hong)
9. Re: Killing Freenet (Re: [freenet-devl] Aardvark) (Theodore Hong)
10. Re: Killing Freenet (Re: [freenet-devl] Aardvark) (Theodore Hong)
11. Re: Re: Simulations... (Theodore Hong)
12. Re: Re: Simulations... (Sebastian Spaeth)
13. Re: Killing Freenet (Re: [freenet-devl] Aardvark) (Oskar Sandberg)
--__--__--
Message: 1
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Please Examine 0.3.7
From: Mr.Bad <[email protected]>
Organization: Pigdog Journal
Date: 03 Feb 2001 18:29:02 -0800
Reply-To: devl at freenetproject.org
>>>>> "SH" == Steven Hazel <sah at thalassocracy.org> writes:
SH> .jar files are .zips -- you can use PKZIP to unpack them (or
SH> to create them).
Could the default compression be different? I dunno.
~Mr. Bad
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/\____/\ Mr. Bad <mr.bad at pigdog.org>
\ / Pigdog Journal | http://pigdog.org/ | *Stay*Real*Bad*
| (X \x)
( ((**) "If it's not bad, don't do it.
\ <vvv> If it's not crazy, don't say it." - Ben Franklin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--__--__--
Message: 2
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Please Examine 0.3.7
From: Steven Hazel <[email protected]>
Date: 03 Feb 2001 20:34:58 -0600
Reply-To: devl at freenetproject.org
Sebastian Spaeth <Sebastian at SSpaeth.de> writes:
> No, I alway WinZip my freenet.jar with max compression and gzip
> isn't that much better.
Is that "gzip -9" (max compression, to match your WinZip setting) or
just "gzip"?
> I rather supect that already compressed files have a bad influence
> pn file size when embedded in another zipped file.
They'll lower the compressed size / original size ratio, because
compressed data won't compress much (if at all), but they shouldn't
have much effect on the compression of the rest of the data.
-S
--__--__--
Message: 3
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Please Examine 0.3.7
From: Steven Hazel <[email protected]>
Date: 03 Feb 2001 20:37:25 -0600
Reply-To: devl at freenetproject.org
"Mr.Bad" <mr.bad at pigdog.org> writes:
> >>>>> "SH" == Steven Hazel <sah at thalassocracy.org> writes:
>
> SH> .jar files are .zips -- you can use PKZIP to unpack them (or
> SH> to create them).
>
> Could the default compression be different? I dunno.
Could be.
--__--__--
Message: 4
From: Peter Todd <[email protected]>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] [Freenet-dev] Intermittent booster leaf
Date: Sat, 3 Feb 2001 23:25:45 -0500
Reply-To: devl at freenetproject.org
On Sat, 03 Feb 2001, you wrote:
> This would be useful in three ways, one is that this makes the extra storage
> and
> bandwidth capacity of intermittently connected machines available, people who
> want to keep certain information always available can store it on their own
> hard
> drives, never deleting it and in pro censorship areas, the permanent nodes
> could
> rely entirely on the intermittent leaves for data, storing none on their own,
> so
> they wouldn't have any objectionable data on them. Censors would be forced to
> go
> after more individuals, rather than a few high bandwidth servers.
However the permanent node still know who the leaves are and are
therefore able to backstab the leaves should the permanent node be
compromised.
This idea has already been thought of and discussed before. IIRC the
consencous was that the idea was feasable if all this was done
automaticly, with few nodes in any individual group (no more then say
10 nodes in a group) to reduce the sabotauge any compromised node
(the main objection to this idea) could cause and with many other
checks and balances.
In any case it's not gonna be implemented for awhile, there are
other, more pressing issues at hand.
--
retep at penguinpowered.com http://retep.tripod.com
--__--__--
Message: 5
From: Peter Todd <[email protected]>
To: devl at freenetproject.org
Subject: Re: Killing Freenet (Re: [freenet-devl] Aardvark)
Date: Sat, 3 Feb 2001 23:35:12 -0500
Reply-To: devl at freenetproject.org
On Sat, 03 Feb 2001, you wrote:
> > > According to www.octayne.com, there are 100 working Freenet nodes at
> > > least (undoubtedly many more since the limit is 100 nodes).
> > How many of those Freenet nodes are *really* working? Far, far lower
> > I suspect given that my node, which does get well established in the
> > network, rarely has references to more then 3 nodes at a time. Even
> > if I force a load of a pile of references through nodes.config it
> > takes only an hour or two before it weeds out %95 of the nodes.
>
> They should all be working since the inform.php script regularly tests
> them by opening and closing a TCP connection.
Opening and closing a TCP connection will work on a node that's
almost dead due to other reasons. (bugs)
> > I've also noticed that the set of references that do exist often
> > completely changes as nodes are shutdown on a pretty regular, once
> > every few days, bases.
>
> Such nodes shouldn't get into the inform.php list.
No. Nodes send their information to inform.php after 24 hours. I'm
talking about time spans of about 3-5 days. Plenty of time to get
onto inform.php
In fact wouldn't a highly stable node that never restarts be *less*
likely to be on inform.php then a node that regularly restarts every
25 hours, just long enough to send it's information to inform.php?
Eventually nodes are dropped from inform.php so it stands to reason
that nodes that are always reinserting themselves into inform.php
will have a higher chance of being on there. I delibrately set my
nodes informDelay to 0 and restart it every 24 hours for this reason
and to stop memory and hd leaks. (garbage collection isn't perfect)
> > > Someone was doing some experiments on Freenet's reliability over time -
> > > did anything come of that?
> >
> > I was going to but in the end never did after hearing tales of horrid
> > reliability from another person.
>
> Shame - I actually get pretty darn good reliability, I recently
> downloaded a "Homer Simpson" mp3 which has been on the key list for
> ages, and probably wasn't really requested very much.
>
> I think that the problem is that most content isn't being requested
> enough - and that is because there is no content on the system that is
> really drawing people to use it on a regular basis. This is a catch-22
> situation which will hopefully be addressed when, say, Freenet is
> included in Debian-unstable and forms a viable mechanism for downloading
> new Debian releases. This really needs to wait for a GCJ-compiled
> binary version of Freenet which seems to be getting closer by the day.
Well I just started that retrievial test I planned yesterday. The
results? Not good.
(all files 24 hours old)
data size - hops needed to find
1 - 2
256 - 3
1024 - 2
1meg - 15 (and unreliably at that, only sometimes will it even get to
TRANSFERING) and freezes at TRANSFERRING due to a bug somewhere nor
will it even download from my local node, "Unexpected end of stream"
This reminds me of another issue... Are there mechanisms in place
where slightly buggy nodes that fail on some data will be eliminated
from the reference lists? I think any failure that was probably
causeed by a bug should be counted against the node in such a way
that the same peice of data that caused the problem isn't going to be
requested from that node again. So remove whatever reference got us
to that node immeidiately. Doing this should help out a lot with the
problem of data being consistantly requested from nodes that can't
provide it due to bugs.
Of course this does assume our client is non-buggy but we can hope
can't we?
--
retep at penguinpowered.com http://retep.tripod.com
--__--__--
Message: 6
From: Peter Todd <[email protected]>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] HAR!
Date: Sun, 4 Feb 2001 00:02:16 -0500
Reply-To: devl at freenetproject.org
On Sat, 03 Feb 2001, you wrote:
> OK, whoever updated KSK at KeyIndex.txt -- very clever. *I* am definitely
> convinced.
For whoever updated KSK at KeyIndex.txt -- How did you do it? Pure
flooding or something more tricky?
Hmm... That's something I should be testing... How long it takes for
data to be overwritable, as opposed to requestable.
--
retep at penguinpowered.com http://retep.tripod.com
--__--__--
Message: 7
Date: Sun, 04 Feb 2001 09:48:53 +0100
From: Sebastian Spaeth <[email protected]>
Organization: University of =?iso-8859-1?Q?Link=F6ping?=
To: devl at freenetproject.org
Subject: Re: [freenet-devl] 0.3.7
Reply-To: devl at freenetproject.org
"Mr.Bad" wrote:
> One that I can't figure out is how not to clobber existing config
> files. The .deb doesn't do it, but I don't think we can make a .rpm
> not do it. One option is to have an install script in the tar file
> that wouldn't clobber, but I don't think we want to wait on that for
> 0.3.7.
I could imagine one way without having to use install scripts. We can
include code in the node, that when loading the params, detects if no
config file could be found. In case it doesn't it will just call
"Setup.java .freenetrc silent" and create a default config file on its
first run. Problem solved, in all environments and all distributions...
We might want to check if the environment is a Windows environment and
use freenet.ini in this case, but that should be possible in Java (I
guess).
Sebastian
--__--__--
Message: 8
From: Theodore Hong <[email protected]>
Subject: Re: [freenet-devl] Announcement Protocol
To: devl at freenetproject.org
Date: Sun, 4 Feb 2001 10:47:39 +0000 (GMT)
Reply-To: devl at freenetproject.org
Oskar Sandberg <md98-osa at nada.kth.se> wrote:
> On Fri, Feb 02, 2001 at 08:01:32PM +0100, Ruediger Kapitza wrote:
> > Whats wrong?
>
> She has no more reason to trust the nodes she got from Bob's store than
> the node he sends it to. Freenet does not have a node trust system,
> trying to pretend it does will not solve the problems caused by that.
>
> Besides, the point is that Alice is does not decide where the announcement
> goes. And this misses the important role of the introduction as routing
> space harmonizer, the nodes she get's from Bob1 would be effected by the
> part of the keyspace he is biased toward, which may not be close to the
> established value. And your showing the list of nodes to all the nodes,
> not just Alice. I could go on...
also, Bob doesn't necessarily trust Alice and might not want to tell her
what's in his datastore.
--__--__--
Message: 9
From: Theodore Hong <[email protected]>
Subject: Re: Killing Freenet (Re: [freenet-devl] Aardvark)
To: devl at freenetproject.org
Date: Sun, 4 Feb 2001 10:50:37 +0000 (GMT)
Reply-To: devl at freenetproject.org
Ian Clarke <ian at octayne.com> wrote:
> On Sat, Feb 03, 2001 at 03:15:43AM +0100, Oskar Sandberg wrote:
> > I'm quite aware of the fact that the network is not working very well, and
> > I'm quite pesimistic about whether it ever will, but there is really no
> > point in being here at all if not operating under the assumption that we
> > will get it to work some time...
>
> It is annoying, simulations indicate that a network should have about
> 98% reliability and search-times of under 10 hops.
well, but what size network? that makes a big difference. A sweeping
statement like "98% reliability" doesn't mean much unless you say under
what conditions.
theo
--__--__--
Message: 10
From: Theodore Hong <[email protected]>
Subject: Re: Killing Freenet (Re: [freenet-devl] Aardvark)
To: devl at freenetproject.org
Date: Sun, 4 Feb 2001 11:03:44 +0000 (GMT)
Reply-To: devl at freenetproject.org
Ian Clarke <ian at octayne.com> wrote:
> On Fri, Feb 02, 2001 at 06:40:44PM -0800, Mr.Bad wrote:
> > Really? Do you think there's anything fundamentally wrong with the
> > architecture? Or do you think it's just a matter of the current set of
> > nodes?
>
> Simulations suggest it is not a problem with the current architecture
> (although simulations are rarely 100% accurate). I think the problem is
> that not enough people are actually *using* Freenet right now
I think this is a big factor. From the simulations I did, if the network
grows too quickly without people making enough requests, it works ok up to
a certain size, but then breaks down quickly. The problem is that nodes
need some time and practice to adjust their datastores properly, since they
start off just making random guesses. That's ok if the random route
quickly reaches the old nodes who know what they are doing. But if the
network grows too quickly, then the random routes mostly go to other
nearly-new nodes, who just make more random guesses, and it never gets
anywhere.
Pictorially:
<< old nodes >> -- new nodes
(well-connected) (poorly-connected)
Requests to the new nodes bounce around randomly for a bit, but then reach
the old nodes and get routed correctly.
<< old nodes >> -- newish nodes -- new nodes -- very new nodes
(well-connected) (... poorly-connected ...)
Requests to the new and very new nodes bounce around randomly forever and
never make it to the old nodes.
This is exacerbated by inform.php, since it strings new nodes out in a long
chain out to the right, until the newest nodes get very far away indeed
from any nodes with a clue. What the node announcement protocol does
(which accounts for its improvement) is more like this:
new nodes
|
|
new nodes -- << old nodes >> -- new nodes
|
|
new nodes
theo
--__--__--
Message: 11
From: Theodore Hong <[email protected]>
Subject: Re: [freenet-devl] Re: Simulations...
To: devl at freenetproject.org
Date: Sun, 4 Feb 2001 11:10:27 +0000 (GMT)
Reply-To: devl at freenetproject.org
Sebastian Spaeth <Sebastian at SSpaeth.de> wrote:
> Ian Clarke wrote:
> > Simulations suggest it is not a problem with the current architecture
> > (although simulations are rarely 100% accurate).
>
> I have to admit that I am quite suspicious about the ability of
> Simulations since I took a course covering exactly that topic at
> University. They do have their use, but conducting and designing
> simulations where we have that many unknown variables that can/will
> influence Freenet reliability will lead to *very* inaccurate results.
>
> Unknown variables (e.g. the statistical distributions) that cannot be
> found out or used correctly in simulations, but will influence
> routing/reliability, might be:
>
> - up/downtime of Freenet nodes which will depend on the future user base
> (modem, cable, Windows/Linux,...)
> - connection speed of Freenet nodes depends on user base as well
> - size of disk space dedicated to Freenet nodes (will people spend 10MB
> or 1GB to Freenet)
> - user behavior (will people split up big files into 1000 file chunks or
> insert them as big files (might be determinated by popular client
> bahavior)
> - percentage of transient vs. nontransient nodes
>
> How are these factors considered in the current simulations, if at all?
oh, absolutely. I have not accounted for these factors at all in my
numbers. The problem is that we do not have any model for user
characteristics, which is what all of your factors are. (Maybe we need a
user survey to try to collect some data?)
theo
--__--__--
Message: 12
Date: Sun, 04 Feb 2001 12:36:59 +0100
From: Sebastian Spaeth <[email protected]>
Organization: University of =?iso-8859-1?Q?Link=F6ping?=
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Re: Simulations...
Reply-To: devl at freenetproject.org
Theodore Hong wrote:
> > > Simulations suggest it is not a problem with the current architecture
> > > (although simulations are rarely 100% accurate).
[snip]
> > Unknown variables (e.g. the statistical distributions) that cannot be
> > found out or used correctly in simulations, but will influence
> > routing/reliability, might be:
[snip]
> oh, absolutely. I have not accounted for these factors at all in my
> numbers. The problem is that we do not have any model for user
> characteristics, which is what all of your factors are. (Maybe we need a
> user survey to try to collect some data?)
Not too sure about surveys either as I think persons interested enough
in Freenet now, might not be the typical user base which uses Freenet
later. Nevertheless it could be interesting to set up a Sourceforge
survey and let people specify their characteristics to get a rough
impression.
I just wanted to make clear that not everything can be mathematically
proved or simulated in a reliable and validate way as Freenet
reliability will e.g. depend on (unknown and unmodeled) user
characteristics/behavior as well.
This makes comparing proposals and suggestions quite hard as you might
get (mathematical) precise results which are totally misleading (in a
nonobvious way).
Sebastian
--__--__--
Message: 13
Date: Sun, 4 Feb 2001 13:52:00 +0100
From: Oskar Sandberg <[email protected]>
To: devl at freenetproject.org
Subject: Re: Killing Freenet (Re: [freenet-devl] Aardvark)
Reply-To: devl at freenetproject.org
On Sun, Feb 04, 2001 at 11:03:44AM +0000, Theodore Hong wrote:
> Ian Clarke <ian at octayne.com> wrote:
> > On Fri, Feb 02, 2001 at 06:40:44PM -0800, Mr.Bad wrote:
> > > Really? Do you think there's anything fundamentally wrong with the
> > > architecture? Or do you think it's just a matter of the current set of
> > > nodes?
> >
> > Simulations suggest it is not a problem with the current architecture
> > (although simulations are rarely 100% accurate). I think the problem is
> > that not enough people are actually *using* Freenet right now
>
> I think this is a big factor. From the simulations I did, if the network
> grows too quickly without people making enough requests, it works ok up to
> a certain size, but then breaks down quickly. The problem is that nodes
> need some time and practice to adjust their datastores properly, since they
> start off just making random guesses. That's ok if the random route
> quickly reaches the old nodes who know what they are doing. But if the
> network grows too quickly, then the random routes mostly go to other
> nearly-new nodes, who just make more random guesses, and it never gets
> anywhere.
It seems to me that the datastore filling in the proposed introduction
procedure should more or less solve this problem...
<>
--
'DeCSS would be fine. Where is it?'
'Here,' Montag touched his head.
'Ah,' Granger smiled and nodded.
Oskar Sandberg
md98-osa at nada.kth.se
--__--__--
_______________________________________________
Devl mailing list
Devl at freenetproject.org
http://www.uprizer.com/mailman/listinfo/devl
End of Devl Digest
