[freenet-dev] something we haven't seen for a long time...

[Zlatin Balevsky]

Histogram of requested keys.
This count has nothing to do with keys in your datastore
Dec 1, 2003 1:39:01 AM
keys: 478
scale factor: 0.4383561611175537 (This is used to keep lines < 64 characters)
  0 |===
  1 |==
  2 |
  3 |==
  4 |===
  5 |
  6 |=
  7 |==
  8 |==
  9 |
  a |===
  b |===
  c |=
  d |=
  e |
  f |=
peaks (count/mean)
0 --> (0.30125523)
4 --> (4.887029)
b --> (0.53556484)
f --> (0.43514645)
:)))



pgp0.pgp
Description: PGP signature
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

RE: [freenet-dev] stable net kicking ass

[Pete]

Just looked at my nodes stats before I close it down for the time being
and I noticed this figure
Outbound connections that are to peers not in the routingtable
20.37037% 

Why aren't these peers being added to the routing table? They obviously
are capable of dealing with request which 
Thses nodes >
Node references requesting ARKs 46 

Obviously aren't, I know churning the rt is not exactly a great thing to
do, but I think it may help in the long run if we drop nodes that really
are behaving badly for ones that are working, just a suggestion...

Pete

>>-Original Message-
>>From: [EMAIL PROTECTED]
>>[mailto:[EMAIL PROTECTED] On Behalf Of Ian Clarke
>>Sent: 01 December 2003 00:19
>>To: Discussion of development issues
>>Subject: Re: [freenet-dev] stable net kicking ass
>>
>>
>>Zlatin Balevsky wrote:
>>> I still think its a good idea to invest some toadtime into polishing
>>> it
>>> before 0.5.3, but so far its kicking ass.
>>
>>That is good news, and yes - I suspect there are a few things
>>that need 
>>to be tidied up (for example, didn't someone mention that 
>>acquisition of 
>>new refs was borked?).
>>
>>Ideal scenario is to get 0.5.3 working nicely (perhaps releasing
>>0.5.3rc1 in a week), release it, get $$$, and then start figuring out 
>>what is wrong with NGR using a careful scientific process rather than 
>>the somewhat haphazard approach we have been taking.
>>
>>Ian.
>>___
>>Devl mailing list
>>[EMAIL PROTECTED]
>>http://dodo.freenetproject.org/cgi->bin/mailman/listinfo/devl
>>
>>
>>---
>>Incoming mail is certified Virus Free.
>>Checked by AVG anti-virus system (http://www.grisoft.com).
>>Version: 6.0.543 / Virus Database: 337 - Release Date: 21/11/2003
>> 
>>
>
>---
>Outgoing mail is certified Virus Free.
>Checked by AVG anti-virus system (http://www.grisoft.com).
>Version: 6.0.543 / Virus Database: 337 - Release Date: 21/11/2003
> 
>
>___
>Devl mailing list
>[EMAIL PROTECTED] 
>http://dodo.freenetproject.org/cgi->bin/mailman/listinfo/devl
>
>
>---
>Incoming mail is certified Virus Free.
>Checked by AVG anti-virus system (http://www.grisoft.com).
>Version: 6.0.543 / Virus Database: 337 - Release Date: 21/11/2003
> 
>

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.543 / Virus Database: 337 - Release Date: 21/11/2003
 

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] stable net kicking ass

[Juiceman]

I have always thought that the whole NGR thing was misguided.  Freenet works
because of keyspace specialization.  NGR trashes keyspace specialization in
favor of "speed", which reduces Freenet from an medium speed expressway to a
grid of city-streets with traffic-lights and congestion.  You might be able
to get across town faster by taking the side streets -- until everyone else
starts taking the side streets also and your route becomes a parking-lot
from the cross-traffic.

IMHO, Toad's time would be better spent (post 0.5.3 of course) on MUXing and
other minor optimizations that will benefit the project more for 0.6.  In
the meantime, other dev's can work on hammering out the bugs in NGR (0.7
anyone?  :-p)

- Original Message - 
From: "Ian Clarke" <[EMAIL PROTECTED]>
To: "Discussion of development issues" <[EMAIL PROTECTED]>
Sent: Sunday, November 30, 2003 7:18 PM
Subject: Re: [freenet-dev] stable net kicking ass


> Zlatin Balevsky wrote:
> > I still think its a good idea to invest some toadtime into polishing it
> > before 0.5.3, but so far its kicking ass.
>
> That is good news, and yes - I suspect there are a few things that need
> to be tidied up (for example, didn't someone mention that acquisition of
> new refs was borked?).
>
> Ideal scenario is to get 0.5.3 working nicely (perhaps releasing
> 0.5.3rc1 in a week), release it, get $$$, and then start figuring out
> what is wrong with NGR using a careful scientific process rather than
> the somewhat haphazard approach we have been taking.
>
> Ian.
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
>
>

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Errors in formulas

[Martin Stone Davis]

Zlatin Balevsky wrote:

The very idea of using a formula for making decisions about routing has 
one major flaw and that is the innacuracy of the estimators.  Unless a 
perfect estimator is developed which will give the exact value of a 
given variable, any formula will produce humonguous margin of error.
As you know, if every variable in the formula was 90% accurate (which is 
extremely optimistic in our case), the accuracy of the formula would be 
at most 90% if only additions and substractions are used and much worse 
if there are other operations.  The accuracy of each estimator in a live 
network is anywhere between 0 and 100%, and the older the last 
measurement is, the bigger the error.  If the formula uses 3 or more 
variables from estimators, if a single of those measurements goes under 
80% accuracy you can dump the result altogether.

To judge different formulas it is not enough to just plug them in and 
see which one produces least error; if that was the case I can tell you 
right now that the formula that uses the least # of variables and only 
additions and substractions will be the best ;).  We need to be able to 
know which estimator tends to produce the biggest error and to have some 
known range for that error.  What if estimator A consistently produced 
10 times more error than estimator B?  Wouln't you want to decrease the 
weight or totally get rid of pA in that case?

The only way to find the error of the various estimators is to know what 
the actual values are.  And the only way to have that knowledge is to 
run the tests in controlled environment - either sim or watchme network.

P.s. personally I'm against using a formula alone to make the most 
important decision a node has to make, but if we're going to do it lets 
at least do it the right way.
 Let's say E = 0.1x * 1.0y + 10z . Let's say x and y are 1% 
accurate, but z is 90% accurate. Then E will be about 90% accurate.

 that formula will be 9/11 accurate, roughly
which is around 85% or so
now, multiply y*z and the story changes drastically...
 my point is just that you can't say that "if a single of those 
measurements goes under 80% accuracy you can dump the result 
altogether". It just isn't necessarily true. It depends on the formula 
and how the variables interact in it.

 well yeah, I had the current formula in mind when I wrote that, 
or any other formula which doesn't assign vastly different weights to 
the variables

 All this concern about the accuracy of estimate() is valid. 
But it's not an argument for why we should not implement my formula. 
It's just an argument for the fact that we should pay attention to how 
to make our calculation of estimate() more accurate.

 I'm not the person to convince whether your formula is better or 
not than someone elses.

 Well I know *that*

 as a matter of fact if we indeed produce those experiments in 
controlled environment where each estimator is tested, I believe we'll 
see their accuracy is under 50%

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Errors in formulas

[Zlatin Balevsky]

The very idea of using a formula for making decisions about routing has 
one major flaw and that is the innacuracy of the estimators.  Unless a 
perfect estimator is developed which will give the exact value of a 
given variable, any formula will produce humonguous margin of error. 

As you know, if every variable in the formula was 90% accurate (which is 
extremely optimistic in our case), the accuracy of the formula would be 
at most 90% if only additions and substractions are used and much worse 
if there are other operations.  The accuracy of each estimator in a live 
network is anywhere between 0 and 100%, and the older the last 
measurement is, the bigger the error.  If the formula uses 3 or more 
variables from estimators, if a single of those measurements goes under 
80% accuracy you can dump the result altogether.

To judge different formulas it is not enough to just plug them in and 
see which one produces least error; if that was the case I can tell you 
right now that the formula that uses the least # of variables and only 
additions and substractions will be the best ;).  We need to be able to 
know which estimator tends to produce the biggest error and to have some 
known range for that error.  What if estimator A consistently produced 
10 times more error than estimator B?  Wouln't you want to decrease the 
weight or totally get rid of pA in that case?

The only way to find the error of the various estimators is to know what 
the actual values are.  And the only way to have that knowledge is to 
run the tests in controlled environment - either sim or watchme network.

P.s. personally I'm against using a formula alone to make the most 
important decision a node has to make, but if we're going to do it lets 
at least do it the right way.


pgp0.pgp
Description: PGP signature
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

RE: [freenet-dev] stable net kicking ass

[Pete]

I'm not sure the acquisition of new ref's is borked, I've watched my rt
shrink and regrow, so I'm pretty confident that that's not an issue, but
I have noticed that every ref in my rt eventually backs off, which isn't
very helpful to the cause, it also seems nodes aren't passing data
between each other freely watching the convo is #freenet, (I'm sure zab
will comment on this later). I think someone mentioned that some of the
ref harvesters are down that could be a problem, as everyone
installing/upgrading will be starting with pretty much the same refs and
punishing the nodes in them.

There are also far to many nodes which have arks that never seem to be
found, (49 currently in my rt) that can't be helping the cause as they
don't seem to ever get dropped :(

Anyway nice to see some positive steps being taken in the development of
freenet again 

Pete

>-Original Message-
>From: [EMAIL PROTECTED] 
>[mailto:[EMAIL PROTECTED] On Behalf Of Ian Clarke
>Sent: 01 December 2003 00:19
>To: Discussion of development issues
>Subject: Re: [freenet-dev] stable net kicking ass
>
>
>Zlatin Balevsky wrote:
>> I still think its a good idea to invest some toadtime into polishing 
>> it
>> before 0.5.3, but so far its kicking ass.
>
>That is good news, and yes - I suspect there are a few things 
>that need 
>to be tidied up (for example, didn't someone mention that 
>acquisition of 
>new refs was borked?).
>
>Ideal scenario is to get 0.5.3 working nicely (perhaps releasing 
>0.5.3rc1 in a week), release it, get $$$, and then start figuring out 
>what is wrong with NGR using a careful scientific process rather than 
>the somewhat haphazard approach we have been taking.
>
>Ian.
>___
>Devl mailing list
>[EMAIL PROTECTED] 
>http://dodo.freenetproject.org/cgi->bin/mailman/listinfo/devl
>
>
>---
>Incoming mail is certified Virus Free.
>Checked by AVG anti-virus system (http://www.grisoft.com).
>Version: 6.0.543 / Virus Database: 337 - Release Date: 21/11/2003
> 
>

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.543 / Virus Database: 337 - Release Date: 21/11/2003
 

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] stable net kicking ass

[Ian Clarke]

Zlatin Balevsky wrote:
I still think its a good idea to invest some toadtime into polishing it 
before 0.5.3, but so far its kicking ass.
That is good news, and yes - I suspect there are a few things that need 
to be tidied up (for example, didn't someone mention that acquisition of 
new refs was borked?).

Ideal scenario is to get 0.5.3 working nicely (perhaps releasing 
0.5.3rc1 in a week), release it, get $$$, and then start figuring out 
what is wrong with NGR using a careful scientific process rather than 
the somewhat haphazard approach we have been taking.

Ian.
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] stable net kicking ass

[Aureliano Rama]

> I still think its a good idea to invest some toadtime into polishing it
> before 0.5.3, but so far its kicking ass.

Yeah, just downloaded almost 300MB in half a day!
That's a new record for me with Freenet.

And we must consider that my stable node is completely new, I wiped
out everything I had before (but the datastore).

However, even if my node seems to get to other node very good, others
are not going to me as well: in the same time, I've got only 53
inbound request search keys, a little too low for 5 hours of work.

Aureliano Rama
Corso di Laurea in Informatica, Pisa

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Where now?

[Aureliano Rama]

> Fine, but that's not the point.  The point is to make Joe User happy
> with stable so we don't kill off donations.  Experiments with black 
> holes in stable might make Joe mad.

No tv and no beer make Homer (ehm Joe) go crazy
No tv and no beer make Homer (ehm Joe) go crazy
No tv and no beer make Homer (ehm Joe) go crazy
No tv and no beer make Homer (ehm Joe) go crazy
No tv and no beer make Homer (ehm Joe) go crazy
No tv and no beer make Homer (ehm Joe) go crazy


Aureliano Rama
Corso di Laurea in Informatica, Pisa

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] stable net kicking ass

[Zlatin Balevsky]

Ok, yesterday it didn't work - nodes didn't learn about each other, it 
was busy, etc.

Today however, the request success ratio was more than 10% for a brand 
new node (18 out of 124 externally requested keys successful), and frost 
is chugging along (recorded a download speed of 100k at some point). 
And an insert of a small file with htl 21 took just few seconds (hope 
nobody is running a black hole).  And nodes learn about each other all 
of a sudden...

I still think its a good idea to invest some toadtime into polishing it 
before 0.5.3, but so far its kicking ass.


pgp0.pgp
Description: PGP signature
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Unobtanium routing

[Edward J. Huff]

On Sun, 2003-11-30 at 15:40, Thomas Leske wrote:
> Martin Stone Davis wrote:
> > Okay, so the attacker could censor the current edition.  But he wouldn't 
> > be able to censor all of them, since they are distributed throughout the 
> > keyspace.  The reader could then just click to retrieve one of the 
> > previous day's (or week's) editions.  The fred interface might be 
> > designed to make this even easier, if that becomes a problem.
> 
> Well! If there really is not a problem, with making the specialization of nodes
> public, then we should make use of that fact in our routing algorithm.
> 

I went and read about SSK's and redirects and metadata.  It seems that
the whole structure of Freenet above the CHK level (i.e. the fproxy
interface and the whole browsing experience) absolutely requires
the ability to route requests for keys which are "well known."

Otherwise, there must be some out-of-band distribution system for
revised keys.  Frankly, I think that it will be necessary to find
such a system to achieve the anonymity and censorship resistance
goals of Freenet, but the present implementation needs fixed keys.

-- Ed Huff



signature.asc
Description: This is a digitally signed message part
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Where now?

[Martin Stone Davis]

Zlatin Balevsky wrote:

Zlatin Balevsky wrote:

We already know that old stuff is subject to that attack


No, we don't know if classic routing is.  We suspect it might be to 
some degree, but we are not sure.


Fine, but that's not the point.  The point is to make Joe User happy 
with stable so we don't kill off donations.  Experiments with black 
holes in stable might make Joe mad.

-Martin


IF (<-- big "if") classical routing is vulnerable to black holes, then 
making it immune would be much easier than making NGR immune, because 
the only strength a black hole has in classical routing is that it never 
QRs.  So all it will take to make classical immune is some tweaking of 
the backoff parameters.

and yes, joe user will be unhappy if we do it, but won't be happy
either if someone else ([RI|MP]AA | script kiddie) does it for us.
Admirable goal.  But instead of beating the xxAA to the punch, why not 
advocate for a separate test network in which you can run the experiments?

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Abolishing HTL

[Martin Stone Davis]

Ian Clarke wrote:

The proposal for getting rid of HTL seems to be based on using time 
expired rather than hops expired, but doesn't that require a globally 
agreed time?

Ian.
No, I believe the agreed time isn't global---it's agreed just between 
two nodes at a time.  I think the way Toad intends it to work is that 
instead of passing the number of hops-to-live, we pass the amount of 
time-to-live.  When the next node routes to another node, it has to 
decrease the time-to-live so that it won't timeout before the routed 
node does.  So there's no globally agreed time.

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Unobtanium routing

[Thomas Leske]

Martin Stone Davis wrote:
Okay, so the attacker could censor the current edition.  But he wouldn't 
be able to censor all of them, since they are distributed throughout the 
keyspace.  The reader could then just click to retrieve one of the 
previous day's (or week's) editions.  The fred interface might be 
designed to make this even easier, if that becomes a problem.
Well! If there really is not a problem, with making the specialization of nodes
public, then we should make use of that fact in our routing algorithm.
But I somehow don't like the idea, that the author has to the determine,
if his freesite is under attack (He may just blame the insertion client or
freenet and give up). But I can think of a way to fix unobtanium routing,
so that specialization probing is prohibited. Unfortunately the modifications
would introduce quite some overhead. I'll write about it in my next mail.
--
 Thomas Leske
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Where now?

[Zlatin Balevsky]

Zlatin Balevsky wrote:

We already know that old stuff is subject to that attack


No, we don't know if classic routing is.  We suspect it might be to some 
degree, but we are not sure.


Fine, but that's not the point.  The point is to make Joe User happy 
with stable so we don't kill off donations.  Experiments with black 
holes in stable might make Joe mad.

-Martin
IF (<-- big "if") classical routing is vulnerable to black holes, 
then making it immune would be much easier than making NGR immune, 
because the only strength a black hole has in classical routing is 
that it never QRs.  So all it will take to make classical immune 
is some tweaking of the backoff parameters.

and yes, joe user will be unhappy if we do it, but won't be happy
either if someone else ([RI|MP]AA | script kiddie) does it for us. 



pgp0.pgp
Description: PGP signature
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Unobtanium routing

[Edward J. Huff]

On Sun, 2003-11-30 at 13:33, Thomas Leske wrote:
> Martin Stone Davis wrote:
> > What bad thing would a malicious node operator do with that 
> > knowledge?
> 
> He could censor a certain document. Assume he has the resources to
> lauch a denail of service attack against a limited number of nodes. If he
> does not know, which nodes to attack, he will be out of luck.
> 
> When the nodes that specialized around the key to be censored are taken
> down, it will be pure chance to find the data unless it was already very
> popular before.
> 

This problem is of course an argument for re-inserting data under
a different key:  just append some salt to the file and reinsert it.
You still need to distribute the new key, but that is no harder than
it was to distribute the original key.  It also gets around the
problem of re-insertion failing to keep data stored where the
network looks for it.  And it gets around key based censorship
in general.

-- Ed Huff



signature.asc
Description: This is a digitally signed message part
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Abolishing HTL

[Jim Dixon]

On Sun, 30 Nov 2003 [EMAIL PROTECTED] wrote:

> No it does not. Read my proposial for Positive Trust baised Freenet.

> > The proposal for getting rid of HTL seems to be based on using time
> > expired rather than hops expired, but doesn't that require a globally
> > agreed time?

If clocks are out of sync, rewards will be incorrectly allocated among
participating nodes, but this is not a critical issue.

--
Jim Dixon  [EMAIL PROTECTED]   tel +44 117 982 0786  mobile +44 797 373 7881

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Positive Trust Baised Freenet

[Jim Dixon]

On Sun, 30 Nov 2003 [EMAIL PROTECTED] wrote:

> The idea is this: We eliminate the notion of HTL. Instead we replace
> it with a trust based system with TTL.
...
> void processRequest(Node node,Key key, TTL ttl)
> {
>   float load=getCurrentLoad(); //Check our current load.
>
> //Tenitively subtract the maximum amount of credit that the node could owe 
> us.
> //Subtract returns the amount it took away. (never more than there was there)
>   float credit = node.credit.subtract(ttl * 1.414214);

This is a penalty imposed on the requestor, refunded only if the request
is eventually successful.

> void requestComplete( Node requester, Node provider, TTL requesterTTL, TTL 
> providerTTL, float requesterTime, float providerTime, bool successful)
> {
>   if (successful)
>   {
>   if ( requester != me )
>   {   //add back the amount we took away minus the amount they owe 
> us.
>   requester.trust.add( requesterTTL * 1.414214 - Max( 0 , 
> squareRoot( 2 * requesterTTL^2 - requesterTime^2 ) ) );
>   requester.trustInMe.add(  Max( 0 , squareRoot( 2 * 
> requesterTTL^2 - requesterTime^2 ) ) );
>   }
>   //Pay out. Pay up.
>   if ( provider != me )
>   {
>   provider.trust.add( Max( 0 , squareRoot( 2 * providerTTL^2 - 
> providerTime^2 ) ) );
>   //add back the amount they took away minus the amount we owe 
> them.
>   provider.trustInMe.add( providerTTL * 1.414214 - Max( 0 , 
> squareRoot( 2 * providerTTL^2 - providerTime^2 ) ) );
>   }
>   }
>   else
>   {
>   //everything already subtracted. Nothing to add.
>   }
> }

While the details are not all that clear and the code doesn't quite make
sense, a network adopting this approach would suffer from grave defects.

A.  THE NETWORK WOULD NEVER LEAVE THE ttl==0 PHASE.

In the early stages of network development, all nodes would be at zero
credit, and so all processRequest calls would be rejected.  End of story,
really ;-)

However, there is a vague proposal for ttl==0 requests giving some small
amount of credit.  These would be treated as requests with large HTL are
now; the proposal is for HTL==25.  Let's say that a node acquires some
credit in this way and then makes a request using the proposed ttl>0
protocol.  Freenet experience so far is that that request is unlikely to
succeed.  The penalty is ttl*1.4.  End of story, really; any credit gained
using the ttl==0 protocol is very likely to disappear immediately.  The
network is stable in a state where all nodes have no credit with one
another.

B.  ONE-SHOT INJECTION OF CREDIT

Let's say that there is a one-time injection of credit into a brand-new
network.  There are two likely outcomes.  In the first, the more likely
case, because of lack of specialization, everyone simply slides back to
zero credit.  This occurs because the penalty for failure is high and the
chance of success is low, as no one is specialized.

C.  METASTABLE: A FEW STARS, MOST NODES BROKE

In the second state (far less likely, in my opinion), a few nodes
specialize narrowly and hold to their specializations.  The rest slide
back to no credit.  This state is at least metastable: everyone learns
who is reliable, and so requests from them succeed and gain them credit.
The request chains to these stars rapidly become very short -- in fact,
most requests should be direct.

Most requests to other nodes fail, and they remain broke.  They trade only
at ttl==0.

The stars can trade among themselves successfully.  They know each other
well, and all requests between them are direct.

In this state, there is minimal replication, because request chains are
short.  There is also minimal anonymity for the same reason.  In other
words, this protocol might work for some nodes, but it ain't Freenet.

D.  ATTACK ON A STAR

A cluster of nodes can easily attack a star by agreeing among themselves
to subdivide its specialization, cooperating among themselves as described
in E.  They will become more attractive in each subdivision, so traffic
will shift to them.  The natural result is for the star to fall back to
zero credit status.

Alternatively, all nodes in the cluster could just adopt the same
specialization as the star.  If there are N nodes in the cluster, the star
should lose N/(N+1) of its traffic.  The natural result is that the star
slides back towards zero credit.

One would have to experiment to see which tactic is more successful. ;-)

E.  COOPERATE TO CONQUER

A cluster of nodes can agree to divide all of keyspace among them.  Each
will give unlimited credit to any node.  Each either handles requests
directly if the key lies in its range or forwards the request to the
cluster node responsible for the key.  Requests are never rejected.  If
the key is not held by the appropriate node, it gets it, if necessary
usin

[freenet-dev] Re: Where now?

[Martin Stone Davis]

Zlatin Balevsky wrote:

We already know that old stuff is subject to that attack


No, we don't know if classic routing is.  We suspect it might be to some 
degree, but we are not sure.

Fine, but that's not the point.  The point is to make Joe User happy 
with stable so we don't kill off donations.  Experiments with black 
holes in stable might make Joe mad.

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Where now?

[Zlatin Balevsky]

We already know that old stuff is subject to that attack
No, we don't know if classic routing is.  We suspect it might be to some degree, but we are not sure.



pgp0.pgp
Description: PGP signature
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Please nobody "blackhole" freenet's future (was Re: Where now?)

[Martin Stone Davis]

Aureliano Rama wrote:

Also, I'd really like to test the black hole in stable as well; it should be
much less potent, but the never-QR-ing element is still there.


Sincerely, i think this would be pointless.

We already know that old stuff is subject to that attack. That anyone
can bring freenet down even with low resources.
And this is why we need to bring on unstable development: to replace
stable some day. And then stable flaws will be gone in the wind.
We already got protest in support list for your BH that, as i
answered, was needed to debug.
But if don't let users something that can be used, what the need to
fork out the stable was?
We need Joe User happy, to make his donation come in.

Aureliano Rama
Corso di Laurea in Informatica, Pisa
Hear, hear!

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Where now?

[Aureliano Rama]

> Also, I'd really like to test the black hole in stable as well; it should be
> much less potent, but the never-QR-ing element is still there.

Sincerely, i think this would be pointless.

We already know that old stuff is subject to that attack. That anyone
can bring freenet down even with low resources.
And this is why we need to bring on unstable development: to replace
stable some day. And then stable flaws will be gone in the wind.

We already got protest in support list for your BH that, as i
answered, was needed to debug.

But if don't let users something that can be used, what the need to
fork out the stable was?

We need Joe User happy, to make his donation come in.


Aureliano Rama
Corso di Laurea in Informatica, Pisa

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Unobtanium routing

[Martin Stone Davis]

Thomas Leske wrote:

Martin Stone Davis wrote:

I don't think this is a major problem.  If the author realized this 
was happening, the same document (essentially) could always be 
inserted at multiple places in the keyspace.


A good target document for censoring would be the manifest of a 
DBR-freesite.
The key of the next edition is predetermined. The attacker even can 
guess the
time, when the key will be inserted (and maybe verified). The readers will
think the freesite just was not inserted, when it does not load.
Okay, so the attacker could censor the current edition.  But he wouldn't 
be able to censor all of them, since they are distributed throughout the 
keyspace.  The reader could then just click to retrieve one of the 
previous day's (or week's) editions.  The fred interface might be 
designed to make this even easier, if that becomes a problem.

It would be impractical for the author to insert the manifest under a
different key (= at another place in the keyspace), because it will 
break the
bookmarks of his readers.If there is a well-known index of new 
freesites, then
the attacker will read it too and try to censor the new site as well.

--
 Thomas Leske
-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Unobtanium routing

[Thomas Leske]

Martin Stone Davis wrote:
I don't think this is a major problem.  If the author realized this was 
happening, the same document (essentially) could always be inserted at 
multiple places in the keyspace.
A good target document for censoring would be the manifest of a DBR-freesite.
The key of the next edition is predetermined. The attacker even can guess the
time, when the key will be inserted (and maybe verified). The readers will
think the freesite just was not inserted, when it does not load.
It would be impractical for the author to insert the manifest under a
different key (= at another place in the keyspace), because it will break the
bookmarks of his readers.If there is a well-known index of new freesites, then
the attacker will read it too and try to censor the new site as well.
--
 Thomas Leske
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: GNUnet economic model (was Re: Separating dev and stable)

[Martin Stone Davis]

[EMAIL PROTECTED] wrote:

I know that may not seem straight forward. But I didn't come up with
this proposial in an hour. I spent a long time thinking about the
problems with the network, and decided that time, rather than HTL was
the way to go. as did you. 
Okay, fine.

That presented a lot of problems too. (and
security issues) 
Well, I'm not convinced that the security issues are the most important 
thing.  Can you show how using time-to-live rather than hops-to-live 
leads to security problems?

The way I worked out how to solve them was to use
trust as a means of deturmaning time.
Read my proposial, it is simple, and easy to impliment, and does
both.
As for the design problems it presents, I outline some of the details 
Toad's plan needs here: 
http://article.gmane.org/gmane.network.freenet.devel/8184.  None of that 
 involves solving whatever security problems are present.  If your plan 
(with the exception of the trust stuff) is a completed version of Toad's 
idea, you should be able to fill in those details.

I would like to understand your idea better, so please don't just say 
"read my proposal".  If the questions are answered there, then show me 
the way.


[EMAIL PROTECTED] wrote:


[EMAIL PROTECTED] wrote:


Postitive Trust == GNUnet modle. My proposial impliments
both.


Martin Stone Davis wrote:

You'll also have to answer the question I had about
GNUnet-style economic models: what crucial problem would be
solved that wouldn't be solved by the plan I outlined 
(http://article.gmane.org/gmane.network.freenet.devel/8191)?


I could ask you the same question. The bottom line is that my 
proposal is a very easy way of implementing exactly what you 
describe.
I'm confused.  You said your proposal was equivalent to
implementing a GNUnet-style model on Freenet.  You are also saying
it's a way of doing what Toad described.  But Toad's idea !=
GNUnet-style model.  All Toad is saying is to use time-to-live
rather than hops-to-live.  That's just one part of your proposal.
-Martin
-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Unobtanium routing

[Martin Stone Davis]

Thomas Leske wrote:

Martin Stone Davis wrote:

The whole point of NGR is that nodes learn the specializations of 
other nodes.


Yes, of course, other nodes should learn about the specialization of
other nodes. But they must not be able to do so without propagating
any data and spending a considerable amount of resources.
What bad thing would a malicious node operator do with that knowledge?


He could censor a certain document. Assume he has the resources to
lauch a denail of service attack against a limited number of nodes. If he
does not know, which nodes to attack, he will be out of luck.
When the nodes that specialized around the key to be censored are taken
down, it will be pure chance to find the data unless it was already very
popular before.
--
 Thomas Leske
I don't think this is a major problem.  If the author realized this was 
happening, the same document (essentially) could always be inserted at 
multiple places in the keyspace.

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: GNUnet economic model (was Re: Separating dev and stable)

[tkaitchuck]

I know that may not seem straight forward. But I didn't come up with this proposial in 
an hour. I spent a long time thinking about the problems with the network, and decided 
that time, rather than HTL was the way to go. as did you. That presented a lot of 
problems too. (and security issues) The way I worked out how to solve them was to use 
trust as a means of deturmaning time.

Read my proposial, it is simple, and easy to impliment, and does both.
> [EMAIL PROTECTED] wrote:
> 
> >> [EMAIL PROTECTED] wrote:
> >> 
> >>> Postitive Trust == GNUnet modle. My proposial impliments both.
> >> 
> 
>  >>> Martin Stone Davis wrote:
> >> You'll also have to answer the question I had about GNUnet-style 
> >> economic models: what crucial problem would be solved that wouldn't
> >> be solved by the plan I outlined 
> >> (http://article.gmane.org/gmane.network.freenet.devel/8191)?
> > 
> > 
> > I could ask you the same question. The bottom line is that my
> > proposal is a very easy way of implementing exactly what you
> > describe.
> 
> I'm confused.  You said your proposal was equivalent to implementing a 
> GNUnet-style model on Freenet.  You are also saying it's a way of doing 
> what Toad described.  But Toad's idea != GNUnet-style model.  All Toad 
> is saying is to use time-to-live rather than hops-to-live.  That's just 
> one part of your proposal.
> 
> -Martin
> 
> 
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Unobtanium routing

[Thomas Leske]

Martin Stone Davis wrote:
The whole point of NGR is that nodes learn the specializations of other 
nodes.
Yes, of course, other nodes should learn about the specialization of
other nodes. But they must not be able to do so without propagating
any data and spending a considerable amount of resources.
What bad thing would a malicious node operator do with that 
knowledge?
He could censor a certain document. Assume he has the resources to
lauch a denail of service attack against a limited number of nodes. If he
does not know, which nodes to attack, he will be out of luck.
When the nodes that specialized around the key to be censored are taken
down, it will be pure chance to find the data unless it was already very
popular before.
--
 Thomas Leske
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: GNUnet economic model (was Re: Separating dev and stable)

[Martin Stone Davis]

[EMAIL PROTECTED] wrote:

[EMAIL PROTECTED] wrote:

Postitive Trust == GNUnet modle. My proposial impliments both.


>>> Martin Stone Davis wrote:
You'll also have to answer the question I had about GNUnet-style 
economic models: what crucial problem would be solved that wouldn't
be solved by the plan I outlined 
(http://article.gmane.org/gmane.network.freenet.devel/8191)?


I could ask you the same question. The bottom line is that my
proposal is a very easy way of implementing exactly what you
describe.
I'm confused.  You said your proposal was equivalent to implementing a 
GNUnet-style model on Freenet.  You are also saying it's a way of doing 
what Toad described.  But Toad's idea != GNUnet-style model.  All Toad 
is saying is to use time-to-live rather than hops-to-live.  That's just 
one part of your proposal.

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Where now?

[Martin Stone Davis]

Ian Clarke wrote:

One thing that is important is simply to figure out how accurate NGR's 
estimates actually are, and whether their estimates are statistically 
significant.  
If you look at the diff* diagnostics, I think you'll be convinced that 
we are consistently wrong in one direction.  Until recently, we were 
consistently too optimistic (last time I checked, I think it reversed to 
being too pessimistic).  I think the reason for this is simply that we 
start off with an overly-optimistic or pessimistic view of new nodes.

Perhaps a more rational approach would be to view new nodes as being no 
different from the global average.  Then, a backoff scheme, by itself, 
would encourage nodes to sometimes try new routes.

> Also, understanding which parts of the NGR estimate
calculation have the most bearing on the routing decision.
Good goal, but let's first fix the estimate() formula so that the 
routing decision is made rationally.  Then, yes, we should be able to 
easily see which factors have the biggest influence.  Hopefully, we can 
use that information to guide us on our future plans.

Thoughts?

Ian.
-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Where now?

[Martin Stone Davis]

Ian Clarke wrote:

Ok, the new stable build seems to be working quite well, are other 
people experiencing the same thing?

We need to take stock of the situation with NGR.  I think one problem 
has been a willingness to dream up solutions, and implement them, before 
actually understanding what the problem is.

I would like to propose that now that the time-pressure is off, we try 
to be more cautious - we need to form theories about what the problem 
is, figure out how to test these theories, and if they prove true, 
*then* we implement a solution.
Well, okay, but how does that relate to the plan I outlined 
(http://article.gmane.org/gmane.network.freenet.devel/8191)?

I understand why you would want more proof that we should drop HTL in 
favor of time-to-live (Toad's idea, which I support).  To do so *right 
now* would be a big change, since we still have many details to be 
filled in (http://article.gmane.org/gmane.network.freenet.devel/8184). 
Therefore we should only do it if we're confident that it will cure a 
major ill.

However, I think we can implement the other ideas right away.  We know 
(Toad has proven) that we have more queries than we can actually deal 
with, so your back-off scheme should be implemented.  We know (I have 
proven) that the current estimate() formula is incorrect, so we should 
fix it to match its original purpose.  And as for Unobtanium routing, I 
can't prove to you that it would cure a major ill, but it would not be 
hard to implement, it couldn't hurt, and it just might help.

One thing that is important is simply to figure out how accurate NGR's 
estimates actually are, and whether their estimates are statistically 
significant.  
>
Also, understanding which parts of the NGR estimate 
calculation have the most bearing on the routing decision.

Thoughts?

Ian.
-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Where now?

[Edward J. Huff]

On Sun, 2003-11-30 at 08:55, Ian Clarke wrote:
> Ok, the new stable build seems to be working quite well, are other 
> people experiencing the same thing?
> 
> We need to take stock of the situation with NGR.  I think one problem 
> has been a willingness to dream up solutions, and implement them, before 
> actually understanding what the problem is.
> 
Yes, although that was a consequence of the need to get it done
yesterday...  

> I would like to propose that now that the time-pressure is off, we try 
> to be more cautious - we need to form theories about what the problem 
> is, figure out how to test these theories, and if they prove true, 
> *then* we implement a solution.

I agree.  Meanwhile, perhaps Toad should be implementing obvious
improvements independent of routing, such as connection multiplexing.
Not that the theory that multiplexing would help should be exempt from 
testing.  

Another theory is that message traffic is larger than data traffic, 
and hence compressing the message traffic would be worthwhile.  
Hand crafted binary message formats, or Huffman encoding based 
on actual traffic, or even just gz or bz2 right before encription
might help.

> 
> One thing that is important is simply to figure out how accurate NGR's 
> estimates actually are, and whether their estimates are statistically 
> significant.  

Hear hear!!  Some of the data may already be in the diagnostics,
but it bears careful analysis.  If the things we want to measure
change drastically before we can get enough samples to average out
the noise, we better figure out something else to measure.

> Also, understanding which parts of the NGR estimate 
> calculation have the most bearing on the routing decision.
> 

Lets get a clear understandable explanation written up.  If I
can understand it, I'll try writing it.

-- Ed Huff



signature.asc
Description: This is a digitally signed message part
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: GNUnet economic model (was Re: Separating dev and stable)

[tkaitchuck]

> [EMAIL PROTECTED] wrote:
> > Postitive Trust == GNUnet modle.
> > My proposial impliments both.
> 
> In which case, you'll have to answer Ian's objection:
> > And the disadvantage of such a system, as I have pointed out many
> > times in the past, is that new users get a terrible experience of
> > Freenet, which is a *very* bad thing for the network as new users
> > should be encouraged, not punished.

Read my proposial. I submit 2 ways of doing this. I agree that this is the biggest 
problem, so I welcome more ideas, but we have no idea of how much of a problem it is 
untel we impliment it. So lets do that.

> You'll also have to answer the question I had about GNUnet-style 
> economic models: what crucial problem would be solved that wouldn't be 
> solved by the plan I outlined 
> (http://article.gmane.org/gmane.network.freenet.devel/8191)?

I could ask you the same question. The bottom line is that my proposal is a very easy 
way of implementing exactly what you describe. I also agree with you that we need a 
network fork with a good stable branch. So I say: set tDNF to infinity, so we have 
unobtanum routing, then fix any problems and make that 0.5.3. Then on the 
unstable/experimental branch we can play with time and trust routing.

> Martin Stone Davis wrote:
> >I just skimmed through http://www.ovmj.org/GNUnet/download/ebe.ps, which 
> >describes GNUnet's economic model.  In short, we would prioritize 
> >requests based, in part, on how well requesting nodes have serviced our 
> >requests in the past.  The benefit would be a system in which 
> >well-behaved nodes are rewarded during times when resources are short.
> >
> >Some kind of GNUnet-style economic model could be built into Freenet, 
> >but what crucial problem would it solve that won't be solved by the plan 
> >I outlined?
> >
> >I'm not saying that it wouldn't be a good thing to have down the road, 
> >as I can see it protecting us from freeloaders without needlessly 
> >punishing them.  However, now is the time to get Freenet to work well 
> >even in a network of mostly well-behaved peers (with a few malicious 
> >black-holes and freeloaders thrown in for good measure).
> 
> -Martin
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: GNUnet economic model (was Re: Separating dev and stable)

[Martin Stone Davis]

[EMAIL PROTECTED] wrote:
Postitive Trust == GNUnet modle.
My proposial impliments both.
In which case, you'll have to answer Ian's objection:
And the disadvantage of such a system, as I have pointed out many
times in the past, is that new users get a terrible experience of
Freenet, which is a *very* bad thing for the network as new users
should be encouraged, not punished.
You'll also have to answer the question I had about GNUnet-style 
economic models: what crucial problem would be solved that wouldn't be 
solved by the plan I outlined 
(http://article.gmane.org/gmane.network.freenet.devel/8191)?

Martin Stone Davis wrote:
I just skimmed through http://www.ovmj.org/GNUnet/download/ebe.ps, which 
describes GNUnet's economic model.  In short, we would prioritize 
requests based, in part, on how well requesting nodes have serviced our 
requests in the past.  The benefit would be a system in which 
well-behaved nodes are rewarded during times when resources are short.

Some kind of GNUnet-style economic model could be built into Freenet, 
but what crucial problem would it solve that won't be solved by the plan 
I outlined?

I'm not saying that it wouldn't be a good thing to have down the road, 
as I can see it protecting us from freeloaders without needlessly 
punishing them.  However, now is the time to get Freenet to work well 
even in a network of mostly well-behaved peers (with a few malicious 
black-holes and freeloaders thrown in for good measure).
-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Unobtanium routing

[Martin Stone Davis]

Thomas Leske wrote:

Toad wrote:

as load goes up, we make what we accept more and
more specialized. Overload pressure strengthens specialization.


Won't unobtanium routing make network analysis very easy?
If an attacker wants to determine the specialisation of a node, he will 
just
overload it with requests for random keys. For the keys outside its 
specialization
he will receive query rejects and for the ones inside its specialization 
DNFs.
The whole point of NGR is that nodes learn the specializations of other 
nodes.  What bad thing would a malicious node operator do with that 
knowledge?

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Positive Trust Baised Freenet

[tkaitchuck]

Again, read my origional post. It explains all of these things and has code for how to 
do much of it.

You get more trust for routing Faster so you route baised on speed and speed alone. So 
you just use NGrouting but take out the DNF estimators. A DNF is intreprited as "I 
cannot get the data in less than 1.4142*TTL" and so you adjust your estimator.
> Hi,
> 
> Am I correct in assuming that this model is fairly much independent of the
> routing algorithm used, or am I missing the point completely?   I understand
> that with your proposal a node needs to have sufficient trust to be able to 
> route
> but I am not clear how you decide to route to a node.
> 
> Ed 
> 
> On November 30, 2003 10:29 am, [EMAIL PROTECTED] wrote:
> > Nodes A,B,C,D all have 15 ecounds of trust in each other.
> > A wants some data.
> > Routs it to B with TTL of 10.
> > B docks A 14.14 trust.
> > 1 secound passes.
> > B routes to C with a TTL of 9.
> > C docks B 12.73 trust.
> > 1 secound passes.
> > C routes to D with TTL of 8.
> > D docks C 11.31 trust.
> > 1 secound passes.
> > D returns the data.
> > D adds .04428 Trust back into C's account.
> > C passes the data allong to B.
> > C adds .158117 Trust back into B's account.
> > B passes the data allong to A.
> > B adds .3218 trust back into A's account.
> > A adds 13.82 Trust into B's account.
> >
> > A gets the Data in 3 secounds when it asked for 10.
> >
> > *The Formula for Trust is (2*TTL^2-ReturnTime^2)^.5
> > *Every node In the chain has a net Trust gain.
> > *The Requeter has to pay for it's request.
> > *No node gives out more trust than it receives.
> > *Nodes are greedy and accept querrys on the baisis of personal profit.
> > *The faster a node can fetch data above the rest of the nodes on the
> > network the more it profits. *Returning Data from your store is very
> > profitable.
> > *The more Greedy the network becomes the faster it routes.
> > *All data will be fetched in 1.414 * TTL time or fail.
> > *Nobody can launch a DOS attack.
> > *If a node rejects your request it indicates that it cannot fetch that key
> > in that TTL profitable. So you learn about specilization even if you don't
> > get the data. *You keep retrying nodes to see if anyone will take the
> > request untel it would not be profitable to you even if they returned in
> > TTL time. Then you reject. *A request for data that is not in the network
> > uses the maximum ammount of resources, and costs the requester the maximum
> > ammount. *No intermediate node can be cheated, unless they drop data, or
> > dirmaticaly slow it down on the return path.
> >
> > Please read my proposial for more details. It still need review. Also think
> > of ways of injecting trust in the first place. I have 2 but more couldn't
> > hurt.
> >
> > > [EMAIL PROTECTED] wrote:
> > > > LOL, yeah. This is exactly what you and Toad were talking about, plus
> > > > lots of other goodies thrown in. I was so busy writing this that I
> > > > wasn't reading my email, now as I look back through, I see the two of
> > > > you were descussing many of the issues that I struggled to sort out
> > > > to write this. It was realy a let down to see that this wasn't going
> > > > to be totaly unexpected.
> > > >
> > > > Anywho, It ammounts to killing querys baised on time, not HTL. It
> > > > makes Freenet a positive trust baised network, (read the archives) I
> > > > bleeve I responded to you a few times talking about some of the
> > > > implications of this. It eliminates pDNF entirely, which I had come
> > > > to the conclusion was causing NGrouting to fail. It eliminates
> > > > vunerability to Black Hole nodes, which I did not know existed, when
> > > > I wrote this. It impliments proportional querry rejecting. (see the
> > > > archives where I post sample data talking about a split lifo/fifo
> > > > system.) It also enables QRing baised on key, etc.
> > > >
> > > > Toad: Please give this a very thourough look over. But I think this
> > > > should be a general solution to most network problems / flooding
> > > > attacks.
> > >
> > > It would still help if you could give an illustration of how your system
> > > works.
> > >
> > > -Martin
> > >
> > >
> > > ___
> > > Devl mailing list
> > > [EMAIL PROTECTED]
> > > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
> >
> > ___
> > Devl mailing list
> > [EMAIL PROTECTED]
> > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Positive Trust Baised Freenet

[Ed Tomlinson]

Hi,

Am I correct in assuming that this model is fairly much independent of the
routing algorithm used, or am I missing the point completely?   I understand
that with your proposal a node needs to have sufficient trust to be able to route
but I am not clear how you decide to route to a node.

Ed 

On November 30, 2003 10:29 am, [EMAIL PROTECTED] wrote:
> Nodes A,B,C,D all have 15 ecounds of trust in each other.
> A wants some data.
> Routs it to B with TTL of 10.
> B docks A 14.14 trust.
> 1 secound passes.
> B routes to C with a TTL of 9.
> C docks B 12.73 trust.
> 1 secound passes.
> C routes to D with TTL of 8.
> D docks C 11.31 trust.
> 1 secound passes.
> D returns the data.
> D adds .04428 Trust back into C's account.
> C passes the data allong to B.
> C adds .158117 Trust back into B's account.
> B passes the data allong to A.
> B adds .3218 trust back into A's account.
> A adds 13.82 Trust into B's account.
>
> A gets the Data in 3 secounds when it asked for 10.
>
> *The Formula for Trust is (2*TTL^2-ReturnTime^2)^.5
> *Every node In the chain has a net Trust gain.
> *The Requeter has to pay for it's request.
> *No node gives out more trust than it receives.
> *Nodes are greedy and accept querrys on the baisis of personal profit.
> *The faster a node can fetch data above the rest of the nodes on the
> network the more it profits. *Returning Data from your store is very
> profitable.
> *The more Greedy the network becomes the faster it routes.
> *All data will be fetched in 1.414 * TTL time or fail.
> *Nobody can launch a DOS attack.
> *If a node rejects your request it indicates that it cannot fetch that key
> in that TTL profitable. So you learn about specilization even if you don't
> get the data. *You keep retrying nodes to see if anyone will take the
> request untel it would not be profitable to you even if they returned in
> TTL time. Then you reject. *A request for data that is not in the network
> uses the maximum ammount of resources, and costs the requester the maximum
> ammount. *No intermediate node can be cheated, unless they drop data, or
> dirmaticaly slow it down on the return path.
>
> Please read my proposial for more details. It still need review. Also think
> of ways of injecting trust in the first place. I have 2 but more couldn't
> hurt.
>
> > [EMAIL PROTECTED] wrote:
> > > LOL, yeah. This is exactly what you and Toad were talking about, plus
> > > lots of other goodies thrown in. I was so busy writing this that I
> > > wasn't reading my email, now as I look back through, I see the two of
> > > you were descussing many of the issues that I struggled to sort out
> > > to write this. It was realy a let down to see that this wasn't going
> > > to be totaly unexpected.
> > >
> > > Anywho, It ammounts to killing querys baised on time, not HTL. It
> > > makes Freenet a positive trust baised network, (read the archives) I
> > > bleeve I responded to you a few times talking about some of the
> > > implications of this. It eliminates pDNF entirely, which I had come
> > > to the conclusion was causing NGrouting to fail. It eliminates
> > > vunerability to Black Hole nodes, which I did not know existed, when
> > > I wrote this. It impliments proportional querry rejecting. (see the
> > > archives where I post sample data talking about a split lifo/fifo
> > > system.) It also enables QRing baised on key, etc.
> > >
> > > Toad: Please give this a very thourough look over. But I think this
> > > should be a general solution to most network problems / flooding
> > > attacks.
> >
> > It would still help if you could give an illustration of how your system
> > works.
> >
> > -Martin
> >
> >
> > ___
> > Devl mailing list
> > [EMAIL PROTECTED]
> > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
>
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [EMAIL PROTECTED]: [freenet-dev] Corrections to NGR formula (improves routing and protects from black hole attack)]

[tkaitchuck]

OK, I should clarify. That was not directed at, you, Toad, Ian, Martin, Ed, Susa, or 
any of the other contributers here. I acknowledge that NGrouting is complicated and is 
hard to make work. I acknowledge there is a need for a working network even if it does 
not work. I acknowledge the need for attack resistance testing, although I think at 
this point you've made your point, and can stop. My message was directed at, thouse 
that said that NGrouting can never work, by virtue of it being complicated, those who 
said that alchemy is inherently better because it is simpler, and anyone who used the 
phrase "Horse sense". 

Now I also must apologize because my post was not productive ether. Now in the mean 
time I have done something, so please please read my proposal for positive trust 
biased freenet. Read the code. Think of attack. Tear it appart. It is infinitely 
better to find some theoretical problem now, before it is implemented, when it can be 
easily fixed, then having to wait until it is, and you have to write attack code to 
convince anyone. Perhaps I am over optimistic, but I think my proposal will solve 
every routing related problem is currently facing the network that I am aware of. So 
your help is needed, read the code, think of ways to attack it.
> 
> > 
> > Ok, I'm a bit behind in my e-mail so I just read this. But my postive trust
> > baised solution should solve this.
> > 
> > Hopefully this should shut up all those [EMAIL PROTECTED]&* people, and you know 
> > who you
> > are; who are talking about removing NGrouting, or saying that we should route
> > indpendent of the key value or that "It's too complicated, lets do something
> > stupid", or favoring bandwidth. To those who I'm refering, who don't have a
> > @#$%!^& clue what their talking about, just stop posting stuff that's not
> > helpful.
> > 
> 
> how about all those !#$%%^ people who give all those fancy ideas actually get
> off their cloud and implement them?
> 
> FYI, if I had not done the black hole, but had instead posted here outlinging a
> "potential" attack, it would have been dismissed, ignored, and few people in
> particular (no names mentioned, but they know who they are) would go into great
> lengths explaining what an idiot I am not to fully and unconditionally trust
> into NGR's ability to solve world hunger.
> 
> And a specific hint for you Mr. Kaitchuk - in the world of open source, if you
> believe an idea is good then you go implement it and only then if it proves to
> work others will join you.  Nobody's going to do someone else's crazy ideas for
> them.  (and yes, Ian coded the skeleton of the first freenet implementation back
> in 0.1/0.2 days, it was proven to work and only then it became popular; a paid
> programmer was hired much much later)
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Separating dev and stable

[Niklas Bergh]

Sorry, I should have been clearer.. I think that this 0.5.3 idea is a good
one but what I meant was:
1. Do the 0.5.3
2. Muxing or something
3. More NGR.

/N

- Original Message - 
From: "Ian Clarke" <[EMAIL PROTECTED]>
To: "Discussion of development issues" <[EMAIL PROTECTED]>
Sent: Sunday, November 30, 2003 12:16 PM
Subject: Re: [freenet-dev] Separating dev and stable


> Niklas Bergh wrote:
> > A thought.. Why not give NGR development a couple of weeks break and try
to
> > get muxing or something up and running in the mean time? When you have
no
> > relly good ideas on some specific problem it can be good to move over to
a
> > completely different matter for a while and then get back to tackling
the
> > original hurdles with renewed energy afterwards.
>
> Because the current situation is untenable and we have to do something
> to rectify it - implementing muxing probably won't fix the issues that
> are preventing NGR from working, and while NGR doesn't work, Freenet
> doesn't work.
>
> The proposal is to release a non-NGR 0.5.3 which includes all or most
> other innovations since 0.5.2.  This will give Toad a break from NGR
> while others keep working on it.  It will relieve the time-pressure to
> get NGR working, and it will mean that Joe-user has a working version of
> Freenet (we hope) while those interested in helping with NGR can do-so.
>
> Ian.
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
>

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Where now?

[Jonathan Howard]

Ian Clarke wrote:
Ok, the new stable build seems to be working quite well, are other 
people experiencing the same thing?
I don't think you can judge it yet. Only a fraction of users have updated.

I'm running a new (always) transient node on stable and it's routing 
other requests. I thought a change was made so transients will not get 
requests?

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Re: Positive Trust Baised Freenet

[tkaitchuck]

Nodes A,B,C,D all have 15 ecounds of trust in each other.
A wants some data.
Routs it to B with TTL of 10.
B docks A 14.14 trust.
1 secound passes.
B routes to C with a TTL of 9.
C docks B 12.73 trust.
1 secound passes.
C routes to D with TTL of 8.
D docks C 11.31 trust.
1 secound passes.
D returns the data. 
D adds .04428 Trust back into C's account.
C passes the data allong to B.
C adds .158117 Trust back into B's account.
B passes the data allong to A.
B adds .3218 trust back into A's account. 
A adds 13.82 Trust into B's account.

A gets the Data in 3 secounds when it asked for 10.

*The Formula for Trust is (2*TTL^2-ReturnTime^2)^.5
*Every node In the chain has a net Trust gain.
*The Requeter has to pay for it's request.
*No node gives out more trust than it receives.
*Nodes are greedy and accept querrys on the baisis of personal profit.
*The faster a node can fetch data above the rest of the nodes on the network the more 
it profits.
*Returning Data from your store is very profitable.
*The more Greedy the network becomes the faster it routes.
*All data will be fetched in 1.414 * TTL time or fail.
*Nobody can launch a DOS attack.
*If a node rejects your request it indicates that it cannot fetch that key in that TTL 
profitable. So you learn about specilization even if you don't get the data.
*You keep retrying nodes to see if anyone will take the request untel it would not be 
profitable to you even if they returned in TTL time. Then you reject.
*A request for data that is not in the network uses the maximum ammount of resources, 
and costs the requester the maximum ammount.
*No intermediate node can be cheated, unless they drop data, or dirmaticaly slow it 
down on the return path.

Please read my proposial for more details. It still need review. Also think of ways of 
injecting trust in the first place. I have 2 but more couldn't hurt.
> [EMAIL PROTECTED] wrote:
> > LOL, yeah. This is exactly what you and Toad were talking about, plus
> > lots of other goodies thrown in. I was so busy writing this that I
> > wasn't reading my email, now as I look back through, I see the two of
> > you were descussing many of the issues that I struggled to sort out
> > to write this. It was realy a let down to see that this wasn't going
> > to be totaly unexpected.
> > 
> > Anywho, It ammounts to killing querys baised on time, not HTL. It
> > makes Freenet a positive trust baised network, (read the archives) I
> > bleeve I responded to you a few times talking about some of the
> > implications of this. It eliminates pDNF entirely, which I had come
> > to the conclusion was causing NGrouting to fail. It eliminates
> > vunerability to Black Hole nodes, which I did not know existed, when
> > I wrote this. It impliments proportional querry rejecting. (see the
> > archives where I post sample data talking about a split lifo/fifo
> > system.) It also enables QRing baised on key, etc.
> > 
> > Toad: Please give this a very thourough look over. But I think this
> > should be a general solution to most network problems / flooding
> > attacks.
> 
> It would still help if you could give an illustration of how your system 
> works.
> 
> -Martin
> 
> 
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [EMAIL PROTECTED]: [freenet-dev] Corrections to NGR formula (improves routing and protects from black hole attack)]

[zbalevsk]

> 
> Ok, I'm a bit behind in my e-mail so I just read this. But my postive trust
> baised solution should solve this.
> 
> Hopefully this should shut up all those [EMAIL PROTECTED]&* people, and you know who 
> you
> are; who are talking about removing NGrouting, or saying that we should route
> indpendent of the key value or that "It's too complicated, lets do something
> stupid", or favoring bandwidth. To those who I'm refering, who don't have a
> @#$%!^& clue what their talking about, just stop posting stuff that's not
> helpful.
> 

how about all those !#$%%^ people who give all those fancy ideas actually get
off their cloud and implement them?

FYI, if I had not done the black hole, but had instead posted here outlinging a
"potential" attack, it would have been dismissed, ignored, and few people in
particular (no names mentioned, but they know who they are) would go into great
lengths explaining what an idiot I am not to fully and unconditionally trust
into NGR's ability to solve world hunger.

And a specific hint for you Mr. Kaitchuk - in the world of open source, if you
believe an idea is good then you go implement it and only then if it proves to
work others will join you.  Nobody's going to do someone else's crazy ideas for
them.  (and yes, Ian coded the skeleton of the first freenet implementation back
in 0.1/0.2 days, it was proven to work and only then it became popular; a paid
programmer was hired much much later)
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re: Where now?

[zbalevsk]

>Ok, the new stable build seems to be working quite well, are other 
>people experiencing the same thing?

quick comment:  there is still some polishing that should be done to the current
stable codebase before 0.5.3 can go out.  For example, 5046 wouldn't learn about
other new nodes through DataReplys; wild guess because StoreData was changed to
carry estimator data and not standard refs.

Also, I'd really like to test the black hole in stable as well; it should be
much less potent, but the never-QR-ing element is still there.
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] GNUnet economic model (was Re: Separating dev and stable)

[tkaitchuck]

Postitive Trust == GNUnet modle.
My proposial impliments both.
> S M wrote:
> 
> > "Martin Stone Davis" <[EMAIL PROTECTED] 
> > > 
> > wrote in message news:[EMAIL PROTECTED]
> > 
> >  > 1. We probably should go with Toad's idea of dropping the HTL system 
> >  >in&nb sp;favor of a timeout-based system.  This is a radical change, 
> > and >needs to be thought-out a bit more before it is implemented.  
> > However, I >would think we should be able to finish the design "on 
> > paper" in less >than a week.
> > 
> > If radical changes are going to be made it is a good opportunity to 
> > borrow the best aspects from GNUnet, (including their economic routing 
> > model, and the ability to host permanent specific files). 
> > 
> > What's more is there is no need to start from scratch to design anything 
> > "on paper" since it’s already done; taking the best parts from open 
> > source is one reason why it's so good.
> 
> I just skimmed through http://www.ovmj.org/GNUnet/download/ebe.ps, which 
> describes GNUnet's economic model.  In short, we would prioritize 
> requests based, in part, on how well requesting nodes have serviced our 
> requests in the past.  The benefit would be a system in which 
> well-behaved nodes are rewarded during times when resources are short.
> 
> Some kind of GNUnet-style economic model could be built into Freenet, 
> but what crucial problem would it solve that won't be solved by the plan 
> I outlined?
> 
> I'm not saying that it wouldn't be a good thing to have down the road, 
> as I can see it protecting us from freeloaders without needlessly 
> punishing them.  However, now is the time to get Freenet to work well 
> even in a network of mostly well-behaved peers (with a few malicious 
> black-holes and freeloaders thrown in for good measure).
> 
> -Martin
> 
> 
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Abolishing HTL

[tkaitchuck]

No it does not. Read my proposial for Positive Trust baised Freenet.
> The proposal for getting rid of HTL seems to be based on using time 
> expired rather than hops expired, but doesn't that require a globally 
> agreed time?
> 
> Ian.
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Where now?

[Ian Clarke]

Ok, the new stable build seems to be working quite well, are other 
people experiencing the same thing?

We need to take stock of the situation with NGR.  I think one problem 
has been a willingness to dream up solutions, and implement them, before 
actually understanding what the problem is.

I would like to propose that now that the time-pressure is off, we try 
to be more cautious - we need to form theories about what the problem 
is, figure out how to test these theories, and if they prove true, 
*then* we implement a solution.

One thing that is important is simply to figure out how accurate NGR's 
estimates actually are, and whether their estimates are statistically 
significant.  Also, understanding which parts of the NGR estimate 
calculation have the most bearing on the routing decision.

Thoughts?

Ian.
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Abolishing HTL

[Ian Clarke]

The proposal for getting rid of HTL seems to be based on using time 
expired rather than hops expired, but doesn't that require a globally 
agreed time?

Ian.
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Unobtanium routing

[Thomas Leske]

Toad wrote:
as load goes up, we make what we accept more and
more specialized. Overload pressure strengthens specialization.
Won't unobtanium routing make network analysis very easy?
If an attacker wants to determine the specialisation of a node, he will just
overload it with requests for random keys. For the keys outside its specialization
he will receive query rejects and for the ones inside its specialization DNFs.
--
 Thomas Leske
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Separating dev and stable

[Ian Clarke]

Niklas Bergh wrote:
A thought.. Why not give NGR development a couple of weeks break and try to
get muxing or something up and running in the mean time? When you have no
relly good ideas on some specific problem it can be good to move over to a
completely different matter for a while and then get back to tackling the
original hurdles with renewed energy afterwards.
Because the current situation is untenable and we have to do something 
to rectify it - implementing muxing probably won't fix the issues that 
are preventing NGR from working, and while NGR doesn't work, Freenet 
doesn't work.

The proposal is to release a non-NGR 0.5.3 which includes all or most 
other innovations since 0.5.2.  This will give Toad a break from NGR 
while others keep working on it.  It will relieve the time-pressure to 
get NGR working, and it will mean that Joe-user has a working version of 
Freenet (we hope) while those interested in helping with NGR can do-so.

Ian.
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Apparent specialization but weird estimators

[Niklas Bergh]

Same here.. I am running build 6366 and 40+ of the 50 estimators look
exactly the same.. However the look of that *the same* seems to be changing
pretty rapidly..

/N
- Original Message - 
From: "Ian Clarke" <[EMAIL PROTECTED]>
To: "Discussion of development issues" <[EMAIL PROTECTED]>
Sent: Saturday, November 29, 2003 10:33 AM
Subject: [freenet-dev] Apparent specialization but weird estimators


> My node appears to be specializing somewhat, however all of the
> estimators on nodestatus.html now look very similar... what gives?
>
> Ian.
> ___
> Devl mailing list
> [EMAIL PROTECTED]
> http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
>

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Separating dev and stable

[Niklas Bergh]

A thought.. Why not give NGR development a couple of weeks break and try to
get muxing or something up and running in the mean time? When you have no
relly good ideas on some specific problem it can be good to move over to a
completely different matter for a while and then get back to tackling the
original hurdles with renewed energy afterwards.


/N

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] GNUnet economic model (was Re: Separating dev and stable)

[Ian Clarke]

Martin Stone Davis wrote:
I just skimmed through http://www.ovmj.org/GNUnet/download/ebe.ps, which 
describes GNUnet's economic model.  In short, we would prioritize 
requests based, in part, on how well requesting nodes have serviced our 
requests in the past.  The benefit would be a system in which 
well-behaved nodes are rewarded during times when resources are short.
And the disadvantage of such a system, as I have pointed out many times 
in the past, is that new users get a terrible experience of Freenet, 
which is a *very* bad thing for the network as new users should be 
encouraged, not punished.

Ian.
___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] GNUnet economic model (was Re: Separating dev and stable)

[Martin Stone Davis]

S M wrote:

"Martin Stone Davis" <[EMAIL PROTECTED] 
> 
wrote in message news:[EMAIL PROTECTED]

 > 1. We probably should go with Toad's idea of dropping the HTL system 
 >in&nb sp;favor of a timeout-based system.  This is a radical change, 
and >needs to be thought-out a bit more before it is implemented.  
However, I >would think we should be able to finish the design "on 
paper" in less >than a week.

If radical changes are going to be made it is a good opportunity to 
borrow the best aspects from GNUnet, (including their economic routing 
model, and the ability to host permanent specific files). 

What's more is there is no need to start from scratch to design anything 
"on paper" since it’s already done; taking the best parts from open 
source is one reason why it's so good.
I just skimmed through http://www.ovmj.org/GNUnet/download/ebe.ps, which 
describes GNUnet's economic model.  In short, we would prioritize 
requests based, in part, on how well requesting nodes have serviced our 
requests in the past.  The benefit would be a system in which 
well-behaved nodes are rewarded during times when resources are short.

Some kind of GNUnet-style economic model could be built into Freenet, 
but what crucial problem would it solve that won't be solved by the plan 
I outlined?

I'm not saying that it wouldn't be a good thing to have down the road, 
as I can see it protecting us from freeloaders without needlessly 
punishing them.  However, now is the time to get Freenet to work well 
even in a network of mostly well-behaved peers (with a few malicious 
black-holes and freeloaders thrown in for good measure).

-Martin

___
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl