Ian writes:
> I think that the issue is that a request forwarded to the node
> in-question, the chance that the DataSource (ie. the reference with
> which a datastore entry will be replaced) will actually be *that* node
> is quite high, in fact, it is 100% if that node is maliciously setting
> all
Ian writes:
> Perhaps, but given the current mechanism it does influence how often
> that node is contacted - a node will be contacted roughly in proportion
> to the number of references it has in the datastore, which will create a
> positive feedback loop and make the problem even worse.
I don't
If anything I think you should lower the maximum HTL allowed by the
nodes (not just clients, I mean network nodes) to the 20-30 range or
perhaps even lower. Theoretically this should make things work better.
My feeling is that all those inserts with HTL=100 are flattening the
search space and keep
Ray Heasman wrote:
> 3. Insertion of data
>
> My understanding is that FreeNet performs a depth-first backtracking graph
> traversal of nodes on the network.
Technically, yes, but keep in mind that backtracking happens only in
two circumstances: if you hit a dead node (hopefully rare), or if you
h
As long as everyone keeps inserting everything onto all nodes, the Freenet
search algorithm won't work. There will be no way to follow a path to a
particular node where the data is stored, if all nodes have approximately
the same data. This causes requests to fail, so people try to fix it
by inse
Theo writes:
> I would argue the opposite. I think that our best defense is to try as
> hard as possible to make a blocking mechanism impossible.
> [quoting an essay on eff.org:]
> "A file-sharing technology provider such as
> Freenet that is incapable of blocking access to users or disabling file
Theo writes, regarding random announcement:
> A priori it seems like it might be a problem, but in practice when I did
> simulations it seemed to work well enough. Certainly it's a huge
> improvement over inform.php -- the inform-style announcement really broke
> down over 10,000 nodes, and no twe
I ain't no lawyer either, but I looked at the DMCA and wrote a quick
analysis of how various provisions might or might not apply to Freenet
nodes, at http://www.geocrawler.com/archives/3/929/2000/3/50/3523198/.
My conclusion:
In summary, Freenet nodes in the US will probably be obligated to h
Oskar writes:
> - I don't see why all the Bob's can collude to control the value, since
> Alice also has a value XORed into the result. Of course, they can decide
> as they wish what key value they actually reference Alice from, so it
> doesn't matter. I think better stated the goal is that for any
I've looked some more at Oskar's announcement protocol,
http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/~checkout~/docs/announcement.txt?rev=1.1&content-type=text/plain&cvsroot=freenet
As far as I can see, the protocol is cryptographically sound. I have
two concerns, which don't deal with any crypt
A new mailing list is getting started to discuss general issues of
peer-to-peer style file-sharing systems like Freenet, MojoNation, Publius,
Gnutella and the like. Information is at:
http://www.transarc.com/~ota/bluesky/index.html.
Their charter:
The purpose of the mailing list is to foste
Mark Roberts wrote:
> mjr::mjr$ gcj --main=Test -o test BigInteger.java Test.java ; ./test
> -1937774878715506038028866200512230656843600612196659062553089891228649461609973656722618166188603602526225775155515485421847013634702706240071776472595388606430915043103270813449621149194273928529962997365
Mark J. Roberts, , writes:
> import java.math.BigInteger;
>
> public class Test {
> public static void main(String[] args) {
> BigInteger x = new BigInteger("1000");
> BigInteger y = new BigInteger("4294967296");
> System.out.println(x.divide(y));
> }
Mark J. Roberts, , writes:
> You can find BigInteger.java at
>
> CHK at J9WTpJLqAfc9Y7yaj1V1JHJdeEQOAwE,qba6avE3DyH6qGsgYIZS1A
>
> or http://24.131.185.22/BigInteger.java. Check out the big scary private
> divide method.
Actually that divide method doesn't even do the actual divide. It just
forma
Oskar wrote:
> Hal, if you have time I would still appreciate some comments on the
> announcement proposal:
> http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/~checkout~/docs/announcement.txt?rev=1.1&content-type=text/plain&cvsroot=freenet
I need to think about it some more, but I did see one possibl
Matthew Toseland writes about insert problems:
> Sounds reasonable, except that I can insert many other files with different
> filenames. It is definitely key specific.
The main developers have been at the P2P conference this week so you
may have to wait a few days for an authoritative answer.
M
Oskar writes:
> The DataLength includes all the control characters and alike. This is for
> abstractions sake, DataLength is used by the connection layer to know how much
> to read of the socket, it shouldn't have to deal with control bytes and alike
> (except for being able to interrupted) when th
More comments on http://whiterose.sourceforge.net/r3proto.pdf:
> New Key Types
>
> The KHK is dead, long live the new key types. From the protocol point
> of view, a KSK is strictly a subset of the more general SVK keytype. The
> key type for SVK is 0x0201 and for KSK it s 0x0202.
>
> When you hav
I apologize for the use of text, which I know is not as portable or widely
supported as PDF. Hopefully users can find a text-to-pdf converter so
they will be able to read this message.
Adam wrote at http://whiterose.sourceforge.net/r3proto.pdf:
> Progressive CHKs
>
> The concept of a Content Has
Bram writes:
> I'm trying to change the logic to decide which neigbor to send a file to
> to be based on comparison in a ring - where's the code which does this? I
> just spent a while trying to figure it out.
Changing the forwarding logic should not be done lightly, as it is important
that all n
Ian writes:
> One idea might be to make nodes aquire the addresses of other nodes
> greedily - ie. if they see the address of another node (perhaps one which
> has sent a request) they add it to their datastore with a configurable
> probability. Additionally, we could support an additional field i
Itamar writes:
> Why is storing the directory contents in numbered "files" a good idea?
> Consider our original scenario - Bob loads directory listing - say,
> '.listing31', Alice does the same, adds new file and inserts an updated
> '.listing32'.
>
> Now, Bob adds a file, and tries to insert '.li
> > With C/C++ development it's easier, because CVS knows to ignore .o files.
> > ...But CVS doesn't know to ignore .class files,
>
> That is tricially solved by .cvsignore files. That's one of the first
> things you should do to set up for this project: $HOME/.cvsignore should
> have at least "*.
> I am aware that it is an easy mistake to make, which is why it is
> important to draw people's attention to the risk of it happening, so
> that they can watch out for it in future. It was not my intention to,
> nor would I ever, yell at anyone for making a mistake (this would be
> highly hypocri
> the trouble is, we don't have interoperability with 0.2 either! If
> some people are using a non-encrypting 0.2 release and other people
> are using an encrypting snapshot, there is going to be a big mess
> when their nodes try to talk to each other. (Probably there is
> already.)
Why don't w
> Ian, we agreed a couple of weeks ago that *we were breaking freenet on a
> regular basis now*. So don't be surprised that interoperability was
> hosed.=20
The problem with this is that most people aren't exercising the new code.
Having people actually use code as it is developed is the best way
> > I still don't see how the insert and search will find each other.
> > Think of a network with 100,000 nodes, and the data you want has been
> > inserted, but you are the first person fetching it. It lies on maybe
> > 10 nodes in the network. There are 10,000 to 1 odds against any given
> > no
Alex writes:
> This is why in my proposal, the data would be inserted in five different
> directions (one to the epicentre of A, one to the epicentre of B etc.)
Wouldn't this approach have the problem of overloading the nodes?
Keywords like "music", "sex", "freenet" would be essentially useless b
Neil writes (with Ian's qualified agreement):
> I think you put to much emphasis on the Inserts.
> The success factor for Freenet's routing is the distributed cashing, the
> fact that information migrates to where it is requested, this means that as
> long as enough nodes look in the same place the
Ian writes:
> The idea is that you have a search query like "contains("hello") and
> contains("goodbye") or matches("fred")". On recieving a request for
> data matching this key, each plain-text key in the datastore is compared
> against the search and given a score between 0 and 1, 1 being a per
> because I've written a lot of notwork protocol code and I know what
Yeah, I've written a lot of code like that, too...
(Sorry, couldn't resist.)
Hal
___
Freenet-dev mailing list
Freenet-dev at lists.sourceforge.net
http://lists.sourceforge.net/mailm
Lee writes:
> I wouldn't count Twofish out yet; in fact I'd be surprized in Rijndael
> won by itself. Twofish will also have better performance in Java.
>
> But I think the correct answer to the question of "which algorithm
> do we implement now" is "XOR"; i.e., let's make sure we can plug one
> i
> > Which cipher did we pick? Blowfish or Twofish?
>
> Let's go with Twofish.
If you're going to pick an AES candidate, you might want to go for the
one that has the best chance of winning. As Lucky says, Rijndael appears
to have a slight edge at this point.
Hal
___
> > Essentially this is a way of turning a block cipher into a stream
> > cipher.
> This isn't standard CFB, its a variation using a block-cipher as a
> keystream generator. And it just...might...work .
This is pretty standard, really. It represents the situation where the
"character size" is th
> > You're describing 8-bit-shift CFB. The variant that shifts 64 bits at
> > a time requires no more work than straight encryption.
> Yes, but the problem we're currently having is that we need to be able to
> send byte-at-a-time data over a wire. If we use a 64-bit-shift CFB, we
> have exactly
> I propose we go ahead and move to CFB mode in order to avoid this padding
> problem. The downside is that it requires blocksize/8 times as much CPU
> time (since it effectively encrypts one byte as one block. For blowfish,
> this would be 64/8=8 times more than ECB.
You're describing 8-bit-shi
The language of the regulation says:
3. Also in ?740.13, to, in part, take into account the "open source"
approach to software development, unrestricted encryption source
code not subject to an express agreement for the payment of a
licensing fee or royalty for commercial p
Regarding hashes:
> I'm not sure it is necessary for us though. I can see the advantage when
> sending large pieces of data where each part needs to be a certain size, but
> that is not really the case for us. Firstly data on freenet is limited in size
> anyways, and secondly it is not necessary fo
> Hmmm, yes, I see your point. I guess we should not go with the
> public-key in address idea - but I still fail to see why people are
> saying inter-node encryption is so difficult to achieve.
The main problem in the face of active attacks is to securely get the
right keys for other nodes. If a
A while back I described a way of hashing stream data such that bad data
could be detected and rejected before waiting for the end of the stream.
That was a two-level hash where each block of data gets hashed, and a
table of all these hashes is sent before the data; that table is itself
hashed to f
Theo writes:
> It is true that that won't stop someone from finding if you in particular
> are running a Freenet node. But something along these lines is still
> useful to make it much more difficult for an upstream provider to find out
> if any of its users are running Freenet nodes. With a text
Lee writes:
> I'm not sure there's any value at all to half a loaf here. Certainly we
> have to implement data store encryption, and even a simple implementation
> of that accomplishes important goals. But what important goal can be
> accomplished by simple node-to-node encryption? Without the f
> DH is a key exchange algorithm. If you already have a secret key shared
> between
> the nodes, or possibly each other public keys, then I can't see any reason in
> the world to do DH.
Actually there is good reason to do DH even if there are shared public
keys. (If you have shared secrets it i
> > The benchmark protocol for secure communications is SSL. Generally I
> > try to follow its principles in terms of what attacks it guards against.
> > SSL does not try to hide which symmetric cipher is used. And of course
> > it can't hide what kind of asymmetric ciphering is done since that h
On Thursday, Apr 27, 2000, "Scott G. Miller" writes:
>I can take this one in the manner Oskar suggested. I've already completed
>a key exchange and socket crypto layer. Its just a matter of hooking it
>up to freenet at the connection layer.
I assume that either the server or the client would hav
As far as ciphers go, the new US government standard cipher, called
AES, will be announced later this year. Whatever cipher we use now,
we should probably plan on switching to AES. Information on the ciphers
and timetable is available at http://csrc.nist.gov/encryption/aes/.
Five final candidate
Adam writes:
> I just want to clear something up.
>
> With StringKeys (the ones currently in use) is this true:
>
> is closer to ZZZ than AAAB?
Yes, although your question is worded a little awkwardly. What you
are really asking is, which is ZZZ closer to: or AAAB? The answer,
as you s
> After watching activity with full debugging on, It seems that most of the
> QueryRestarted messages are coming from loops. On a data request with high
> htl, 50 -100, from me and other sources, I would see the same message hit
> my node four or five times. I never saw one of these be successful.
Bill writes:
> Therefore, I propose that the chances of the DataSource in a DataSend
> being changed be something like
>
> 1 in 4 + 30 * sqrt(currentDataStoreSize/totalDataStoreSize)
>
> Under this regime, the chances of a change are as follows:
>
> sizeprobability
>
> > DSA signatures are 320 bits long.
>
> Isn't the public key between 512-1024 bits (given that the modulus and other
> constants are shared)?
Yes, sorry, that is basically correct. For some reason I got started
thinking of signatures, but that is not relevant here.
Actually a DSA public key ha
Lee writes:
> All keys, of any type, are 160-bit numbers with a
> 16-bit keytype value, so keyspace is 176 bits. KHKs
> are made by applying SHA to a text name; CHKs by
> applying SHA to the Document (including the metadata
> section, if any); SVKs by digitally signing a KHK
> with DS
> I fixed this, the findClosestKey method was using searchKey != null to check
> that maskKey was actually in the datastore, but I added null references. I
> changed it to h.get(maskKey) and I can no longer reproduce the bug.
Oh I see. What exactly do the null references mean? Data for which the
> kmm.lastAttempt keeps coming back as the same value, which suggests
> to me that findClosestKey() is doing something wrong. Strangely
> enough, kmm.lastAttempt is not very far from searchKey (searchKey
> = 33E9505D12942E8259A3C96FB6F88ED325B95797, kmm.lastAttempt =
> 3309CB034F7B337E9FADF0AEBC0D
Theo writes:
> Let's just consider a set of inserts and requests for the same routing key
> (assuming multiple inserts can share the same routing key). Imagine
> inserts and requests as ants and data as sugar. Initially one insert ant
> wanders into the network dropping sugar at various random pl
Hal says,
> Furthermore, the routing algorithm is the same for inserts as for
> requests, so overlaps occur for inserts as well. If someone inserts under
> key "mp3" it goes onto 5-10 nodes. Assuming we allow multiple docs per
> key, every such insert will overlap with each other on at least one
> I don't think this is really a problem. The thing is, the routing is not
> absolute -- it's not the case that globally, 123.56.78.* might have a
> really big affinity for the hash of the keyword mp3. Each node decides for
> itself which target it thinks might have an affinity for mp3, based on
Jim writes:
> For example:
> A = "1129948930"
> B = "1129148930"
> C = "1129633462"
>
> C is closer to A than B is.
>
> First, have I read the code correctly and second, is C really closer to
> A than B?
First, yes, that's how it works, and second, that is not a meaningful
question. "Closeness"
I want to reiterate a comment I made earlier, with regard to storing
things into the Freenet under a "searchkey" like mp3. This is not going
to work, because too many documents will use that keyword, and they will
all try to go onto that one node (even if the "documents" are just index
or metadata
Michael Wiktowy wrote:
> the general request (aka search):
> Someone is interested in a general subject (say mp3s) and maybe even a
> specific topic (say a particular music group). They make an attempt at
> guessing a key. This may just be a keyword. Since they don't know exactly
> what they are
Mike writes:
> The cpu load of all these CHK generations is the only thing that may be
> a problem if the CH algorithm is too complex. However, it doesn't need
> to be overly complex since more uniqueness will be provided by hashing
> it wil the KHK. Also, generating the hash once on insert and the
Oskar writes:
> So the standard we will follow for now is: Storeable fields stay, but they
> should not contain any meta-data that is content information or gives away
> what
> the data is (including things like filetype, mime-type, etc). Instead, such
> data should be appended before the document
JBS wrote:
>
> hal at finney.org wrote:
> >
> > JBS wrote, regarding Yarrow:
> > > The code measures the amount of time in between two real time clock
> > > ticks in terms of the number of iterations of a tight loop (closely
> > > approximates clock cycles).
> >
> > I wasn't aware that Yarrow did
JBS wrote, regarding Yarrow:
> The code measures the amount of time in between two real time clock
> ticks in terms of the number of iterations of a tight loop (closely
> approximates clock cycles). This number varies based on a number of
> factors including random fluctuations in the speed of the
> The comments in the Linux kernel claim that SHA does not leak
> information. Are you saying that it does, or that it could? The two
> are not the same thing at all. A block cypher could leak information
> (i.e. be flawed), in theory.
The original version of Yarrow did use a hash function rath
As a US resident, I am very happy that the government has relaxed the
rules for crypto export. All that is necessary is to send a single email
to the proper government address informing them that crypto export is
going on from the given address (the sourceforge server). I don't have
the address h
Mike writes:
> The TOS flags that I would suggest would be:
> normal : (default) the way things are routed now
> throughput : the node accesses the node status table and strongly biases the
> routing
> to nodes with high bandwidth
> speed : the node accesses the node status table and biases routi
I'm concerned that adding node latency/throughput to the routing algorithm
will make it less successful. The whole point of the algorithm is to
take me to the node that gives me the best chance of finding the data.
That's why we think that we can have a million nodes and still find
items even with
> > I'm having trouble doing doing it from the command-line client. I'll take a
> > whack at debugging the argument parser when I get home.
>
> Just enclose the key in doublequotes. That should work on most shells.
It doesn't work. The relevant line in freenet_request is:
java Freenet.client.$
Adam writes:
> Correct me if i am mistaken, but currently the only method for host
> discovery is for someone to specific connect to or be connected to
> another freenet node.
No, there are several mechanisms, and the one you suggest isn't one.
Connecting to another node does not allow it to "dis
Bill writes:
> WHICH MEANS, the spec needs a precise, detailed, and
> humanly-intelligible description of the closeness metric/algorithm.
> I'm not sure I really understand it, even after reading the source,
> so I'll go willy-nilly here and propose one: simply order the keys by
> alphabetical orde
Lee writes:
> Here's a possible way to make "Depth" useful: have the originator of
> a Request set it to a small random value (say 1-10). On each forward
> hop of the request, Depth is incremented as HopsToLive is decremented--
> but unlike HopsToLive, it is not further incremented on subsequent
> acme.com - isnt that by the same guy who made the netpbm/pbmplus
> image libraries which are like the basis for all graphics conversion
> stuff on *nix?
Right, it's Jef Poskanzer. Pretty sharp guy.
Hal
___
Freenet-dev mailing list
Freenet-dev at lis
BTW another source for free Java crypto is http://www.acme.com/. They
seem to be straightforward and clean implementations. The distribution
policy is liberal:
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditi
Mike asks:
> Is that right? I understand where HTL comes in -- you don't want to be
> passing each request to millions of nodes. But what does Depth do? From
> what's been said, it comes into play when, for example, B crashes before
> receiving the reply from C. Yes? But in that case, C would
> in.available() returns 0 on two conditions, there is nothing to read
> because the other node has not sent a message, and there is nothing
> to read because the stream has ended. This is very annoying. Someone
> tell me a good way (short of reading the data, which I can figure out
> for myself, b
Oskar writes:
> It looks like, especially from Hal's logs, that we are seeing a lot of
> restarting of chains when it isn't necessary. My experimenting last night did
> not seem to produce too much of this (I sent a message for an unexistant key
> with htl 100 which TimedOut after about a minute wi
Paul writes:
> java.lang.NullPointerException:
> at Freenet.SendFailedException.(SendFailedException.java:9)
> at Freenet.Message.sendBack(Message.java:278)
> at Freenet.message.InsertReply.pReceived(InsertReply.java:33)
> at Freenet.Message.received(Message.java:166
> > I am also bothered by 127.0.0.1:19114 being in the list; could this cause
> > the node to try to connect to itself?
>
> Trying to connect to yourself should be caught an handled correctly anyways.
> L185 in Message.java.
if (myAddress.equals(peer))
throw new SendFailedExcep
I don't think the problem with searching is "exponential growth" of the
number of nodes searched (with increasing HTL).
Rather, it is "N squared growth" in the load on the network, where N is
the number of nodes.
With smart searches, the load on the network grows linearly with the
number of nodes
> > java.lang.NullPointerException
> > at Freenet.Message.sendBack(Message.java, Compiled Code)
> > at Freenet.message.InsertReply.pReceived(InsertReply.java, Compiled
> > Code)
> > at Freenet.Message.received(Message.java, Compiled Code)
> > at
> > Freenet.node.St
FYI here is the patch I have applied locally to ConnectionManager.java for
the server to print a one line summary of each message sent or received.
This can be useful to see overall patterns of behavior (or misbehavior)
and then you can dig into the detailed logs (which I have set to debugging
leve
David writes:
>
> Just updated from cvs, when I started the node it produced this -
>
>
> java.lang.IllegalArgumentException:
> java.lang.reflect.InvocationTargetException
> at Freenet.node.Node.main(Node.java, Compiled Code)
>
> The node is running, I'm not sure what this means.
I th
I wrote:
> Src:tcp/129.116.44.244:4865 htl:5 depth:25 id:1f6d7522f9041601
> type:InsertRequest In
> Dst:tcp/198.11.16.81:19114 htl:4 depth:26 id:1f6d7522f9041601
> type:InsertRequest Out
> Dst:tcp/129.116.44.244:4865 htl:26 depth:0 id:1f6d7522f9041601
> type:QueryRestarted Out
> Src:tcp/140.239.
I have patched my ConnectionHandler.java to print a one liner for every
incoming and outgoing message. This gives me a "big picture" printout
of what is going on. Looking at this I see some things which look like
bugs. For example:
Src:tcp/207.96.1.41:2399 htl:21 depth:0 id:13a8750574694912 typ
> I'm writing this into the installation notes right now, and reading
> the notes, something that disturbs me is that the Freenet Node is
> being refered to a Freenet Server everywhere. Nodes are not servers,
> they are nodes. Refering to it as a server is bad because:
>
> b) If they are servers th
> > available() has the nice feature of not blocking. However it will return 0
> > even if the stream has been closed, like you noticed. What I ultimately
> > had
> > to do was use setSoTimeout() on the Socket so it wouldn't block forever
> > (and
> > ignore the InterruptedIOExceptions), and
I changed the java clients to return nonzero exit status if the request
or insert command failed.
Hal
___
Freenet-dev mailing list
Freenet-dev at lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/freenet-dev
87 matches
Mail list logo
