Re: Celsius 451 -the melting point of Cat-5 Re: network topology
On Fri, 29 Mar 2002, Major Variola (ret) wrote: 3. Slow connections, slow machines Thanks to gamers, ping latencies are getting better. ADSL is a pain, but even 128 kBit upstream can be useful, if aggregated from multiple sites. Queries for distributed P2P search engines should use ACKless protocols, obviously. To resist 1. you can use port 80, which ISPs can't block without losing most 'legitimate' utility for the masses :-) Or you use randomly Um, you can, just block incoming connections. It's a problem with REST. varying ports and have to do more door-knocking. If you run a P2P-agnostic firewall, you'll have a problem with random incoming ports. I suggest camouflaging as bona fide traffic, including gaming and streaming multimedia. To resist 2. you have to be able to randomly probe IP addresses to find a node. Yes, probabilistic headless node discovery vs. a centralist approach. Now that I write it up, I realize a tree has the flaw that child nodes' queries must go through slow upstream links. So I will think about algorithms to grow meshes dynamically, robustly, to overcome that problem. Don't use trees, trees are stupid. Use high-dimensional meshes. We welcome comments pointers, and apologize for the rambling.
Re: CDR: Re: network topology
Ben Laurie wrote: James B. DiGriz wrote: Jim Choate wrote: Draw a picture. If you don't have a place to post it I can arrange a page gratis. You take three nodes. Arrange them in a ring/triangle. Each node branches to 295(?) other nodes (making it a member of three 100 node subnets - somehow these numbers don't add up). It's not clear if those are a 'one to many' branch or if that node simply has two links to two other nodes in the ring (which has a total of 100 nodes). And where did the '2 other triangles' come from? We start with a single triange that is a member of a larger set the nodes of which are the members of a -two triangle- set? Why is 'our' triangle 'single'? Is this a 'big version' of the 'Caveman World'? The evil triangles have been banished for now. I played with graphviz for a while last night and it's easy enough to see that this is a torus. Surely not - in a torus you have loops of nodes, whereas here we have each node directly connected to 99 others in each segment. It may be a bit like a torus, but it isn't one. Spose it might be a set of interconnected 100-dimensional toruses (my head hurts). Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff Yes, that's what was throwing me, too, and why I couldn't reconcile 1,000,000 with the three segments. The connections to the other 297 nodes. This means you have 100 cycles for each segment. In an ordinary torus there's only one. Am I wrong or isn't this just the hypercube anolog for a torus, that is, a hypertorus. If not I suppose you'd just have to call it a polycyclic torus. It does provide a small diameter for such a large number of nodes. jbdigriz
Celsius 451 -the melting point of Cat-5 Re: network topology
I've been thinking about noncentralized self-organizing network topologies since George posted his query. First, there are several problems that any P2P network faces in the future hostile world: 1. ISPs blocking its ports 2. The entry points to P2P are vulnerable ---web sites that point to dynamic list of *tella servents, or the Kazaa site that points to active Kazaa supernode servents. Simply sue any of the sites with lists of *tella hosts. Even better, get the ISPs to drop host lists as fast as they drop stuff under DMCA. 3. Slow connections, slow machines 4. Active, hostile attacks, not just doofuses querying with too-common keywords To resist 1. you can use port 80, which ISPs can't block without losing most 'legitimate' utility for the masses :-) Or you use randomly varying ports and have to do more door-knocking. To resist 2. you have to be able to randomly probe IP addresses to find a node. To manage 3., nodes should announce their capacities and the network should work with this info. (Cf. self-management in Morpheus/Kazaa, where users can disable the 'supernode' function). Caching probably helps a lot. So what you have is your client randomly 'knocking on doors' looking for someone to answer with the handshake that means, 'yes, I'm in the network'. Then the knocker asks to join, and joins the resident's cell :-) or is told to try another known to be in the network. I then started thinking about how to keep nodes connected in a single connected graph. I ended up with an algorithm which I realized resembled funky tree growing/balancing algorithms I had seen in datastruct/algorithm CS classes years ago. Only the nodes were hosts, the links were communication links (IP addr=pointers) instead of in-memory locations pointers. Nodes can add children, keep track of their parents (and several great-ancestors for redundancy), and defer requests to join to their children. When nodes leave, they tell their children to reparent to the leaving-node's parent. When parents defer join-requests to their children, they first do an application-ping to see if the child is still alive. When the 'root' node leaves, it tells each of its children that they are each other's parents, to keep the graph connected. I'm going to code up a simulation when I get the time, with nodes randomly receiving join requests and randomly leaving, gracefully and not gracefully. This does not handle various hostile/spoofing attacks, crypto authentication, content-finding-query routing, managing slow connections, etc. Its about keeping a virtual network together. Now that I write it up, I realize a tree has the flaw that child nodes' queries must go through slow upstream links. So I will think about algorithms to grow meshes dynamically, robustly, to overcome that problem. We welcome comments pointers, and apologize for the rambling. -- In the future, you will need a state license for a general-purpose computer, otherwise you can only run state-approved code. Similarly for Inet connections which transport other than approved protocols on well known ports. Celsius 451 -the melting point of Cat-5
Re: Celsius 451 -the melting point of Cat-5 Re: network topology
At 09:28 AM 3/29/02 -0800, Greg Broiles wrote: At 08:55 AM 3/29/2002 -0800, Major Variola (ret) wrote: 1. ISPs blocking its ports 2. The entry points to P2P are vulnerable ---web sites that point to dynamic list of *tella servents, or the Kazaa site that points to active Kazaa supernode servents. Simply sue any of the sites with lists of *tella hosts. Even better, get the ISPs to drop host lists as fast as they drop stuff under DMCA. [...] To resist 2. you have to be able to randomly probe IP addresses to find a node. This sounds like a bad assumption to me - both because it seems unworkable given the size of the IPv4 address space (without even thinking about IPv6), and because randomly probing other machines isn't likely to be allowed (or successful) in a more security-aware environment, which is what the DMCA and its ilk are creating. Re IPv6: yes, you'd have to restrict search to netblocks known to be used by folks at home. Randomly probing machines is legal if not intended to disable the destination. And how else can you find peers if a fixed site that refers you to them is succeptible to attack? The very fact that such a fixed site would be advertizing its services brings (unwanted legal) attention to it, even as it helps the user community through this same attention. If the probe were disguised as an HTTP request it would not be seen as a probe, but as a misconfigured browser request. Also, from an inbound perspective, it's not sensible to respond to incoming queries from unknown users with potentially incriminating information - e.g., If he's connected to my port 31337, he's here for my warez, I'll give him a full list! - because what looks like an inbound random probe may be a sweep performed by hostile actors, e.g., http://www.mediaenforcer.com or http://www.baytsp.com. But how do you admit new nodes without also admitting spies? This is how those spy-sites work. Naive self-organization is not a reasonable approach for a hostile environment. P2P content networks exist (and have always existed) in a hostile environment. Designs which depend on friendly behavior on the part of unknown counterparties are doomed. Eliminate the friendly assumption, or eliminate the unknown aspect of the counterparties before transacting with them. How *do* you stop hostile entities from finding your network? How do you admit benign users without accepting spies? Passwords and the like offer small obstacles to spies and prevent network deployment. Yes, you can have the equivalent of invisible 'private clubs' but how do you open a general gallery to the public without admitting spies who report that you're reading copyrighted poems without paying the author? I don't see how crypto for authentication, confidentiality, or stego for concealment can help. All the RIAA has to do is get the congresshits to pass a few laws making Freenet *tella list sites illegal, ISP's responsible, and publicly-accessable P2P is toast. Random probing and forged-source encrypted UDP packets seem like a good place to start... when the nightmare of the RIAA shredding the 1st and 4th becomes the present. Thanks
gnutella's problems (Re: network topology)
On Wed, Mar 27, 2002 at 04:56:32PM -0800, [EMAIL PROTECTED] wrote: I got the impression (maybe wrong) that guntella as it exists is something much worse than a tree, that connections are pretty much haphazard and when you send out a query it reaches the same node by multiple paths, and that you really need the query ID to keep from forwarding massive duplicates. I think you're right and that's how it works, which is why it scales badly as the search messages flooded seven hops deep into the randomly organized network even with duplicate suprresion based on query-id (or whatever they use), end up consuming some significant proportion of the network bandwidth. The other problem apparently is that for a low bandwidth node (eg dialup) the searches can saturate your link, so you can hardly do anything but receive and answer search queries. Apparently there are some hacks to reduce this problem, but Gnutella's other big problem is that there are lots of independent clients so some of the problems come from interoperability problems, bugs etc. And gnutella is not able to resume a transfer that dies part way through which is very bad for download reliability. FastTrack/Kazza (but no longer Morpheus since the Kazza / Morpheus fall-out) on the other hand can resume, and in fact do multiple simultaneous downloads from multiple nodes having the same content so that it gets the content both much faster and much more reliably. Also helps cope with different link speeds as a group of slow nodes or asymmetric bandwidth nodes (like cable with fast down but limited up) can satisfy the download of cable and other broadband users. There's a nice write-up about the gnutella's problem's on openp2p.com [1]. Contrary to what article [2] claims FastTrack/Kazza really does blow Gnutella away, the supernode concept with high performance nodes elected to be search hubs makes all the difference. Gnutella last I tried it was barely functional for downloads, ~95% of downloads failed, and searches were much slower. Adam [1] Gnutella: Alive, Well and Changing Fast, by Kelly Truelove http://www.openp2p.com/pub/a/p2p/2001/01/25/truelove0101.html [2] Gnutella Blown Away? Not Exactly, by Serguei Osokine http://www.openp2p.com/pub/a/p2p/2001/07/11/numbers.html
Re: network topology
On Wednesday, March 27, 2002, at 04:56 PM, [EMAIL PROTECTED] wrote: On 27 Mar 2002 at 22:43, Eugene Leitl wrote: On Wed, 27 Mar 2002 [EMAIL PROTECTED] wrote: I don't recall ever having read of this type of structure before, but it seems so obvious that I'm sure it's been discussed before. So is there a name for it? Does anyone use it? has it been shown to be utterly worthless? You don't mean something like this: http://www.perfdynamics.com/Papers/Gnews.html do you? Yeah, I think what I was describing was more or less what they call a hypercube, or maybe just a cube. I'm not one of those people that can actually envision multidimensional structures, so I only know this is a 4-cube if I see the coordinates. No need to visualize 4D spaces, 5D spaces, and so on. Think in terms of how many nearest neighbors a point is, a la my last post: -- in a 1-dimensional space/topology, 2 nearest neighbors (left, right) -- in a 2-dimensional space/topology, 4 nearest neighbors (left, right, above, below) -- in a 3-dimensional space/topology, 6 nearest neighbors (6 faces of a cube) -- in a 4-dimensional space/topology, 8 nearest neighbors -- in an n-dimensional space/topology, 2n nearest neighbors (all using the definition of distance in terms of unit vectors, not diagonals) Actual machine hypercubes are just made in the obvious way: by connecting vertices/nodes the way a physical hypercube would be connected. In fact, since wires are small [though not infinitely small, and certainly not of zero propagation delay], it's possible to connect every node to every other node. --Tim May That the said Constitution shall never be construed to authorize Congress to infringe the just liberty of the press or the rights of conscience; or to prevent the people of the United States who are peaceable citizens from keeping their own arms. --Samuel Adams
Re: gnutella's problems (Re: network topology)
On 28 Mar 2002 at 2:18, Adam Back wrote: And gnutella is not able to resume a transfer that dies part way through which is very bad for download reliability. FastTrack/Kazza (but no longer Morpheus since the Kazza / Morpheus fall-out) on the other hand can resume, and in fact do multiple simultaneous downloads from multiple nodes having the same content so that it gets the content both much faster and much more reliably. Actually, the gnucleus client will do both of these, so presumably the gnutella morpheus does also since it's based on gnucleus. Also helps cope with different link speeds as a group of slow nodes or asymmetric bandwidth nodes (like cable with fast down but limited up) can satisfy the download of cable and other broadband users. There's a nice write-up about the gnutella's problem's on openp2p.com [1]. Contrary to what article [2] claims FastTrack/Kazza really does blow Gnutella away, the supernode concept with high performance nodes elected to be search hubs makes all the difference. Gnutella last I tried it was barely functional for downloads, ~95% of downloads failed, and searches were much slower. Adam I think the idea (used in alpine) of using UDP for search queries and only establishing a persistent connection when you actually want to transfer a file is a good one. George [1] Gnutella: Alive, Well and Changing Fast, by Kelly Truelove http://www.openp2p.com/pub/a/p2p/2001/01/25/truelove0101.html [2] Gnutella Blown Away? Not Exactly, by Serguei Osokine http://www.openp2p.com/pub/a/p2p/2001/07/11/numbers.html
Re: gnutella's problems (Re: network topology)
In article [EMAIL PROTECTED], Adam Back [EMAIL PROTECTED] wrote: And gnutella is not able to resume a transfer that dies part way through which is very bad for download reliability. FastTrack/Kazza (but no longer Morpheus since the Kazza / Morpheus fall-out) on the other hand can resume, and in fact do multiple simultaneous downloads from multiple nodes having the same content so that it gets the content both much faster and much more reliably. Also helps cope with different link speeds as a group of slow nodes or asymmetric bandwidth nodes (like cable with fast down but limited up) can satisfy the download of cable and other broadband users. Wait; as far as I know, Fasttrack's and Gnutella's file-transfer protocols are *identical* (but not their search protocols). If Gnutella doesn't support resuming downloads and grabbing from many people at once, that's just a client-side issue, not a protocol issue. [That being said, grabbing from multiple people at once requires you know *who's got* the _very same file_. The FastTrack protocol supports search by hash value, but Gnutella doesn't seem to. Should be easy to fix, though.] - Ian
Re: gnutella's problems (Re: network topology)
Adam Back writes: Contrary to what article [2] claims FastTrack/Kazza really does blow Gnutella away, the supernode concept with high performance nodes elected to be search hubs makes all the difference. Gnutella last I tried it was barely functional for downloads, ~95% of downloads failed, and searches were much slower. You should try again. That article you quoted about Gnutella's problems was over a year old. The network has improved enormously in the past year. Today I would say that 70% of downloads succeed, although of course it works better for more popular files. We have restarting of failed downloads, and simultaneous downloads from multiple users. With a widely shared file I sometimes max out this DSL line at 80-90 KB/sec downloads (that's bytes not bits). One problem is that searches are still slow and it is still hard to find uncommon files. FastTrack is going to die. The spyware installed by the clients pisses users off, and the use of centralized servers is just too big a target for the infinitely deep pockets of the recording industry. Gnutella is now a very adequate replacement, which was certainly not true a year ago. In the long run, decentralized networks are the only ones which can survive.
