On Nov 7, 2011, at 5:07 PM, Robert Horton wrote: > On Mon, 2011-11-07 at 15:45 +0100, Vincent Diepeveen wrote: >> What's the second one doing, is this just in case the switch fails, >> a >> kind of 'backup' port? >> >> In my naivity i had thought that both ports together formed the >> bidirectional link to the switch. >> So i thought that 1 port was for 10 gigabit upstream and the other >> port was for 10 gigabit downstream, >> did i misunderstood that? > > It's "normal" to just use single port cards in a compute server. You > might want to use 2 (or more) to increase the bandwidth to a > particular > machine (might be useful for a fileserver, for instance) or if you are > linking nodes to each other (rather than via a switch) in a taurus- > type > topology. > > Rob >
It's still not clear to me what exactly the 2nd link is doing. If i want to ship th emaximum amount of short messages, say 128 bytes each message, is a 2nd cable gonna increase the number of messages i can ship? In fact the messages i'll be shipping out is requests to read remote in a blocking manner 128 bytes. So say this proces P at node N0 wants from some other node N1 exactly 128 bytes from the gigabytes big hashtable. That's a blocked read. The number of blocked reads per second that can read a 128 bytes is the only thing that matters for the network, nothing else. Note it will also do writes, but with writes you always can be doing things in a more tricky manner. So to speak you can queue up a bunch and ship them. Writes do not need to be non-blocking. If they flow at a tad slower speed to the correct node and get written that's also ok. The write is 32 bytes max. In fact i don't want to read 128 bytes. As that 128 bytes is 4 entries and as in such cluster network it's a layered system, if i would be able to modify the source code doing the read, all i would give is a location, the host processor then can do the read of 32 bytes and give that. As i assume the network to be silly and not able to execute remote code, i read 128 bytes and figure out here which of the 4 positions *possible* is the correct position stored (odds about a tad more than 5% that a position already was stored before). So ideally i'd be doing reads of 32 bytes, yet as the request for the read is not capable of selecting the correct position, it has to scan 128 bytes for it, so i get the entire 128 bytes. The number of 128 byte reads per second randomized over the hashtable that's spreaded over the nodes, is the speed at which the 'mainsearch' can search. I'm guessing blocked reads to eat nearly 10 microseconds with infiniband 4x, so that would mean i can do about a 100k lookups a card. Question is whether connecting the 2nd port would speedup that to more than 100k reads per second. _______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
