> > I am including complete setup details below, because I would > > like some help making sure I did everything right. > > > > *Robbie says:*
*When you figure that out, perhaps it could drive adding some* > * documentation so others dont have the same issue.* I'd be happy to work with you on that. Please note, however, that I am probably a Lot More Paranoid than most users. I mean --- I don't know if I'm *really* paranoid --- but I'm sure that's what everybody says about me! On Fri, Mar 12, 2021 at 11:01 AM Robbie Gemmell <[email protected]> wrote: > On Fri, 12 Mar 2021 at 11:10, Michael Goulish <[email protected]> wrote: > > > > Dispatch Router TCP throughput test over a 40 Gbit/sec link, > > with 0, 1, and 2 routers. > > ================================================================= > > > > > > I am including complete setup details below, because I would > > like some help making sure I did everything right. > > > > When you figure that out, perhaps it could drive adding some > documentation so others dont have the same issue. > > > > > > > > > *Basic idea * > > > > Two good-sized machines with a 40 Gbit link between > > them. An iperf3 server running on one machine, an > > iperf3 client running on the other. Dispatch Routers > > running on both machines, and connected to each other. > > The iperf3 client connects directly to the iperf3 server > > for the 0-routers test, or it connects to the router on > > the other box for the 1-router test, or it connects to > > the router on its own box for the two-router test. > > > > I make sure that the data really is following the path > > I think it is (in the 1- and 2-router tests) by killing > > one or more involved routers and confirming that the > > client can no longer connect. > > > > Both Dispatch Routers have BUFFER_SIZE = 4096 > > in src/buffer.c . > > > > The tests run for 60 seconds. > > > > > > *Result * > > > > I'll just give the summary numbers for now. I'll make some > > graphs later so we can see how smooth or choppy the throughput > > was. Also I will run all of these 5x to see variability. > > > > But for now, mostly interested in help checking correctness. > > > > > > > > *0 routers : 22.1 Gbits / sec* ( iperf3 server at 100% CPU ) > > > > *1 router : 21.9 GBits / sec* > > > > *2 routers : 21.7 GBits / sec* > > > > > > > > *Setup* > > > > *The machines :* > > > > Brontonomicon : 32 core, 256 GB -- the server runs here > > 10.10.10.1 on the 40 Gbe interface > > > > Colossus-Guardian : 16 core, 128 GB -- the client runs here > > 10.10.10.2 on the 40 Gbe interface > > > > > > > > > > *Router Config Files :* > > > > > > Brontonomicon > > > > router { > > debugDump: qddebug.txt > > mode: interior > > id: A > > workerThreads: 32 > > } > > address { > > prefix: closest > > distribution: closest > > } > > listener { > > stripAnnotations: no > > idleTimeoutSeconds: 120 > > saslMechanisms: ANONYMOUS > > host: 0.0.0.0 > > role: normal > > authenticatePeer: no > > port: 5672 > > linkCapacity: 10 > > } > > listener { > > role: inter-router > > host: 10.10.10.1 > > port: 5800 > > saslMechanisms: ANONYMOUS > > } > > > > # Accept TCP traffic over the fast link. > > tcpListener { > > host: 10.10.10.1 > > port: 5801 > > address: pair-1 > > siteId: my-site > > } > > > > # Send TCP traffic to server on this box. > > tcpConnector { > > host: 0.0.0.0 > > port: 5802 > > address: pair-1 > > siteId: my-site > > } > > > > > > > > > > Colossus-Guardian > > > > > > router { > > debugDump: qddebug.txt > > mode: interior > > id: B > > workerThreads: 16 > > } > > address { > > prefix: closest > > distribution: closest > > } > > listener { > > stripAnnotations: no > > idleTimeoutSeconds: 120 > > saslMechanisms: ANONYMOUS > > host: 0.0.0.0 > > role: normal > > authenticatePeer: no > > port: 5672 > > linkCapacity: 10 > > } > > listener { > > role: inter-router > > host: 10.10.10.2 > > port: 5860 > > saslMechanisms: ANONYMOUS > > } > > connector { > > role: inter-router > > host: 10.10.10.1 > > port: 5800 > > saslMechanisms: ANONYMOUS > > } > > > > # Accept TCP traffic from client on this box. > > tcpListener { > > host: 0.0.0.0 > > port: 5861 > > address: pair-1 > > siteId: my-site > > } > > > > # Send TCP traffic to other router over fast link. > > tcpConnector { > > host: 10.10.10.1 > > port: 5801 > > address: pair-1 > > siteId: my-site > > } > > > > > > > > > > > > > > *The Iperf3 Commands* > > > > > > > > On Br (10.10.10.1), the server command is always the same : > > > > iperf3 -s -p 5802 > > > > > > > > On CG (10.10.10.2), the client commands are : > > > > For 0-router, connect directly to the server : > > > > iperf3 -c 10.10.10.1 -p 5802 -t 60 -P 3 > > > > ( "-P 3" means "use 3 parallel streams". This causes the > > server on the other box to go to 100% CPU. ) > > > > > > For 1-router, connect to the router on Bronto : > > > > iperf3 -c 10.10.10.1 -p 5801 -t 60 -P 3 > > > > > > For 2-router, connect to the router on this box : > > > > iperf3 -c 0.0.0.0 -p 5861 -t 60 -P 3 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
