On Fri, 12 Mar 2021 at 11:10, Michael Goulish <[email protected]> wrote:
>
> Dispatch Router TCP throughput test over a 40 Gbit/sec link,
> with 0, 1, and 2 routers.
> =================================================================
>
>
> I am including complete setup details below, because I would
> like some help making sure I did everything right.
>
When you figure that out, perhaps it could drive adding some
documentation so others dont have the same issue.
>
>
> *Basic idea *
>
> Two good-sized machines with a 40 Gbit link between
> them. An iperf3 server running on one machine, an
> iperf3 client running on the other. Dispatch Routers
> running on both machines, and connected to each other.
> The iperf3 client connects directly to the iperf3 server
> for the 0-routers test, or it connects to the router on
> the other box for the 1-router test, or it connects to
> the router on its own box for the two-router test.
>
> I make sure that the data really is following the path
> I think it is (in the 1- and 2-router tests) by killing
> one or more involved routers and confirming that the
> client can no longer connect.
>
> Both Dispatch Routers have BUFFER_SIZE = 4096
> in src/buffer.c .
>
> The tests run for 60 seconds.
>
>
> *Result *
>
> I'll just give the summary numbers for now. I'll make some
> graphs later so we can see how smooth or choppy the throughput
> was. Also I will run all of these 5x to see variability.
>
> But for now, mostly interested in help checking correctness.
>
>
>
> *0 routers : 22.1 Gbits / sec* ( iperf3 server at 100% CPU )
>
> *1 router : 21.9 GBits / sec*
>
> *2 routers : 21.7 GBits / sec*
>
>
>
> *Setup*
>
> *The machines :*
>
> Brontonomicon : 32 core, 256 GB -- the server runs here
> 10.10.10.1 on the 40 Gbe interface
>
> Colossus-Guardian : 16 core, 128 GB -- the client runs here
> 10.10.10.2 on the 40 Gbe interface
>
>
>
>
> *Router Config Files :*
>
>
> Brontonomicon
>
> router {
> debugDump: qddebug.txt
> mode: interior
> id: A
> workerThreads: 32
> }
> address {
> prefix: closest
> distribution: closest
> }
> listener {
> stripAnnotations: no
> idleTimeoutSeconds: 120
> saslMechanisms: ANONYMOUS
> host: 0.0.0.0
> role: normal
> authenticatePeer: no
> port: 5672
> linkCapacity: 10
> }
> listener {
> role: inter-router
> host: 10.10.10.1
> port: 5800
> saslMechanisms: ANONYMOUS
> }
>
> # Accept TCP traffic over the fast link.
> tcpListener {
> host: 10.10.10.1
> port: 5801
> address: pair-1
> siteId: my-site
> }
>
> # Send TCP traffic to server on this box.
> tcpConnector {
> host: 0.0.0.0
> port: 5802
> address: pair-1
> siteId: my-site
> }
>
>
>
>
> Colossus-Guardian
>
>
> router {
> debugDump: qddebug.txt
> mode: interior
> id: B
> workerThreads: 16
> }
> address {
> prefix: closest
> distribution: closest
> }
> listener {
> stripAnnotations: no
> idleTimeoutSeconds: 120
> saslMechanisms: ANONYMOUS
> host: 0.0.0.0
> role: normal
> authenticatePeer: no
> port: 5672
> linkCapacity: 10
> }
> listener {
> role: inter-router
> host: 10.10.10.2
> port: 5860
> saslMechanisms: ANONYMOUS
> }
> connector {
> role: inter-router
> host: 10.10.10.1
> port: 5800
> saslMechanisms: ANONYMOUS
> }
>
> # Accept TCP traffic from client on this box.
> tcpListener {
> host: 0.0.0.0
> port: 5861
> address: pair-1
> siteId: my-site
> }
>
> # Send TCP traffic to other router over fast link.
> tcpConnector {
> host: 10.10.10.1
> port: 5801
> address: pair-1
> siteId: my-site
> }
>
>
>
>
>
>
> *The Iperf3 Commands*
>
>
>
> On Br (10.10.10.1), the server command is always the same :
>
> iperf3 -s -p 5802
>
>
>
> On CG (10.10.10.2), the client commands are :
>
> For 0-router, connect directly to the server :
>
> iperf3 -c 10.10.10.1 -p 5802 -t 60 -P 3
>
> ( "-P 3" means "use 3 parallel streams". This causes the
> server on the other box to go to 100% CPU. )
>
>
> For 1-router, connect to the router on Bronto :
>
> iperf3 -c 10.10.10.1 -p 5801 -t 60 -P 3
>
>
> For 2-router, connect to the router on this box :
>
> iperf3 -c 0.0.0.0 -p 5861 -t 60 -P 3
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
