Dispatch Router TCP throughput test over a 40 Gbit/sec link,
with 0, 1, and 2 routers.
=================================================================
I am including complete setup details below, because I would
like some help making sure I did everything right.
*Basic idea *
Two good-sized machines with a 40 Gbit link between
them. An iperf3 server running on one machine, an
iperf3 client running on the other. Dispatch Routers
running on both machines, and connected to each other.
The iperf3 client connects directly to the iperf3 server
for the 0-routers test, or it connects to the router on
the other box for the 1-router test, or it connects to
the router on its own box for the two-router test.
I make sure that the data really is following the path
I think it is (in the 1- and 2-router tests) by killing
one or more involved routers and confirming that the
client can no longer connect.
Both Dispatch Routers have BUFFER_SIZE = 4096
in src/buffer.c .
The tests run for 60 seconds.
*Result *
I'll just give the summary numbers for now. I'll make some
graphs later so we can see how smooth or choppy the throughput
was. Also I will run all of these 5x to see variability.
But for now, mostly interested in help checking correctness.
*0 routers : 22.1 Gbits / sec* ( iperf3 server at 100% CPU )
*1 router : 21.9 GBits / sec*
*2 routers : 21.7 GBits / sec*
*Setup*
*The machines :*
Brontonomicon : 32 core, 256 GB -- the server runs here
10.10.10.1 on the 40 Gbe interface
Colossus-Guardian : 16 core, 128 GB -- the client runs here
10.10.10.2 on the 40 Gbe interface
*Router Config Files :*
Brontonomicon
router {
debugDump: qddebug.txt
mode: interior
id: A
workerThreads: 32
}
address {
prefix: closest
distribution: closest
}
listener {
stripAnnotations: no
idleTimeoutSeconds: 120
saslMechanisms: ANONYMOUS
host: 0.0.0.0
role: normal
authenticatePeer: no
port: 5672
linkCapacity: 10
}
listener {
role: inter-router
host: 10.10.10.1
port: 5800
saslMechanisms: ANONYMOUS
}
# Accept TCP traffic over the fast link.
tcpListener {
host: 10.10.10.1
port: 5801
address: pair-1
siteId: my-site
}
# Send TCP traffic to server on this box.
tcpConnector {
host: 0.0.0.0
port: 5802
address: pair-1
siteId: my-site
}
Colossus-Guardian
router {
debugDump: qddebug.txt
mode: interior
id: B
workerThreads: 16
}
address {
prefix: closest
distribution: closest
}
listener {
stripAnnotations: no
idleTimeoutSeconds: 120
saslMechanisms: ANONYMOUS
host: 0.0.0.0
role: normal
authenticatePeer: no
port: 5672
linkCapacity: 10
}
listener {
role: inter-router
host: 10.10.10.2
port: 5860
saslMechanisms: ANONYMOUS
}
connector {
role: inter-router
host: 10.10.10.1
port: 5800
saslMechanisms: ANONYMOUS
}
# Accept TCP traffic from client on this box.
tcpListener {
host: 0.0.0.0
port: 5861
address: pair-1
siteId: my-site
}
# Send TCP traffic to other router over fast link.
tcpConnector {
host: 10.10.10.1
port: 5801
address: pair-1
siteId: my-site
}
*The Iperf3 Commands*
On Br (10.10.10.1), the server command is always the same :
iperf3 -s -p 5802
On CG (10.10.10.2), the client commands are :
For 0-router, connect directly to the server :
iperf3 -c 10.10.10.1 -p 5802 -t 60 -P 3
( "-P 3" means "use 3 parallel streams". This causes the
server on the other box to go to 100% CPU. )
For 1-router, connect to the router on Bronto :
iperf3 -c 10.10.10.1 -p 5801 -t 60 -P 3
For 2-router, connect to the router on this box :
iperf3 -c 0.0.0.0 -p 5861 -t 60 -P 3
