Module Name: src
Committed By: pooka
Date: Mon Mar 29 02:01:47 UTC 2010
Added Files:
src/share/examples/rump/virtual_ip_router: README.txt rumprouter.c
Log Message:
Add an example for a rump router cluster setup along with a README.
some contributions to the code from Martti Kuparinen
To generate a diff of this commit:
cvs rdiff -u -r0 -r1.1 src/share/examples/rump/virtual_ip_router/README.txt \
src/share/examples/rump/virtual_ip_router/rumprouter.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Added files:
Index: src/share/examples/rump/virtual_ip_router/README.txt
diff -u /dev/null src/share/examples/rump/virtual_ip_router/README.txt:1.1
--- /dev/null Mon Mar 29 02:01:47 2010
+++ src/share/examples/rump/virtual_ip_router/README.txt Mon Mar 29 02:01:47 2010
@@ -0,0 +1,107 @@
+ $NetBSD: README.txt,v 1.1 2010/03/29 02:01:47 pooka Exp $
+
+Using rump it is possible to build a router test setup consisting
+of thousands of NetBSD IP stacks within a single host OS, one
+networking stack per application process. Each IP stack instance
+has its own set of interfaces, addresses and routing tables. These
+instances may or may not share the same code, i.e. it is possible
+to do compatibility testing of new features. The advantage over
+using full-fledged virtual OS setups (qemu, Xen, etc.) is scalability:
+the rump IP router base runtime takes less than 500kB of memory
+per instance.
+
+The code is _ONLY AN EXAMPLE_ as opposed a fully featured test kit.
+Some code tweaking is probably required to make this do what you
+want. Usage examples follow.
+
+To use one single rump networking stack instance with access to
+two real networks, you need tap and bridge on the host system (yes,
+this involves some memory copies. the resulting router setup can
+still saturate a GigE, though. it should not be difficult to bring
+performance to be ~the same as an in-kernel stack, but haven't
+managed to implement that yet).
+
+Anyway, the following can be done with the current code:
+
+/*
+ * Usage:
+ *
+ * # ifconfig yourrealif0 up
+ * # ifconfig tap0 create
+ * # ifconfig tap0 up
+ * # ifconfig bridge0 create
+ * # brconfig bridge0 add tap0 add yourrealif0
+ * # brconfig bridge0 up
+ * #
+ * # ifconfig yourrealif1 up
+ * # ifconfig tap1 create
+ * # ifconfig tap1 up
+ * # ifconfig bridge1 create
+ * # brconfig bridge1 add tap1 add yourrealif1
+ * # brconfig bridge1 up
+ * #
+ * # ./router virt0 192.168.1.1 255.255.255.0 192.168.1.255 \
+ * # virt1 192.168.2.1 255.255.255.0 192.168.2.255
+ *
+ * This will bind virtN to tapN and act as a router.
+ */
+
+As brilliant ascii art, it would look something like this:
+
+ network network
+ ^ ^
+ | |
+ /----v-------------\ /------------v----\
+ kernel | realif0 <-> tap0 | | tap1 -> realif1 |
+ \---------------^--/ \---^-------------/
+-------------------------|-------------------|--------------------
+ /----v-------------------v----\
+ user | virt0 <-> rump IP <-> virt1 |
+ \-----------------------------/
+
+(ok, no more drawing)
+
+The addresses configured to the rump virt0 and virt1 interfaces
+will be visible on the physical network, and their traffic can be
+examined with e.g. wireshark. You can also use wireshark on
+tap0/tap1.
+
+The alternate approach is to use purely internal simulation. The
+shmif rump driver uses a memory-mapped file as an ethernet "bus"
+between multiple rump networking stack instances. Just use
+rump_pub_shmif_create() in the code. This can also of course be
+combined with the tap setup, and you can have setups where border
+nodes talk to an internal mesh of shmif's. Semi-drawn, it looks
+like this:
+
+net1 <-> virt0, shm0 <-> shm1, shm2 <-> .... <-> shmN, virt1 <-> net1
+ (rump0) (rump1) .... (rumpN)
+
+Linear setups (where router n talks to exactly router n-1 and n+1)
+can be easily autogenerated. Here's a snippet of executed commands
+I used to start a few hundred routers (NOTE! the usage of the
+example code is different!):
+
+./a.out 10.0.0.1 10.0.0.255 /tmp/rumpshm_0 0 10.0.1.2 10.0.1.255 /tmp/rumpshm_1 10.0.1.1
+./a.out 10.0.1.1 10.0.1.255 /tmp/rumpshm_1 10.0.1.2 10.0.2.2 10.0.2.255 /tmp/rumpshm_2 10.0.2.1
+./a.out 10.0.2.1 10.0.2.255 /tmp/rumpshm_2 10.0.2.2 10.0.3.2 10.0.3.255 /tmp/rumpshm_3 10.0.3.1
+./a.out 10.0.3.1 10.0.3.255 /tmp/rumpshm_3 10.0.3.2 10.0.4.2 10.0.4.255 /tmp/rumpshm_4 10.0.4.1
+....
+./a.out 10.0.252.1 10.0.252.255 /tmp/rumpshm_252 10.0.252.2 10.0.253.2 10.0.253.
+255 /tmp/rumpshm_253 10.0.253.1
+./a.out 10.0.253.1 10.0.253.255 /tmp/rumpshm_253 10.0.253.2 10.0.255.1 10.0.255.
+255 /tmp/rumpshm_255 0
+
+Unfortunately I lost script used to produce that, but the algorithm
+is quickly obvious.
+
+Easy but slightly more interesting setups, such as a M^N matrix
+(hyper-matrix?) are also possible, but left as an exercise to the
+reader.
+
+Compiling the router depends a little on what networking domain
+and what interface you want to use for testing. The very basic
+setup with IP+virtif will get you quite far:
+
+cc rumprouter.c -lrumpnet_virtif -lrumpnet_netinet -lrumpnet_net -lrumpnet \
+ -lrump -lrumpuser -lpthread
Index: src/share/examples/rump/virtual_ip_router/rumprouter.c
diff -u /dev/null src/share/examples/rump/virtual_ip_router/rumprouter.c:1.1
--- /dev/null Mon Mar 29 02:01:47 2010
+++ src/share/examples/rump/virtual_ip_router/rumprouter.c Mon Mar 29 02:01:47 2010
@@ -0,0 +1,207 @@
+/* $NetBSD: rumprouter.c,v 1.1 2010/03/29 02:01:47 pooka Exp $ */
+
+/*
+ * Copyright (c) 2008 Antti Kantee. All Rights Reserved.
+ *
+ * Development of this software was supported by then
+ * Finnish Cultural Foundation.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
+ * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+
+#include <arpa/inet.h>
+
+#include <netinet/in.h>
+#include <net/route.h>
+
+#include <rump/rump.h>
+#include <rump/rump_syscalls.h>
+
+#include <err.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <sys/sockio.h>
+#include <arpa/inet.h>
+#include <net/if.h>
+#include <net/route.h>
+
+#undef DEBUG
+
+#ifdef DEBUG
+#define DP if (1) printf
+#else
+#define DP if (0) printf
+#endif
+
+static void
+configure_interface(const char *ifname, const char *addr, const char *mask,
+ const char *bcast)
+{
+ struct ifaliasreq ia;
+ struct sockaddr_in *sin;
+ int s, rv;
+
+ DP("Entering %s\n", __FUNCTION__);
+
+ DP("Create an interface(%s)\n", ifname);
+ s = atoi(ifname + strlen(ifname) - 1); /* XXX FIXME XXX */
+ if ((s = rump_pub_virtif_create(s)) != 0) {
+ err(1, "rump_pub_virtif_create(%d)", s);
+ }
+
+ DP("Get a socket for configuring the interface\n");
+ if ((s = rump_sys_socket(PF_INET, SOCK_DGRAM, 0)) < 0) {
+ err(1, "rump_sys_socket");
+ }
+
+ /* Address */
+ memset(&ia, 0, sizeof(ia));
+ strcpy(ia.ifra_name, ifname);
+ sin = (struct sockaddr_in *)&ia.ifra_addr;
+ sin->sin_family = AF_INET;
+ sin->sin_len = sizeof(struct sockaddr_in);
+ sin->sin_addr.s_addr = inet_addr(addr);
+
+ /* Netmask */
+ sin = (struct sockaddr_in *)&ia.ifra_mask;
+ sin->sin_family = AF_INET;
+ sin->sin_len = sizeof(struct sockaddr_in);
+ sin->sin_addr.s_addr = inet_addr(mask);
+
+ /* Broadcast address */
+ sin = (struct sockaddr_in *)&ia.ifra_broadaddr;
+ sin->sin_family = AF_INET;
+ sin->sin_len = sizeof(struct sockaddr_in);
+ sin->sin_addr.s_addr = inet_addr(bcast);
+
+ DP("Set the addresses\n");
+ rv = rump_sys_ioctl(s, SIOCAIFADDR, &ia);
+ if (rv) {
+ err(1, "SIOCAIFADDR");
+ }
+ rump_sys_close(s);
+ DP("Done with %s\n", __FUNCTION__);
+}
+
+static void
+configure_routing(const char *dst, const char *mask, const char *gw)
+{
+ size_t len;
+ struct {
+ struct rt_msghdr m_rtm;
+ uint8_t m_space;
+ } m_rtmsg;
+#define rtm m_rtmsg.m_rtm
+ uint8_t *bp = &m_rtmsg.m_space;
+ struct sockaddr_in sinstore;
+ int s, rv;
+
+ DP("Entering %s\n", __FUNCTION__);
+
+ DP("Open a routing socket\n");
+ s = rump_sys_socket(PF_INET, SOCK_DGRAM, 0);
+ if (s == -1) {
+ err(1, "rump_sys_socket");
+ }
+
+ memset(&m_rtmsg, 0, sizeof(m_rtmsg));
+ rtm.rtm_type = RTM_ADD;
+ rtm.rtm_flags = RTF_UP | RTF_GATEWAY | RTF_STATIC;
+ rtm.rtm_version = RTM_VERSION;
+ rtm.rtm_seq = 2;
+ rtm.rtm_addrs = RTA_DST | RTA_GATEWAY | RTA_NETMASK;
+
+ /* dst */
+ memset(&sinstore, 0, sizeof(sinstore));
+ sinstore.sin_family = AF_INET;
+ sinstore.sin_len = sizeof(sinstore);
+ sinstore.sin_addr.s_addr = inet_addr(dst);
+ memcpy(bp, &sinstore, sizeof(sinstore));
+ bp += sizeof(sinstore);
+
+ /* gw */
+ memset(&sinstore, 0, sizeof(sinstore));
+ sinstore.sin_family = AF_INET;
+ sinstore.sin_len = sizeof(sinstore);
+ sinstore.sin_addr.s_addr = inet_addr(gw);
+ memcpy(bp, &sinstore, sizeof(sinstore));
+ bp += sizeof(sinstore);
+
+ /* netmask */
+ memset(&sinstore, 0, sizeof(sinstore));
+ sinstore.sin_family = AF_INET;
+ sinstore.sin_len = sizeof(sinstore);
+ sinstore.sin_addr.s_addr = inet_addr(mask);
+ memcpy(bp, &sinstore, sizeof(sinstore));
+ bp += sizeof(sinstore);
+
+ len = bp - (uint8_t *)&m_rtmsg;
+ rtm.rtm_msglen = len;
+
+ DP("Set the route\n");
+ rv = rump_sys_write(s, &m_rtmsg, len);
+ if (rv != (int)len) {
+ err(1, "rump_sys_write");
+ }
+ rump_sys_close(s);
+ DP("Done with %s\n", __FUNCTION__);
+}
+
+static void
+usage(const char *argv0)
+{
+ printf("Usage: %s if1 if2 [route]\n", argv0);
+ printf("\n");
+ printf("where both \"if1\" and \"if2\" are\n");
+ printf("\n");
+ printf("ifname address netmask broadcast\n");
+ printf("\n");
+ printf("and \"route\" is an optional default route\n");
+ exit(1);
+}
+
+int
+main(int argc, char *argv[])
+{
+ if (argc < 9 || argc > 10) {
+ usage(argv[0]);
+ }
+
+ rump_init();
+ configure_interface(argv[1], argv[2], argv[3], argv[4]);
+ configure_interface(argv[5], argv[6], argv[7], argv[8]);
+ if (argc == 10) {
+ configure_routing("192.168.3.0", "255.255.255.0", "192.168.1.2");
+ }
+ printf("Press Ctrl+C to quit...");
+ pause();
+
+ return 0;
+}
