This is an automated email from the ASF dual-hosted git repository.
weizhou pushed a commit to branch network-namespace
in repository https://gitbox.apache.org/repos/asf/cloudstack-extensions.git
The following commit(s) were added to refs/heads/network-namespace by this push:
new 5bfe166 Network Namespace: support VPC
5bfe166 is described below
commit 5bfe166c2ebb9ae0968135d499e55c696928b5c2
Author: Wei Zhou <[email protected]>
AuthorDate: Tue Apr 7 21:48:38 2026 +0200
Network Namespace: support VPC
---
Network-Namespace/README.md | 590 ++++++++++++++++++-------
Network-Namespace/network-namespace-wrapper.sh | 519 +++++++++++++++++++---
Network-Namespace/network-namespace.sh | 66 ++-
3 files changed, 963 insertions(+), 212 deletions(-)
diff --git a/Network-Namespace/README.md b/Network-Namespace/README.md
index 3bfe1bf..b57982b 100644
--- a/Network-Namespace/README.md
+++ b/Network-Namespace/README.md
@@ -67,8 +67,8 @@ required**.
│ NetworkExtensionElement.java │
│ │ executes (path resolved from Extension record) │
│ ▼ │
-│ /etc/cloudstack/extensions/<ext-name>/ │
-│ network-namespace.sh │
+│ /usr/share/cloudstack-management/extensions/<ext-name>/ │
+│ <ext-name>.sh (network-namespace.sh) │
└──────────────────────┬───────────────────────────────────┘
│ SSH (host : port from extension details)
│ credentials from extension_resource_map_details
@@ -82,23 +82,24 @@ required**.
│ │
│ HOST side │
│ ───────────────────────────────────────────────── │
-│ eth0.1910 ─────────────────────────────────┐ │
+│ eth1.1910 ─────────────────────────────────┐ │
│ (VLAN sub-iface) │ │
-│ breth0-1910 (bridge) │
-│ veth-host-1910 ────────────────────────────┘ │
+│ breth1-1910 (bridge) │
+│ vh-1910-d1 ─────────────────────────────────┘ │
│ │ │
-│ NAMESPACE cs-net-209 (or cs-net-<vpcId>) │
+│ NAMESPACE cs-net-209 (isolated) │
+│ cs-vpc-5 (VPC, vpc-id=5) │
│ ───────────────────────────────────────────────── │
-│ veth-ns-1910 ← gateway IP 10.1.1.1/24 │
+│ vn-1910-d1 ← gateway IP 10.1.1.1/24 │
│ │
│ PUBLIC side (source-NAT IP 10.0.56.4 on VLAN 101): │
│ │
│ HOST side │
-│ eth0.101 ─────────────────────────────────┐ │
-│ breth0-101 (bridge) │
+│ eth1.101 ─────────────────────────────────┐ │
+│ breth1-101 (bridge) │
│ vph-101-209 ────────────────────────────────┘ │
│ │ │
-│ NAMESPACE cs-net-209 (or cs-net-<vpcId>) │
+│ NAMESPACE cs-net-209 (or cs-vpc-<vpcId>) │
│ vpn-101-209 ← source-NAT IP 10.0.56.4/32 │
│ default route → 10.0.56.1 (upstream gateway) │
└──────────────────────────────────────────────────────────┘
@@ -108,25 +109,23 @@ required**.
| Object | Name pattern | Example (VLAN 1910, net 209, pub-VLAN 101) |
|--------|--------------|-------------------------------------------|
-| Namespace (standalone network) | `cs-net-<networkId>` | `cs-net-209` |
-| Namespace (VPC network) | `cs-net-<vpcId>` | `cs-net-5` |
-| Guest host bridge | `br<ethX>-<vlan>` | `breth0-1910` |
+| Namespace (isolated network) | `cs-net-<networkId>` | `cs-net-209` |
+| Namespace (VPC network) | `cs-vpc-<vpcId>` | `cs-vpc-5` |
+| Guest host bridge | `br<ethX>-<vlan>` | `breth1-1910` |
| Guest veth – host side | `vh-<vlan>-<id>` | `vh-1910-d1` |
| Guest veth – namespace side | `vn-<vlan>-<id>` | `vn-1910-d1` |
-| Public host bridge | `br<pub_ethX>-<pvlan>` | `breth0-101` |
+| Public host bridge | `br<pub_ethX>-<pvlan>` | `breth1-101` |
| Public veth – host side | `vph-<pvlan>-<id>` | `vph-101-209` |
| Public veth – namespace side | `vpn-<pvlan>-<id>` | `vpn-101-209` |
-`ethX` (and `pub_ethX`) is the **physical NIC** resolved from the
-`kvmnetworklabel` (`public_kvmnetworklabel`) stored in the physical-network
-extension details:
+`ethX` (and `pub_ethX`) is the NIC specified in the `guest.network.device`
+(and `public.network.device`) key when registering the extension on the
+physical network. Both default to `eth1` when not explicitly set.
-* `eth1` → `eth1` (not in `/sys/devices/virtual/net/` → already physical)
-* `cloudbr1` → `eth1` (virtual bridge → first non-virtual
- `/sys/class/net/cloudbr1/brif/` member)
-
-Both labels are automatically included in
`--physical-network-extension-details`
-by `NetworkExtensionElement` — no extra registration step is needed.
+> **Note:** when `<vlan>` or `<id>` would make the interface name exceed the
+> Linux 15-character limit, the `<id>` portion is shortened to its hex
+> representation (for numeric IDs) or a 6-character MD5 prefix (for
+> non-numeric IDs).
**Key design principles:**
@@ -139,9 +138,10 @@ by `NetworkExtensionElement` — no extra registration step
is needed.
VLAN sub-interfaces live on the **host** (not inside the namespace) so that
guest VMs whose NICs are connected to `brethX-<vlan>` reach the namespace
gateway without any additional configuration.
-* **VPC networks** share a single namespace per VPC (`cs-net-<vpcId>`).
Multiple
- guest VLANs are each connected via their own veth pair (`veth-host-<vlan>` /
- `veth-ns-<vlan>`).
+* **VPC networks** share a single namespace per VPC (`cs-vpc-<vpcId>`).
Multiple
+ guest VLANs are each connected via their own veth pair (`vh-<vlan>-<id>` /
+ `vn-<vlan>-<id>`).
+* **Isolated networks** each get their own namespace (`cs-net-<networkId>`).
* The two scripts are intentionally decoupled: you can replace either script
with a custom implementation (Python, Go, etc.) as long as the interface
contract (arguments and exit codes) is maintained.
@@ -208,36 +208,56 @@ by `NetworkExtensionElement` — no extra registration step
is needed.
During package installation the `network-namespace.sh` script is deployed to:
```
-/etc/cloudstack/extensions/<extension-name>/network-namespace.sh
+/usr/share/cloudstack-management/extensions/<extension-name>/<extension-name>.sh
```
+The extension path is set to `network-namespace` at creation time;
+`NetworkExtensionElement` looks for `<extensionName>.sh` inside the directory.
In **developer mode** the extensions directory defaults to `extensions/`
relative
-to the repo root working directory, so
`extensions/network-namespace/network-namespace.sh`
-is found automatically when `path=network-namespace` is set at extension
creation
-time (CloudStack looks for `<extensionName>.sh` inside the directory).
+to the repo root, so `extensions/network-namespace/network-namespace.sh` is
+found automatically.
### Remote network device
-Copy `network-namespace-wrapper.sh` to each remote device that will act as the
-network gateway:
+Copy `network-namespace-wrapper.sh` to **each** remote device that will act as
the
+network gateway, inside a subdirectory named after the extension:
```bash
# From the CloudStack source tree:
+DEVICE=root@<kvm-host>
+EXT_NAME=network-namespace # must match the extension name in CloudStack
+
+ssh ${DEVICE} "mkdir -p /etc/cloudstack/extensions/${EXT_NAME}"
scp extensions/network-namespace/network-namespace-wrapper.sh \
- root@<device>:/etc/cloudstack/extensions/network-namespace-wrapper.sh
-chmod +x /etc/cloudstack/extensions/network-namespace-wrapper.sh
+ ${DEVICE}:/etc/cloudstack/extensions/${EXT_NAME}/${EXT_NAME}-wrapper.sh
+ssh ${DEVICE} "chmod +x
/etc/cloudstack/extensions/${EXT_NAME}/${EXT_NAME}-wrapper.sh"
```
-The default path expected by `network-namespace.sh` is
-`/etc/cloudstack/extensions/network-namespace/network-namespace-wrapper.sh`.
-You can override this per-physical-network by passing a `script_path` detail
-when calling `registerExtension` (see below).
+The wrapper derives its state directory and log path from the directory it is
+installed in:
+
+* **State:** `/var/lib/cloudstack/<ext-name>/`
+ (e.g. `/var/lib/cloudstack/network-namespace/`)
+* **Log (wrapper):** `/var/log/cloudstack/extensions/<ext-name>/<ext-name>.log`
+ (e.g.
`/var/log/cloudstack/extensions/network-namespace/network-namespace.log`)
+* **Log (proxy, on management server):**
`/var/log/cloudstack/extensions/<ext-name>.log`
**Prerequisites on the remote device:**
-* `iproute2` (`ip link`, `ip addr`, `ip netns`)
-* `iptables`
-* `sshd` running and reachable from the management server
-* The SSH user must have permission to run `ip` and `iptables` (root or `sudo`)
+
+| Package / tool | Purpose |
+|----------------|---------|
+| `iproute2` (`ip`, `ip netns`) | Namespace, bridge, veth, route management |
+| `iptables` + `iptables-save` | NAT and filter rules inside namespace |
+| `arping` | Gratuitous ARP after public IP assignment |
+| `dnsmasq` | DHCP and DNS service inside namespace |
+| `haproxy` | Load balancing inside namespace |
+| `apache2` (Debian/Ubuntu) or `httpd` (RHEL/CentOS) | Metadata / user-data
HTTP service (port 80) |
+| `python3` | DHCP options parsing, haproxy config generation, vm-data
processing |
+| `util-linux` (`flock`) | Serialise concurrent operations per network |
+| `sshd` | Reachable from the management server on the configured port
(default 22) |
+
+The SSH user must have permission to run `ip`, `iptables`, `iptables-save`,
+and `ip netns exec` (root or passwordless `sudo` for those commands).
---
@@ -321,6 +341,24 @@ This creates a **Network Service Provider** (NSP) entry
named `my-extnet` on the
physical network and enables it automatically. The NSP name is the **extension
name** — not the generic string `NetworkExtension`.
+After registering, set the connection details for the remote KVM device(s):
+
+```bash
+cmk updateRegisteredExtension \
+ extensionid=<extension-uuid> \
+ resourcetype=PhysicalNetwork \
+ resourceid=<phys-net-uuid> \
+ "details[0].key=hosts"
"details[0].value=192.168.10.1,192.168.10.2" \
+ "details[1].key=username" "details[1].value=root" \
+ "details[2].key=sshkey"
"details[2].value=<pem-key-contents>" \
+ "details[3].key=guest.network.device" "details[3].value=eth1" \
+ "details[4].key=public.network.device" "details[4].value=eth1"
+```
+
+The `hosts` value is a comma-separated list of KVM host IPs;
`ensure-network-device`
+picks one per network and stores it in `--network-extension-details`. Use
`sshkey`
+(PEM private key) for passwordless authentication, or `password` + `sshpass`.
+
Verify:
```bash
cmk listNetworkServiceProviders physicalnetworkid=<phys-net-uuid>
@@ -369,8 +407,8 @@ cmk updateNetworkOffering id=<offering-uuid> state=Enabled
```
> The `serviceCapabilityList` entries must match the values declared in the
-> extension's `network.capabilities` detail. If the extension's JSON does not
-> declare a capability value for a service, CloudStack accepts any value (or no
+> extension's `network.service.capabilities` detail. If the extension's JSON
does
+> not declare a capability value for a service, CloudStack accepts any value
(or no
> value) without error.
### 4. Create an isolated network
@@ -402,9 +440,11 @@ network-namespace-wrapper.sh implement \
--cidr 10.0.1.0/24
```
-The wrapper creates a VLAN sub-interface and Linux bridge, assigns the gateway
-IP to the bridge, enables IP forwarding, and creates dedicated per-network
-iptables chains (`CS_EXTNET_42` in `nat` and `CS_EXTNET_FWD_42` in `filter`).
+The wrapper creates a VLAN sub-interface and Linux bridge, a guest veth pair
+(`vh-100-2a`/`vn-100-2a`), assigns the gateway IP to the namespace veth,
+enables IP forwarding inside the namespace, and creates per-network iptables
+chains: `CS_EXTNET_42_PR` (nat PREROUTING), `CS_EXTNET_42_POST` (nat
+POSTROUTING), and `CS_EXTNET_FWD_42` (filter FORWARD).
### 5. Acquire a public IP and enable Source NAT
@@ -426,10 +466,15 @@ network-namespace.sh assign-ip \
```
The wrapper:
-1. Adds `203.0.113.10/32` as a secondary address on the physical interface.
-2. Adds an iptables SNAT rule: traffic from `10.0.1.0/24` outbound → source
`203.0.113.10`.
-3. Adds an iptables FORWARD rule allowing traffic from the guest CIDR to the
- physical interface.
+1. Creates public VLAN sub-interface `eth1.<pvlan>` and bridge
`breth1-<pvlan>` on the host.
+2. Creates veth pair `vph-<pvlan>-42` (host, in bridge) / `vpn-<pvlan>-42`
(namespace).
+3. Assigns `203.0.113.10/32` to `vpn-<pvlan>-42` **inside the namespace**.
+4. Adds host route `203.0.113.10/32 dev vph-<pvlan>-42` so the host can reach
it.
+5. Adds an iptables SNAT rule in `CS_EXTNET_42_POST`: traffic from
`10.0.1.0/24`
+ out `vpn-<pvlan>-42` → source `203.0.113.10`.
+6. Adds an iptables FORWARD ACCEPT rule in `CS_EXTNET_FWD_42` for the guest
CIDR.
+7. If `--public-gateway` is set, adds/replaces the namespace default route via
+ `vpn-<pvlan>-42`.
When the IP is released (via `disassociateIpAddress`), `release-ip` is called,
which removes all associated rules and the IP address.
@@ -454,15 +499,15 @@ network-namespace.sh add-static-nat \
--private-ip 10.0.1.5
```
-iptables rules added:
+iptables rules added (all run inside the namespace via `ip netns exec`):
```bash
-# DNAT inbound
-iptables -t nat -A CS_EXTNET_42 -d 203.0.113.20 -j DNAT --to-destination
10.0.1.5
-# SNAT outbound
-iptables -t nat -A CS_EXTNET_42 -s 10.0.1.5 -o eth0 -j SNAT --to-source
203.0.113.20
-# FORWARD inbound + outbound
-iptables -t filter -A CS_EXTNET_FWD_42 -d 10.0.1.5 -o cs-br-42 -j ACCEPT
-iptables -t filter -A CS_EXTNET_FWD_42 -s 10.0.1.5 -i cs-br-42 -j ACCEPT
+# DNAT inbound (CS_EXTNET_42_PR = nat PREROUTING chain)
+iptables -t nat -A CS_EXTNET_42_PR -d 203.0.113.20 -j DNAT --to-destination
10.0.1.5
+# SNAT outbound (CS_EXTNET_42_POST = nat POSTROUTING chain)
+iptables -t nat -A CS_EXTNET_42_POST -s 10.0.1.5 -o vpn-<pvlan>-42 -j SNAT
--to-source 203.0.113.20
+# FORWARD inbound + outbound (CS_EXTNET_FWD_42 = filter FORWARD chain)
+iptables -t filter -A CS_EXTNET_FWD_42 -d 10.0.1.5 -o vn-100-2a -j ACCEPT
+iptables -t filter -A CS_EXTNET_FWD_42 -s 10.0.1.5 -i vn-100-2a -j ACCEPT
```
```bash
@@ -498,14 +543,14 @@ network-namespace.sh add-port-forward \
--protocol TCP
```
-iptables rules added:
+iptables rules added (inside the namespace):
```bash
-# DNAT inbound
-iptables -t nat -A CS_EXTNET_42 -p tcp -d 203.0.113.20 --dport 2222 \
+# DNAT inbound (CS_EXTNET_42_PR = nat PREROUTING chain)
+iptables -t nat -A CS_EXTNET_42_PR -p tcp -d 203.0.113.20 --dport 2222 \
-j DNAT --to-destination 10.0.1.5:22
-# FORWARD
+# FORWARD (CS_EXTNET_FWD_42 = filter FORWARD chain)
iptables -t filter -A CS_EXTNET_FWD_42 -p tcp -d 10.0.1.5 --dport 22 \
- -o cs-br-42 -j ACCEPT
+ -o vn-100-2a -j ACCEPT
```
Port ranges (e.g. `80:90`) are supported and passed verbatim to iptables
`--dport`.
@@ -533,11 +578,16 @@ network-namespace.sh destroy --network-id 42 --vlan 100
The wrapper:
1. Removes jump rules from PREROUTING, POSTROUTING, and FORWARD.
-2. Flushes and deletes iptables chains `CS_EXTNET_42` and `CS_EXTNET_FWD_42`.
-3. Brings down and deletes bridge `cs-br-42`.
-4. Brings down and deletes VLAN interface `eth0.100` (VLAN ID read from state
if
- not passed in arguments).
-5. Removes all state under `/var/lib/cloudstack/network-namespace/42/`.
+2. Flushes and deletes iptables chains `CS_EXTNET_42_PR`, `CS_EXTNET_42_POST`,
+ `CS_EXTNET_FWD_42`, and any `CS_EXTNET_FWRULES_42` / `CS_EXTNET_FWI_*`
chains.
+3. Deletes public veth pairs (`vph-<pvlan>-42` / `vpn-<pvlan>-42`) that were
+ created during `assign-ip` (read from state files).
+4. On `destroy`: also deletes the guest veth host-side (`vh-100-2a`) and
removes
+ the namespace `cs-net-42` entirely.
+5. Removes all state under `/var/lib/cloudstack/<ext-name>/network-42/`.
+
+> The host bridge `breth1-100` and VLAN sub-interface `eth1.100` are **not**
+> removed — they may still be used by other networks or for VM connectivity.
### 9. Unregister and delete the extension
@@ -547,10 +597,15 @@ cmk updateNetworkServiceProvider id=<nsp-uuid>
state=Disabled
cmk deleteNetworkServiceProvider id=<nsp-uuid>
# Remove external network device credentials (if any)
-# Device credentials are stored as `extension_resource_map_details` for the
-# extension registration. Remove or update them via `updateExtension` or
-# by unregistering the extension from the physical network
(unregisterExtension)
-# and then updating the Extension record if necessary.
+# Device credentials are stored as extension_resource_map_details for the
+# extension registration. Remove or update them via `updateRegisteredExtension`
+# (set cleanupdetails=true to wipe all details) or by supplying new details.
+# Example: clear all registration details for a physical network:
+cmk updateRegisteredExtension \
+ extensionid=<extension-uuid> \
+ resourcetype=PhysicalNetwork \
+ resourceid=<phys-net-uuid> \
+ cleanupdetails=true
# Unregister the extension from the physical network
cmk unregisterExtension \
@@ -575,14 +630,18 @@ network:
# Register two extensions, each backed by a different device
cmk registerExtension id=<ext-a-uuid> resourcetype=PhysicalNetwork
resourceid=<pn-uuid>
cmk registerExtension id=<ext-b-uuid> resourcetype=PhysicalNetwork
resourceid=<pn-uuid>
-```
-# Store device connection details and script_path as registration details
-# (use updateNetworkServiceProvider or updateExtension details in the API /
CMK)
-# Example: set hosts, sshkey, script_path for the registered extension on the
physical network
-# Note: details are stored in extension_resource_map_details for the
registration
-cmk updateExtension id=<ext-uuid> "details[0].key=hosts"
"details[0].value=10.0.0.1,10.0.0.2" \
- "details[1].key=script_path"
"details[1].value=/etc/cloudstack/extensions/network-namespace/network-namespace-wrapper.sh"
+# Store device connection details as registration details for each extension.
+# Details are stored in extension_resource_map_details for the registration.
+# Example: set hosts and guest/public network devices for ext-a on the
physical network:
+cmk updateRegisteredExtension \
+ extensionid=<ext-a-uuid> \
+ resourcetype=PhysicalNetwork \
+ resourceid=<pn-uuid> \
+ "details[0].key=hosts"
"details[0].value=10.0.0.1,10.0.0.2" \
+ "details[1].key=guest.network.device" "details[1].value=eth1" \
+ "details[2].key=public.network.device" "details[2].value=eth1"
+```
When creating network offerings, reference the specific extension name:
@@ -613,10 +672,11 @@ It receives the command as its first positional argument
followed by named
`--option value` pairs.
All commands:
-* Write timestamped entries to `/var/log/cloudstack/network-namespace.log`.
-* Use a per-network flock file
(`/var/lib/cloudstack/network-namespace/lock-<id>`)
- to serialise concurrent operations.
-* Persist state under `/var/lib/cloudstack/network-namespace/<network-id>/`.
+* Write timestamped entries to
`/var/log/cloudstack/extensions/<ext-name>/<ext-name>.log`.
+* Use a per-network flock file (`${STATE_DIR}/lock-network-<id>`) — or
+ `lock-vpc-<id>` for VPC networks — to serialise concurrent operations.
+* Persist state under `/var/lib/cloudstack/<ext-name>/network-<network-id>/`
+ (or `vpc-<vpc-id>/` for VPC-wide shared state such as public IPs).
### `implement`
@@ -632,17 +692,17 @@ network-namespace-wrapper.sh implement \
```
Actions:
-1. Create namespace `cs-net-<vpc-id>` (VPC) or `cs-net-<network-id>`
(standalone).
-2. Resolve `ethX` from `kvmnetworklabel` in
`--physical-network-extension-details`.
-3. Create VLAN sub-interface `ethX.<vlan>` on the host.
-4. Create host bridge `brethX-<vlan>` and attach `ethX.<vlan>` to it.
-5. Create veth pair `veth-host-<vlan>` (host) / `veth-ns-<vlan>` (namespace).
- Attach host end to `brethX-<vlan>`.
-6. Assign `<gateway>/<prefix>` to `veth-ns-<vlan>` inside the namespace.
+1. Create namespace `cs-vpc-<vpc-id>` (VPC) or `cs-net-<network-id>`
(isolated).
+2. Resolve `GUEST_ETH` from `guest.network.device` in
`--physical-network-extension-details`
+ (defaults to `eth1` when absent).
+3. Create VLAN sub-interface `GUEST_ETH.<vlan>` on the host.
+4. Create host bridge `br<GUEST_ETH>-<vlan>` and attach `GUEST_ETH.<vlan>` to
it.
+5. Create veth pair `vh-<vlan>-<id>` (host, in bridge) / `vn-<vlan>-<id>`
(namespace).
+6. Assign `<gateway>/<prefix>` to `vn-<vlan>-<id>` inside the namespace.
7. Enable IP forwarding inside the namespace.
-8. Create iptables chains `CS_EXTNET_<id>_PR` (PREROUTING DNAT),
- `CS_EXTNET_<id>_POST` (POSTROUTING SNAT), and `CS_EXTNET_FWD_<id>`
(FORWARD).
-9. Save VLAN, gateway, CIDR, namespace, network-id or vpc-id to state files.
+8. Create iptables chains `CS_EXTNET_<id>_PR` (nat PREROUTING DNAT),
+ `CS_EXTNET_<id>_POST` (nat POSTROUTING SNAT), and `CS_EXTNET_FWD_<id>`
(filter FORWARD).
+9. Save VLAN, gateway, CIDR, namespace, and network-id / vpc-id to state files.
### `shutdown`
@@ -654,12 +714,15 @@ network-namespace-wrapper.sh shutdown \
```
Actions:
-1. Flush and remove iptables chains (PREROUTING, POSTROUTING, FORWARD jumps +
- chain contents).
-2. Delete public veth pairs (`vph-<pvlan>-<id>`) that were created during
- `assign-ip` (read from state).
-3. Keep namespace and guest veth (`veth-host-<vlan>` / `veth-ns-<vlan>`)
intact —
- guest VMs can still connect to `brethX-<vlan>`.
+1. Stop dnsmasq, haproxy, apache2, and password-server processes running inside
+ the namespace (if any).
+2. Flush and remove iptables chains (PREROUTING, POSTROUTING, FORWARD jumps +
+ chain contents), including `CS_EXTNET_FWRULES_<id>` and all
`CS_EXTNET_FWI_*`
+ ingress chains.
+3. Delete public veth pairs (`vph-<pvlan>-<id>` / `vpn-<pvlan>-<id>`) that were
+ created during `assign-ip` (read from state).
+4. Keep namespace and guest veth (`vh-<vlan>-<id>` / `vn-<vlan>-<id>`) intact —
+ guest VMs can still connect to `br<GUEST_ETH>-<vlan>`.
### `destroy`
@@ -671,14 +734,14 @@ network-namespace-wrapper.sh destroy \
```
Actions (superset of shutdown):
-1. Delete guest veth host-side (`veth-host-<vlan>`).
-2. Delete public veth pairs.
+1. Delete guest veth host-side (`vh-<vlan>-<id>`).
+2. Delete public veth pairs (`vph-<pvlan>-<id>` / `vpn-<pvlan>-<id>`).
3. Delete the namespace (removes all interfaces inside it).
-4. Remove state directory.
+4. Remove per-network state directory `network-<id>/`.
-> The host bridge `brethX-<vlan>` and VLAN sub-interface `ethX.<vlan>` are NOT
-> removed on destroy — they may still be used by other networks or for VM
-> connectivity.
+> The host bridge `br<GUEST_ETH>-<vlan>` and VLAN sub-interface
`GUEST_ETH.<vlan>`
+> are NOT removed on destroy — they may still be used by other networks or for
+> VM connectivity.
### `assign-ip`
@@ -699,10 +762,11 @@ network-namespace-wrapper.sh assign-ip \
```
Actions:
-1. Resolve `pub_ethX` from `public_kvmnetworklabel` (falls back to
`kvmnetworklabel`).
-2. Create VLAN sub-interface `pub_ethX.<pvlan>` and bridge
`brpub_ethX-<pvlan>` on the host.
+1. Resolve `PUB_ETH` from `public.network.device` in
`--physical-network-extension-details`
+ (defaults to `eth1` when absent).
+2. Create VLAN sub-interface `PUB_ETH.<pvlan>` and bridge
`br<PUB_ETH>-<pvlan>` on the host.
3. Create veth pair `vph-<pvlan>-<id>` (host) / `vpn-<pvlan>-<id>` (namespace).
- Attach host end to `brpub_ethX-<pvlan>`.
+ Attach host end to `br<PUB_ETH>-<pvlan>`.
4. Assign `<public-ip>/32` (or `/<prefix>` if `--public-cidr` given) to
`vpn-<pvlan>-<id>` inside the namespace.
5. Add host route `<public-ip>/32 dev vph-<pvlan>-<id>` so the host can reach
it.
@@ -760,10 +824,10 @@ iptables rules added (chains `CS_EXTNET_<id>_PR` /
`_POST` / `FWD_<id>`):
|-------|-------|------|
| `nat` | `CS_EXTNET_<id>_PR` | `-d <public-ip> -j DNAT --to-destination
<private-ip>` |
| `nat` | `CS_EXTNET_<id>_POST` | `-s <private-ip> -o vpn-<pvlan>-<id> -j SNAT
--to-source <public-ip>` |
-| `filter` | `CS_EXTNET_FWD_<id>` | `-d <private-ip> -o veth-ns-<vlan> -j
ACCEPT` |
-| `filter` | `CS_EXTNET_FWD_<id>` | `-s <private-ip> -i veth-ns-<vlan> -j
ACCEPT` |
+| `filter` | `CS_EXTNET_FWD_<id>` | `-d <private-ip> -o vn-<vlan>-<id> -j
ACCEPT` |
+| `filter` | `CS_EXTNET_FWD_<id>` | `-s <private-ip> -i vn-<vlan>-<id> -j
ACCEPT` |
-State saved to
`/var/lib/cloudstack/network-namespace/<id>/static-nat/<public-ip>`.
+State saved to `${STATE_DIR}/network-<id>/static-nat/<public-ip>`.
### `delete-static-nat`
@@ -792,17 +856,17 @@ network-namespace-wrapper.sh add-port-forward \
--protocol tcp|udp
```
-iptables rules added:
+iptables rules added (inside the namespace):
| Table | Chain | Rule |
|-------|-------|------|
-| `nat` | `CS_EXTNET_<id>` | `-p <proto> -d <public-ip> --dport <public-port>
-j DNAT --to-destination <private-ip>:<private-port>` |
-| `filter` | `CS_EXTNET_FWD_<id>` | `-p <proto> -d <private-ip> --dport
<private-port> -o cs-br-<id> -j ACCEPT` |
+| `nat` | `CS_EXTNET_<id>_PR` | `-p <proto> -d <public-ip> --dport
<public-port> -j DNAT --to-destination <private-ip>:<private-port>` |
+| `filter` | `CS_EXTNET_FWD_<id>` | `-p <proto> -d <private-ip> --dport
<private-port> -o vn-<vlan>-<id> -j ACCEPT` |
Port ranges (`80:90`) are passed verbatim to iptables `--dport`.
State saved to
-`/var/lib/cloudstack/network-namespace/<id>/port-forward/<proto>_<public-ip>_<public-port>`.
+`${STATE_DIR}/network-<id>/port-forward/<proto>_<public-ip>_<public-port>`.
### `delete-port-forward`
@@ -818,6 +882,217 @@ network-namespace-wrapper.sh delete-port-forward \
Removes the DNAT and FORWARD rules added by `add-port-forward`.
+### `apply-fw-rules`
+
+Called when CloudStack applies or removes firewall rules for the network.
+
+```
+network-namespace-wrapper.sh apply-fw-rules \
+ --network-id <id> \
+ --vlan <vlan-id> \
+ --fw-rules <base64-json> \
+ [--vpc-id <vpc-id>]
+```
+
+The `--fw-rules` value is a Base64-encoded JSON object:
+```json
+{
+ "default_egress_allow": true,
+ "cidr": "10.0.1.0/24",
+ "rules": [
+ {
+ "type": "ingress",
+ "protocol": "tcp",
+ "portStart": 22,
+ "portEnd": 22,
+ "publicIp": "203.0.113.10",
+ "sourceCidrs": ["0.0.0.0/0"]
+ },
+ {
+ "type": "egress",
+ "protocol": "all",
+ "sourceCidrs": ["0.0.0.0/0"]
+ }
+ ]
+}
+```
+
+iptables design (two independent parts, both inside the namespace):
+
+* **Ingress** (mangle PREROUTING, per public IP):
+ Per-public-IP chains `CS_EXTNET_FWI_<pubIp>` check traffic *before* DNAT so
+ the match is against the real public destination IP. Traffic not matched by
+ explicit ALLOW rules is dropped.
+
+* **Egress** (filter FORWARD, chain `CS_EXTNET_FWRULES_<networkId>`):
+ Inserted at position 1 of `CS_EXTNET_FWD_<networkId>`. Applies the
+ `default_egress_allow` policy (allow-by-default or deny-by-default) to VM
+ outbound traffic on `-i vn-<vlan>-<id>`.
+
+### `config-dhcp-subnet` / `remove-dhcp-subnet`
+
+Configure or tear down dnsmasq DHCP service for the network inside the
namespace.
+
+**`config-dhcp-subnet` arguments:**
+```
+network-namespace-wrapper.sh config-dhcp-subnet \
+ --network-id <id> \
+ --gateway <gw> \
+ --cidr <cidr> \
+ [--dns <dns-server>] \
+ [--domain <domain>] \
+ [--vpc-id <vpc-id>]
+```
+
+Actions: writes a dnsmasq configuration file under
+`${STATE_DIR}/network-<id>/dnsmasq/` and starts or reloads the dnsmasq process
+inside the namespace. DNS on port 53 is **disabled** by `config-dhcp-subnet`
+(use `config-dns-subnet` to enable it).
+
+**`remove-dhcp-subnet` arguments:**
+```
+network-namespace-wrapper.sh remove-dhcp-subnet --network-id <id>
+```
+
+Actions: stops dnsmasq and removes the dnsmasq configuration directory.
+
+### `add-dhcp-entry` / `remove-dhcp-entry`
+
+Add or remove a static DHCP host reservation (MAC → IP mapping) from dnsmasq.
+
+```
+network-namespace-wrapper.sh add-dhcp-entry \
+ --network-id <id> \
+ --mac <mac> \
+ --ip <vm-ip> \
+ [--hostname <name>] \
+ [--default-nic true|false]
+```
+
+When `--default-nic false`, the DHCP option 3 (default gateway) is suppressed
+for that MAC so the VM does not get a competing default route via a secondary
NIC.
+
+```
+network-namespace-wrapper.sh remove-dhcp-entry \
+ --network-id <id> \
+ --mac <mac>
+```
+
+### `set-dhcp-options`
+
+Set extra DHCP options for a specific NIC (identified by `--nic-id`) using a
+JSON map of option-code → value pairs.
+
+```
+network-namespace-wrapper.sh set-dhcp-options \
+ --network-id <id> \
+ --nic-id <nic-id> \
+ --options '{"119":"example.com"}'
+```
+
+### `config-dns-subnet` / `remove-dns-subnet`
+
+Enable or disable DNS (port 53) in the dnsmasq instance.
+
+```
+network-namespace-wrapper.sh config-dns-subnet \
+ --network-id <id> \
+ --gateway <gw> \
+ --cidr <cidr> \
+ [--extension-ip <ip>] \
+ [--domain <domain>] \
+ [--vpc-id <vpc-id>]
+```
+
+Actions: like `config-dhcp-subnet` but enables DNS on port 53. Also registers
a
+`data-server` hostname entry (using `--extension-ip` if provided, otherwise
+`--gateway`) for metadata service discovery.
+
+```
+network-namespace-wrapper.sh remove-dns-subnet --network-id <id>
+```
+
+Actions: disables DNS (rewrites config to disable port 53) but keeps DHCP
running.
+
+### `add-dns-entry` / `remove-dns-entry`
+
+Add or remove a hostname → IP mapping in the dnsmasq hosts file.
+
+```
+network-namespace-wrapper.sh add-dns-entry \
+ --network-id <id> \
+ --ip <vm-ip> \
+ --hostname <name>
+
+network-namespace-wrapper.sh remove-dns-entry \
+ --network-id <id> \
+ --ip <vm-ip>
+```
+
+### `save-vm-data`
+
+Write the full VM metadata/userdata/password set for a VM in a single call.
+Called on network restart and VM deploy.
+
+```
+network-namespace-wrapper.sh save-vm-data \
+ --network-id <id> \
+ --ip <vm-ip> \
+ --vm-data <base64-json>
+```
+
+The `--vm-data` value is a Base64-encoded JSON array of `{dir, file, content}`
+entries (same format as `generateVmData()` in the Java layer). Writes files
+under `${STATE_DIR}/network-<id>/metadata/<vm-ip>/latest/`. After writing,
+starts or reloads both the **apache2 metadata HTTP service** (port 80) and the
+**VR-compatible password server** (port 8080) inside the namespace.
+
+### `save-userdata` / `save-password` / `save-sshkey` /
`save-hypervisor-hostname`
+
+Granular variants that write individual VM metadata fields:
+
+```
+network-namespace-wrapper.sh save-userdata --network-id <id> --ip
<vm-ip> --userdata <base64>
+network-namespace-wrapper.sh save-password --network-id <id> --ip
<vm-ip> --password <plain>
+network-namespace-wrapper.sh save-sshkey --network-id <id> --ip
<vm-ip> --sshkey <base64>
+network-namespace-wrapper.sh save-hypervisor-hostname \
+ --network-id <id> --ip <vm-ip> --hypervisor-hostname <name>
+```
+
+Each command writes the relevant file and restarts/reloads apache2 (and
+the password server, for `save-password`).
+
+### `apply-lb-rules`
+
+Apply or revoke load-balancing rules via haproxy inside the namespace.
+
+```
+network-namespace-wrapper.sh apply-lb-rules \
+ --network-id <id> \
+ --lb-rules <json-array> \
+ [--vpc-id <vpc-id>]
+```
+
+`--lb-rules` is a JSON array of LB rule objects. Set `"revoke": true` on a
+rule to remove it. The wrapper regenerates the haproxy configuration from the
+persistent per-rule JSON files under `${STATE_DIR}/network-<id>/haproxy/` and
+reloads haproxy inside the namespace. haproxy is stopped when no active rules
+remain.
+
+### `restore-network`
+
+Batch-restore DHCP/DNS/metadata/services for all VMs on a network in a single
+call. Invoked on network restart to rebuild all state at once instead of N
+per-VM calls.
+
+```
+network-namespace-wrapper.sh restore-network \
+ --network-id <id> \
+ --restore-data <base64-json> \
+ [--gateway <gw>] [--cidr <cidr>] [--dns <dns>] \
+ [--domain <dom>] [--extension-ip <ip>] [--vpc-id <vpc-id>]
+```
+
### `custom-action`
```
@@ -830,11 +1105,12 @@ Built-in actions:
| Action | Description |
|--------|-------------|
-| `reboot-device` | Bounces the bridge: `ip link set cs-br-<id> down && up` |
-| `dump-config` | Prints iptables rules and bridge/interface state to stdout |
+| `reboot-device` | Bounces the guest veth pair (`vh-<vlan>-<id>` down → up) |
+| `dump-config` | Prints namespace IP addresses, iptables rules, and
per-network state to stdout |
To add custom actions, place an executable script at
-`/var/lib/cloudstack/network-namespace/hooks/custom-action-<name>.sh`.
+`${STATE_DIR}/hooks/custom-action-<name>.sh`
+(e.g. `/var/lib/cloudstack/network-namespace/hooks/custom-action-<name>.sh`).
Unknown action names are delegated to the hook if present; otherwise the
command
fails with a descriptive error.
@@ -861,37 +1137,34 @@ These keys are explicitly set when calling
`registerExtension`:
| `username` | SSH user — default: `root` |
| `password` | SSH password via `sshpass` — sensitive, not logged |
| `sshkey` | PEM-encoded SSH private key — sensitive, not logged; preferred
over password |
+| `guest.network.device` | Host NIC for guest (internal) traffic, e.g. `eth1`
— defaults to `eth1` when absent |
+| `public.network.device` | Host NIC for public (NAT/external) traffic, e.g.
`eth1` — defaults to `eth1` when absent |
-These keys are **automatically injected** by `NetworkExtensionElement` from the
-physical network record — no manual registration needed:
+This key is **automatically injected** by `NetworkExtensionElement` from the
+physical network record:
| JSON key | Description |
|----------|-------------|
| `physicalnetworkname` | Physical network name from CloudStack DB |
-| `kvmnetworklabel` | KVM guest traffic label (e.g. `eth0`, `cloudbr0`) |
-| `vmwarenetworklabel` | VMware guest traffic label |
-| `xennetworklabel` | XenServer guest traffic label |
-| `public_kvmnetworklabel` | KVM public traffic label (used for public
bridges) |
-| `public_vmwarenetworklabel` | VMware public traffic label |
-The wrapper script uses `kvmnetworklabel` (and `public_kvmnetworklabel`) to
-derive the physical NIC `ethX` via `/sys/devices/virtual/net/` inspection, then
-names bridges as `brethX-<vlan>`.
+The wrapper script uses `guest.network.device` (and `public.network.device`) to
+name bridges as `br<eth>-<vlan>` and veth pairs as `vh-<vlan>-<id>` /
+`vn-<vlan>-<id>` (guest) and `vph-<pvlan>-<id>` / `vpn-<pvlan>-<id>` (public).
### Per-network details (keys in `--network-extension-details`)
| JSON key | Description |
|----------|-------------|
| `host` | Previously selected host IP (set by `ensure-network-device`) |
-| `namespace` | Linux network namespace name (e.g. `cs-net-<networkId>`) |
+| `namespace` | Linux network namespace name (e.g. `cs-net-<networkId>` or
`cs-vpc-<vpcId>`) |
### Additional per-command arguments
| CLI Argument | Commands | Description |
|--------------|----------|-------------|
-| `--vpc-id <id>` | all | Inject when network belongs to a VPC; namespace
becomes `cs-net-<vpcId>` |
+| `--vpc-id <id>` | all | Present when the network belongs to a VPC; namespace
becomes `cs-vpc-<vpcId>` |
| `--public-vlan <pvlan>` | `assign-ip`, `release-ip` | Public IP's VLAN tag
(e.g. `101`) |
-| `--network-id <id>` | `assign-ip`, `release-ip`, `add-static-nat`,
`delete-static-nat`, `add-port-forward`, `delete-port-forward` | VPC ID if VPC
network, else network ID — used in public veth names (`vph-<pvlan>-<id>`,
`vpn-<pvlan>-<id>`) |
+| `--network-id <id>` | most | Network ID — CHOSEN_ID for veth names is
`<vpc-id>` when VPC, else `<network-id>` |
### Action parameters (custom-action only)
@@ -955,44 +1228,51 @@ argument; hook scripts should parse the JSON argument as
needed.
The integration smoke test at
`test/integration/smoke/test_network_extension_namespace.py`
-exercises the full lifecycle using a **Linux network namespace** on the Marvin
-node as the simulated remote device:
+exercises the full lifecycle against real KVM hosts in the zone.
```
-Marvin node (this machine — also acts as the remote network device)
- ├── ip netns add cs-extnet-<id> ← isolated namespace
- ├── ~/.ssh/authorized_keys ← test RSA key ← management server connects here
- └── /etc/cloudstack/extensions/
- └── network-namespace-wrapper.sh ← copied from repo by test
-
-KVM hosts in the zone (best-effort, skipped if none found)
- └── /etc/cloudstack/extensions/
- └── network-namespace-wrapper.sh ← copied by KvmHostDeployer
+Management server
+ └── /usr/share/cloudstack-management/extensions/<ext-name>/
+ └── network-namespace.sh ← deployed / referenced by test
+ SSHes to KVM host
+ runs network-namespace-wrapper.sh <cmd> <args>
-Management server (may be the same machine)
+KVM host(s) in the zone
└── /etc/cloudstack/extensions/<ext-name>/
- └── network-namespace.sh ← deployed by test (static copy)
- reads CS_PHYSICAL_NETWORK_EXTENSION_DETAILS (JSON)
- CS_NETWORK_EXTENSION_DETAILS (JSON)
- SSHes back to Marvin node :22
- runs ip netns exec cs-extnet-<id> <script> <args>
+ └── network-namespace-wrapper.sh ← copied to KVM hosts by test
setup
+ creates cs-net-<id> or cs-vpc-<id> namespaces
+ manages bridges, veth pairs, iptables, dnsmasq, haproxy, apache2
```
The test covers:
* Create / list / update / delete external network device.
* Full network lifecycle: implement → assign-ip (source NAT) → static NAT →
- port forwarding → shutdown / destroy.
+ port forwarding → firewall rules → DHCP/DNS → shutdown / destroy.
* NSP state transitions: Disabled → Enabled → Disabled → Deleted.
+* Tests `test_04`, `test_05`, `test_06` (DHCP, DNS, LB) require `arping`,
+ `dnsmasq`, and `haproxy` on the KVM hosts; the test skips them automatically
+ if these tools are not installed.
Run the test:
```bash
cd test/integration/smoke
-nosetests test_network_extension_provider.py \
+python -m pytest test_network_extension_namespace.py \
--with-marvin --marvin-config=<config.cfg> \
-s -a 'tags=advanced,smoke' 2>&1 | tee /tmp/extnet-test.log
```
-**Prerequisites:**
-* `iproute2` on the Marvin node (`ip netns list` must succeed).
-* The Marvin node must be reachable by SSH from the management server on port
22.
-* Set `MARVIN_NODE_IP=<ip>` if auto-detection of the Marvin node IP fails.
+**Prerequisites on KVM hosts:**
+* `iproute2` (`ip`, `ip netns`)
+* `iptables` + `iptables-save`
+* `arping` (for GARP on IP assignment)
+* `dnsmasq` (DHCP + DNS — required for `test_04` / DNS tests)
+* `haproxy` (LB — required for `test_05` / LB tests)
+* `apache2` / `httpd` (metadata HTTP service — required for UserData tests)
+* `python3` (vm-data processing, haproxy config generation)
+* `util-linux` (`flock`) (lock serialization)
+* SSH access from management server (root or sudo-capable user)
+
+**Prerequisites on the Marvin / test runner node:**
+* Python Marvin library installed (`pip install -r requirements.txt`)
+* A valid Marvin config file pointing to the CloudStack environment
+* The test runner must be able to SSH to the management server and to KVM hosts
diff --git a/Network-Namespace/network-namespace-wrapper.sh
b/Network-Namespace/network-namespace-wrapper.sh
index 8051c01..c8af377 100755
--- a/Network-Namespace/network-namespace-wrapper.sh
+++ b/Network-Namespace/network-namespace-wrapper.sh
@@ -349,6 +349,7 @@ pub_veth_ns_name() {
nat_chain() { echo "${CHAIN_PREFIX}_${1}"; }
filter_chain() { echo "${CHAIN_PREFIX}_FWD_${1}"; }
firewall_chain() { echo "${CHAIN_PREFIX}_FWRULES_${1}"; }
+acl_chain() { echo "${CHAIN_PREFIX}_ACL_${1}"; }
ensure_public_ip_on_namespace() {
local public_ip="$1" public_cidr="$2" pveth_n="$3" pveth_h="$4" addr_spec
prefix
@@ -460,10 +461,15 @@ parse_args() {
LB_RULES_JSON="[]"
DEFAULT_NIC="true"
VM_DATA=""
+ VM_DATA_FILE=""
DOMAIN=""
EXTENSION_IP=""
RESTORE_DATA=""
+ RESTORE_DATA_FILE=""
FW_RULES_JSON=""
+ FW_RULES_FILE=""
+ ACL_RULES_JSON=""
+ ACL_RULES_FILE=""
while [ $# -gt 0 ]; do
case "$1" in
@@ -495,10 +501,15 @@ parse_args() {
--lb-rules) LB_RULES_JSON="$2"; shift 2 ;;
--default-nic) DEFAULT_NIC="$2"; shift 2 ;;
--vm-data) VM_DATA="$2"; shift 2 ;;
+ --vm-data-file) VM_DATA_FILE="$2"; shift 2 ;;
--domain) DOMAIN="$2"; shift 2 ;;
--extension-ip) EXTENSION_IP="$2"; shift 2 ;;
--restore-data) RESTORE_DATA="$2"; shift 2 ;;
+ --restore-data-file) RESTORE_DATA_FILE="$2"; shift 2 ;;
--fw-rules) FW_RULES_JSON="$2"; shift 2 ;;
+ --fw-rules-file) FW_RULES_FILE="$2"; shift 2 ;;
+ --acl-rules) ACL_RULES_JSON="$2"; shift 2 ;;
+ --acl-rules-file) ACL_RULES_FILE="$2"; shift 2 ;;
# consumed by _pre_scan_args — skip silently
--physical-network-extension-details|--network-extension-details)
shift 2 ;;
@@ -577,7 +588,7 @@ _load_state() {
}
##############################################################################
-# Command: implement
+# Command: implement-network
#
# 1. Create namespace cs-net-<id>
# 2. Create host bridge br<ethX>-<vlan> with ethX.<vlan> sub-interface
@@ -586,11 +597,11 @@ _load_state() {
# 5. Set up iptables chains inside namespace
##############################################################################
-cmd_implement() {
+cmd_implement_network() {
parse_args "$@"
acquire_lock "${NETWORK_ID}"
- log "implement: network=${NETWORK_ID} ns=${NAMESPACE} vlan=${VLAN}
gw=${GATEWAY} cidr=${CIDR}"
+ log "implement-network: network=${NETWORK_ID} ns=${NAMESPACE} vlan=${VLAN}
gw=${GATEWAY} cidr=${CIDR}"
local veth_h veth_n nchain_pr nchain_post fchain
veth_h=$(veth_host_name "${VLAN}" "${CHOSEN_ID}")
@@ -706,22 +717,22 @@ cmd_implement() {
_dump_iptables "${NAMESPACE}"
release_lock
- log "implement: done network=${NETWORK_ID} namespace=${NAMESPACE}"
+ log "implement-network: done network=${NETWORK_ID} namespace=${NAMESPACE}"
}
##############################################################################
-# Command: shutdown
+# Command: shutdown-network
# Flush iptables chains and remove this network's veth pairs.
# For VPC networks the shared namespace is preserved (other tiers still use
it).
# For isolated networks the namespace is also removed.
##############################################################################
-cmd_shutdown() {
+cmd_shutdown_network() {
parse_args "$@"
_load_state
acquire_lock "${NETWORK_ID}"
- log "shutdown: network=${NETWORK_ID} ns=${NAMESPACE} vpc=${VPC_ID}"
+ log "shutdown-network: network=${NETWORK_ID} ns=${NAMESPACE} vpc=${VPC_ID}"
local nchain_pr nchain_post fchain
nchain_pr="${CHAIN_PREFIX}_${NETWORK_ID}_PR"
@@ -752,7 +763,7 @@ cmd_shutdown() {
if [ -f "${tier_f}" ]; then
local owner_tier; owner_tier=$(cat "${tier_f}" 2>/dev/null ||
true)
if [ -n "${owner_tier}" ] && [ "${owner_tier}" !=
"${NETWORK_ID}" ]; then
- log "shutdown: skipping veth for $(basename "${f%.pvlan}")
(owned by tier ${owner_tier})"
+ log "shutdown-network: skipping veth for $(basename
"${f%.pvlan}") (owned by tier ${owner_tier})"
continue
fi
fi
@@ -760,7 +771,7 @@ cmd_shutdown() {
pvlan=$(cat "${f}")
pveth_h=$(pub_veth_host_name "${pvlan}" "${CHOSEN_ID}")
ip link del "${pveth_h}" 2>/dev/null || true
- log "shutdown: removed public veth ${pveth_h}"
+ log "shutdown-network: removed public veth ${pveth_h}"
done
fi
@@ -768,7 +779,7 @@ cmd_shutdown() {
local veth_h
veth_h=$(veth_host_name "${VLAN}" "${CHOSEN_ID}")
ip link del "${veth_h}" 2>/dev/null || true
- log "shutdown: removed guest veth ${veth_h}"
+ log "shutdown-network: removed guest veth ${veth_h}"
# Clean transient public IP state.
# For isolated networks the state dir is per-network, so wipe it entirely.
@@ -791,28 +802,28 @@ cmd_shutdown() {
# across tiers and must only be deleted when the last tier is destroyed.
if [ -z "${VPC_ID}" ]; then
ip netns del "${NAMESPACE}" 2>/dev/null || true
- log "shutdown: deleted namespace ${NAMESPACE}"
+ log "shutdown-network: deleted namespace ${NAMESPACE}"
else
- log "shutdown: preserved shared namespace ${NAMESPACE} (VPC tier)"
+ log "shutdown-network: preserved shared namespace ${NAMESPACE} (VPC
tier)"
fi
release_lock
- log "shutdown: done network=${NETWORK_ID}"
+ log "shutdown-network: done network=${NETWORK_ID}"
}
##############################################################################
-# Command: destroy
+# Command: destroy-network
# Delete this network's state entirely.
-# For VPC networks the shared namespace is only deleted when this is the last
-# remaining tier (tracked via vpc-<vpcId>/tiers/<networkId> marker files).
+# For VPC tier networks the shared namespace is preserved — the namespace is
+# removed only when shutdownVpc() calls shutdown-vpc or destroy-vpc.
##############################################################################
-cmd_destroy() {
+cmd_destroy_network() {
parse_args "$@"
_load_state
acquire_lock "${NETWORK_ID}"
- log "destroy: network=${NETWORK_ID} ns=${NAMESPACE} vpc=${VPC_ID}"
+ log "destroy-network: network=${NETWORK_ID} ns=${NAMESPACE} vpc=${VPC_ID}"
# Remove this tier's guest veth host-side
local veth_h
@@ -831,7 +842,7 @@ cmd_destroy() {
if [ -f "${tier_f}" ]; then
local owner_tier; owner_tier=$(cat "${tier_f}" 2>/dev/null ||
true)
if [ -n "${owner_tier}" ] && [ "${owner_tier}" !=
"${NETWORK_ID}" ]; then
- log "destroy: skipping veth for $(basename "${f%.pvlan}")
(owned by tier ${owner_tier})"
+ log "destroy-network: skipping veth for $(basename
"${f%.pvlan}") (owned by tier ${owner_tier})"
continue
fi
fi
@@ -855,29 +866,18 @@ cmd_destroy() {
# Deregister this tier from VPC tracking
if [ -n "${VPC_ID}" ]; then
rm -f "${vsd}/tiers/${NETWORK_ID}" 2>/dev/null || true
- # Only delete the VPC-wide namespace and state when no more tiers
remain
- local remaining_tiers
- remaining_tiers=$(find "${vsd}/tiers/" -type f 2>/dev/null | wc -l)
- if [ "${remaining_tiers}" -le 0 ]; then
- if ip netns list 2>/dev/null | grep -q "^${NAMESPACE}\b"; then
- ip netns del "${NAMESPACE}"
- log "destroy: deleted shared namespace ${NAMESPACE} (last VPC
tier)"
- fi
- rm -rf "${vsd}"
- log "destroy: removed VPC state dir ${vsd}"
- else
- log "destroy: preserved namespace ${NAMESPACE} (${remaining_tiers}
tier(s) remaining)"
- fi
+ # The VPC namespace is managed by shutdown-vpc / destroy-vpc — do NOT
delete it here.
+ log "destroy-network: deregistered tier ${NETWORK_ID} from VPC
${VPC_ID} (namespace preserved)"
else
# Isolated network: delete the namespace directly
if ip netns list 2>/dev/null | grep -q "^${NAMESPACE}\b"; then
ip netns del "${NAMESPACE}"
- log "destroy: deleted namespace ${NAMESPACE}"
+ log "destroy-network: deleted namespace ${NAMESPACE}"
fi
fi
release_lock
- log "destroy: done network=${NETWORK_ID}"
+ log "destroy-network: done network=${NETWORK_ID}"
}
##############################################################################
@@ -961,7 +961,9 @@ cmd_assign_ip() {
fi
# ---- Source NAT ----
- if [ "${SOURCE_NAT}" = "true" ] && [ -n "${CIDR}" ]; then
+ # For VPC tiers the SNAT rule covers the entire VPC CIDR and is set up by
+ # implement-vpc (using --vpc-cidr). Duplicate SNAT rules here would
conflict.
+ if [ "${SOURCE_NAT}" = "true" ] && [ -n "${CIDR}" ] && [ -z "${VPC_ID}" ];
then
ip netns exec "${NAMESPACE}" iptables -t nat \
-C "${nchain_post}" -s "${CIDR}" -o "${pveth_n}" -j SNAT
--to-source "${PUBLIC_IP}" 2>/dev/null || \
ip netns exec "${NAMESPACE}" iptables -t nat \
@@ -971,6 +973,8 @@ cmd_assign_ip() {
ip netns exec "${NAMESPACE}" iptables -t filter \
-A "${fchain}" -o "${pveth_n}" -s "${CIDR}" -j ACCEPT
log "Source NAT: ${CIDR} -> ${PUBLIC_IP} via ${pveth_n}"
+ elif [ "${SOURCE_NAT}" = "true" ] && [ -n "${VPC_ID}" ]; then
+ log "assign-ip: skipping SNAT rules for VPC tier (managed by
implement-vpc)"
fi
# ---- Persist state ----
@@ -2231,20 +2235,36 @@ cmd_save_vm_data() {
acquire_lock "${NETWORK_ID}"
log "save-vm-data: network=${NETWORK_ID} ip=${VM_IP}"
[ -z "${VM_IP}" ] && die "save-vm-data: missing --ip"
- [ -z "${VM_DATA}" ] && die "save-vm-data: missing --vm-data"
+
+ local vm_data_file="${VM_DATA_FILE}"
+ local cleanup_vm_data_file="false"
+ if [ -z "${vm_data_file}" ]; then
+ [ -z "${VM_DATA}" ] && die "save-vm-data: missing --vm-data or
--vm-data-file"
+ vm_data_file=$(mktemp /tmp/cs-extnet-vm-data-XXXXXX)
+ cleanup_vm_data_file="true"
+ printf '%s' "${VM_DATA}" > "${vm_data_file}"
+ fi
+ [ -f "${vm_data_file}" ] || die "save-vm-data: payload file not found:
${vm_data_file}"
local meta_dir; meta_dir=$(_metadata_dir)
local passwd_f; passwd_f=$(_passwd_file)
mkdir -p "${meta_dir}" "$(dirname "${passwd_f}")"
touch "${passwd_f}"
- python3 - "${VM_IP}" "${meta_dir}" "${passwd_f}" "${VM_DATA}" << 'PYEOF'
+ python3 - "${VM_IP}" "${meta_dir}" "${passwd_f}" "${vm_data_file}" <<
'PYEOF'
import base64, json, os, sys
vm_ip = sys.argv[1]
meta_dir = sys.argv[2]
passwd_f = sys.argv[3]
-data_b64 = sys.argv[4]
+data_file = sys.argv[4]
+
+try:
+ with open(data_file, 'r', encoding='utf-8') as f:
+ data_b64 = f.read().strip()
+except Exception as e:
+ print(f"save-vm-data: failed to read vm-data file: {e}", file=sys.stderr)
+ sys.exit(1)
# Decode the outer base64 wrapper, then parse the JSON array
try:
@@ -2321,6 +2341,10 @@ if password_written:
print(f"save-vm-data: wrote {len(entries)} entries for {vm_ip}")
PYEOF
+ if [ "${cleanup_vm_data_file}" = "true" ]; then
+ rm -f "${vm_data_file}" 2>/dev/null || true
+ fi
+
_write_apache2_conf
_svc_start_or_reload_apache2
_svc_start_or_reload_passwd_server
@@ -2389,6 +2413,15 @@ cmd_apply_fw_rules() {
acquire_lock "${NETWORK_ID}"
log "apply-fw-rules: network=${NETWORK_ID} ns=${NAMESPACE}"
+ local fw_rules_file="${FW_RULES_FILE}"
+ local cleanup_fw_rules_file="false"
+ if [ -z "${fw_rules_file}" ]; then
+ fw_rules_file=$(mktemp /tmp/cs-extnet-fw-rules-XXXXXX)
+ cleanup_fw_rules_file="true"
+ printf '%s' "${FW_RULES_JSON:-}" > "${fw_rules_file}"
+ fi
+ [ -f "${fw_rules_file}" ] || die "apply-fw-rules: payload file not found:
${fw_rules_file}"
+
local veth_n fchain fw_chain
veth_n=$(veth_ns_name "${VLAN}" "${CHOSEN_ID}")
fchain=$(filter_chain "${NETWORK_ID}")
@@ -2406,15 +2439,22 @@ cmd_apply_fw_rules() {
ip netns exec "${NAMESPACE}" iptables -t filter -N "${fw_chain}"
# ---- 4. Build iptables rules via Python ----
- python3 - "${NAMESPACE}" "${FW_RULES_JSON:-}" "${veth_n}" \
+ python3 - "${NAMESPACE}" "${fw_rules_file}" "${veth_n}" \
"${fw_chain}" << 'PYEOF'
import base64, json, re, subprocess, sys
namespace = sys.argv[1]
-rules_b64 = sys.argv[2]
+rules_file = sys.argv[2]
veth_n = sys.argv[3]
fw_chain = sys.argv[4] # filter table egress chain (CS_EXTNET_FWRULES_<N>)
+try:
+ with open(rules_file, 'r', encoding='utf-8') as f:
+ rules_b64 = f.read().strip()
+except Exception as e:
+ print(f"apply-fw-rules: failed to read rules file: {e}", file=sys.stderr)
+ sys.exit(1)
+
# Prefix for per-public-IP ingress chains in the mangle table.
# e.g. CS_EXTNET_FWI_10.0.56.20
FW_INGRESS_PREFIX = 'CS_EXTNET_FWI_'
@@ -2592,6 +2632,11 @@ print(f"apply-fw-rules: built {n_in} ingress rule(s)
across {len(pub_ip_rules)}
PYEOF
local py_exit=$?
+
+ if [ "${cleanup_fw_rules_file}" = "true" ]; then
+ rm -f "${fw_rules_file}" 2>/dev/null || true
+ fi
+
if [ ${py_exit} -ne 0 ]; then
# Python script failed — leave chain empty but continue so that the
# fchain catch-all rules remain effective (fail-open for existing
traffic).
@@ -2768,7 +2813,16 @@ cmd_restore_network() {
_load_state
acquire_lock "${NETWORK_ID}"
log "restore-network: network=${NETWORK_ID} ns=${NAMESPACE}"
- [ -z "${RESTORE_DATA}" ] && die "restore-network: missing --restore-data"
+
+ local restore_data_file="${RESTORE_DATA_FILE}"
+ local cleanup_restore_data_file="false"
+ if [ -z "${restore_data_file}" ]; then
+ [ -z "${RESTORE_DATA}" ] && die "restore-network: missing
--restore-data or --restore-data-file"
+ restore_data_file=$(mktemp /tmp/cs-extnet-restore-data-XXXXXX)
+ cleanup_restore_data_file="true"
+ printf '%s' "${RESTORE_DATA}" > "${restore_data_file}"
+ fi
+ [ -f "${restore_data_file}" ] || die "restore-network: payload file not
found: ${restore_data_file}"
local dhcp_hosts; dhcp_hosts=$(_dnsmasq_dhcp_hosts)
local dhcp_opts; dhcp_opts=$(_dnsmasq_dhcp_opts)
@@ -2780,19 +2834,26 @@ cmd_restore_network() {
touch "${dhcp_hosts}" "${dhcp_opts}" "${dns_hosts}" "${passwd_f}"
python3 - \
- "${RESTORE_DATA}" \
+ "${restore_data_file}" \
"${dhcp_hosts}" "${dhcp_opts}" "${dns_hosts}" \
"${meta_dir}" "${passwd_f}" \
<< 'PYEOF'
import base64, json, os, sys
-restore_b64 = sys.argv[1]
+restore_file = sys.argv[1]
dhcp_hosts_f = sys.argv[2]
dhcp_opts_f = sys.argv[3]
dns_hosts_f = sys.argv[4]
meta_dir = sys.argv[5]
passwd_f = sys.argv[6]
+try:
+ with open(restore_file, 'r', encoding='utf-8') as f:
+ restore_b64 = f.read().strip()
+except Exception as e:
+ print(f"restore-network: failed to read restore-data file: {e}",
file=sys.stderr)
+ sys.exit(1)
+
# ---- Decode the outer base64, then parse JSON ----
try:
data = json.loads(base64.b64decode(restore_b64).decode('utf-8'))
@@ -2938,16 +2999,16 @@ print("restore-network: done")
PYEOF
# ------------------------------------------------------------------
- # Decode dhcp_enabled / dns_enabled from RESTORE_DATA so we can
+ # Decode dhcp_enabled / dns_enabled from restore_data_file so we can
# reconfigure dnsmasq correctly even when the namespace (and therefore
- # dnsmasq.conf) was deleted and recreated. RESTORE_DATA is passed as
- # a positional argument to avoid shell-quoting issues with base64 data.
+ # dnsmasq.conf) was deleted and recreated.
# ------------------------------------------------------------------
local _r_flags
- _r_flags=$(python3 - "${RESTORE_DATA}" 2>/dev/null << 'PYFLAGSEOF'
-import base64, json, sys
+ _r_flags=$(python3 - "${restore_data_file}" 2>/dev/null << 'PYFLAGSEOF'
+import base64, json, sys, pathlib
try:
- d = json.loads(base64.b64decode(sys.argv[1]).decode('utf-8'))
+ b64_data = pathlib.Path(sys.argv[1]).read_text(encoding='utf-8').strip()
+ d = json.loads(base64.b64decode(b64_data).decode('utf-8'))
dhcp = 'true' if d.get('dhcp_enabled', False) else 'false'
dns = 'true' if d.get('dns_enabled', False) else 'false'
print(dhcp + ' ' + dns)
@@ -2955,6 +3016,10 @@ except Exception:
print('false false')
PYFLAGSEOF
)
+
+ if [ "${cleanup_restore_data_file}" = "true" ]; then
+ rm -f "${restore_data_file}" 2>/dev/null || true
+ fi
local _r_dhcp _r_dns
_r_dhcp="${_r_flags%% *}"; _r_dhcp="${_r_dhcp:-false}"
_r_dns="${_r_flags##* }"; _r_dns="${_r_dns:-false}"
@@ -2989,6 +3054,344 @@ PYFLAGSEOF
log "restore-network: done network=${NETWORK_ID}"
}
+##############################################################################
+# Helpers: parse VPC-level args (no --network-id required)
+##############################################################################
+
+parse_vpc_args() {
+ VPC_ID=""
+ NAMESPACE=""
+ VPC_CIDR=""
+ PUBLIC_IP=""
+ PUBLIC_VLAN=""
+ PUBLIC_GATEWAY=""
+ PUBLIC_CIDR=""
+ SOURCE_NAT="false"
+
+ while [ $# -gt 0 ]; do
+ case "$1" in
+ --vpc-id) VPC_ID="$2"; shift 2 ;;
+ --namespace) NAMESPACE="$2"; shift 2 ;;
+ --cidr) VPC_CIDR="$2"; shift 2 ;;
+ --public-ip) PUBLIC_IP="$2"; shift 2 ;;
+ --public-vlan) PUBLIC_VLAN="$2"; shift 2 ;;
+ --public-gateway) PUBLIC_GATEWAY="$2"; shift 2 ;;
+ --public-cidr) PUBLIC_CIDR="$2"; shift 2 ;;
+ --source-nat) SOURCE_NAT="$2"; shift 2 ;;
+ # consumed by _pre_scan_args — skip silently
+ --physical-network-extension-details|--network-extension-details)
+ shift 2 ;;
+ *) shift ;;
+ esac
+ done
+
+ [ -z "${VPC_ID}" ] && die "Missing --vpc-id"
+
+ if [ -z "${NAMESPACE}" ]; then
+ local NS_FROM_DETAILS
+ NS_FROM_DETAILS=$(_json_get "${EXTENSION_DETAILS}" "namespace")
+ NAMESPACE="${NS_FROM_DETAILS:-cs-vpc-${VPC_ID}}"
+ fi
+
+ # Normalise VLANs
+ if [ -n "${PUBLIC_VLAN}" ]; then
+ PUBLIC_VLAN=$(normalize_vlan "${PUBLIC_VLAN}")
+ fi
+}
+
+##############################################################################
+# Command: implement-vpc
+# Creates the VPC namespace, enables IP forwarding, and optionally sets up
+# VPC-level source NAT (--public-ip / --public-vlan / --source-nat true).
+# State is persisted to ${STATE_DIR}/vpc-<vpcId>/
+##############################################################################
+
+cmd_implement_vpc() {
+ parse_vpc_args "$@"
+ log "implement-vpc: vpc=${VPC_ID} ns=${NAMESPACE} cidr=${VPC_CIDR}"
+
+ # ---- 1. Create (or ensure) VPC namespace ----
+ if ! ip netns list 2>/dev/null | grep -q "^${NAMESPACE}\b"; then
+ ip netns add "${NAMESPACE}"
+ log "implement-vpc: created namespace ${NAMESPACE}"
+ fi
+ ip netns exec "${NAMESPACE}" ip link set lo up 2>/dev/null || true
+
+ # Disable IPv6 inside the namespace
+ ip netns exec "${NAMESPACE}" sysctl -w net.ipv6.conf.all.disable_ipv6=1
>/dev/null 2>&1 || true
+ ip netns exec "${NAMESPACE}" sysctl -w
net.ipv6.conf.default.disable_ipv6=1 >/dev/null 2>&1 || true
+ ip netns exec "${NAMESPACE}" sysctl -w net.ipv6.conf.lo.disable_ipv6=1
>/dev/null 2>&1 || true
+
+ # ---- 2. IP forwarding ----
+ ip netns exec "${NAMESPACE}" sysctl -w net.ipv4.ip_forward=1 >/dev/null
2>&1 || true
+
+ # ---- 3. VPC-level source NAT (if source NAT IP provided) ----
+ # Creates the public veth pair and SNAT rule for the entire VPC CIDR.
+ if [ -n "${PUBLIC_IP}" ] && [ -n "${PUBLIC_VLAN}" ] && [ "${SOURCE_NAT}" =
"true" ] && [ -n "${VPC_CIDR}" ]; then
+ local pveth_h pveth_n pub_br
+ pveth_h=$(pub_veth_host_name "${PUBLIC_VLAN}" "${VPC_ID}")
+ pveth_n=$(pub_veth_ns_name "${PUBLIC_VLAN}" "${VPC_ID}")
+ ensure_host_bridge "${PUB_ETH}" "${PUBLIC_VLAN}"
+ pub_br=$(host_bridge_name "${PUB_ETH}" "${PUBLIC_VLAN}")
+
+ if ! ip link show "${pveth_h}" >/dev/null 2>&1; then
+ ip link add "${pveth_h}" type veth peer name "${pveth_n}"
+ ip link set "${pveth_n}" netns "${NAMESPACE}"
+ ip link set "${pveth_h}" master "${pub_br}"
+ ip link set "${pveth_h}" up
+ ip netns exec "${NAMESPACE}" ip link set "${pveth_n}" up
+ log "implement-vpc: created public veth ${pveth_h} <-> ${pveth_n}"
+ else
+ ip link set "${pveth_h}" up 2>/dev/null || true
+ ip netns exec "${NAMESPACE}" ip link set "${pveth_n}" up
2>/dev/null || true
+ fi
+
+ # Assign public IP
+ local ADDR_SPEC
+ if [ -n "${PUBLIC_CIDR}" ] && echo "${PUBLIC_CIDR}" | grep -q '/'; then
+ local PREFIX
+ PREFIX=$(echo "${PUBLIC_CIDR}" | cut -d'/' -f2)
+ ADDR_SPEC="${PUBLIC_IP}/${PREFIX}"
+ else
+ ADDR_SPEC="${PUBLIC_IP}/32"
+ fi
+ ip netns exec "${NAMESPACE}" ip addr show "${pveth_n}" 2>/dev/null | \
+ grep -q "${PUBLIC_IP}/" || \
+ ip netns exec "${NAMESPACE}" ip addr add "${ADDR_SPEC}" dev
"${pveth_n}"
+
+ # Host route
+ ip route show | grep -q "^${PUBLIC_IP}" || \
+ ip route add "${PUBLIC_IP}/32" dev "${pveth_h}" 2>/dev/null || true
+
+ # Default route inside namespace toward upstream gateway
+ if [ -n "${PUBLIC_GATEWAY}" ]; then
+ ip netns exec "${NAMESPACE}" ip route replace default \
+ via "${PUBLIC_GATEWAY}" dev "${pveth_n}" 2>/dev/null || \
+ ip netns exec "${NAMESPACE}" ip route add default \
+ via "${PUBLIC_GATEWAY}" dev "${pveth_n}" 2>/dev/null || true
+ log "implement-vpc: default route via ${PUBLIC_GATEWAY} dev
${pveth_n}"
+ fi
+
+ # VPC SNAT rule — covers the entire VPC CIDR (all tiers)
+ # Use a VPC-level POSTROUTING chain: CS_EXTNET_<vpcId>_VPC_POST
+ local vpc_post_chain="${CHAIN_PREFIX}_${VPC_ID}_VPC_POST"
+ ensure_chain nat "${vpc_post_chain}"
+ ensure_jump nat POSTROUTING "${vpc_post_chain}"
+
+ ip netns exec "${NAMESPACE}" iptables -t nat \
+ -C "${vpc_post_chain}" -s "${VPC_CIDR}" -o "${pveth_n}" -j SNAT
--to-source "${PUBLIC_IP}" 2>/dev/null || \
+ ip netns exec "${NAMESPACE}" iptables -t nat \
+ -A "${vpc_post_chain}" -s "${VPC_CIDR}" -o "${pveth_n}" -j SNAT
--to-source "${PUBLIC_IP}"
+ log "implement-vpc: VPC SNAT ${VPC_CIDR} -> ${PUBLIC_IP} via
${pveth_n}"
+
+ # Persist public IP state
+ local vsd="${STATE_DIR}/vpc-${VPC_ID}"
+ mkdir -p "${vsd}/ips"
+ echo "true" > "${vsd}/ips/${PUBLIC_IP}"
+ echo "${PUBLIC_VLAN}" > "${vsd}/ips/${PUBLIC_IP}.pvlan"
+ fi
+
+ # ---- 4. Persist VPC state ----
+ local vsd="${STATE_DIR}/vpc-${VPC_ID}"
+ mkdir -p "${vsd}"
+ echo "${NAMESPACE}" > "${vsd}/namespace"
+ [ -n "${VPC_CIDR}" ] && echo "${VPC_CIDR}" > "${vsd}/cidr"
+
+ log "implement-vpc: done vpc=${VPC_ID} namespace=${NAMESPACE}"
+}
+
+##############################################################################
+# Command: shutdown-vpc
+# Removes the VPC namespace after all tiers have been shut down.
+# Called by shutdownVpc() in NetworkExtensionElement after all tiers are gone.
+##############################################################################
+
+cmd_shutdown_vpc() {
+ parse_vpc_args "$@"
+ log "shutdown-vpc: vpc=${VPC_ID} ns=${NAMESPACE}"
+
+ if ip netns list 2>/dev/null | grep -q "^${NAMESPACE}\b"; then
+ ip netns del "${NAMESPACE}"
+ log "shutdown-vpc: deleted namespace ${NAMESPACE}"
+ else
+ log "shutdown-vpc: namespace ${NAMESPACE} not found (already removed?)"
+ fi
+
+ log "shutdown-vpc: done vpc=${VPC_ID}"
+}
+
+##############################################################################
+# Command: destroy-vpc
+# Destroys the VPC namespace and removes all VPC state.
+##############################################################################
+
+cmd_destroy_vpc() {
+ parse_vpc_args "$@"
+ log "destroy-vpc: vpc=${VPC_ID} ns=${NAMESPACE}"
+
+ if ip netns list 2>/dev/null | grep -q "^${NAMESPACE}\b"; then
+ ip netns del "${NAMESPACE}"
+ log "destroy-vpc: deleted namespace ${NAMESPACE}"
+ fi
+
+ local vsd="${STATE_DIR}/vpc-${VPC_ID}"
+ rm -rf "${vsd}"
+ log "destroy-vpc: removed VPC state dir ${vsd}"
+
+ log "destroy-vpc: done vpc=${VPC_ID}"
+}
+
+##############################################################################
+# Command: apply-network-acl
+# Applies VPC network ACL rules to the FORWARD chain inside the namespace.
+# Rules are passed as a Base64-encoded JSON array via --acl-rules-file or
+# --acl-rules. Each rule has:
+# number, action (allow|deny), trafficType (ingress|egress),
+# protocol, portStart, portEnd, icmpType, icmpCode, sourceCidrs[]
+##############################################################################
+
+cmd_apply_network_acl() {
+ parse_args "$@"
+ _load_state
+ acquire_lock "${NETWORK_ID}"
+ log "apply-network-acl: network=${NETWORK_ID} ns=${NAMESPACE} cidr=${CIDR}"
+
+ local acl_rules_file="${ACL_RULES_FILE}"
+ local cleanup_acl_file="false"
+ if [ -z "${acl_rules_file}" ]; then
+ acl_rules_file=$(mktemp /tmp/cs-extnet-acl-rules-XXXXXX)
+ cleanup_acl_file="true"
+ printf '%s' "${ACL_RULES_JSON:-}" > "${acl_rules_file}"
+ fi
+ [ -f "${acl_rules_file}" ] || die "apply-network-acl: payload file not
found: ${acl_rules_file}"
+
+ local veth_n acl_chain_name fchain
+ veth_n=$(veth_ns_name "${VLAN}" "${CHOSEN_ID}")
+ acl_chain_name=$(acl_chain "${NETWORK_ID}")
+ fchain=$(filter_chain "${NETWORK_ID}")
+
+ # ---- 1. Remove existing jump from fchain to acl chain (idempotent) ----
+ ip netns exec "${NAMESPACE}" iptables -t filter \
+ -D "${fchain}" -j "${acl_chain_name}" 2>/dev/null || true
+
+ # ---- 2. Flush and delete old ACL chain ----
+ ip netns exec "${NAMESPACE}" iptables -t filter -F "${acl_chain_name}"
2>/dev/null || true
+ ip netns exec "${NAMESPACE}" iptables -t filter -X "${acl_chain_name}"
2>/dev/null || true
+
+ # ---- 3. Create fresh ACL chain ----
+ ip netns exec "${NAMESPACE}" iptables -t filter -N "${acl_chain_name}"
+
+ # ---- 4. Build iptables ACL rules via Python ----
+ python3 - "${NAMESPACE}" "${acl_rules_file}" "${veth_n}" \
+ "${acl_chain_name}" "${CIDR:-}" << 'PYEOF'
+import base64, json, subprocess, sys
+
+namespace = sys.argv[1]
+rules_file = sys.argv[2]
+veth_n = sys.argv[3]
+acl_chain = sys.argv[4]
+net_cidr = sys.argv[5] # tier CIDR (may be empty)
+
+try:
+ with open(rules_file, 'r', encoding='utf-8') as f:
+ rules_b64 = f.read().strip()
+except Exception as e:
+ print(f"apply-network-acl: failed to read rules file: {e}",
file=sys.stderr)
+ sys.exit(1)
+
+def _run(table, *args):
+ cmd = ['ip', 'netns', 'exec', namespace, 'iptables', '-t', table] +
list(args)
+ r = subprocess.run(cmd, capture_output=True)
+ if r.returncode != 0:
+ print(f"iptables ({table}): {r.stderr.decode().strip()}",
file=sys.stderr)
+ return r
+
+def iptf(*args):
+ _run('filter', *args)
+
+if rules_b64:
+ try:
+ rules = json.loads(base64.b64decode(rules_b64).decode('utf-8'))
+ except Exception as e:
+ print(f"apply-network-acl: failed to decode rules: {e}",
file=sys.stderr)
+ sys.exit(1)
+else:
+ rules = []
+
+# Always allow RELATED,ESTABLISHED so active sessions are not dropped.
+iptf('-A', acl_chain, '-m', 'state', '--state', 'RELATED,ESTABLISHED', '-j',
'ACCEPT')
+
+for rule in sorted(rules, key=lambda r: r.get('number', 999)):
+ direction = rule.get('trafficType', 'ingress').lower()
+ protocol = (rule.get('protocol') or 'all').lower()
+ port_start = rule.get('portStart')
+ port_end = rule.get('portEnd')
+ icmp_type = rule.get('icmpType')
+ icmp_code = rule.get('icmpCode')
+ src_cidrs = rule.get('sourceCidrs') or ['0.0.0.0/0']
+ action = 'ACCEPT' if rule.get('action', 'deny').lower() == 'allow'
else 'DROP'
+
+ for src_cidr in src_cidrs:
+ a = []
+ if direction == 'ingress':
+ # Traffic toward VMs (going OUT on guest veth_n into the tier
subnet)
+ a = ['-o', veth_n]
+ if net_cidr:
+ a += ['-d', net_cidr]
+ if src_cidr and src_cidr not in ('0.0.0.0/0', '::/0', ''):
+ a += ['-s', src_cidr]
+ else:
+ # Traffic FROM VMs (coming IN on guest veth_n from the tier subnet)
+ a = ['-i', veth_n]
+ if net_cidr:
+ a += ['-s', net_cidr]
+ # For egress rules sourceCidrs is used as destination filter
+ if src_cidr and src_cidr not in ('0.0.0.0/0', '::/0', ''):
+ a += ['-d', src_cidr]
+
+ if protocol not in ('all', ''):
+ a += ['-p', protocol]
+ if protocol in ('tcp', 'udp') and port_start is not None:
+ port_spec = str(port_start)
+ if port_end is not None and port_end != port_start:
+ port_spec = f"{port_start}:{port_end}"
+ a += ['--dport', port_spec]
+ elif protocol == 'icmp' and icmp_type is not None and icmp_type !=
-1:
+ icmp_spec = str(icmp_type)
+ if icmp_code is not None and icmp_code != -1:
+ icmp_spec += f"/{icmp_code}"
+ a += ['--icmp-type', icmp_spec]
+
+ iptf('-A', acl_chain, *a, '-j', action)
+
+# Default: DROP all unmatched traffic (implicit deny at end of ACL)
+iptf('-A', acl_chain, '-j', 'DROP')
+
+print(f"apply-network-acl: applied {len(rules)} ACL rule(s) to chain
{acl_chain}")
+PYEOF
+
+ local py_exit=$?
+
+ if [ "${cleanup_acl_file}" = "true" ]; then
+ rm -f "${acl_rules_file}" 2>/dev/null || true
+ fi
+
+ if [ ${py_exit} -ne 0 ]; then
+ log "apply-network-acl: Python rule builder exited ${py_exit}; ACL
chain may be incomplete"
+ fi
+
+ # ---- 5. Insert jump from fchain to acl chain at position 1 ----
+ # ACL rules take precedence over the catch-all ACCEPT rules in fchain.
+ if ip netns exec "${NAMESPACE}" iptables -t filter -n -L "${fchain}"
>/dev/null 2>&1; then
+ ip netns exec "${NAMESPACE}" iptables -t filter \
+ -I "${fchain}" 1 -j "${acl_chain_name}" 2>/dev/null || true
+ log "apply-network-acl: inserted ACL jump in ${fchain}"
+ fi
+
+ release_lock
+ log "apply-network-acl: done network=${NETWORK_ID}"
+}
+
##############################################################################
# Main dispatcher
##############################################################################
@@ -2999,9 +3402,13 @@ COMMAND="${1:-}"
shift || true
case "${COMMAND}" in
- implement) cmd_implement "$@" ;;
- shutdown) cmd_shutdown "$@" ;;
- destroy) cmd_destroy "$@" ;;
+ implement-network) cmd_implement_network "$@" ;;
+ shutdown-network) cmd_shutdown_network "$@" ;;
+ destroy-network) cmd_destroy_network "$@" ;;
+ # VPC lifecycle
+ implement-vpc) cmd_implement_vpc "$@" ;;
+ shutdown-vpc) cmd_shutdown_vpc "$@" ;;
+ destroy-vpc) cmd_destroy_vpc "$@" ;;
assign-ip) cmd_assign_ip "$@" ;;
release-ip) cmd_release_ip "$@" ;;
add-static-nat) cmd_add_static_nat "$@" ;;
@@ -3028,15 +3435,19 @@ case "${COMMAND}" in
# Load balancing (haproxy)
apply-fw-rules) cmd_apply_fw_rules "$@" ;;
apply-lb-rules) cmd_apply_lb_rules "$@" ;;
+ # ACL rules (VPC network ACLs)
+ apply-network-acl) cmd_apply_network_acl "$@" ;;
# Custom actions
custom-action) cmd_custom_action "$@" ;;
"")
- echo "Usage: $0 {implement|shutdown|destroy|assign-ip|release-ip|" \
+ echo "Usage: $0 {implement-network|shutdown-network|destroy-network|" \
+ "implement-vpc|shutdown-vpc|destroy-vpc|" \
+ "assign-ip|release-ip|" \
"add-static-nat|delete-static-nat|add-port-forward|delete-port-forward|" \
"config-dhcp-subnet|remove-dhcp-subnet|add-dhcp-entry|remove-dhcp-entry|set-dhcp-options|"
\
"config-dns-subnet|remove-dns-subnet|add-dns-entry|remove-dns-entry|" \
"save-userdata|save-password|save-sshkey|save-hypervisor-hostname|save-vm-data|restore-network|"
\
- "apply-fw-rules|apply-lb-rules|custom-action} [options]" >&2
+ "apply-fw-rules|apply-lb-rules|apply-network-acl|custom-action}
[options]" >&2
exit 1 ;;
*)
echo "Unknown command: ${COMMAND}" >&2
diff --git a/Network-Namespace/network-namespace.sh
b/Network-Namespace/network-namespace.sh
index 4155f01..090b46f 100755
--- a/Network-Namespace/network-namespace.sh
+++ b/Network-Namespace/network-namespace.sh
@@ -152,6 +152,10 @@ EXTENSION_DETAILS="{}"
NETWORK_ID=""
CURRENT_DETAILS="{}"
VPC_ID=""
+VM_DATA_FILE=""
+FW_RULES_FILE=""
+RESTORE_DATA_FILE=""
+ACL_RULES_FILE=""
FORWARD_ARGS=()
while [ $# -gt 0 ]; do
@@ -173,6 +177,18 @@ while [ $# -gt 0 ]; do
--current-details)
CURRENT_DETAILS="${2:-{}}"
shift 2 ;;
+ --vm-data-file)
+ VM_DATA_FILE="${2:-}"
+ shift 2 ;;
+ --fw-rules-file)
+ FW_RULES_FILE="${2:-}"
+ shift 2 ;;
+ --acl-rules-file)
+ ACL_RULES_FILE="${2:-}"
+ shift 2 ;;
+ --restore-data-file)
+ RESTORE_DATA_FILE="${2:-}"
+ shift 2 ;;
*)
FORWARD_ARGS+=("$1")
shift ;;
@@ -280,12 +296,28 @@ ssh_exec() {
fi
}
+upload_file_to_remote() {
+ local host="$1" local_file="$2" tag="$3"
+ [ -f "${local_file}" ] || die "Missing local payload file: ${local_file}" 1
+
+ local remote_tmp
+ remote_tmp=$(ssh_exec "${host}" "mktemp /tmp/cs-extnet-${tag}-XXXXXX") || \
+ die "Failed to create remote temp file for ${tag}" 2
+ remote_tmp=$(printf '%s' "${remote_tmp}" | tr -d '\r\n')
+ [ -n "${remote_tmp}" ] || die "Failed to resolve remote temp file for
${tag}" 2
+
+ cat "${local_file}" | ssh_exec "${host}" "cat > '${remote_tmp}' && chmod
600 '${remote_tmp}'" || \
+ die "Failed to upload payload file for ${tag}" 2
+
+ printf '%s' "${remote_tmp}"
+}
+
# ---------------------------------------------------------------------------
# ensure-network-device
# ---------------------------------------------------------------------------
if [ "${COMMAND}" = "ensure-network-device" ]; then
- [ -z "${NETWORK_ID}" ] && die "ensure-network-device: missing
--network-id" 1
+ [ -z "${NETWORK_ID}" ] && [ -z "${VPC_ID}" ] && die
"ensure-network-device: missing --network-id or --vpc-id" 1
if [ ${#HOST_LIST[@]} -eq 0 ]; then
die "ensure-network-device: no hosts configured. Set 'hosts' in
registerExtension details." 1
@@ -311,7 +343,7 @@ if [ "${COMMAND}" = "ensure-network-device" ]; then
h="${h// /}"
if [ "${h}" = "${CURRENT_HOST}" ]; then
if host_reachable "${CURRENT_HOST}"; then
- log "ensure-network-device: network=${NETWORK_ID} keeping
current host=${CURRENT_HOST}"
+ log "ensure-network-device:
${NETWORK_ID:+network=${NETWORK_ID} }${VPC_ID:+vpc=${VPC_ID} }keeping current
host=${CURRENT_HOST}"
if [ -n "${VPC_ID}" ]; then
printf
'{"host":"%s","namespace":"%s","vpc_id":"%s"}\n' \
"${CURRENT_HOST}" "${NAMESPACE}" "${VPC_ID}"
@@ -350,7 +382,7 @@ if [ "${COMMAND}" = "ensure-network-device" ]; then
_H="${HOST_LIST[$_IDX]// /}"
if host_reachable "${_H}"; then
_SELECTED_HOST="${_H}"
- log "ensure-network-device: network=${NETWORK_ID} hash-selected
host=${_SELECTED_HOST} (key=${_ROUTE_KEY}, idx=${_IDX})"
+ log "ensure-network-device: ${NETWORK_ID:+network=${NETWORK_ID}
}${VPC_ID:+vpc=${VPC_ID} }hash-selected host=${_SELECTED_HOST}
(key=${_ROUTE_KEY}, idx=${_IDX})"
break
fi
log "ensure-network-device: host ${_H} not reachable, trying next"
@@ -386,6 +418,28 @@ for arg in "${FORWARD_ARGS[@]}"; do
remote_args+=("'${arg//"'"/"'\\''"}'" )
done
+REMOTE_PAYLOAD_FILES=()
+if [ -n "${VM_DATA_FILE}" ]; then
+ REMOTE_VM_DATA_FILE=$(upload_file_to_remote "${REMOTE_HOST}"
"${VM_DATA_FILE}" "vm-data")
+ REMOTE_PAYLOAD_FILES+=("${REMOTE_VM_DATA_FILE}")
+ remote_args+=("'--vm-data-file'" "'${REMOTE_VM_DATA_FILE//"'"/"'\\''"}'")
+fi
+if [ -n "${FW_RULES_FILE}" ]; then
+ REMOTE_FW_RULES_FILE=$(upload_file_to_remote "${REMOTE_HOST}"
"${FW_RULES_FILE}" "fw-rules")
+ REMOTE_PAYLOAD_FILES+=("${REMOTE_FW_RULES_FILE}")
+ remote_args+=("'--fw-rules-file'" "'${REMOTE_FW_RULES_FILE//"'"/"'\\''"}'")
+fi
+if [ -n "${ACL_RULES_FILE}" ]; then
+ REMOTE_ACL_RULES_FILE=$(upload_file_to_remote "${REMOTE_HOST}"
"${ACL_RULES_FILE}" "acl-rules")
+ REMOTE_PAYLOAD_FILES+=("${REMOTE_ACL_RULES_FILE}")
+ remote_args+=("'--acl-rules-file'"
"'${REMOTE_ACL_RULES_FILE//"'"/"'\\''"}'")
+fi
+if [ -n "${RESTORE_DATA_FILE}" ]; then
+ REMOTE_RESTORE_DATA_FILE=$(upload_file_to_remote "${REMOTE_HOST}"
"${RESTORE_DATA_FILE}" "restore-data")
+ REMOTE_PAYLOAD_FILES+=("${REMOTE_RESTORE_DATA_FILE}")
+ remote_args+=("'--restore-data-file'"
"'${REMOTE_RESTORE_DATA_FILE//"'"/"'\\''"}'")
+fi
+
PHYS_ESCAPED="${PHYS_DETAILS//\'/\'\\\'\'}"
EXT_ESCAPED="${EXTENSION_DETAILS//\'/\'\\\'\'}"
REMOTE_CMD="'${REMOTE_SCRIPT}' '${COMMAND}' ${remote_args[*]}
--physical-network-extension-details '${PHYS_ESCAPED}'
--network-extension-details '${EXT_ESCAPED}'"
@@ -395,6 +449,12 @@ log "Remote: ${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_PORT}
cmd=${COMMAND}"
RC=0
ssh_exec "${REMOTE_HOST}" "${REMOTE_CMD}" || RC=$?
+if [ ${#REMOTE_PAYLOAD_FILES[@]} -gt 0 ]; then
+ for _rf in "${REMOTE_PAYLOAD_FILES[@]}"; do
+ ssh_exec "${REMOTE_HOST}" "rm -f '${_rf}'" >/dev/null 2>&1 || true
+ done
+fi
+
if [ ${RC} -ne 0 ]; then
if [ ${RC} -eq 255 ]; then
log "SSH connection failed (rc=255):
host=${REMOTE_HOST}:${REMOTE_PORT} user=${REMOTE_USER}"
