With dnsmasq as the dns server, I can use tap_ip as the nameserver. The
boottime back to normal. I create two scripts to setup the network.
setup_iptable.sh sets the ip table. setup_dnsmasq configures dnsmasq and
start it. After running these two scripts, I can run firecracker.py with
only -n. I'm not familiar with sending patch through email with git. I
include the patch below.
>From bbc12eb31878bdacaf20313be444ac71f9ff6578 Mon Sep 17 00:00:00 2001
From: Zhiting Zhu <zhiti...@cs.utexas.edu>
Date: Wed, 9 Oct 2019 00:32:35 -0500
Subject: [PATCH] scripts to setup dnsmasq and NAT
Signed-off-by: Zhiting Zhu <zhiti...@cs.utexas.edu>
---
scripts/delete_dnsmasq_setting.sh | 6 ++++++
scripts/restore_iptable.sh | 5 +++++
scripts/setup_dnsmasq.sh | 20 ++++++++++++++++++++
scripts/setup_iptable.sh | 8 ++++++++
4 files changed, 39 insertions(+)
create mode 100755 scripts/delete_dnsmasq_setting.sh
create mode 100755 scripts/restore_iptable.sh
create mode 100755 scripts/setup_dnsmasq.sh
create mode 100755 scripts/setup_iptable.sh
diff --git a/scripts/delete_dnsmasq_setting.sh
b/scripts/delete_dnsmasq_setting.sh
new file mode 100755
index 00000000..aa9c3c15
--- /dev/null
+++ b/scripts/delete_dnsmasq_setting.sh
@@ -0,0 +1,6 @@
+if [ "$#" -ne 1 ]; then
+ echo "Need to specify interface name"
+fi
+DEV=$1
+sudo rm -rf /var/lib/dnsmasq/$DEV
+sudo rm -rf /etc/dnsmasq.d/$DEV.conf
diff --git a/scripts/restore_iptable.sh b/scripts/restore_iptable.sh
new file mode 100755
index 00000000..e882948a
--- /dev/null
+++ b/scripts/restore_iptable.sh
@@ -0,0 +1,5 @@
+if [ -f iptables.rules.old ]; then
+ sudo iptables-restore < iptables.rules.old
+ rm iptables.rules.old
+fi
+sudo sh -c "echo 0 > /proc/sys/net/ipv4/ip_forward"
diff --git a/scripts/setup_dnsmasq.sh b/scripts/setup_dnsmasq.sh
new file mode 100755
index 00000000..b93c88df
--- /dev/null
+++ b/scripts/setup_dnsmasq.sh
@@ -0,0 +1,20 @@
+if [ "$#" -ne 1 ]; then
+ echo "need to specify the interface name"
+fi
+DEV=$1
+sudo mkdir -p /var/lib/dnsmasq/$DEV
+sudo touch /var/lib/dnsmasq/$DEV/hostsfile
+sudo touch /var/lib/dnsmasq/$DEV/leases
+sudo touch /var/lib/dnsmasq/$DEV/dnsmasq.conf
+sudo sh -c "cat << 'EOF' >/var/lib/dnsmasq/$DEV/dnsmasq.conf
+except-interface=lo
+interface=$DEV
+bind-dynamic
+strict-order
+EOF"
+sudo mkdir -p /etc/dnsmasq.d/
+sudo touch /etc/dnsmasq.d/$DEV.conf
+sudo bash -c "echo "except-interface=$DEV" >> /etc/dnsmasq.d/$DEV.conf"
+sudo bash -c "echo "bind-interfaces" >> /etc/dnsmasq.d/$DEV.conf"
+sudo mkdir -p /var/run/dnsmasq/
+sudo dnsmasq --conf-file=/var/lib/dnsmasq/$DEV/dnsmasq.conf
--pid-file=/var/run/dnsmasq/$DEV.pid
diff --git a/scripts/setup_iptable.sh b/scripts/setup_iptable.sh
new file mode 100755
index 00000000..ce638b7b
--- /dev/null
+++ b/scripts/setup_iptable.sh
@@ -0,0 +1,8 @@
+INTERFACE=enp7s3
+sudo iptables-save > iptables.rules.old
+sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
+sudo iptables -t nat -A POSTROUTING -o $INTERFACE -j MASQUERADE
+sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j
ACCEPT
+sudo iptables -A FORWARD -i fc_tap0 -o $INTERFACE -j ACCEPT
+sudo iptables -A INPUT -i fc_tap0 -p udp -m udp -m multiport --dports 53
-j ACCEPT
+sudo iptables -A INPUT -i fc_tap0 -p tcp -m tcp -m multiport --dports 53
-j ACCEPT
--
2.17.1
On Tue, Oct 8, 2019 at 5:19 PM Waldek Kozaczuk <jwkozac...@gmail.com> wrote:
> Quick update - using Google public DNS IP - made DNS working for me (on
> top of NAT setup described by firecracker doc:
>
> sudo iptables -t nat -A POSTROUTING -o ens9 -j MASQUERADE
> sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j
> ACCEPT
> sudo iptables -A FORWARD -i fc_tap0 -o ens9 -j ACCEPT
>
>
> diff --git a/scripts/firecracker.py b/scripts/firecracker.py
> index 42a5e8ee..bf1aed10 100755
> --- a/scripts/firecracker.py
> +++ b/scripts/firecracker.py
> @@ -229,7 +229,7 @@ def main(options):
> setup_tap_interface('fc_tap0', tap_ip, options.bridge)
> if not options.bridge:
> client_ip = '172.16.0.2'
> - cmdline = '--ip=eth0,%s,255.255.255.252 --defaultgw=%s %s' %
> (client_ip, tap_ip, cmdline)
> + cmdline = '--ip=eth0,%s,255.255.255.252 --defaultgw=%s
> --nameserver=%s %s' % (client_ip, tap_ip, "8.8.8.8", cmdline)
>
> if options.verbose:
> cmdline = '--verbose ' + cmdline
>
> I think using public google DNS is not ideal and ideally we should figure
> out why DNS does not work if nameserver is equal to tap_ip.
>
> Waldek
>
> On Tuesday, October 8, 2019 at 5:59:34 PM UTC-4, Waldek Kozaczuk wrote:
>>
>> Hi,
>>
>> I have realized that our firecracker script misses passing '--nameserver'
>> argument that is used when DHCP is off.
>>
>> diff --git a/scripts/firecracker.py b/scripts/firecracker.py
>> index 42a5e8ee..65345efc 100755
>> --- a/scripts/firecracker.py
>> +++ b/scripts/firecracker.py
>> @@ -229,7 +229,7 @@ def main(options):
>> setup_tap_interface('fc_tap0', tap_ip, options.bridge)
>> if not options.bridge:
>> client_ip = '172.16.0.2'
>> - cmdline = '--ip=eth0,%s,255.255.255.252 --defaultgw=%s %s' %
>> (client_ip, tap_ip, cmdline)
>> + cmdline = '--ip=eth0,%s,255.255.255.252 --defaultgw=%s
>> --nameserver=%s %s' % (client_ip, tap_ip, tap_ip, cmdline)
>>
>> if options.verbose:
>> cmdline = '--verbose ' + cmdline
>>
>> I have also noticed that even though it sets up the tap device, it does
>> not set up routing outside of VM. For now, I have manually followed the NAT
>> setup examples (the iptables fragment) from firecracker readme -
>> https://github.com/firecracker-microvm/firecracker/blob/master/docs/network-setup.md.
>> In my case, after enabling NAT I was able to make outgoing traffic using an
>> IP. However, I was not able to make DNS working so trying to fetch files
>> with URLs with DNS names would fail. I think DNS relies on UDP and so maybe
>> NAT setup is missing something.
>>
>> I looked at the document you pointed to which does mention DNS setup so
>> maybe it will all work for you.
>>
>> Let us know if it works so I can update the script accordingly. If you
>> feel like sending a patch and updating wiki -
>> https://github.com/cloudius-systems/osv/wiki/Running-OSv-on-Firecracker -
>> I would appreciate it.
>>
>> Waldek
>>
>> On Tuesday, October 8, 2019 at 2:26:17 PM UTC-4, zhiting zhu wrote:
>>>
>>> Hey,
>>>
>>> With the -n option on firecracker.py, client on the same machine can
>>> send packet to the VM. But the VM cannot access Internet. I setup NAT based
>>> network following:
>>> https://jamielinux.com/docs/libvirt-networking-handbook/custom-nat-based-network.html
>>> I can get Internet access but the boot time is very large(9.2 second for
>>> me).
>>>
>>> Here is the booting log:
>>>
>>> bsd: initializing - done
>>> VFS: mounting ramfs at /
>>> VFS: mounting devfs at /dev
>>> net: initializing - done
>>> Detected virtio-mmio device: (2,0)
>>> Detected virtio-mmio device: (1,0)
>>> queue 0 size 256
>>> queue 1 size 256
>>> queue 2 size 0
>>> eth0: ethernet address: 52:54:00:12:34:56
>>> queue 0 size 256
>>> queue 1 size 0
>>> virtio-blk: Add blk device instances 0 as vblk0, devsize=536870912
>>> Warning: No hardware source of entropy available to your platform,
>>> CSPRNG will rely on software source of entropy to provide high-quality
>>> randomness.
>>> random: <Software, Yarrow> initialized
>>> VFS: unmounting /dev
>>> VFS: mounting rofs at /rofs
>>> failed to mount /rofs, error = No error information
>>> VFS: mounting zfs at /zfs
>>> zfs: mounting osv/zfs from device /dev/vblk0.1
>>> VFS: mounting devfs at /dev
>>> VFS: mounting procfs at /proc
>>> VFS: mounting sysfs at /sys
>>> random: device unblocked.
>>> program zpool.so returned 1
>>> BSD shrinker: event handler list found: 0xffffa00000989080
>>> BSD shrinker found: 1
>>> BSD shrinker: unlocked, running
>>> [I/26 dhcp]: Broadcasting DHCPDISCOVER message with xid: [1850312527]
>>> [I/26 dhcp]: Waiting for IP...
>>> [I/26 dhcp]: Broadcasting DHCPDISCOVER message with xid: [1771944366]
>>> [I/26 dhcp]: Waiting for IP...
>>> [I/26 dhcp]: Broadcasting DHCPDISCOVER message with xid: [1285738661]
>>> [I/26 dhcp]: Waiting for IP...
>>> [I/26 dhcp]: Broadcasting DHCPDISCOVER message with xid: [1533719522]
>>> [I/26 dhcp]: Waiting for IP...
>>> [I/195 dhcp]: Received DHCPOFFER message from DHCP server: 192.168.100.1
>>> regarding offerred IP address: 192.168.100.173
>>> [I/195 dhcp]: Broadcasting DHCPREQUEST message with xid: [1533719522] to
>>> SELECT offered IP: 192.168.100.173
>>> [I/195 dhcp]: Received DHCPOFFER message from DHCP server: 192.168.100.1
>>> regarding offerred IP address: 192.168.100.173
>>> [W/195 dhcp]: Got packet with wrong transaction ID (1533719522, 1285738661)
>>> [I/195 dhcp]: DHCP received hostname: osv
>>>
>>> [I/195 dhcp]: Received DHCPACK message from DHCP server: 192.168.100.1
>>> regarding offerred IP address: 192.168.100.173
>>> [I/195 dhcp]: Server acknowledged IP 192.168.100.173 for interface eth0
>>> with time to lease in seconds: 3600
>>> eth0: 192.168.100.173
>>> [I/195 dhcp]: Configuring eth0: ip 192.168.100.173 subnet mask
>>> 255.255.255.0 gateway 192.168.100.1 MTU 1500
>>> [I/195 dhcp]: Set hostname to: osv
>>> Booted up in 9237.45 ms
>>>
>>> Is there any other recommended way to setup VM with internet?
>>>
>>> Best,
>>> Zhiting
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/7b0585d6-e7f2-49dc-afe6-c9a43d10b90f%40googlegroups.com
> <https://groups.google.com/d/msgid/osv-dev/7b0585d6-e7f2-49dc-afe6-c9a43d10b90f%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/CA%2B3q14xPOa0hK8y_KdbRUK-sggp%3DAAVuUb7JM0MteG0Ez8r75w%40mail.gmail.com.
