On Thursday, 27 October 2016 23:48:28 UTC+8, Max wrote: > On Monday, 24 October 2016 08:30:28 UTC+8, Unman wrote: > > On Sun, Oct 23, 2016 at 02:11:48AM -0700, Max wrote: > > > Hi, > > > > > > I am a new user of Qubes OS so apologies in advance if the question here > > > has been answered already in a separate topic (there are similar issues) > > > and I haven’t discovered this or it is not one suited to this mailing > > > list. I am running Qubes 3.2 and attempting to ping from one VM to > > > another VM, specifically from a Standalone Windows 7 VM to a Qubes VM > > > based on the Debian 8 template. > > > > > > All my VM’s were initially connected in the default manner i.e. to a > > > sys-firewall and through to the sys-net VM, both of which are Fedora 23. > > > There are no firewall rules on these VMs restricting which IP addresses > > > can be accessed. > > > > > > Current status: > > > - I am able to ping from my Windows 7 VM (10.137.2.19) to the Firewall VM > > > (10.137.1.8) using the IP address visible in the VM Manager > > > > > > - I am unable to ping the Debian 8 VM (10.137.2.18) from my Windows VM. > > > > > > Steps taken: > > > 1) I followed the instructions here > > > (https://www.qubes-os.org/doc/qubes-firewall/#enabling-networking-between-two-vms) > > > and in the firewall VM’s terminal enter the following iptables rule... > > > > > > sudo iptables -I FORWARD 2 -s <IP address of Windows 7 VM> -d <IP address > > > of Debian 8 VM> -j ACCEPT > > > > > > … In VM B’s terminal (Debian 8) I entered the following iptables rule... > > > > > > sudo iptables -I INPUT -s <IP address of Windows 7 VM> -j ACCEPT > > > > > > ...but from here when using the ping function to my Debian 8 VM in the > > > cmd prompt in Windows, all packets were lost. > > > > > > 2) As this was not successful I attempted to see if I could connect to > > > VMs from an external machine and followed the instructions here > > > https://www.qubes-os.org/doc/qubes-firewall/#port-forwarding-to-a-vm-from-the-outside-world. > > > > > > The Eth0 IP address (192.168.1.6) appeared to be what I should expose the > > > service to. > > > > > > I put the below rule in the sys-net VM’s Terminal... > > > > > > iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -d 192.168.x.x > > > -j DNAT --to-destination 10.137.1.x > > > > > > ...and this rule into the sys-firewall VM’s Terminal > > > > > > iptables -I FORWARD 2 -i eth0 -d 10.137.1.x -p tcp --dport 443 -m > > > conntrack --ctstate NEW -j ACCEPT > > > > > > But using ping or Telnet resulted in lost packets and failed to increase > > > the counters when using the iptables -t nat -L -v -n command in the > > > sys-firewall VM's terminal. > > > > > > 3) With this not being successful either I attempted to add a “sys-proxy” > > > VM as described here > > > https://groups.google.com/forum/#!searchin/qubes-users/intervm%7Csort:relevance/qubes-users/lA2SgPcV9fU/U969uapYAAAJ > > > and entered the following in the new sys-proxy VM's terminal: > > > > > > iptables -I FORWARD 1 -i vif+ -o vif+ -s $intervm_internalnet/24 -d > > > $intervm_internalnet/24 -m state --state NEW -p tcp -m tcp -j ACCEPT > > > > > > iptables -I FORWARD 1 -i vif+ -o vif+ -s $intervm_internalnet/24 -d > > > $intervm_internalnet/24 -p udp -m udp -j ACCEPT > > > > > > After this, I was still unable to ping the Debian 8 VM from my Windows VM. > > > > > > Questions: > > > > > > 1) Are there any obvious errors in the steps I took and does anyone have > > > any suggestions how I can resolve this issue? > > > > > > 2) There are a number of other incidences of what seemed to be a similar > > > issue here: > > > https://groups.google.com/forum/?nomobile=true#!msg/qubes-users/59kOjfQFBI4/bjS47-jJJgAJ, > > > > > > https://groups.google.com/forum/#!msg/qubes-users/vSyUaOSloYU/ONZNJlhrBAAJ. > > > Are the enabling networking between VMs steps described here still > > > correct and applicable for Qubes 3.2? > > > > > > 3) The IP address assignment suggests that the VMs are on the same > > > network – the Subnet Mask is 255.255.255.0 so surely any devices with an > > > IP address of 10.137.2.x would be able to communicate with each other? > > > What is unique in Xen / Qubes that stops this? > > > > > > 4) Is there a way in which the current routing rules can be displayed and > > > reset back to the default if required? > > > > > > > Hi Max, > > > > I would make sure the basics work before moving on. > > > > 1. You haven't allowed return traffic from the Debian qube. > > Put in an ACCEPT FORWARD rule as you have with source and destination > > reversed. > > > > The rules you have entered to allow forwarding are for traffic to port > > 443. You don't seem to have either ping (icmp) or telnet(tcp port 23) > > enabled. > > > > These look like obvious mistakes. > > > > 2) Yes, I believe the instructions are still correct. > > > > 3) qubes are connected through a netvm - the default firewall rules > > there prohibit traffic between qubes connected downstream: in the > > FORWARD chain is - > > DROP all vif+ vif+ > > > > 4) The firewall and routing rules can be displayed using standard Linux > > commands: iptables and ip route. (I'm not sure exactly what you are > > looking for with this question.) Any changes that you make to a qube > > will be reset on reboot UNLESS you explicitly make changes in /rw/config > > using rc.local and the qubes-firewall script, or use some other > > mechanism. > > There isn't currently a way to see the full route from sys-net through > > to a target qube, but it isnt difficult to get output from the > > intermediates. > > > > Hope this helps - try the simple window7 deb-8 connection first,and > > then move on to the port forwarding. > > > > Cheers > > > > unman > > Hi Unman and Daniel, > > Thank you for your detailed and quick responses. I have attempted to follow > the instructions but have had not much luck yet. Some responses first: > > > 4) The firewall and routing rules can be displayed using standard Linux > > commands: iptables and ip route. (I'm not sure exactly what you are > > looking for with this question.) Any changes that you make to a qube > > will be reset on reboot UNLESS you explicitly make changes in /rw/config > > using rc.local and the qubes-firewall script, or use some other > > mechanism. > > Thanks - I have now rebooted so at least previous changes I have made will > not affect the new commands I input. > > > 3) qubes are connected through a netvm - the default firewall rules > > there prohibit traffic between qubes connected downstream: in the > > FORWARD chain is - > > DROP all vif+ vif+ > > Great, makes sense. > > > The rules you have entered to allow forwarding are for traffic to port > > 443. You don't seem to have either ping (icmp) or telnet(tcp port 23) > > enabled. > > I have held off now adding these. Ultimately, I don't need to connect > externally. So I probably jumped ahead here and added extra complexity. > > > 1. You haven't allowed return traffic from the Debian qube. > > Put in an ACCEPT FORWARD rule as you have with source and destination > > reversed. > > I have now added this step after a reboot but no luck. > > Below is a copy of the iptables lists with the before and after results: > > BEFORE - FIREWALL > [user@sys-firewall ~]$ sudo iptables -L -vx > Chain INPUT (policy DROP 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 DROP udp -- vif+ any anywhere > anywhere udp dpt:bootpc > 0 0 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT icmp -- any any anywhere > anywhere > 0 0 ACCEPT all -- lo any anywhere > anywhere > 0 0 REJECT all -- any any anywhere > anywhere reject-with icmp-host-prohibited > > Chain FORWARD (policy DROP 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT all -- vif0.0 any anywhere > anywhere > 0 0 DROP all -- vif+ vif+ anywhere > anywhere > 0 0 ACCEPT udp -- any any 10.137.2.19 > gateway udp dpt:domain > 0 0 ACCEPT udp -- any any 10.137.2.19 > 10.137.1.254 udp dpt:domain > 0 0 ACCEPT tcp -- any any 10.137.2.19 > gateway tcp dpt:domain > 0 0 ACCEPT tcp -- any any 10.137.2.19 > 10.137.1.254 tcp dpt:domain > 0 0 ACCEPT icmp -- any any 10.137.2.19 > anywhere > 0 0 DROP tcp -- any any 10.137.2.19 > 10.137.255.254 tcp dpt:us-cli > 0 0 ACCEPT all -- any any 10.137.2.19 > anywhere > > Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > > AFTER - FIREWALL > [user@sys-firewall ~]$ sudo iptables -L -vx > Chain INPUT (policy DROP 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 DROP udp -- vif+ any anywhere > anywhere udp dpt:bootpc > 0 0 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT icmp -- any any anywhere > anywhere > 0 0 ACCEPT all -- lo any anywhere > anywhere > 0 0 REJECT all -- any any anywhere > anywhere reject-with icmp-host-prohibited > > Chain FORWARD (policy DROP 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 116 9277 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT all -- any any 10.137.2.19 > 10.137.2.18 > 0 0 ACCEPT all -- vif0.0 any anywhere > anywhere > 0 0 DROP all -- vif+ vif+ anywhere > anywhere > 0 0 ACCEPT udp -- any any 10.137.2.19 > gateway udp dpt:domain > 0 0 ACCEPT udp -- any any 10.137.2.19 > 10.137.1.254 udp dpt:domain > 0 0 ACCEPT tcp -- any any 10.137.2.19 > gateway tcp dpt:domain > 0 0 ACCEPT tcp -- any any 10.137.2.19 > 10.137.1.254 tcp dpt:domain > 0 0 ACCEPT icmp -- any any 10.137.2.19 > anywhere > 0 0 DROP tcp -- any any 10.137.2.19 > 10.137.255.254 tcp dpt:us-cli > 0 0 ACCEPT all -- any any 10.137.2.19 > anywhere > 8 536 ACCEPT udp -- any any 10.137.2.18 > gateway udp dpt:domain > 0 0 ACCEPT udp -- any any 10.137.2.18 > 10.137.1.254 udp dpt:domain > 0 0 ACCEPT tcp -- any any 10.137.2.18 > gateway tcp dpt:domain > 0 0 ACCEPT tcp -- any any 10.137.2.18 > 10.137.1.254 tcp dpt:domain > 0 0 ACCEPT icmp -- any any 10.137.2.18 > anywhere > 0 0 DROP tcp -- any any 10.137.2.18 > 10.137.255.254 tcp dpt:us-cli > 16 1216 ACCEPT all -- any any 10.137.2.18 > anywhere > > Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > > BEFORE - DEBIAN 8 AppVM > user@work-apps:~$ sudo iptables -L -vx > Chain INPUT (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 DROP udp -- vif+ any anywhere > anywhere udp dpt:bootpc > 49 3688 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT icmp -- vif+ any anywhere > anywhere > 1 52 ACCEPT all -- lo any anywhere > anywhere > 0 0 REJECT all -- vif+ any anywhere > anywhere reject-with icmp-host-prohibited > 16 2678 DROP all -- any any anywhere > anywhere > > Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 DROP all -- vif+ vif+ anywhere > anywhere > 0 0 ACCEPT all -- vif+ any anywhere > anywhere > 0 0 DROP all -- any any anywhere > anywhere > > Chain OUTPUT (policy ACCEPT 71 packets, 6762 bytes) > pkts bytes target prot opt in out source > destination > > AFTER - DEBIAN 8 AppVM > user@work-apps:~$ sudo iptables -L -vx > Chain INPUT (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 ACCEPT all -- any any 10.137.2.19 > anywhere > 0 0 DROP udp -- vif+ any anywhere > anywhere udp dpt:bootpc > 165 567437 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT icmp -- vif+ any anywhere > anywhere > 1 52 ACCEPT all -- lo any anywhere > anywhere > 0 0 REJECT all -- vif+ any anywhere > anywhere reject-with icmp-host-prohibited > 21 3043 DROP all -- any any anywhere > anywhere > > Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > 0 0 ACCEPT all -- any any anywhere > anywhere ctstate RELATED,ESTABLISHED > 0 0 ACCEPT all -- any any 10.137.2.18 > 10.137.2.19 > 0 0 DROP all -- vif+ vif+ anywhere > anywhere > 0 0 ACCEPT all -- vif+ any anywhere > anywhere > 0 0 DROP all -- any any anywhere > anywhere > > Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > > I have only started the relevant VMs to keep it as simple as possible and I > have also provided the Kernel IP routing tables in case the virtual interface > assignment is related to these rules: > > [user@sys-firewall ~]$ route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 0.0.0.0 10.137.1.1 0.0.0.0 UG 0 0 0 eth0 > 10.137.1.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0 > 10.137.2.18 0.0.0.0 255.255.255.255 UH 32744 0 0 vif8.0 > 10.137.2.19 0.0.0.0 255.255.255.255 UH 32746 0 0 vif6.0 > > I used the above vif numbers for the instructions from Daniel which I did for > a previous attempt to fix my inter-VM networking (sysctl -w > net/ipv4/conf/vifX.0/proxy_arp=1 <= where vifX.0 are the interfaces to the > VMs you want to network) and succeeded in identifying a broadcast mac but > this didn't allow successful pinging from Windows 7 to the Debian 8 VM by > itself. Was there a further step I was missing? > > I have read up on iptables configuration to see if I can understand this > further but I can't see what I am doing incorrectly. My firewall rules seem > quite encompassing i.e. the input rule 2 allows any protocol and states it > will accept data from the IP address of my Windows VM and send this onto the > Debian AppVM. I presume that the order of rules should influence the routing > and this would not base this off packets of data that match more criteria in > the table? > > Thanks again and apologies if I am requiring baby steps here.
Brief update on this. After attempting to use the Qubes Network Server from Manuel Amador (Rudd-O) to solve this issue with no luck I have gone back to looking at solving this by adjusting the iptables rules. I ran through the steps listed here again: https://www.qubes-os.org/doc/qubes-firewall/#enabling-networking-between-two-vms but instead of trying to ping my Debian 8 VM (10.137.2.18) from the Windows VM (10.137.2.19), I did this from a new Fedora VM (10.137.2.16) to test the results. I simply did the following: Firewall sudo iptables -I FORWARD 2 -s 10.137.2.16 -d 10.137.2.18 -j ACCEPT work-apps iptables -I INPUT -s 10.137.2.16 -j ACCEPT This enabled me to ping from Fedora to the Debian VM. No additional rules were required such as adding ports or adding an ACCEPT FORWARD rule in the Debian VM with the destination and source reversed. Given the ease of achieving this, it seems that the issue here stopping me pinging my Debian VM from Windows is specific to Windows being an HVM. Pinging from an HVM to a PVM does not seem to work but PVM to PVM networking does. Please note that the HVM can ping the firewall and vice versa. Does anyone have any suggestions given this information? Many thanks. -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/d14804d7-c901-48ad-8193-54328035c055%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
