Re: [qubes-users] Re: Is it possible to create a fast clone/copy-on-write Qube?

[fsharpn00b]

> When you clone a VM (lets say an appVM), you'll notice another private
> volume is created and shows the same figures as the volume it
> originates from. But looking at pool00 (the overall container for our
> VM volumes), the Data% barely changes if at all.

I did see in Qubes Manager that some template-based VMs, sys-net, sys-firewall, 
and untrusted, have small sizes (< 100 MB) after I played around with them. I'm 
guessing that represents stuff that has accumulated in their private volumes.

> The Linux storage options won't show a volume that grows from nothing
> and has a size equal to the delta from the parent volume.

That's just what I was (mistakenly) looking for, based on my experience with 
Hyper-V.

> And that brings me to another point: On Btrfs if you have two related
> files (.img volumes) but suspect they have diverged too much over
> time, you can tell Btrfs to perform deduplication to find new
> commonalities and reduce the overall space used.

Btrfs looks interesting, but given I don't fully understand how Qubes 4 handles 
storage even in its vanilla configuration, I'd better get some more experience 
with it under my belt before I experiment with a non-default file system.

Thank you again for your help! This has cleared things up and taught me some 
things.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/39f1776f-6b2a-4b36-9556-3619080ed42e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[qubes-users] Re: Is it possible to create a fast clone/copy-on-write Qube?

[fsharpn00b]

Ok, I discovered pvs to tell me the disk usage, ran it before and after cloning 
a ~10 GB Windows VM, and verified usage was the same before and after:

$ sudo pvs
  PV VG Fmt  Attr PSize  PFree
  /dev/sdb2  qubes_dom0 lvm2 a--  58.76g 8.71g

$ qvm-clone windows windows2
windows2: Cloning private volume
windows2: Cloning root volume

$ sudo pvs
  PV VG Fmt  Attr PSize  PFree
  /dev/sdb2  qubes_dom0 lvm2 a--  58.76g 8.71g

Qubes manager gives the disk of the clone VM as 0 before I run it:

https://imgur.com/NGAjTxg

Though afterward it gets the same disk size as the original:

https://imgur.com/hCGUVqs

So again I assume that's the virtual space allocated to the clone rather than 
the actual space.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/426fb451-b320-4ea8-8ed9-211f2530912c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] Is it possible to create a fast clone/copy-on-write Qube?

[fsharpn00b]

Hi Chris,

Thank you very much for your fast and helpful reply.

For the sake of my own learning, I've been trying to verify what you said by 
doing the following:

1. Clone a VM.
2. Compare the sizes of the parent and child VMs.
3. The child VM (which contains no delta from the parent VM) should have a much 
smaller size than the parent VM (which is about 10 GB).

I haven't been able to do this, I presume due to my ignorance. The above 
procedure would work with Hyper-V VMs, but it might be conceptually wrong for 
regard to Qubes/Xen/Linux. I'm new to logical volumes on Linux, thin 
provisioning, etc., though I've been trying to read up on them today. (If 
anyone knows of good conceptual material on this stuff, I'd certainly 
appreciate it.)

I ran qvm-clone to clone my Windows 8 HVM, named "windows". The clone is named 
"windows2". I have never touched "windows2", so it should have no delta from 
"windows".

In case it matters, the Windows 8 HVM started life as a Hyper-V .vhdx file, 
which I converted to a .vhd using Powershell, then to an .img file using 
StarWind Converter. I then imported it into Qubes using qvm-create with 
--root-move-from.

My Qubes install is plain vanilla and I haven't tweaked any settings, file 
systems, etc.

Here are some commands I've run to try to determine the actual size of 
"windows2", and the output (edited to remove irrelevant info):

$ sudo lvm lvs
  LV  VG Attr   LSize  Pool   
Origin  Data%  Meta%  Move Log Cpy%Sync Convert
  pool00  qubes_dom0 twi-aotz-- 43.98g  
  45.81  27.26   
  rootqubes_dom0 Vwi-aotz-- 43.98g pool00   
  9.03   
  swapqubes_dom0 -wi-ao  5.98g  
 
... 
  vm-windows-private  qubes_dom0 Vwi-a-tz--  2.00g pool00   
  0.00   
  vm-windows-private-1526795188-back  qubes_dom0 Vwi-a-tz--  2.00g pool00   
  0.00   
  vm-windows-root qubes_dom0 Vwi-a-tz-- 10.00g pool00   
  83.61  
  vm-windows-root-1526795191-back qubes_dom0 Vwi-a-tz-- 10.00g pool00   
  82.08  
  vm-windows2-private qubes_dom0 Vwi-a-tz--  2.00g pool00 
vm-windows-private  0.00   
  vm-windows2-rootqubes_dom0 Vwi-a-tz-- 10.00g pool00 
vm-windows-root 83.61  
...

I'm guessing the LSize column means apparent size rather than actual size, so 
vm-windows2-root having size 10 G wouldn't be unexpected.

Now I want to find the block special files for these VMs.

$ ls -la /dev/qubes_dom0
...
lrwxrwxrwx  1 root root8 May 23 13:24 vm-windows2-private -> ../dm-32
lrwxrwxrwx  1 root root8 May 23 13:24 vm-windows2-root -> ../dm-31
lrwxrwxrwx  1 root root8 May 23 13:24 vm-windows-private -> ../dm-18
lrwxrwxrwx  1 root root8 May 23 13:13 vm-windows-private-1526795188-back -> 
../dm-17
lrwxrwxrwx  1 root root8 May 23 13:24 vm-windows-root -> ../dm-20
lrwxrwxrwx  1 root root8 May 23 13:13 vm-windows-root-1526795191-back -> 
../dm-19
lrwxrwxrwx  1 root root8 May 23 13:13 vm-work-private -> ../dm-16

So the parent (vm-windows-root) is dm-20, and the child (vm-windows2-root) is 
dm-31.

$ sudo blockdev --getsize64 /dev/dm-20
10737418240
$ sudo blockdev --getsize64 /dev/dm-31
10737418240

Still getting 10 GB for both of them.

$ sudo fdisk -l /dev/dm-20
Disk /dev/dm-20: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: dos
Disk identifier: 0x18b40766

Device Boot  Start  End  Sectors  
Size Id Type
/dev/mapper/qubes_dom0-vm--windows--root-part1 *  2048   718847   716800  
350M  7 HPFS/NTFS/exFAT
/dev/mapper/qubes_dom0-vm--windows--root-part2  718848 25163775 2928 
11.7G  7 HPFS/NTFS/exFAT

$ sudo fdisk -l /dev/dm-31
Disk /dev/dm-31: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: dos
Disk identifier: 0x18b40766

Device  Boot  Start  End  Sectors  
Size Id Type
/dev/mapper/qubes_dom0-vm--windows2--root-part1 *  2048   718847   716800  
350M  7 HPFS/NTFS/exFAT
/dev/mapper/qubes_dom0-vm--windows2--root-part2  718848 

[qubes-users] Is it possible to create a fast clone/copy-on-write Qube?

[fsharpn00b]

I want to do the following.

1. Create an HVM Qube. This Qube contains a clean install of an OS such as 
Windows 8 or Arch.

2. Clone the Qube from #1. The files that make up this Qube should just contain 
the delta from the parent Qube. Writes are made to the child Qube; reads go to 
the child Qube (if the data being read was written previously) or to the parent 
Qube if not (in other words, copy-on-write.) I want to install applications in 
this Qube, so changes should persist.

Hyper-V does this with differencing disks (see 
https://technet.microsoft.com/en-us/library/cc720381.aspx.)

It looks like Xen does this also, with fast cloning (see 
https://docs.citrix.com/en-us/xencenter/6-2/xs-xc-vms/xs-xc-vms-copy.html.)

My question is, can this be done in Qubes? From searching the Web and the docs, 
it looks like not, but I thought I'd see if I've overlooked anything.

One way would be to have a TemplateVM and a TemplateBasedVM, but have changes 
to the TemplateBasedVM (outside of /home) be persistent. I don't know if Qubes 
allows that.

One possible way (I think) would be to use Btrfs as the dom0 file system, but I 
don't know if that's allowed either.

Otherwise, I'll have to create the parent Qube and then clone it with 
qvm-clone, but it looks like that creates a full copy, which I'd rather avoid 
if possible due to the disk space consumption.

Thank you for any suggestions!

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/9e1e5425-58e3-40f4-9919-e08d16723bd1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[qubes-users] Qubes 4.0: Can't connect to network over Ethernet

[fsharpn00b]

Hello. First-time Qubes user, somewhat new to Linux.

I'm not able to connect to the network over Ethernet. In the system tray is a 
red network icon with an x. If I click it and select "Wired connection", the 
icon changes to two circling red dots (i.e. a "wait" icon). Occasionally a 
message pops up saying "The network connection has been disconnected." 
Eventually it gives up and goes back to the red network icon.

I'm able to connect to the network using the same computer with a different OS.

I tried replacing my cat-5 cable, but no luck.

The cat-5 cable connects to a wifi repeater, if that makes any difference. The 
wifi repeater is programmed with the wifi network information; when I connect a 
computer to it, it's typically treated like any other wired connection.

I'm using wifi, so I don't have a modem I can connect to. I do have a wifi USB 
card, but I don't know how to make Qubes use that (I know how to attach USB 
storage devices, but not this).

I'm running Qubes from a USB stick, if that makes any difference. I'm on the 
same computer I used to install it.

I have an old Biostar TPower X79 mainboard.

When I look at the properties for sys-net, it shows the following device 
attached:

0b:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 
PCI Express Gigabit Ethernet Contoller (rev 06)

I don't see any other network-related devices listed.

sudo lspci -v
00:05.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. 
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 06)
Subsystem: Biostar Microtech Int'l Corp Device [1565:230a]
Physical slot: 5
Flags: bus master, fast devsel, latency 64, IRQ 71
I/O ports at c200 [size=256]
Memory at f2029000 (64-bit, prefetchable) [size=4K]
Memory at f2024000 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [d0] Vital Product Data
Kernel driver in use: r8169
Kernel modules: r8169

sudo systemctl | grep 'eth'
  sys-devices-vif\x2d0-net-eth0.device  loaded active 
plugged   /sys/devices/vif-0/net/eth0   
  sys-subsystem-net-devices-eth0.device loaded active 
plugged   /sys/subsystem/net/devices/eth0   
LOAD   = Reflects whether the unit definition was properly loaded.

sudo journalctl | grep 'eth'
Apr 04 17:02:34 sys-firewall kernel: xen_netfront: Initialising Xen virtual 
ethernet driver

ifconfig -a
eth0: flags=4163  mtu 1500
inet 10.137.0.3  netmask 255.255.255.255  broadcast 10.255.255.255
inet6 fe80::216:3eff:fe5e:6c00  prefixlen 64  scopeid 0x20
ether 00:16:3e:5e:6c:00  txqueuelen 1000  (Ethernet)
RX packets 44  bytes 3192 (3.1 KiB)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 44  bytes 3136 (3.0 KiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
inet 127.0.0.1  netmask 255.0.0.0
inet6 ::1  prefixlen 128  scopeid 0x10
loop  txqueuelen 1000  (Local Loopback)
RX packets 26  bytes 2104 (2.0 KiB)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 26  bytes 2104 (2.0 KiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Last night for some reason ifconfig was listing my ethernet connection as 
"ens5". Not sure why that happened but it seems to be gone now.

Thank you very much for any suggestions, and if I can provide any other info 
just let me know.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/d562bea2-f93b-4bf1-9253-a6ace853b925%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.