-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

Most of the features of Qubes 4.0 are done, but there are few more things to do
be fully useable. Below is (not so) quick summary.

0. (Mostly) rewrite of Qubes core management scripts.

Over time, our codebase grew significantly, and the focus was on delivering
features and refining overall architecture. This allowed for steady growth of
Qubes OS as both the system and the community. However less thought was given to
code quality and practices. Also, the code documentation was nonexistent.
So, we've decided to rewrite this part, having modularization and extensibility
in mind when designing new code. 

The work have started before Qubes 3.0 release by Wojtek and initially was
scheduled to land there (hence the name "core3" still used internally in some
places). But finalizing it turned out to be much more time consuming than
expected and in consequence this task was deferred to Qubes 4.0.

Today I can say that the core part of it is done. Every major functionality
is already (re-)implemented[5]. And thanks to the new, much easier to extend
code, we've already implemented few cool new features:
 - multiple DispVM support[6]
 - new storage API[7], allowing instant VM cloning, keeping arbitrary number of
   volume revisions (somehow extended qvm-revert-template-changes feature) and
   possibly more in the future
 - improved firewall internal interface[8], allowing more rules, not interfering
   with other firewall tools[9]
 - and many more

1. Moving away from PV[1].

Initially we wanted to move to PVHv2 (aka HVMlite) domains, which are just like
HVM, but without qemu. But unfortunately it turned out PVHv2 is still far from
complete[2], so we've decided to go with "normal" HVM, with qemu in (still PV)
stubdomain and move to PVHv2 when it will be complete. But for Qubes 4.0
release, update qemu to more recent version - instead of the ancient,
unmaintained fork of it, from early Xen days. Mostly to improve
interoperability (see [4] for example problem) and some missing features in old
version (direct kernel boot, just to name one). Unfortunately only that old
qemu fork is supported in upstream Xen stubdomain and updating it there isn't a
priority to the Xen project[3] (the message is year old but nothing really 
changed
since then). So, we've decided to do it ourself, going Linux-based stubdomain
way (see [3] for other options). 

This task is handled by HW42, the current status is that the majority of work is
done (it's bootable, most of things just work), but still have issues. For
example PCI passthrough doesn't work yet, which is essential for NetVM...
The current code referenced below still use PV domains by default.

2. Qubes manager

Since the internal API have changed, Qubes Manager also require adjustments.
And since the code quality degraded here too and its overall design was quite
poor[10], we've decided to throw it away and write new one, improving its
interface at the same time[11]. While there was some progress on this, it
turned out to be too much work to finish it without delaying Qubes 4.0 even
more. So, we've fall back to simpler idea: replace its main window (which was
the most horrible part) with a new one[12], as nice applet, then mostly reuse 
other
parts of the old manager code (like settings window, backup/restore wizards 
etc).

This task is handled by Kalkin. The current state is that most of frontend work
is done, but it still needs to be plugged into new backend API. See [12] for 
details.

3. VM Admin API

While working on connecting GUI parts and new command line tools, it turned out
it will be easier if we'd implement administration API[13] right now - not
delay it until Qubes 4.1. We've designed it the way to allow later expose part
of it to semi-trusted management VM - for example allowing it to manage
_subset_ of VMs. For now, it is mostly to have just one entity (qubesd - qubes
daemon) which handle reads and writes to /var/lib/qubes/qubes.xml and avoid all
the problems with access synchronization.

This task is handled by Wojtek. The current state is protocol design is done,
but it wasn't implemented yet. The current code use racy method of accessing
qubes.xml and may occasionally result in failure on some VM modification. We've
added an band aid to detect such situation and fail the operation, instead of
breaking qubes.xml. This is just temporary solution, until said administration
API got implemented.

4. Summary, the code

Besides the above listed things, it is possible to compile and run what's
already done for Qubes 4.0. You can use qubes-builder[14] with the config
linked below. In addition to the above list, there are plenty of
minor issues, some of them may result in data loss. So, at this point I would
not recommend using it for anything important. I think it's fair to name it
"technology preview 2".

There are also still a couple of rough edges during installation/first run. For
example "LVM thin" storage should be used, but currently it needs to be
selected manually (using custom partitioning option). And depending on the disk
layout, it may require creating those partitions manually.

There is also no package repository yet and most of components are still
versioned with 3.2.x. This means no easy update path from such built system.
The only option to migrate later to the official build is to reinstall the 
system.

The linked configuration use Fedora 23 as a base for dom0 (which will
probably stay the same in final version), and also Fedora 23 as the only
template. But templates will be updated later to more recent version. And
additionally, templates from Qubes 3.2 should be mostly compatible - some
feature new in Qubes 4.0 may not work, but all the basic things should be fine.

Builder configuration:
https://gist.github.com/marmarek/3e59944a890c440157c4a596f6f38b6a

$ sha256sum builder-r4.0.conf
0c0814608b07dc215b2640eb09d0ce07f4eee2ddddb6f9646a7cbdf867c1e4c6 
builder-r4.0.conf


[1] https://github.com/QubesOS/qubes-issues/issues/2185
[2] http://markmail.org/message/pndwc3igjjt5p5qe
[3] http://markmail.org/message/fhor7ojwkfdopkyz
[4] https://github.com/QubesOS/qubes-issues/issues/1943
[5] https://github.com/QubesOS/qubes-issues/issues/1825
[6] https://github.com/QubesOS/qubes-issues/issues/866
[7] https://github.com/QubesOS/qubes-issues/issues/1842
[8] https://github.com/QubesOS/qubes-issues/issues/1815
[9] https://github.com/QubesOS/qubes-issues/issues/974
[10] https://github.com/QubesOS/qubes-issues/issues/1288
[11] https://github.com/QubesOS/qubes-issues/issues/1870
[12] https://github.com/QubesOS/qubes-issues/issues/2132
[13] https://github.com/QubesOS/qubes-issues/issues/853
[14] https://www.qubes-os.org/doc/qubes-builder/

- -- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYRNX/AAoJENuP0xzK19cso0EH/1hoAemx3WnnbUhxQMLEr0zU
3PuO1ENY3rXHSI/UVhn3N4XmjLRLV9jXlaR7E8b139jztyisrxL4dHrfE3rGPaIH
IsoV3UPo1+rySOBraAPt8Lrae77hMPnWoelVOEZzz1bgVcqKxAIALEb22CGm9KAp
A9EPKOrqANgBpwtbOhEwThB4CfHjaFbvCHljbgL5HipZh29i//dfZ4bsSJ7U218o
6+8yzX8HV4B/VFzqknpWtt+xKiccPEQMZdt97ExOCBn6Dq+TjYt0i9x9KucB7C/J
l5FggNS8E55DaJsajQkktziU9gRCVT9fDFdulk+jd4ktKc2FGWMJvnIu0m0NCWQ=
=cg+8
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-devel@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/20161205025038.GV1145%40mail-itl.
For more options, visit https://groups.google.com/d/optout.

Reply via email to