Re: [qubes-devel] Re: GVT-g Discussion

[thorsten...@gmail.com]

Hello everyone,

it has been a while since the last post here, is there any update on this 
topic?
Any gpu virtualization on the horizon in the (near) future?

Thorsten Schierer 

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/8ab4ca70-eca5-453f-9b89-a5cb5e3c58a4n%40googlegroups.com.

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

On 2020-02-11 17:51, Marek Marczykowski-Górecki wrote:
> On Thu, Feb 06, 2020 at 02:53:09PM +0300, Ivan Kardykov wrote:
>> Greetings,
> 
>> there is an update of our experimental GVT-g integration with Qubes.
> 
>> Code was slightly improved to get rid of qemu. Vgpu init routine was
>> moved to libxl and it works with stubdomain.
> 
> 
>> https://github.com/tabit-pro/tabit-qubes-repo
> 
> This is really interesting development!
> Can you say something more about it? Specifically:
> > 1. What a VM have access to really? Full GPU, some context? If a
> context, how is it enforced?
> 
> 2. What components are involved in GPU commands processing and how are
> they isolated?
> 

In theory [1] I see that each VM has access to frame and command buffers
in system memory. The command buffer reused across VMs with smart
shadowing and GVT framework validates graphics memory addresses before
they used by the GPU.
Access to the privileged resources (GPU Page Table Entries and I/O
registers) is handled by xengt module, based on trap-and-emulation.
Render context switching is used per vGPU.

In practice, security audit is the matter of our further experiments.

[1]
https://www.usenix.org/system/files/conference/atc14/atc14-paper-tian.pdf

> 3. Is it possible to enable it only for some VMs - in a way outside of VM
> control?
> 

We made several changes to the existing implementation to use it with
qubes via qvm-features (video-model):
- Patch to libxl creates vGPU instance via sysfs, during hvm domain
execution.
- Stubdomain part extends video model to prevent unnecessary
initialization of emulated VGA.
- Libvirt configuration adapted to parse xengt parameters.


---
Regads,
IK
tabit.pro

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/a2c9b683-171f-6186-9a4c-4e0eca739802%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

[Marek Marczykowski-Górecki]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Thu, Feb 06, 2020 at 02:53:09PM +0300, Ivan Kardykov wrote:
> Greetings,
> 
> there is an update of our experimental GVT-g integration with Qubes.
> 
> Code was slightly improved to get rid of qemu. Vgpu init routine was
> moved to libxl and it works with stubdomain.
> 
> 
> https://github.com/tabit-pro/tabit-qubes-repo

This is really interesting development!
Can you say something more about it? Specifically:

1. What a VM have access to really? Full GPU, some context? If a
context, how is it enforced?

2. What components are involved in GPU commands processing and how are
they isolated?

3. Is it possible to enable it only for some VMs - in a way outside of VM
control?

- -- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-BEGIN PGP SIGNATURE-

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl5Cv4gACgkQ24/THMrX
1ywSQAf/bZhoYV4kd++KmF/o38mZkBUblyZZFfRlXRYuvDYqAFWdmCMVmLyLzLBn
fRDTjph/4XuY3UC0hADBXpVYDSAoSVlCw4E2dza5g+gFipvXEC8l0dCAP3hS3nob
N03t4UGmqBS+Vq6dekVpvgCyLnUGVc1SUDtgM04LNdtJbt+NfeRdsL9QFa5tBaCI
MFmg1HTvdz5/Qg4gmxvLIuwMRwXi6tSPWAcd84a3+3c+kGR8paZsJhztdNbyhrY+
QPPWqNxwNbL5aUEv/HqljM9Qcxt/iXRrVkv0y3ZXHnZMN3ff2ZVru6r3+Ie+tCWc
ZjgSKxHf6YnEsEkxyPd8DOrkdwacZA==
=TCpb
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/20200211145152.GO15453%40mail-itl.

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

Greetings,

there is an update of our experimental GVT-g integration with Qubes.

Code was slightly improved to get rid of qemu. Vgpu init routine was
moved to libxl and it works with stubdomain.


https://github.com/tabit-pro/tabit-qubes-repo

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/3277ad93-e5c6-ccba-9007-639b1195775d%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

On 2020-01-02 18:34, Demi M. Obenour wrote:
> On 2020-01-02 15:33, Kardykov Ivan wrote:
>> Sorry for that bumping, but looks like my posts from work domain (tabit.pro) 
>> stucked at moderate or
>> spam checks.. So my message was about GVT-g.
>>
>> We have some success with adaptation Intel gtv-g technology to Qubes OS.
>> At now it is rather proof of concept state, but it probably could help
>> to test guivm features and discuss of applicability in practice.
>>
>> In this [1] repository we placed specs and patches to build core Qubes
>> components with ability to run and work with GPU accelerated VM. We
>> oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
>> compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
>> this versions in some time.
>>
>> We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
>> xorg dummy driver to guest environment. Our work mostly based on Intel
>> [2][3] and XenServer [4] public repositories.
>>
>> The most controversial question is probably running qemu without any
>> restrictions, but there are several ways to improve it.
> 
> As Marek has stated before, this is a non-starter.  QEMU is not
> considered trusted.  Xen’s attack surface is already too high.
> 

As I see it, useful results could be with testing it to get more about
advantages and finding ways to minimize risks. May be smth with this [1]
movements are compatible with gvt-g.

[1] https://wiki.xenproject.org/wiki/Linux_stub_domains

> The approach I would like to see is a paravirtualized approach
> similar to Genode’s: a small (<10K SLOC), auditable (ideally
> formally verified) driver running in dom0 and/or its own domain that
> communicates with a paravirtualized driver in the VMs.  Any needed
> emulation is done in the stubdomain.  This is likely to be difficult
> for many reasons, but is the only way that I see can hardware
> accelerated graphics being secure enough for Qubes with commodity
> hardware.  If you are willing to pay literally thousands of dollars
> for server-grade GPUs, then SR-IOV and friends become options, which
> make secure hardware acceleration much easier.  However, those are
> far too expensive for virtually all Qubes users.  In particular,
> they are far more expensive than just buying another computer for
> graphics-intensive activities.
> 
> The main activities I know of that need hardware-accelerated graphics,
> and which a Qubes user is likely to want to partake in, are gaming
> and GPGPU.  Video games, in particular, should be considered completely
> untrusted, and so need to be walled off as tightly as possible.
> 
> Sincerely,
> 
> Demi
> 


-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/8dce113c-6afa-3181-9f1c-7eb140b0b725%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

On 2020-01-02 15:26, Kardykov Ivan wrote:
> bump thread
> 
> On Friday, November 22, 2019 at 5:02:14 AM UTC-5, Dylanger Daly wrote:
> 
> Will the Qubes team consider enabling the use of GVT-g so we can
> enjoy hardware accelerated graphics?
> 
> After upgrading to DDR4 Memory I've noticed a significant increase
> in performance, I assume this is because of the many, many memory
> copies taking place, GVT-g should lighten the load.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to qubes-devel+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qubes-devel/f608a18f-15e2-4ad1-94b3-da7ae8b1c553%40googlegroups.com
> .

Sorry for that bumping, but looks like my posts stuck at moderate or
spam checks.. So my message was about GVT-g.

We have some success with adaptation Intel gtv-g technology to Qubes OS.
At now it is rather proof of concept state, but it probably could help
to test guivm features and discuss of applicability in practice.

In this [1] repository we placed specs and patches to build core Qubes
components with ability to run and work with GPU accelerated VM. We
oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
this versions in some time.

We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
xorg dummy driver to guest environment. Our work mostly based on Intel
[2][3] and XenServer [4] public repositories.

The most controversial question is probably running qemu without any
restrictions, but there are several ways to improve it.

At now it works with some issues and limitations, most of them described
in repository readme. We have tested it on Lenovo X1 gen5/gen6 and T480
laptops with KabyLake/KabyLake R chipsets.

[1] https://github.com/tabit-pro/tabit-qubes-repo
[2] https://github.com/intel/gvt-linux
[3] https://github.com/intel/igvtg-qemu
[4] https://github.com/xenserver/xen.pg.git


Regards,
Ivan Kardykov

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/9ffc85a2-9b21-b126-bfbb-831ebad2d7ed%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

['Dylanger Daly' via qubes-devel]

 My bad, I accidentally replied directly to Demi

Hey Ivan,

This is fantastic work! I'll test this out today, I think this will help 
with Firefox's WebRender engine, meaning much less CPU spikes, and reduced 
wattage when I have like 2 Firefox Tabs open.
Not to mention watching YouTube.

Demi you make some valid Security points, however one major GPU activity is 
WebRender (Web Content being rendered within the iGPU)
It's also my understanding Marek mentioned R4.1 would support GVT-g.

I would love to see an approach similar to Genode as well, I just don't 
think it will happen, I'm almost certain GVT-g will be used.

Cheers!

*Demi Replied with*

> Hey Ivan,
> 
> This is fantastic work! I'll test this out today, I think this will help
> with Firefox's WebRender engine, meaning much less CPU spikes, and reduced
> wattage when I have like 2 Firefox Tabs open.
> Not to mention watching YouTube.
> 
> Demi you make some valid Security points, however one major GPU activity 
is
> WebRender (Web Content being rendered within the iGPU)
> It's also my understanding Marek mentioned R4.1 would support GVT-g.
> 
> I would love to see an approach similar to Genode as well, I just don't
> think it will happen, I'm almost certain GVT-g will be used.
> 
> Cheers!

Regarding WebRender: My personal approach (and recommendation) is to
just accept the higher power consumption.  If you can’t, a separate
computer is probably your best option.  The trend in QubesOS is to
emulate more devices (such as audio hardware), and rely on QEMU being
sandboxed for security.

GVT-g may be supported in QubesOS at some point, but not until it
doesn’t require unsandboxed QEMU.  Honestly, I consider QEMU
being in stubdoms to be one of QubesOS’s main selling points.
Especially since dom0 is so out of date that there is almost certain
to be a trivially-exploitable vulnerability there.

Sincerely,

Demi

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/4e5e8079-8aba-4c9f-95c1-7d119db418eb%40googlegroups.com.

Re: [qubes-devel] Re: GVT-g Discussion

[Demi M. Obenour]

On 2020-01-02 15:33, Kardykov Ivan wrote:
> Sorry for that bumping, but looks like my posts from work domain (tabit.pro) 
> stucked at moderate or
> spam checks.. So my message was about GVT-g.
> 
> We have some success with adaptation Intel gtv-g technology to Qubes OS.
> At now it is rather proof of concept state, but it probably could help
> to test guivm features and discuss of applicability in practice.
> 
> In this [1] repository we placed specs and patches to build core Qubes
> components with ability to run and work with GPU accelerated VM. We
> oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
> compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
> this versions in some time.
> 
> We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
> xorg dummy driver to guest environment. Our work mostly based on Intel
> [2][3] and XenServer [4] public repositories.
> 
> The most controversial question is probably running qemu without any
> restrictions, but there are several ways to improve it.

As Marek has stated before, this is a non-starter.  QEMU is not
considered trusted.  Xen’s attack surface is already too high.

The approach I would like to see is a paravirtualized approach
similar to Genode’s: a small (<10K SLOC), auditable (ideally
formally verified) driver running in dom0 and/or its own domain that
communicates with a paravirtualized driver in the VMs.  Any needed
emulation is done in the stubdomain.  This is likely to be difficult
for many reasons, but is the only way that I see can hardware
accelerated graphics being secure enough for Qubes with commodity
hardware.  If you are willing to pay literally thousands of dollars
for server-grade GPUs, then SR-IOV and friends become options, which
make secure hardware acceleration much easier.  However, those are
far too expensive for virtually all Qubes users.  In particular,
they are far more expensive than just buying another computer for
graphics-intensive activities.

The main activities I know of that need hardware-accelerated graphics,
and which a Qubes user is likely to want to partake in, are gaming
and GPGPU.  Video games, in particular, should be considered completely
untrusted, and so need to be walled off as tightly as possible.

Sincerely,

Demi

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/430e381e-7118-5660-38b5-9db65399b02d%40gmail.com.


signature.asc
Description: OpenPGP digital signature