Re: Informal voting proposal

[Rich Persaud]

On Nov 6, 2023, at 13:53, Kelly Choi  wrote:
> 
> Hi all,
> 
> As an open-source community, there will always be differences of opinion in 
> approaches and the way we think. It is imperative, however, that we view this 
> diversity as a source of strength rather than a hindrance.
> 
> Recent deliberations within our project have led to certain matters being put 
> on hold due to an inability to reach a consensus. While formal voting 
> procedures serve their purpose, they can be time-consuming and may not always 
> lead to meaningful progress.
> 
> Having received agreement from a few maintainers already, I would like to 
> propose the following:
> 
> Informal voting method:
> Each project should ideally have more than 2 maintainers to facilitate 
> impartial discussions. Projects lacking this configuration will be addressed 
> at a later stage.
> Anyone in the community is welcome to voice their opinions, ideas, and 
> concerns about any patch or contribution.
> If members cannot agree, the majority informal vote of the maintainers will 
> be the decision that stands. For instance, if, after careful consideration of 
> all suggestions and concerns, 2 out of 3 maintainers endorse a solution 
> within the x86 subsystem, it shall be the decision we move forward with.
> Naturally, there may be exceptional circumstances, as such, a formal vote may 
> be warranted but should happen only a few times a year for serious cases only.
> Informal votes can be as easy as 2 out of 3 maintainers providing their 
> Acked-by/Reviewed-by tag. Alternatively, Maintainers can call an informal 
> vote by simply emailing the thread with "informal vote proposed, option 1 and 
> option 2." 
> All maintainers should reply with their vote within 5 working days.  
> Please note that with any new process, there will always be room for 
> improvement and we will reiterate where needed.
> Ultimately our goal here is to prevent the project coming to a standstill 
> while deliberating decisions that we all cannot agree on. This may mean 
> compromising in the short term but I am sure the long-term benefits will 
> stand for themselves.  
> 
> If you have any strong objections to the informal voting, please let me know 
> by 30th November 2023. 
> Should I receive no objections, the process will be implemented as of 1st 
> December 2023.
> 

Apologies for the late response, I was recently asked to look at this thread, 
and it's now the end of my Nov 30th USA day.

In order to evaluate new governance proposals, historical test cases are 
needed.  Then the existing process, proposed process (and other candidate 
processes!) can be applied to each test case in turn, so we can evaluate the 
benefits and costs of each candidate.  

If the problem is not defined, how can candidate solutions be evaluated?  
Perhaps those who have responded to the thread have already discussed the 
problem(s) elsewhere, but we need to include them in the public, on-list 
discussion record.


> Again there will be times for that call for flexibility, but we should always 
> aim to have a vote for two of the best solutions to avoid the project coming 
> to another standstill. 


Unless I am mistaken, only one solution has been proposed for a problem that 
has zero on-list examples or test cases.  The community is being given a choice 
between one solution and no solution?  

If we can define the problem, with more than one historical example, then we 
can consider multiple solutions, pick two of the best solutions, and approve 
one of the solutions for implementation.

Regards,
Rich

p.s. This is a strong objection to the absence of a problem definition.

Re: [RFC PATCH] xen/arm: Rebranding dom0less feature

[Rich Persaud]

On Jul 8, 2023, at 03:29, Luca Fancellu  wrote:
> 
 
 Instead, the use case configurations should themselves be describable.
>>> 
>>> Thanks Christopher, Daniel and all!
>>> 
>>> So if I understand correctly, you are in favor if renaming Dom0less to
>>> Hyperlaunch throughout the Xen codebase? And we need a clarification of
>>> the docs/, especially docs/features/dom0less.pandoc?
>> 
>> Christopher wrote:
 = Community resourcing
>> 
>> Note the pre-requisite work items for upstream Xen, listed under "Community 
>> Resourcing", to merge code for Hyperlaunch common interfaces and test cases, 
>> with docs on configuration of Hyperlaunch to deliver functionality for 
>> dom0less use cases.
> 
> Are you saying that before renaming the “dom0less” feature, we should wait 
> for it to be ported to the common code?

Why "wait"? In what timeframe do you expect dom0less to use Hyperlaunch code?

Can kernel component foo adopt the name of kernel component bar without code 
change?

Can dom0less stakeholders derive Hyperlaunch benefits without using Hyperlaunch 
code?

Rich


P.S. An intellectual property story from Cambridge, UK
https://en.wikipedia.org/wiki/Leibniz–Newton_calculus_controversy
https://www.smithsonianmag.com/history/ten-famous-intellectual-property-disputes-18521880/

"By the early 18th century, many credited the German mathematician and 
philosopher Gottfried Wilhelm Leibniz with inventing the study of calculus. 
Leibniz had, after all, been the first to publish papers on the topic in 1684 
and 1686. But when Englishman Isaac Newton published a book called Opticks in 
1704, in which he asserted himself as the father of calculus, a debate arose. 
Each of the thinkers’ respective countries wanted to stake a claim in what was 
one of the biggest advances in mathematics."

Re: [RFC PATCH] xen/arm: Rebranding dom0less feature

[Rich Persaud]

On Jul 7, 2023, at 21:17, Stefano Stabellini  wrote:
> 
> On Fri, 7 Jul 2023, Christopher Clark wrote:
>> +CC openxt, Jason, Marek
>> 
>> On Fri, Jul 7, 2023 at 2:06 PM Christopher Clark 
>>  wrote:
>>  +CC members of the Hyperlaunch Working Group + participants on earlier 
>> Hyperlaunch threads
>> 
>> On Thu, Jul 6, 2023 at 2:39 PM Stefano Stabellini 
>>  wrote:
>>>  On Thu, 6 Jul 2023, George Dunlap wrote:
 On Wed, Jul 5, 2023 at 11:14 PM Stefano Stabellini 
  wrote:
On Wed, 5 Jul 2023, George Dunlap wrote:
>>>> On Mon, Jul 3, 2023 at 9:55 PM P S  wrote:
>>>>   > On Jul 3, 2023, at 15:45, Luca Fancellu 
>>>  wrote:
>>>>   >
>>>>   >> On 3 Jul 2023, at 18:48, Stefano Stabellini 
>>>  wrote:
>>>>   >>
>>>>   >>> On Mon, 3 Jul 2023, Daniel P. Smith wrote:
>>>>   >>> On 7/1/23 11:13, Luca Fancellu wrote:
>>>>   > On 1 Jul 2023, at 08:53, Andrew Cooper 
>>>  wrote:
>>>>   >
>>>>   > On 30/06/2023 10:12 am, Luca Fancellu wrote:
>>>>   >> The "dom0less" feature was intended to be the feature 
>>> where a domU
>>>>   >> domain could be launched without the control domain 
>>> (Dom0)
>>>>   >> intervention, however the name seems to suggest that 
>>> Dom0 cannot
>>>>   >> be part of the configuration, while instead it's a 
>>> possible use case.
>>>>   >>
>>>>   >> To avoid that, rename the "dom0less" configuration 
>>> with the name
>>>>   >> "hyperlaunch", that is less misleading.
>>>>   >>
>>>>   >> Signed-off-by: Luca Fancellu 
>>>>   >> ---
>>>>   >> This is an RFC to get the feeling of the community 
>>> about the name
>>>>   >> change, for now it's everything in one patch just to 
>>> see how it
>>>>   >> will look like, if there is interest on proceeding 
>>> into it, I can
>>>>   >> split in more commit.
>>>>   >
>>>>   > Have you discussed this with Dan and Chris at all?  
>>> You haven't even
>>>>   > CC'd them.
>>>>   
>>>>    No, this rename idea started from a chat during the 
>>> summit, anyway Julien
>>>>    promptly add them to the CC, because I forgot.
>>>>   >>>
>>>>   >>> No worries and thank you for considering and taking the 
>>> time to do this RFC.
>>>>   >>> It is greatly appreciated that there is a strong 
>>> willingness to have dom0less
>>>>   >>> and hyperlaunch merged.
>>>>   >>>
>>>>   >
>>>>   > While there is a lot of end-goal in common between the 
>>> dom0less and
>>>>   > hyperlaunch, and that the name dom0less is deeply 
>>> misleading,
>>>>   > hyperlaunch is specifically not this.
>>>>   
>>>>    Yes Hyperlaunch is more than this, however as I said, 
>>> with this RFC I would
>>>>    like
>>>>    to ear opinions, @Daniel @Christopher could it be a 
>>> proper name for the
>>>>    dom0less
>>>>    feature?
>>>>   >>>
>>>>   >>> As Andy has alluded, hyperlaunch is meant to provide a 
>>> flexible means to
>>>>   >>> handle domain construction at boot to meet a wide range 
>>> of possible use cases.
>>>>   >>> One of those use cases is dom0less, so yes, ultimately 
>>> what dom0less does
>>>>   >>> today will be achievable under hyperlaunch. Our intended 
>>> approach to align the
>>>>   >>> two implementations is one that is meant to be minimally 
>>> disruptive, since
>>>>   >>> dom0less is considered a supported (SUPPORT.md) 
>>> capability. As mentioned, we
>>>>   >>> are greatly appreciative to the openness to adopt the 
>>> name,
>>>>   >>
>>>>   >> Thanks Daniel!
>>>>   >>
>>>>   >>
>>>>   >>> but a big concern
>>>>   >>> I personally have is the confusion it could cause a 
>>> general user. A blanket
>>>>   >>> rename would end up with two documents in the docs tree 
>>> that provide two
>>>>   >>> different explanations of hyperlaunch and two different 
>>> device tree
>>>>   >>> definitions. So I think a more measured approach should 
>>> be considered here.
>>>>   >>>
>>>>    If this patch makes things more difficult for the 
>>> Hyperlunch serie, I’m ok
>>>>    to drop it,
>>>>    my only aim was just to find a less misleading name for 
>>> the feature.
>>>>   >>>
>>>>   >>> What I would like to 

Re: [RFC PATCH] xen/arm: Rebranding dom0less feature

[Rich Persaud]

Hi Luca,

On Jun 30, 2023, at 05:12, Luca Fancellu  wrote:
> 
> The "dom0less" feature was intended to be the feature where a domU
> domain could be launched without the control domain (Dom0)
> intervention, however the name seems to suggest that Dom0 cannot
> be part of the configuration, while instead it's a possible use case.

Thanks for your interest in Xen boot integrity. Please see the 2018 domB RFC:
https://lists.xenproject.org/archives/html/xen-devel/2018-06/msg01306.html

At Xen Summit 2018 (Nanjing) and Xen Summit 2019 (Chicago), OpenXT contributors 
made a case to Xen-on-Arm contributors for the architectural unification of 
incumbent dom0less (Arm) and the domB (x86) proposal for improving Xen boot 
integrity.

> To avoid that, rename the "dom0less" configuration with the name
> "hyperlaunch", that is less misleading.

2018-2022 work on Xen launch integrity, thanks to Apertus and Star Lab: 
https://wiki.xenproject.org/wiki/Hyperlaunch
https://www.theregister.com/2022/12/16/xen_4_17_hyperlaunch/

2023 Hyperlaunch design session last week, thanks to Apertus and AMD:
https://lists.xenproject.org/archives/html/xen-devel/2023-06/msg01870.html

> Signed-off-by: Luca Fancellu 

If Arm is now ready to invest engineering resources into new Xen launch 
integrity features for security and safety-critical use cases, that is exciting 
news, 5 years into the on-again-off-again bootstrapped Hyperlaunch project! The 
roadmap would benefit from new funding.

Would you like to attend the next Xen working group call for Hyperlaunch?

Rich

Re: How to isolate vital part of a host with the Xen Hypervisor?

[Rich Persaud]

On Aug 21, 2022, at 1:36 AM, Jason Long  wrote:
> 
> Hello,
> Is it possible to install the Xen Hypervisor for just isolate my host OS and 
> disable its Virtualization features (Install guest OS)?

You can often disable hardware virtualization via a UEFI/BIOS setting.  Even if 
hardware virtualization support was disabled in UEFI/BIOS or already claimed by 
a bare-metal hypervisor like Xen or ESXi, it would still be possible to create 
a guest VM using software-only virtualization, e.g. VMware Workstation.

Rich

Re: [PATCH] xen-pcifront: Handle missed Connected state

[Rich Persaud]

On Aug 29, 2022, at 11:16 AM, Jason Andryuk  wrote:
> 
> An HVM guest with linux stubdom and 2 PCI devices failed to start as
> libxl timed out waiting for the PCI devices to be added.  It happens
> intermittently but with some regularity.  libxl wrote the two xenstore
> entries for the devices, but then timed out waiting for backend state 4
> (Connected) - the state stayed at 7 (Reconfiguring).  (PCI passthrough
> to an HVM with stubdomain is PV passthrough to the stubdomain and then
> HVM passthrough with the QEMU inside the stubdomain.)
> 
> The stubdom kernel never printed "pcifront pci-0: Installing PCI
> frontend", so it seems to have missed state 4 which would have
> called pcifront_try_connect -> pcifront_connect_and_init_dma

Is there a state machine doc/flowchart for LibXL and Xen PCI device passthrough 
to Linux? This would be a valuable addition to Xen's developer docs, even as a 
whiteboard photo in this thread.

Rich

Re: PROPOSAL: Delete www-archive.xenproject.org

[Rich Persaud]

On May 12, 2022, at 11:55 AM, George Dunlap  wrote:
> 
>> On Apr 14, 2022, at 4:49 PM, George Dunlap  wrote:
>> 
>> I’m pretty sure www-archive.xenproject.org is at least N-2 for websites; 
>> last updated nearly 9 years ago.  As far as I can tell there’s nothing 
>> terribly interesting stored on the site itself.  I’m going to pursue 
>> deleting it within 4 weeks unless someone objects.
> 
> I have instructed Credativ to take the site down, but keep it in a tarball 
> should we ever need to recover anything from it.

Is all of the historical content from www-archive replicated on the current 
site? That content was indexed by search engines and linked from other parts of 
the web. Are the existing links going to break, or be redirected?

Hosting static content is essentially free, e.g. the entire site could be 
hosted on GH Pages. Is there a cost to keeping the current site online? How 
large are the uncompressed HTML files and images? We don't delete historical 
mailing list archives, why delete historical web site archives?

While on the topic of historical content, do we have a self-hosted mirror of 
slides/videos from from SlideShare, Vimeo and YouTube? More than one publisher 
has found their content deleted due to policy changes. It would be prudent to 
have our own backups.

Rich

[events] BigBlueButton enhancements

[Rich Persaud]


LPC and LF have issued an RFQ that includes upstream usability enhancements for 
BBB and integration with Matrix chat. 

Some feature gaps (page 7, HTML5 client) may be familiar to Xen Summit 
participants. If Xen Summit has additional enhancement requests for 2022, these 
could be coordinated with LF/LPC, as a follow-on project.

https://www.linuxplumbersconf.org/event/11/attachments/732/1341/LPC2021-RFQ01.pdf

Rich

Re: [PATCH v2 01/13] docs: Warn about incomplete vtpmmgr TPM 2.0 support

[Rich Persaud]

On May 6, 2021, at 10:00, Jason Andryuk  wrote:
> The vtpmmgr TPM 2.0 support is incomplete.  Add a warning about that to
> the documentation so others don't have to work through discovering it is
> broken.
> 
> Signed-off-by: Jason Andryuk 
> Acked-by: Andrew Cooper 
> ---
> docs/man/xen-vtpmmgr.7.pod | 11 +++
> 1 file changed, 11 insertions(+)
> 
> diff --git a/docs/man/xen-vtpmmgr.7.pod b/docs/man/xen-vtpmmgr.7.pod
> index af825a7ffe..875dcce508 100644
> --- a/docs/man/xen-vtpmmgr.7.pod
> +++ b/docs/man/xen-vtpmmgr.7.pod
> @@ -222,6 +222,17 @@ XSM label, not the kernel.
> 
> =head1 Appendix B: vtpmmgr on TPM 2.0
> 
> +=head2 WARNING: Incomplete - cannot persist data
> +
> +TPM 2.0 support for vTPM manager is incomplete.  There is no support for
> +persisting an encryption key, so vTPM manager regenerates primary and 
> secondary
> +key handles each boot.
> +
> +Also, the vTPM manger group command implementation hardcodes TPM 1.2 
> commands.
> +This means running manage-vtpmmgr.pl fails when the TPM 2.0 hardware rejects
> +the TPM 1.2 commands.  vTPM manager with TPM 2.0 cannot create groups and
> +therefore cannot persist vTPM contents.
> +
> =head2 Manager disk image setup:
> 
> The vTPM Manager requires a disk image to store its encrypted data. The image
> -- 
> 2.30.2

Should SUPPORT.md also be updated?

https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=SUPPORT.md;hb=refs/heads/master

Rich

Re: [openxt-dev] VirtIO-Argo initial development proposal

[Rich Persaud]

On Dec 17, 2020, at 07:13, Jean-Philippe Ouellet  wrote:
> On Wed, Dec 16, 2020 at 2:37 PM Christopher Clark
>  wrote:
>> Hi all,
>> 
>> I have written a page for the OpenXT wiki describing a proposal for
>> initial development towards the VirtIO-Argo transport driver, and the
>> related system components to support it, destined for OpenXT and
>> upstream projects:
>> 
>> https://openxt.atlassian.net/wiki/spaces/~cclark/pages/1696169985/VirtIO-Argo+Development+Phase+1
>> 
>> Please review ahead of tomorrow's OpenXT Community Call.
>> 
>> I would draw your attention to the Comparison of Argo interface options 
>> section:
>> 
>> https://openxt.atlassian.net/wiki/spaces/~cclark/pages/1696169985/VirtIO-Argo+Development+Phase+1#Comparison-of-Argo-interface-options
>> 
>> where further input to the table would be valuable;
>> and would also appreciate input on the IOREQ project section:
>> 
>> https://openxt.atlassian.net/wiki/spaces/~cclark/pages/1696169985/VirtIO-Argo+Development+Phase+1#Project:-IOREQ-for-VirtIO-Argo
>> 
>> in particular, whether an IOREQ implementation to support the
>> provision of devices to the frontends can replace the need for any
>> userspace software to interact with an Argo kernel interface for the
>> VirtIO-Argo implementation.
>> 
>> thanks,
>> Christopher
> 
> Hi,
> 
> Really excited to see this happening, and disappointed that I'm not
> able to contribute at this time. I don't think I'll be able to join
> the call, but wanted to share some initial thoughts from my
> middle-of-the-night review anyway.
> 
> Super rough notes in raw unedited notes-to-self form:
> 
> main point of feedback is: I love the desire to get a non-shared-mem
> transport backend for virtio standardized. It moves us closer to an
> HMX-only world. BUT: virtio is relevant to many hypervisors beyond
> Xen, not all of which have the same views on how policy enforcement
> should be done, namely some have a preference for capability-oriented
> models over type-enforcement / MAC models. It would be nice if any
> labeling encoded into the actual specs / guest-boundary protocols
> would be strictly a mechanism, and be policy-agnostic, in particular
> not making implicit assumptions about XSM / SELinux / similar. I don't
> have specific suggestions at this point, but would love to discuss.
> 
> thoughts on how to handle device enumeration? hotplug notifications?
> - can't rely on xenstore
> - need some internal argo messaging for this?
> - name service w/ well-known names? starts to look like xenstore
> pretty quickly...
> - granular disaggregation of backend device-model providers desirable
> 
> how does resource accounting work? each side pays for their own delivery ring?
> - init in already-guest-mapped mem & simply register?
> - how does it compare to grant tables?
>  - do you need to go through linux driver to alloc (e.g. xengntalloc)
> or has way to share arbitrary otherwise not-special userspace pages
> (e.g. u2mfn, with all its issues (pinning, reloc, etc.))?
> 
> ioreq is tangled with grant refs, evt chans, generic vmexit
> dispatcher, instruction decoder, etc. none of which seems desirable if
> trying to move towards world with strictly safer guest interfaces
> exposed (e.g. HMX-only)
> - there's no io to trap/decode here, it's explicitly exclusively via
> hypercall to HMX, no?
> - also, do we want argo sendv hypercall to be always blocking & synchronous?
>  - or perhaps async notify & background copy to other vm addr space?
>  - possibly better scaling?
>  - accounting of in-flight io requests to handle gets complicated
> (see recent XSA)
>  - PCI-like completion request semantics? (argo as cross-domain
> software dma engine w/ some basic protocol enforcement?)
> 
> "port" v4v driver => argo:
> - yes please! something without all the confidence-inspiring
> DEBUG_{APPLE,ORANGE,BANANA} indicators of production-worthy code would
> be great ;)
> - seems like you may want to redo argo hypercall interface too? (at
> least the syscall interface...)
>  - targeting synchronous blocking sendv()?
>  - or some async queue/completion thing too? (like PF_RING, but with
> *iov entries?)
>  - both could count as HMX, both could enforce no double-write racing
> games at dest ring, etc.
> 
> re v4vchar & doing similar for argo:
> - we may prefer "can write N bytes? -> yes/no" or "how many bytes can
> write? -> N" over "try to write N bytes -> only wrote M, EAGAIN"
> - the latter can be implemented over the former, but not the other way around
> - starts to matter when you want to be able to implement in userspace
> & provide backpressure to peer userspace without additional buffering
> & potential lying about durability of writes
> - breaks cross-domain EPIPE boundary correctness
> - Qubes ran into same issues when porting vchan from Xen to KVM
> initially via vsock
> 
> some virtio drivers explicitly use shared mem for more than just
> communication rings:
> - e.g. virtio-fs, which can map pages as DAX-like fs backing to 

Re: [openxt-dev] Re: Follow up on libxl-fix-reboot.patch

[Rich Persaud]

(adding xen-devel & toolstack devs)

On Dec 14, 2020, at 16:12, Jason Andryuk  wrote:
> 
> On Fri, Dec 11, 2020 at 3:56 PM Chris Rogers  wrote:
>> 
>> This is a follow up to a request during our roadmapping meeting to clarify 
>> the purpose of libxl-fix-reboot.patch on the current version of Xen in 
>> OpenXT (4.12).  It's pretty simple.  While the domctl API does define a 
>> trigger for reset in xen/include/public/domctl.h:
>> 
> 
>> The call stack looks like this:
>>> libxl_send_trigger(ctx, domid, LIBXL_TRIGGER_RESET, 0);
>>> xc_domain_send_trigger(ctx->xch, domid, XEN_DOMCTL_SENDTRIGGER_RESET, 
>>> vcupid);
>>> do_domctl()
>>> arch_do_domctl()
>> and reaching the case statement in arch_do_domctl() for 
>> XEN_DOMCTL_sendtrigger, with RESET, we get -ENOSYS as illustrated above.
> 
> Thanks, Chris.  It's surprising that xl trigger reset exists, but
> isn't wired through to do anything.  And that reboot has a fallback
> command to something that doesn't work.
> 
> If we have to turn reboot into shutdown + start, it seems like that
> could be done in xenmgr instead of libxl.  Similarly, this may avoid
> the signaling between xenmgr and libxl.

If upstream Xen's libxl cannot support VM reset, can we drop/hide reset support 
from the OpenXT CLI and UIVM? That would avoid incurring costs for a fake 
feature with no long-term future. A reset is not the same as shutdown + start.  
If reset is not supportable, the user can perform shutdown + reboot manually. 
Then they would at least be aware of the consequences, e.g. temporary storage 
snapshots will be deleted and changes lost immediately.

OpenXT derivatives which need reset support can use another Xen toolstack which 
provides this capability, e.g. the Citrix XenServer xapi ocaml toolstack, for 
this single function.  Or the old XenClient xenops fork of xapi.

The long-term direction, based on an upstream prototype in Rust, is a low level 
toolstack daemon that accepts input over an RPC protocol that is stable and 
versioned, which will drive a stable hypercall ABI for Xen. We can ask for 
reset support to be prioritized in the Rust prototype, which would then enable 
testing of OpenXT integration.

Hopefully an upstream Xen LibXL developer will recall why the reset logic isn't 
yet fully wired up.

Rich

Re: [PATCH v2] xen: EXPERT clean-up and introduce UNSUPPORTED

[Rich Persaud]


On Nov 20, 2020, at 05:09, Jan Beulich  wrote:
> On 19.11.2020 22:40, Stefano Stabellini wrote:
>> On Thu, 19 Nov 2020, Jan Beulich wrote:
>>> On 18.11.2020 22:00, Stefano Stabellini wrote:
 On Wed, 18 Nov 2020, Jan Beulich wrote:
> On 18.11.2020 01:50, Stefano Stabellini wrote:
>> 1) It is not obvious that "Configure standard Xen features (expert
>> users)" is actually the famous EXPERT we keep talking about on xen-devel
> Which can be addressed by simply changing the one prompt line.
>> 2) It is not obvious when we need to enable EXPERT to get a specific
>> feature
>> In particular if you want to enable ACPI support so that you can boot
>> Xen on an ACPI platform, you have to enable EXPERT first. But searching
>> through the kconfig menu it is really not clear (type '/' and "ACPI"):
>> nothing in the description tells you that you need to enable EXPERT to
>> get the option.
> And what causes this to be different once you switch to UNSUPPORTED?
 Two things: firstly, it doesn't and shouldn't take an expert to enable
 ACPI support, even if ACPI support is experimental. So calling it
 UNSUPPORTED helps a lot. This is particularly relevant to the ARM Kconfig
 options changed by this patch. Secondly, this patch is adding
 "(UNSUPPORTED)" in the oneline prompt so that it becomes easy to match
 it with the option you need to enable.
>>> There's redundancy here then, which I think is in almost all cases
>>> better to avoid. That's first and foremost because the two places
>>> can go out of sync. Therefore, if the primary thing is to help
>>> "make menuconfig" (which I admit I don't normally use, as it's
>>> nothing that gets invoked implicitly by the build process afaict,
>>> i.e. one has to actively invoke it), perhaps we should enhance
>>> kconfig to attach at least a pre-determined subset of labels to
>>> the prompts automatically?
>>> And second, also in reply to what you've been saying further down,
>>> perhaps we would better go with a hierarchy of controls here, e.g.
>>> EXPERT -> EXPERIMENTAL -> UNSUPPORTED?
>> 
>> Both these are good ideas worth discussing; somebody else made a similar
>> suggestion some time back. I was already thinking this could be a great
>> candidate for one of the first "working groups" as defined by George
>> during the last community call because the topic is not purely
>> technical: a working group could help getting alignment and make
>> progress faster. We can propose it to George when he is back.
>> 
>> However, I don't think we need the working group to make progress on
>> this limited patch that only addresses the lowest hanging fruit.
>> 
>> I'd like to suggest to make progress on this patch in its current form,
>> and in parallel start a longer term discussion on how to do something
>> like you suggested above.
> 
> Okay, I guess I can accept this. So FAOD I'm not objecting to the
> change (with some suitable adjustments, as discussed), but I'm
> then also not going to be the one to ack it. Nevertheless I'd like
> to point out that doing such a partial solution may end up adding
> confusion rather than reducing it.

Seconded.

> Much depends on how exactly consumers interpret what we hand to them.

How have Xen consumers changed during the last 15 years?  The UX (user 
experience) community uses a technique called "user stories" [0][1][2].  It may 
be worth writing a few sentences about the users (distro packagers? Xen 
derivatives? hypervisor developers? hosting companies? malware developers?) 
whose needs are addressed by proposed changes.  One could then reason about UX 
of Xen feature selection, before and after proposed changes.

Rich

[0] https://uxdict.io/design-thinking-methods-user-stories-3b9467313a04
[1] https://uxdesign.cc/fostering-ux-on-a-devops-culture-bb92716e3f43
[2] 
https://about.gitlab.com/blog/2020/03/27/how-we-utilize-user-stories-as-a-collaborative-design-tool/

Re: [ANNOUNCE] Call for agenda items for 5 November 2020 Community Call @ 16:00 UTC

[Rich Persaud]

> On Nov 5, 2020, at 11:01, Jan Beulich  wrote:
> On 30.10.2020 15:47, George Dunlap wrote:
>> Hi all,
>> 
>> The proposed agenda is in 
>> https://cryptpad.fr/pad/#/2/pad/edit/k-0Aj+Sxb5SliLWrFRBwx49V/ and you can 
>> edit to add items.  Alternatively, you can reply to this mail directly.
>> 
>> Agenda items appreciated a few days before the call: please put your name 
>> besides items if you edit the document.
>> 
>> Note the following administrative conventions for the call:
>> * Unless, agreed in the pervious meeting otherwise, the call is on the 1st 
>> Thursday of each month at 1600 British Time (either GMT or BST)
>> * I usually send out a meeting reminder a few days before with a provisional 
>> agenda
>> 
>> * To allow time to switch between meetings, we'll plan on starting the 
>> agenda at 16:05 sharp.  Aim to join by 16:03 if possible to allocate time to 
>> sort out technical difficulties 
>> 
>> * If you want to be CC'ed please add or remove yourself from the 
>> sign-up-sheet at 
>> https://cryptpad.fr/pad/#/2/pad/edit/D9vGzihPxxAOe6RFPz0sRCf+/
>> 
>> Best Regards
>> George
>> 
>> 
>> 
>> == Dial-in Information ==
>> ## Meeting time
>> 16:00 - 17:00 UTC
>> Further International meeting times: 
>> https://www.timeanddate.com/worldclock/meetingdetails.html?year=2020=11=5=16=0=0=1234=37=224=179
>> 
>> 
>> ## Dial in details
>> Web: https://www.gotomeet.me/GeorgeDunlap
>> 
>> You can also dial in using your phone.
>> Access Code: 168-682-109
>> 
>> China (Toll Free): 4008 811084
>> Germany: +49 692 5736 7317
> 
> FYI: This number continues to not work.
> 
> Jan

This appears to be the new number since May 2020, based on [0].  I called and 
confirmed that the number works and is expecting a number to be input, but 
can't otherwise verify the German prompts:

   Germany: +49 721 9881 4161

Rich

[0] https://www.youtube.com/watch?v=Embn9_JEqRo

Re: [RFC PATCH] xen: EXPERT clean-up

[Rich Persaud]

> On Nov 3, 2020, at 16:15, Stefano Stabellini  wrote:
> 
> On Tue, 3 Nov 2020, Rich Persaud wrote:
>>> On Nov 3, 2020, at 14:37, Stefano Stabellini  wrote:
>>> 
>>> On Tue, 3 Nov 2020, Jan Beulich wrote:
>>>>> On 02.11.2020 22:41, Stefano Stabellini wrote:
>>>>> On Mon, 2 Nov 2020, Jan Beulich wrote:
>>>>>> On 31.10.2020 01:24, Stefano Stabellini wrote:
>>>>>>> @@ -79,8 +79,8 @@ config SBSA_VUART_CONSOLE
>>>>>>> SBSA Generic UART implements a subset of ARM PL011 UART.
>>>>>>> 
>>>>>>> config ARM_SSBD
>>>>>>> -bool "Speculative Store Bypass Disable" if EXPERT
>>>>>>> -depends on HAS_ALTERNATIVE
>>>>>>> +bool "Speculative Store Bypass Disable"
>>>>>>> +depends on HAS_ALTERNATIVE && EXPERT
>>>>>>>   default y
>>>>>> 
>>>>>> At the example of this, I'm afraid when the default isn't "n"
>>>>>> (or there's no default directive at all, as ought to be
>>>>>> equivalent to and preferred over "default n"), such a
>>>>>> transformation is not functionally identical: Before your
>>>>>> change, with !EXPERT this option defaults to y. After your
>>>>>> change this option is unavailable (which resolves to it being
>>>>>> off for all consuming purposes).
>>>>>> 
>>>>>> IOW there are reasons to have "if ..." attached to the prompts
>>>>>> (for this construct indeed only making the prompt conditional,
>>>>>> not the entire option), but there are also cases where the
>>>>>> cleanup you do is indeed desirable / helpful.
>>>>> 
>>>>> Yeah, thanks for catching it, it is obviously a problem.
>>>>> 
>>>>> My intention was just to "tag" somehow the options to EXPERT so that it
>>>>> would show on the menu. Maybe a better, simpler, way to do it is
>>>>> to add the word "EXPERT" to the one line prompt:
>>>>> 
>>>>> config ARM_SSBD
>>>>> -bool "Speculative Store Bypass Disable" if EXPERT
>>>>> +bool "Speculative Store Bypass Disable (EXPERT)" if EXPERT
>>>>>   depends on HAS_ALTERNATIVE
>>>>>   default y
>>>>>   help
>>>>> 
>>>>> 
>>>>> What do you think?
>>>> 
>>>> While on the surface this may look like an improvement, I don't
>>>> see how it would actually help: If you read the Kconfig file
>>>> itself, the dependency is seen anyway. And on the menu I don't
>>>> see the point of telling someone who has enabled EXPERT that a
>>>> certain option is (or is not) an expert one. If they think
>>>> they're experts, so should they be treated. (It was, after all,
>>>> a deliberate decision to make enabling expert mode easier, and
>>>> hence easier to use for what one might consider not-really-
>>>> experts. I realize saying so may be considered tendentious; I
>>>> mean it in a purely technical sense, and I'd like to apologize
>>>> in advance to anyone not sharing this as a possible perspective
>>>> to take.)
>>>> 
>>>> Plus, of course, the addition of such (EXPERT) markers to
>>>> future options' prompts is liable to get forgotten now and then,
>>>> so sooner or later we'd likely end up with an inconsistent
>>>> mixture anyway.
>>> 
>>> I tend to agree with you on everything you wrote. The fundamental issue
>>> is that we are (mis)using EXPERT to tag features that are experimental,
>>> as defined by SUPPORT.md.
>>> 
>>> It is important to be able to distinguish clearly at the kconfig level
>>> options that are (security) supported from options that are
>>> unsupported/experimental. Today the only way to do it is with EXPERT
>>> which is not great because:
>>> 
>>> - it doesn't convey the idea that it is for unsupported/experimental
>>> features
>>> - if you want to enable one unsupported feature, it is not clear what
>>> you have to do
>>> 
>>> So maybe we should replace EXPERT with UNSUPPORTED (or EXPERIMENTAL) in
>>> the Kconfig menu?
>>> 
>>> It would make it clearer that by enabling UNSUPPORTE

Re: [RFC PATCH] xen: EXPERT clean-up

[Rich Persaud]

On Nov 3, 2020, at 14:37, Stefano Stabellini  wrote:
> 
> On Tue, 3 Nov 2020, Jan Beulich wrote:
>>> On 02.11.2020 22:41, Stefano Stabellini wrote:
>>> On Mon, 2 Nov 2020, Jan Beulich wrote:
 On 31.10.2020 01:24, Stefano Stabellini wrote:
> @@ -79,8 +79,8 @@ config SBSA_VUART_CONSOLE
>  SBSA Generic UART implements a subset of ARM PL011 UART.
> 
> config ARM_SSBD
> -bool "Speculative Store Bypass Disable" if EXPERT
> -depends on HAS_ALTERNATIVE
> +bool "Speculative Store Bypass Disable"
> +depends on HAS_ALTERNATIVE && EXPERT
>default y
 
 At the example of this, I'm afraid when the default isn't "n"
 (or there's no default directive at all, as ought to be
 equivalent to and preferred over "default n"), such a
 transformation is not functionally identical: Before your
 change, with !EXPERT this option defaults to y. After your
 change this option is unavailable (which resolves to it being
 off for all consuming purposes).
 
 IOW there are reasons to have "if ..." attached to the prompts
 (for this construct indeed only making the prompt conditional,
 not the entire option), but there are also cases where the
 cleanup you do is indeed desirable / helpful.
>>> 
>>> Yeah, thanks for catching it, it is obviously a problem.
>>> 
>>> My intention was just to "tag" somehow the options to EXPERT so that it
>>> would show on the menu. Maybe a better, simpler, way to do it is
>>> to add the word "EXPERT" to the one line prompt:
>>> 
>>> config ARM_SSBD
>>> -bool "Speculative Store Bypass Disable" if EXPERT
>>> +bool "Speculative Store Bypass Disable (EXPERT)" if EXPERT
>>>depends on HAS_ALTERNATIVE
>>>default y
>>>help
>>> 
>>> 
>>> What do you think?
>> 
>> While on the surface this may look like an improvement, I don't
>> see how it would actually help: If you read the Kconfig file
>> itself, the dependency is seen anyway. And on the menu I don't
>> see the point of telling someone who has enabled EXPERT that a
>> certain option is (or is not) an expert one. If they think
>> they're experts, so should they be treated. (It was, after all,
>> a deliberate decision to make enabling expert mode easier, and
>> hence easier to use for what one might consider not-really-
>> experts. I realize saying so may be considered tendentious; I
>> mean it in a purely technical sense, and I'd like to apologize
>> in advance to anyone not sharing this as a possible perspective
>> to take.)
>> 
>> Plus, of course, the addition of such (EXPERT) markers to
>> future options' prompts is liable to get forgotten now and then,
>> so sooner or later we'd likely end up with an inconsistent
>> mixture anyway.
> 
> I tend to agree with you on everything you wrote. The fundamental issue
> is that we are (mis)using EXPERT to tag features that are experimental,
> as defined by SUPPORT.md.
> 
> It is important to be able to distinguish clearly at the kconfig level
> options that are (security) supported from options that are
> unsupported/experimental. Today the only way to do it is with EXPERT
> which is not great because:
> 
> - it doesn't convey the idea that it is for unsupported/experimental
>  features
> - if you want to enable one unsupported feature, it is not clear what
>  you have to do
> 
> So maybe we should replace EXPERT with UNSUPPORTED (or EXPERIMENTAL) in
> the Kconfig menu?
> 
> It would make it clearer that by enabling UNSUPPORTED you are going to
> get a configuration that is not security supported.

If going down this path, there should be one, authoritative, in-tree definition 
of feature-level support from which Kconfig (build-time policy enforcement) and 
SUPPORT.md (documentation) can be derived.  Later, even run-time enforcement 
can be similarly classified.  FuSA may also wish for documented policy to align 
with enforcement.

Rich

> And ideally we would
> also tag features like ACPI as UNSUPPORTED as I suggested above.
> 

Re: [Xen-devel] [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset with 'reset' SysFS attribute

[Rich Persaud]

On Oct 19, 2020, at 11:52, Håkon Alstadheim  wrote:
> 
> 
> Den 19.10.2020 17:16, skrev Håkon Alstadheim:
>> Den 19.10.2020 13:00, skrev George Dunlap:
>>> 
>>>> On Jan 31, 2020, at 3:33 PM, Wei Liu  wrote:
>>>> 
>>>> On Fri, Jan 17, 2020 at 02:13:04PM -0500, Rich Persaud wrote:
>>>>> On Aug 26, 2019, at 17:08, Pasi Kärkkäinen  wrote:
>>>>>> Hi,
>>>>>> 
>>>>>>> On Mon, Oct 08, 2018 at 10:32:45AM -0400, Boris Ostrovsky wrote:
>>>>>>>> On 10/3/18 11:51 AM, Pasi Kärkkäinen wrote:
>>>>>>>> On Wed, Sep 19, 2018 at 11:05:26AM +0200, Roger Pau Monné wrote:
>>>>>>>>> On Tue, Sep 18, 2018 at 02:09:53PM -0400, Boris Ostrovsky wrote:
>>>>>>>>>> On 9/18/18 5:32 AM, George Dunlap wrote:
>>>>>>>>>>>> On Sep 18, 2018, at 8:15 AM, Pasi Kärkkäinen  wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> On Mon, Sep 17, 2018 at 02:06:02PM -0400, Boris Ostrovsky wrote:
>>>>>>>>>>>>>> What about the toolstack changes? Have they been accepted? I 
>>>>>>>>>>>>>> vaguely
>>>>>>>>>>>>>> recall there was a discussion about those changes but don't 
>>>>>>>>>>>>>> remember how
>>>>>>>>>>>>>> it ended.
>>>>>>>>>>>>> I don't think toolstack/libxl patch has been applied yet either.
>>>>>>>>>>>>> "[PATCH V1 0/1] Xen/Tools: PCI reset using 'reset' SysFS 
>>>>>>>>>>>>> attribute":
>>>>>>>>>>>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00664.html
>>>>>>>>>>>>>  
>>>>>>>>>>>>> "[PATCH V1 1/1] Xen/libxl: Perform PCI reset using 'reset' SysFS 
>>>>>>>>>>>>> attribute":
>>>>>>>>>>>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00663.html
>>>>>>>>>>>>>  
>>>>>>>>>>> Will this patch work for *BSD? Roger?
>>>>>>>>>> At least FreeBSD don't support pci-passthrough, so none of this works
>>>>>>>>>> ATM. There's no sysfs on BSD, so much of what's in libxl_pci.c will
>>>>>>>>>> have to be moved to libxl_linux.c when BSD support is added.
>>>>>>>>> Ok. That sounds like it's OK for the initial pci 'reset' 
>>>>>>>>> implementation in xl/libxl to be linux-only..
>>>>>>>> Are these two patches still needed? ISTR they were written originally
>>>>>>>> to deal with guest trying to use device that was previously assigned to
>>>>>>>> another guest. But pcistub_put_pci_dev() calls
>>>>>>>> __pci_reset_function_locked() which first tries FLR, and it looks like
>>>>>>>> it was added relatively recently.
>>>>>>> Replying to an old thread.. I only now realized I forgot to reply to 
>>>>>>> this message earlier.
>>>>>>> 
>>>>>>> afaik these patches are still needed. Håkon (CC'd) wrote to me in 
>>>>>>> private that
>>>>>>> he gets a (dom0) Linux kernel crash if he doesn't have these patches 
>>>>>>> applied.
>>>>>>> 
>>>>>>> 
>>>>>>> Here are the links to both the linux kernel and libxl patches:
>>>>>>> 
>>>>>>> 
>>>>>>> "[Xen-devel] [PATCH V3 0/2] Xen/PCIback: PCI reset using 'reset' SysFS 
>>>>>>> attribute":
>>>>>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00659.html
>>>>>>> 
>>>>>>> [Note that PATCH V3 1/2 "Drivers/PCI: Export pcie_has_flr() interface" 
>>>>>>> is already applied in upstream linux kernel, so it's not needed anymore]
>>>>>>> 
>>>>>>> "[Xen-devel] [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus 
>>>>>>> reset with 'reset' SysFS attribute":
>>>>>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00661.html
>>

Re: Xen 4.14.0 fails on Dell IoT Gateway without efi=no-rs

[Rich Persaud]

On Aug 20, 2020, at 07:24, George Dunlap  wrote:
> 
>> On Thu, Aug 20, 2020 at 9:35 AM Jan Beulich  wrote:
> 
>> 
>> As far as making cases like this work by default, I'm afraid it'll
>> need to be proposed to replace me as the maintainer of EFI code in
>> Xen. I will remain on the position that it is not acceptable to
>> apply workarounds for firmware issues by default unless they're
>> entirely benign to spec-conforming systems. DMI data based enabling
>> of workarounds, for example, is acceptable in the common case, as
>> long as the matching pattern isn't unreasonably wide.
> 
> 
> It sort of sounds like it would be useful to have a wider discussion on this 
> then, to hash out what exactly it is we want to do as a project.
> 
>  -George

Sometimes a middle ground is possible, e.g. see this Nov 2019 thread about a 
possible Xen Kconfig option for EFI_NONSPEC_COMPATIBILITY, targeting 
Edge/IoT/laptop hardware:

https://lists.archive.carbon60.com/xen/devel/571670#571670

In the years to come, edge devices will only grow in numbers.  Some will be 
supported in production for more than a decade, which will require new 
long-term commercial support mechanisms for device BIOS, rather than firmware 
engineers shifting focus after a device is launched. 

In parallel to (opt-in) Xen workarounds for a constrained and documented set of 
firmware issues, we need more industry efforts to support open firmware, like 
coreboot and OCP Open System Firmware with minimum binary blobs.  At least one 
major x86 OEM is expected to ship open firmware in one of their popular 
devices, which may encourage competing OEM devices to follow.

PC Engines APU2 (dual-core AMD, 4GB RAM, 6W TDP, triple NIC + LTE) is one 
available edge device which supports Xen and has open (coreboot) firmware.  It 
would be nice to include APU2 in LF Edge support, if only to provide 
competition to OEM devices with buggy firmware. Upcoming Intel Tiger Lake 
(Core) and Elkhart Lake (Atom Tremont) are expected to expand edge-relevant 
security features, which would make such devices attractive to Xen deployments. 
 

We also need edge software vendors to encourage device OEMs to enable open 
firmware via coreboot, OCP OSF, Intel MinPlatform and similar programs. See 
https://software.intel.com/content/www/us/en/develop/articles/minimum-platform-architecture-open-source-uefi-firmware-for-intel-based-platforms.html
 and other talks from the open firmware conference, https://osfc.io/archive

Rich

Re: Porting Xen to Jetson Nano

[Rich Persaud]

On Jul 28, 2020, at 13:19, Srinivas Bangalore  wrote:
> 
> 
>> 
>> I struggled to find your comment inline as your e-mail client doesn't 
>> quote my answer. Please configure your e-mail client to use some form 
>> of quoting (the usual is '>').
>> 
>> [] Done! Sorry about that.
> 
> Thanks this is a good start. Unfortunately, it doesn't fully help it when you 
> have a reply split accross multiple line. This is become more proeminent 
> after a few back and forth. Which e-mail client are you using?
> 
> [] I'm using Microsoft Outlook

Might be worth trying these instructions:
https://www.slipstick.com/outlook/email/to-use-internet-style-quoting/

Rich

Re: Saved notes for design sessions

[Rich Persaud]

> On Jul 16, 2020, at 07:55, George Dunlap  wrote:
> Hey all,
> 
> PDFs of the saved shared notes for the design sessions can be found in this 
> zipfile:
> 
> https://cryptpad.fr/file/#/2/file/LoJZpSq+vHKNoisVqdsPj3Z9/
> 
> The files are labeled with the start/end time and the room in which they were 
> held; that should help you find out the notes which are pertinent to you.
> 
> Please remember to post a summary of your design session to xen-devel for 
> posterity.
> 
> Thanks!
> 
> -George

After 2020 design session notes are published on xen-devel, they can be indexed 
on a wiki page similar to the 2019 index: 

  https://wiki.xenproject.org/wiki/Design_Sessions_2019

Rich

Re: [XEN RFC for-4.14] Re: use of "stat -"

[Rich Persaud]

On Jun 24, 2020, at 22:39, Jason Andryuk  wrote:
> 
> On Wed, Jun 24, 2020 at 12:19 PM Ian Jackson  wrote:
>> 
>> Jan Beulich writes ("Re: use of "stat -""):
>>> [CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments 
>>> unless you have verified the sender and know the content is safe.
 On 14.05.2020 13:02, Ian Jackson wrote:
> I've read this thread.  Jan, I'm sorry that this causes you
> inconvenience.  I'm hoping it won't come down to a choice between
> supporting people who want to ship a dom0 without perl, and people who
> want a dom0 using more-than-a-decade-old coreutils.
> 
> Jan, can you tell me what the output is of this on your ancient
> system:
> 
>  $ rm -f t
>  $ >t
>  $ exec 3  $ stat -L -c '%F %i' /dev/stdin <&3
>  regular empty file 393549
>  $ rm t
>  $ stat -L -c '%F %i' /dev/stdin <&3
>  regular empty file 393549
>  $ strace -ou stat -L -c '%F %i' /dev/stdin <&3
>  $
>>> 
>>> $ rm -f t
>>> $ >t
>>> $ exec 3>> $ stat -L -c '%F %i' /dev/stdin <&3
>>> regular empty file 3380369
>>> $ rm t
>>> $ stat -L -c '%F %i' /dev/stdin <&3
>>> regular empty file 3380369
>>> $ strace -ou stat -L -c '%F %i' /dev/stdin <&3
>>> regular empty file 3380369
>>> 
 Also, the contents of the file "u" afterwards, please.
>>> 
>>> Attached.
>> 
>> Thanks.
>> 
>> I think this means that `stat -' can be replaced by `stat /dev/stdin'.
>> 
>> This script is only run on Linux where /dev/stdin has existed
>> basically forever.  The strace output shows
>>  stat("/dev/stdin", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
>> and the transcript shows that your stat(1) behaves as I hope.
>> 
>> Jan, will you send a patch ?  It is best if someone else but me writes
>> it and tests it because then I am a "clean" reviewer.
>> 
>> Paul, supposing I review such a patch and say it is low risk, and we
>> commit it by Friday, can it have a release-ack ?
> 
> I was going to just write a patch to replace - with /dev/stdin and ask
> Jan to test it.  When I opened the script, this comment was staring at
> me:
># We can't just stat /dev/stdin or /proc/self/fd/$_lockfd or
># use bash's test -ef because those all go through what is
># actually a synthetic symlink in /proc and we aren't
># guaranteed that our stat(2) won't lose the race with an
># rm(1) between reading the synthetic link and traversing the
># file system to find the inum.
> 
> On my system:
> $ ls -l /dev/stdin
> lrwxrwxrwx 1 root root 15 Jun 24 21:13 /dev/stdin -> /proc/self/fd/0
> $ ls -l /proc/self/fd/0 0 lrwx-- 1 jason jason 64 Jun 24 21:26 /proc/self/fd/0 -> 
> /home/jason/lockfile
> 
> stat /dev/stdin will work around the lack of `stat -` support, but it
> doesn't address the race in the comment.  Is the comment valid?  How
> would we prove there is no race for /dev/stdin?  And as a refresher,
> `stat -` does an fstat(0), so there is no symlink lookup.  Or is there
> no race since `stat /proc/self/fd/0` isn't a symlink lookup but just
> accessing the already open fd via the proc special file? i.e.
> equivalent to fstat.
> 
> I've mentioned it before, but maybe we should just use the Qubes
> patch.  It leaves the lockfile even when no-one is holding the lock,
> but it simplifies the code and sidesteps the issues we are discussing
> here.
> https://github.com/QubesOS/qubes-vmm-xen/blob/xen-4.13/patch-tools-hotplug-drop-perl-usage-in-locking-mechanism.patch

Is there any practical downside to the Qubes patch, which is already deployed 
on production systems?  The complexity of other approaches has been reflected 
in code and prior discussion.  If needed, the comments in the Qubes patch could 
be expanded, when merging that well-tested code into 4.14.

Rich

Re: [PATCH v1] tools: fix usage of strncpy

[Rich Persaud]

On Jun 8, 2020, at 10:12, Olaf Hering  wrote:
> 
> Am Mon, 8 Jun 2020 08:43:50 -0400
> schrieb Jason Andryuk :
> 
>> I added a length check in this patch:
> 
> gcc will not recognize such runtime checks and will (most likely) complain 
> about the strncpy usage anyway, just as it does now in 
> libxl__prepare_sockaddr_un. While the usage in libxl__prepare_sockaddr_un is 
> fatal due to -Werror, libvchan is apparently built without -Werror.
> 
> Olaf

Is there any reason not to take a patch that builds libvchan with -Werror?

Rich

Re: XenSummit 2020 *will* be held virtuaully in June!

[Rich Persaud]


> 
> On May 7, 2020, at 18:50, George Dunlap  wrote:
> We’re still ironing out all the details, but it’s absolutely confirmed that 
> XenSummit 2020 will be held virtually in June.
> 
> In addition, the new version of the Design Sessions website is now live:
> 
> https://design-sessions.xenproject.org
> 
> Make space on your calendars, and submit your design sessions, and watch this 
> space for further information!

Thanks for the update.  Might be worth adding the above note and expected 
pricing to one of the public status pages and Twitter, while the new schedule 
is being worked out:

https://events.linuxfoundation.org/xen-summit/attend/novel-coronavirus-update/
https://xenproject.org/2020/03/26/xenproject-developer-and-design-summit-update-in-light-of-covid-19/
https://events.linuxfoundation.org/xen-summit/

Rich

Re: XenSummit 2020 *will* be held virtuaully in June!

[Rich Persaud]

On May 7, 2020, at 18:50, George Dunlap  wrote:
> 
> We’re still ironing out all the details, but it’s absolutely confirmed that 
> XenSummit 2020 will be held virtually in June.
> 
> In addition, the new version of the Design Sessions website is now live:
> 
> https://design-sessions.xenproject.org
> 
> Make space on your calendars, and submit your design sessions, and watch this 
> space for further information!

Thanks for the update.  Might be worth adding the above note and expected 
pricing to one of the public status pages and Twitter, while the new schedule 
is being worked out:

https://events.linuxfoundation.org/xen-summit/attend/novel-coronavirus-update/
https://xenproject.org/2020/03/26/xenproject-developer-and-design-summit-update-in-light-of-covid-19/
https://events.linuxfoundation.org/xen-summit/

Rich

Re: [Xen-devel] Moving Forward on XenSummit

[Rich Persaud]

On Mar 24, 2020, at 14:03, George Dunlap  wrote:
> 
> I wanted to let everyone know that the XenProject is moving forward with 
> plans to hold XenSummit this year, one way or another.
> 
> There are two basic approaches the Advisory Board has been considering:  
> Postponing the even until later in the year, or holding a virtual event 
> during the same timeframe.  Additionally, if we hold a virtual event during 
> the same timeframe, the Board wants to keep the option open of having a 
> smaller, in-person event later in the year, if circumstances permit.

Due to variation in scope/timing of geo and company restrictions on travel, 
could some speakers present remotely for the in-person event?  

Could the Xen Summit CFP be re-opened for those who can present virtually, who 
may not have submitted due to travel restrictions?

Rich

Re: [Xen-devel] [PATCH 1/2] tools/helpers: Introduce cmp-fd-file-inode utility

[Rich Persaud]

On Feb 26, 2020, at 11:07, Jason Andryuk  wrote:
> 
>> On Wed, Feb 26, 2020 at 10:48 AM Ian Jackson  wrote:
>> Jason Andryuk writes ("[PATCH 1/2] tools/helpers: Introduce 
>> cmp-fd-file-inode utility"):
>>> This is a C implementation of the perl code inside of locking.sh to
>>> check that the locked file descriptor and lock file share the same inode
>>> and therefore match.  One change from the perl version is replacing
>>> printing "y" on success with exit values of 0 (shell True) and 1 (shell
>>> False).
>> Maybe it would be better to use stat(1) ?  On Linux
>> stat -L -c%D.%i /dev/stdin blah.lock
>> or some such, and then compare the two numbers.
>> I'm reluctant to host a general-purpose shell utility in xen.git, no
>> matter how useful...

We've used this downstream in OpenXT to avoid pulling in the general-purpose 
Perl runtime, which is..larger than this special-purpose utility.  This shell 
utility was developed for this specific use case in Xen.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 0/9] Add hypervisor sysfs-like support

[Rich Persaud]

On Jan 21, 2020, at 03:45, Juergen Gross  wrote:
> 
> On the 2019 Xen developer summit there was agreement that the Xen
> hypervisor should gain support for a hierarchical name-value store
> similar to the Linux kernel's sysfs.

Is there a short summary of the most recent use cases for this feature and 
expected interactions with other Xen features (e.g. Panopticon Xen, security 
controls on information that is visible to guests, e.g. recent discussion on 
version number hiding). This would impact many subsystems.

Presumably Kconfig could enable/disable this optional feature and all 
dependencies, and the Xen toolstack would continue to function normally in its 
absence.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 12/16] libxl: use vchan for QMP access with Linux stubdomain

[Rich Persaud]

On Jan 24, 2020, at 09:07, Jason Andryuk  wrote:
> 
> On Tue, Jan 21, 2020 at 6:46 PM Marek Marczykowski-Górecki
>  wrote:
> 
 +
 +sdss->qmp_proxy_spawn.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 
 1000;
 +sdss->qmp_proxy_spawn.midproc_cb = libxl__spawn_record_pid;
 +sdss->qmp_proxy_spawn.confirm_cb = qmp_proxy_confirm;
 +sdss->qmp_proxy_spawn.failure_cb = qmp_proxy_startup_failed;
 +sdss->qmp_proxy_spawn.detached_cb = qmp_proxy_detached;
 +
 +const int arraysize = 6;
 +GCNEW_ARRAY(args, arraysize);
 +args[nr++] = STUBDOM_QMP_PROXY_PATH;
 +args[nr++] = GCSPRINTF("--state-path=%s", 
 sdss->qmp_proxy_spawn.xspath);
 +args[nr++] = GCSPRINTF("%u", dm_domid);
 +args[nr++] = GCSPRINTF("%s/device-model/%u/qmp-vchan", dom_path, 
 guest_domid);
>>> Thinking of OpenXT"s qmp-helper, this path isn't useful.  But it is
>>> for vchan-socket-proxy, so qmp-helper could just change to ignore it.
>> For vchan we could use also a port number (and then it will encode it
>> into a xenstore path). This is in fact how we use libvchan in Qubes. I
>> opted for explicit path only because of lack of idea for some meaningful
>> port number ;) But I'm open for suggestions.
>> I guess that would be useful for Argo version then.
> 
> The argo version hard codes the port number, so it's not a command
> line argument.  The port number would need to get passed to the
> stubdom or it would need to be standardized.
> 
> I think the arguments for vchan-socket-proxy make sense.  Since it's
> the one that's submitted upstream, it makes sense to use them.
> 
> Put another way, do we want this to support alternate implementations
> for a qmp proxy?  Should the arguments be generic for that case?


One advantage of the server+client approach of vchan-socket-proxy is the 
absence of patches for Qemu.  OpenXT qmp-helper requires a Qemu patch for Argo 
support.  If there was a qmp socket proxy with Argo support, unpatched Qemu 
could be used with libxl and Argo access controls.

A generalized qmp-socket-proxy may be useful to other projects.  It would be 
good if the design supported single-purpose (client or server) binaries, e.g. 
common functions in a library shared by separate client and server source 
files, with conditional compilation for vchan and Argo interfaces.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 13/16] Regenerate autotools files

[Rich Persaud]

On Jan 21, 2020, at 15:58, Marek Marczykowski-Górecki 
 wrote:
> 
> On Wed, Jan 15, 2020 at 04:57:29PM -0500, Rich Persaud wrote:
>>>> On Jan 14, 2020, at 21:42, Marek Marczykowski-Górecki 
>>>>  wrote:
>>> Since we have those generated files committed to the repo (why?!),
>>> update them after changing configure.ac.
>> 
>> Is there any reason not to remove the generated configure files?  A 
>> developer using generated files on system B would be incorporating 
>> configuration assumptions from system A where the configure script was 
>> generated.  If we are going to ship configure scripts, do we need to 
>> document a "system A" reference distro/environment where all configure 
>> scripts from Xen will be generated?
>> 
>> 
>> Other notes:
>> 
>> 1.  Debian autoreconf works in the Xen root directory, but the default 
>> OpenEmbedded autoreconf uses Gnu libtoolize and fails because some Xen build 
>> subdirectories don't have configure.ac/.in.   
>> 
>> 2.  If OpenEmbedded autoreconf is run only in the tools directory (where it 
>> works and generates a new tools configure), then root configure (generated 
>> from older configure.ac) will silently ignore the newer tools configure and 
>> write config.h _without_ tools-specific config, such as the vchan QMP proxy.
>> 
>> 3. If autoreconf runs successfully in the root directory, then 
>> tools-specific configure is correctly generated and everything works as 
>> expected.
>> 
>> This silent failure could be avoided by deleting the generated configure 
>> scripts.  There may be other failure modes for using System A generated 
>> scripts on downstream build system B.
> 
> Yes, I think general good practices are:
> 1. don't keep generated autotools files in version control system
> 2. generate them into release tarballs

A potential topic for the next Xen community call:  can we delete generated 
autotools files from the Xen tree and update the release process to 
generate+bundle them with release tarballs?

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset with 'reset' SysFS attribute

[Rich Persaud]

On Aug 26, 2019, at 17:08, Pasi Kärkkäinen  wrote:
> Hi,
> 
>> On Mon, Oct 08, 2018 at 10:32:45AM -0400, Boris Ostrovsky wrote:
>>> On 10/3/18 11:51 AM, Pasi Kärkkäinen wrote:
>>> On Wed, Sep 19, 2018 at 11:05:26AM +0200, Roger Pau Monné wrote:
 On Tue, Sep 18, 2018 at 02:09:53PM -0400, Boris Ostrovsky wrote:
> On 9/18/18 5:32 AM, George Dunlap wrote:
>>> On Sep 18, 2018, at 8:15 AM, Pasi Kärkkäinen  wrote:
>>> Hi,
>>> On Mon, Sep 17, 2018 at 02:06:02PM -0400, Boris Ostrovsky wrote:
 What about the toolstack changes? Have they been accepted? I vaguely
 recall there was a discussion about those changes but don't remember 
 how
 it ended.
>>> I don't think toolstack/libxl patch has been applied yet either.
>>> "[PATCH V1 0/1] Xen/Tools: PCI reset using 'reset' SysFS attribute":
>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00664.html
>>> "[PATCH V1 1/1] Xen/libxl: Perform PCI reset using 'reset' SysFS 
>>> attribute":
>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00663.html
> Will this patch work for *BSD? Roger?
 At least FreeBSD don't support pci-passthrough, so none of this works
 ATM. There's no sysfs on BSD, so much of what's in libxl_pci.c will
 have to be moved to libxl_linux.c when BSD support is added.
>>> Ok. That sounds like it's OK for the initial pci 'reset' implementation in 
>>> xl/libxl to be linux-only..
>> 
>> Are these two patches still needed? ISTR they were  written originally
>> to deal with guest trying to use device that was previously assigned to
>> another guest. But pcistub_put_pci_dev() calls
>> __pci_reset_function_locked() which first tries FLR, and it looks like
>> it was added relatively recently.
> 
> Replying to an old thread.. I only now realized I forgot to reply to this 
> message earlier.
> 
> afaik these patches are still needed. Håkon (CC'd) wrote to me in private that
> he gets a (dom0) Linux kernel crash if he doesn't have these patches applied.
> 
> 
> Here are the links to both the linux kernel and libxl patches:
> 
> 
> "[Xen-devel] [PATCH V3 0/2] Xen/PCIback: PCI reset using 'reset' SysFS 
> attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00659.html
> 
> [Note that PATCH V3 1/2 "Drivers/PCI: Export pcie_has_flr() interface" is 
> already applied in upstream linux kernel, so it's not needed anymore]
> 
> "[Xen-devel] [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset 
> with 'reset' SysFS attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00661.html
> 
> 
> "[Xen-devel] [PATCH V1 0/1] Xen/Tools: PCI reset using 'reset' SysFS 
> attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00664.html
> 
> "[Xen-devel] [PATCH V1 1/1] Xen/libxl: Perform PCI reset using 'reset' SysFS 
> attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00663.html

[dropping Linux mailing lists]

What is required to get the Xen patches merged?  Rebasing against Xen master?  
OpenXT has been carrying a similar patch for many years and we would like to 
move to an upstream implementation.  Xen users of PCI passthrough would benefit 
from more reliable device reset.

  2017 thread, including OpenXT patch: https://lists.gt.net/xen/devel/492945
  2017-2019 thread: https://lists.gt.net/xen/devel/532648

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen: xen-pciback: Reset MSI-X state when exposing a device

[Rich Persaud]

On Sep 26, 2019, at 06:17, Pasi Kärkkäinen  wrote:
> 
> Hello Stanislav,
> 
>> On Fri, Sep 13, 2019 at 11:28:20PM +0800, Chao Gao wrote:
>>> On Fri, Sep 13, 2019 at 10:02:24AM +, Spassov, Stanislav wrote:
>>> On Thu, Dec 13, 2018 at 07:54, Chao Gao wrote:
 On Thu, Dec 13, 2018 at 12:54:52AM -0700, Jan Beulich wrote:
 On 13.12.18 at 04:46,  wrote:
>> On Wed, Dec 12, 2018 at 08:21:39AM -0700, Jan Beulich wrote:
>> On 12.12.18 at 16:18,  wrote:
 On Wed, Dec 12, 2018 at 01:51:01AM -0700, Jan Beulich wrote:
 On 12.12.18 at 08:06,  wrote:
>> On Wed, Dec 05, 2018 at 09:01:33AM -0500, Boris Ostrovsky wrote:
>>> On 12/5/18 4:32 AM, Roger Pau Monné wrote:
 On Wed, Dec 05, 2018 at 10:19:17AM +0800, Chao Gao wrote:
> I find some pass-thru devices don't work any more across guest 
> reboot.
> Assigning it to another guest also meets the same issue. And the 
> only
> way to make it work again is un-binding and binding it to pciback.
> Someone reported this issue one year ago [1]. More detail also 
> can be
> found in [2].
> 
> The root-cause is Xen's internal MSI-X state isn't reset properly
> during reboot or re-assignment. In the above case, Xen set 
> maskall bit
> to mask all MSI interrupts after it detected a potential security
> issue. Even after device reset, Xen didn't reset its internal 
> maskall
> bit. As a result, maskall bit would be set again in next write to
> MSI-X message control register.
> 
> Given that PHYSDEVOPS_prepare_msix() also triggers Xen resetting 
> MSI-X
> internal state of a device, we employ it to fix this issue rather 
> than
> introducing another dedicated sub-hypercall.
> 
> Note that PHYSDEVOPS_release_msix() will fail if the mapping 
> between
> the device's msix and pirq has been created. This limitation 
> prevents
> us calling this function when detaching a device from a guest 
> during
> guest shutdown. Thus it is called right before calling
> PHYSDEVOPS_prepare_msix().
 s/PHYSDEVOPS/PHYSDEVOP/ (no final S). And then I would also drop 
 the
 () at the end of the hypercall name since it's not a function.
 
 I'm also wondering why the release can't be done when the device is
 detached from the guest (or the guest has been shut down). This 
 makes
 me worry about the raciness of the attach/detach procedure: if 
 there's
 a state where pciback assumes the device has been detached from the
 guest, but there are still pirqs bound, an attempt to attach to
 another guest in such state will fail.
>>> 
>>> I wonder whether this additional reset functionality could be done 
>>> out
>>> of xen_pcibk_xenbus_remove(). We first do a (best effort) device 
>>> reset
>>> and then do the extra things that are not properly done there.
>> 
>> No. It cannot be done in xen_pcibk_xenbus_remove() without modifying
>> the handler of PHYSDEVOP_release_msix. To do a successful Xen 
>> internal
>> MSI-X state reset, PHYSDEVOP_{release, prepare}_msix should be 
>> finished
>> without error. But ATM, xen expects that no msi is bound to pirq when
>> doing PHYSDEVOP_release_msix. Otherwise it fails with error code 
>> -EBUSY.
>> However, the expectation isn't guaranteed in 
>> xen_pcibk_xenbus_remove().
>> In some cases, if qemu fails to unmap MSIs, MSIs are unmapped by Xen
>> at last minute, which happens after device reset in 
>> xen_pcibk_xenbus_remove().
> 
> But that may need taking care of: I don't think it is a good idea to 
> have
> anything left from the prior owning domain when the device gets reset.
> I.e. left over IRQ bindings should perhaps be forcibly cleared before
> invoking the reset;
 
 Agree. How about pciback to track the established IRQ bindings? Then
 pciback can clear irq binding before invoking the reset.
>>> 
>>> How would pciback even know of those mappings, when it's qemu
>>> who establishes (and manages) them?
>> 
>> I meant to expose some interfaces from pciback. And pciback serves
>> as the proxy of IRQ (un)binding APIs.
> 
> If at all possible we should avoid having to change more parties (qemu,
> libxc, kernel, hypervisor) than really necessary. Remember that such
> a bug fix may want backporting, and making sure 

Re: [Xen-devel] [PATCH v4 11/16] tools: add simple vchan-socket-proxy

[Rich Persaud]

On Jan 14, 2020, at 21:42, Marek Marczykowski-Górecki 
 wrote:
> 
> diff --git a/tools/libvchan/Makefile b/tools/libvchan/Makefile
> index 7892750..1c845ca 100644
> --- a/tools/libvchan/Makefile
> +++ b/tools/libvchan/Makefile
> @@ -13,6 +13,7 @@ LIBVCHAN_PIC_OBJS = $(patsubst %.o,%.opic,$(LIBVCHAN_OBJS))
> LIBVCHAN_LIBS = $(LDLIBS_libxenstore) $(LDLIBS_libxengnttab) 
> $(LDLIBS_libxenevtchn)
> $(LIBVCHAN_OBJS) $(LIBVCHAN_PIC_OBJS): CFLAGS += $(CFLAGS_libxenstore) 
> $(CFLAGS_libxengnttab) $(CFLAGS_libxenevtchn)
> $(NODE_OBJS) $(NODE2_OBJS): CFLAGS += $(CFLAGS_libxengnttab) 
> $(CFLAGS_libxenevtchn)
> +vchan-socket-proxy.o: CFLAGS += $(CFLAGS_libxenstore) $(CFLAGS_libxenctrl) 
> $(CFLAGS_libxengnttab) $(CFLAGS_libxenevtchn)
> 
> MAJOR = 4.14
> MINOR = 0
> @@ -39,7 +40,7 @@ $(PKG_CONFIG_LOCAL): PKG_CONFIG_LIBDIR = $(CURDIR)
> $(PKG_CONFIG_LOCAL): PKG_CONFIG_CFLAGS_LOCAL = $(CFLAGS_xeninclude)
> 
> .PHONY: all
> -all: libxenvchan.so vchan-node1 vchan-node2 libxenvchan.a $(PKG_CONFIG_INST) 
> $(PKG_CONFIG_LOCAL)
> +all: libxenvchan.so vchan-node1 vchan-node2 vchan-socket-proxy libxenvchan.a 
> $(PKG_CONFIG_INST) $(PKG_CONFIG_LOCAL)
> 
> libxenvchan.so: libxenvchan.so.$(MAJOR)
>ln -sf $< $@
> @@ -59,6 +60,9 @@ vchan-node1: $(NODE_OBJS) libxenvchan.so
> vchan-node2: $(NODE2_OBJS) libxenvchan.so
>$(CC) $(LDFLAGS) -o $@ $(NODE2_OBJS) $(LDLIBS_libxenvchan) 
> $(APPEND_LDFLAGS)
> 
> +vchan-socket-proxy: vchan-socket-proxy.o libxenvchan.so
> +$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenvchan) $(LDLIBS_libxenstore) 
> $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
> +
> .PHONY: install
> install: all
>$(INSTALL_DIR) $(DESTDIR)$(libdir)
> @@ -66,6 +70,7 @@ install: all
>$(INSTALL_PROG) libxenvchan.so.$(MAJOR).$(MINOR) $(DESTDIR)$(libdir)
>ln -sf libxenvchan.so.$(MAJOR).$(MINOR) 
> $(DESTDIR)$(libdir)/libxenvchan.so.$(MAJOR)
>ln -sf libxenvchan.so.$(MAJOR) $(DESTDIR)$(libdir)/libxenvchan.so
> +$(INSTALL_PROG) vchan-socket-proxy $(DESTDIR)$(bindir)

Does this need directory creation, to avoid vchan binary being named "bin"?
+   $(INSTALL_DIR) $(DESTDIR)$(bindir)


> diff --git a/tools/libvchan/vchan-socket-proxy.c 
> b/tools/libvchan/vchan-socket-proxy.c
> new file mode 100644
> index 000..6b4ae09
> --- /dev/null
> +++ b/tools/libvchan/vchan-socket-proxy.c
> @@ -0,0 +1,469 @@
> +/**
> + * @file
> + * @section AUTHORS
> + *
> + * Copyright (C) 2010  Rafal Wojtczuk  
> + *
> + *  Authors:
> + *   Rafal Wojtczuk  
> + *   Daniel De Graaf 
> + *   Marek Marczykowski-Górecki  
> + *
> + * @section LICENSE
> + *
> + *  This program is free software; you can redistribute it and/or
> + *  modify it under the terms of the GNU Lesser General Public
> + *  License as published by the Free Software Foundation; either
> + *  version 2.1 of the License, or (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + *  Lesser General Public License for more details.
> + *
> + *  You should have received a copy of the GNU Lesser General Public
> + *  License along with this program; If not, see 
> .
> + *
> + * @section DESCRIPTION
> + *
> + * This is a vchan to unix socket proxy. Vchan server is set, and on client
> + * connection, local socket connection is established. Communication is 
> bidirectional.
> + * One client is served at a time, clients needs to coordinate this 
> themselves.
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +#include 
> +#include 
> +#include 
> +
> +static void usage(char** argv)
> +{
> +fprintf(stderr, "usage:\n"
> +"\t%s [options] domainid nodepath [socket-path|file-no|-]\n"
> +"\n"
> +"options:\n"
> +"\t-m, --mode=client|server - vchan connection mode\n"
> +"\t-m, --state-path=path - xenstore path where write \"running\" to 
> at startup\n"
> +"\t-v, --verbose - verbose logging\n"
> +"\n"
> +"client: client of a vchan connection, fourth parameter can be:\n"
> +"\tsocket-path: listen on a UNIX socket at this path and connect to 
> vchan\n"
> +"\t  whenever new connection is accepted;\n"
> +"\t  handle multiple _subsequent_ connections, until terminated\n"
> +"\tfile-no: except open FD of a socket in listen mode; otherwise 
> similar to socket-path\n"
> +"\t-: open vchan connection immediately and pass the data from 
> stdin/stdout;\n"
> +"\t  terminate when vchan connection is closed\n"
> +"server: server of a vchan connection, fourth parameter can be:\n"
> +"\tsocket-path: connect to this UNIX socket when new vchan 
> connection is accepted\n"
> +"\t  handle multiple _subsequent_ connections, until terminated\n"

Re: [Xen-devel] [PATCH v4 13/16] Regenerate autotools files

[Rich Persaud]

> On Jan 14, 2020, at 21:42, Marek Marczykowski-Górecki 
>  wrote:
> Since we have those generated files committed to the repo (why?!),
> update them after changing configure.ac.

Is there any reason not to remove the generated configure files?  A developer 
using generated files on system B would be incorporating configuration 
assumptions from system A where the configure script was generated.  If we are 
going to ship configure scripts, do we need to document a "system A" reference 
distro/environment where all configure scripts from Xen will be generated?


Other notes:

1.  Debian autoreconf works in the Xen root directory, but the default 
OpenEmbedded autoreconf uses Gnu libtoolize and fails because some Xen build 
subdirectories don't have configure.ac/.in.   

2.  If OpenEmbedded autoreconf is run only in the tools directory (where it 
works and generates a new tools configure), then root configure (generated from 
older configure.ac) will silently ignore the newer tools configure and write 
config.h _without_ tools-specific config, such as the vchan QMP proxy.

3. If autoreconf runs successfully in the root directory, then tools-specific 
configure is correctly generated and everything works as expected.

This silent failure could be avoided by deleting the generated configure 
scripts.  There may be other failure modes for using System A generated scripts 
on downstream build system B.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH net-next] xen-netback: get rid of old udev related code

[Rich Persaud]

> On Dec 16, 2019, at 03:31, Jürgen Groß  wrote:
> 
> On 16.12.19 09:18, Durrant, Paul wrote:
>>> -Original Message-
>>> From: Jürgen Groß 
>>> Sent: 16 December 2019 08:10
>>> To: Durrant, Paul ; David Miller
>>> 
>>> Cc: xen-devel@lists.xenproject.org; wei@kernel.org; linux-
>>> ker...@vger.kernel.org; net...@vger.kernel.org
>>> Subject: Re: [Xen-devel] [PATCH net-next] xen-netback: get rid of old udev
>>> related code
 On 13.12.19 11:12, Durrant, Paul wrote:
>> -Original Message-
>> From: Jürgen Groß 
>> Sent: 13 December 2019 10:02
>> To: Durrant, Paul ; David Miller
>> 
>> Cc: xen-devel@lists.xenproject.org; wei@kernel.org; linux-
>> ker...@vger.kernel.org; net...@vger.kernel.org
>> Subject: Re: [Xen-devel] [PATCH net-next] xen-netback: get rid of old
>>> udev
> related code
> On 13.12.19 10:24, Durrant, Paul wrote:
>>> -Original Message-
>>> From: Jürgen Groß 
>>> Sent: 13 December 2019 05:41
>>> To: David Miller ; Durrant, Paul
>>> 
>>> Cc: xen-devel@lists.xenproject.org; wei@kernel.org; linux-
>>> ker...@vger.kernel.org; net...@vger.kernel.org
>>> Subject: Re: [Xen-devel] [PATCH net-next] xen-netback: get rid of old
> udev
>>> related code
 On 12.12.19 20:05, David Miller wrote:
> From: Paul Durrant 
> Date: Thu, 12 Dec 2019 13:54:06 +
>> In the past it used to be the case that the Xen toolstack relied
 upon
>> udev to execute backend hotplug scripts. However this has not been
>> the
>> case for many releases now and removal of the associated code in
>> xen-netback shortens the source by more than 100 lines, and removes
 much
>> complexity in the interaction with the xenstore backend state.
>> NOTE: xen-netback is the only xenbus driver to have a functional
 uevent()
>> method. The only other driver to have a method at all is
>> pvcalls-back, and currently pvcalls_back_uevent() simply
>> returns
 0.
>> Hence this patch also facilitates further cleanup.
>> Signed-off-by: Paul Durrant 
> If userspace ever used this stuff, I seriously doubt you can remove
>> this
> even if it hasn't been used in 5+ years.
 Hmm, depends.
 This has been used by Xen tools in dom0 only. If the last usage has
>> been
 in a Xen version which is no longer able to run with current Linux in
 dom0 it could be removed. But I guess this would have to be a rather
>> old
 version of Xen (like 3.x?).
 Paul, can you give a hint since which Xen version the toolstack no
 longer relies on udev to start the hotplug scripts?
>>> The udev rules were in a file called tools/hotplug/Linux/xen-
>> backend.rules (in xen.git), and a commit from Roger removed the NIC
 rules
>> in 2012:
>>> commit 57ad6afe2a08a03c40bcd336bfb27e008e1d3e53
>> Xen 4.2
>>> The last commit I could find to that file modified its name to xen-
>> backend.rules.in, and this was finally removed by George in 2015:
>>> commit 2ba368d13893402b2f1fb3c283ddcc714659dd9b
>> Xen 4.6
>>> So, I think this means anyone using a version of the Xen tools within
>> recent memory will be having their hotplug scripts called directly by
>> libxl (and having udev rules present would actually be counter-
 productive,
>> as George's commit states and as I discovered the hard way when the
 change
>> was originally made).
>> The problem are systems with either old Xen versions (before Xen 4.2)
 or
>> with other toolstacks (e.g. Xen 4.4 with xend) which want to use a new
>> dom0 kernel.
>> And I'm not sure there aren't such systems (especially in case someone
>> wants to stick with xend).
> But would someone sticking with such an old toolstack expect to run on
 an unmodified upstream dom0? There has to be some way in which we can
 retire old code.
 As long as there are no hypervisor interface related issues
 prohibiting running dom0 unmodified I think the expectation to be
 able to use the kernel in that environment is fine.
>> I think we need a better policy in future then otherwise we will only 
>> collect baggage.
> 
> The Linux kernel policy regarding user interfaces and existing use cases
> is rather clear and we should not deviate without very strong reasons.
> 
>>> Another question coming up would be: how is this handled in a driver
>>> domain running netback? Which component is starting the hotplug script
>>> there? I don't think we can assume a standard Xen toolset in this case.
>>> So I'd rather leave this code as it is instead of breaking some rare
>>> but valid use cases.
>> I am not sure there is a standard. Do we 'support' driver domains with any 
>> sort of tools API or do they really just have to 

[Xen-devel] Making save/restore optional in toolstack, for edge/embedded derivatives

[Rich Persaud]

Linux stubdom patches currently require qemu in dom0 for consoles [1], due to 
the upstream toolstack need for save/restore.  Until a long-term solution is 
available (multiple console support in xenconsoled), would tools maintainers 
consider a patch that made save/restore build-time configurable for the 
toolstack?  This would avoid Xen edge/embedded derivatives having to patch 
downstream to remove save/restore, e.g. to avoid qemu in dom0.

Rich

https://github.com/marmarek/xen/commit/13c6d27259929eca56a11bd0dacbe557274224d3

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [MirageOS-devel] [PATCH v2 4/6] Add Code Review Guide

[Rich Persaud]

On Nov 28, 2019, at 09:05, Lars Kurth  wrote:
> 
> On 28/11/2019, 07:37, "Jan Beulich"  wrote:
> 
>>On 28.11.2019 14:06, Lars Kurth wrote:
>> I can certainly add something on the timing , along the lines of
>> * For complex series, consider the time it takes to do reviews (maybe with a 
>> guide of LOC per hour) and give reviewers enough time to
>> * For series with design issues or large questions, try and highlight the 
>> key open issues in cover letters clearly and solicit feedback from key 
>> maintainers who can comment on the open issue. The idea is to save both the 
>> contributor and the reviewers time by focussing on what needs to be resolved 
>> * Don’t repost a series, unless all review comments are addressed
>> or the reviewers asked you to do so. The problem with this is that
>> this is somewhat in conflict with the "let's focus on the core
>> issues and not get distracted by details early on in a review cycle".
>> In other words, this can only work, if reviewers focus on major
>> issues in early reviews only and do not focus on style, coding
>> standards, etc.
> 
>But this doesn't make much sense either, because then full re-reviews
>need to happen anyway on later versions, to also deal with the minor
>issues. For RFC kind of series omitting style and alike feedback
>certainly makes sense, but as soon as a patch is non-RFC, it should
>be considered good to go in by the submitter.
> 
> OK, I think we have a disconnect between ideal and reality. 
> 
> I see two issues today
> * Key maintainers don't always review RFC series [they end up at the bottom 
> of the priority list, even though spending time on RFCs will save time 
> elsewhere later]. So the effect is that then the contributor assumes there 
> are no major issues and ends it as a proper series
> * In practice what has happened often in the past is that design, 
> architecture, assumption flaws are found in early versions of a series.
>   - This usually happens because of an oversight or because there was no 
> design discussion prior to the series being posted and agreed
>   - Common sense would dictate that the biggest benefit for both the 
> reviewer, the contributor and the community as a whole would be to try and 
> focus on such flaws and leave everything aside
>   - Of course there may be value in doing a detailed reviews of such a series 
> as there may be bits that are unaffected by such a flaw
>   - But there will likely be parts which are not: doing a detailed review of 
> such portions wastes everyone's time
> 
> So coming back to your point. Ideally, it would be nice if we had the 
> capability to call out parts of a series as "problematic" and treating such 
> parts differently.

We may be able to reuse some "Shift Left" terminology, including citations of 
previous Xen code reviews to illustrate categories of design issues that can be 
shifted left:

  https://devopedia.org/shift-left

Rich

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] Add Code Review Guide

[Rich Persaud]

On Nov 28, 2019, at 05:12, Jan Beulich  wrote:
> 
> On 28.11.2019 01:54, Stefano Stabellini wrote:
>>> On Thu, 26 Sep 2019, Lars Kurth wrote:
>>> From: Lars Kurth 
>>> 
>>> This document highlights what reviewers such as maintainers and committers 
>>> look
>>> for when reviewing code. It sets expectations for code authors and provides
>>> a framework for code reviewers.
>> 
>> I think the document is missing a couple of things:
>> 
>> - a simple one line statement that possibly the most important thing in
>>  a code review is to indentify any bugs in the code
>> 
>> - an explanation that requests for major changes to the series should be
>>  made early on (i.e. let's not change the architecture of a feature at
>>  v9 if possible) I also made this comment in reply to patch #5. I'll
>>  let you decide where is the best place for it.
> 
> This needs balancing. People crucial to the evaluation of a new
> feature and its implementation simply may not have the time to
> reply prior to v9. We've had situations where people posted new
> revisions every other day, sometimes even more than one per day.
> 
> As indicated in several other contexts before - imo people not
> helping to shoulder the review load should also not have the
> expectation that their (large) contributions will be looked at
> in due course. 

To make this actionable, we could have:

- reviewer demand index:  automated index of open patches still in need of 
review, sorted by decreasing age

- review flow control:  each new patch submission cites one recent review by 
the patch submitter, for a patch of comparable size

- reviewer supply growth:  a bootstrapping guide for new reviewers and 
submitters, with patterns, anti-patterns, and examples to be emulated

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)

[Rich Persaud]

On Nov 27, 2019, at 04:16, Jan Beulich  wrote:
> 
> On 26.11.2019 22:20, Rich Persaud wrote:
>> As an intermediate step, could we have an umbrella opt-in
>> Kconfig option (CONFIG_EFI_NONSPEC_COMPATIBILITY?) that
>> enables multiple EFI options for maximum hardware compatibility?
>> For this thread and Xen 4.13, that would be
>> EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc.  If more
>> options/quirks are added in the future, downstreams using
>> EFI_NONSPEC_COMPATIBILITY would get them by default.
> 
> While I don't particularly like it, I'd be okay with having such
> an option, provided it doesn't hamper code readability too much.
> However - why would you stop at those two things? Why not also
> exclude reboot through UEFI (as indicated by Andrew), or use of
> runtime services as a whole? What about /mapbs? The fundamental
> problem I see here really is - where would we draw the line?

If we take this thread as an example, a middle ground was found among 
developers motivated to maintain the workarounds for downstream projects with 
affected hardware.  Qubes, EVE & OpenXT are used on edge/client devices that 
often have (relative to servers) a shorter lifetime, with more device churn and 
support costs. 

These two initial options would address current pain points and enable the use 
of upstream Xen + EFI RS on more devices, e.g. for OTA updates with 
forward-sealed integrity measurements.  The line could change if more 
downstreams adopt the option and/or new devices appear that have both customer 
adoption and problematic firmware behavior.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)

[Rich Persaud]

On Nov 26, 2019, at 15:23, Andrew Cooper  wrote:
> On 26/11/2019 20:12, Roman Shaposhnik wrote:
>>> On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-Górecki
>>>  wrote:
>>> On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote:
 Hi Marek, after applying Jan's patch I'm making much further progress.
 Xen boots fine and Dom0 seems to be OK (more tests are needed tho on
 my end).
 I'm attaching the logs from Xen and Dom0.
 At this point it seems that adding efi=attr=uc is a better option for
 these boxes than a wholesale efi=no-rs
 Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was
 supposed to cover by default (so I don't have to add efi=attr=uc)?
>>> No, this looks like some different firmware (?) issue.
 Question #2: is there any downside to *always* specifying efi=attr=uc?
 Even for servers that, strictly speaking, don't need it?
>>> TL;DR: It should be fine. It is what Linux does too.
>>> Details:
>>> Lets take a look why 'efi=attr=uc' helps, and how can we make it work
>>> out of the box:
>>> The issue is about memory marked as type=11 (EfiMemoryMappedIO) with
>>> attr=8000 (EFI_MEMORY_RUNTIME). Indeed none of cachability
>>> attribute is defined. For the record, defined attributes are (UEFI spec
>>> .6):
>>>EFI_MEMORY_UC Memory cacheability attribute: The memory region supports
>>>being configured as not cacheable.
>>>EFI_MEMORY_WC Memory cacheability attribute: The memory region supports
>>>being configured as write combining.
>>>EFI_MEMORY_WT Memory cacheability attribute: The memory region supports
>>>being configured as cacheable with a “write through” policy.
>>>Writes that hit in the cache will also be written to main memory.
>>>EFI_MEMORY_WB Memory cacheability attribute: The memory region supports
>>>being configured as cacheable with a “write back” policy. Reads
>>>and writes that hit in the cache do not propagate to main memory.
>>>Dirty data is written back to main memory when a new cache line
>>>is allocated.
>>>EFI_MEMORY_UCE Memory cacheability attribute: The memory region supports
>>>being configured as not cacheable, exported, and supports the
>>>“fetch and add” semaphore mechanism.
>>> My reading of UEFI spec doesn't give much hints what to do with memory
>>> mappings without any cachability attribute. The only related info I've
>>> found is about EfiMemoryMappedIO:
>>>This memory is not used by the OS. All system memory-mapped IO
>>>information should come from ACPI tables.
>>> So, maybe there is some more info?
>>> Anyway, if I understand correctly, MMIO region should be mapped as UC,
>>> right?
>>> I've also taken look at what Linux does. And basically, the only bit
>>> Linux care about is EFI_MEMORY_WB - if it's absent, then set the region
>>> as uncachable (page cache disabled bit in page table entry). So,
>>> basically Linux by default does what Xen's efi=attr=uc does.
>> Very interesting! Thanks for doing the research.
>> 
>>> So, to improve Xen's hardware/firmware compatibility, I have two ideas:
>>> 1. Make efi=attr=uc the default (it's still possible to disable it with
>>> efi=attr=no).
>> I'd be very much in favor of that too (especially since it seems to match
>> Linux behaviour) What do others think?
> 
> Its more than just this.  Linux also doesn't use EFI reboot because it
> is broken almost everywhere (because Windows doesn't use it because its
> broken almost everywhere, so it never gets fixed).
> 
> Xen should be following Linux, but I'm exhausted arguing this point.
> 
> A consequence is that downstream tend to share a pile of "unbreak Xen on
> UEFI" patches which have been rejected upstream on philosophical rather
> than technical grounds, despite this being a toxic environment to work in.

As an intermediate step, could we have an umbrella opt-in Kconfig option 
(CONFIG_EFI_NONSPEC_COMPATIBILITY?) that enables multiple EFI options for 
maximum hardware compatibility?  For this thread and Xen 4.13, that would be 
EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc.  If more options/quirks are added 
in the future, downstreams using EFI_NONSPEC_COMPATIBILITY would get them by 
default.

The long-term solution is an OSS virtualization-security test tool (e.g. with 
Xen and QEMU KVM) that can be run by OEM/ODM QA factory teams on pre-production 
firmware and hardware.  That is the most OEM-actionable development window 
where firmware quality issues can be detected and fixed.  Microsoft's hardware 
logo/certification work with Windows 10 OEMs on "secured core" features is also 
tackling firmware improvements for virtualization-based security. 

From the business side, Dell/HP/Lenovo + other OEMs and ODMs could add premium 
"FirmCare" SKUs into their custom build ordering systems, where customers could 
pay a small fee for additional firmware support, custom root-of-trust (e.g. 
BootGuard) key management, or even coreboot.  

Re: [Xen-devel] Status of 4.13

[Rich Persaud]

On Nov 21, 2019, at 17:11, Marek Marczykowski-Górecki 
 wrote:
> 
> On Thu, Nov 21, 2019 at 11:39:14AM -0800, Roman Shaposhnik wrote:
>>> On Thu, Nov 21, 2019 at 9:38 AM Andrew Cooper  
>>> wrote:
>>> On 21/11/2019 17:31, Roman Shaposhnik wrote:
>> On Wed, Nov 20, 2019 at 10:06 PM Jürgen Groß  wrote:
>>> Where do we stand with Xen 4.13 regarding blockers and related patches?
>>> 1. OSStest failure regarding nested test:
>>> I'm not quite sure whether the currently debated patch of Andrew is
>>> fixing the problem. If not, do we know what is missing or how to
>>> address the issue? If yes, could we please come to an agreement?
>>> As an alternative: any thoughts about ignoring this test failure for
>>> 4.13-RC3 (IOW: doing a force push)?
>>> 2. Ryzen/Rome failures with Windows guests:
>>> What is the currently planned way to address the problem? Who is
>>> working on that?
>>> 3. Pending patches for 4.13:
>>> Could I please have feedback which patches tagged as "for-4.13" are
>>> fixing real regressions or issues? I don't want to take any patches
>>> not fixing real problems after RC3, and I hope to be able to get a
>>> push rather sooner than later to be able to let Ian cut RC3.
>>> 4. Are there any blockers for 4.13 other than 1. and 2. (apart of any
>>> pending XSAs)?
>> Any chance the efi=no-rs regression can be added to the list? I 
>> understand
>> that I'm still on the hook to provide more details (I promise to do it 
>> on Fri
>> when I get to my lab to actually have a serial console on all these 
>> boxes).
>> At the same time this is a pretty serious regression for an entire class 
>> of
>> devices where Xen was perfectly happy even during RC1.
> https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=534f9e29ce28580892b3856036b5e5cd805667cc
> has been committed.  It is in staging, but not in master yet (because
> master is blocked by my regression in 1).
>> I'll make sure to test it on Fri, but here's where I'm lost -- my
>> understanding that
>> activation of this patch requires a special build flag to be passed.

Draft doc for the Xen 4.13 improvement:
https://wiki.xen.org/wiki/Xen_EFI#Compatibility_of_UEFI_Host_Firmware.2C_Xen_and_UEFI_Runtime_Services

Corrections and compatibility test reports would be welcome.

Rich

>> Which means,
>> we're still very much in a regresses state when it comes to building
>> out-of-the-box,
>> no?
> 
> No, there are two thing:
> 1. A bug triggered by efi=no-rs flag - fixed in the above commit
> 2. A second commit making efi=no-rs unnecessary on some machines - this
> is what require build flag (CONFIG_EFI_SET_VIRTUAL_ADDRESS_MAP=y).
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Likely regression in efi=no-rs option

[Rich Persaud]

On Nov 19, 2019, at 02:13, Roman Shaposhnik  wrote:
> 
> Ok, to sum up -- there's definitely a pretty major regression on all
> this hardware with Xen 4.13 RC2:
>   
> https://www.dell.com/en-us/work/shop/gateways-embedded-computing/sc/gateways-embedded-pcs/edge-gateway?~ck=bt
> 
> Without efi=no-rs option Xen panics on boot (sorry for attaching the
> screenshot -- I know it is not super helpful but it gets the point
> across)

I don't think the screenshot came through..

By "without efi=no-rs" do you mean "efi=rs" or no explicit value for efi on the 
Xen command line?

> With efi=no-rs Xen boots fine, but Dom0 can't come up.

Is that with RC2, or RC2 + the recent patch from Marek for efi=no-rs?  Any 
serial logs from dom0 boot?  Can you share the dom0 kernel command line with 
all options?

Thanks,
Rich

> And, once again, this is clearly a regression from RC1 (just verified).
> 
> Thanks,
> Roman.
> 
>>>> On Sat, Nov 16, 2019 at 12:47 PM Rich Persaud  wrote:
>> I don't know if there's a change in efi=no-rs behavior, but some EFI fixes 
>> were merged on 10/25, which (on some machines) have reduced the need to 
>> disable UEFI runtime services to work around non-spec UEFI firmware.  This 
>> should increase hardware compatibility with Xen.  Of course, there could 
>> still be other reasons to disable UEFI runtime services.
>> Could you try booting the affected systems with efi=rs?
>> Rich
>>>> On Nov 16, 2019, at 00:27, Roman Shaposhnik  wrote:
>>> Hi!
>>> as I've reported earlier -- part of my testing of Xen 4.13 RC2 failed
>>> in a massive way with Dom0 never coming up. I've traced that problem
>>> to the option that we're using to boot Xen:
>>> efi=no-rs
>>> We've been using this option for quite sometime and Xen 4.13 RC2
>>> is the first one that seems to make Dom0 boot fail with this option
>>> present (note that RC1 was fine).
>>> I was wondering whether there were any changes in the areas related
>>> to UEFI in Xen that may have triggered this.
>>> Here's the boot line that works with RC2:
>>> dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
>>> adding efi=no-rs make Dom0 boot process fail:
>>> efi=no-rs dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
>>> Attaching xl info and dmesg just in case
>>> Thanks,
>>> Roman.
>>> 
>>> 
>>> ___
>>> Xen-devel mailing list
>>> Xen-devel@lists.xenproject.org
>>> https://lists.xenproject.org/mailman/listinfo/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Likely regression in efi=no-rs option

[Rich Persaud]

I don't know if there's a change in efi=no-rs behavior, but some EFI fixes were 
merged on 10/25, which (on some machines) have reduced the need to disable UEFI 
runtime services to work around non-spec UEFI firmware.  This should increase 
hardware compatibility with Xen.  Of course, there could still be other reasons 
to disable UEFI runtime services.

Could you try booting the affected systems with efi=rs?

Rich

> On Nov 16, 2019, at 00:27, Roman Shaposhnik  wrote:
> 
> Hi!
> 
> as I've reported earlier -- part of my testing of Xen 4.13 RC2 failed
> in a massive way with Dom0 never coming up. I've traced that problem
> to the option that we're using to boot Xen:
>efi=no-rs
> We've been using this option for quite sometime and Xen 4.13 RC2
> is the first one that seems to make Dom0 boot fail with this option
> present (note that RC1 was fine).
> 
> I was wondering whether there were any changes in the areas related
> to UEFI in Xen that may have triggered this.
> 
> Here's the boot line that works with RC2:
>dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
> adding efi=no-rs make Dom0 boot process fail:
>efi=no-rs dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin 
> smt=false
> 
> Attaching xl info and dmesg just in case
> 
> Thanks,
> Roman.
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Virtualization videos from Platform Security Summit 2019

[Rich Persaud]

Xen, OpenXT, QubesOS and embedded developers may be interested in these 
videos.  

The first (IBM ppc Ultravisor with extended Q) is related to past discussions 
of minimal L0 Xen in firmware, similar to HP/Bromium nesting-optimized 
hypervisor.  The second is related to kexec and TrenchBoot, which (video TBD) 
was demoed booting Xen with AMD SKINIT DRTM on PC Engines $150 APU2 with TPM 
2.0.  The third describes a custom AMD Jaguar-derived SoC with a precursor to 
AMD's PSP, running a minimal version of Hyper-V called Nanovisor.  The fourth 
describes changes to Windows and x86 firmware/hardware for virtualization-based 
security.

Rich


Protected Execution Facility: We present the Protected Execution Facility ― an 
architecture modification for IBM Linux and OpenPower Linux servers ― along 
with the associated firmware, the Protected Execution Ultravisor which provides 
additional security to virtual machines ― called secure virtual machines 
(SVMs). The Protected Execution Facility concurrently supports both normal VMs 
and SVMs.
https://www.platformsecuritysummit.com/2019/speaker/hunt/


LinuxBoot progress: boot anything from Linux: LinuxBoot replaces traditionally 
closed source firmware (e.g. UEFI) with an open, auditable, and measurable 
Linux kernel and initramfs. We’ll present an overview of LinuxBoot, its part in 
the boot integrity story, and talk about newly gained abilities to boot VMware, 
Xen, and Windows from Linux, and future plans.
https://www.platformsecuritysummit.com/2019/speaker/koch/


Guarding Against Physical Attacks: The Xbox One Story: ... describe the Xbox 
security design goals and why it needs to guard against hardware attacks, 
followed by descriptions of the hardware and software architecture to keep the 
Xbox secure. This includes details about the custom SoC we built with AMD and 
how we addressed the fact that all data read from flash, the hard drive, and 
even DRAM cannot be trusted. We will also discuss the corresponding software 
changes we made to keep the system and the games secure.
https://www.platformsecuritysummit.com/2019/speaker/chen


Advancing Windows Security: ... the OS security engineering team at Microsoft 
has built a strategy to address new and challenging attacks. This talk will 
walk attendees through Windows current and future security strategy and the 
engineering challenges with scaling across new devices, form factors, and 
threat models ...
https://www.platformsecuritysummit.com/2019/speaker/weston/



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] Documentation formats, licenses and file system structure

[Rich Persaud]

On Oct 17, 2019, at 12:55, Stefano Stabellini  wrote:
> 
> On Thu, 17 Oct 2019, Rich Persaud wrote:
>>> On Oct 17, 2019, at 12:32, Stefano Stabellini  
>>> wrote:
>>> 
>>> On Thu, 17 Oct 2019, Lars Kurth wrote:
>>>> On 16/10/2019, 17:35, "Rich Persaud"  wrote:
>>>> 
>>>>>> On Oct 15, 2019, at 08:27, Lars Kurth  wrote:
>>>>> ...
>>>>> 
>>>>> My point really was is that due to storing the files in git, we 
>>>>> essentially do NOT today do this.
>>>>> So we would need to take extra action: e.g. manually or through tooling
>>>>> 
>>>>>>> 4.2: We could require individual authors to be credited: in that
>>>>>>> case we probably ought to lead by example and list the authors
>>>>>>> in a credit/license section and extract the information from
>>>>>>> git logs when we generate it (at some point in the future)
>>>>>>> 5: You give an indication whether you made changes ... in practice
>>>>>>> this means you have to state significant changes made to the works
>>>>>> 
>>>>>> This is also helpful for provenance of changes, which is relevant in 
>>>>>> safety-oriented documentation.  It can be used to clearly delineate 
>>>>>> CC-licensed content (which may be reused by many companies) from "All 
>>>>>> Rights Reserved" commercial content that may be added for a specific 
>>>>>> commercial audience or purpose.
>>>>> 
>>>>> I agree
>>>>> 
>>>>> I think the outcome of this analysis is really that the only significant 
>>>>> difference between BSD and CC-BY in this context is the  "All Rights 
>>>>> Reserved" portion
>>>> 
>>>>   Also - BSD is a "software" license while CC-BY is a "content" license, 
>>>> so they are not strictly comparable, even if they use similar terminology.
>>>> 
>>>> True, but as we have noticed the boundary between content and in-code docs 
>>>> content is fuzzy.
>>>> 
>>>>>> There is a difference between "software" which "runs on machines" and 
>>>>>> "documentation" which "runs on humans".  Combined software (e.g. BSD 
>>>>>> code from two origins) is executed identically, despite origin.  Humans 
>>>>>> make value judgements based on the author/origin of content, hence the 
>>>>>> focus on attribution.  Yes, there is a provenance graph in git 
>>>>>> (software/data), but that's not typically visible to human readers, 
>>>>>> except as a generated report, i.e. documentation.
>>>>> 
>>>>> Yes true. But also true for CC-BY-4 sources stored in git unless extra 
>>>>> action is taken 
>>>>> 
>>>>> But my point is: 
>>>>> * If we take extra action as e.g. proposed in 4.2 we can apply this 
>>>>> uniformly to BSD as well as CC-BY pages
>>>>> * We can add a section on re-use as proposed in 4.2 which recommends best 
>>>>> practices around 5.  
>>>>> * We can highlight sections that are BSD vs CC-BY in such a section, such 
>>>>> that someone who has issue can remove these easily
>>>>> 
>>>>> In addition to these points: maybe it is too impractical to create ABI 
>>>>> documentation based on CC-BY-4 (given that a lot of what we need is 
>>>>> already in BSD sources). 
>>>>> We could just copy some of the content in the BSD sources to new CC-BY-4 
>>>>> sources, but in practice it would just be hiding the potential legal 
>>>>> issues behind it. 
>>>>> Someone could contest the creation and argue that portions of the now 
>>>>> CC-BY-4 sources are in fact BSD: in practice this is extremely unlikely, 
>>>>> but it is possible.
>>>>> 
>>>>>>> As such, BSD-2/3-Clause in our context works similarly to CC-BY-4
>>>>>>> from a downstream's perspective. In fact CC-BY-4 is somewhat stricter
>>>>>> 
>>>>>> If we don't want the incentives and provenance properties of CC-BY, 
>>>>>> there is the option of CC0, which is the equivalent of public domain.  
>>>>>> This would d

Re: [Xen-devel] [RFC] Documentation formats, licenses and file system structure

[Rich Persaud]

On Oct 17, 2019, at 12:32, Stefano Stabellini  wrote:
> 
> On Thu, 17 Oct 2019, Lars Kurth wrote:
>> On 16/10/2019, 17:35, "Rich Persaud"  wrote:
>> 
>>>> On Oct 15, 2019, at 08:27, Lars Kurth  wrote:
>>> ...
>>> 
>>> My point really was is that due to storing the files in git, we essentially 
>>> do NOT today do this.
>>> So we would need to take extra action: e.g. manually or through tooling
>>> 
>>>>>  4.2: We could require individual authors to be credited: in that
>>>>>  case we probably ought to lead by example and list the authors
>>>>>  in a credit/license section and extract the information from
>>>>>  git logs when we generate it (at some point in the future)
>>>>> 5: You give an indication whether you made changes ... in practice
>>>>> this means you have to state significant changes made to the works
>>>> 
>>>> This is also helpful for provenance of changes, which is relevant in 
>>>> safety-oriented documentation.  It can be used to clearly delineate 
>>>> CC-licensed content (which may be reused by many companies) from "All 
>>>> Rights Reserved" commercial content that may be added for a specific 
>>>> commercial audience or purpose.
>>> 
>>> I agree
>>> 
>>> I think the outcome of this analysis is really that the only significant 
>>> difference between BSD and CC-BY in this context is the  "All Rights 
>>> Reserved" portion
>> 
>>Also - BSD is a "software" license while CC-BY is a "content" license, so 
>> they are not strictly comparable, even if they use similar terminology.
>> 
>> True, but as we have noticed the boundary between content and in-code docs 
>> content is fuzzy.
>> 
>>>> There is a difference between "software" which "runs on machines" and 
>>>> "documentation" which "runs on humans".  Combined software (e.g. BSD code 
>>>> from two origins) is executed identically, despite origin.  Humans make 
>>>> value judgements based on the author/origin of content, hence the focus on 
>>>> attribution.  Yes, there is a provenance graph in git (software/data), but 
>>>> that's not typically visible to human readers, except as a generated 
>>>> report, i.e. documentation.
>>> 
>>> Yes true. But also true for CC-BY-4 sources stored in git unless extra 
>>> action is taken 
>>> 
>>> But my point is: 
>>> * If we take extra action as e.g. proposed in 4.2 we can apply this 
>>> uniformly to BSD as well as CC-BY pages
>>> * We can add a section on re-use as proposed in 4.2 which recommends best 
>>> practices around 5.  
>>> * We can highlight sections that are BSD vs CC-BY in such a section, such 
>>> that someone who has issue can remove these easily
>>> 
>>> In addition to these points: maybe it is too impractical to create ABI 
>>> documentation based on CC-BY-4 (given that a lot of what we need is already 
>>> in BSD sources). 
>>> We could just copy some of the content in the BSD sources to new CC-BY-4 
>>> sources, but in practice it would just be hiding the potential legal issues 
>>> behind it. 
>>> Someone could contest the creation and argue that portions of the now 
>>> CC-BY-4 sources are in fact BSD: in practice this is extremely unlikely, 
>>> but it is possible.
>>> 
>>>>> As such, BSD-2/3-Clause in our context works similarly to CC-BY-4
>>>>> from a downstream's perspective. In fact CC-BY-4 is somewhat stricter
>>>> 
>>>> If we don't want the incentives and provenance properties of CC-BY, there 
>>>> is the option of CC0, which is the equivalent of public domain.  This 
>>>> would delegate the task of separating commercial vs CC content to each 
>>>> reader, without any license-required attribution or separation.
>>>> 
>>>> Some background on licenses designed for documentation, which has 
>>>> different legal requirements than software:
>>>> 
>>>> https://www.dreamsongs.com/IHE/IHE-50.html
>>>> https://creativecommons.org/faq/#what-are-creative-commons-licenses (not 
>>>> for s/w)
>>> 
>>> I will have a look. But the core issue - which is why I have proposed what 
>>> I have - is the question on how practically ABI documentation publish

Re: [Xen-devel] [RFC] Documentation formats, licenses and file system structure

[Rich Persaud]

> On Oct 15, 2019, at 08:27, Lars Kurth  wrote:
> Hi Rich,
> 
>>>> On 15 Oct 2019, at 02:58, Rich Persaud  wrote:
>>>>> On Oct 11, 2019, at 07:11, Lars Kurth  wrote:
>>> On 11/10/2019, 02:24, "Stefano Stabellini"  wrote:
>>>>   On Thu, 10 Oct 2019, Lars Kurth wrote:
>>>> @Stefano: as you and I believe Brian will be spending time on improving the
>>>> ABI docs, I think we need to build some agreement here on what/how
>>>> to do it. I was assuming that generally the consensus was to have
>>>> docs close to the code in source, but this does not seem to be the case.
>>>> But if we do have stuff separately, ideally we would have a tool that helps
>>>> point people editing headers to also look at the relevant docs. Otherwise 
>>>> it will
>>>> be hard to keep them in sync.
>>>   In general, it is a good idea to keep the docs close to the code to make
>>>   it easier to keep them up to date. But there is no one-size-fits-all
>>>   here. For public ABI descriptions, I agree with Andrew that ideally they
>>>   should not be defined as C header files.
>>>   But it is not an issue: any work that we do here won't be wasted. For
>>>   instance, we could start by adding more comments to the current header
>>>   files. Then, as a second step, take all the comments and turn them into
>>>   a proper ABI description document without any C function declarations.
>>>   It is easy to move English text around, as long as the license allows it
>>>   -- that is the only potential blocker I can see.
>>> This is likely to be problematic. First of all, we are talking about 
>>> BSD-3-Clause
>>> or BSD-2-Clause code (the latter is more dominant in headers I believe) in
>>> all known cases.
>>> The main properties of the BSD are
>>> 1: Can be pretty much used anywhere for any purpose
>>> 2: Can be modified for any purpose
>>> 3: But the original license header must be retained in derivates
>> 
>> This is equivalent to attribution of the copyright owner of the originally 
>> created file.
>> 
>>> Does *not* have requirements around attribution as CC-BY-4: however,
>>> as we store everything in git attribution is handled by us by default
>> 
>> See above, the license header attributes copyright, since BSD was created 
>> for "software" and people who work on "software" would typically be looking 
>> at source code, hence the primary attribution takes place there, with 
>> secondary attribution in EULAs, "About" panels, etc.
>> 
>>> CC-BY-4 also has properties 1-3
>>> In addition: it does require that
>>> 4: Derived works are giving appropriate credit to authors
>>>   We could clarify in a COPYING how we prefer to do this
>>>   4.1: We could say that "referring to the Xen Project community"
>>>   is sufficient to comply with the attribution clause
>> 
>> One motivation for CC-BY (with attribution) is to create an incentive 
>> (credit) for the creation of documentation, which is not commonly a favorite 
>> pastime of developers.   Credit typically goes at least to the original 
>> author of a section of documentation, with varying ways of crediting 
>> subsequent contributors.  The documentation can be structured to make 
>> crediting easier.  The mechanism for crediting can be designed to encourage 
>> specific outcomes, along our projected doc lifecycle for safety 
>> certification, contributors, evaluators and commercial investors.
> 
> My point really was is that due to storing the files in git, we essentially 
> do NOT today do this.
> So we would need to take extra action: e.g. manually or through tooling
> 
>>>   4.2: We could require individual authors to be credited: in that
>>>   case we probably ought to lead by example and list the authors
>>>   in a credit/license section and extract the information from
>>>   git logs when we generate it (at some point in the future)
>>> 5: You give an indication whether you made changes ... in practice
>>> this means you have to state significant changes made to the works
>> 
>> This is also helpful for provenance of changes, which is relevant in 
>> safety-oriented documentation.  It can be used to clearly delineate 
>> CC-licensed content (which may be reused by many companies) from "All Rights 
>> Reserved" commercial content that may be added for a specific commercial 
>> audience or pur

Re: [Xen-devel] [RFC] Documentation formats, licenses and file system structure

[Rich Persaud]

> On Oct 11, 2019, at 07:11, Lars Kurth  wrote:
> 
> On 11/10/2019, 02:24, "Stefano Stabellini"  wrote:
> 
>>On Thu, 10 Oct 2019, Lars Kurth wrote:
>> * Would we ever include API docs generated from GPLv2 code? E.g. for safety 
>> use-cases?
>> @Stefano, @Artem: I guess this one is for you. 
>> I suppose if we would have a similar issue for a safety manual
>> I am also assuming we would want to use sphinx docs and rst to generate a 
>> future safety manual
> 
>Hi Lars,
> 
>Thanks for putting this email together.
> 
>In terms of formats, I don't have a preference between rst and pandoc,
>but if we are going to use rst going forward, I'd say to try to use rst
>for everything, including converting all the old stuff. The fewer
>different formats, the better.
> 
> I think the proposal that needs to follow on from this (which would at some
> point need to be voted on) would then be to go for rst. 
> 
>As I mentioned during the FuSa call, I agree with you, Andrew, and
>others that it would be best to have the docs under a CC license. I do
>expect that we'll end up copy/pasting snippets of in-code comments into
>the docs, so I think it is important that we are allowed to do that from
>a license perspective. It is great that GPLv2 allows it (we need to be
>sure about this).
> 
> The GPL does *not* allow this, but (c) law and fair use clauses do. So 
> typically
> stuff such as
> * Referring to function names, signatures, etc. tend to be all fine
> * Copying large portions of in-line comments would not be fine, but
> If they are large, they would in most cases be re-written in a more suitable
> language. 
> 
> So, I think overall, we should be fine. It's a bit of a grey area though.
> 
> And as you point out below, most of the code in question is typically BSD 
> 
>Yes, I expect that some docs might be automatically generated, but from
>header files, not from source code. Especailly public/ header files,
>which are typically BSD, not GPLv2. I cannot come up with examples of
>docs we need to generated from GPLv2-only code at the moment, hopefully
>there won't be any.
> 
> That makes things a lot easier.
> 
>>I wasn't planning on reusing any of the markup, and wasn't expecting to
>>use much of the text either.  I'm still considering the option of
>>defining that xen/public/* isn't the canonical description of the ABI,
>>because C is the wrong tool for the job.
>> 
>>Its fine to provide a C set of headers implementing an ABI, but there is
>>a very deliberate reason why the canonical migration v2 spec is in a
>>text document.
>> 
>> @Stefano: as you and I believe Brian will be spending time on improving the
>> ABI docs, I think we need to build some agreement here on what/how
>> to do it. I was assuming that generally the consensus was to have
>> docs close to the code in source, but this does not seem to be the case.
>> 
>> But if we do have stuff separately, ideally we would have a tool that helps
>> point people editing headers to also look at the relevant docs. Otherwise it 
>> will
>> be hard to keep them in sync.
> 
>In general, it is a good idea to keep the docs close to the code to make
>it easier to keep them up to date. But there is no one-size-fits-all
>here. For public ABI descriptions, I agree with Andrew that ideally they
>should not be defined as C header files.
> 
>But it is not an issue: any work that we do here won't be wasted. For
>instance, we could start by adding more comments to the current header
>files. Then, as a second step, take all the comments and turn them into
>a proper ABI description document without any C function declarations.
>It is easy to move English text around, as long as the license allows it
>-- that is the only potential blocker I can see.
> 
> This is likely to be problematic. First of all, we are talking about 
> BSD-3-Clause
> or BSD-2-Clause code (the latter is more dominant in headers I believe) in
> all known cases.
> 
> The main properties of the BSD are
> 1: Can be pretty much used anywhere for any purpose
> 2: Can be modified for any purpose 
> 3: But the original license header must be retained in derivates

This is equivalent to attribution of the copyright owner of the originally 
created file.

> Does *not* have requirements around attribution as CC-BY-4: however,
> as we store everything in git attribution is handled by us by default

See above, the license header attributes copyright, since BSD was created for 
"software" and people who work on "software" would typically be looking at 
source code, hence the primary attribution takes place there, with secondary 
attribution in EULAs, "About" panels, etc.

> CC-BY-4 also has properties 1-3
> In addition: it does require that 
> 4: Derived works are giving appropriate credit to authors 
>We could clarify in a COPYING how we prefer to do this
>4.1: We could say 

Re: [Xen-devel] [ANNOUNCE] Call for agenda items for September 5th Community Call @ 15:00 UTC

[Rich Persaud]

On Sep 5, 2019, at 12:12, Lars Kurth  wrote:
>  
> > We can defer the xenstoreless name service topic to the October monthly 
> > call.
> > 
> > For today's call, can we discuss the previously posted high-level design 
> > for unification of the domB RFC with dom0less, as "domB mode" for 
> > disaggregation of Xen's dom0?
> > 
> > - domB mode for dom0less (July 2019):  https://lists.gt.net/xen/devel/557782
> > - domB RFC (June 2018):  https://lists.gt.net/xen/devel/519367
>  
> I had seen this too late. Apologies
> Lars

Since we did cover the xenstoreless name service today, we can cover "domB mode 
for dom0less" in the October call.

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Call for agenda items for September 5th Community Call @ 15:00 UTC

[Rich Persaud]

> On Sep 5, 2019, at 04:36, Lars Kurth  wrote:
> 
> On 05/09/2019, 09:33, "Juergen Gross"  wrote:
> 
>>On 05.09.19 10:14, Andrew Cooper wrote:
>>> On 05/09/2019 08:49, Lars Kurth wrote:
>>>> On 05/09/2019, 08:41, "Rich Persaud"  wrote:
>>>> 
>>>> On Sep 5, 2019, at 03:19, Jan Beulich  wrote:
>>>> 
>>>> Forgive me asking, but why is this put up as an agenda item here?
>>>> IMO this is the kind of thing where you would send a proposal and
>>>> request feedback by email first, and put it up as an agenda item
>>>> here only if it got stalled there. (Apologies if I've overlooked
>>>> such a stalled thread.)
>>> 
>>> If Xen community call topics are limited to escalations of xen-devel 
>>> threads, then a new thread can be created for this topic. Xen community 
>>> calls have also provided real-time, interactive feedback on candidate 
>>> proposals, along with guidance on areas which need documentation before a 
>>> formal proposal is made to xen-devel.   Such agenda items are typically 
>>> covered after all series and priority topics have been addressed.
>>> 
>>> I don't mind having items such these on the agenda and to be fair have 
>>> added similar items onto the agenda in the past.
>>> Clearly, they are forward looking [like an RFC], for which reason I tend to 
>>> add them to the end of an agenda if there is a busy schedule
>>> 
>>> Personally, on this specific item, it is not really clear what the 
>>> questions are.  In other words: is this about UUIDS/domain ids only, or is 
>>> there something else.
>> 
>> Requiring something to be blocked on xen-devel before we discuss it on
>> the call is monumentally short sighted, and off-putting for contributors.
>> 
>> In this case, it is very definitely not the first time this problem has
>> been raised, as it is an XSA shaped elephant in the room.  Its no secret
>> that id wraps cause problems, and while our security policy doesn't
>> comment on the matter, it also doesn't say "warning - stuff *will* break
>> in weird, wonderful, and security-relevant ways when domid's wrap".
>> 
>> The order of the agenda is important, and I don't think this should be
>> at the top, but even if we only end up with 2 minutes to discuss it,
>> then so be it.  (2 minutes of talking can still be far more valuable
>> than a weeks worth of emailing.)
>> 
>> What is not acceptable is suggesting that it should be veto'd simply
>> because it is perceived to be a very fresh idea/query.
> 
>I still think it would help if a short high level design would be posted
>on xen-devel first.
> 
> I think that is a valid point and I agree. In the past when we had similar
> discussions often the outcome was not that clear and due to the nature of
> the calls the discussions were also not well reflected in the notes.
> 
> So, there is an argument, that discussions typically are more productive when
> there is the possibility for some preparation or an e-mail thread to refer to.

We can defer the xenstoreless name service topic to the October monthly call.

For today's call, can we discuss the previously posted high-level design for 
unification of the domB RFC with dom0less, as "domB mode" for disaggregation of 
Xen's dom0?

- domB mode for dom0less (July 2019):  https://lists.gt.net/xen/devel/557782
- domB RFC (June 2018):  https://lists.gt.net/xen/devel/519367

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Call for agenda items for September 5th Community Call @ 15:00 UTC

[Rich Persaud]

> On Sep 5, 2019, at 03:19, Jan Beulich  wrote:
> 
>> On 05.09.2019 04:32, Rich Persaud wrote:
>> Agenda item:  Domain name service for nested virt and disaggregation 
>> 
>> (text based on draft by Daniel Smith, who will speak to this agenda item)
>> 
>> If a future, minimal "L0 Xen" hypervisor can be optimized for nested 
>> virtualization in greenfield deployments (i.e. no requirement to maintain 
>> existing hypervisor-guest interfaces), then PV driver mechanisms other than 
>> grants, events and xenstore could be considered.  This was discussed in a 
>> Xen Summit 2019 design session:
>> https://lists.gt.net/xen/devel/560973
>> 
>> For some OpenXT use cases, we are in the process of further disaggregating 
>> the platform.  We need a name service to enable the disaggregated service 
>> domains to discover the other service domains with which they need to 
>> communicate.  Xenstore is not sufficient, as we would like to use Flask to 
>> control the data flow, as well as applying mandatory access control to 
>> service calls. 
>> 
>> We are reaching out to the Xen Community to elicit input on approaches, such 
>> that we might be able to submit an upstream RFC based on our early work:
>> 
>> - For a communication channel we are looking to leverage Argo, which is 
>> currently in experimental status. Its predecessor (v4v) is already being 
>> used in a similar fashion by Bromium uXen (https://github.com/openxt/uxen), 
>> which functions well across nested hypervisors.  uXen v4v includes a 
>> mechanism to control information flow.
>> 
>> - An open question is how to address the domains. Xen domain ids are reused 
>> and have no guarantee for uniqueness.  UUIDs are available and can provide 
>> better guarantees for uniqueness. Another approach is to use the name string 
>> which allows the ability for punctuation characters, eg. : or /, to create 
>> namespaced names for the domains.
> 
> Forgive me asking, but why is this put up as an agenda item here?
> IMO this is the kind of thing where you would send a proposal and
> request feedback by email first, and put it up as an agenda item
> here only if it got stalled there. (Apologies if I've overlooked
> such a stalled thread.)

If Xen community call topics are limited to escalations of xen-devel threads, 
then a new thread can be created for this topic. Xen community calls have also 
provided real-time, interactive feedback on candidate proposals, along with 
guidance on areas which need documentation before a formal proposal is made to 
xen-devel.   Such agenda items are typically covered after all series and 
priority topics have been addressed.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Call for agenda items for September 5th Community Call @ 15:00 UTC

[Rich Persaud]

Agenda item:  Domain name service for nested virt and disaggregation 

(text based on draft by Daniel Smith, who will speak to this agenda item)

If a future, minimal "L0 Xen" hypervisor can be optimized for nested 
virtualization in greenfield deployments (i.e. no requirement to maintain 
existing hypervisor-guest interfaces), then PV driver mechanisms other than 
grants, events and xenstore could be considered.  This was discussed in a Xen 
Summit 2019 design session:
https://lists.gt.net/xen/devel/560973

For some OpenXT use cases, we are in the process of further disaggregating the 
platform.  We need a name service to enable the disaggregated service domains 
to discover the other service domains with which they need to communicate.  
Xenstore is not sufficient, as we would like to use Flask to control the data 
flow, as well as applying mandatory access control to service calls. 

We are reaching out to the Xen Community to elicit input on approaches, such 
that we might be able to submit an upstream RFC based on our early work:

- For a communication channel we are looking to leverage Argo, which is 
currently in experimental status. Its predecessor (v4v) is already being used 
in a similar fashion by Bromium uXen (https://github.com/openxt/uxen), which 
functions well across nested hypervisors.  uXen v4v includes a mechanism to 
control information flow.

- An open question is how to address the domains. Xen domain ids are reused and 
have no guarantee for uniqueness.  UUIDs are available and can provide better 
guarantees for uniqueness. Another approach is to use the name string which 
allows the ability for punctuation characters, eg. : or /, to create namespaced 
names for the domains.



> On Sep 4, 2019, at 10:05, Lars Kurth  wrote:
> 
> Hi all,
> 
> the proposed agenda is in 
> https://cryptpad.fr/pad/#/2/pad/edit/xwUTm6b5f5ijPTQcF9IFgkBg/ and you can 
> edit to add items
> Alternatively, you can reply to this mail directly
> Agenda items appreciated ASAP: please put your name besides items if you edit 
> the document
> 
> Apologies for dropping the ball on this one, I forgot to add the CC list to 
> the earlier mail I sent
> 
> Regards
> Lars
> P.S.: If you want to be added or removed from the CC list please reply 
> privately
> 
> == Dial-in Information ==
> 
> ## Meeting time
> 15:00 - 16:00 UTC
> Further International meeting times: 
> https://www.timeanddate.com/worldclock/meetingdetails.html?year=2019=9=5=15=0=0=225=224=24=179=136=37=33
> 
> ## Dial in details
> Web: https://www.gotomeet.me/larskurth
> 
> You can also dial in using your phone.
> Access Code: 906-886-965
> 
> China (Toll Free): 4008 811084
> Germany: +49 692 5736 7317
> Poland (Toll Free): 00 800 1124759
> United Kingdom: +44 330 221 0088
> United States: +1 (571) 317-3129
> 
> More phone numbers
> Australia: +61 2 9087 3604
> Austria: +43 7 2081 5427
> Argentina (Toll Free): 0 800 444 3375
> Bahrain (Toll Free): 800 81 111
> Belarus (Toll Free): 8 820 0011 0400
> Belgium: +32 28 93 7018
> Brazil (Toll Free): 0 800 047 4906
> Bulgaria (Toll Free): 00800 120 4417
> Canada: +1 (647) 497-9391
> Chile (Toll Free): 800 395 150
> Colombia (Toll Free): 01 800 518 4483
>  Czech Republic (Toll Free): 800 500448
> Denmark: +45 32 72 03 82
> Finland: +358 923 17 0568
> France: +33 170 950 594
> Greece (Toll Free): 00 800 4414 3838
> Hong Kong (Toll Free): 30713169
> Hungary (Toll Free): (06) 80 986 255
> Iceland (Toll Free): 800 7204
> India (Toll Free): 18002669272
> Indonesia (Toll Free): 007 803 020 5375
> Ireland: +353 15 360 728
> Israel (Toll Free): 1 809 454 830
> Italy: +39 0 247 92 13 01
> Japan (Toll Free): 0 120 663 800
> Korea, Republic of (Toll Free): 00798 14 207 4914
> Luxembourg (Toll Free): 800 85158
> Malaysia (Toll Free): 1 800 81 6854
> Mexico (Toll Free): 01 800 522 1133
> Netherlands: +31 207 941 377
> New Zealand: +64 9 280 6302
> Norway: +47 21 93 37 51
> Panama (Toll Free): 00 800 226 7928
> Peru (Toll Free): 0 800 77023
> Philippines (Toll Free): 1 800 1110 1661
> Portugal (Toll Free): 800 819 575
> Romania (Toll Free): 0 800 410 029
> Russian Federation (Toll Free): 8 800 100 6203
> Saudi Arabia (Toll Free): 800 844 3633
> Singapore (Toll Free): 18007231323
> South Africa (Toll Free): 0 800 555 447
> Spain: +34 932 75 2004
> Sweden: +46 853 527 827
> Switzerland: +41 225 4599 78
> Taiwan (Toll Free): 0 800 666 854
> Thailand (Toll Free): 001 800 011 023
> Turkey (Toll Free): 00 800 4488 23683
> Ukraine (Toll Free): 0 800 50 1733
> United Arab Emirates (Toll Free): 800 044 40439
> Uruguay (Toll Free): 0004 019 1018
> Viet Nam (Toll Free): 122 80 481
> 
> First GoToMeeting? Let's do a quick system check:
> https://link.gotomeeting.com/system-check
> 
> 
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
___
Xen-devel mailing list

Re: [Xen-devel] More questions about Xen memory layout/usage, access to guest memory

[Rich Persaud]

> On Aug 22, 2019, at 09:51, Andrew Cooper  wrote:
> 
>> On 22/08/2019 03:06, Johnson, Ethan wrote:
>> 
>> For HVM, obviously anything that can't be virtualized natively by the 
>> hardware needs to be emulated by Xen/QEMU (since the guest kernel isn't 
>> expected to be cooperative to issue PV hypercalls instead); but I would 
>> expect emulation to be limited to the relatively small subset of the ISA 
>> that VMX/SVM can't natively virtualize. Yet I see that x86_emulate.c 
>> supports emulating just about everything. Under what circumstances does 
>> Xen actually need to put all that emulation code to use?
> 
> Introspection, as I said earlier, which is potentially any instruction.

Could introspection-specific emulation code be disabled via KConfig?

Rich

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] Code of Conduct

[Rich Persaud]

On Aug 16, 2019, at 07:19, George Dunlap  wrote:
> 
> On 8/15/19 6:23 PM, Rich Persaud wrote:
>>> On Aug 9, 2019, at 13:48, Lars Kurth  wrote:
>>> 
>>> Hi all,
>> 
>> Hi Lars,
>> 
>>> 
>>> Following the discussion we had at the Developer Summit (see 
>>> https://wiki.xenproject.org/wiki/Design_Sessions_2019#Community_Issues_.2F_Improvements_-_Communication.2C_Code_of_Conduct.2C_etc.
>>>  for notes) I put together a draft for the Code of Conduct which can be 
>>> found here as well as inlined below
>>> https://docs.google.com/document/d/1NnWdU_VnC1N_ZzxQG6jU9fnY2GPVCcfPJT5KY61WXJM/edit?usp=sharing
>>>  
>>> 
>>> It is based on the LF Events CoC as we agreed on (the diff is attached). I 
>>> took the scope and enforcement sections from 
>>> https://www.contributor-covenant.org/version/1/4/code-of-conduct.html and 
>>> simplified it rather than inventing something new.
>> 
>> Is there precedent for applying a legal contract (Code of Conduct) that was 
>> designed for physical space (conference event) to an online context?   Is 
>> there an existing Code of Conduct that was legally designed for a similar, 
>> online open-source community context, e.g. operating system or hypervisor or 
>> other systems-level software dev?
> 
> This is sort of a strange question.
> 
> Generally speaking, there was a link Lars pointed to in an earlier
> thread in preparation for this, making two suggestions about adopting a CoC:
> 
> 1. Don't create your own CoC from scratch.  Learn from other people's
> experiences, mistakes, and so on, rather than re-inventing the wheel.
> This will hopefully reduce the chance of re-hashing mistakes other
> communities have made.
> 
> 2. Don't copy-and-paste a CoC unmodified from another project.  Consider
> it, adapt it to your own community culture and situation.  This makes
> sure that the CoC is not a tick-box exercise, but that people in your
> community have thoughfully considered various issues and genuinely
> decided to commit to them.
> 
> I think both of those bits of advice are good; and it appears to me that
> this is exactly what Lars (with input from a number of others) has done.
> 
> There are two things that we want, in general:
> 
> 1. To cast a vision for what ideal contributor behavior should be
> 
> 2. To set a bar for minimum acceptable behavior, and a way for excluding
> people whose behavior consistently falls below that bar.
> 
> One area in particular where Lars thought other CoCs were weak was in
> trying to combine #1 and #2.  They need different responses.  #1 needs
> encouragement and vision.  #2 needs teeth: We need to be able to apply
> penalties and exclude people.
> 
> As a result, Lars has suggested (and many people have agreed), that we
> separate the two functions.  This document is about #2, not #1.  We plan
> to do #1 after #2 is completed.
> 
>>> # Expected Behavior
>>> All Xen Project community members are expected to behave in accordance with 
>>> professional standards, with both the Xen Project Code of Conduct as well 
>>> as their 
>>> respective employer’s policies governing appropriate workplace behavior, 
>>> and 
>>> applicable laws.
>> 
>> In the x86 community call where this was first discussed, I suggested that 
>> we try to define desirable behavior, which we would like to incentivize and 
>> promote.   In this current draft, we have a single sentence on positive 
>> behavior, with inclusion-by-reference to:
> 
> We plan on doing this, but in another document.
> 
>> If incorporation-by-reference is not sufficient, e.g. if we will maintain a 
>> blacklist of unacceptable behavior for collaborative, online open-source 
>> development, do we also need a whitelist of acceptable behavior?  Within Xen 
>> source code, we have been moving away from blacklists towards whitelists.
> 
> Unlike hypercalls, all human behavior cannot be enumerated; and if it
> could, 100% certainty cannot be obtained about what a certain behavior
> is, or even exactly what did or did not happen.  No matter what we write
> down, at some point, you're just going to have to either trust the
> people making the decisions.
> 
>>> # Unacceptable Behavior
>>> Harassment will not be tolerated in the Xen Project Community in any form, 
>>> including but not limited to harassment based on gender, gender identity 
>>> and 
>>> expression, sexual orientation, disability, physical appearance, body size, 
>>> race, 
>>> age, religion, ethnicity, nationality, level of exp

Re: [Xen-devel] [win-pv-devel] [RFC] Code of Conduct

[Rich Persaud]

> On Aug 15, 2019, at 14:46, Lars Kurth  wrote:
>> On 15 Aug 2019, at 19:27, Rich Persaud  wrote:
>> On Aug 15, 2019, at 14:01, Lars Kurth  wrote:
>> 
>>> Hi Rich,
>>> 
>>> thanks for the feedback. I am going to 
>>> 
>>> On 15/08/2019, 18:23, "Rich Persaud"  wrote:
>>> 
>>>> On Aug 9, 2019, at 13:48, Lars Kurth  wrote:
>>>> 
>>>> Hi all,
>>> 
>>>   Hi Lars,
>>> 
>>>> 
>>>> Following the discussion we had at the Developer Summit (see 
>>>> https://wiki.xenproject.org/wiki/Design_Sessions_2019#Community_Issues_.2F_Improvements_-_Communication.2C_Code_of_Conduct.2C_etc.
>>>>  for notes) I put together a draft for the Code of Conduct which can be 
>>>> found here as well as inlined below
>>>> https://docs.google.com/document/d/1NnWdU_VnC1N_ZzxQG6jU9fnY2GPVCcfPJT5KY61WXJM/edit?usp=sharing
>>>> 
>>>> It is based on the LF Events CoC as we agreed on (the diff is attached). I 
>>>> took the scope and enforcement sections from 
>>>> https://www.contributor-covenant.org/version/1/4/code-of-conduct.html and 
>>>> simplified it rather than inventing something new.
>>> 
>>>   Is there precedent for applying a legal contract (Code of Conduct) that 
>>> was designed for physical space (conference event) to an online context?   
>>> Is there an existing Code of Conduct that was legally designed for a 
>>> similar, online open-source community context, e.g. operating system or 
>>> hypervisor or other systems-level software dev?
>>> 
>>> If you look at 
>>> https://www.contributor-covenant.org/version/1/4/code-of-conduct.html or 
>>> many other examples, what we ended up with is almost identical. The same is 
>>> true for most other CoCs which are used as “gold standard”.
>> 
>> Thanks for the pointer, that's exactly what I was hoping to find.  Here is 
>> some text from Contributor Covenant:
>> 
>> "Instances of abusive, harassing, or otherwise unacceptable behavior may be 
>> reported by contacting the project team at [INSERT EMAIL ADDRESS]. All 
>> complaints will be reviewed and investigated and will result in a response 
>> that is deemed necessary and appropriate to the circumstances. The project 
>> team is obligated to maintain confidentiality with regard to the reporter of 
>> an incident. Further details of specific enforcement policies may be posted 
>> separately.
>> Project maintainers who do not follow or enforce the Code of Conduct in good 
>> faith may face temporary or permanent repercussions as determined by other 
>> members of the project’s leadership."
>> 
>> This is different from the proposed CoC, because:
>> 
>> (a) repercussions are not specified, i.e. they can be contextual
>> (b) there is a confidentiality provision
>> (c) decisions are made by open-source project leadership, not a separate 
>> "CoC team" with TBD members, electoral process and governance 
>> 
>> Can Xen Project adopt Contributor Covenant directly?  It has a large base of 
>> adopters, including Intel and Google projects, so we would benefit from 
>> upstream improvements as the CoC is tested in the real world:  
>> https://www.contributor-covenant.org/adopters
> 
> We most definitely could and I am open to the idea. However, when Linux 
> adopted it, there was significant controversy because of the origin of the 
> Contributor Covenant
> 
> See https://itsfoss.com/linux-code-of-conduct/
> 
> I am not sure what the risk would be if we followed Linux
> 
> However, we can address all of the above with what we have: The section you 
> quoted was indeed from the covenant (see attribution) and I simply modified 
> it based on the discussion we had at the summit. 
> 
> 
> a) We could leave the repercussion section out - I think it is clearer to 
> have one, but we can clearly debate the pros and cons of not having one
> b) There is a confidentiality provision: "The Xen Project’s CoC team is 
> obligated to maintain confidentiality with regard to the reporter of an 
> incident."
> c) In the design session at the summit the present project leadership team 
> members felt we should have a CoC team, which is why I changed it
> 
> In any case, the Covenant suggested to customise the template to our needs. 
> And that's what I have done.
> 
> It was also interesting that when I started with the LF events CoC, I still 
> ended up with something very similar to most of the other CoCs out there

Differenc

Re: [Xen-devel] [RFC] Code of Conduct

[Rich Persaud]

On Aug 15, 2019, at 14:01, Lars Kurth  wrote:
> 
> Hi Rich,
>  
> thanks for the feedback. I am going to
>  
> On 15/08/2019, 18:23, "Rich Persaud"  wrote:
>  
> > On Aug 9, 2019, at 13:48, Lars Kurth  wrote:
> >
> > Hi all,
>
> Hi Lars,
>
> >
> > Following the discussion we had at the Developer Summit (see 
> https://wiki.xenproject.org/wiki/Design_Sessions_2019#Community_Issues_.2F_Improvements_-_Communication.2C_Code_of_Conduct.2C_etc.
>  for notes) I put together a draft for the Code of Conduct which can be found 
> here as well as inlined below
> > 
> https://docs.google.com/document/d/1NnWdU_VnC1N_ZzxQG6jU9fnY2GPVCcfPJT5KY61WXJM/edit?usp=sharing
> >
> > It is based on the LF Events CoC as we agreed on (the diff is 
> attached). I took the scope and enforcement sections from 
> https://www.contributor-covenant.org/version/1/4/code-of-conduct.html and 
> simplified it rather than inventing something new.
>
> Is there precedent for applying a legal contract (Code of Conduct) that 
> was designed for physical space (conference event) to an online context?   Is 
> there an existing Code of Conduct that was legally designed for a similar, 
> online open-source community context, e.g. operating system or hypervisor or 
> other systems-level software dev?
>  
> If you look at 
> https://www.contributor-covenant.org/version/1/4/code-of-conduct.html or many 
> other examples, what we ended up with is almost identical. The same is true 
> for most other CoCs which are used as “gold standard”.

Thanks for the pointer, that's exactly what I was hoping to find.  Here is some 
text from Contributor Covenant:

"Instances of abusive, harassing, or otherwise unacceptable behavior may be 
reported by contacting the project team at [INSERT EMAIL ADDRESS]. All 
complaints will be reviewed and investigated and will result in a response that 
is deemed necessary and appropriate to the circumstances. The project team is 
obligated to maintain confidentiality with regard to the reporter of an 
incident. Further details of specific enforcement policies may be posted 
separately.
Project maintainers who do not follow or enforce the Code of Conduct in good 
faith may face temporary or permanent repercussions as determined by other 
members of the project’s leadership."

This is different from the proposed CoC, because:

  (a) repercussions are not specified, i.e. they can be contextual
  (b) there is a confidentiality provision
  (c) decisions are made by open-source project leadership, not a separate "CoC 
team" with TBD members, electoral process and governance 

Can Xen Project adopt Contributor Covenant directly?  It has a large base of 
adopters, including Intel and Google projects, so we would benefit from 
upstream improvements as the CoC is tested in the real world:  
https://www.contributor-covenant.org/adopters

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] Code of Conduct

[Rich Persaud]

> On Aug 9, 2019, at 13:48, Lars Kurth  wrote:
> 
> Hi all,

Hi Lars,

> 
> Following the discussion we had at the Developer Summit (see 
> https://wiki.xenproject.org/wiki/Design_Sessions_2019#Community_Issues_.2F_Improvements_-_Communication.2C_Code_of_Conduct.2C_etc.
>  for notes) I put together a draft for the Code of Conduct which can be found 
> here as well as inlined below
> https://docs.google.com/document/d/1NnWdU_VnC1N_ZzxQG6jU9fnY2GPVCcfPJT5KY61WXJM/edit?usp=sharing
>  
> 
> It is based on the LF Events CoC as we agreed on (the diff is attached). I 
> took the scope and enforcement sections from 
> https://www.contributor-covenant.org/version/1/4/code-of-conduct.html and 
> simplified it rather than inventing something new.

Is there precedent for applying a legal contract (Code of Conduct) that was 
designed for physical space (conference event) to an online context?   Is there 
an existing Code of Conduct that was legally designed for a similar, online 
open-source community context, e.g. operating system or hypervisor or other 
systems-level software dev?


> You can provide feedback by commenting on the google doc or by replying to 
> the in-lined version below. 
> I expect it will some more discussion to get consensus. 
> 
> Note that I am not very attached to some of the terms, such as "Xen Project 
> CoC  Team" and in some cases "participant" should probably be replaced by 
> community 
> members. 
> 
> But I felt, we should have something more concrete to discuss compared to 
> previous discussions.
> 
> A Code of Conduct is a project wide policy change: thus, all subprojects 
> lists are CC'ed
> 
> Regards
> Lars
> 
> Here is the actual text
> ---
> # Our Pledge
> In the interest of fostering an open and welcoming environment, we as 
> community 
> members of the Xen Project pledge to making participation in our project and 
> our 
> community a harassment-free experience for everyone.
> 
> We believe that a Code of Conduct can help create a harassment-free 
> environment, 
> but is not sufficient to create a welcoming environment on its own: guidance 
> on creating 
> a welcoming environment, how to communicate in an effective and friendly way, 
> etc. 
> can be found .
> 
> # Scope
> This Code of Conduct applies within all Xen Project project spaces, and it 
> also applies 
> when an individual is representing the Xen Project or its community in public 
> spaces. 
> Examples of representing the Xen Project include using an official project 
> email address, 
> posting via an official social media account, or acting as an appointed 
> representative 
> at an online or offline event. 
> 
> # Expected Behavior
> All Xen Project community members are expected to behave in accordance with 
> professional standards, with both the Xen Project Code of Conduct as well as 
> their 
> respective employer’s policies governing appropriate workplace behavior, and 
> applicable laws.

In the x86 community call where this was first discussed, I suggested that we 
try to define desirable behavior, which we would like to incentivize and 
promote.   In this current draft, we have a single sentence on positive 
behavior, with inclusion-by-reference to:

- professional standards
- corporate policy
- city, state and national/federal law

If it is sufficient to define acceptable behavior by reference to external 
governance institutions and cultural practices, can we do the same for 
unacceptable behavior, i.e. anything that violates the above?

If incorporation-by-reference is not sufficient, e.g. if we will maintain a 
blacklist of unacceptable behavior for collaborative, online open-source 
development, do we also need a whitelist of acceptable behavior?  Within Xen 
source code, we have been moving away from blacklists towards whitelists.


> # Unacceptable Behavior
> Harassment will not be tolerated in the Xen Project Community in any form, 
> including but not limited to harassment based on gender, gender identity and 
> expression, sexual orientation, disability, physical appearance, body size, 
> race, 
> age, religion, ethnicity, nationality, level of experience, education, or 
> socio-economic status or any other status protected by laws in jurisdictions 
> in 
> which community members are based. Harassment includes the use of abusive, 
> offensive or degrading language, intimidation, stalking, harassing 
> photography 
> or recording, inappropriate physical contact, sexual imagery and unwelcome 
> sexual advances, requests for sexual favors, publishing others' private 
> information such as a physical or electronic address without explicit 
> permission

Picking one item at random:  would a conference-originated blacklist 
prohibition be appropriate for online open-source development?  E.g. if 
someone's email address were included in a xen-devel thread (on the cc line), 
without obtaining explicit permission, would that be unacceptable behavior for 
a Xen developer?  That could disqualify 

[Xen-devel] Xen Summit 2019 Design Session - Nested Virtualization

[Rich Persaud]

Session notes attached in markdown and PDF format, please revise as needed.Rich

xen-summit-2019-nested-virt-design-session.pdf
Description: Adobe PDF document
# Nested Virtualization Design Session
Xen Design and Developer Summit, [11 July 2019](https://design-sessions.xenproject.org/uid/discussion/disc_1NVcnOZyDZM1LpQbIsJm/view)

**Related Presentations**

- (2019) Jürgen Groß, [Support of PV devices in nested Xen](https://youtube.com/watch?v=HA_teA6hV7c)
- (2019) Christopher Clark and Kelli Little, [The Xen-Blanket](https://youtube.com/watch?v=i5w9sF9VerE)
- (2018) Ian Pratt, [Hypervisor Security: Lessons Learned](https://youtube.com/watch?v=bNVe2y34dnM) (uXen)
- (2018) David Weston, [Windows: Hardening with Hardware](https://youtube.com/watch?v=8V0wcqS22vc) (Credential Guard)

**Use Cases**

- Xen on Xen, some work was done for the Shim (Meltdown mitigation).
- Xen on another hypervisor, involves teaching Xen how to use enlightenments from other hypervisors.  
- Qubes runs Xen on AWS bare-metal instances that use Nitro+KVM, mostly works.
- Windows Credential Guard (Hyper-V on Xen)
- Bromium Type-2 uXen in Windows and Linux guests on Xen

**Issues**

1.  Need to be careful with features, eg. Ballooning down memory.
2. Dom0 is exposed to things that it should not see.
3. Nested virtualization works when both L0 and L1 agree, e.g Xen on Xen.  When replacing Xen with another hypervisor, all falls apart.
4. Need more audit checks for what the VM can read or write, i.e. guest requirements.
5. Virtual vmentry and vmexit emulation "leaking", doesn't cope well.
6. Context switching bug fixed a while ago: doesn't understand AFR(?) loading or whether it should do it or leave alone.
7. Missing instructions to virtualize vmexit.
8. Enlightened EPT shootdown is easy on top of the other features working.

**Dependent on CPUID and MSR work**

1. Auditing of changes.  Can then fix virtual vmentry and vmexit, one bit at a time.  Once all features are covered, it should work fine.
2. hwcaps: needed to tell the guest about the security state of the hardware.
3. Reporting CPU topology representation to guests, which is blocking core-scheduling work (presented by Juergen at Xen Summit)
4. Andrew is working on the prerequisites for the policy.

**Validation of Nested Virtualization**

1.  First priority is correctness. 
2. Second priority is performance.
3. There is a unit testing prototype which exercises vmxon, vmxoff instructions.
4. Depends on regression testing, which depends upon (a) formal security support, (b) approval of the Xen security team.
5. Other hypervisors must be tested with Xen.

**Guest State**

Nesting requires merge of L1 and L0 state.

1.  AMD interface is much easier: it has "clean bits": if any bit is clear, must resync.  Guest state is kept separately.
2. Intel guest state is kept in an opaque blob in memory, with special instructions to access it.  Memory layout in RAM is unknown, behavior changes with microcode updates and there are 150 pages of relevant Intel manuals.
4. Bromium does some fun stuff to track guest state in software, poisoning RAM and then inspecting it, which is still faster than Intel's hardware-based VMCS shadowing. L1 hypervisor (Type-2 uXen): https://github.com/openxt/uxen
5. Viridian emulates the AMD way, i.e. Microsoft has Intel bits formatted in an AMD-like structure, then L0 translates the AMD structure into Intel's opaque blob.

**Secure Variable Storage**

1. Need an agreed sane way for multiple hypervisors to handle it, eg. a pair of ioports, translation from VMX, guest handles the interrupts via a standard ioport interception to secondary emulator: tiny.
2. Easy case:  ioports + memory page for data.
3. Citrix XenServer has a closed-source implementation (varstored?)

**Interface for nested PV devices**

PV driver support currently involves grants and interrupts.

Requirements:

1. Should Xen's ABI include hypercall nesting level?
2. Each layer of nesting must apply access control decisions to the operation invoked by its guest.
3. Brownfield: if Xen and other L1 hypervisors must be compatible with existing Xen bare-metal deployments, the L0 hypervisor must continue to support grants, events and xenstore.
4. Greenfield: if the L0 hypervisor can be optimized for nesting, then PV driver mechanisms other than grants, events and xenstore could be considered.

Live migration with PCI graphics (has been implemented on AWS):

- need to make it look the same, regardless of nesting level
- 1 or more interrupts
- 1 or shared pages of RAM
- share xenstore
- virtio guest physical address space DMA: done right
- _*need to get rid of domid*_ as the endpoint identifier

Access Control:

Marek: use virtio?
David: do whatever you like in L1
Juergen: new "nested hypercall", to pass downwards an opaque payload
David: how does access control work with that approach?

Christopher: xenblanket RFC series implements support for one level of nesting. Its implementation below the hypercall 

Re: [Xen-devel] Design session notes: Build system gripe session

[Rich Persaud]

> On Jul 15, 2019, at 09:27, George Dunlap  wrote:
> ...
> https://hackmd.io/mAGT5bU9T6-aXJVj88deYw
> ...
> 
> 3. meta virtualization build system (yocto) needs to pull simlink
> tricks, xen's build system stomp on that

Are more details available on this item?

This script can be used to cross-compile Xen for x86 or Arm, using OE/Yocto 
meta-virtualization:  https://github.com/dozylynx/oe-build-xen

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Terminology for "guest" - Was: [PATCH] docs/sphinx: Introduction

[Rich Persaud]

On Aug 8, 2019, at 06:49, Jan Beulich  wrote:
> 
>> On 08.08.2019 11:13, Julien Grall wrote:
>> Hi Jan,
>> 
>>> On 08/08/2019 10:04, Jan Beulich wrote:
 On 08.08.2019 10:43, Andrew Cooper wrote:
> On 08/08/2019 07:22, Jan Beulich wrote:
>> On 07.08.2019 21:41, Andrew Cooper wrote:
>> --- /dev/null
>> +++ b/docs/glossary.rst
>> @@ -0,0 +1,37 @@
>> +Glossary
>> +
>> +
>> +.. Terms should appear in alphabetical order
>> +
>> +.. glossary::
>> +
>> +   control domain
>> + A :term:`domain`, commonly dom0, with the permission and
>> responsibility
>> + to create and manage other domains on the system.
>> +
>> +   domain
>> + A domain is Xen's unit of resource ownership, and generally has
>> at the
>> + minimum some RAM and virtual CPUs.
>> +
>> + The terms :term:`domain` and :term:`guest` are commonly used
>> + interchangeably, but they mean subtly different things.
>> +
>> + A guest is a single virtual machine.
>> +
>> + Consider the case of live migration where, for a period of
>> time, one
>> + guest will be comprised of two domains, while it is in transit.
>> +
>> +   domid
>> + The numeric identifier of a running :term:`domain`.  It is
>> unique to a
>> + single instance of Xen, used as the identifier in various APIs,
>> and is
>> + typically allocated sequentially from 0.
>> +
>> +   guest
>> + See :term:`domain`
> 
> I think you want to mention the usual distinction here: Dom0 is,
> while a domain, commonly not considered a guest.
 
 To be honest, I had totally forgotten about that.  I guess now is the
 proper time to rehash it in public.
 
 I don't think the way it currently gets used has a clear or coherent set
 of rules, because I can't think of any to describe how it does get used.
 
 Either there are a clear and coherent (and simple!) set of rules for
 what we mean by "guest", at which point they can live here in the
 glossary, or the fuzzy way it is current used should cease.
>>> 
>>> What's fuzzy about Dom0 not being a guest (due to being a part of the
>>> host instead)?
>> Dom0 is not part of the host if you are using an hardware domain.
> 
> It's still the control domain then, and hence still part of the host.

With disaggregation and dom0less (how might we describe that term in the 
intro?) for edge/embedded Xen systems, there could be a mode where the control 
domain has never had privilege over the domain that handles the physical TPM, 
or the provider of the virtual TPM:  

  https://lists.gt.net/xen/devel/557782

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tests/cpu-policy: fix format-overflow warning by null terminating strings

[Rich Persaud]

> On Jul 31, 2019, at 04:11, Jan Beulich  wrote:
> 
>> On 31.07.2019 02:22, Dario Faggioli wrote:
>> Jan's example above, seem to compile **without any warnings** for me as
>> well. If I add a main(), I can even get the code above to print the
>> content of the array.
>> 
>> And yet, building the tools without a patch like Christoper's one
>> (which was also what I was using locally, and raised to Andy), I get:
> 
> Sure - I'm aware that we're using a (potentially heavily) patched gcc.
> Hence my question to Christopher whether he's using a plain one.

The Xen test-cpu-policy.c build error and workaround reported by Christopher 
can be reproduced by building Xen staging against upstream OpenEmbedded/Yocto 
meta-virtualization master and GCC.  I've tested on Devuan Ascii with 
"oe-build-xen x86-64 staging master":

  http://github.com/dozylynx/oe-build-xen

If there is an upstream GCC compiler issue, Xen will not be the only affected 
project.  We can expect new data as other projects start building with recent 
GCC versions.  In the meantime, the workaround can unblock Xen builds.

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] Adding Intel TXT maintainter

[Rich Persaud]

> On Jul 29, 2019, at 10:45, Jan Beulich  wrote:
> 
>> On 29.07.2019 16:39, Lukasz Hawrylko wrote:
>> Support for Intel TXT has orphaned status right now because
>> no active maintainter is listed. Adding myself as active maintainter,
>> so it could be reverted to supported state.
> 
> Which you should then do ...
> 
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -240,6 +240,7 @@ S:Maintained
>>  F:tools/golang
>> 
>>  INTEL(R) TRUSTED EXECUTION TECHNOLOGY (TXT)
>> +M:Lukasz Hawrylko 
>>  S:Orphaned
> 
> ... right here. The question is what new state you want to put
> it into. But it was suggested anyway that you add yourself as
> reviewer first, at which point the new state would probably be
> "Odd Fixes".

There is no status for Intel TXT documented in SUPPORT.md [1].  What's the 
established mapping between "Odd Fixes" in Maintainers and support status in 
SUPPORT.md?

At present, MAINTAINERS generalizes "tboot support" to "Intel TXT".  This will 
change with TrenchBoot (discussed at Xen Summit) [2] and "secure launch" [3] 
for the Linux kernel, which will support Intel TXT and AMD SKINIT variants of 
DRTM.  At that point, there will be tboot and TrenchBoot for Intel TXT, with 
different Xen maintainers and potentially different levels of support.

We will eventually need to document Xen's DRTM status on Intel, AMD and 
Arm/Qualcomm, in both SUPPORT.md and MAINTAINERS.

Rich


[1] SUPPORT.md, 
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=SUPPORT.md;hb=refs/heads/stable-4.12

[2] TrenchBoot, https://youtube.com/watch?v=f0LZFSq4Ack

[3] kernel_info, https://lkml.org/lkml/2019/5/24/330

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] docs/sphinx: todo/wishlist

[Rich Persaud]

> On Jul 22, 2019, at 15:20, Andrew Cooper  wrote:
> 
> a.k.a. (at least in this form) Andrew's "work which might be offloadable to
> someone else" list.
> 
> Signed-off-by: Andrew Cooper 
> ---
> CC: George Dunlap 
> CC: Ian Jackson 
> CC: Jan Beulich 
> CC: Stefano Stabellini 
> CC: Tim Deegan 
> CC: Wei Liu 
> CC: Julien Grall 
> CC: Lars Kurth 
> CC: Paul Durrant 
> CC: Roger Pau Monné 
> 
> RFC for obvious reasons.
> 
> A rendered version of this can be found at:
> https://andrewcoop-xen.readthedocs.io/en/docs-wishlist/misc/wishlist.html
> 
> During XenSummit in Chicago, it was expressed several times that having some
> todo lists would be a benefit, to help coordinate work in related areas.
> 
> Here is an attempt to start one.  For now, it covers one single
> item (xenstored's use of non-stable APIs) to get some feedback about the
> general approach.  I have plenty to get stuck into in Xen itself if this way
> of expressing them isn't deemed unacceptable.
> 
> As for the wishlist itself, I think it is important that it be restricted to
> concrete actions (i.e. already partially groomed, if you speak agile), which
> are identified problems, and suggested fixes.
> 
> In particular, I don't think it is appropriate to devolve into a bullet point
> list of new features, or tasks like "document $whotsit".  It should be
> restricted to things which are real problems, on existing systems, which have
> some forward plan of action.  That way, any developer should be able to
> cross-reference at least at a high level, and see if there are areas of
> overlapping work, or whether a slightly tweaked approach might be suitable for
> multiple areas.
> 
> Anyway - thoughts from the peanut gallery?

Would you consider a permissive, documentation-oriented license, e.g. Creative 
Commons CC-BY 4.0, for Xen's Sphinx/RST documentation?
https://creativecommons.org/licenses/by/4.0/

As Xen moves beyond cloud computing into multi-vendor, edge/embedded supply 
chains [1], the audience and context for Xen's technical docs is expanding.  
Beyond operating system user/dev/admin, there may be: nested hypervisor 
user/dev/admin, certification (FuSA), security, firmware/device/accelerator 
dev, processor architects, formal verification (e.g. TLA+ models), ecosystem 
building (e.g. blogs, books, videos, training, research) and commercial 
maintenance manuals for long-lived products with multiple Xen configs and 
embedded processors.

A permissive license would encourage reuse and tailoring of Xen docs.  With 
healthy OSS projects, there will remain an incentive to contribute long-lived 
improvements upstream, even if those improvements are not mandated by the CC 
license. The Xen wiki license is historically CC-BY-SA 3.0, so that content 
would be incompatible with CC-BY 4.0.  But Xen's Sphinx/RST docs appear to be 
focused on new content, so we have an opportunity to choose a license which 
reflects current community priorities.

Rich

[1] 
https://dornerworks.com/blog/high-performance-space-computing-platform-nasa-sbir

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tboot: remove maintainers and declare orphaned

[Rich Persaud]

(cc Intel and tboot-devel)

Hi Roger,

Thanks for your interest in documenting the status of maintenance for Intel TXT 
support in Xen.  Intel TXT and Xen are deployed in production today by OpenXT 
and QubesOS for boot integrity.  Xen was a pioneering adopter of DRTM, almost a 
decade ago, but mainstream enterprise computing is now catching up with the May 
2019 release of Windows 10 SystemGuard.  It would be nice to avoid "orphaning" 
one of Xen's competitive advantages in 2019.

> On Jul 25, 2019, at 09:51, Roger Pau Monne  wrote:
> 
> Gang Wei Intel email address has been bouncing for some time now,

Gang Wei's replacement is Lukasz Hawrylko, who posted on March 6, 2019:
https://lists.gt.net/xen/devel/546401

Could you include Lukasz patch, along with Julien's requested formatting 
changes, in your update to the MAINTAINERS file?  As a new Xen maintainer and 
contributor, Lukasz may not yet be familiar with the procedures and practices 
of the Xen community.  We can welcome his new maintainership role without 
dropping support for a feature, that (a) he is maintaining, (b) is used by Xen.

> and
> the other maintainer is non-responsive to patches [0], so remove
> maintainers and declare INTEL(R) TRUSTED EXECUTION TECHNOLOGY (TXT)
> orphaned.
> 
> [0] https://lists.xenproject.org/archives/html/xen-devel/2019-05/msg00563.html

Since we have at least one Intel maintainer, Lukasz, the feature need not be 
orphaned.  If Shawn is not responding to the request to confirm Lukasz as 
maintainer, the Xen community has multiple communication channels with Intel.  
Pragmatically, a review of the tboot-devel archives shows that Lukasz is 
working on tboot development.  

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] After upgrade to Xen 4.12.0 iommu=no-igfx

[Rich Persaud]

> On Jul 19, 2019, at 15:31, Roman Shaposhnik  wrote:
> 
> Hi!
> 
> we're using Xen on Advantech ARK-2250 Embedded Box PC:
>
> https://www.elmark.com.pl/web/uploaded/karty_produktow/advantech/ark-2250l/ark-2250l_instrukcja-uzytkownika.pdf

Roman, 

Good to see Xen being used on fanless devices.  Does the AMI BIOS for the i7 
6600U Skylake CPU [1] variant of ARK-2250 [2] support Intel TXT DRTM and 
discrete TPM, which would enable boot integrity [3] protection for Xen, 
read-only dom0 and stateless VMs?  Boot integrity is valuable on edge devices.

Rich


[1] CPU spec: 
https://ark.intel.com/content/www/us/en/ark/products/88192/intel-core-i7-6600u-processor-4m-cache-up-to-3-40-ghz.html

[2] PC spec: 
https://www.advantech.com/products/ark-2000_series_embedded_box_pcs/ark-2250l/mod_66ebc4e0-9a0c-489c-96a5-70a8054e9037

[3] TrenchBoot, Xen Summit 2019, https://youtube.com/watch?v=f0LZFSq4Ack

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] preparations for 4.12.1

[Rich Persaud]

> On Jul 23, 2019, at 17:05, Roman Shaposhnik  wrote:
> 
>> On Tue, Jul 23, 2019 at 2:00 PM Pasi Kärkkäinen  wrote:
>> 
>> Hi,
>> 
>>> On Fri, Jul 19, 2019 at 02:23:44PM +, Jan Beulich wrote:
>>> All,
>>> 
>>> the release is due in early August. Please point out backports you
>>> find missing from the respective staging branch, but which you
>>> consider relevant. The one commit I've queued already on top of
>>> what was just pushed is
>>> 
>>> ec2ab491b5x86/ept: pass correct level to p2m_entry_modify
>>> 
>> 
>> I'd like to request backport of the following commit for 4.12.1:
>> 
>> "libxl: fix pci device re-assigning after domain reboot":
>> commit  c19434d9284e93e6f9aaec9a70f5f361adbfaba6
>> 
>> https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=c19434d9284e93e6f9aaec9a70f5f361adbfaba6
> 
> FWIW: I'd like to second that.
> 
> Thanks,
> Roman.

Another vote for this backport, for rebooting stateless, measured network 
driver domains which have more than one NIC.

Rich

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] xen: Add .astylerc for automated style-formatting

[Rich Persaud]

On Jul 19, 2019, at 09:31, Julien Grall  wrote:
>> On 19/07/2019 14:24, Julien Grall wrote:
>> Hi Tamas,
>>> On 19/07/2019 14:14, Tamas K Lengyel wrote:
 On Fri, Jul 19, 2019 at 7:11 AM Julien Grall  wrote:
 
 Hi Tamas,
 
> On 19/07/2019 14:00, Tamas K Lengyel wrote:
>> On Fri, Jul 19, 2019 at 2:43 AM Julien Grall  
>> wrote:
>> 
>> Hi Tamas,
>> 
>> On 18/07/2019 18:48, Tamas K Lengyel wrote:
   - Line 1025: The tools needs to be able to deal 
 for_each_vcpu(...) & co.
>>> 
>>> These can be made OK by adding braces. Other than that the only way I
>>> found to make it not change the indentation is to add the comment "/*
>>> *INDENT-OFF* */" before the block and "/* *INDENT-ON* */" afterwards.
>> 
>> None of them looks really appealing because it means astyle will not 
>> correctly
>> indent if the user does not add braces or comments.
>> 
>> Could astyle be easily modified to recognize foreach macros?
> 
> Not that I'm aware of. If you don't want to manually annotate files
> with unsupported macros then just exclude those files from astyle. I
> wouldn't recommend adding this to the CI for all files, only for those
> that their respective maintainers have confirmed to conform to the
> style and want to enforce it going forward.
 
 So a couple use of an unsupported macros would make impossible to enforce 
 the
 coding style. This is not a very ideal position to be in.
 
 _if_ we are going to adopt astyle then we need to be able to enforce it on 
 every
 Xen files long-term. If it is not possible to do it with astyle, then 
 maybe this
 is not the right tool to use.
 
 For instance, I know that tools such as clang-format is able to deal with
 foreach macros.
>>> 
>>> If there are better tools then sure, I don't really mind using
>>> something else. I just don't have time to do the manual style check
>>> back-and-forth anymore, so the sooner we have something in place the
>>> better.
>> I totally agree we need a tool so the reviewer can free-up some time to 
>> focus on more important things. However, I think we should be careful on 
>> what we adopt here.
>> Similar to Andrew, I am open with modifying the coding style to help the 
>> automatic style check. But I am not happy to disable automatic style on part 
>> (or entire) of files forever.
>> At the moment, clang-format feels more powerful and there are people working 
>> on it.
> 
> FYI, below a link to the clang-format changes:
> 
> https://github.com/xen-troops/Xen-Clang-format/blob/devel/clang-format.patch

Were these clang-format changes done for FuSa work?  Are they intended to be 
run within OSStest and/or Xen's Gitlab CI, which do not currently support 
OpenEmbedded/Yocto and xen-troops?

It would be helpful to have a xen-devel thread on the motivation for the 
clang-format work, the specific style being enforced (including the nuances 
discussed in this thread) and additional work needed before clang-format can 
perform automated style checking to address (a) existing Xen/Linux style 
requirements, (b) FuSa requirements.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] xen: Add .astylerc for automated style-formatting

[Rich Persaud]

> On Jul 18, 2019, at 10:43, Tamas K Lengyel  wrote:
> 
> Using astyle (http://astyle.sourceforge.net) can greatly reduce the overhead 
> of
> manually checking and applying style-fixes to source-code. The included
> .astylerc is the closest approximation of the established Xen style (including
> styles not formally spelled out by CODING_STYLE but commonly requested).
> 
> Checking the comment styles are not included in the automation.
> 
> Incorporating Xen's exception to the do-while style is only partially 
> possible,
> thus a change is proposed to the CODING_STYLE of moving the brace from "do {"
> to the next line.
> 
> Most of Xen's code-base is non-conforming at the moment: 289 files pass
> unchanged, 876 have some style issues.
> 
> Ideally we can slowly migrate the entire code-base to be conforming, thus
> eliminating the need of discussing and enforcing style issues manually on the
> mailinglist.

Thanks for taking the lead on this, Tamas.  New Xen contributors are unlikely 
to be aware of the style ambiguities discussed in this thread. A frequent topic 
on Xen monthly calls is the lack of time to perform patch reviews.  Automated 
style checking will increase the S/N ratio of xen-devel, reviewer efficiency, 
patch quality from new contributors, and style consistency across Xen trees.  
This automation will complement upcoming static analysis of Xen for functional 
safety compliance.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] golang/xenlight: Fixing compilation for go 1.11

[Rich Persaud]

> On Apr 25, 2019, at 07:41, George Dunlap  wrote:
> 
> On 4/25/19 12:40 PM, Jan Beulich wrote:
> On 18.04.19 at 15:11,  wrote:
 On 4/18/19 2:52 AM, Daniel P. Smith wrote:
 This deals with two casting issues for compiling under go 1.11:
 - explicitly cast to *C.xentoollog_logger for Ctx.logger pointer
 - add cast to unsafe.Pointer for the C string cpath
 
 Signed-off-by: Daniel P. Smith 
>>> 
>>> Reviewed-by: George Dunlap 
>>> 
>>> BTW, do you know if this will compile for older versions of go?
>>> 
>>> This should be considered for backport as well (cc'ing Jan).
>> 
>> Did you mean Ian, this being a tools patch?
> 
> I guess so.  Sorry, I didn't realize Ian was doing the tools backports.
> 
> -George

With the golang tools maintainer change completed, is anything further needed 
for this patch to be merged?

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen Project Community Call June 27th (instead of July 4th): @15:00 UTC Call for agenda items

[Rich Persaud]

> On Jun 26, 2019, at 06:45, Lars Kurth  wrote:
> 
> 
> 
> On 25/06/2019, 21:27, "Rich Persaud"  wrote:
> 
>> On Jun 25, 2019, at 16:17, Julien Grall  wrote:
>> 
>> Hi Rich,
>> 
>> On 6/25/19 8:38 PM, Rich Persaud wrote:
>>>> On Jun 25, 2019, at 12:36, Lars Kurth  wrote:
>>>> 
>>>> Hi all:
>>>> please add your agenda items. I had only ONE series which was highlighted 
>>>> as needing attention from others. Is this seriously the only one?
> 
> We had quite a few additions to the agenda. One problem I have is that 
> cryptpad history does not tell me who added an agenda item. We will have to 
> manage this appropriately in the meeting.
> 
>>> Proposed agenda item: in the absence of Jira tickets, would it be useful to 
>>> have a list (e.g. generated by a script) with the lifecycle status of all 
>>> outstanding patch series, e.g.
>>> METADATA
>>> - bug fix / improvement / refactor / cleanup / new feature
>>> - impacted Xen subsystems/components/features
>>> - targeted version of Xen
>>> - contributing person/org
>>> - relevance of patch series to the goals set by RM for the next Xen release
>>> - related patch series (with below status info)
>>> STATUS:
>>> - patch series version
>>> - date of patch series v1
>>> - no responses received + ping count + days since submission + days since 
>>> ping
>>> - reviewed with objections
>>> - reviewed without objections, awaiting ack
>>> - acked, awaiting merge
>>> From such a summary, patch series could be prioritized for review/triage in 
>>> the community call, based on uniform criteria and project-wide context.
>> 
>> While I think raising awareness of the stuck series is a good idea. I still 
>> have some concern regarding the prioritization. Who is going to consume the 
>> result of the discussion? Is it the maintainers?
> 
>   Anyone who typically answers the question raised by Lars in this thread, 
> presumably a subset of call attendees.
> 
> This would only work if there was consensus on the priority amongst the key 
> stake-holders. I believe that some limited prioritization has happened in the 
> past, e.g. for some Arm related features for Xen 4.12 where, if I recall 
> correctly you, Stefano and EPAM did this. 
> 
> Maybe we can trial this type of approach for a small number of series first. 
> At the end of the day this is about queue management. Right now, when a new 
> series comes in it ends up in one big queue (xen-devel@). Most complex series 
> have to go through a series of gates (aka code reviews from different people) 
> before they get applied and when a new version comes out the series ends up 
> in the queue again: each reviewer today prioritizes their own review queues, 
> where no-one else sees the prioritisation of other reviewers. Unless there is 
> lot of spare review capacity (which there isn't) we essentially end up in 
> grid-lock, with an ever-growing queue. We seem to have specific additional 
> complexity in Xen because most recent series, typically involve a large 
> number of reviewers.
> 
> In theory, there are several ways to address this:
> * Queue management either by a set of agreed criteria which all reviewers 
> follow or through some agreement about which series we actively try and 
> shepherd through the gates
> * We have an additional issue that in many areas we have multiple 
> reviewers/committers reviewing the same area of code, which also frequently 
> leads to slow-downs, because the multiple reviewers/committers can disagree. 
> We could look at a model where the reviewers/committers agree that one takes 
> the lead on reviewing a specific series. We could try and streamline the 
> ownership structure to create a clearer mapping.
> * The queues of each reviewer are somehow made public (assuming this is 
> possible) and we hope that the system self-regulates. Not sure this will 
> actually work
> 
> The big problem I have is that mailing lists really don't lend themselves 
> well to highlight what is going on. We have been grappling with this for 
> years and things are getting worse, not better.
> 
> In past community calls when we tried to do this with specific series, in 
> practice we ended up discovering obstacles that were concerning a specific 
> series, such as unexposed dependencies, lack of HW, specs against which to 
> review being too complex, ...
> 
> In any case, given that quite a few series were added to the agenda, maybe we 
> shouldn't talk about process in the meeting, but see whether we can unblock 
> those series

Re: [Xen-devel] Xen Project Community Call June 27th (instead of July 4th): @15:00 UTC Call for agenda items

[Rich Persaud]

> On Jun 25, 2019, at 16:17, Julien Grall  wrote:
> 
> Hi Rich,
> 
> On 6/25/19 8:38 PM, Rich Persaud wrote:
>>> On Jun 25, 2019, at 12:36, Lars Kurth  wrote:
>>> 
>>> Hi all:
>>> please add your agenda items. I had only ONE series which was highlighted 
>>> as needing attention from others. Is this seriously the only one?
>> Proposed agenda item: in the absence of Jira tickets, would it be useful to 
>> have a list (e.g. generated by a script) with the lifecycle status of all 
>> outstanding patch series, e.g.
>> METADATA
>> - bug fix / improvement / refactor / cleanup / new feature
>> - impacted Xen subsystems/components/features
>> - targeted version of Xen
>> - contributing person/org
>> - relevance of patch series to the goals set by RM for the next Xen release
>> - related patch series (with below status info)
>> STATUS:
>> - patch series version
>> - date of patch series v1
>> - no responses received + ping count + days since submission + days since 
>> ping
>> - reviewed with objections
>> - reviewed without objections, awaiting ack
>> - acked, awaiting merge
>> From such a summary, patch series could be prioritized for review/triage in 
>> the community call, based on uniform criteria and project-wide context.
> 
> While I think raising awareness of the stuck series is a good idea. I still 
> have some concern regarding the prioritization. Who is going to consume the 
> result of the discussion? Is it the maintainers?

Anyone who typically answers the question raised by Lars in this thread, 
presumably a subset of call attendees.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen Project Community Call June 27th (instead of July 4th): @15:00 UTC Call for agenda items

[Rich Persaud]

> On Jun 25, 2019, at 12:36, Lars Kurth  wrote:
> 
> Hi all:
> please add your agenda items. I had only ONE series which was highlighted as 
> needing attention from others. Is this seriously the only one?

Proposed agenda item: in the absence of Jira tickets, would it be useful to 
have a list (e.g. generated by a script) with the lifecycle status of all 
outstanding patch series, e.g.

METADATA

- bug fix / improvement / refactor / cleanup / new feature
- impacted Xen subsystems/components/features
- targeted version of Xen
- contributing person/org
- relevance of patch series to the goals set by RM for the next Xen release
- related patch series (with below status info)

STATUS:

- patch series version
- date of patch series v1
- no responses received + ping count + days since submission + days since ping
- reviewed with objections
- reviewed without objections, awaiting ack
- acked, awaiting merge

From such a summary, patch series could be prioritized for review/triage in the 
community call, based on uniform criteria and project-wide context.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Rich Persaud]

> On Jun 25, 2019, at 12:34, Lars Kurth  wrote:
> 
> On 25/06/2019, 14:47, "Andrew Cooper"  wrote:
> 
>>   On 25/06/2019 13:15, Lars Kurth wrote:
>> On 25/06/2019, 10:03, "Julien Grall"  wrote:
>> 
> The point here is that we can be flexible and creative about the way to
> maintain the docs on xen.git. But as a technology is certainly better
> than the wiki: we don't have to keep them all up-to-date with the code,
> but at least this way we have a chance (if we want to). If we leave them
> on the wiki, there is no chance.
 
 I can't see how xen.git is going to be better if "we don't have to keep 
 them
 all up-to-date".
>>> 
>>> That's because a contributor could add a patch at the end of a series to
>>> update one of the docs, even if the doc in question comes with no
>>> promises of being up-to-date.
>> 
>>   I think this is going the wrong direction. The goal of using xen.git is to 
>> try 
>>   to keep the documentation up-to-date.
>> 
>> I agree with Julien and this was also not my intention. The reason why I 
>> brought this up now is that the in-tree docs are pretty much a mess today 
>> and are stale in many ways. And they look TERRIBLE and are not easily 
>> searchable. However, Andy's latest set of patches provide an opportunity to 
>> consolidate some of the in-tree docs in a nicely rendered and searchable 
>> format.
> 
>   So the plan here is to get a consistent and uniform set of high quality
>   docs.
> 
>   As fair warning, I'm intending to be fairly strict with what goes in
>   (quality wise), because I'm going to do my best to ensure that the
>   sphinx documentation doesn't devolve into the mess that wiki or the
>   majority of docs/ currently is.
> 
> I wholeheartedly agree
> 
>> I have been focussing on process related and key developer related docs, 
>> because who maintains them is not actually an issue in theory. Everyone 
>> really ought to care, because everyone is impacted by these.
> 
>   The key point is for maintainers/reviewers to be aware of whether
>   documentation exists for the area of code being modified, and if so,
>   whether the submitted patch should also patch the documentation.
> 
> I am wondering whether this is something which could be addressed. One 
> possibility may be to have SUPPORT.md point to documentation. But that is 
> kind of assuming that SUPPORT.md works and is widely used. There may be 
> better or orthogonal ways to point to relevant docs (e.g. by pointing to docs 
> in header files and other source files). 
> 
>   Reviewers tend to be fairly good at noticing patches which also need to
>   patch docs/misc/xen-command-line.pandoc (submitters, less so), but this
>   approach needs extending to the whole of the sphinx docs (which in turn
>   requires the docs to stay high quality so its easy for maintainers to
>   know what is where).
> 
> Although this does not apply in my proposal, I think the key issue has been 
> that reviewers and submitters of code often don't use our documentation. The 
> wiki is seen as a separate thing and also has the disadvantage that it 
> doesn't lend itself to supporting different versions of Xen. And most of the 
> time, developers do not use it and neither contribute to it.
> 
> My hope was that by hosting documentation related to contribution workflow 
> and other essential tasks close to other useful documentation this would 
> enable change.
> 
> @Andy and others: I need to know whether you agree with my proposal and 
> whether anyone has other suggestions.

If not already present in the release manager process checklist, we could 
specify documentation-related updates for each release, e.g. minimum text for 
new features, revisions to modified features, SUPPORT.md updates.

Rich

(resend with non-bulkmail address)
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 0/3] mwait support for AMD processors

[Rich Persaud]

> On Mar 28, 2019, at 11:04, Woods, Brian  wrote:
> 
> This patch series add support and enablement for mwait on AMD Naples
> and Rome processors.  Newer AMD processors support mwait, but only for
> c1, and for c2 halt is used.  The mwait-idle driver is modified to be
> able to use both mwait and halt for idling.

Would you mind if I create a Xen Project JIRA ticket, or a wiki page, to track 
requirements and implementations related to this patch series?

From the initial thread [1]:
>>> On certain AMD families that support mwait, only c1 can be reached by
>>> + * mwait and to reach c2, halt has to be used.
>>> + */
>>> +#define CPUIDLE_FLAG_USE_HALT0x2
>> 
>> Could you point us at where in the manuals this behavior is described?
>> While PM Vol 2 has a chapter talking about P-states, I can't seem to
>> find any mention of C-states there.
> 
> IIRC it's in the NDA PPR and internally it's in some other documents. 
> We don't have support to use mwait while in CC6 due to caches being 
> turned off etc.  If we did have mwait suport for CC6, we'd use that here 
> (basically mirroring Intel).  Sadly I don't think we have any public 
> information directly detailing this information.  

Can this be documented in the patch comment, or an AMD-specific page on 
wiki.xenproject.org?  It's a requirement/input to all possible implementations. 
 

From a comment in the April 2018 Linux patch by Yazen [2]:
> x86/smpboot: Don't use mwait_play_dead() on AMD systems
> Recent AMD systems support using MWAIT for C1 state. However, MWAIT will
> not allow deeper cstates than C1 on current systems.
> 
> play_dead() expects to use the deepest state available.  The deepest state
> available on AMD systems is reached through SystemIO or HALT. If MWAIT is
> available, it is preferred over the other methods, so the CPU never reaches
> the deepest possible state.
> 
> Don't try to use MWAIT to play_dead() on AMD systems. Instead, use CPUIDLE
> to enter the deepest state advertised by firmware. If CPUIDLE is not
> available then fallback to HALT.

For the ticket/wiki: what are the expected benefits of the proposed Xen change? 
 Would it reduce idle power consumption on Ryzen 1000/2000/3000? Epyc 
3000/7000? Any sample data available for idle power before/after the v2 patch?

From a thread [3] posted by Jan this week, "x86/AMD: make C-state handling 
independent of Dom0":
> The 3rd patch is my counterproposal to Brian's intended abuse (as I would 
> call it) of the mwait-idle driver. 

Do we need a new, patch-independent, thread for convergence of candidate 
implementations which address the requirements (documented in ticket/wiki)?  
Should discussion move from the initial thread [1] to the counter-proposal 
thread [3]?  Or this thread?

Rich

[1] https://lists.gt.net/xen/devel/545688

[2] 
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=x86-urgent-for-linus=da6fa7ef67f07108a1b0cb9fd9e7fcaabd39c051

[3] https://lists.xenproject.org/archives/html/xen-devel/2019-05/msg01894.html

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Anyone using blktap2?

[Rich Persaud]

> On May 13, 2019, at 11:34, Wei Liu  wrote:
> 
> Hello
> 
> Seeing that you were the last people who changed blktap2 in a meaningful
> way: do you use it at all?

As discussed F2F in a Xen Summit 2017 design session: OpenXT and Citrix 
XenServer use blktap. VHD encryption support was recently upstreamed from 
OpenXT to the Citrix-maintained repo [1] for blktap3 [2], now used by OpenXT.  
Prior versions of OpenXT use blktap2.

Citrix changed the license of XAPI blktap from GPL to BSD, to enable non-OSS 
features in the paid version of XenServer. The XAPI blktap repo is actively 
maintained, with dozens of commits in 2018 and 2019, the most recent being this 
week.

If the XAPI [3] blktap repos are part of Xen Project, should LibXL support a 
Xen Project feature that is actively maintained and shipping in production 
systems? Does the blktap3 repo [1] need to be separated from XAPI?  Perhaps a 
topic for discussion at the upcoming Xen Summit.

> I'm thinking about dropping it (again).

What would be the proposed mechanism for Xen VM block storage backed by 
thin-provisioned VHD files with per-VM encryption keys? This capability was 
sufficiently valuable to be upstreamed by Citrix, from OpenXT to Xen Project 
XAPI in 2018.  

With the imminent arrival of Windows Hyper-V and WSL2 Linux 4.19 on developer 
desktops, VHD support (i.e. Microsoft storage interoperability) in Xen may be a 
feature to improve rather than remove.

Rich

[1] https://github.com/xapi-project/blktap

[2] https://wiki.xenproject.org/wiki/Blktap3

[3] https://xenproject.org/developers/teams/xen-api/

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Guest Testing in OSSTEST - What distros and versions should we test against

[Rich Persaud]

> On May 9, 2019, at 21:28, Lars Kurth  wrote:
> 
> Hi all,
> 
> following a discussion with committers about Guest testing in OSSTEST, it 
> surfaced that we have not updated what distros we test in OSSTEST for a very 
> long time. All agreed that we should regularly review what we test against: 
> maybe at the beginning of a release cycle
> 
> In any case, currently we test against
> 
> x86 HVM guests:
>  debian-9.4.0-{i386,amd64}-CD-1.iso
>  rhel-server-6.1-i386-dvd.iso
>  win10v1703-x86.iso
>  win7-x64.iso
>  ws16-x64.iso
>  FreeBSD-10.1-CUSTOM-{i386,amd64}-20150525.raw.xz
>  Debian HVM {i386,amd64} via debian-installer netinst [1]
> 
> x86 PV guests:
>  Debian PV {i386,amd64} via debian-installer netinst [1]
> 
> ARM guests:
>  Debian PV via debian-installer netinst [1]
> 
> [1] whatever Debian release osstest itself mostly runs
> 
> So I am opening the floor to suggestions.
> 
> With regards to Windows testing we have some restrictions. We have tried 
> several times to buy additional test licenses, but this never went anywhere 
> (some of the VM licenses are not available for our environment, unless you 
> bulk buy, which is very expensive). The only approach that would allow us to 
> test against different windows versions would be to require everyone who may 
> touch OSSTEST which is not doable.

Are the 90-day test/eval versions of Windows incompatible with OSSTEST?

  https://www.microsoft.com/en-us/evalcenter/evaluate-windows-10-enterprise
  https://www.microsoft.com/en-us/evalcenter/

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [VOTE] tagging for operational messages sent to xen-devel@ (was Re: Xen 4.13 Development Update)

[Rich Persaud]

> On May 1, 2019, at 14:37, Lars Kurth  wrote:
> 
> Rich,
> as nobody replied to the mail, I am inclined to dismiss the proposal of ANN 
> for now
> Lars

What do you think about the suggestion to apply a tag ("ANNOUNCE"?) for emails 
that are mirrored to xen-devel from the -announce mailing list?  Jan commented 
on that suggestion in a separate thread.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [VOTE] tagging for operational messages sent to xen-devel@ (was Re: Xen 4.13 Development Update)

[Rich Persaud]

> On Apr 25, 2019, at 12:36, Lars Kurth  wrote:
> 
> Alright,
> 
> there was a lengthy discussion on this topic on IRC - log attached. The 
> consensus appears to be to use Canonical messages with a CAPITALISED tag. 
> E.g. "[TAG] Xen 4.13 Development Update".
> 
> The options which seemed to have least objections are
> 1: [ANNOUNCE]
> 2: [OPERATIONS] 
> 3: [PROCESS]
> 
> And that we should use these for other messages/announcements related to the 
> operation of Xen Project Development.

On mobile devices, shorter subjects are better.  A [xen-devel] email already 
has one 11-character tag. Since tags are in CAPITALS, abbreviated tags = less 
SHOuting.

> [Diziet] Only because we copy everything from -announce to -devel.

Some mailing lists use [ANN] for announcements.  Email mirrored to xen-devel 
from -announce could prefix the [ANN] tag, which would not be used for 
non-mirrored email, since all announcements would be directed to -announce.

> [gwd] But in my mind, things like RM updates (which happen pretty regularly) 
> and say, Developer Summit announcements, are different things.

The messages which prompted this discussion were related to release management. 
 These were called RM in the IRC discussion, which suggests [RM] as a possible 
tag.  It's quick to type and non-distracting to read.  This would not preclude 
other tags for non [RM] messages.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] libxl: Add virtio vga interface support for qemu

[Rich Persaud]

> On Apr 10, 2019, at 20:02, Chris Patterson  wrote:
> 
> For anyone looking at this... while I have tested and verified that
> both virtio-gpu and VirGL work, it's not without some hiccup.
> 
> I have been running Ubuntu 19.04 with this config for a few days and I
> have had a couple VM freezes. 'xl dmesg' is spamming the following
> repeatedly:
> (XEN) irq.c:2479: dom3: pirq 143 or emuirq -3 already mapped
> 
> Seems to be related to MSI-X for the device, but I haven't yet had a
> chance to dig into it.  Maybe someone knows if this is to be expected
> given the state of things? :)

Adding Paul Durrant (vGPU/graphics) to the thread.

Rich

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] XSA-283: Post-mortem

[Rich Persaud]

On Feb 25, 2019, at 10:14, George Dunlap  wrote:
> 
> This is an attempt to do a 'post-mortem' on XSA-283, to find out how
> the error came about, and consider what changes we could make to code
> / processes to prevent such errors from happening in the future.  The
> Security Team hopes to make it a habit to perform such analyses in the
> future.

...

> There was no public review of this patch; the commit contains no R-b
> or A-b.  Keir's only comment on the patch before committing it was to
> note that it conflicted with a different patch and ask for a rebase.

Would it be useful to count/triage commits without R-b or A-b?

Rich

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 00/15] Argo: hypervisor-mediated interdomain communication

[Rich Persaud]

> On Feb 4, 2019, at 05:07, Roger Pau Monné  wrote:
> 
>> On Sun, Feb 03, 2019 at 10:04:29AM -0800, Christopher Clark wrote:
>> On Thu, Jan 31, 2019 at 5:39 AM Roger Pau Monné  
>> wrote:
>> 
>> On Wed, Jan 30, 2019 at 08:05:30PM -0800, Christopher Clark wrote:
>> On Tue, Jan 22, 2019 at 6:19 AM Roger Pau Monné  
>> wrote:
>> 
>> On Mon, Jan 21, 2019 at 01:59:40AM -0800, Christopher Clark wrote:
>> Version five of this patch series:
>> 
>> * Changes are primarily addressing feedback from the v4 series reviews.
>> Many points noted on the invididual commit posts.
>> 
>> * Critical sections have been shrunk, with allocations and frees
>> pulled outside where possible, reordering logic within hypercall ops.
>> 
>> * A new ring hash function implemented, derived from the djb2 string
>> hash function.
>> 
>> * Flags returned by the notify op have been simplified.
>> 
>> * Now uses a single argo boot parameter, taking a list:
>> - top level boolean to enable/disable Argo
>> - mac-permissive option to enable/disable wildcard rings
>> - command line doc edit: no "CONFIG_ARGO" but refers to build config
>> 
>> * Switched to use the standard list data structures used by Xen's
>> common code.
> 
> AFAIK this was not requested by any reviewer, so I wonder why you made
> such change. The more that you open coded some of the list_ macros
> instead of just doing a s/hlist_/list_/ replacement.
> I'm fine with using list instead of hlist,
 
 At your request, v7 replaces open coding with Xen's list macros. The
 hlist macros were not used by any of the common code in Xen.
 
> but I don't understand why
> you decided to open code list_for_each and list_for_each_safe instead
> of using the macros provided by Xen. Is there an issue with such
> macros?
 
 As discussed offline:
 
 - Using Xen's list macros will expedite Argo's merge for Xen 4.12
 - List macros in Xen list.h originated in Linux list.h and have diverged
 - OpenXT has use cases for measured launch and nested virtualization,
 which influence downstream performance and security requirements for
 Argo and Xen
 - OpenXT can temporarily patch Xen 4.12 for downstream use
 
> I've made a couple of minor comments, but I think the current status
> is good, and fixing those minor comments is going to be trivial.
 
 Ack, thanks. Hopefully v7 looks good.
>>> 
>>> As a note, the common flow of interactions usually involves the
>>> contributor replying to the comments made by the reviewer in order to
>>> try to reach an agreement before sending a new version.
>> 
>> Yes, v7 was sent to address Jan and Julien's review comments in parallel
>> with our ongoing discussion on v5 macros. v7 also provided a checkpoint
>> for Argo testers to maximize test coverage as the series converges into
>> a Xen 4.12 merge candidate for Juergen. It addressed:
>> 
>> - Jan's v6 review comments
>> - Julien's v1 review comment
>> - most of your xen-devel and offline review comments
> 
> I think it will benefit the community to give this review in public,
> so other reviewers know whats going on. IMO getting this private
> review makes it harder for me (as a reviewer) to know the motivation
> of some of the changes between versions, and likely also makes it
> harder for you since you have to keep track of comments from multiple
> sources on different channels.
> 
> Is there anything that prevents those people from making the review
> comments publicly on xen-devel?
> 
> We should very much try to fix that so everyone can make review
> comments on the public mailing list.

I've advocated for open-source principles in several large organizations.  At 
XenSource and Citrix, we created organizational separation between the OSS Xen 
dev team and product teams.  I don't know if that structure remains today, but 
it was once helpful in reducing conflict between public OSS and private product 
roadmaps.

The separation between server and client Xen product teams was less ideal, 
which eventually lead to OpenXT.  Six years after v4v was posted to xen-devel, 
Xen Argo is the first step to possible reunification, a small chance at 
reversal, via public open-source, of architectural and resource fragmentation 
that took place privately.

Like QubesOS, OpenXT (and predecessor Citrix XenClient) development is spread 
across many open-source projects, including Xen, enabling user workflows that 
balance hardware-assisted security with usability.  Spanning ecosystems, OpenXT 
is:

- unbundling OSS capabilities, e.g. TrenchBoot and coreboot for launch integrity
- moving code upstream (Argo, stubdom, blktap, Qemu, OpenEmbedded meta-virt)
- refactoring for peer & downstream derivatives, on client devices and beyond

To achieve this cross-community integration, we work with many stakeholders 

Re: [Xen-devel] [PATCH v5 00/15] Argo: hypervisor-mediated interdomain communication

[Rich Persaud]

> On Feb 4, 2019, at 13:22, Stefano Stabellini  wrote:
> 
> On Mon, 4 Feb 2019, Roger Pau Monné wrote:
>>> Yes, v7 was sent to address Jan and Julien's review comments in parallel
>>> with our ongoing discussion on v5 macros. v7 also provided a checkpoint
>>> for Argo testers to maximize test coverage as the series converges into
>>> a Xen 4.12 merge candidate for Juergen. It addressed:
>>> 
>>> - Jan's v6 review comments
>>> - Julien's v1 review comment
>>> - most of your xen-devel and offline review comments
>> 
>> I think it will benefit the community to give this review in public,
>> so other reviewers know whats going on. IMO getting this private
>> review makes it harder for me (as a reviewer) to know the motivation
>> of some of the changes between versions, and likely also makes it
>> harder for you since you have to keep track of comments from multiple
>> sources on different channels.
> 
> There is one more reason to require public comments which I have only
> learned recently: for safety certifications we need to keep a record of
> all review comments and patches that address them for traceability.

Do you mean:

(A) all _merged_ patches and their review comments

 or

(B) all comments and patches (merged or not) that address them

i.e. would the certification process be seeking traceability of 
safety-impacting patches (code, scenario A) or decisions (including decisions 
to leave code unchanged, scenario B)?

If you mean (B), would we need an update to the Xen Security Problem Response 
Process [1]?  e.g. public archive of all comments from pre-disclosure 
discussion, along with content hashes stored immutably?  

Rich

[1] https://www.xenproject.org/security-policy.html


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Macro supply chains

[Rich Persaud]

Synopsys, which owns both Coverity and Black Duck, wrote about software 
supply-chain integrity for a library with almost two million downloads per week:

"EventStream, a highly popular JavaScript library, was compromised with the 
addition of a third-party dependency, flatmap-stream, containing encrypted 
malicious code. The attack targeted other Node.js libraries used in 
cryptocurrency wallets."
"Keep a bill of materials (BoM), a list of components and dependencies in your 
codebase. Just knowing what your code depends on will help make you aware of 
the third-party risks that you might be exposed to."

Are there existing best practices for tracking and maintaining macros which 
originated in other open-source communities, e.g. QEMU or Linux?  Some Xen 
macros have diverged [2][3][4] from the versions used by other communities.  
Would such macros benefit from a documented relationship with upstream, e.g.

- "Ignore upstream changes"
- "Mirror upstream changes"
- "Review upstream changes"

For the latter case, build/test tooling could trigger manual macro review when 
a change is detected in an upstream dependency.  Which other cases should be 
considered?

Rich

[1] Compromised npm packages lead to supply chain attack

https://www.synopsys.com/blogs/software-security/malicious-dependency-supply-chain/

https://github.com/dominictarr/event-stream/issues/116

[2] bitops.h

http://xenbits.xen.org/gitweb/?p=xen.git;a=history;f=xen/include/xen/bitops.h;h=a103e490894829a445cc743b98f8956b7d44e022;hb=HEAD

https://github.com/torvalds/linux/commits/master/arch/x86/include/asm/bitops.h

[3] list.h

http://xenbits.xen.org/gitweb/?p=xen.git;a=history;f=xen/include/xen/list.h;h=1387abb21192668f4103620520cf4256902a45aa;hb=HEAD

https://github.com/torvalds/linux/commits/master/include/linux/list.h

[4] delay.h

http://xenbits.xen.org/gitweb/?p=xen.git;a=history;f=xen/include/xen/delay.h;h=9d70ef035fc20bb6708a5ff33eb83a55c6ad460c;hb=HEAD

https://github.com/torvalds/linux/commits/master/include/linux/delay.h


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen (both x86 and Arm) Community Call: Feb 13 - 16:00 - 17:00 UTC - Call for Agenda Items

[Rich Persaud]

> On Feb 11, 2019, at 05:05, Lars Kurth  wrote:
> 
> Hi all, 
> 
> we have the community call for February coming up this Wednesday. My sincere 
> apologies, that I have not asked for agenda items last week. A current agenda 
> (primarily a skeleton) is available at  
> https://docs.google.com/document/d/15ZLzQcH794jufDZW1oNYVY2D12CnVqxQ-klFAqkd2bU/edit#heading=h.mz1wjb9vekjn
> 
> Please propose topics by either editing the running agenda document at 
> https://docs.google.com/document/d/1Ufv9XcQO0zIAVeFbFCAHAeEIB9Ap4Y4srAm4vI8I01I/edit#
>  or by replying to the mail. Ideally by a few hours before the meeting!

Proposed agenda items:

1.  Tailored instances of Xen: continuing the Nov 2018 discussion of KCONFIG/L0 
 hypervisor use cases.  More details upcoming via wiki page.

2.  Macro supply chains:  what are best practices for maintaining Xen macros 
which originate in other open-source communities, e.g. QEMU or Linux?  Would 
each macro benefit from a documented status (e.g. "Ignore upstream changes", 
"Monitor upstream changes", "Mirror upstream changes") with associated tooling?

3. Go toolchain:  is there community interest in collaborating on the 
development of golang tools for local management of Xen?  Historically, OpenXT 
used a combination of Haskell and Ocaml tools.  Some OpenXT community members 
are using golang with Xen. Could these new tools find a home in upstream Xen?

4. NVME passthrough performance: this is improved when VMEXITs are avoided by 
using "posted interrupts" [1] available on Broadwell and later Xeon processors 
or AWS nested hypervisor "metal" [2].  For commodity x86 CPUs which do not have 
posted interrupts, Linux [3] and Hyper-V [4] have used "hybrid polling" to 
achieve good I/O performance at the cost of CPU cycles.  Is this applicable to 
Xen?

Rich

[1] "VT-d Posted Interrupts" - Intel, 2012
https://events.static.linuxfound.org/sites/events/files/slides/VT-d%20Posted%20Interrupts-final%20.pdf
https://www.linux-kvm.org/images/7/70/2012-forum-nakajima_apicv.pdf

[2] Running Thousands of KVM Guests on Amazon i3.metal Instances, twosix, 2017
https://www.twosixlabs.com/running-thousands-of-kvm-guests-on-amazons-new-i3-metal-instances/

[3] "I/O Latency Optimization with Polling" - Western Digital, 2017
https://events.static.linuxfound.org/sites/events/files/slides/lemoal-nvme-polling-vault-2017-final_0.pdf

[4] "Achieving 10-Million IOPS from a single VM on Windows Hyper-V" - MS, 2018
https://www.snia.org/sites/default/files/SDC/2018/presentations/Cloud_Storage/Yang_L_Zhu_D_Achieving_10-Million_IOPS_from_a_single_VM_on_Windows_Hyper-V.pdf

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v8 for-4.12 10/17] argo: implement the notify op

[Rich Persaud]

On Feb 7, 2019, at 04:04, Julien Grall  wrote:
> 
> Hi,
> 
>> On 2/7/19 6:32 AM, Christopher Clark wrote:
>>> On Wed, Feb 6, 2019 at 10:28 AM Julien Grall  wrote:
>>> 
>>> Hi,
>>> 
 On 2/6/19 8:55 AM, Christopher Clark wrote:
 +/*
 + * XEN_ARGO_OP_notify
 + *
 + * Asks Xen for information about other rings in the system.
 + *
 + * ent->ring is the xen_argo_addr_t of the ring you want information on.
 + * Uses the same ring matching rules as XEN_ARGO_OP_sendv.
 + *
 + * ent->space_required : if this field is not null then Xen will check
 + * that there is space in the destination ring for this many bytes of 
 payload.
 + * If the ring is too small for the requested space_required, it will set 
 the
 + * XEN_ARGO_RING_EMSGSIZE flag on return.
 + * If sufficient space is available, it will set XEN_ARGO_RING_SUFFICIENT
 + * and CANCEL any pending notification for that ent->ring; otherwise it
 + * will schedule a notification event and the flag will not be set.
 + *
 + * These flags are set by Xen when notify replies:
 + * XEN_ARGO_RING_EXISTS ring exists
 + * XEN_ARGO_RING_SHARED ring is registered for wildcard partner
 + * XEN_ARGO_RING_EMPTY  ring is empty
 + * XEN_ARGO_RING_SUFFICIENT sufficient space for space_required is there
 + * XEN_ARGO_RING_EMSGSIZE   space_required is too large for the ring size
 + * XEN_ARGO_RING_EBUSY  too many domains waiting for available space 
 signals
 + *
 + * arg1: XEN_GUEST_HANDLE(xen_argo_ring_data_t) ring_data (may be NULL)
 + * arg2: NULL
>>> 
>>> NIT: While looking at all the hypercall, I noticed you say NULL here. In
>>> most of the cases, NULL will be 0 but there are place where it might not
>>> be. So what is the exact value you expect here?
>> The implementation tests both arg1 and arg2 with: guest_handle_is_null
>> which on both x86 and ARM expands to: ((hnd).p == NULL)
> 
> My point here is someone reading the public headers may not read the 
> implementation. So he/she would not know whether NULL is equivalent to 0 or 
> another value.
> 
>> It uses that null test because both are XEN_GUEST_HANDLE_PARAM type in
>> the function signature:
>> long do_argo_op(
>> unsigned int cmd,
>> XEN_GUEST_HANDLE_PARAM(void) arg1,
>> XEN_GUEST_HANDLE_PARAM(void) arg2,
>> unsigned long arg3,
>> unsigned long arg4);
>> and since it does that, the comment states NULL rather than zero.
>> It is Xen's definition for NULL that is used, so the expected value in
>> the register when the hypercall is invoked is: zero.
> 
> As above, I don't think it is clearly define in the headers that NULL means 
> 0. This is rather an implicit written rules in the hope that every OS are 
> going to follow that.
> 
> I know, I am pedantic here (hence the NIT) :). And I realize this is not 
> related to this series and a few places in the code assumes the same.
> 
 + * arg3: 0 (ZERO)
 + * arg4: 0 (ZERO)
>>> 
>>> NIT: I guess those to will be 0 in an unsigned long value?
>> Yes.
> 
> Can this be clarified in a follow-up patch?

Since this is a comment revision, could it be made by the committer during the 
imminent merge of this Argo series?

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v8 for-4.12 00/17] Argo: hypervisor-mediated interdomain communication

[Rich Persaud]

On Feb 6, 2019, at 08:57, Jan Beulich  wrote:

 On 06.02.19 at 14:53,  wrote:
>> On 06/02/2019 14:45, Jan Beulich wrote:
>> On 06.02.19 at 09:54,  wrote:
 Version eight of this series:
 
 Note: This version may not address the currently open discussion on the
 ARM hypercall argument convention and type selection for hypercall
 parameters.
 
 * Range check applied to numeric args in native hypercall entry
 (ref: the above open discussion)
 
 * Revises the compat ABI and implementation
 - avoids duplication of hypercall op implementations via
   forwarding to native for ops other than sendv
 - register op uses an always-64-bit fixed width pfn type
   for consistent ABI as well as compat reuse of the native op
 - tested communication between VMs on x86-64 host with:
   32-bit PV, 32-bit HVM and 64-bit PV guests
 
 * Applies list_first_entry_or_null macro in multiple loops to
 replace previous use of a list foreach to address review feedback
 
 * Removed stale comments from the public header
 
 New to this series:
 
 * Adds an initial version of a design document for Argo
 - based on work previously sent to the mailing list, covers
   the implementation's granular locking
 
 * Adds a SUPPORT.md section for the feature and Experimental statement
 
 Christopher Clark (17):
 argo: Introduce the Kconfig option to govern inclusion of Argo
 argo: introduce the argo_op hypercall boilerplate
 argo: define argo_dprintk for subsystem debugging
 argo: init, destroy and soft-reset, with enable command line opt
 errno: add POSIX error codes EMSGSIZE, ECONNREFUSED to the ABI
 xen/arm: introduce guest_handle_for_field()
 argo: implement the register op
 argo: implement the unregister op
 argo: implement the sendv op; evtchn: expose send_guest_global_virq
 argo: implement the notify op
 xsm, argo: XSM control for argo register
 xsm, argo: XSM control for argo message send operation
 xsm, argo: XSM control for any access to argo by a domain
 xsm, argo: notify: don't describe rings that cannot be sent to
 MAINTAINERS: add new section for Argo and self as maintainer
 SUPPORT.md : add new entry for the Argo feature
 docs, argo: add design document for Argo
>>> 
>>> Where necessary and not already present
>>> Acked-by: Jan Beulich 
>>> 
>>> Jürgen, for this to be committed, your Rab would be needed, assuming
>>> you're still comfortable with this going in this late.
>> 
>> What about the ARM hypercall parameters? Is this settled?
> 
> My interpretation of Stefano's latest response was "yes, it is".

Julien's latest response said, "None of this is specific to Argo and I would be 
happy to defer this as a follow-up series."

>> If yes or if this question is solved this week:
>> 
>> Release-acked-by: Juergen Gross 

If necessary to unblock the merge of Argo to Xen 4.12, we can limit/constrain 
Arm support via SUPPORT.md.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

[Rich Persaud]

> On Jan 14, 2019, at 06:32, Roger Pau Monné  wrote:
> 
> On Mon, Jan 14, 2019 at 9:32 AM Christopher Clark
>  wrote:
>> 
>> I've written a document about the locking to add to the tree with the
>> series, and a copy is at github here:
>> 
>> https://github.com/dozylynx/xen/blob/0cb95385eba696ecf4856075a524c5e528e60455/docs/misc/argo-locking.md
> 
> Thanks. It would have been better to send the contents of the document
> to the list, so inline comments can be added. It's hard to comment on
> the document now since it's only on github AFAICT.

Here's an inline copy of the doc:

=== begin document ===

# Argo: Locking

## Introduction

Argo is an interdomain communication mechanism. It has requirements for 
performance isolation between domains, to prevent negative performance impact 
from malicious or disruptive activity of other domains, or even other vcpus of 
the same domain operating other rings.

Since Argo operates a data path between domains, sections of this code are 
*hot* when the communication paths are in use. To encourage high performance, a 
goal is to limit mutual exclusion to only where required and enable significant 
concurrency.

Avoidance of deadlock is essential and since state must frequently be updated 
that pertains to more than one domain, a locking protocol defines which locks 
are needed and the order of their acquistion.

## Structure

The granular locking structure of Argo enables:

1. Performance isolation of guests
2. Avoidance of DoS of rings by domains that are not authorized to send to them
3. Deadlock-free teardown of state across multiple domains on domain destroy
4. Performance of guests using Argo with concurrent operation of rings.

Argo uses three per-domain locks to protect three separate data structures.  
Access to the ring_hash data structure is confined to domains that a 
ring-registering domain has authorized to send data via the ring.  The complete 
set of Argo locks is:

* Global : `L1_global_argo_rwlock`
* Per-domain: `rings_L2_rwlock`
* Per-domain: `send_L2_lock`
* Per-domain: `wildcard_L2_lock`
* Per-ring: `L3_lock`

## Protected State

The data structures being protected by the locks are all per-domain. The only 
global Argo state is the `L1_global_argo_rwlock` used to coordinate access to 
data structures of other domains.

### State: Rings registered and owned by a domain

This includes the state to run that ring, such as memory frame numbers and 
established mappings. Per-ring state is protected by its own lock, so that 
multiple VCPUs of the same domain operating different rings do not inhibit the 
performance of each other.

The per-domain ring state also includes the list of pending notifications for 
other domains that are waiting for ring space availability.

### State: Partner rings registered by other domains that this domain is the 
single allowed sender

This state belonging to the permitted sender is written to when a ring is 
registered by another domain. The lock that protects this state is subject to 
locking at arbitrary frequency by those foreign domains when registering rings 
-- which do not need any permission granted by this domain in order to register 
a ring to communicate with it --  so it must not inhibit the domain's own 
ability to use its own rings, to protect them from DoS. For this reason, this 
state is protected by its own lock.

### State: Pending notifications to this domain about wildcard rings registered 
by other domains

This data structure is needed when a domain is destroyed, to cancel the 
outstanding space availability notifications about the wildcard rings of other 
domains that this domain has queried.

Data is entered into this data structure by the domain that owns it, either by 
a space-inhibited sendv or a notify operation.

Data is removed from this data structure in one of three cases: when space 
becomes available in the destination ring and the notification is sent, when 
the ring is torn down, or when the awaiting domain is destroyed.

In the case where a notification is sent, access to the data structure is 
triggered by the ring owner domain, rather than the domain waiting for 
notification. This data structure is protected by its own lock since doing so 
entails less contention than the alternative of reusing an existing lock owned 
by the domain.

## Hierarchical Locking Model and Protocol

The locking discipline within the Argo code is heirarchical and utilizes 
reader/writer locks to enable increased concurrency when operations do not 
conflict. None of the Argo locks are reentrant.

The hierarchy:

* There is a global rwlock (`L1`) to protect access to all of the per-domain 
argo data structures. 
* There is a rwlock per-domain (`rings_L2`) to protect the hashtable of the 
per-ring data structures. 
* There is a lock per ring (`L3`) to protect the per-ring data structure, 
`struct argo_ring_info`. 

There are a two other per-domain L2 locks; their operation is similar and they 
are described 

Re: [Xen-devel] Xen (both x86 and Arm) Community Call: Jan 9 - 16:00 - 17:00 UTC - Call for Agenda Items

[Rich Persaud]

On Jan 9, 2019, at 10:46, Lars Kurth  wrote:
> 
> Quick note: the meeting is in 15 minutes - the agenda is at 
> https://docs.google.com/document/d/1Ufv9XcQO0zIAVeFbFCAHAeEIB9Ap4Y4srAm4vI8I01I/edit
>  

On the topic of 4.12, I would like to propose moving the merge date by one 
week, since about 50% of Xen  maintainers who can review the Argo patch series 
for 4.12 are unavailable this week.

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] x86 Community Call: Nov 14 - 15:00 - 16:00 UTC - Call reminder

[Rich Persaud]

> On Nov 14, 2018, at 04:56, Lars Kurth  wrote:
> 
> Hi all,
> 
> the agenda is as follows: 
> https://docs.google.com/document/d/1RxW-iwcFFuKzNjjEqLEtiwFVHgAUlk35c0EtTkRE1k4/edit
> * Follow up on previous actions
> * PVH resource Mapping: Rian Quinn (AIS)
> * TMEM (Jan)

If there is time on today's call, I would like to add an agenda item to discuss 
requirements collection for two Kconfig-defined subsets of Xen:

- minimal Xen hypervisor optimized for hardware support, 
isolation/partitioning, secrets-free, safety certification, reduced attack 
surface and performant nesting of Xen and other hypervisors, including Hyper-V, 
KVM, ESX, Bromium uXen, bhyve

- full-featured Xen hypervisor optimized for guest operating systems, 
unikernels, application-specific workloads and performant nesting on Xen and 
other hypervisors

Related topics for discussion, possibly on a future call:

- which hypervisor has priority for inter-VM communication policy enforcement
- scheduler interactions across bare-metal Xen, nested hypervisor and guest OS

Daniel Smith (Apertus) will introduce the topic.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] x86 Community Call: Wed Oct 10, 14:00 - 15:00 UTC - Call for agenda items

[Rich Persaud]

Lars,

This NIST document ("A Methodology for Determining Forensic Data Requirements 
for Detecting Hypervisor Attacks" [1]) appears to be focused on the application 
of LibVMI in some contexts.  It is a NIST Interagency or Internal Report 
(NISTIR) document with a narrower scope than other NIST publications, e.g. 
Special Publications (SP).  NISTIR documents are:

https://www.nist.gov/nist-research-library/nist-series-publications
"... Interim or final reports on work performed by NIST for outside sponsors 
(both government and non-government).  May also report results of NIST projects 
of transitory or limited interest, including those that will be published 
subsequently in more comprehensive form."

If the Xen community wishes to provide feedback on this NISTIR draft, I suggest 
compiling a single document, including:

 - any inaccuracies + supporting references
 - vulnerability scope boundaries, including Xen hypervisor, Linux kernel 
affecting KVM, KVM module for Linux kernel, QEMU and hypervisor toolstack(s)
 - additional sample attack(s) and evidence coverage for forensic analysis
 - additional references on hypervisor security / vulnerability analysis
 - missing perspectives (e.g. impact of features selected via KCONFIG, 
disaggregation)
 - other feedback

If a single list can be compiled, each item can be numbered and Xen community 
viewpoints can be aggregated for possible consensus in unified feedback, or 
individuals could submit their feedback separately.

Rich

[1] 
https://csrc.nist.gov/CSRC/media/Publications/nistir/8221/draft/documents/nistir-8221-draft.pdf

> On Oct 9, 2018, at 14:20, Lars Kurth  wrote:
> 
> Hi all,
> I added a NIST Security Paper to the agenda which is currently under review 
> and is full of inaccuracies and could potentially become very problematic to 
> the project and vendors using Xen if officially published by NIST without 
> being corrected (it needs responses by the end of week). I will be struggling 
> to do this alone and would like to enlist help, in particular from people 
> with a security background. That would also be significantly more powerful 
> than me providing the feedback.
> Regards
> Kars
> 
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] x86 Community Call - Wed Sept 12, 14:00 - 15:00 UTC - Call for agenda items

[Rich Persaud]

> On Sep 6, 2018, at 11:35, Lars Kurth  wrote:
> 
> Dear community members,
>  
> please send me agenda items for next week’s community call by next 
> Monday. 

"Argo" (formerly v4v) inter-VM communication mechanism for Xen 4.12, discussed 
at Xen Summit 2018 design session and PSEC 2018, design doc forthcoming.  
Variants of this technology have been used in OpenXT and uXen/Bromium for 
several years.
Video:  https://www.platformsecuritysummit.com/2018/speaker/clark/

> One suggestion which was made to me recently by Rich,
> Christopher and Daniel was for Daniel to give an overview of
> Secure Boot, Measured Launch and TrenchBoot.
>  
> @Daniel, @Christopher, @Rich: would this be possible during the
> next call? How much time does the talk require.

If attendees can watch Daniel Smith’s TrenchBoot video as a pre-requisite for 
the call, his overview will build on the video: 15 min talk + 15 mins of 
discussion.
Video: https://www.platformsecuritysummit.com/2018/speaker/smith/

There were boot integrity talks on Xen and/or UEFI, from AIS, Dell, Intel, 
Oracle:
https://www.platformsecuritysummit.com/2018/topic/boot/

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 5/5] x86: use PDEP for PTE flags insertion when available

[Rich Persaud]

On Aug 17, 2018, at 03:24, Jan Beulich  wrote:
> 
> This replaces 5 instructions by a single one, further reducing code size,
> cache, and TLB footprint (in particular on systems supporting BMI2).

This link claims that BMI2 may be less performant/consistent on AMD Ryzen than 
Intel:
https://www.reddit.com/r/Amd/comments/60i6er/ryzen_and_bmi2_strange_behavior_and_high_latencies/

Would this patch series have any benefit to L0 hypervisors/rootkits (e.g. 
Bromium, Bareflank or similar hypervisors) which could be monitoring L1 Xen?  
Or Xen as L0 hypervisor and Hyper-V as L1 hypervisor?

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH] production-config: Temporarily drop arm64

[Rich Persaud]

On Aug 15, 2018, at 05:29, Julien Grall  wrote:
> 
> Hi,
> 
>> On 08/15/2018 12:25 AM, Rich Persaud wrote:
>>> On Aug 14, 2018, at 18:46, Julien Grall  wrote:
>>> 
>>> Hi Rich,
>>> 
>>>>> On 08/14/2018 11:42 PM, Rich Persaud wrote:
>>>>> On Aug 13, 2018, at 10:57, Ian Jackson  wrote:
>>>>> 
>>>>> Both our arm64 boxes are out of commission and repairing them is
>>>>> taking too long.
>>>> Apologies if this is already documented elsewhere - does OSStest use Qemu 
>>>> for arm64 testing?
>>> 
>>> Osstest does not use QEMU for testing, but I think it would be too slow to 
>>> have result in timely manner and use x86 resource as well.
>> To avoid having zero test coverage for one target architecture, it may be 
>> acceptable to temporarily reduce test capacity for other target 
>> architectures.  QEMU has the advantage of being faster to "rack" a test 
>> architecture for temporary use.
> 
> Well, arm64 test coverage was already reduced because we had only 2 platforms 
> ready for testing. I can hardly imagine how this code be reduced more for 
> fitting QEMU. Beware that we compile natively in Osstest so this will take an 
> awful lot of time here.
> 
> However, the main problem here is not the lack of platform but the lack of 
> time for OSSTest team (mostly Ian and Wei) to investigate and bring-up new 
> platforms. Maybe you can help finding contributors to help on Arm64?

This may happen via testing of the OpenEmbedded meta-virtualization layer 
support for Xen, where OE enables cross-compilation.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH] production-config: Temporarily drop arm64

[Rich Persaud]

On Aug 14, 2018, at 18:46, Julien Grall  wrote:
> 
> Hi Rich,
> 
>> On 08/14/2018 11:42 PM, Rich Persaud wrote:
>>> On Aug 13, 2018, at 10:57, Ian Jackson  wrote:
>>> 
>>> Both our arm64 boxes are out of commission and repairing them is
>>> taking too long.
>> Apologies if this is already documented elsewhere - does OSStest use Qemu 
>> for arm64 testing?
> 
> Osstest does not use QEMU for testing, but I think it would be too slow to 
> have result in timely manner and use x86 resource as well.

To avoid having zero test coverage for one target architecture, it may be 
acceptable to temporarily reduce test capacity for other target architectures.  
QEMU has the advantage of being faster to "rack" a test architecture for 
temporary use.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH] production-config: Temporarily drop arm64

[Rich Persaud]

On Aug 13, 2018, at 10:57, Ian Jackson  wrote:
> 
> Both our arm64 boxes are out of commission and repairing them is
> taking too long.

Apologies if this is already documented elsewhere - does OSStest use Qemu for 
arm64 testing?

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen 4.12 Development Update

[Rich Persaud]

> On Jul 27, 2018, at 12:19, Juergen Gross  wrote:
> 
> This email only tracks big items for xen.git tree. Please reply for items you
> would like to see in 4.12 so that people have an idea what is going on and
> prioritise accordingly.

Two additional items:

1.  Linux stub domains, series posted by the Qubes team.  A variant of this 
approach has been used by OpenXT for several years.
https://lists.xen.org/archives/html/xen-devel/2018-07/msg02616.html

2.  "Argo" inter-VM communication mechanism, discussed at Xen Summit 2018 
design session, design doc forthcoming.  Variants of this technology have been 
used in OpenXT and uXen/Bromium for several years.

> We recently introduced a jira instance to track all the tasks (not only big)
> for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.
> 
> Most of the tasks tracked by this e-mail also have a corresponding jira task
> referred by XEN-N.

Could we start using the "Affects Version" and "Fix Version" of JIRA, with 
values for currently supported releases, plus the next two releases (4.12 and 
4.13)?  This will enable the use of filters and issue tracking for prior 
releases.

I can create JIRA tickets for the two tasks listed above, if there are no 
objections.

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen for Automotive - white paper on virtualization

[Rich Persaud]

Thanks to everyone who contributed to the AGL virtualization white paper and to 
Michele Paolino for stewarding it from concept to final publication.   I’ve 
added it to the Xen wiki:

https://wiki.xenproject.org/wiki/Automotive_Whitepapers

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Rich Persaud]

On Jul 5, 2018, at 22:54, Tamas K Lengyel  wrote:
> 
>> On Thu, Jul 5, 2018 at 12:52 PM George Dunlap  
>> wrote:
>> 
>>> 
 Again, there was a sense that some of the issues we are seeing could be 
 solved if we had better
 CI capability: in other words, some of the issues we were seeing could be 
 resolved by
 * Better CI capability as suggested in the Release Cadence discussion
 * Improving some of the internal working practices of the security team
 * Before we commit to a change (such as improved batching), we should try 
 them first informally.
  E.g. the security team could try and work towards more predictable dates 
 for batches vs. a
  concrete process change
>>> 
>>> My feeling on CI is clear in this thread and other threads. But I think
>>> what would help OSSTEST bottlenecks if we do better at separating up
>>> different parts of the testing process into more parallel tasks that
>>> also provide feedback to the contributor faster. I'll obviously never
>>> suggest the GitHub/GitLab PR/MR model to a ML driven project because I
>>> wouldn't survive the hate mail but there is something that those models
>>> do provide.
>> 
>> FWIW we (IanJ, Wei, Roger, Anthony and I) just had a fairly extended 
>> discussion about this in our team meeting today, and everyone basically 
>> agreed that there are some things about the web-based PR model that are 
>> *really* nice:
>> 
>> 1. Effective tracking of submission state — open / assigned to a reviewer / 
>> merged / rejected
>> 2. Automation
>> 3. Not having to marshal git commits into email, and then marshal them back 
>> into git commits again
>> 
>> On the other hand, the general consensus, from people who had used such 
>> websites “in anger” (as they say here in the UK) was that they really didn’t 
>> like the way that reviews worked.  Email was seen as:
>> 1. Much more convenient for giving feedback and having discussions
>> 2. Easier for people to “listen in” on other people’s reviews
>> 3. More accessible for posterity
>> 
>> In the end we generally agreed that it was an idea worth thinking about 
>> more.  Not sure how the wider community feels, but there are at least a 
>> decent cohort who wouldn’t send you hate mail. :-)
> 
> I for one would very much welcome a PR-style model. Keeping track of
> patches in emails I need to review is not fun (and I'm pretty bad at
> it) and then just to find a patch that doesn't even compile is a waste
> of everyone's time. Automatic style checks and compile checks are the
> bare minimum I would consider any project should have today. There is
> already a Travis CI script shipped with Xen yet it's not used when
> patches are submitted.. Perhaps the reviews are more accessible for
> posterity but I personally never end up reading old reviews, even in
> the depths of the worst code archeology it's always just looking at
> git blame and commit messages. Giving feedback and discussions I also
> find a lot more easier to navigate on say Github then on the
> mailinglist - and I do get email copies of PRs and can reply inline
> via email if I want to.. We are already keeping track of open patch
> series on Jira - or at least there was an attempt to do so, not sure
> how up-to-date that is - but that's not the right way as that requires
> manual porting of tasks from the mailinglist. Perhaps it should be the
> other way around.
> 
> Just my 2c.
> 
> Tamas

OpenXT uses JIRA for issue tracking and Github for pull requests and  approval 
workflow.  JIRA can link  issues to PRs, based on ticket number in the PR 
description.

Both JIRA and Github can mirror  issue/PR comments and content to email 
(individual or mailing list).  Replies to these emails will be associated with 
issues/PRs, if the sender has an account on the service.

Would there be interest in testing a Gitlab/Github workflow in a Xen sub 
project, where contributors are already inclined to use such tools?   Windows 
PV drivers could be a candidate, as QubesOS uses Github PRs and the volume of 
changes is not high.

The value of these services is not just the metadata archive and structure that 
it brings to dev workflow, but the ever-expanding ecosystem of analytics tools 
that can use the  historical data.  Yes, in theory, similar data can be 
extracted from xen-devel archives, but it often requires custom tooling.  Case 
in point - the lack of accessible quantitative data to substantiate the 
intuitions expressed in this thread, about the differences in dev patterns with 
6-month vs. longer release cycles.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen Project Security Whitepaper v1 is ready for community review

[Rich Persaud]

> On May 18, 2018, at 06:13, Lars Kurth  wrote:
> 
> Dear Community Members,
> 
> just under 3 months ago, we started a community consultation titled "Xen 
> Security Process Consultation: is there a case to change anything?" (see 
> https://lists.xenproject.org/archives/html/xen-announce/2018-02/msg0.html).
>  As promised, I would collate the input - together with further analysis 
> trying to genuinely consider the implications of what respondents to the 
> consultation have been suggesting - in a white paper. The white paper is 
> attached

The paper title would be more precise if "Security" was replaced by "Security 
Process".

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen and safety certification, Minutes of the meeting on Apr 4th

[Rich Persaud]

> On May 10, 2018, at 15:51, Stefano Stabellini  wrote:
> 
> On Thu, 10 May 2018, Praveen Kumar wrote:
>>> Yeah, you are right. It looks like turning Dom0 into a DomU is not good
>>> enough. Maybe for this option to be viable we would actually have to
>>> terminate (or pause and never unpause?) dom0 after boot.
>> 
>> Just a thought !
>> How about keeping Dom0 still be there, but DomUs given Dom0 privilege, with
>> restricted permission on mission critical resources ? And if anyhow Dom0
>> crashes,
>> the best contended among the existing DomUs take the ownership of Dom0 ?
> 
> I don't think this is easily doable, also it wouldn't solve the issue of
> removing dom0 from the system. But see below.
> 
> 
 However, you surely need an entity to handle domain crash. You don't
>> want to
 reboot your platform (and therefore you safety critical domain) for a
>> crashed
 UI, right? So how this is going to be handled in your option?
>> 
>>> We need to understand the certification requirements better to know the
>>> answer to this. I am guessing that UI crashes are not handled from the
>>> certification point of view -- maybe we only need to demonstrate that
>>> the system is not affected by them?
>> 
>> Where can we find the certification requirements details ?
> 
> Yes, I think we need to understand the requirements better to figure out
> the right way forward for Dom0.
> 
> For instance, here is another idea: we could have Xen boot multiple
> domains at boot time from device tree, as suggested in the dom0-less
> approach. All of the domains booted from Xen are "mission-critical". The
> first domain could still be dom0. Once booted, Dom0 can start other VMs,
> however, Xen would restrict Dom0 from doing any operations affecting the
> first set of mission-critical domains.
> 
> This way, we would get the flexibility of being able to start/stop
> domains at run time, but at the same time we might still be able to
> avoid certifications for Dom0, because Dom0 cannot affect the mission
> critical applications.
> 
> Is this approach actually feasible? We need to read the requirements to
> know. I am hoping Artem will chime in on this :-)

Is any of the x86 hardware domain (non dom0) work applicable to Arm?
https://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00314.html

Daniel is giving a talk on TCB reduction with a Xen hardware domain:
http://platformsecuritysummit.com/#degraaf

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] reboot driver domain, vifX.Y = NO-CARRIER?

[Rich Persaud]

> On May 1, 2018, at 08:53, Jason Cooper  wrote:
> 
> add the link to xen-users thread of me talking to myself.  :-))
> 
>> On Tue, May 01, 2018 at 12:37:51PM +, Jason Cooper wrote:
>> When I was first digging into this, I started a thread on xen-users [1],
>> I've attached my xl-reboot.sh script here so you can see exactly what
>> I'm attempting to do:
> 
> [1] https://marc.info/?l=xen-users=152389443206023=2

You may want to look at the code (toolstack and/or frontend-backend drivers) 
for Qubes and OpenXT, both of which use network driver domains and support 
wired/wireless networks.  

Operational restart of a measured, non-persistent driver domain (instead of 
host) is a benefit of Xen disaggregation architectures.

Rich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Community call: PCI Emulation - Future Direction (Wed, May 2nd, UTC 16:00-17:00 / BST 17:00-18:00)

[Rich Persaud]

> On Apr 24, 2018, at 05:19, Lars Kurth  wrote:
> 
> Hi all,
> as agreed please find attached the meeting invite
> Regards
> Lars
> 
> ## Agenda (provisional)
> I copied what was discussed on this thread so far 
> https://docs.google.com/document/d/1RWylmNmBXOrgGLARj6_ynK50P7SZPl4LpnmhGaPglJw/edit?usp=sharing,
>  which I will use as pad to write down minutes. Feel free to make agenda 
> suggestions and copy relevant information into the doc, prior to the meeting.

I would like to add an agenda item to discuss the level of security support 
that will be asserted in SUPPORT.md for driver domains which contain untrusted 
PCI devices.  Will Xen security support be different for SR-IOV devices?  GPUs 
vs. NICs?

There have been past discussions on this topic and a proposed 
PCI-iommu-bugs.txt file to help Xen users and developers understand the risks 
[2][3][4] that may arise from a hostile device and potentially buggy firmware.  
If we can document specific risks, we can ask firmware developers to make 
specific improvements to improve the security of PCI emulation.

There is an active effort [4] underway to improve firmware security in servers 
(and eventually desktops), including a reduction of attack surface due to SMM.  
There is also work underway [5][6] to perform secure boot between individual 
PCI devices and server motherboards.  Some of these concepts may already be 
deployed in Azure.

Several stakeholders will be attending or presenting at the PSEC [6] conference.

Rich

[1] Performance Isolation Exposure in Virtualized Platforms with PCI 
Passthrough I/O Sharing, https://mediatum.ub.tum.de/doc/1187609/972322.pdf

[2]  Securing Self-Virtualizing Ethernet Devices, 
https://www.usenix.org/system/files/conference/usenixsecurity15/sec15-paper-smolyar.pdf

[3]  Denial-of-Service Attacks on PCI Passthrough Devices, 
http://publications.andre-richter.com/richter2015denial.pdf

[4] Open Compute Open System Firmware, 
http://www.opencompute.org/wiki/Open_System_Firmware

[5] Open Compute Security, http://www.opencompute.org/wiki/Security

[6] Firmware attestation: 
https://www.platformsecuritysummit.com/prepare/#attestation

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel