[Xen-devel] FOSDEM Virtualization and IaaS Devroom CfP (closes Dec 1) and Xen Project Stand

[Lars Kurth]

Hi all,

as we usually have a big presence at FOSDEM, here are a few relevant CfPs. To 
make it easier for people who submit talks, I collated some of the info.

== FOSDEM Virtualization and IaaS Devroom CfP ==
See https://lists.fosdem.org/pipermail/fosdem/2017-October/002632.html

Submission deadline: 01 December 2017
Acceptance notifications: 14 December 2017
Final schedule announcement: 21 December 2017
Devroom: 03 and 04 February 2018 (two days- different rooms)

== Other relevant Devroom CfP ==
* BSD (cfp closes at 2017-11-26): see 
https://lists.fosdem.org/pipermail/fosdem/2017-October/002614.html
* Embedded, Mobile and Automotive, TBD: TBD - see 
https://fosdem.org/2018/news/2017-10-04-accepted-developer-rooms/


== Xen Project Stand ==
As usual, we applied for a stand again. Acceptance notifications are not due 
until 20 November

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Is that possible to merge MBA into Xen 4.10?

[Lars Kurth]

Hi all,

> On 24 Oct 2017, at 09:35, Jan Beulich  wrote:
> 
 On 24.10.17 at 04:10,  wrote:
>> As you may know, MBA patch set has got enough Reviewed-by/Acked-by in last 
>> week.
>> It is ready to be merged. 
>> 
>> This is a feature for Skylake, Intel has launched Skylake and KVM already
>> supported MBA, so including it in Xen 4.10 will quickly fill this gap.
>> 
>> MBA missed the 4.10 feature freeze date for only a few days due to lack of
>> timely review for earlier versions which slowed down the patch iteration 
>> notably.
>> It seems maintainers are very busy recently so that the review progress for 
>> 4.10
>> is slower than before. So I am wondering if it is possible to merge it into 
>> 4.10?
>> 
>> This patch set mainly touches codes related to PSR in 
>> tools/domctl/sysctl/hypervisor.
>> It does not touch other features. So, the risk is low to merge it.
> 
> While I agree the risk is low, I think we should not start making
> exceptions from the "no freeze exceptions" rule. Even less so
> for a secondary (or should I pull out my favorite "niche" again?)
> feature like this one. But in the end it's Julien's call., of course.

I think there are a number of separate issues concerns, which are somewhat 
orthogonal:

a) Sticking to the rules
I think in some cases where a few days have been missed, we should have enough 
flexibility to bend the rules.
In fact, if say a crucial part of PVHv2 missed the deadline by a few days, we 
would probably bend the rules. 

Of course whether something is a "niche" feature is in view of the beholder.

b) Risk 
This is something which is clearly important.

c) PR Perspective
I am somewhat concerned that we do not have a lot of stuff for good media 
coverage for Xen 4.10

This is a relatively small release: from what I can see we have approximately 
only around 235 patch series (4.9 had 481, 4.8 had 575)
Admittedly the number of patches in 4.10 is quite high: approx 1707 patches 
(with 1549 in 4.9 and 1245 in 4.8)
A lot of it seems to be groundwork for 4.11 (or even 4.12)

Besides PVHv2, we don't have a lot of big marketable new stuff which would 
enable us to get press quotes. 
Admittedly, I have not put a list of marketable features together yet.
A feature such as MBA, would help from a PR perspective.


But ultimately this is going to be Julien's call.

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Please Welcome Stefano Stabellini as new Security Team Member

[Lars Kurth]

Dear Community members,

I am pleased to announce that Stefano Stabellini has been nominated and
voted to become a new member of the Xen Project security team. The vote
was unanimous with all security team members voting in favour of
Stefano's membership.

Stefano has made significant contributions to the Xen Project over the 
years (see https://xen.biterg.io:443/goto/73478f514bcbb3eafc68354837a6b70e),
has been a maintainer and project leadership team member for a long time,
and is also a member of the QEMU security team.

Best Regards 
Lars Kurth 
Xen Project Community Manager 
Chairman of Xen Project Advisory Board

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.9.1 and 4.7.4

[Lars Kurth]

Jan,

I had sent the mail an hour after I ran the scripts (had a meeting in between). 

I will look into the issue with XSA 226
On commits c7783d9c26fc191862d9883da22387340b1fab18 & 
d6aad635097d901b96df650e87f04687c9fb7bd2: I have to look into why these didn’t 
get picked up.
Maybe there is a bug in the tool

Lars

On 13/10/2017, 08:13, "Jan Beulich"  wrote:

>>> On 12.10.17 at 19:08,  wrote:
> for 4.9.1 the XSA status is
> 
> XSA 226 : Some patches not applied => check
> There is an extra chunk in the tree: see xsa226.png

I see there are outdated patches for this XSA still in xsa.git,
which I've now removed.

I can't seem to be able to confirm the difference your attached
image indicates - both xen.git and xsa.git have that change. I
could understand the tool spotting a difference in there being a
seemingly extra hunk changing xen/common/compat/grant_table.c
in xsa.git - this change was applied incrementally to the various
xen.git branches, as the issue was found (by osstest iirc) only
after the original patches had already gone in.

> XSA 237 : No patch found => check
> XSA 238 : No patch found => check
> XSA 239 : No patch found => check
> XSA 240 : No patch found => check
> XSA 241 : No patch found => check
> XSA 242 : No patch found => check
> XSA 243 : No patch found => check
> XSA 244 : No patch found => check
> These have only been released today and have not yet been applied

By the time you had sent the message, these had been applied for
several hours. But of course I don't know when you ran the script.

> For 4.7.4 the XSA status is
> 
> XSA 226 : Some patches not applied => check
> There is an extra chunk in the tree: see xsa226.png

Same as for 4.9.

> XSA 234 : No patch found => check
> ISSUE: This is genuinely missing (aka xsa234-4.8.patch)

Commit c7783d9c26fc191862d9883da22387340b1fab18.

> XSA 245 : Some patches not applied => check
> This is missing 
> xsa245/0002-xen-arm-Correctly-report-the-memory-region-in-the-du.patch

Commit d6aad635097d901b96df650e87f04687c9fb7bd2.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.9.1 and 4.7.4

[Lars Kurth]

Hi all,

for 4.9.1 the XSA status is

XSA 226 : Some patches not applied => check
There is an extra chunk in the tree: see xsa226.png

XSA 227 : All patches found (no need to check)
XSA 228 : All patches found (no need to check)

XSA 229 : No patch found => check
Linux only => no issue

XSA 230 : All patches found (no need to check)
XSA 231 : All patches found (no need to check)
XSA 232 : All patches found (no need to check)
XSA 233 : All patches found (no need to check)
XSA 234 : All patches found (no need to check)
XSA 235 : All patches found (no need to check)

XSA 237 : No patch found => check
XSA 238 : No patch found => check
XSA 239 : No patch found => check
XSA 240 : No patch found => check
XSA 241 : No patch found => check
XSA 242 : No patch found => check
XSA 243 : No patch found => check
XSA 244 : No patch found => check
These have only been released today and have not yet been applied

XSA 245 : Some patches not applied => check
This is a minor change to a comment at check-in => no issue

For 4.7.4 the XSA status is

XSA 226 : Some patches not applied => check
There is an extra chunk in the tree: see xsa226.png

XSA 227 : All patches found (no need to check)
XSA 228 : All patches found (no need to check)

XSA 229 : No patch found => check
XSA 229 : No patch found => check
Linux only => no issue

XSA 230 : All patches found (no need to check)
XSA 231 : All patches found (no need to check)
XSA 232 : All patches found (no need to check)
XSA 233 : All patches found (no need to check)

XSA 234 : No patch found => check
ISSUE: This is genuinely missing (aka xsa234-4.8.patch)

XSA 235 : All patches found (no need to check)

XSA 237 : No patch found => check
XSA 238 : No patch found => check
XSA 239 : No patch found => check
XSA 240 : No patch found => check
XSA 241 : No patch found => check
XSA 242 : No patch found => check
XSA 243 : No patch found => check
XSA 244 : No patch found => check
These have only been released today and have not yet been applied

XSA 245 : Some patches not applied => check
This is missing 
xsa245/0002-xen-arm-Correctly-report-the-memory-region-in-the-du.patch

Best Regards
Lars

On 06/10/2017, 14:33, "Jan Beulich"  wrote:

All,

with the goal of releasing around the end of the month, please point
out backport candidates you find missing from the respective staging
branches, but which you consider relevant. Note that commits

1c2ea5ee05 x86/hvm/dmop: fix EFAULT condition
4e383df865 x86/PV: fix/generalize guest nul selector handling

are already on my list.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.9.1 and 4.7.4

[Lars Kurth]

I will try and run the XSA scripts this week and get back to you
Lars

On 06/10/2017, 14:33, "Jan Beulich"  wrote:

All,

with the goal of releasing around the end of the month, please point
out backport candidates you find missing from the respective staging
branches, but which you consider relevant. Note that commits

1c2ea5ee05 x86/hvm/dmop: fix EFAULT condition
4e383df865 x86/PV: fix/generalize guest nul selector handling

are already on my list.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC v2] Add SUPPORT.md

[Lars Kurth]

> On 27 Sep 2017, at 13:57, Robert VanVossen  
> wrote:
> 
> 
> 
> On 9/26/2017 3:12 AM, Dario Faggioli wrote:
>> [Cc-list modified by removing someone and adding someone else]
>> 
>> On Mon, 2017-09-25 at 16:10 -0700, Stefano Stabellini wrote:
>>> On Mon, 11 Sep 2017, George Dunlap wrote:
 +### RTDS based Scheduler
 +
 +Status: Experimental
 +
 +A soft real-time CPU scheduler built to provide guaranteed CPU
 capacity to guest VMs on SMP hosts
 +
 +### ARINC653 Scheduler
 +
 +Status: Supported, Not security supported
 +
 +A periodically repeating fixed timeslice scheduler. Multicore
 support is not yet implemented.
 +
 +### Null Scheduler
 +
 +Status: Experimental
 +
 +A very simple, very static scheduling policy 
 +that always schedules the same vCPU(s) on the same pCPU(s). 
 +It is designed for maximum determinism and minimum overhead
 +on embedded platforms.

...

>> Actually, the best candidate for gaining security support, is IMO
>> ARINC. Code is also rather simple and "stable" (hasn't changed in the
>> last... years!) and it's used by DornerWorks' people for some of their
>> projects (I think?). It's also not tested in OSSTest, though, and
>> considering how special purpose it is, I think we're not totally
>> comfortable marking it as Sec-Supported, without feedback from the
>> maintainers.
>> 
>> George, Josh, Robert?
>> 
> 
> Yes, we do still use the ARINC653 scheduler. Since it is so simple, it hasn't
> really needed any modifications in the last couple years.
> 
> We are not really sure what kind of feedback you are looking from us in 
> regards
> to marking it sec-supported, but would be happy to try and answer any 
> questions.
> If you have any specific questions or requests, we can discuss it internally 
> and
> get back to you.

I think there are two sets of issues: one around testing, which Dario outlined.

For example, if you had some test harnesses that could be run on Xen release 
candidates, which verify that the scheduler works as expected, that would
help. It would imply a commitment to run the tests on release candidates.

The second question is what happens if someone reported a security issue on
the scheduler. The security team would not have the capability to fix issues in 
the ARINC scheduler: so it would be necessary to pull in an expert under 
embargo to help triage the issue, fix the issue and prove that the fix works. 
This 
would most likely require "the expert" to work to the timeline of the security
team (which may require prioritising it over other work), as once a security 
issue 
has been reported, the reporter may insist on a disclosure schedule. If we 
didn't 
have a fix in time, because we don't get expert bandwidth, we could be forced 
to 
disclose an XSA without a fix.

Does this make sense?

Lars



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC v2] Add SUPPORT.md

[Lars Kurth]

> On 12 Sep 2017, at 16:35, Rich Persaud  wrote:
> 
>> On Sep 11, 2017, at 13:01, George Dunlap  wrote:
>> 
>> +### XSM & FLASK
>> +
>> +Status: Experimental
>> +
>> +Compile time disabled
>> +
>> +### XSM & FLASK support for IS_PRIV
>> +
>> +Status: Experimental
> 
> In which specific areas is XSM lacking in Functional completeness, Functional 
> stability and/or Interface stability, resulting in "Experimental" status?  
> What changes to XSM would be needed for it to qualify for "Supported" status?

I think the issue in this case may be lack of automated testing or known 
testing - see 
https://www.mail-archive.com/xen-devel@lists.xen.org/msg123768.html 
  
I am not quite sure what the status of XSM testing in OSSTEST is: I think there 
is something there, but not sure what. 

> If there will be no security support for features in Experimental status, 
> would Xen Project accept patches to fix XSM security issues?  Could 
> downstream projects issue CVEs for XSM security issues, if these will not be 
> issued by Xen Project?

This question I have to defer to members of the security team.

Lars___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Results of Formal vote on Unicore Proposal : Passed & Next Steps

[Lars Kurth]

Hi everyone.

this is a quick note that the proposal for the Unicore Proposal (see 
http://markmail.org/message/ly3f6e53nv3jaxd7) has passed.

The calculation is as follows:
1) XAPI subproject: quorum not met; subproject’s vote discounted
2) Hypervisor subproject: quorum met
7 votes in favour (min required 3), no votes against => 100%

This means 100% agreement for the proposal

@Simon, @Felipe:
we need to start planning next steps and put the infrastructure in place
1) Lars: Update https://www.xenproject.org/help/mailing-list.html minios-devel 
section to reflect for the fact that the two projects will share a mailing list 
for now
2) Lars: Upload proposal to wiki
3) Simon, Felipe: Create some content for 
https://wiki.xenproject.org/wiki/Category:Unicore_Devel and 
https://wiki.xenproject.org/wiki/Category:Unicore, or let Lars know what 
roughly should be on the pages. Both, Simon & Felipe to ley Lars know of wiki 
user names such that write access can be given. We should probably highlight 
and link to some of the basic workflows for contributions. Eventually build, 
etc. instructions and other items specific to the project should be added. 
4) Lars to create wiki pages. 
5) Simon, Felipe: verify repositories which need to be created. We can start 
with personal repos, if that helps

@Simon, @Felipe, @Zibby:
6) Simon, Felipe, Lars, Zibby: Draft content for website landing page (see 
https://xenproject.org/developers/teams.html for examples). If you let me know 
of any highlights and/or presentations to add, we can just go ahead with this
7) Plan some PR (after enough of 1-6 has been done): e.g. blog post, linux.com 
article, xen-announce announcement, …

Regards
Lars

On 29/09/2017, 13:07, "Lars Kurth"  wrote:

Dear committers,

in accordance with https://www.xenproject.org/governance.html, I need the 
leadership teams of the two mature projects – the Hypervisor and the XAPI 
project – to vote on this proposal.

The Advisory Board is endorsing the proposal and there seems to be wide 
consensus amongst community members.

The specific voting rules in this case are outlined in section 
https://www.xenproject.org/governance.html#project-decisions

People allowed to vote on behalf of the Hypervisor project are:
Julien Grall, Andy Cooper, George Dunlap, Ian Jackson, Jan Beulich, Konrad 
R Wilk, Stefano Stabellini, Tim Deegan, Wei Liu

People allowed to vote on behalf of the XAPI project are:
Jon Ludlam, Chandrika Srinivasan, David Scott, Euan Harris, Germano 
Percossi, Siddharth Vinoth Kumar, John Else, Mate Lakat, Konstantina Chremmou, 
Rob Hoes, Si Beaumont, Thanos Makatos, Thomas Sanders, Vineeth Thampi 
Raveendran, Zheng Li

I propose to tally the votes by Friday the 6th of October. You can reply via
+1: for proposal
-1: against proposal
in public or private.

Votes will be tallied by subproject – aka the Hypervisor and XAPI project 
by % for the proposal - and then averaged across sub-projects that achieved the 
quorum. 

Sub-project needs to achieve the following quorum of votes in favour for 
the sub-project’s vote to count
Hypervisor: 3 + votes
XAPI: 5 + votes

The proposals are attached

Regards
Lars

PROPOSAL: Unicore
=

Roles
-
Project Leads:Simon Kuenzer  
 (co-lead)Felipe Huici   
 (co-lead)Florian Schmidt
Project Mentor:   Lars Kurth 
Project Sponsors: Stefano Stabellini 
  Wei Liu

Background
--
In recent years, several papers and projects dedicated to unikernels
have shown the immense potential for performance gains that these
have. By leveraging specialization and the use of minimalistic OSes,
unikernels are able to yield impressive numbers, including fast
instantiation times (tens of milliseconds or less), tiny memory
footprints (a few MBs or even KBs), high network throughput (10-40
Gb/s), and high consolidation (e.g., being able to run thousands of
instances on a single commodity server), not to mention a reduced
attack surface and the potential for easier certification. Unikernel
projects worthy of mention include MirageOS, ClickOS, Erlang on Xen,
OSv, HALVM, and Minicache, Rump, among others.

The fundamental drawback of unikernels is that they require that
applications be manually ported to the underlying minimalistic OS (e.g.
having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance
with unikernels, or no porting effort but decreased performance
and decreased efficiency with standard OS/VM images.
The goal of this proposal is to change this status quo by providing
a highly confi

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

On 18/09/2017, 05:17, "Felipe Huici"  wrote:

Hi Lars, all,

[cc’ing authors of Erlang on Xen, HalVM and Rump].

Thanks everyone for all of the support and useful comments. We’ve
incorporated a number of them into a new version of the document (attached
and pasted at the bottom for convenience) and for those that didn’t make
it we’re keeping track of them.

Lars, FYI, Simon also did a blog post regarding Unicore on unikernel.org
(https://devel.unikernel.org/t/unicore-a-new-unikernel-project/274).

Please let us know what the next steps are.

 I will set up the formal vote next week! It doesn’t need the full distribution 
list
Lars


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

Felipe, Simon,
a quick note to let you know that the Advisory Board in today’s AB meeting 
decided to endorse your proposal.
Let me know how you proceed: from my perspective, we can kick off a formal vote 
before you make modifications to the proposal, but I think it is better to post 
v2 first.
Lars

On 07/09/2017, 05:26, "Felipe Huici"  wrote:

Dear all,

Following up on discussions that Simon Kuenzer had with several of you at
the last Xen summit, we’re now submitting a Xen subproject proposal based
on our Unicore work. Could you please review it?

Thanks,

Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.


PROPOSAL: Unicore
=

Roles
-
Project Leads: Simon Kuenzer  (main lead)
   Felipe Huici  (co-lead)
   Florian Schmidt  (co-lead)
Project Mentor:  Lars Kurth 
Project Sponsor: -To be found-

Background
--
In recent years, several papers and projects dedicated to unikernels have
shown the immense potential for performance gains that these have. By
leveraging specialization and the use of minimalistic OSes, unikernels are
able to yield impressive numbers, including fast instantiation times (tens
of milliseconds or less), tiny memory footprints (a few MBs or even KBs),
high network throughput (10-40 Gb/s), and high consolidation (e.g., being
able to run thousands of instances on a single commodity server), not to
mention a reduced attack surface and the potential for easier
certification. Unikernel projects worthy of mention include MirageOS,
ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.

The fundamental drawback of unikernels is that they require that
applications be manually ported to the underlying minimalistic OS (e.g.
having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance with unikernels,
or no porting effort but decreased performance and decreased efficiency
with standard OS/VM images. The goal of this proposal is to change this
status quo by providing a highly configurable unikernel code base; we call
this base Unicore.

This project also aims to concentrate the various efforts currently going
on in the Xen community regarding minimalistic OSes (essentially different
variants of MiniOS). We think that splitting the community across these
variants is counter-productive and hope that Unicore will provide a common
place for all or most improvements and customizations of minimalistic
OSes. The long term goal is to replace something like MiniOS with a tool
that can automatically build such a minimalistic OS.

Unicore - The "Unikernel Core"
-
The high level goal of Unicore is to be able to build unikernels targeted
at specific applications without requiring the time-consuming, expert work
that building such a unikernel requires today. An additional goal (or
hope) of Unicore is that all developers interested in unikernel
development would contribute by supplying libraries rather than working on
independent projects with different code bases as it is done now. The main
idea behind Unicore is depicted in Figure 1 and consists of two basic
components:
 

[Attachment: unicore-oneslider.pdf]


Figure 1. Unicore architecture.

 
Library pools would contain libraries that the user of Unicore can select
from to create the unikernel. From the bottom up, library pools are
organized into (1) the architecture library tool, containing libraries
specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
virtualization) and user-space Linux; and (3) the main library pool,
containing a rich set of functionality to build the unikernel from. This
last library includes drivers (both virtual such as netback/netfront and
physical such as ixgbe), filesystems, memory allocators, schedulers,
network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
Python interpreter and debugging and profiling tools. These pools of
libraries constitute a code base for creating unikernels. As shown, a
library can be relatively large (e.g libc) or quite small (a scheduler),
which should allow for a fair amount of customization for the unikernel.
 

The Unicore build tool is in charge of compiling the application and the
selected libraries together to create a binary for a specific platform and
architecture (e.g., Xen on x86_64). The tool is currently inspired by
Linux’s kconfig system and consists of a set of Ma

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

Simon,

It looks to me as if there is some feedback: so it may make some sense to 
incorporate some of it and send out a version 2. We may also want to CC some 
reps from other unikernel projects but Mirage OS. Or you could point them to 
this thread in a separate mail through respective channels or a hub like 
unikernel.org. Whatever you think may work best. It’s your call.

And then have a formal vote, a week after v2 of the proposal. Does this work?

Lars

On 11/09/2017, 05:08, "Simon Kuenzer"  wrote:

Hi Alexander,

thanks a lot for your review.

On 10.09.2017 22:48, Alexander Dubinin wrote:
> Hi Felipe, all,
> 
> Great that it's going to start :) Looking forward to join :)

I am looking forward to your contributions. ;)

> 
> Just my 2 cents:
> 
> 1. Is this academic project, or it have specific goals and areas of 
> application? Would be good to have some practical use-cases and well 
> formulated list of problems (we all feel these by guts, but...), it 
> aiming to solve. IMHO that will help to prioritize functionality and get 
> usable result faster :)

It is kind of both, however we aim a strong focus on real world 
problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network 
Functions (VNFs), and others.
We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and 
others) and tried to apply them in the several areas. While doing this, 
we noticed that each area benefits differently from the properties that 
Unikernels give - which is great (e.g., instant boot times for MEC, high 
performance for NFV, resource efficiency for IoT). However, building and 
maintaining new Unikernels (as we did with ClickOS, MiniCache, and 
Minipython) is currently painful.
Because of different focuses on properties and ported/implemented 
applications, most Unikernel today are bound to their own OS layers 
(e.g., ClickOS uses a different Mini-OS than Mirage). Each application 
requires a different subset of OS layers but also enables different 
optimizations of them.

In order to solve this, we came up with the Unicore proposal. But I 
agree with your suggestion at this point: It helps for the project start 
to focus on some initial areas. For now, I hope this is driven by the 
first contributors, and I have personally IoT in mind. Since the project 
goal is so ambitious, we should keep the long-term goal in mind from the 
beginning.

> 
> 2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is 
> should be supplemented by some security layer in control/stub domains as 
> well. So far only known implementation is OpenXT, but it is very 
> specific. Probably some generalized security layer needed in Unicore to 
> supplement FLASK/XSM... Correct me please, if I misunderstanding :)

I agree that many projects (especially embedded, stubdomains, driver 
domains, NFV) have a vested interest in security and isolation. In my 
view, XSM/FLASK further restricts what a VM can do and sounds kind of 
orthogonal to the functionality of a VM (am I right?). The fact that 
Unikernels should only pick components that are actually required to do 
the job reduces the attack surface compared to general purpose OSes.
Do you see further value with FLASK/XSM which requires early 
implementation and design decisions for Unicore? As far as I can tell 
something like Flask is implemented mostly in the hypervisor and 
toolstack, not in the guests themselves, is this right?


Thanks,

Simon

> 
> Regards,
>Alexander
> 
> On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici  <mailto:felipe.hu...@neclab.eu>> wrote:
> 
> Hi Wei, Stefano,
> 
> Thank you so much for agreeing to be sponsors! I’ll update the 
document.
> 
> — Felipe
> 
> 
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
> 
> e-mail:
> felipe.hu...@neclab.eu <mailto:felipe.hu...@neclab.eu>
> 
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
> 
> 
> 
> 
> On 9/8/17, 1:00 PM, "Lars Kurth"  <mailto:lars.ku...@citrix.com

Re: [Xen-devel] Xen 4.8.2 released

[Lars Kurth]

Fixed
Apologies
Lars

On 11/09/2017, 03:26, "Marek Marczykowski-Górecki" 
 wrote:

On Mon, Sep 11, 2017 at 03:06:48AM -0600, Jan Beulich wrote:
> >>> On 10.09.17 at 01:53,  wrote:
> > On Wed, Sep 06, 2017 at 08:42:33AM -0600, Jan Beulich wrote:
> >> All,
> >> 
> >> I am pleased to announce the release of Xen 4.8.2. This is
> >> available immediately from its git repository
> >> 
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.8 
> >> (tag RELEASE-4.8.2) or from the XenProject download page
> >> 
http://www.xenproject.org/downloads/xen-archives/xen-project-48-series/xen-482.html
 
> >> (where a list of changes can also be found).
> >> 
> >> We recommend all users of the 4.8 stable series to update to this
> >> latest point release.
> > 
> > The announcement on the website has wrong link (to 4.8.1 instead of
> > 4.8.2).
> 
> Hmm, thanks for pointing this out, but unless Lars knows without
> further context where this wrong link is, it would have helped if
> you identified the page having that bad link. I can spot such a
> wrong link in the blog post - is that what you're referring to? 

Yes, this one:
https://blog.xenproject.org/2017/09/06/xen-project-4-8-2-is-available/

> Lars,
> could you correct that?
> 
> Jan
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

@Wei, @Stefano,

On 07/09/2017, 22:16, "Stefano Stabellini"  wrote:

Hi all,

I would be glad to sponsor this proposal. I think it will be of great
benefit to the ecosystem. Let me know if I need to do anything specific.

Basically, all which is needed is an agreement. Which we have from you both. 
Felipe, can then add your names to the proposal.

Looking out for the evolving project and helping (e.g. through advice) is not 
strictly necessary, but always welcome.

Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

On 07/09/2017, 14:24, "Andrew Cooper"  wrote:
> Unicore - The "Unikernel Core"
> -
> The high level goal of Unicore is to be able to build unikernels targeted
> at specific applications without requiring the time-consuming, expert work
> that building such a unikernel requires today. An additional goal (or
> hope) of Unicore is that all developers interested in unikernel
> development would contribute by supplying libraries rather than working on
> independent projects with different code bases as it is done now. The main
> idea behind Unicore is depicted in Figure 1 and consists of two basic
> components:

Have you encountered the netbsd rumpkernel project?  I don't it
referenced in your text (apologies if I've missed it).


http://events.linuxfoundation.org/sites/events/files/slides/xdps15-talk-final_0.pdf
is a presentation on from a previous Xen Developer Summit, and

One particular need build solution there was to not alter the build
system of the individual apps, and pass in the rest of the microkernel
as a cross-compile environment.  It's not entirely clear how you plan to
do this part of the building, but anything which involves modifying the
end applications is going to cause a non-trivial maintenance burden.

I don’t think we have to answer design questions up-front. Although it may make 
sense, to track some key open design and architectural decisions in a section 
at the end of the document, such that they are not forgotten.

> [Attachment: unicore-oneslider.pdf]
>
>
> Figure 1. Unicore architecture.
>
>  
> Library pools would contain libraries that the user of Unicore can select
> from to create the unikernel. From the bottom up, library pools are
> organized into (1) the architecture library tool, containing libraries
> specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
> platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
> virtualization) and user-space Linux;

On the x86 Xen side of things, you should treat PV and HVM guests as
different platforms, and their tradeoffs are quite different.

The one semi-supported microkernel in the Xen world is stub-qemu, and in
principle this does give better isolation than qemu running in dom0, but
it also exposes other attack surfaces.  If you assume an HVM guest has
compromised its stub-qemu, it means that security issues exposed only to
PV guests are within the reach of an HVM guest.  In this circumstance,
having an HVM stub qemu would give the system a reduced attack surface
compared to a PV stub qemu.

I think this question could also be treated like an open design/architecture 
decision which should be recorded.

Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC] Unicore Subproject Proposal

[Lars Kurth]

Hi all,

there is a technical issue which still needs resolving: we need a Sponsor. I am 
thinking of Wei – he would qualify as a Hypervisor Leadership team member and 
it would have the benefit of making sure that the MiniOS angle is covered. I 
asked Wei, and he will get back to us once he read the proposal.

I also want to highlight this proposal at the next AB board meeting, Sept 19th. 
It would be good, if most feedback could be given in the next week, such that 
a) we have time to make mods, b) I have a good baseline to share with the AB. I 
would need to share an updated proposal on the 18th at the latest.

Technically, the subproject does not need AB approval, as there is no financial 
impact, but it is always good to have it. 

Regards
Lars

On 07/09/2017, 11:26, "Felipe Huici"  wrote:

Dear all,

Following up on discussions that Simon Kuenzer had with several of you at
the last Xen summit, we’re now submitting a Xen subproject proposal based
on our Unicore work. Could you please review it?

Thanks,

Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.


PROPOSAL: Unicore
=

Roles
-
Project Leads: Simon Kuenzer  (main lead)
   Felipe Huici  (co-lead)
   Florian Schmidt  (co-lead)
Project Mentor:  Lars Kurth 
Project Sponsor: -To be found-

Background
--
In recent years, several papers and projects dedicated to unikernels have
shown the immense potential for performance gains that these have. By
leveraging specialization and the use of minimalistic OSes, unikernels are
able to yield impressive numbers, including fast instantiation times (tens
of milliseconds or less), tiny memory footprints (a few MBs or even KBs),
high network throughput (10-40 Gb/s), and high consolidation (e.g., being
able to run thousands of instances on a single commodity server), not to
mention a reduced attack surface and the potential for easier
certification. Unikernel projects worthy of mention include MirageOS,
ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.

The fundamental drawback of unikernels is that they require that
applications be manually ported to the underlying minimalistic OS (e.g.
having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance with unikernels,
or no porting effort but decreased performance and decreased efficiency
with standard OS/VM images. The goal of this proposal is to change this
status quo by providing a highly configurable unikernel code base; we call
this base Unicore.

This project also aims to concentrate the various efforts currently going
on in the Xen community regarding minimalistic OSes (essentially different
variants of MiniOS). We think that splitting the community across these
variants is counter-productive and hope that Unicore will provide a common
place for all or most improvements and customizations of minimalistic
OSes. The long term goal is to replace something like MiniOS with a tool
that can automatically build such a minimalistic OS.

Unicore - The "Unikernel Core"
-
The high level goal of Unicore is to be able to build unikernels targeted
at specific applications without requiring the time-consuming, expert work
that building such a unikernel requires today. An additional goal (or
hope) of Unicore is that all developers interested in unikernel
development would contribute by supplying libraries rather than working on
independent projects with different code bases as it is done now. The main
idea behind Unicore is depicted in Figure 1 and consists of two basic
components:
 

[Attachment: unicore-oneslider.pdf]


Figure 1. Unicore architecture.

 
Library pools would contain libraries that the user of Unicore can select
from to create the unikernel. From the bottom up, library pools are
organized into (1) the architecture library tool, containing libraries
specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
virtualization) and user-space Linux; and (3) the main library pool,
containing a rich set of functionality to build the unikernel from. This
last library includes drivers (both virtual such as netback/netfront and
physical such as ixgbe), filesystems, memory allocators, schedulers,
network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
Python interpreter and debugging and profiling tools. These pools of
libraries constitute a code base for creating unikernels. As 

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

Julien,
we are waiting for George to come back from holiday to post the proposal for 
review, after Ian and I had done the prep work
This was then discussed at the summit, and there were a couple off-line 
responses on ARM support, etc.
Lars

On 17/08/2017, 15:48, "Julien Grall"  wrote:

Hi,

I wanted to bump this thread. I saw that the page still contain "Note 
that we will add complete information related to Xen Project 4.9, in the 
week after the 4.9 release.".

It looks like to me we added some features, but I am not sure if we 
added all of them.

Cheers,

    On 27/06/17 09:53, Lars Kurth wrote:
> Hi all, (I think I CCed all stake-holders)
>
> to finish off the release documentation for 4.9, I need to add an extra
> column
> to https://wiki.xenproject.org/wiki/Xen_Project_Release_Features –
> because I was travelling, this dropped of my radar. There several
> decisions to be made:
> A) Decide which "features" to add
> B) Decide on the status of the feature
> C) Deal with status changes of any past features
>
> The first goal would be to decide on A and any new "features" under C.
> For B, I am OK to add "???" for now and point to this thread, until we
> have concluded the discussion
>
> Note that I tracked some of this as preparation for getting CNA status.
>  Items marked with * are not yet in the discussion document that I
> created for the security team and which we intend to discuss at the 
summit.
>
> For all of these, the naming convention is "Section in document" >
> "Feature" : "Support status". The definition of support status is added
> at the end of the mail: note that the text has not yet been fully
> agreed, but seems to reflect fairly well how we handled stuff in the past.
>
> == On A / B: I think we should add ==
> - Resource Management > Null Scheduler : tech preview or experimental
> - Virtual Firmware or PV Bootloader Support (not sure which) >  x86/Boot
> Xen on EFI platforms using GRUB2*  : ???
> - Hardware > ARM/Alternative Runtime Patching (ARM32 and ARM64): ???
> [note that this should probably have been added for 4.8, but I didn't
> add it]
> - Hardware > ARM/System Error Protection* : ???
> - Hardware > ARM/Wait for Virtual Interrupt* : ???
> - Hardware > x86/AVX512/Neural Network Instructions AVX512_4VNNIW* : ???
> - Hardware > x86/AVX512/Multiply Accumulation Single precision
> AVX512_4FMAPS* : ???
> - Device Models > DMOP (Device Model Operation Hypercall)  : ???
>
> New Heading:  PV Protocols and Drivers
> - PV Protocols and Drivers > pvcalls : tech preview or experimental
> - PV Protocols and Drivers > 9pfs : tech preview or experimental
> - PV Protocols and Drivers* > sndif (sound device) : tech preview or
> experimental
> - PV Protocols and Drivers* > displif (PV display) : tech preview or
> experimental
>
> Did I miss anything?
>
> == On C ==
> - Security > Live Patching -
> see 
https://lists.xenproject.org/archives/html/xen-devel/2017-06/threads.html#03039
> - Security > Alternative 2pm : Supported – I think we should split this
> out – it is currently implicitly covered under "Virtual Machine
> Introspection"
>
> If we introduce a new heading "PV Protocols and Drivers" we should
> probably list all the common ones as supported in this heading, e.g.
> - PV Protocols and Drivers* > default (net, block, console, keyboard,
> mouse) : supported
>
> There are also USB and framebuffer, which I am not sure whether they
> should be supported and if not, what their status is
> - PV Protocols and Drivers* > USB : ???
> - PV Protocols and Drivers* > framebuffer : ???
>
> Suggestions are welcome
>
> Best Regards
> Lars
>
> 
>
> ## Definitions
>
> ### User-facing Support Criteria
>
>  * **Functionally complete:** Does it behave like a fully functional
>feature?  Does it work on all expected platforms, or does it only work
>for a very specific sub-case?  Does it have a sensible UI, or do you
>have to have a deep understanding of the internals to get it to work
>properly?
>
>  * **Functional stability:** What is the risk of it exhibiting bugs?
>
>General answers to the above:
>
>- *Here be dragons*: Pretty likely to still crash / fail to work.  Not
 

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Jan,
it’s been a while. Did you want to pick this up at some point again? I guess 
the check we have done so far is by now out-of-date. Not sure whether anyone 
tagged anything
It would also be a good opportunity for you guys to test run my script (Wei ran 
it and it worked fine, but he didn’t comb through any results)
Lars

On 27/07/2017, 19:34, "Lars Kurth"  wrote:

Quick info/update:

> XSA-222: line 51 in the log shows a real difference: this is a known bug
> in the tool where the diff file chunks are in a different order

This is now fixed in the last version of the scripts and the script
correctly handles this case

Lars

On 18/07/2017, 18:43, "Lars Kurth"  wrote:

>Hi all,
>
>@Jan: you may want to check the note on XSA-218 and XSA-224
>
>I removed Text::Diff module, which should fix the dependency problem.
>
>I also fixed the script such that it will fetch patches from
>http://xenbits.xenproject.org/xsa if the xsa.git has not been checked out
>in the location in
>
>The script still depends on: Getopt, Cwd, File packages, which I hope are
>standard.
>
>Crude check
>===
>I first ran the scripts using
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --getlogs
>--html > xsamatch.html
>
>Which checks name signatures only.
>Note that 
>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-48
>1
>.html tells us that XSA 212 was applied last.
>
>The output shows that XSA-215 has not been applied. Not a problem, because
>XSA-215 applies to 64-bit Xen versions of 4.6 and earlier only.
>
>All the other ones have patches with matching names that have been
>applied.
>
>Detailed check
>==
>I then ran using
>
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart > xsamatchsmart.html
>
>
>which requires that xsa.git is checked out, which has restricted access
>(security team members only).
>
>The output shows some problems, for which I used
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart --debug > xsamatchsmartdebug.html
>
>
>This then tells me that there are a few real differences between 4.8.2 and
>the XSA database
>
>XSA-218: line 32 in the log shows a real difference: see XSA-218-32.png
>XSA-224: line 72 in the log shows a real difference: see XSA-224-72a.png &
>XSA-224-72b.png
>
>
>XSA-222: line 51 in the log shows a real difference: this is a known bug
>in the tool where the diff file chunks are in a different order
>
>Script Improvements
>===
>I can't use --xsadir https://xenbits.xenproject.org/xsa as I can't read
>files from a website. I can, fetch the file from
>https://xenbits.xenproject.org/xsa via the LWP:Simple package, which I
>don't think is installed on Linux distros by default. Alternatively I
>could use wget, which may be better.
>
>
>I will play with this and see whether I can add it.
>
>Cheers
>Lars
>
>
>On 18/07/2017, 14:53, "Wei Liu"  wrote:
>
>>On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
>>> Wei,
>>> I attached the list output from xsa-list-send starting from 206
>>> If you look at 
>>> 
>>>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-
>>>4
>>>81
>>> .html, you may want to start using from 213+
>>
>>[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
>>Can't locate Text/Diff.pm in @INC (you may need to install the
>>Text::Diff module) (@INC contains: /etc/perl
>>/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
>>/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
>>/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
>>/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
>>./match-xsa line 14.
>>BEGIN failed--compilation aborted at ./match-xsa line 14.
>>
>>Would be useful to give a list of perl modules required.
>



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/3] docs: add xen-release-management.pandoc

[Lars Kurth]

Wei

On 31/07/17 12:22, Wei Liu wrote:
> A document for the release manager.
>
> Signed-off-by: Wei Liu 

Reviewed-by: Lars Kurth 

  Regards
  Lars


On 08/08/2017, 12:03, "Julien Grall"  wrote:

Hi Wei,

On 31/07/17 12:22, Wei Liu wrote:
> A document for the release manager.
>
> Signed-off-by: Wei Liu 

Reviewed-by: Julien Grall 

Cheers,

> ---
>  docs/process/xen-release-management.pandoc | 594 
+
>  1 file changed, 594 insertions(+)
>  create mode 100644 docs/process/xen-release-management.pandoc
>
> diff --git a/docs/process/xen-release-management.pandoc 
b/docs/process/xen-release-management.pandoc
> new file mode 100644
> index 00..afdf559429
> --- /dev/null
> +++ b/docs/process/xen-release-management.pandoc
> @@ -0,0 +1,594 @@
> +% Xen Release Management
> +% Wei Liu <>
> +% Revision 1
> +
> +# Motivation
> +
> +Over the years we have had different people signing up as the Release 
Manager
> +of Xen. It would be rather wasteful if every new Release Manager has to 
go over
> +everything and tripped over by the same mistakes again and again.
> +
> +This file intends to document the process of managing a Xen release. It 
is
> +mainly written for Release Manager, but other roles (contributors,
> +maintainers and committers) are also encouraged to read this document, so
> +that they can have an idea what to expect from the Release Manager.
> +
> +# Xen release cycle
> +
> +The Xen hypervisor project now releases twice a year, at the beginning of
> +June and the beginning of December. The actual release date depends on a 
lot
> +of factors.
> +
> +We can roughly divide one release into two periods. The development 
period
> +and the freeze period. The former is 4 months long and the latter is 
about 2
> +months long.
> +
> +During development period, contributors submit patches to be reviewed and
> +committed into xen.git. All feature patches must be committed before a 
date,
> +which is normally called the "cut-off date", after which the freeze 
period
> +starts. There will be a date before which all patches that wish to be 
merged
> +for the release should be posted -- it is normally called the "last 
posting
> +date" and it is normally two weeks before the "cut-off date".
> +
> +During freeze period, the tree is closed for new features. Only bug 
fixes are
> +accepted. This period can be shorter or longer than 2 months. If it ends 
up
> +longer than 2 months, it eats into the next development period.
> +
> +Here is a conjured up example (use ```cal 2017``` to get an idea):
> +
> +* Development period: 2017 June 11 - 2017 September 29
> +* the "cut-off date" is 2017 September 29
> +* the "last posting date" is 2017 September 15
> +* Freeze period: 2017 October 2 - 2017 December 7
> +* the anticipated release date is 2017 December 7
> +
> +# The different roles in a Xen release
> +
> +## Release Manager
> +
> +A trusted developer in the community that owns the release process. The 
major
> +goal of the Release Manager is to make sure a Xen release has high 
quality
> +and doesn't slip too much.
> +
> +The Release Manager will not see much workload during development 
period, but
> +expects to see increasing workload during the freeze period until the 
final
> +release. He or she is expected to keep track of issues, arrange RCs,
> +negotiate with relevant stakeholders, balance the need from various 
parties
> +and make difficult decisions when necessary.
> +
> +The Release Manager essentially owns xen-unstable branch during the 
freeze
> +period. The Committers will act on the wishes of the Release Manager 
during
> +that time.
> +
> +## Maintainers
> +
> +A group of trusted developers who are responsible for certain components 
in
> +xen.git. They are expected to respond to patches / questions with regard 
to
> +their components in a timely manner, especially during the freeze period.
> +
> +## Committers
> +
> +A group of trusted maintainers who can commit to xen.git. During the
> +development window they normally push things as they see fit. During the
> +freeze period they transfer xen-unstable branch ownership and act on the
> +wishes of the Release Manager. That normally means they need to have an
> +Rele

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Quick info/update:

> XSA-222: line 51 in the log shows a real difference: this is a known bug
> in the tool where the diff file chunks are in a different order

This is now fixed in the last version of the scripts and the script
correctly handles this case

Lars

On 18/07/2017, 18:43, "Lars Kurth"  wrote:

>Hi all,
>
>@Jan: you may want to check the note on XSA-218 and XSA-224
>
>I removed Text::Diff module, which should fix the dependency problem.
>
>I also fixed the script such that it will fetch patches from
>http://xenbits.xenproject.org/xsa if the xsa.git has not been checked out
>in the location in
>
>The script still depends on: Getopt, Cwd, File packages, which I hope are
>standard.
>
>Crude check
>===
>I first ran the scripts using
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --getlogs
>--html > xsamatch.html
>
>Which checks name signatures only.
>Note that 
>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-48
>1
>.html tells us that XSA 212 was applied last.
>
>The output shows that XSA-215 has not been applied. Not a problem, because
>XSA-215 applies to 64-bit Xen versions of 4.6 and earlier only.
>
>All the other ones have patches with matching names that have been
>applied.
>
>Detailed check
>==
>I then ran using
>
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart > xsamatchsmart.html
>
>
>which requires that xsa.git is checked out, which has restricted access
>(security team members only).
>
>The output shows some problems, for which I used
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart --debug > xsamatchsmartdebug.html
>
>
>This then tells me that there are a few real differences between 4.8.2 and
>the XSA database
>
>XSA-218: line 32 in the log shows a real difference: see XSA-218-32.png
>XSA-224: line 72 in the log shows a real difference: see XSA-224-72a.png &
>XSA-224-72b.png
>
>
>XSA-222: line 51 in the log shows a real difference: this is a known bug
>in the tool where the diff file chunks are in a different order
>
>Script Improvements
>===
>I can't use --xsadir https://xenbits.xenproject.org/xsa as I can't read
>files from a website. I can, fetch the file from
>https://xenbits.xenproject.org/xsa via the LWP:Simple package, which I
>don't think is installed on Linux distros by default. Alternatively I
>could use wget, which may be better.
>
>
>I will play with this and see whether I can add it.
>
>Cheers
>Lars
>
>
>On 18/07/2017, 14:53, "Wei Liu"  wrote:
>
>>On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
>>> Wei,
>>> I attached the list output from xsa-list-send starting from 206
>>> If you look at 
>>> 
>>>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-
>>>4
>>>81
>>> .html, you may want to start using from 213+
>>
>>[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
>>Can't locate Text/Diff.pm in @INC (you may need to install the
>>Text::Diff module) (@INC contains: /etc/perl
>>/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
>>/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
>>/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
>>/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
>>./match-xsa line 14.
>>BEGIN failed--compilation aborted at ./match-xsa line 14.
>>
>>Would be useful to give a list of perl modules required.
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen checkpatch infrastructure design

[Lars Kurth]

> We did not plan any changes into the clang-format yet. We have to check with 
> Artem our next steps.

I don't think this is a huge task. I think it is also something that is 
important enough for the Advisory Board to make funds available – assuming this 
can't be done otherwise (but I can't promise it would be approved and going 
this route will take fairly long and would require quite a bit of paperwork)

> Artem is on vacations now.
Let me know when you talked to Artem

Lars

From: Iurii Artemenko 
mailto:iurii_arteme...@epam.com>>
Date: Wednesday, 26 July 2017 at 17:08
To: Lars Kurth mailto:lars.ku...@citrix.com>>, Juergen 
Gross mailto:jgr...@suse.com>>
Cc: Wei Liu mailto:wei.l...@citrix.com>>, Andrew Cooper 
mailto:andrew.coop...@citrix.com>>, Ian Jackson 
mailto:ian.jack...@citrix.com>>, 'Jan Beulich' 
mailto:jbeul...@suse.com>>, xen-devel 
mailto:xen-de...@lists.xenproject.org>>, cardoe 
mailto:car...@cardoe.com>>, Andrii Anisov 
mailto:andrii_ani...@epam.com>>, Lars Kurth 
mailto:lars.kurth@gmail.com>>, Artem Mygaiev 
mailto:artem_myga...@epam.com>>
Subject: Re: [Xen-devel] Xen checkpatch infrastructure design


Hi Lars,


> I was wondering how you deal with the gaps. I suppose these gaps could 
> possibly be covered in clang-format-diff.py

> Of course this info may be out-of-date


I assumed that everything is fine with clang-format and started with python 
script.

Now I have checked both clang-format-3.9 and 5.0, all of those gaps are still 
remaining.

Also there are no corespondent changes expected in Clang-6.0 which currently 
in-progress.


> I wanted to double check, as we had previously looked into clang-format, and 
> it showed some gaps with what it can be used for in Xen coding styles.  Which 
> is why we tried to get > agreement - > and got it - to upstream Xen related 
> changes into clang–format


We did not plan any changes into the clang-format yet. We have to check with 
Artem our next steps.

Artem is on vacations now.


Regards

Iurii
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling

[Lars Kurth]

@Committers, @Julien,

> On 26 Jul 2017, at 13:53, Lars Kurth  wrote:

>> Improving the Process
>> =
>> 
>> JIRA
>> 
>> Open source projects do NOT need licenses for JIRA: these was raised by 
>> OpenXT folks, who use JIRA
>> 
>> ACTION: Lars to follow up with Atlassian - 
>> https://www.atlassian.com/software/views/open-source-license-request 
>> <https://www.atlassian.com/software/views/open-source-license-request>
> 
> This is covered and I raised two tickets
> #CA-297630 confirmed: Open Source license request by Xen Project
>   #CA-297631 confirmed: Purchasing and licensing request (this covers how to 
> migrate the account if approved)

Hi everyone, our account was converted to an open source account. So in theory, 
we should not have a number limitation on people signed up. See mail below.
If there are any issues/unexpected changes in behaviour, please let me know

> Begin forwarded message:
> 
> From: Sen Geronimo 
> Subject: [SUPPORT] Comment posted to request #CA-297630: Open Source license 
> request by Xen Project
> Date: 26 July 2017 at 16:04:01 BST
> To: 
> 
> Hi Lars,
> 
> Thanks for reaching out.
> 
> We are happy to grant you the requested Open Source Cloud subscription. Your 
> Atlassian Cloud Instance (xenproject.atlassian.net 
> <http://xenproject.atlassian.net/>) has been converted to Open Source.
> 
> Please note, third-party Cloud add-ons are not available within the Open 
> Source program. If you wish to include Marketplace 
> <https://marketplace.atlassian.com/> plug-ins, you will be required to 
> provide payment.
> 
> Yo
> 

The next step would be to draft some sort of policy (I am happy to draft it), 
but want to chat with key committers and Julien to get a sense of
a) how open we want https://xenproject.atlassian.net to be
b) what the process/criteria would be for giving people write access
c) what we expect them to do/not do

There is still the issue of labels vs components for wiki page. Maybe we can 
roll this into the proposal
@Julien: when would you have time for a chat

Lars___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] A document for Xen release management, v2

[Lars Kurth]

Hi all,

> On 25 Jul 2017, at 14:25, Wei Liu  wrote:
> 
> Hi all
> 
> This is v2 of this document.
> 
> Lars, please insert your section as you see fit.

Done + some other minor mods

> ---8<---
> % Xen Release Management
> % Wei Liu <>
> % Revision 1
> 
> # Motivation
> 
> Over the years we have had different people signing up as the Release Manager
> of Xen. It would be rather wasteful if every new Release Manager has to go 
> over
> everything and tripped over by the same mistakes again and again.
> 
> This file intends to document the process of managing a Xen release. It is
> mainly written for Release Manager, but other roles (contributors,
> maintainers and committers) are also encouraged to read this document, so
> that they can have an idea what to expect from the Release Manager.
> 
> # Xen release cycle
> 
> The Xen hypervisor project now releases twice a year, at the beginning of
> June and the beginning of December. The actual release date depends on a lot
> of factors.
> 
> We can roughly divide one release into two periods. The development period
> and the freeze period. The former is 4 months long and the latter is about 2
> months long.
> 
> During development period, contributors submit patches to be reviewed and
> committed into xen.git. All feature patches must be committed before a date,
> which is normally called the "cut-off date", after which the freeze period
> starts. There will be a date before which all patches that wish to be merged
> for the release should be posted -- it is normally called the "last posting
> date" and it is normally two weeks before the "cut-off date".
> 
> During freeze period, the tree is closed for new features. Only bug fixes are
> accepted. This period can be shorter or longer than 2 months. If it ends up
> longer than 2 months, it eats into the next development period.
> 
> Here is a conjured up example (use cal 2017 to get an idea):
> 
> * Development period: 2017 June 11 - 2017 September 29
>* the "cut-off date" is 2017 September 29
>* the "last posting date" is 2017 September 15
> * Freeze period: 2017 October 2 - 2017 December 7
>* the anticipated release date is 2017 December 7
> 
> # The different roles in a Xen release
> 
> ## Release Manager
> 
> A trusted developer in the community that owns the release process. The major
> goal of the Release Manager is to make sure a Xen release has high quality
> and doesn't slip too much.
> 
> The Release Manager will not see much workload during development period, but
> expects to see increasing workload during the freeze period until the final
> release. He or she is expected to keep track of issues, arrange RCs,
> negotiate with relevant stakeholders, balance the need from various parties
> and make difficult decisions when necessary.
> 
> The Release Manager essentially owns xen-unstable branch during the freeze
> period. The committers will act on the wishes of the Release Manager during
> that time.
> 
> ## Maintainers
> 
> A group of trusted developers who are responsible for certain components in
> xen.git. They are expected to respond to patches / questions with regard to
> their components in a timely manner, especially during the freeze period.
> 
> ## Committers
> 
> A group of trusted maintainers who can commit to xen.git. During the
> development window they normally push things as they see fit. During the
> freeze period they transfer xen-unstable branch ownership and act on the
> wishes of the Release Manager. That normally means they need to have an
> Release Ack in order to push a patch.
> 
> ## Contributors
> 
> Contributors are also expected to respond quickly to any issues regarding the
> code they submitted during development period. Failing that, the Release
> Manager might decide to revert the changes, declare feature unsupported or
> take any action he / she deems appropriate.
> 
> ## The Security Team
> 
> The Security Team operates independently. The visibility might be rather
> limited due to the sensitive nature of security work. The best action the
> Release Manager can take is to set aside some time for potential security
> issues to be fixed.
> 
> ## The Release Technician
> 
> The Release Technician is the person who tags various trees, prepares tarball
> etc. He or she acts on the wishes of the Release Manager. Please make sure
> the communication is as clear as it can be.
> 
> ## The Community Manager
> 
> The Community Manager owns xenproject.org infrastructure. He or she is
> responsible for updating various web archives, updating wiki pages and
> coordinating with the PR Personnel.
> 
> ## The PR Personnel
> 
> They are responsible for coordinating with external reporters to publish Xen
> release announcement. The Release Manager should be absolutely sure the
> release is going out on a particular date before giving them the signal to
> proceed, because there is a point of no return once they schedule a date with
> external reporters.
> 
> # What h

Re: [Xen-devel] Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling

[Lars Kurth]

Update on some actions

> On 18 Jul 2017, at 13:18, Lars Kurth  wrote:
> 
> 
> ACTION (Wei, Julien, Lars): Have a how to be release manager file (needs to 
> contain some of the stuff in Release technician checklist)
> * RM file: clear set of criteria on PR. Go/NoGo part
> * See 
> https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#01500
>  for a starting point
> 
> ACTION (Lars/Wei/Julien) to write "standard e-mail templates for common 
> stuff" rather than re-doing these every single time
> 
> ACTION (Ian): Clean up release technician checklist after we have the how to 
> be release manager file
> 
> ACTION (Wei/Julien): Additional stuff to add to the templates/RM guide
> * Add hand-over of tasks for Release Manager responsibility to the "how to be 
> release manager" file
> * Add clear reminders in particular at the beginning of a release into e-mail 
> templates: such as "put dates X,Y, Z in your calendar"
> * Communicate better when tree is open again (add to checklist and templates)

This is covered in 
* http://markmail.org/message/lvig3u2gcqt3nwgs 
<http://markmail.org/message/lvig3u2gcqt3nwgs> (v2)
* http://markmail.org/message/4as7zfl2tfqhpdyo 
<http://markmail.org/message/4as7zfl2tfqhpdyo> (v1)

> Improving the Process
> =
> 
> JIRA
> 
> Open source projects do NOT need licenses for JIRA: these was raised by 
> OpenXT folks, who use JIRA
> 
> ACTION: Lars to follow up with Atlassian - 
> https://www.atlassian.com/software/views/open-source-license-request 
> <https://www.atlassian.com/software/views/open-source-license-request>

This is covered and I raised two tickets
#CA-297630 confirmed: Open Source license request by Xen Project
  #CA-297631 confirmed: Purchasing and licensing request (this covers how to 
migrate the account if approved)

> ACTION: Lars to follow up with Intel on owner and describe simple process to 
> nominate new people

Will follow up, once approved.

> We then had a brief discussion on whether to use labels or components. OpenXT 
> believe components are easier to manage
> 
> ACTION: Julien - Decide on whether to use labels or components for wiki page
> ACTION: Julien - Make a proposal on components on list for review (note: Lars 
> could help based on 
> https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#01590)

No progress yet

Lars___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen checkpatch infrastructure design

[Lars Kurth]

Hi Iurii,

I wanted to double check, as we had previously looked into clang-format, and it 
showed some gaps with what it can be used for in Xen coding styles.  Which is 
why we tried to get agreement - and got it - to upstream Xen related changes 
into clang–format


== Indentation: ok ==

[+] Indent using spaces = ok with clang-format

[+] Indent level = 4 spaces = ok with clang-format

[+] Code within blocks is indented by one extra indent level = ok with 
clang-format

[+] The enclosing braces of a block are indented the same as the code _outside_ 
the block = ok with clang-format


== Whitespace: partly ok ==

[?] spread out logical statements = so no complete support (to do that we have 
add spaces before and after parenthesis but they apply to all others 
parenthesis as well)

[?] spaces are placed between the keyword and the brackets surrounding the 
condition, between the 40 brackets and the condition itself = same as above

[+] around binary operators (except the structure access operators, '.' and 
'->') = ok

[-] There should be no trailing white space at the end of lines = no support


== Line length: partly ok ==

[+] Lines should be less than 80 characters in length = ok with clang-format

[-] Split at "sensible places" = no tool can do that

[-] Trailing portions indented = no support in clang-format

[-] User visible strings (e.g., printk() messages) should not be split = no 
support in clang-format (it splits everything)


== Brackets: partly ok ==

[+] Usually placed on a line of their own = ok

[-] Except for the do/while loop = no support

[-] Brackets should be omitted for blocks with a single statement = no support


== Comments: no support ==

[-] Only C style /* ... */ comments are to be used = no support

[-] Multi-word comments should begin with a capital letter. = no support

[-] Comments containing a single sentence may end with a full stop; = no support

[-] Comments containing several sentences must have a full stop after each 
sentence = no support

[-] Multi-line comment blocks should start and end with comment markers on 
separate lines and each line should begin with a leading '*'. = No support


Emacs local variables = no support


I was wondering how you deal with the gaps. I suppose these gaps could possibly 
be covered in clang-format-diff.py

Of course this info may be out-of-date


Regards

Lars

From: Iurii Artemenko 
mailto:iurii_arteme...@epam.com>>
Date: Monday, 24 July 2017 at 17:55
To: Juergen Gross mailto:jgr...@suse.com>>
Cc: Lars Kurth mailto:lars.ku...@citrix.com>>, Wei Liu 
mailto:wei.l...@citrix.com>>, Andrew Cooper 
mailto:andrew.coop...@citrix.com>>, Ian Jackson 
mailto:ian.jack...@citrix.com>>, 'Jan Beulich' 
mailto:jbeul...@suse.com>>, xen-devel 
mailto:xen-de...@lists.xenproject.org>>, cardoe 
mailto:car...@cardoe.com>>, Andrii Anisov 
mailto:andrii_ani...@epam.com>>, Lars Kurth 
mailto:lars.kurth@gmail.com>>
Subject: Re: [Xen-devel] Xen checkpatch infrastructure design


Hello Juergen,


I've started to work on checkpatch-like python script. I make it based on  
clang-format-diff.py and it works as pre-commit hook.

> The easiest way to accomplish that is a file in the repository's root
> directory containing the necessary information. It will be named
> "STYLES" and contains lines in the format:

I will follow this approach.


> Remains the question how to design the style checker itself. It could
> be:
>
> (a) a monolithic script (perl, python, whatever) being capable of
>handling all the different coding styles
> (b) a main script checking the patch header and calling a code style
>specific script for each source file modified by the patch

It seems like specific script for style checking is not needed. Because 
clang-format tool does style checking by itself.
All we need is just to provide appropriate coding style description file for 
each.
Clang-format is a bit specific tool, so we can not specify explicitly file with 
coding style description.
It just looks for a .clang-format file in one of a parent directories of a file 
being checked.
As we got at least three coding-styles we have to substitute needed file in 
sources top directory for each check.
It could be done by generating .clang-format file dynamically depending on 
style/path from the STYLES file.
Another way could be using appropriate symlink on existing .clang-format file 
which is located somewhere in  tools/clang-format/coding-style-file like:
tools/clang-format/xen-style
tools/clang-format/linux-style
tools/clang-format/xl-style


Regards
Iurii
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Notes of Design Session: Xen Certification in Automotive Industrial

[Lars Kurth]

There were also attendees from Advanced Driver Information Technology, Audi & 
Bosch whose e-mail addresses I don't have
This was quite a muddled discussion, which was not easy to capture. 
This is a first go at capturing it, but I missed large swathes of it
Feel free to add/correct as needed

Also, I am not quite sure who said what, because it was quite fast at points

The discussion was prompted by http://sched.co/Aj9l
The design session is at http://sched.co/AjE3

Regards
Lars

Introduction


Critical systems/automation control systems require isolation but certification
Security such as XSM/Flask is also a requirement

Xen is the best solution from a security PoV

Can use Xen as HV and isolation system
In such a scenario the Hypervisor and uKernels running guests would have to be 
certified for specific use-cases


Certification Problem
=

Requires development process to be executed in a specific way (e.g. ISO2 
26262/ASIL-B certification)
Note that different certifications will have slightly different requirements

The development model required by to certify is quite different from FOSS 
development models

Only way to do this today: 
Fork a specific version of Xen (or any FOSS project), follow the process with 
maybe pooling resources, ... 

This would require key stake-holders collaborating on a certified version of a 
Xen "distro" around some defined use-cases


Iurii's proposal

See http://sched.co/Aj9l and attached slides
The basic idea is to separate the system below the hypervisor (e.g. via 
TrustZone, TEE)

In other words, we would try to not certify the hypervisor
In such a scenario, the main focus of certification would be to prove that one 
part of the system does not influnce the other
E.g. this is for example done for QNX with software running outside the 
certification boundary. E.g an uncertified vault interacting with a certified 
one

E.g. shared memory between certified + uncertified
=> basically we will need to prove that ANY data in shared memory written by 
uncertified world, does not negatively affect certified world

Iurii also highlighted that the key point of certification is NOT that you can 
avoid mistakes, but that YOU CAN DETECT IT

Then we looked at some Xen examples, such as whether 0-copy in Xen would still 
be doable
To which the answer was yes

There was also a question on whether HW would need to be certified
To which the answer is that for example the Renesas R-CarH3 is compliant with 
ISO 26262 (ASIL-B)

Then there was a bit of discussion around use-cases and specifications
==
The first thing which was discussed was use-cases.
The basic premise is that you don't certify code, but use-cases (including 
everything from HW to SW which are touched by a use-case)
Changes in code (e.g new features or security fixes) may require a 
certification to lapse and require it to be redone
Bit it is possible to safety certify deltas (e.g. patches or security fixes)

If we don't know the exact use-cases to be certified, we can't be ready for the 
certified world

One approach which can help is to cover as many possible use-cases upfront, for 
example in PV protocols
By getting protocols as future-proof as possible and by supporting a maximum 
number of possible use-cases, we can avoid future re-certification 

Aka: try to not pollute future designs and keep things as stable as possible in 
the long run


Is there anything we can do as a community to make certification easier
===
This was a little bit muddled

* Try to get protocols as watertight and waterproof as possible
  - Need to understand the UC's: this could be reflected in version of 
protocols and ABIs - e.g. alpha version, beta version, ... 

* Compile time code removal: 
  - KCONFIG may help
  - However as a community we can only security support a very small number of 
combinations

* Is the Security process (https://www.xenproject.org/security-policy.html) is 
incompatible with the certification process?
  - No clear answer, but this sounded like a yes
  - There was a little bit of discussion around this or afterwards along the 
lines that certification only requires
to take action if an organisation was made aware of an issue (being on a 
security pre-disclosure list us therefore a
possible disadvantage for a vendor with a certified product)
  - But certification is only one issue: a high-profile vulnerability may case 
all sorts of other issues

* Around the Software - Hardware interface
  - Need to ensure that software can handle the HW bugs
  - SW can only be used on only specific certified HW (e.g. Renesas R-CarH3)
  - Starting to test against such HW is a good first step

* Feature Classification and the work we are/have been doing for becoming a CNA 
may help
  - See http://sched.co/AjHl & http://markmail.org/message/zxti

[Xen-devel] Notes Design Session: Testing & CI Process and Workflow Improvements, x86/ARM/Embedded Testing, etc. - Does what we do today work?

[Lars Kurth]

Hi all,

these are the notes from 

http://sched.co/AjHT

There are ACTIONS on the following people: Lars. Ian/Julien, Rich/Christopher 

Also see http://markmail.org/message/7e2mdpimvrmsppq5, which duplicated some of 
the discussion we had here 

Regards
Lars

ARM Server Testing
==

Unreliable testing on ARM64/32 
---

* Ongoing problems with ARM64 box (hardware issue) ... ticket 92394
* Other ARM64 box (firmware issue) ... ticket 91727
These are stuck waiting for SoftIron
We have escalated through a few channels and waiting for a response
Order of new SoftIron boxes on hold.

* ThunderX hardware ... ticket 91730
The ThunderX boxes are installed, but not yet commissioned
Threads going on regarding netbook support (or lack thereof) 

* Failing Arndale boards in ARM crate
We do not have a solution yet. 
For the general principle on how to deal with such cases, see the next section

We had a discussion on how to move forward:
a) Drop ARM32 testing (as Julien was not sure whether there is any demand for 
ARM32 based testing)
b) Try and find a solution, in line with general principle for non-server form 
factor Hardware

However, we agreed the following action

ACTION: Ian to make a proposal to drop/downgrade ARM32 testing (make it 
non-blocking) and remove specific tests if there is no demand


Embedded / Client Testing (ARM & x86)
=

General Principle
-

Fundamentally, the project OSSTEST and Test Lab maintainers do not have the 
expertise and bandwidth to be involved in evaluating development boards for 
stability and suitability. We also cannot have under-NDA hardware in the Test 
Lab. And we want to ensure that all non-server equipment is mounted on a 
sliding rack shelf, properly mounted and maintainable, such that we do not have 
outages due to loose equipment and or unplugged cables.

This means, we can accept Embedded / Client under the following conditions:

* We have enough space (Advisory Board approval needed)

* Step 1: The proposer does some basic testing to ensure that the board is 
stable and suitable

* Step 2: Investigate form factor issues
We don't need a perfect server chassis, but we would really like everything in 
the rack to be properly mounted and maintainable.

That means that shelves should be sliding shelves to which all the equipment, 
aka boards, power adaptor bricks, possibly fans if needed are fixed somehow 
(self-adhesive hooks, cable ties, etc.) and confidence that we won't have 
issues with height (spare height on the shelf when everything is mounted).

* Step 3: Bench-test the final set-up 

We would expect either working HW sent to us (after some initial discussion) or 
a list of parts with clear instructions on how to assemble these. We would run 
these past Credativ, who manage the Test Lab for us. This would need to be a 
discussion with which Credativ and the maintainers are happy, but we would not 
drive it.

Ongoing Work


* Adding Renesas Car-X gen HW to the Test Lab ... ticket 91996
There has been some discussion around requirements. The goal is to get Renesas 
Car boards in a server chassis.
There is AB approval.

ACTION: Lars to chase EPAM based on ticket as we seem to have provided all 
necessary info (done)

From EPAM: 
"Design is ready, we are about to start production as soon as we execute 
paperwork.
Artem will send an update soon. Currently stalled on legal review and sign-off 
of
paperwork with Renesas."

* Intel NUCs in Test Lab
Status: some initial discussion (stuck somewhere in step 2). Ball is in 
Christopher and Rich's court
No board approval yet. Project may carry costs for chassis/fans (TBD)

ACTION Christopher and Rich to follow up and drive 

Note this will be slow due to people being on vacation. There was also a 
discussion I was not part of on client specific test cases


x86 Testing & Heisenbugs


On the x86 side we had capacity issues (in hand)
But in the 4.9.0 release we had problems with x86 Heisenbugs 
(https://en.wikipedia.org/wiki/Heisenbug) for some tests, which have created 
problems

We had a bit of a discussion around this involving Ian and a few other people. 

In summary:
* The push gate assumes that tests fail when someone introduces a bug, which 
should motivate contributors to fix their bugs
* This breaks down with Heisenbugs: where the wrong people's contributions get 
stalled

There is no simple solution: but we discussed options such as

1: Manually special case test cases with Heisenbugs (make them non-blocking) 
=> underlying problems won't be fixed, but solved release bottlenecks

2: Add functionality to notice Heisenbugs and treat them differently (by 
re-running and doing log analysis) 
=> this is a more sophisticated version of 1
Note that OpenStack has some ElasticSearch based tools to help with 2, but 
tests are manually made non-blocking if they fail intermittently

3: Release Manager to

Re: [Xen-devel] Xen checkpatch infrastructure design

[Lars Kurth]

CC'ing Doug Goldstein as he has been reviewing some of Ishani's work (see below)
Both Andy Cooper and Doug Goldstein had done some groundwork earlier on this 
topic

> On 24 Jul 2017, at 09:50, Juergen Gross  wrote:
> 
> On the Xen Developer Summit 2017 in Budapest we agreed to add a
> script to the Xen repository capable to test patches for style
> correctness, similar to checkpatch.pl of the Linux kernel.
> 
> This is a first draft of the interface visible to users and
> developers.

...

> RFC: Design Considerations
> --
> Remains the question how to design the style checker itself. It could
> be:
> 
> (a) a monolithic script (perl, python, whatever) being capable of
>handling all the different coding styles
> (b) a main script checking the patch header and calling a code style
>specific script for each source file modified by the patch
> 
> I believe (b) would be easier to maintain and to develop (we could start
> with the main script and add style specific scripts later).

I don't have a view on this, but wanted to point the following docs which cover 
a little bit of groundwork on the subject, that can possibly be built upon
https://docs.google.com/document/d/10NJn-QvO1TvyJJJGE2PD6FtElYCT3neBAffIqeWHdiE/edit
 

http://markmail.org/message/tmdv2zzd4dvjin7v 


Regards
Lars___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] A document for Xen release management

[Lars Kurth]

Hi all,

I went over this with a few of the actions from 
https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#01645 


Lars/Wei/Julien 
A1 ACTION to write "standard e-mail templates for common stuff", rather than 
re-doing these every single time
Ian release manager file 

A2 ACTION : Clean up release technician checklist after we have the how to be
* Add hand-over of tasks for Release Manager responsibility to the "how to be 
release manager file"

A3 ACTION: Additional stuff to add to the templates/RM guide
A3.1: Add clear reminders in particular at the beginning of a release into 
e-mail templates: such as put dates X,Y, Z in your calendar add to checklist 
and templates
A3.2: Communicate better when tree is open again
A3.3: Release manager can say "not releasing now" because of too many bugs, 
"until someone fixes these". "no more patches until XYZ"

Looking through it, I am not sure whether A3.2 and 3.3 are covered.

Lars

> On 17 Jul 2017, at 16:09, Wei Liu  wrote:
> 
> It is agreed during the summit we should write down such document. Here
> is my attempt of doing so.
> 
> We should probably commit something like this into xen.git so that it
> gets updated regularly.
> 
> Comments are welcome.
> 
> -
> 
> % Xen Release Management
> % Wei Liu <>
> % Revision 1
> 
> # Motivation
> 
> Over the years we have had different people from different company signning
> up as the Release Manager of Xen. It would be rather wasteful if every new
> Release Manager has to go over everything and tripped over by the same
> mistakes again and again.
> 
> This file intends to document the process of managing a Xen release. It is
> mainly written for Release Manager, but other roles (contributors,
> maintainers and committers) are also encouraged to read this document, so
> that they can have an idea what to expect from the Release Manager.
> 
> # Xen release cycle
> 
> The Xen hypervisor project now releases twice a year, at the beginning of
> June and the beginning of December. The actual release date depends on a lot
> of factors. 
> 
> We can roughly divide one release into two periods. The development period
> and the freeze period. The former is 4 months long and the latter is about 2
> months long.
> 
> During development period, contributors submit patches to be reviewed and
> committed into xen.git.
> 
> During freeze period, the tree is closed for new features. Only bug fixes are
> accepted. This period can be shorter or longer than 2 months. If it ends up
> longer than 2 months, it eats into the next development period.
> 
> # The different roles in a Xen release
> 
> ## Release Manager
> 
> A trusted developer in the community that owns the release process. The major
> goal of the Release Manager is to make sure a Xen release has high quality
> and doens't slip too much.
> 
> The Release Manager will not see much workload during development period, but
> expects to see increasing workload during the freeze period until the final
> release. He or she is expected to keep track of issues, arrange RCs,
> negotiate with relevant stakeholders, balance the need from various parties
> and make difficult decisions when necessary.
> 
> The Release Manager essentially owns xen-unstable branch during the freeze
> period. The committers will act on the wishes of the Release Manager during
> that time.
> 
> ## Maintainers
> 
> A group of trusted developers who are responsible for certain components in
> xen.git. They are expected to respond to patches / questions with regard to
> their components in a timely manner, especially during the freeze period.
> 
> ## Committers
> 
> A group of trusted maintainers who can commit to xen.git. During the
> development window they normally push things as they see fit. During the
> freeze period they transfer xen-unstable branch ownership and act on the
> wishes of the Release Manager. That normally means they need to have an
> Release Ack in order to push a patch.
> 
> ## Contributors
> 
> Contributors are also expected to respond quickly to any issues regarding the
> code they submitted during development period. Failing that, the Release
> Manager might decide to revert the changes, declare feature unsupported or
> take any action he / she deems appropriate.
> 
> ## The Security Team
> 
> The Security Team operates independently. The visibility might be rather
> limited due to the sensitive nature of security work. The best action the
> Release Manager can take is to set aside some time for potential security
> issues to be fixed.
> 
> ## The Release Technician
> 
> The Release Technician is the person who tags various trees, prepares tarball
> etc. He or she acts on the wishes of the Release Manager. Please make sure
> the communication is as clear as it can be.
> 
> ## The Community Manager
> 
> The Community Manager owns xenproject.org infrastructure. He or she is
> respons

[Xen-devel] Notes from Design Session: Solving Community Problems: Patch Volume vs Review Bandwidth, Community Meetings ... and other problems

[Lars Kurth]

Hi all,
please find attached my notes. 
Lars

Session URL: http://sched.co/AjB3

ACTIONS on Lars, Andy and Juergen
ACTIONS on Stefano and Julien

Community Call
==
This was a discussion about whether we should do more community calls, 
in critical areas. The background was whether we should have an x86 call 
to mirror the ARM call.

Jan and Andy asked whether the ARM calls are useful

Julien: 
They are very useful. On average about 10 people attend.
On ARM we don't yet have a real plan of what's needed for the future.
We are hoping to use the call to establish a firmer plan.

Lars:
Was asking whether we always have an agenda at the beginning.

Julien:
Sometimes, but often the agenda is established/refined in the first
5 minutes at the beginning of the call. Typically Julien or Stefano
handle this at the beginning

Lars asks whether we need one for tools
Ian: there is currently not much a need for technical coordination

Lars: it feels that a call on x86 would be helpful
But we can only cover non-NDA information as with the other calls

Jan and Andy agree that they are happy to try this, but are concerned
that it may fizzle out. Also neither want to own agenda and note-taking
(notes and call info are posted on xen-devel@)

ACTION: Lars to work with Intel on setting this up
(note, I was asked by Susie Li to include John Ji and Chao Peng on this
thread and discuss with them at a separate call)

Timing wise, a call at from 9-10 UK time once a month should work. 

Example of ARM call minutes:
* http://markmail.org/message/myjllcngy3lqveji
* http://markmail.org/message/d4kuqxxhj6dfnf23
* There also ought to be a reminder of call details (someone to
  highlight an example)

Contributions vs. Review Bandwidth 
==

A potential bottleneck issue was raised in the area of ARM and x86
 
ARM
---
Lars asks what issue have been observed

Julien: 
Lots of new features and lots of design discussion

Stefano: 
Design discussions are creating trouble: sometimes we have complex 
proposals without a clear answer on the right way forward.

Complicated design 
=> 2/3 options 
=> not clear which way is the best forward 
=> ARM maintainers can provide advice, can say what is going to work

Right now ARM maintainers expect the contributor has to lead and 
drive it (e.g. an example where we were stuck was BIG.Little support)

A pattern we have seen is:
- Complex problem
- Not an obviously clear answer
- Gets stuck
- Design discussion fizzles out without an artefact in the codebase
  (in other words, there is an unfinished mail thread)

Lars:
Asks whether maybe the issue is one of sufficient confidence by the
contributor to move the discussion further or whether expectations 
were not communicated clearly (e.g. tell contributors to pick a
solution and move forward).

Stefano and Julien: 
Agree that this may indeed be the case

It is unusual to be in a technical leadership position when it comes
to driving designs and new solutions, but not from a process perspective.
Contributors need to be reminded of that.

It is also possible that embedded vendors may want to contribute,
but have only a small time window to do this.

Agreements:
* Create a couple of boilerplate mails or checklists to set 
  expectations better

ACTION: on ARM maintainers to trial

* Agreed to allow draft design into the git tree, as long as 
  interface status (Draft and unresolved issues) are clearly 
  documented. In that case, contributors can show progress
  and others - even if a design is not finished - can build on
  it. Feature docs already allow for that and so do Design
  Docs (although there is no example).

ACTION: on ARM maintainers to trial and pick a suitable
location in tree.

x86
---

Lars prompts Jan, Andy on some of the challenges

Jan, Andy:
A Typically series are large and fully formed 
  (e.g. 30 size series)
B Often we don't have enough context to understand design behind code
  This has improved through Hackathons, meetings under NDA, ...
C In the past, series have existed for 2 years earlier in private
  (e.g. SGX was developed against 4.6) and is posted against a newer version.
  At that point, some assumptions may have changed: e.g. on 5-level-paging
  we agreed at the summit that PV support is not needed (only HVM and PVH)
D There is not normally lack of driving and managing the submission of an issue

Roger: 
feels that when he is reviewing x86 stuff it does not actually take work off 
Jan or Andrew, as sometimes one of them will pick up and re-reviews. That 
sometimes puts him off.

Jan: that is a risk to take and shouldn't put you off. 

Wei: says that when his responsibility on a patch is not clear, he says 
"subject to the agreement of XXX". That sets expectations with other 
maintainers and contributors.

Then we went a little bit onto reasons behind bandwidth issues

Jan: large series are often hard to understand and consume. Also, sometimes
there is a lack of understanding that there is

[Xen-devel] Notes from Design Summit Hypervisor Fuzzing Session

[Lars Kurth]

Hi all,
please find attached my notes. A lot of it went over my head, so I may have 
gotten things wrong and some are missing
Feel free to modify, chip in, clarify, as needed
Lars

Session URL: http://sched.co/AjHN

OPTION 1: Userspace Approach


 Dom0  Domu
[AFL] [VM nested with Xen and XTF]
[Xen ]

Would need 
1. nested HVM support
2. VM forking

Not an option as too hard

OPTION 2:
=

 Dom0DomU
[AFL   ][VM XTF   ] 
[  ] <> [  [e]] e = executor
   /\  ||
   ||  \/
[Xen  ]

This approach would need

1. Tracing (instrument binary and write to shared memory for AFL)

Almost done, but not completely deterministic yet

2. Implemented a special hypercall that returns return code that can be 
converted into expected AFL output for branching info

Submitted

3. Communication channel between AFL and XTF

Almost done

4. Using XTF because it should be the fastest option and allows us to restrict 
the scope of what to fuzz

Key challenge: not making unnecessary indeterministic hyper calls in the 
background
Use of XTF constrains the degrees of freedom and focusses the fuzzing

5. Need some way to feed info back into AFL

I believe there was some discussion around this, which I did not get

Discussion
==

Dismissed Option 1. All agreed that Option 2 is best.

I missed quite a bit of this, because the discussion was quite fast at times

George: 
recommends to test one thing at the time to reduce the problem space
Such as iteration, feedback, ...  
Based on outcome iterate

There was a little bit of discussion around determinism:

Andy: blacklist shadop_??? with ??? = shutdown, suspend, watchdog, ... 
Possibly there are some more functions that need to be blacklisted
This should help with determinism

Andy: Going to have problems such as dealing with partial hypercall operations
Wei: Already included this - only 1 thread in XTF => deterministic
Andy: What happoens if HV gets interrupted
Juergen: put XTF into null scheduler pool to minimise risk of interrupts and 
increase determinism
Wei: That would exclude IRQs in such a scenario

There was a little bit of around feedback loop and protocol between AFL and XTF

Andy: easiest way to get a feedback loop starting. XTF to boot, wait on event 
channel (shadop call with - 0 timeout)
AFL does the hypercall with edge tracing, ...

Jurgen: starting measurement can be done be initiated AFL (Dom0), and disabled 
from XTF (DomU)
Wei: follow the same pattern as xl already does (I don't know the sample code 
though)

There was a bit of discussion on the impact pf QEMU

Wei: can't use QEMU to emulate a machine with vhdx (following on from a 
question by Ian)

Ian: this will be fast, not quite so reliable. But a good first step

And some other topics

Andy: there is also syzkaller, with fuzzing entity being some userspace calls
Wei: used as a reference material as Oracle did something similar
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

On 18/07/2017, 14:53, "Wei Liu"  wrote:

>On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
>> Wei,
>> I attached the list output from xsa-list-send starting from 206
>> If you look at 
>> 
>>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-4
>>81
>> .html, you may want to start using from 213+
>
>[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
>Can't locate Text/Diff.pm in @INC (you may need to install the
>Text::Diff module) (@INC contains: /etc/perl
>/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
>/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
>/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
>/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
>./match-xsa line 14.
>BEGIN failed--compilation aborted at ./match-xsa line 14.
>
>Would be useful to give a list of perl modules required.

These are at the top of the file: Getopt::Long qw(GetOptions), Cwd,
File::Slurp, Text::Diff, File::Spec;
Text::Diff may be obsolete - I used the diff function and then removed it
later because system ('diff ...') worked better for me. I can check and
remove the "use"

Lars 



>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] A document for Xen release management

[Lars Kurth]

On 17/07/2017, 16:09, "Wei Liu"  wrote:

>It is agreed during the summit we should write down such document. Here
>is my attempt of doing so.
>
>We should probably commit something like this into xen.git so that it
>gets updated regularly.
>
>Comments are welcome.

Just to x-ref, see the notes from the meeting at
https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#0
1645
I think I ought to go through the document when the next version is in
place and check, whether something has been missed

Lars
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling

[Lars Kurth]

Hi all,
these are my notes from http://sched.co/AjES
These are a little crude and I may have missed some things or misrepresented

ACTIONS for
* Wei Liu
* Julien Grall
* Lars Kurth
* Ian Jackson

Feel free to correct
Lars

Releases Lessons learned


Process issues in 4.9.0 (checklists, guides, templates)
---

Ian:
* Lots of fiddly bits need to be executed (right now we have a release 
technician checklist)
* We don't have a comprehensive checklist that is useful for Release Managers
* Started the final steps for 4.9.0 on Tue of the release week
* By this tine we were committed to releasing on Wed (no point of return due to 
PR)
* Some items were release managers job, but discovered by Release Technician 
(Ian)
* Release technician checklist is in xen.git

ACTION (Wei, Julien, Lars): Have a how to be release manager file (needs to 
contain some of the stuff in Release technician checklist)
 * RM file: clear set of criteria on PR. Go/NoGo part
 * See 
https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#01500 
for a starting point

ACTION (Lars/Wei/Julien) to write "standard e-mail templates for common stuff" 
rather than re-doing these every single time
 
ACTION (Ian): Clean up release technician checklist after we have the how to be 
release manager file

ACTION (Wei/Julien): Additional stuff to add to the templates/RM guide
* Add hand-over of tasks for Release Manager responsibility to the "how to be 
release manager" file
* Add clear reminders in particular at the beginning of a release into e-mail 
templates: such as "put dates X,Y, Z in your calendar"
* Communicate better when tree is open again (add to checklist and templates)

ARM issues in 4.9.0
---

Julien: Unreliable testing on ARM32/64 
- ongoing problems with ARM64 box (hardware issue), other ARM64 box (firmware 
issue), ARM32 Arndale hardware problem (replace)

On ARM everything is in progress and not much more can be done => no concrete 
ACTION as those already executed

x86 Heisenbugs in 4.9.0
---
- x86 migration Heisenbugs have caused us issues this release cycle: not 
completely understood the issues and hard to debug

Ian: These are most likely software problems, most likely in Xen.
ISSUE: nobody wants to debug Windows Heisenbugs
ISSUE: have nobody to do formal triage - doing back-pressure on developers

ACTION: Ian, Julien
- Manually add ignore list for specific Heisenbugs until we found another 
solution
- Requires a proposal to the list - Julien to propose


ACTION: Julien to propose 
- OSSTEST to CC people blocked/proposed on test failures instead just the list
- Release manager can say "not releasing now" because of too many bugs
  "until someone fixes these". "no more patches until XYZ" 


Improving the Process
=

JIRA

Open source projects do NOT need licenses for JIRA: these was raised by OpenXT 
folks, who use JIRA

ACTION: Lars to follow up with Atlassian - 
https://www.atlassian.com/software/views/open-source-license-request
ACTION: Lars to follow up with Intel on owner and describe simple process to 
nominate new people

We then had a brief discussion on whether to use labels or components. OpenXT 
believe components are easier to manage

ACTION: Julien - Decide on whether to use labels or components for wiki page
ACTION: Julien - Make a proposal on components on list for review (note: Lars 
could help based on 
https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#01590)

XSA Testing 
---
ACTION: include Lars'es tool into checklist

Arguments on the mailing list on XSAs
-
Delayed RC by two days: security patches produced by the security team - we 
don't always consult the maintainer

ACTION: committers@ + maintainer of relevant code of Advisory should be on 
pre-disclosure list 
* Explicitly say in public that security team should be informed pre-disclosure 
...
  With appropriate template

Resolved in 
https://lists.xenproject.org/archives/html/xen-devel/2017-07/threads.html#01590

Release Manager for 4.11

Julien stepping down after 4.10 - we should find another volunteer

ACTION: Lars to sound out SUSE (Juergen Gross), Ross Phillipson (Oracle)
Note: Juergen would be happy to do this for one release







___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] A document for Xen release management

[Lars Kurth]

I added Jan because I think we should probably add a section for point
releases, which I assume would be a subset of the steps in this document
Lars

I will post the notes of the design session also. I am doing the ones
first that I moderated and where I took notes in handwriting and/or photos
of whiteboard discussions first though, such that I don't miss too much
stuff

Lars
 
On 17/07/2017, 16:09, "Wei Liu"  wrote:

>It is agreed during the summit we should write down such document. Here
>is my attempt of doing so.
>
>We should probably commit something like this into xen.git so that it
>gets updated regularly.
>
>Comments are welcome.
>
>-
>
>% Xen Release Management
>% Wei Liu <>
>% Revision 1
>
># Motivation
>
>Over the years we have had different people from different company
>signning
>up as the Release Manager of Xen. It would be rather wasteful if every new
>Release Manager has to go over everything and tripped over by the same
>mistakes again and again.
>
>This file intends to document the process of managing a Xen release. It is
>mainly written for Release Manager, but other roles (contributors,
>maintainers and committers) are also encouraged to read this document, so
>that they can have an idea what to expect from the Release Manager.
>
># Xen release cycle
>
>The Xen hypervisor project now releases twice a year, at the beginning of
>June and the beginning of December. The actual release date depends on a
>lot
>of factors. 
>
>We can roughly divide one release into two periods. The development period
>and the freeze period. The former is 4 months long and the latter is
>about 2
>months long.
>
>During development period, contributors submit patches to be reviewed and
>committed into xen.git.
>
>During freeze period, the tree is closed for new features. Only bug fixes
>are
>accepted. This period can be shorter or longer than 2 months. If it ends
>up
>longer than 2 months, it eats into the next development period.
>
># The different roles in a Xen release
>
>## Release Manager
>
>A trusted developer in the community that owns the release process. The
>major
>goal of the Release Manager is to make sure a Xen release has high quality
>and doens't slip too much.
>
>The Release Manager will not see much workload during development period,
>but
>expects to see increasing workload during the freeze period until the
>final
>release. He or she is expected to keep track of issues, arrange RCs,
>negotiate with relevant stakeholders, balance the need from various
>parties
>and make difficult decisions when necessary.
>
>The Release Manager essentially owns xen-unstable branch during the freeze
>period. The committers will act on the wishes of the Release Manager
>during
>that time.
>
>## Maintainers
>
>A group of trusted developers who are responsible for certain components
>in
>xen.git. They are expected to respond to patches / questions with regard
>to
>their components in a timely manner, especially during the freeze period.
>
>## Committers
>
>A group of trusted maintainers who can commit to xen.git. During the
>development window they normally push things as they see fit. During the
>freeze period they transfer xen-unstable branch ownership and act on the
>wishes of the Release Manager. That normally means they need to have an
>Release Ack in order to push a patch.
>
>## Contributors
>
>Contributors are also expected to respond quickly to any issues regarding
>the
>code they submitted during development period. Failing that, the Release
>Manager might decide to revert the changes, declare feature unsupported or
>take any action he / she deems appropriate.
>
>## The Security Team
>
>The Security Team operates independently. The visibility might be rather
>limited due to the sensitive nature of security work. The best action the
>Release Manager can take is to set aside some time for potential security
>issues to be fixed.
>
>## The Release Technician
>
>The Release Technician is the person who tags various trees, prepares
>tarball
>etc. He or she acts on the wishes of the Release Manager. Please make sure
>the communication is as clear as it can be.
>
>## The Community Manager
>
>The Community Manager owns xenproject.org infrastructure. He or she is
>responsible for updating various web archives, updating wiki pages and
>coordinating with the PR Personnel.
>
>## The PR Personnel
>
>They are responsible for corrdinating with external reporters to publish
>Xen
>release announcement. The Release Manager should be absolutely sure the
>release is going out on a particular date before giving them the signal to
>proceed, because there is a point of no return once they schedule a date
>with
>external reporters.
>
># What happens during a release
>
>## Development period
>
>Send out monthly update email. The email contains the timeline of the
>release, the major work items and any other information the Release
>Manager
>sees fit. Please consider adding a recurring event to your calendar.
>
>Occasionally che

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Wei,
I attached the list output from xsa-list-send starting from 206
If you look at 
https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-481
.html, you may want to start using from 213+
Lars

On 17/07/2017, 12:40, "Wei Liu"  wrote:

>On Mon, Jul 17, 2017 at 09:17:23AM +0100, Lars Kurth wrote:
>> Folks,
>> 
>> I didn't run the XSA script. Maybe someone can have a go and test out
>>the
>> instructions in 
>> 
>>https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts
>>.g
>> it;a=summary
>> The scripts does requireS XSA.GIT to be checked out, but can be changed
>> easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR
>> 
>> In fact --xsadir http://xenbits.xenproject.org/xsa may just work
>> 
>> Lars
>> 
>
>I tried to follow the instructions in README for match-xsa. I believe
>the xsa-list-send script in step 3 depends on xsa.git, which I don't
>have access to.

206 
xsa206-unstable/0001-xenstored-apply-a-write-transaction-rate-limit.patch   
xenstored: apply a write transaction rate limit
206 
xsa206-unstable/0002-xenstored-Log-when-the-write-transaction-rate-limit-.patch 
xenstored: Log when the write transaction rate limit bites
206 
xsa206-unstable/0003-oxenstored-comments-explaining-some-variables.patch
oxenstored: comments explaining some variables
206 
xsa206-unstable/0004-oxenstored-handling-of-domain-conflict-credit.patch
oxenstored: handling of domain conflict-credit
206 
xsa206-unstable/0005-oxenstored-ignore-domains-with-no-conflict-credit.patch
oxenstored: ignore domains with no conflict-credit
206 
xsa206-unstable/0006-oxenstored-add-transaction-info-relevant-to-history-.patch 
oxenstored: add transaction info relevant to history-tracking
206 xsa206-unstable/0007-oxenstored-support-commit-history-tracking.patch   
oxenstored: support commit history tracking
206 
xsa206-unstable/0008-oxenstored-only-record-operations-with-side-effects-.patch 
oxenstored: only record operations with side-effects in history
206 
xsa206-unstable/0009-oxenstored-discard-old-commit-history-on-txn-end.patch 
oxenstored: discard old commit-history on txn end
206 xsa206-unstable/0010-oxenstored-track-commit-history.patch  
oxenstored: track commit history
206 
xsa206-unstable/0011-oxenstored-blame-the-connection-that-caused-a-transa.patch 
oxenstored: blame the connection that caused a transaction conflict
206 xsa206-unstable/0012-oxenstored-allow-self-conflicts.patch  
oxenstored: allow self-conflicts
206 
xsa206-unstable/0013-oxenstored-do-not-commit-read-only-transactions.patch  
oxenstored: do not commit read-only transactions
206 
xsa206-unstable/0014-oxenstored-don-t-wake-to-issue-no-conflict-credit.patch
oxenstored: don't wake to issue no conflict-credit
206 
xsa206-unstable/0015-oxenstored-transaction-conflicts-improve-logging.patch 
oxenstored transaction conflicts: improve logging
206 
xsa206-unstable/0016-oxenstored-trim-history-in-the-frequent_ops-function.patch 
oxenstored: trim history in the frequent_ops function
206 xsa206-4.4/0001-xenstored-apply-a-write-transaction-rate-limit.patch
xenstored: apply a write transaction rate limit
206 
xsa206-4.4/0002-xenstored-Log-when-the-write-transaction-rate-limit-.patch  
xenstored: Log when the write transaction rate limit bites
206 xsa206-4.4/0003-oxenstored-exempt-dom0-from-domU-node-quotas.patch  
oxenstored: exempt dom0 from domU node quotas
206 
xsa206-4.4/0004-oxenstored-perform-a-3-way-merge-of-the-quota-after-.patch  
oxenstored: perform a 3-way merge of the quota after a transaction
206 
xsa206-4.4/0005-oxenstored-catch-the-error-when-a-connection-is-alre.patch  
oxenstored: catch the error when a connection is already deleted
206 
xsa206-4.4/0006-oxenstored-use-hash-table-to-store-socket-connection.patch  
oxenstored: use hash table to store socket connections
206 
xsa206-4.4/0007-oxenstored-enable-domain-connection-indexing-based-o.patch  
oxenstored: enable domain connection indexing based on eventchn port
206 
xsa206-4.4/0008-oxenstored-only-process-domain-connections-that-noti.patch  
oxenstored: only process domain connections that notify us by events
206 
xsa206-4.4/0009-oxenstored-add-a-safe-net-mechanism-for-existing-ill.patch  
oxenstored: add a safe net mechanism for existing ill-behaved clients
206 xsa206-4.4/0010-oxenstored-refactor-putting-response-on-wire.patch  
oxenstored: refactor putting response on wire
206 xsa206-4.4/0011-oxenstored-remove-some-unused-p

Re: [Xen-devel] Notes for Design Session: Loose ends for becoming a CNA (CVE Numbering Authorities) and other Security Team Operational Questions

[Lars Kurth]

A small correction

> On 18 Jul 2017, at 11:37, Lars Kurth  wrote:
> 
> ...
> b) The alternative would be for the security team to fix issues in private 
> and bundle as we already do and pre-disclose 2 weeks before a fixed monthly 
> embargo date. The effect would be that
> b.1) information leakage is reduced as confined to the security team only
this should be "risk of information leakage is reduced as handling of 
confidential information is confined to the security team only"

Regards
Lars



signature.asc
Description: Message signed with OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Notes for Design Session: Loose ends for becoming a CNA (CVE Numbering Authorities) and other Security Team Operational Questions

[Lars Kurth]

Hi all,

on the following topic in session http://sched.co/AjHl we discussed the 
following issues
Here are my notes

ACTIONS for
* Julien Grall/Stefano Stabellini
* Andrew Cooper
* Ian Jackson
* Lars Kurth
* Security team members

Regards
Lars

= Consolidate Security Coverage Documents for CNA =

Consolidate security coverage documents where possible (we have a proposal). 
Specifically

== Review the proposal ==
See 
https://docs.google.com/document/d/17LiK-C3oBFZNpxeihXxkM2Bagn7A2L3Dv1u1ZGZe_TQ/edit

The ghist is to
* Replace https://wiki.xenproject.org/wiki/Xen_Project_Release_Features, 
xen.git:docs/misc/qemu-xen-security and some other sources with a SUPPORT file 
in some form of mark-up in xen.git trees (starting with master and back-porting 
to security supported trees)
* Link to all relevant pages from https://xenproject.org/security-policy.html 

This has been agreed in principle

== Review the security scope of Features == 

See 
https://docs.google.com/spreadsheets/d/1wLb37mbxN715rlYD8eKV8htgatDI41noOmQ1H428QX4/edit#gid=0
Note that this google doc has taken information from 
https://wiki.xenproject.org/wiki/Xen_Project_Release_Features, 
xen.git:docs/misc/qemu-xen-security and some other sources as prep work for 
consolidation. Some areas were unclear and required discussion.

We discussed and resolved some of the categorisation in preparation for a full 
proposal on xen-devel. 

IMPORTANT: the attached google docs are *not a proposal*. They are preparation 
for a *formal proposal* on xen-devel@ in future. No final decisions were being 
made at the summit. The discussion at the summit was intended to resolve thorny 
issues *before* making a formal proposal on xen-devel@, making use of key 
people being in one room. The idea is to save time, for what could otherwise be 
a lengthy debate on xen-devel@ by creating maximum alignment upfront. 

We made good progress, BUT there were a few areas to were to tough to resolve
* The question of how to handle limits => we agreed that we would provide 
security support for maximum theoretical limits (we already do), but wanted to 
better set expectations for what we actually test in general, because we expect 
that Xen consumers will use the proposed SUPPORT file outside security support 
* We agreed a new "Obsolete" status, which needs to be encoded (the agreement 
would be to treat it like Deprecated without Security support)
* The ARM area needs a wholesale review, as it does not make much sense

ACTIONS:
* Julien Grall/Stefano Stabellini: Get the ARM/* items in the list into a 
sensible shape 
* Andrew Cooper: Make a concrete proposal on limits (either now, or once the 
* Ian Jackson: Develop a script that takes the google doc and translates it 
into an appropriate text file to be included in each Xen tree (starting from 
master) and post on xen-devel@ for community review

Once we have agreement, we basically just need to document the outcome, publish 
it and get the process started.

= Other Operational Issues =

== Automated checking of XSA patch levels against releases ==
See 
https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts.git;a=summary

The problem with the script is that it currently relies on xsa.git that only is 
accessible to security team members

Lars walked through the script and explained how it worked: this was well 
received.
We agreed to include running of the script into the Release Manager checklist

This script currently has a number of issues
a) It relies on a file that can only be generated by the security team which is 
run on xsa.git (a private repository)

The file format is:
xsa-numberpatch-pathpatch commit message

Example:
210 xsa210.patcharm/p2m: remove the page from p2m->pages list 
before freeing it
211 xsa211-qemut.patch  cirrus/vnc: zap drop bitblit support 
from console code.
211 xsa211-qemut-4.5.patch  cirrus/vnc: zap drop bitblit support 
from console code.
211 xsa211-qemuu.patch  cirrus/vnc: zap bitblit support from 
console code.
...

Attached a complete example at the end of the mail. It should be possible to 
generate the file from
* generate the file from http://xenbits.xenproject.org/xsa/
* publish such a file on http://xenbits.xenproject.org/xsa/
* update the tool such that it reads http://xenbits.xenproject.org/xsa/ 

Note that the tool or mechanism should be changed to generate a file based on a 
start-date, not an XSA number as currently done

b) It relies on xsa.git to be checked out
Note that "--xsadir https://xenbits.xenproject.org/xsa"; should work assuming 
that read_file() from File::Read can open files with an http address
I have not tested this

ACTION: Lars to work with Julien/Wei on inclusion of tool into Release Manager 
checklist
ACTION: Lars to gather community feedback (as the tool is potentially useful 
for patch management for Xen users)

Ian noted that the tool should be able to ru

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

> I tried to follow the instructions in README for match-xsa. I believe
> the xsa-list-send script in step 3 depends on xsa.git, which I don't
> have access to.
That is unfortunately correct: we ought to fix this.
Lars


On 17/07/2017, 12:40, "Wei Liu"  wrote:

>On Mon, Jul 17, 2017 at 09:17:23AM +0100, Lars Kurth wrote:
>> Folks,
>> 
>> I didn't run the XSA script. Maybe someone can have a go and test out
>>the
>> instructions in 
>> 
>>https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts
>>.g
>> it;a=summary
>> The scripts does requireS XSA.GIT to be checked out, but can be changed
>> easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR
>> 
>> In fact --xsadir http://xenbits.xenproject.org/xsa may just work
>> 
>> Lars
>> 
>
>I tried to follow the instructions in README for match-xsa. I believe
>the xsa-list-send script in step 3 depends on xsa.git, which I don't
>have access to.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Folks,

I didn't run the XSA script. Maybe someone can have a go and test out the
instructions in 
https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts.g
it;a=summary
The scripts does requireS XSA.GIT to be checked out, but can be changed
easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR

In fact --xsadir http://xenbits.xenproject.org/xsa may just work

Lars

On 17/07/2017, 10:01, "Wei Liu"  wrote:

>On Thu, Jul 06, 2017 at 01:17:02AM -0600, Jan Beulich wrote:
>> All,
>> 
>> with the goal of releasing in the first half of August (once I'm back
>> from vacation and had time to sync back up, and the tree has got
>> the necessary push), please point out backport candidates you
>> find missing from the respective staging branches, but which you
>> consider relevant. Note that commit 2ff229643b ("livepatch: Don't
>> crash on encountering STN_UNDEF relocations") is already on my
>> list; I'm not fully decided on bd53b85156 ("livepatch: Use zeroed
>> memory allocations for arrays") yet, but I tend towards taking it as
>> long as it applies reasonably cleanly (which I expect it will do).
>> 
>> Thanks, Jan
>> 
>
>xen-RELEASE-4.8.2 tagged in mini-os.git.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Is anyone interested in hosting the following to Design Sessions at our Summit: "Versioning ABC" and "Improvements to Continuous Integration Workflow"?

[Lars Kurth]

These will now be removed: I merged some of 
https://xendeveloperanddesignsummit2017.sched.com/event/AjGa/improvements-to-continuous-integration-workflow-doug-goldstein-star-lab
 
<https://xendeveloperanddesignsummit2017.sched.com/event/AjGa/improvements-to-continuous-integration-workflow-doug-goldstein-star-lab>
 into 
https://xendeveloperanddesignsummit2017.sched.com/event/AjHP/open-session-testing-ci-process-and-workflow-improvements-x86armembedded-testing-etc-does-what-we-do-today-work
 
<https://xendeveloperanddesignsummit2017.sched.com/event/AjHP/open-session-testing-ci-process-and-workflow-improvements-x86armembedded-testing-etc-does-what-we-do-today-work>
Lars

> On 30 Jun 2017, at 11:04, Lars Kurth  wrote:
> 
> Hi all,
> 
> Doug will unfortunately not be able to attend the summit next week. He 
> proposed two sessions, which although he will try and participate in 
> remotely, does not feel comfortable to chair. I am organizing a portable 
> speaker/mic and see whether we can pull this off, 
> 
> Is anyone willing to take these sessions over?
> 
> See 
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjGa/improvements-to-continuous-integration-workflow-doug-goldstein-star-lab
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjHT/versioning-abc-doug-goldstein-star-lab
> 
> If yes, please let me know and reach out to Doug: I will then change the 
> session owner. If no, I will cancel them. Please let me know before Friday.
> 
> Best Regards
> Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] New Design Sessions for the Developer Summit

[Lars Kurth]

Hi all,
two more sessions were added
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjEW/design-session-improved-xendomu-ring-mapping-api
 
<https://xendeveloperanddesignsummit2017.sched.com/event/AjEW/design-session-improved-xendomu-ring-mapping-api>
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjHK/design-session-improvements-to-in-hypervisor-emulation
 
<https://xendeveloperanddesignsummit2017.sched.com/event/AjHK/design-session-improvements-to-in-hypervisor-emulation>
Lars

> On 29 Jun 2017, at 18:43, Lars Kurth  wrote:
> 
> Hi all, (proposers CC'ed)
> 
> I added several new design sessions today: people on the CC list may want to 
> propose different time-slots
> 
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjEI/design-session-the-missing-toolstack-side-of-pvh-domu
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjB3/do-we-need-more-community-meetings
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjES/making-releases-lessons-learned-improving-our-release-process-and-tooling
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjHP/open-session-testing-process-testing-improvements-x86armembedded-testing-etc-does-what-we-do-today-work
> * 
> https://xendeveloperanddesignsummit2017.sched.com/event/AjHl/design-session-loose-ends-for-becoming-a-cna-cve-numbering-authorities
> 
> You can still submit sessions via 
> http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session
>  until next Friday. After that, it is possible to submit sessions on the day, 
> but it would be easier if we got as many as possible raised before.
> 
> Regards
> Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] WhatsApp group for Xen Project Developer Summit social stuff

[Lars Kurth]

To make this easier for me, you can also use 
https://chat.whatsapp.com/CXjUXzCFdCbF4P0SzNHE0N 
<https://chat.whatsapp.com/CXjUXzCFdCbF4P0SzNHE0N> to join the group (either 
from a phone or via a WhatsApp Desktop app)
Lars

> On 30 Jun 2017, at 11:45, Lars Kurth  wrote:
> 
> Folks,
> some us are arriving a little earlier in Budapest and we may want to 
> coordinate for dinner on the 10th. If you arrive early, please drop me a 
> private mail (NOT PUBLIC UNLESS YOU WANT YOUR NUMBER TO BE PUBLIC) with your 
> name and phone number and I will set up a WhatsApp group and we can see 
> whether we can pull something together. 
> Regards
> Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] WhatsApp group for Xen Project Developer Summit social stuff

[Lars Kurth]

Folks,
some us are arriving a little earlier in Budapest and we may want to coordinate 
for dinner on the 10th. If you arrive early, please drop me a private mail (NOT 
PUBLIC UNLESS YOU WANT YOUR NUMBER TO BE PUBLIC) with your name and phone 
number and I will set up a WhatsApp group and we can see whether we can pull 
something together. 
Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Is anyone interested in hosting the following to Design Sessions at our Summit: "Versioning ABC" and "Improvements to Continuous Integration Workflow"?

[Lars Kurth]

Hi all,

Doug will unfortunately not be able to attend the summit next week. He proposed 
two sessions, which although he will try and participate in remotely, does not 
feel comfortable to chair. I am organizing a portable speaker/mic and see 
whether we can pull this off, 

Is anyone willing to take these sessions over?

See 
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjGa/improvements-to-continuous-integration-workflow-doug-goldstein-star-lab
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjHT/versioning-abc-doug-goldstein-star-lab

If yes, please let me know and reach out to Doug: I will then change the 
session owner. If no, I will cancel them. Please let me know before Friday.

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] New Design Sessions for the Developer Summit

[Lars Kurth]

Hi all, (proposers CC'ed)

I added several new design sessions today: people on the CC list may want to 
propose different time-slots

* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjEI/design-session-the-missing-toolstack-side-of-pvh-domu
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjB3/do-we-need-more-community-meetings
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjES/making-releases-lessons-learned-improving-our-release-process-and-tooling
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjHP/open-session-testing-process-testing-improvements-x86armembedded-testing-etc-does-what-we-do-today-work
* 
https://xendeveloperanddesignsummit2017.sched.com/event/AjHl/design-session-loose-ends-for-becoming-a-cna-cve-numbering-authorities

You can still submit sessions via 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session
 until next Friday. After that, it is possible to submit sessions on the day, 
but it would be easier if we got as many as possible raised before.

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

On 27/06/2017, 17:16, "Jan Beulich"  wrote:

>>>> Lars Kurth  06/27/17 10:54 AM >>>
>>- Hardware > x86/AVX512/Neural Network Instructions AVX512_4VNNIW* : ???
>>- Hardware > x86/AVX512/Multiply Accumulation Single precision
>>AVX512_4FMAPS* : ???
>
>For one I think mentioning enabling of individual instructions (rather
>than larger
>sets) isn't really worthwhile. Furthermore, as will all AVX512
>sub-features, what
>we have for now is really only partial enablement, as the insn emulator
>doesn't
>support any of it yet.

Fine with me. I wanted to ask the question
Lars
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

Removing security@

On 27/06/2017, 14:28, "Oleksandr Andrushchenko"  wrote:

>
>> Did I miss anything?
>>
>Please consider adding new topics:
>1. Shared coprocessor framework
>2. Native applications, e.g. EL0 app, stubdoms

I am only adding committed stuff: these are 4.10 items
Lars
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

On 27/06/2017, 12:02, "George Dunlap"  wrote:

>On Tue, Jun 27, 2017 at 9:53 AM, Lars Kurth  wrote:
>> Hi all, (I think I CCed all stake-holders)
>>
>> to finish off the release documentation for 4.9, I need to add an extra
>> column to https://wiki.xenproject.org/wiki/Xen_Project_Release_Features
>>–
>
>Also -- the table is getting awfully large, and half of it is out of
>security support now.  Would it make sense to start trimming some
>releases off the left-hand side?
>
>I think 8 outstanding releases should really be an upper limit.
>
Agreed, I was already thinking about this. I am going to make an archived
copy and delete everything before 4.4
Lars
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

On 27/06/2017, 11:57, "Juergen Groß"  wrote:

>On 06/27/2017 12:33 PM, George Dunlap wrote:
>> On Tue, Jun 27, 2017 at 9:53 AM, Lars Kurth 
>>wrote:
>>> Hi all, (I think I CCed all stake-holders)
>>>
>>> to finish off the release documentation for 4.9, I need to add an extra
>>> column to 
>>>https://wiki.xenproject.org/wiki/Xen_Project_Release_Features –
>>> because I was travelling, this dropped of my radar. There several
>>>decisions
>>> to be made:
>>> A) Decide which "features" to add
>>> B) Decide on the status of the feature
>>> C) Deal with status changes of any past features
>>>
>>> The first goal would be to decide on A and any new "features" under C.
>>>For
>>> B, I am OK to add "???" for now and point to this thread, until we have
>>> concluded the discussion
>>>
>>> Note that I tracked some of this as preparation for getting CNA status.
>>> Items marked with * are not yet in the discussion document that I
>>>created
>>> for the security team and which we intend to discuss at the summit.
>>>
>>> For all of these, the naming convention is "Section in document" >
>>>"Feature"
>>> : "Support status". The definition of support status is added at the
>>>end of
>>> the mail: note that the text has not yet been fully agreed, but seems
>>>to
>>> reflect fairly well how we handled stuff in the past.
>>>
>>> == On A / B: I think we should add ==
>>> - Resource Management > Null Scheduler : tech preview or experimental
>> 
>> I think that the interface is probably in a final form, the code is
>> complete, and there are no known bugs or issues; so I'd say tech
>> preview.
>> 
>> Dario, any opinions?
>> 
>>> - Virtual Firmware or PV Bootloader Support (not sure which) >
>>>x86/Boot Xen
>>> on EFI platforms using GRUB2*  : ???
>> 
>> If I'm interpreting your phrase correctly, this probably goes under
>> "interoperability / hardware support"
>> 
>>> - Hardware > ARM/Alternative Runtime Patching (ARM32 and ARM64): ???
>>>[note
>>> that this should probably have been added for 4.8, but I didn't add it]
>>> - Hardware > ARM/System Error Protection* : ???
>>> - Hardware > ARM/Wait for Virtual Interrupt* : ???
>>> - Hardware > x86/AVX512/Neural Network Instructions AVX512_4VNNIW* :
>>>???
>>> - Hardware > x86/AVX512/Multiply Accumulation Single precision
>>> AVX512_4FMAPS* : ???
>>> - Device Models > DMOP (Device Model Operation Hypercall)  : ???
>> 
>> This is already used by default in the version of qemu released in
>> 4.9, right?  I think I'd call that "supported".
>> 
>>> == On C ==
>>> - Security > Live Patching - see
>>> 
>>>https://lists.xenproject.org/archives/html/xen-devel/2017-06/threads.htm
>>>l#03039
>>> - Security > Alternative 2pm : Supported – I think we should split
>>>this out
>>> – it is currently implicitly covered under "Virtual Machine
>>>Introspection"
>>>
>>> If we introduce a new heading "PV Protocols and Drivers" we should
>>>probably
>>> list all the common ones as supported in this heading, e.g.
>>> - PV Protocols and Drivers* > default (net, block, console, keyboard,
>>>mouse)
>>> : supported
>> 
>> I don't think there are keyboard and mouse PV protocols.  On the other
>> hand, I just realized that I have no idea how one uses PV guests with
>> a framebuffer but no mouse.
>
>xen/include/public/io/kbdif.h does exist it it is being used.
>
>>> There are also USB and framebuffer, which I am not sure whether they
>>>should
>>> be supported and if not, what their status is
>>> - PV Protocols and Drivers* > USB : ???
>> 
>> This one has been around a long time, then languished for a bit, but
>> it seems has been picked up by SuSE again.
>> 
>> I don't *think* we're doing USB testing in osstest yet (either HVM or
>>PVUSB).
>> 
>> Juergen, are you guys actively doing internal testing of PVUSB?  If so
>> we could probably label that as 'supported' anyway.
>
>Yes, we do.

Alright: as we seem to have consensus, I will add

PV Protocols and Drivers* > default (net, block, console, keyboard, mouse,
USB, framebuffer) : supported


And I am going to backdate this up to 4.5 (and put ? before), as these
seem to have been around forever

Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.9] livepatch: Declare live patching as a supported feature

[Lars Kurth]

On 27/06/2017, 11:49, "Ian Jackson"  wrote:


>> * There was a proposal to declare live patching supported for older
>> releases (aka "back port" docs/features/livepatching.pandoc), but Royger
>> pointed out that the toolstack in question needs to support buildid. If
>> so, we should include back-porting requests and d
>
>("and do it then" or something?)  Yes.

Correct: "and do it then" ... Must have deleted the text when hovering
over it. 
Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

On 27/06/2017, 11:33, "George Dunlap"  wrote:

>On Tue, Jun 27, 2017 at 9:53 AM, Lars Kurth  wrote:
>>
>> - Virtual Firmware or PV Bootloader Support (not sure which) >
>>x86/Boot Xen
>> on EFI platforms using GRUB2*  : ???
>
>If I'm interpreting your phrase correctly, this probably goes under
>"interoperability / hardware support"

I wasn't quite sure whether it should go into
A) 
https://wiki.xenproject.org/wiki/Xen_Project_Release_Features#Interoperabil
ity_.2F_Hardware_Support (Interoperability / Hardware Support) or
B) 
https://wiki.xenproject.org/wiki/Xen_Project_Release_Features#Device_Models
_and_Virtual_Firmware (Device Models and Virtual Firmware)

If B), then there is the option to put it under B.1) "Device Models and
Virtual Firmware for HVM guests" or B.2) "PV Bootloader support"

I will put it wherever you guys think is best

Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

Removed security@ to avoid creating unnecessary RT tickets. Please respond
to this mail. I suppose on this specific topic, Tamas and Andy Cooper
should voice an opinion also
Regards
Lars

On 27/06/2017, 10:48, "Razvan Cojocaru"  wrote:

>Hello,
>
>> - Security > Alternative 2pm : Supported – I think we should split this
>> out – it is currently implicitly covered under "Virtual Machine
>> Introspection"
>
>I agree that altp2m deserves its own space. While we're interested in
>it, our current solution makes no use of it, so it's certainly possible
>to do introspection without any help from altp2m.
>
>From my, albeit limited, experience, altp2m probably fits under "Tech
>preview".
>
>If we subtract altp2m from the general "Virtual Machine Introspection"
>category, I'd say that the upstream support is between "Tech Preview"
>and "Supported". I'll explain.
>
>The interface is largely stable, but we may need to add optimizations
>which might change it - so while we don't want this to be happening a
>lot, it will happen.
>
>There's also functional stability with the XenServer patches (which
>bypass the emulator (single-step) when it can't emulate, and has a
>version of the old smp_lock() emulator race-condition patch).
>Unfortunately, upstream Xen still has race condition issues when
>emulating LOCKed instructions in SMP scenarios - this will be addressed
>ASAP with a proper CMPXCHG patch that currently depends on a patch from
>Andrew and will require rework of a patch from Jan that adds a specific
>emulation return code for CMPXCHG failures.
>
>There's also an issue with #UD injections which is covered by the
>emulator bypass patch in XenServer but can cause IPIs to get lost with
>the upstream code - that will also require some more thinking.
>
>So there are still (emulation-related) quirks upstream. We'd very much
>like to get them ironed out.
>
>I hope this answers the question.
>
>
>Thanks,
>Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [For 4.9] Updating https://wiki.xenproject.org/wiki/Xen_Project_Release_Features to reflect support status of new features

[Lars Kurth]

Hi all, (I think I CCed all stake-holders)

to finish off the release documentation for 4.9, I need to add an extra column 
to https://wiki.xenproject.org/wiki/Xen_Project_Release_Features – because I 
was travelling, this dropped of my radar. There several decisions to be made:
A) Decide which "features" to add
B) Decide on the status of the feature
C) Deal with status changes of any past features

The first goal would be to decide on A and any new "features" under C. For B, I 
am OK to add "???" for now and point to this thread, until we have concluded 
the discussion

Note that I tracked some of this as preparation for getting CNA status.  Items 
marked with * are not yet in the discussion document that I created for the 
security team and which we intend to discuss at the summit.

For all of these, the naming convention is "Section in document" > "Feature" : 
"Support status". The definition of support status is added at the end of the 
mail: note that the text has not yet been fully agreed, but seems to reflect 
fairly well how we handled stuff in the past.

== On A / B: I think we should add ==
- Resource Management > Null Scheduler : tech preview or experimental
- Virtual Firmware or PV Bootloader Support (not sure which) >  x86/Boot Xen on 
EFI platforms using GRUB2*  : ???
- Hardware > ARM/Alternative Runtime Patching (ARM32 and ARM64): ??? [note that 
this should probably have been added for 4.8, but I didn't add it]
- Hardware > ARM/System Error Protection* : ???
- Hardware > ARM/Wait for Virtual Interrupt* : ???
- Hardware > x86/AVX512/Neural Network Instructions AVX512_4VNNIW* : ???
- Hardware > x86/AVX512/Multiply Accumulation Single precision AVX512_4FMAPS* : 
???
- Device Models > DMOP (Device Model Operation Hypercall)  : ???

New Heading:  PV Protocols and Drivers
- PV Protocols and Drivers > pvcalls : tech preview or experimental
- PV Protocols and Drivers > 9pfs : tech preview or experimental
- PV Protocols and Drivers* > sndif (sound device) : tech preview or 
experimental
- PV Protocols and Drivers* > displif (PV display) : tech preview or 
experimental

Did I miss anything?

== On C ==
- Security > Live Patching - see 
https://lists.xenproject.org/archives/html/xen-devel/2017-06/threads.html#03039
- Security > Alternative 2pm : Supported – I think we should split this out – 
it is currently implicitly covered under "Virtual Machine Introspection"

If we introduce a new heading "PV Protocols and Drivers" we should probably 
list all the common ones as supported in this heading, e.g.
- PV Protocols and Drivers* > default (net, block, console, keyboard, mouse) : 
supported

There are also USB and framebuffer, which I am not sure whether they should be 
supported and if not, what their status is
- PV Protocols and Drivers* > USB : ???
- PV Protocols and Drivers* > framebuffer : ???

Suggestions are welcome

Best Regards
Lars



## Definitions

### User-facing Support Criteria

 * **Functionally complete:** Does it behave like a fully functional
   feature?  Does it work on all expected platforms, or does it only work
   for a very specific sub-case?  Does it have a sensible UI, or do you
   have to have a deep understanding of the internals to get it to work
   properly?

 * **Functional stability:** What is the risk of it exhibiting bugs?

   General answers to the above:

   - *Here be dragons*: Pretty likely to still crash / fail to work.  Not
 recommended unless you like life on the bleeding edge.
   - *Quirky*: Mostly works but may have odd behavior here and there.
 Recommended for playing around or for non-production use cases.
   - *Normal*: Ready for production use

*  **Interface stability:**  If I build a system based on the current
   interfaces, will they still work when I upgrade to the next version?

   - *Not stable*: Interface is still in the early stages and still fairly
 likely to be broken in future updates.
   - *Provisionally stable*: We're not yet promising backwards
 compatibility, but we think this is probably the final form of the
 interface.  It may still require some tweaks.
   - *Stable*: We will try very hard to avoid breaking backwards
 compatibility, and to fix any regressions that are reported.

*  **Security supported:** Will XSAs be issued if security-related bugs are
   discovered in the functionality?

### Definition of Support Labels

Rather than specify each level above, we have some short-hand labels that we 
use to denote general answer to the above questions.

# Experimental
 Functional completeness: No
 Functional stability: Here be dragons
 Interface stability: Not stable
 Security supported: No

# Tech Preview
 Functional completeness: Yes
 Functional stability: Quirky
 Interface stability: Provisionally stable
 Security supported: No.

# Supported
 Functional completeness: Yes
 Functional stability: Normal
 Interface stability: Yes
 Security supported: Yes

# Deprecated
 Functional completeness: Yes
 Functional stab

Re: [Xen-devel] [PATCH for-4.9] livepatch: Declare live patching as a supported feature

[Lars Kurth]

Hi all,

there was also a discussion on IRC, which Ian said we should formally
summarise in e-mail, just so there is no doubt. So here is my go at it. As
far as I can tell - besides the technical discussion in this thread, there
are several issues which need to be clarified:

* For Xen 4.9 we can declare live patching supported, without spinning
another RC to update the in-tree documentation: in other words, we would
apply the documentation/policy changes + to the 4.9 tree sometimes after
this discussion has been concluded. In effect this means that
docs/features/livepatching.pandoc (or similar) and associated changes to
KCONFIG options would not show up until Xen 4.9.1 is spun, but the
security team would treat live patching as supported for 4.9. In other
words for now, we can update the table in the wiki
(https://wiki.xenproject.org/wiki/Xen_Project_Release_Features) and live
with in-tree artefacts being out-of-sync with the support status for a few
months. We need to fix this anyway in-tree and there is a concrete
proposal which should be discussed at the summit.

* There was a proposal to declare live patching supported for older
releases (aka "back port" docs/features/livepatching.pandoc), but Royger
pointed out that the toolstack in question needs to support buildid. If
so, we should include back-porting requests and d

* Julien pointed out that maybe we shouldn't declare live patching as
supported for ARM32/64. I don't see an issue to declare it supported for
x86/amd64 only for now. But it is obviously up to committers to make that
call.

I think that covers the ghist of the IRC discussion

Regards
Lars

On 27/06/2017, 08:24, "Julien Grall"  wrote:

>
>
>On 06/26/2017 10:07 PM, Konrad Rzeszutek Wilk wrote:
>> On Mon, Jun 26, 2017 at 07:29:22PM +0100, Julien Grall wrote:
>>> Hi,
>>>
>>> On 06/26/2017 04:36 PM, Ross Lagerwall wrote:
 Xen Live Patching has been available as tech preview feature since Xen
 4.7 and has now had a couple of releases to stabilize. Xen Live
patching
 has been used by multiple vendors to fix several real-world security
 issues without any severe bugs encountered. Additionally, there are
now
 tests in OSSTest that test live patching to ensure that no regressions
 are introduced.

 Based on the amount of testing and usage it has had, we are ready to
 declare live patching as a 'Supported' feature.
>>>
>>> There are only test for x86 and amd64. We likely want to have those
>>>test
>> 
>> The test-cases are also for ARM32.
>> 
>>> enabled for all architectures by default.
>> 
>> And the OSSTest can test all of those.
>
>Can we enable them by default? I know that we limited the number of
>tests for ARM64 due to limited bandwidth. But I don't think we have
>anything preventing it on ARM32.
>
>>>
>>> Also, I am not aware of anyone using in production livepatch on ARM64
>>>and
>>> ARM32. So did anyone give a good kick at the ARM implementaton?
>> 
>> I am not aware of anybody using it on production on ARM32 or ARM64.
>> 
>> The test-cases are there, the code is there, but yes nobody has kicked
>> the tires on ARM32/ARM64 extensively with it. I would be excited to
>> see vendors that use it and their reports but I am not aware of any.
>> 
>>>
>>> If not, then we should  do it before even considering as a supported
>>>feature
>>> for ARM.
>> 
>> OK. Perhaps then only for x86 until ARM operational users pipe up?
>
>That would be my preference. My main concern is to handle security issue
>afterwards because we didn't give any kick at the code.
>
>Cheers,
>
>-- 
>Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations 4.7.3 and 4.6.6

[Lars Kurth]

Hi all,
I am not going to be able to do the website work until Monday, as
travelling until late Friday
Lars

On 20/06/2017, 20:51, "Wei Liu"  wrote:

>On Fri, Jun 09, 2017 at 06:07:56AM -0600, Jan Beulich wrote:
>> All,
>> 
>> with the goal of releasing in about 3 weeks time, please point out
>> backport candidates you find missing from the respective staging
>> branches, but which you consider relevant. Please note that 4.6.6
>> is expected to be the last xenproject.org managed release from
>> its branch.
>
>I don't think I have any backport to do. Please let me know when I
>should tag the tree.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.9: Release date

[Lars Kurth]

Hi all, 

removed xen-announce

I created the following docs

https://wiki.xenproject.org/wiki/Category:Xen_4.9

If anyone created any 4.9 specific docs, feel free to add to the page or
let me know: I added links to generated 9pfs and pvcalls docs

https://wiki.xenproject.org/wiki/Xen_Project_4.9_Release_Notes

@Julien & everyone else: any restrictions, known issues, ... should go
here!

https://wiki.xenproject.org/wiki/Xen_Project_4.9_Feature_List

The only thing missing is the change-list: will add this *after* the last
RC was cut
Edits/additions by people who added features are welcome

https://wiki.xenproject.org/wiki/Xen_Project_4.9_Man_Pages
I added new pages (ran a diff) as there were lots of refactoring changes
Ran link checker: ok

https://wiki.xenproject.org/wiki/Xen_Project_4.9_Acknowledgements

Provisional with data to be updated on final RC (have a simple spreadsheet
which calculates these)
Is missing the individual acknowledgements, which I will do after the
final RC

The only thing which won't change is
https://wiki.xenproject.org/wiki/Xen_Project_4.9_Acknowledgements#4.9_Hyper
visor_Reviewers_.5B_5_.5D
For reviews, I can't map these onto a specific branch, so counted review
comments by people other than proposer in the time from "git-merge-base
staging-4.8 staging-4.9" (did git-merge-base staging-4.7 staging-4.8 for
the previous release).

https://wiki.xenproject.org/wiki/Xen_Project_Release_Features

Have not touched this yet

https://xenproject.org/downloads/xen-archives/xen-project-49-series.html &
other artifacts

Will create, when we cut the tarballs

Regards
Lars

On 02/06/2017 17:15, "Julien Grall"  wrote:

>Hi all,
>
>There are some pending security issues that have been found during the
>hardening period, which haven't been pre-disclosed yet.
>
>I am going to delay the release until one week after the embargo has
>lifted. I will give an exact time frame when they have been pre-disclosed.
>
>Cheers,
>
>-- 
>Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Schedule is live for the Xen Project’s Annual Conference - Discounted Registration closes by end of May

[Lars Kurth]

Dear Community members,
 
Community, embedded, automotive, performance, hardware, and security are a few 
topics that will be covered during the upcoming Xen Project Developer and 
Design Summit 
(http://events.linuxfoundation.org/events/xen-developer-and-design-summit) 
happening July 11-13 in Budapest, Hungary. 

You can find the full schedule here: 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/schedule
 
The conference is combines the formats of the Xen Project Developer Summits 
with the Xen Project Hackathons, with talks in the morning and Design Sessions 
in the afternoon.

Here’s a sampling of some of the talks at the Summit:
* Dedicated Secure Domain as an Approach for Certification of Automotive Sector 
Solutions from Iurii Mykhalskyi of GlobalLogic
* Uniprof: Transparent Unikernel Performance Profiling and Debugging from 
Florian Schmidt of NEC
* Hypervisor-Based Security: Bringing Virtualized Exceptions Into the Game from 
Mihai Dontu of Bitdefender
* Bring up PCI Passthrough on ARM from Julien Grall of ARM
* Shared Coprocessor Framework on ARM - Oleksandr Andrushchenko, EPAM Systems
* Secure Containers with Xen and CoreOS rkt - Stefano Stabellini, Aporeto
* To Grant or Not to Grant? - João Martins, Oracle
 
Currently, we have an early bird special ($250), which will end at the end of 
May (see 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/attend/register).
 Note that the overall ticket price is higher compared to the past, as the 
event is 3 days long and based on your feedback from last year, we included 
lunch.

Also note that you can still submit design sessions: see 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session
 
Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Livepatching and Xen Security

[Lars Kurth]

On 18/05/2017 17:53, "Ian Jackson"  wrote:

>George Dunlap writes ("Livepatching and Xen Security"):
>> # Executive summary
>
>I am completely in agreement with your analysis and your conclusions.

Me too. I am not sure though whether we need a vote or lazy consensus.

For Credit2 (see 
https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg00171.html)
 we used a code patch removing Experimental from KCONFIG, which implicitly
led to active confirmation by project leadership members via Acked-by
tags. In other words, we executed the equivalent of a vote.

From a process perspective
https://xenproject.org/governance.html#lazyconsensus should be sufficient
because we are applying a process, not changing one. In addition lazy
consensus decisions can only be overturned if the project leadership
agrees collectively to do so, because the decision is too important for
lazy consensus. As every leadership member is on the list, there would be
ample opportunity to raise objections.

My gut feeling though is that Leadership Team members and Security Team
members should probably actively agree/disagree to proposals related to
whether a feature is supported and thus security supported to avoid any
future issues. George already expressed an implicit +1 (by making the
proposal) and Ian an explicit +1.

Regards
Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.9 rc4

[Lars Kurth]

Removing the announce and user list.

> Could you please clarify if this [1] is still relevant?
Yes

I will let Julien answer the rest: the main goal is really to ensure that
Xen 4.9 works with the HW you care about.

Lars

On 12/05/2017 15:29, "Andrii Anisov"  wrote:

>Dear Julien,
>
>
>On 08.05.17 21:41, Julien Grall wrote:
>> As a reminder, there will be another Xen Test Day tomorrow (Tuesday 9th
>>May),
>> for the instructions see:
>>
>> 
>>https://blog.xenproject.org/2017/04/13/announcing-xen-project-4-9-rc-and-
>>test-day-schedule/
>>
>> Cheers,
>
>I'm asked to perform a XEN 4.9.0-rcX testing on Renesas R-Car Gen3 board
>(Salvator-X, H3).
>
>Could you please clarify if this [1] is still relevant?
>And if the smoke test described here [2] is enough for RC testing?
>
>[1] 
>https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions#Specific_ARM
>_Test_Instructions
>[2] https://wiki.xenproject.org/wiki/Xen_ARM_Manual_Smoke_Test
>
>-- 
>
>*Andrii Anisov*
>
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Added 5 additional Design sessions to Developer Summit Schedule

[Lars Kurth]

> On 12 May 2017, at 03:44, Zhang, Yu C  wrote:
> 
> Thanks for pointing out, Yi. Our names may be confusing. :-)
> And yes, I will have a discussion session on 5 level paging enabling with 
> Juergen. 
> 
> B.R.
> Yu 

Apologies. This is fixed. It has less to do with names, but more to do with a 
very unwieldy spreadsheet which contains all the data from which I have to copy 
stuff: about 80 x 60 fields.
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Modules support in Xen (WAS: Re: [ARM] Native application design and discussion (I hope))

[Lars Kurth]

> On 11 May 2017, at 18:20, George Dunlap  wrote:
> 
> On Thu, May 11, 2017 at 6:14 PM, Volodymyr Babchuk
>  wrote:
>> Hi George,
>> 
>> On 11 May 2017 at 19:35, George Dunlap  wrote:
>>> Even better would be to skip the module-loading step entirely, and just
>>> compile proprietary code directly into your Xen binary.
>>> 
>>> Both solutions, unfortunately, are illegal.*
>> Look, I don't saying we want to produce closed-source modules or apps.
>> We want to write open source code. Just imagine, that certain header
>> files have some proprietary license (e.g. some device interface
>> definition and this interface is IP of company which developed it).
>> AFAIK, it can't be included into Xen distribution. I thought, that it
>> can be included in some module with different (but still open source)
>> license.  But if you say that it can't... Then I don't know. It is out
>> of my competence. I'm not lawyer also.
> 
> I see.  That's good to know, but it doesn't change the legal aspect of
> things. :-0

The legal issues would be similar to those with Linux Kernel Modules. For more 
information, see 
http://www.ifross.org/en/artikel/license-incompatibility-and-linux-kernel-modules-reloaded

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Added 5 additional Design sessions to Developer Summit Schedule

[Lars Kurth]

Hi everyone,

I added the following sessions to the Design Part of the Summit: you can see 
them via 
https://xendeveloperanddesignsummit2017.sched.com/overview/type/Interactive+Design+%26+Problem+Solving+Session
 (just showing Design sessions)

Please also make use of the "Add to my Sched(ule)" feature in 
https://xendeveloperanddesignsummit2017.sched.com: this will help me identify 
scheduling conflicts *before* the event and will make the event smoother: in 
other words we can make scheduling decisions using real data, before the event.

Added Sessions:

July 12
===
Design Discussion: Intel New QoS (RDT) Features - Yi Sun, Intel 

Design Discussion: SGX virtualization - Kai Huang, Intel

Design Discussion: Support for 5-level paging (including support for PV-guests) 
- Yi Liu, Intel & Jürgen Groß, SUSE 
Note: We merged a proposal from Yi and Jürgen. If we need more time, we can add 
another session on the 13th

July 13
===
Design Discussion: Shared Virtual Memory Virtualization Implementation on Xen - 
Yi Liu, Intel

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Vulnerability embargo dates - add your public holidays

[Lars Kurth]

> On 10 May 2017, at 15:37, Lars Kurth  wrote:
> 
>> 
>> On 10 May 2017, at 15:31, Juergen Gross  wrote:
>> 
>> On 10/05/17 16:07, Ian Jackson wrote:
>>> (dropping announce)
>>> 
>>> Juergen Gross writes ("Re: [Xen-devel] Vulnerability embargo dates - add 
>>> your public holidays"):
>>>> On 10/05/17 15:38, Ian Jackson wrote:
>>>>> Please see:
>>>>> https://wiki.xenproject.org/wiki/HolidayCalendar
>>>> 
>>>> Are you planning to add a link to this page somewhere in the wiki?
>>> 
>>> I haven't done so.  I guess it would be a good idea.  Please go ahead
>>> and do so :-).
>> 
>> Okay. But where?
>> 
>> I guess the most logical place would be the "Xen security problem
>> response process" definition, which I obviously can't change.
>> 
>> Another place would be:
>> 
>> https://wiki.xenproject.org/wiki/Security_Announcements_(Historical)
>> 
>> Any other ideas?
> 
> I can put a widget on the security response page on xenproject.org (there is 
> already one, linking to PGP keys, etc.)

I added "SECURITY POLICY RELATED DOCUMENTS" (top right) to 
https://xenproject.org/security-policy.html

Let me know if that works

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Vulnerability embargo dates - add your public holidays

[Lars Kurth]

> On 10 May 2017, at 15:31, Juergen Gross  wrote:
> 
> On 10/05/17 16:07, Ian Jackson wrote:
>> (dropping announce)
>> 
>> Juergen Gross writes ("Re: [Xen-devel] Vulnerability embargo dates - add 
>> your public holidays"):
>>> On 10/05/17 15:38, Ian Jackson wrote:
 Please see:
  https://wiki.xenproject.org/wiki/HolidayCalendar
>>> 
>>> Are you planning to add a link to this page somewhere in the wiki?
>> 
>> I haven't done so.  I guess it would be a good idea.  Please go ahead
>> and do so :-).
> 
> Okay. But where?
> 
> I guess the most logical place would be the "Xen security problem
> response process" definition, which I obviously can't change.
> 
> Another place would be:
> 
> https://wiki.xenproject.org/wiki/Security_Announcements_(Historical)
> 
> Any other ideas?

I can put a widget on the security response page on xenproject.org (there is 
already one, linking to PGP keys, etc.)

Lars


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen Project Developer Summit Design Sessions (Program published)

[Lars Kurth]

The link to the published program is at 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/schedule
 
<http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/schedule>
We will do PR in the coming weeks (we were ahead of schedule with the program)
Lars

> On 9 May 2017, at 16:10, Lars Kurth  wrote:
> 
> Hi everyone,
> we created a simplified page to submit Design Sessions at 
> http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session
>  
> <http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session>,
>  which is also available from the event page menu under "Program > CFP - 
> DESIGN SESSIONS". The old link will still work, but contains all the extra 
> info related to regular 30 minute sessions and panels, which are not accepted 
> any more.
> Regards
> Lars
> 
>> On 8 May 2017, at 10:32, Lars Kurth > <mailto:lars.kurth@gmail.com>> wrote:
>> 
>> Dear community members (I CC'ed people who submitted design sessions so far)
>> 
>> I got a few questions regarding "acceptance" and scheduling of Design 
>> Sessions for the Developer and Design Summit. As you may recall, the 
>> Developer and Design Summit mixes the formats of past Summits and 
>> Hackathons. The afternoons are reserved for Design sessions, which will 
>> follow the format of Hackathons.
>> 
>> Thus, in the tradition of the Xen Project Hackathons, we will not run Design 
>> Sessions through the Program Management Committee, as long as we have enough 
>> space to host sessions. The submission system is still open for Design 
>> Sessions at 
>> http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp
>>  
>> <http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp>
>>  (although the website says otherwise, which I will get fixed) and sessions 
>> can be submitted until Friday 7th of July. Sessions can still be proposed on 
>> the day of the event (using A5 post-it-notes) as in the past. 
>> 
>> Whenever new sessions come in, I will once a week put them live the 
>> afternoon Design Sessions on the summit schedule, using the following 
>> criteria
>> Schedule sessions on a day which has similar topics to the ones already 
>> proposed (makes the live scheduling sessions at the event easier)
>> Schedule sessions on (or after) a day where related talks take place 
>> (creates more valuable discussions)
>> Schedule such that people which have multiple sessions get some breathing 
>> space (aka avoid that everyone has all their sessions on one day)
>> 
>> Note that we will have room rooms with a projector, others won't. 
>> If you submit design sessions and you need a projector, please add this 
>> under "List any technical requirements that you have for your presentation 
>> over and above the standard projector, screen and wireless internet" 
>> If you need to have your session on a specific day due to travel 
>> constraints, please add this under "List any technical requirements..."
>> The purpose of publishing design sessions as they come in is twofold: 
>> let you know what others are planning to do (avoids duplicate sessions)
>> marketing (attracts attendees)
>> You can propose new design sessions every day of the event, but we prefer if 
>> you proposed them using the submission system. 
>> 
>> As in the past at Hackathons we will try and balance the schedule and merge 
>> related sessions as a group in a 30 minute scheduling session.  I am also 
>> happy getting feedback from community members on this thread: to facilitate 
>> this, I will post schedule updates every other week or so. 
>> 
>> As we have Linux Foundation staff on hand, we should also be able to update 
>> the on-line schedule immediately afterwards. But we will also have a paper 
>> schedule (using post-it notes), as it is easier to manage the scheduling 
>> session.
>> 
>> I attached the schedule for Design sessions as we will publish by the end of 
>> this week

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen Project Developer Summit Design Sessions (updated link for Design Sessions CFP)

[Lars Kurth]

Hi everyone,
we created a simplified page to submit Design Sessions at 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session
 
<http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp-design-session>,
 which is also available from the event page menu under "Program > CFP - DESIGN 
SESSIONS". The old link will still work, but contains all the extra info 
related to regular 30 minute sessions and panels, which are not accepted any 
more.
Regards
Lars

> On 8 May 2017, at 10:32, Lars Kurth  wrote:
> 
> Dear community members (I CC'ed people who submitted design sessions so far)
> 
> I got a few questions regarding "acceptance" and scheduling of Design 
> Sessions for the Developer and Design Summit. As you may recall, the 
> Developer and Design Summit mixes the formats of past Summits and Hackathons. 
> The afternoons are reserved for Design sessions, which will follow the format 
> of Hackathons.
> 
> Thus, in the tradition of the Xen Project Hackathons, we will not run Design 
> Sessions through the Program Management Committee, as long as we have enough 
> space to host sessions. The submission system is still open for Design 
> Sessions at 
> http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp
>  
> <http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp>
>  (although the website says otherwise, which I will get fixed) and sessions 
> can be submitted until Friday 7th of July. Sessions can still be proposed on 
> the day of the event (using A5 post-it-notes) as in the past. 
> 
> Whenever new sessions come in, I will once a week put them live the afternoon 
> Design Sessions on the summit schedule, using the following criteria
> Schedule sessions on a day which has similar topics to the ones already 
> proposed (makes the live scheduling sessions at the event easier)
> Schedule sessions on (or after) a day where related talks take place (creates 
> more valuable discussions)
> Schedule such that people which have multiple sessions get some breathing 
> space (aka avoid that everyone has all their sessions on one day)
> 
> Note that we will have room rooms with a projector, others won't. 
> If you submit design sessions and you need a projector, please add this under 
> "List any technical requirements that you have for your presentation over and 
> above the standard projector, screen and wireless internet" 
> If you need to have your session on a specific day due to travel constraints, 
> please add this under "List any technical requirements..."
> The purpose of publishing design sessions as they come in is twofold: 
> let you know what others are planning to do (avoids duplicate sessions)
> marketing (attracts attendees)
> You can propose new design sessions every day of the event, but we prefer if 
> you proposed them using the submission system. 
> 
> As in the past at Hackathons we will try and balance the schedule and merge 
> related sessions as a group in a 30 minute scheduling session.  I am also 
> happy getting feedback from community members on this thread: to facilitate 
> this, I will post schedule updates every other week or so. 
> 
> As we have Linux Foundation staff on hand, we should also be able to update 
> the on-line schedule immediately afterwards. But we will also have a paper 
> schedule (using post-it notes), as it is easier to manage the scheduling 
> session.
> 
> I attached the schedule for Design sessions as we will publish by the end of 
> this week
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] http://xenproject.org/ is down due to a server failure

[Lars Kurth]

The site is being restored on another server and should be back up at 13:00 UTC
Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Security support scope (apropos of Xen and CNA)

[Lars Kurth]

> On 8 May 2017, at 17:11, Andrew Cooper  wrote:
> 
> On 08/05/17 17:04, Ian Jackson wrote:
>> Tim Deegan writes ("Re: Security support scope (apropos of Xen and CNA)"):
>>> Ah, so it is.  So there is information on the wiki page that's not in
>>> MAINTAINERS.  Could that be moved into MAINTAINERS?  There are
>>> a few things that don't map well to maintainership of specific
>>> files, e.g. "vMCE" or nested virtualization.  But on the whole I
>>> think that adding clauses for them would be OK.
>> I think this is quite awkward, really.  MAINTAINERS is about files,
>> and implementations.  The security support status is about parts of
>> interfaces, which don't map at all well.
>> 
>> We could add no-files stanzas, but how would you tell what they
>> referred to ?
> 
> This is the principle behind introducing docs/features/* which, as part
> of the required metadata, contains a support statement.
> 
> This way, there is an authoritative statement of support on a
> per-feature basis which is easy to keep up to date.
> 
> Lars: Any update on your project level clarifications of support statuses?

No, there is no vendor agreement yet on the testing side (aka what testing they 
commit to and what would happen if 3rd party testing outside of OSSTEST does 
not take place). That and what to do with "security supported" components which 
have poor testing for historical reasons are the only outstanding issue. These 
are really big issues, where there is no real consensus. I will get a Design 
Session in place for this at the summit: maybe we can make some progress.

However, IMHO, we can start off agreeing a format: in nutshell what we need is 
a list that shows feature=status - although we may want to add a few more 
fields, aka feature=(support status=..., maintainer status=, tested=OSSTEST|XTF|3rd Party|..., ...). Someone needs to make a sensible 
proposal.

For example, we could just map "status" on what we have now: preview, 
experimental, supported (which includes security support) and extend/change the 
meaning of the status field as this gets resolved. I don't think the two things 
are strongly linked.

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Security response; public holidays [and 1 more messages] [and 1 more messages]

[Lars Kurth]

> On 8 May 2017, at 16:07, George Dunlap  wrote:
> 
> On 08/05/17 15:50, Ian Jackson wrote:
>> Lars Kurth writes ("Re: Security response; public holidays [and 1 more 
>> messages]"):
>>> I think this makes sense. It is probably also worth looking at 
>>> http://www.officeholidays.com/ (or something similar)
>> 
>> I deliberately decided to avoid one of those portmanteau sites because
>> the set of holidays listed there may not be the set of ones that the
>> Xen Project Community cares about.  There is also an awkardness in
>> referring to a random third-party site with uncertain editorial
>> practices.
>> 
>> Ie I think the right approach is for community members who care about
>> this to record the holidays relevant to them in a Xen Project wiki
>> page.

Agreed: this was more about checking what people add to the wiki page.

> +1 to this.  Those might be good prompts for people to remember their
> own national holidays which are important; but we're not really in a
> position to tell which holidays people actually affect people's
> availability.

+1

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Security response; public holidays [and 1 more messages]

[Lars Kurth]

> On 8 May 2017, at 15:29, Ian Jackson  wrote:
> 
> Ian Jackson writes ("Security response; public holidays"):
>> It would perhaps be possible to find a holiday calendar listing all
>> known public holidays throughout the world.  But we are interested in
>> public holidays affecting Xen Project community members.
>> 
>> I was wondering if we should start a wiki page and invite community
>> members to add their own local holidays.
> 
> I have now created this page:
>  https://wiki.xenproject.org/wiki/HolidayCalendar
> 
> I populated it with holidays from my own jurisdiction, the UK, to
> provide a skeleton.
> 
> Does that look about right ?  If so then we should send out a call on
> xen-announce for community members to add their own dates.
> 
> The policy doesn't explicitly ask us to avoid holidays.  It talks
> about a "usual starting point" and "reasons to diverge".  We have
> usually treated public holidays (at least, ones which are widely
> observed) as reasons to diverge and no-one seems to object to this
> practice.
> 
> If anyone things this needs to be made more explicit in the policy, I
> wouldn't oppose that, but personally I think what we have is OK.
> 
>> I suggest we set a reminder to send out a call for updates in around
>> October of the previous year.
> 
> I changed this to September and put it in the bottom of the page.

I think this makes sense. It is probably also worth looking at 
http://www.officeholidays.com/ (or something similar)

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Xen Project Developer Summit Design Sessions

[Lars Kurth]

Dear community members (I CC'ed people who submitted design sessions so far)

I got a few questions regarding "acceptance" and scheduling of Design Sessions 
for the Developer and Design Summit. As you may recall, the Developer and 
Design Summit mixes the formats of past Summits and Hackathons. The afternoons 
are reserved for Design sessions, which will follow the format of Hackathons.

Thus, in the tradition of the Xen Project Hackathons, we will not run Design 
Sessions through the Program Management Committee, as long as we have enough 
space to host sessions. The submission system is still open for Design Sessions 
at 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit/program/cfp
 

 (although the website says otherwise, which I will get fixed) and sessions can 
be submitted until Friday 7th of July. Sessions can still be proposed on the 
day of the event (using A5 post-it-notes) as in the past. 

Whenever new sessions come in, I will once a week put them live the afternoon 
Design Sessions on the summit schedule, using the following criteria
Schedule sessions on a day which has similar topics to the ones already 
proposed (makes the live scheduling sessions at the event easier)
Schedule sessions on (or after) a day where related talks take place (creates 
more valuable discussions)
Schedule such that people which have multiple sessions get some breathing space 
(aka avoid that everyone has all their sessions on one day)

Note that we will have room rooms with a projector, others won't. 
If you submit design sessions and you need a projector, please add this under 
"List any technical requirements that you have for your presentation over and 
above the standard projector, screen and wireless internet" 
If you need to have your session on a specific day due to travel constraints, 
please add this under "List any technical requirements..."
The purpose of publishing design sessions as they come in is twofold: 
let you know what others are planning to do (avoids duplicate sessions)
marketing (attracts attendees)
You can propose new design sessions every day of the event, but we prefer if 
you proposed them using the submission system. 

As in the past at Hackathons we will try and balance the schedule and merge 
related sessions as a group in a 30 minute scheduling session.  I am also happy 
getting feedback from community members on this thread: to facilitate this, I 
will post schedule updates every other week or so. 

As we have Linux Foundation staff on hand, we should also be able to update the 
on-line schedule immediately afterwards. But we will also have a paper schedule 
(using post-it notes), as it is easier to manage the scheduling session.

I attached the schedule for Design sessions as we will publish by the end of 
this week

Legend  Additional Notes
AV, Projector, Podium, Recording
AV, Projector, Podium, NO Recording SCHEDULING sessions
AV, Projector, Podium, Recording if we can afford itIf not, we may have to 
drop or maybe find a vendor to do a special sponsorship
Projector only, Flipchart or whiteboard 
Flipchart or whiteboard, NO AV  

Time / Duration July 11
14:00 - 14:30   ANNOUNCEMENT OF NEW DESIGN SESSIONS, SCHEDULING OF DESIGN 
SESSIONS
Tas Huba/TohotomDery/Jokai  Krudy/Arany Mikszath/Petrofi
Valletta II
14:30 - 15:15   45 min Design   45 min Design   45 min Design   EFI secure boot 
+ shim + GRUB2 + Xen - how to make it work - Daniel Kiper - Oracle  45 min 
Design   Contributing to Xen: An Introduction - George Dunlap - Citrix
15:20 - 16:05   45 min Design   45 min Design   45 min Design   EFI + Intel TXT 
and TPM + Xen/Linux - how to make it work - Daniel Kiper - Oracle   
PV-IOMMU - Paul Durrant - Citrix45 min Design
16:05 - 16:35   BREAK - Foyer
16:35 - 17:35   60 min Design   60 min Design   60 min Design   60 min Design   
Graphics Virtualization - Rich Persaud - BAE Systems60 min Design

Time / Duration July 12
14:00 - 14:30   ANNOUNCEMENT OF NEW DESIGN SESSIONS, SCHEDULING OF DESIGN 
SESSIONS
Tas Huba/TohotomDery/Jokai  Krudy/Arany Mikszath/Petrofi
Valletta II
14:30 - 15:15   45 min Design   45 min Design   45 min Design   45 min Design   
Improvements to Continuous Integration Workflow - Doug Goldstein - Star Lab 
45 min Design
15:20 - 16:05   45 min Design   45 min Design   45 min Design   45 min Design   
Default Tests and Configuration of Server and Edge Hypervisors - Rich Persaud - 
BAE Systems 45 min Design
16:05 - 16:35   BREAK - Foyer
16:35 - 17:35   60 min Design   60 min Design   60 min Design   60 min Design   
Unikernel support for NFV-like applications on Xen ARM 64bit - Anastassios 
Nanos - OnAppZerocopy on Xen PV drivers - João Martins - Oracle

Time / Duration July 13
14:00 - 14:30   ANNOUNCEMENT OF NEW DESIGN SESSIONS, SCHEDULING OF DESIGN 
SESSIONS
T

Re: [Xen-devel] Security support scope (apropos of Xen and CNA)

[Lars Kurth]

> On 5 May 2017, at 09:43, Tim Deegan  wrote:
> 
> At 13:53 +0100 on 04 May (1493905990), Ian Jackson wrote:
>> To become a CNA (CVE Numbering Authority), which we would like to do,
>> we need to provide MITRE's CNA programme with a definition of the
>> scope of our CNA.  That should be the scope of our general security
>> support, clearly.
>> 
>> At the moment we don't seem to have this written down in a single
>> clear document.  I am aware of the following places which can contain
>> information about security support (normally, in the form of
>> statements saying that certain things are not supported):
>> 
>> * https://wiki.xenproject.org/wiki/Xen_Project_Release_Features has a
>>   table of versions with security support, and information about some
>>   features.
>> 
>> * xen.git:docs/misc/qemu-xen-security, limits security support to
>>   some configurations.
>> 
>> * xen.git:MAINTAINERS might in principle have a status not implying
>>   security support.
>> 
>> * Docs for an individual feature (eg in xl docs) might say that the
>>   feature is not advised, or not supported, or something.
>> 
>> * Previous XSA advisories might withdraw support.
>> 
>> This diversity of information sources is rather unsatisfactory.
>> 
>> I think we need to at least reduce the number of different information
>> sources.  Also we need an overview document which points to them all.
>> 
>> Where should this overview document be ?  Which of the above sources
>> should be coalesced into which others ?
> 
> IMO the overview should on the main xenproject.org site, ideally in
> the security process preamble, or beside it if it gets too long.

I am happy with that

> It should read something like this:
> 
> - Security support is provided for the following versions:
>   [List of versions, + an item on the release checklist to update it.]

A bit more work, but can be done

> - Only features listed as Supported in MAINTAINERS get support.

This seems related to George's proposal of the scope. I am not sure MAINTAINERS 
is correct though (e.g. live-patching is probably listed as Supported but does 
not get security support)

> - Specific exemptions:
>   [ move qemu-xen-security here, and delete it from the tree ]
>   [ brief summary of XSA-77 + a link for details. ] 
>   [ anything else?  I don't think we need to explicitly call out to
> docs for individual features, but there might be some things
> to mention here, e.g. DMA attacks with IOMMU disabled. ]
> 
> Not sure about the Xen_Project_Release_Features wiki page -- it's nice
> to have all that info + historical versions in one place; on the
> other hand it's not the canonical source for most of it and risks
> getting out of date.  Maybe it needs an introduction pointing out
> that MAINTAINERS and the new security scope doc are the official sources.

Also everyone can edit it. To be honest, if we need to make changes frequently, 
we should probably maintain this in-tree. 

Lars


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] differing opinions between maintainers vs patch acks

[Lars Kurth]

On 04/05/2017 13:44, "Ian Jackson"  wrote:

>Andrew Cooper writes ("Re: differing opinions between maintainers vs
>patch acks"):
>> Taking this example, as you have called it out, but without going into
>> the details.
>> 
>> I accept that the issues under debate do not have any impact on the
>> technical correctness of the fix.  Once compiled/assembled, the binary
>> will function correctly.
>> 
>> However, the bikeshedding makes a very real material impact on the
>> understandability and reviewability of the code.
>> 
>> In my mind, all other things being equal, making the code easier to
>> understand and review is of paramount importance, and particularly in
>> this case, not making an already complicated bit of code harder to
>>review.
>
>Well, at one level I agree with Andrew on at least the 1*1 and 0*8
>question.  These seem clearer to me as they state the programmer's
>intent as well as merely the effect.  I found Jan's response hard to
>understand; there doesn't actually seem to be a counterargument.  I
>suspect if I thought about it enough I would agree with Andrew about
>the labels too.
>
>But, earlier I said:
>
>   I definitely agree that there is room for giving the author of some
>   code (whether they are a maintainer or not) some leeway on matters of
>   taste.  I think, though, that while this ought to be a principle
>   applied by maintainers, committers and anyone else making relevant
>   decisions, it is not a rule of governance to be applied in contested
>   cases.
>
>I think this case falls clearly into the category of things where we
>could give the original contributor some leeway.  In this case that
>means Jan.
>
>IOW if I were in Andrew's position I would probably make the same
>requests he has done, but if Jan maintained his position I would
>certainly not block the patch over this.
>
>Stepping back a bit: It is indeed important that our code is easy to
>understand and modify, expresses its intent clearly, and helps future
>programmers avoid writing bugs.  But it is also important that
>contributors feel valued, and feel a sense of ownership.

Agreed.

>The amount of emotional discouragement to a contributor does not scale
>linearly with the size and apparent importance of the disagreement.
>Indeed, turning a tiny issue into a blocker or a big argument can be
>especially demotivating.

Agreed.

Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] differing opinions between maintainers vs patch acks

[Lars Kurth]

On 04/05/2017 10:55, "Ian Jackson"  wrote:

>Jan Beulich writes ("Re: differing opinions between maintainers vs patch
>acks"):
>> On 04.05.17 at 11:21,  wrote:
>> > I'm afraid I disagree.  Someone with a fresh perspective is often
>> > helpful, even if it involves a bit more explanation.
>> > 
>> > And, the use of committers as referees in inter-maintainer disputes is
>> > explicitly laid out in the project governance document:
>> >   https://xenproject.org/governance.html#roles-local
>> >   | Committers
>> >   ...
>> >   | committers can also act as referees should disagreements amongst
>> >   | maintainers arise
>> 
>> I understand this, and I agree as far as actual technical issues
>> go. I just don't think this is suitable / appropriate when it comes
>> to cosmetic questions.
>
>Well, if the cosmetic question is not that important then presumably
>someone would have given ground and it wouldn't need escalation.

In this specific case, I saw a disagreement, but it is not clear whether
this would need an escalation. It could easily be solved on IRC by a quick
chat.

>If the cosmetic question _is_ important enough to have a big debate
>over then I don't see any reason why the same escalation path is not
>appropriate.  Actually, it seems to me that a relative outsider is
>more likely to bring a useful sense of proportion.

However in general, I agree with Ian that "cosmetic" issues which are
contentious and where emotions are attached to it should be covered by the
governance. 


>(And, de jure, the governance doc makes no such distinction.)

As I said, these are intentionally not excluded. Partly, because cosmetic
issues are often more divisive than actual technical issues, and in many
FOSS communities these are exactly the sort of issues people fight over.
Of course, one would hope that we are all mature enough, that we don't
often need to invoke refereeing for it.

Lars

 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable test] 107560: tolerable FAIL - PUSHED

[Lars Kurth]

On 21/04/2017 12:01, "Julien Grall"  wrote:

>Hi,
>
>On 21/04/17 01:01, osstest service owner wrote:
>> flight 107560 xen-unstable real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/107560/
>>
>> Failures :-/ but no regressions.
>>
>> Regressions which are regarded as allowable (not blocking):
>>  test-armhf-armhf-libvirt 13 saverestore-support-checkfail
>>like 107528
>>  test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail
>>like 107539
>>  test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail
>>like 107539
>>  test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail
>>like 107539
>>  test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail
>>like 107539
>>  test-armhf-armhf-libvirt-raw 12 saverestore-support-checkfail
>>like 107539
>>  test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail
>>like 107539
>>  test-amd64-amd64-xl-rtds  9 debian-install   fail
>>like 107539
>>
>> Tests which did not succeed, but are not blocking:
>>  test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail
>>never pass
>>  test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail
>>never pass
>>  test-amd64-i386-libvirt  12 migrate-support-checkfail
>>never pass
>>  test-amd64-amd64-libvirt 12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-xsm  12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-xsm  13 saverestore-support-checkfail
>>never pass
>>  test-arm64-arm64-libvirt-xsm 12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-libvirt-xsm 13 saverestore-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-multivcpu 12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-multivcpu 13 saverestore-support-checkfail
>>never pass
>>  test-arm64-arm64-xl  12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-xl  13 saverestore-support-checkfail
>>never pass
>>  test-arm64-arm64-libvirt 12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-libvirt 13 saverestore-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-credit2  12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-credit2  13 saverestore-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-rtds 12 migrate-support-checkfail
>>never pass
>>  test-arm64-arm64-xl-rtds 13 saverestore-support-checkfail
>>never pass
>
>We don't have migration supported on ARM64 yet, but all the rest is
>successfully passing.
>
>This is the first Xen unstable flight with successful test on Seattle :).
>
>Thank you all for your help!

That is fantastic

Thank you

Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v10 09/25] x86: refactor psr: L3 CAT: set value: implement framework.

[Lars Kurth]

> On 20 Apr 2017, at 14:21, Jan Beulich  wrote:
> 
 On 20.04.17 at 15:02,  wrote:
>>> On 20 Apr 2017, at 10:43, Jan Beulich  wrote:
>>> 
>> On 20.04.17 at 04:14,  wrote:
 On 17-04-19 03:00:29, Jan Beulich wrote:
 On 19.04.17 at 10:22,  wrote:
>> On 17-04-18 05:46:43, Jan Beulich wrote:
>> On 18.04.17 at 12:55,  wrote:
 I made a test patch based on v10 and attached it in mail. Could you 
 please
 help to check it? Thanks!
>>> 

[Item 1]

>>> This looks reasonable at the first glance, albeit I continue to be
>>> unconvinced that this is the only (reasonable) way of solving the
 
 Do you have any other suggestion on this? Thanks!
>>> 
>>> I'm sorry, but no. I'm already spending _much_ more time on this
>>> series than I should be, considering its (afaic) relatively low
>>> priority. I really have to ask you to properly think through both
>>> the data layout and code structure, including considering
>>> alternatives especially in places where you can _anticipate_
>>> controversy.
>> 
>> Jan, can you confirm whether you are happy with the current proposal. You 
>> say "this looks reasonable at the first glance", but then go on to say that 
>> there may be other "reasonable ways" of solving the problem at hand.
>> 
>> This is not very concrete and hard to respond to from a contributors point 
>> of view: it would be good to establish whether a) the v10 proposal in this 
>> area is good enough, b) whether there are any concrete improvements to be 
>> made.

> I understand it's not very concrete, but please understand that with
> the over 100 patches wanting looking at right now it is simply
> impossible for me to give precise suggestions everywhere. I really
> have to be allowed to defer to the originator to come up with
> possible better mechanisms (or defend what there is by addressing
> questions raised),

Jan, I don't object to the principle of deferring issues to a contributor, for 
contributor to defend their viewpoint or to come up with a better mechanism. I 
just observed, that I could not make a lot of sense what you were looking for 
in this particular review. I am assuming that it would be even harder for Yi. 

> especially with - as said - the amount of time spent
> here already having been way higher than is justifiable.

And of course we have passed the 4.9 code freeze, so some of the pressure is 
off. At the same time I understand that because of the upcoming releases you 
need to focus on bug fixes, etc.

> Just go and
> compare v10 with one of the initial versions: Almost all of the data
> layout and code flow have fundamentally changed, mostly based on
> feedback I gave. I'm sorry for saying that, but to me this is an
> indication that things hadn't been thought through well in the design
> phase, i.e. before even submitting a first RFC.

That is good feedback which may contain some valuable lessons. Once we are 
through this (or maybe at the summit) it may be worthwhile to look at what has 
gone wrong and see how we can do better in future.

[Item 2]

>> You say please think through whether "you can _anticipate_ controversy", but 
>> at the same time you can't currently identify/think of any issues. That 
>> seems 
>> to suggest to me that maybe the proposal is good enough. Or that something 
>> is 
>> missing, which has not been articulated. Taking into account language 
>> barriers, more clarity would probably be helpful.
> 
> I've given criteria by which I have the gut feeling (but no more)
> that this isn't the right approach. I'm absolutely fine to be
> convinced that my gut feeling is wrong. That would require to
> simply answer the question I raised multiple times, and which was
> repeatedly "answered" by simply re-stating previously expressed
> facts.

I have not followed the full thread. But it seems that we have communications 
issue there. Normally this happens when expectations don't quite match and one 
party (in this case Yi) does not quite get what the other one is looking for. 
Maybe the best approach would be for Yi to get some of these things resolved 
during a short IRC conversation with you. I did see him and others resolve some 
previous issues more effectively on IRC. 

> If this reaction of mine is not acceptable, all I can do is refrain
> from further looking at this series. And Yi, I certainly apologize
> for perhaps not doing these reviews wholeheartedly, since -
> as also expressed before - I continue to not really view this as
> very important functionality.

As I said earlier, I stepped in, as I didn't really understand what was going 
on. I think this is a little clearer now. 

So to summarise: 

On item 1: it appears that you are looking for a some more justification why 
some of the changes were made, maybe with a rationale for some of the choices 
that were made. Given that this is quite a complex series which has diverged 
quite a lot from the design, the goal is to make it easier for 

Re: [Xen-devel] [PATCH v10 09/25] x86: refactor psr: L3 CAT: set value: implement framework.

[Lars Kurth]

Apologies for stepping in here. But it feels to me that this thread is at risk 
of becoming unproductive.

> On 20 Apr 2017, at 10:43, Jan Beulich  wrote:
> 
 On 20.04.17 at 04:14,  wrote:
>> On 17-04-19 03:00:29, Jan Beulich wrote:
>> On 19.04.17 at 10:22,  wrote:
 On 17-04-18 05:46:43, Jan Beulich wrote:
 On 18.04.17 at 12:55,  wrote:
>> I made a test patch based on v10 and attached it in mail. Could you 
>> please
>> help to check it? Thanks!
> 
> This looks reasonable at the first glance, albeit I continue to be
> unconvinced that this is the only (reasonable) way of solving the
>> 
>> Do you have any other suggestion on this? Thanks!
> 
> I'm sorry, but no. I'm already spending _much_ more time on this
> series than I should be, considering its (afaic) relatively low
> priority. I really have to ask you to properly think through both
> the data layout and code structure, including considering
> alternatives especially in places where you can _anticipate_
> controversy.

Jan, can you confirm whether you are happy with the current proposal. You say 
"this looks reasonable at the first glance", but then go on to say that there 
may be other "reasonable ways" of solving the problem at hand.

This is not very concrete and hard to respond to from a contributors point of 
view: it would be good to establish whether a) the v10 proposal in this area is 
good enough, b) whether there are any concrete improvements to be made.

You say please think through whether "you can _anticipate_ controversy", but at 
the same time you can't currently identify/think of any issues. That seems to 
suggest to me that maybe the proposal is good enough. Or that something is 
missing, which has not been articulated. Taking into account language barriers, 
more clarity would probably be helpful.

>> Per our discussion on v9 patch 05, we decided to adopt option 2. Below are
>> what we discussed. FYR.
> 
> Well, we decided option 2 is better than option 1. I'm still
> unconvinced there's not a yet better alternative.

I suppose that is the same type of argument. Aka we looked at a number of 
options, seem to have agreed one is better than the other. But there is no 
clarity as to whether in this case option 2 is good enough and what concrete 
steps would be needed to get to a better alternative.

Of course I may have missed some of the context here in some of the older 
threads and thus I may have missed some of the context. 

>> If it is not 0 which means the domain's cos id does not need restore
>> to 0, we can directly set it into ASSOC register. That can avoid
>> unnecessary lock. I will send out the test patch again to ask your
>> help to provide review comments (you said there are also 'a number
>> of cosmetics issues').
> 
> And I would hope you would try to eliminate some (if not all) yourself
> first. After all you can easily go over your patch yourself, checking
> for e.g. style violations.

I think this is fair enough.

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] Add clang-format file for Xen Hypervisor format

[Lars Kurth]

Ishani,

> On 12 Apr 2017, at 16:38, Ishani  wrote:
> 
> Hello,
> 
> I seemed to have overwritten the correct file by the output of the 
> clang-format tool. My main idea is to compare output file and correct file 
> and take their comparison. Since there are features which are partially 
> implemented, they are not bound to be same. Will it be better to refine the 
> tests into multiple files referring to features that are fully implemented, 
> partially implemented and not supported?

I think for now this is fine. This is really just a test to see how you get on 
with the community workflow and conventions.

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] "Hello Xen Project" Book.

[Lars Kurth]

> On 13 Apr 2017, at 14:56, Jason Long  wrote:
> 
> It is a good news and I hope Xen experts thinking about beginners like me. 
> Xen is great but have some problems in documenting and easy to use. I hope to 
> see this book on Xen Project website soon.
> 
> Thank you Xen team.

You are welcome. The content has essentially been transferred: see 
https://wiki.xenproject.org/wiki/Category:HelloXenProjectBook
The formatting can probably be improved in some areas, but generally seems 
fine: see 
https://wiki.xenproject.org/wiki/Book/HelloXenProject/Instructions_for_Improvement

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Announcing Xen Project 4.9 RC and Test Day Schedule

[Lars Kurth]

Dear community members, 

today we created Xen 4.9 RC1 (see 
https://lists.xenproject.org/archives/html/xen-devel/2017-04/msg01899.html)


== Subsequent RC's ==

We will release a new release candidate every week on Friday, until we declare 
a release candidate as the final candidate and cut the Xen 4.9 release. The 
projected dates for the next RC's will be

 Xen 4.8 RC2: April 21
 Xen 4.8 RC3: April 28
 Xen 4.8 RC4: May 5
 Xen 4.8 RC5: May 12

Release candidates are available from our git repository at

 git://xenbits.xenproject.org/xen.git (tag 4.9.0-)

where  is rc1, rc2, rc3, etc. and as tarball from

 https://downloads.xenproject.org/release/xen/4.9.0-/xen-4.9.0-.tar.gz 
 
https://downloads.xenproject.org/release/xen/4.9.0-/xen-4.9.0-.tar.gz.sig
 

Detailed build and Install instructions can be found on the Test Day Wiki 
(https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions#Installing). 
Make sure you check the known issues section of the instructions 
(https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions#Known_issues) 
before trying to download an RC.

Note that RC’s are announced on the following mailing lists: 
 xen-announce
 xen-devel
 xen-users 

Make sure that you are subscribed to at least one of those lists.

== Test Day schedule ==

We will also hold a Test Day every TUESDAY for the release candidate that was 
released the week prior to the Test Day starting from RC2. 

This means we will have Test Days on 
 April 25th 
 May 2nd
 May 9th 
 May 16th

Your testing is still valuable on other days, so please feel free to send Test 
Reports as outlined below at any time.

== Testing new Features, Test and Bug Reports ==

You can find Test Instructions for new features on our Test Day Wiki 
(https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions) and 
instructions for general tests on Testing Xen 
(https://wiki.xenproject.org/wiki/Testing_Xen). The following pages provide 
information on how to report successful tests 
(https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions#Reporting_success)
 and how to report bugs and issues 
(https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions#Reporting_Bugs_.28.26_Issues.29).

Happy Testing!

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.9 RC1

[Lars Kurth]

Hi all,
I created https://wiki.xenproject.org/wiki/Xen_4.9_RC_test_instructions - this 
needs checking and addition of specific stuff that you want people to test
I updated https://wiki.xenproject.org/wiki/Xen_Project_Test_Days
Regards
Lars

> On 13 Apr 2017, at 11:38, Julien Grall  wrote:
> 
> Hi all,
> 
> Xen 4.9 RC1 is tagged. You can check that out from xen.git:
> 
>  git://xenbits.xen.org/xen.git 4.9.0-rc1.2
> 
> For your convenience there is also a tarball at:
> https://downloads.xenproject.org/release/xen/4.9.0-rc1.2/xen-4.9.0-rc1.2.tar.gz
>   
> 
> And the signature is at:
> https://downloads.xenproject.org/release/xen/4.9.0-rc1.2/xen-4.9.0-rc1.2.tar.gz.sig
> 
> Please send bug reports and test reports to
> xen-de...@lists.xenproject.org. When sending bug reports, please CC
> relevant maintainers and me (julien.gr...@arm.com).
> 
> Thanks,
> 
> -- 
> Julien Grall
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] "Hello Xen Project" Book.

[Lars Kurth]

Hi all,

I had a go at converting the first chapter
* See https://wiki.xenproject.org/wiki/Category:HelloXenProjectBook 
<https://wiki.xenproject.org/wiki/Category:HelloXenProjectBook> (and 
https://wiki.xenproject.org/wiki/Book/HelloXenProject/Instructions_for_Conversion
 
<https://wiki.xenproject.org/wiki/Book/HelloXenProject/Instructions_for_Conversion>)
* And https://wiki.xenproject.org/wiki/Book/HelloXenProject/0-Contents 
<https://wiki.xenproject.org/wiki/Book/HelloXenProject/0-Contents>

All relevant information for people to help out is there. If anyone wants to 
help and needs some advice, feel free to do so

Best Regards
Lars

> On 4 Apr 2017, at 15:25, Mohsen  wrote:
> 
> Thank you for openSUSE VM and the libreoffice-wiki-publisher but you must 
> split the .odt file? 
> 
> 
> On Monday, April 3, 2017 8:38 AM, Lars Kurth  wrote:
> 
> 
> 
> > On 29 Mar 2017, at 05:51, Juergen Gross  > <mailto:jgr...@suse.com>> wrote:
> > 
> > On 28/03/17 21:33, Mike Wright wrote:
> >> On 03/28/2017 12:11 PM, Mohsen wrote:
> >>> I'm using LibreOffice 4.3.3.2 on Debian amd64 and this version not
> >>> have MediaWiki export function!!
> >> 
> >> There was a deb for the libreoffice extension
> >> libreoffice-wiki-publisher.  Give that a try.
> > 
> > In the end you need that only once for the initial conversion. So
> > instead of trying to find the correct package you could just use
> > your Xen skills to create an openSUSE VM and do it there. :-)
> 
> Juergen, thanks for the tip. I installed an openSUSE VM and the 
> libreoffice-wiki-publisher came as default, which is good. So I ran a few 
> tests.
> 
> First, what I couldn't get to work: I tried Send > MediaWiki in the hope that 
> this would allow transferring of images, but could not get it to work. There 
> seems to be an issue with authentication on the XenProject wiki side. But 
> File > Export [MediaWiki (.txt)] works. 
> 
> Also, I couldn't find any docs for the converter: the help links to pages 
> which do not exist. But hey, we can live with that.
> 
> Here is what I learned:
> ===
> * The export granularity is 1 LibreOffice document to 1 Wiki Page
> * Most of the basic formatting such as lists and tables get correctly 
> converted, but the converter introduces an awful lot of  style"...">... and ... attributes. Basically it 
> does this every time, something slightly out of the ordinary has been done 
> with text. These may have to be stripped with on-line tools such as 
> http://www.unit-conversion.info/texttools/strip-tags/  
> <http://www.unit-conversion.info/texttools/strip-tags/>or similar, otherwise 
> the wiki pages become a nightmare to edit in future.
> * URLs are correctly converted
> * Headlines (aka text marked as "Heading 1", "Heading 2", etc. are not 
> converted) to = ... =, == ... ==, etc.
> * Images are not converted: when an image is found, "[[image:|top]]" is 
> inserted
> * I don't know how code snippets will come across in terms of formatting, as 
> I don't have the ODT source of the book
> 
> What does this mean:
> 
> In principle, this means that should be doable with 1-2 days worth of work. 
> However it's not going to be entirely trivial. What we would need to do is to:
> * Break the original book ODT file into smaller sections (probably along the 
> chapter structure as exposed in the Contents)
> * Then take each of the ODT files and do the following
> ** Save as MediaWiki (.txt) [1]. If appropriate remove tags using 
> http://www.unit-conversion.info/texttools/strip-tags/ 
> <http://www.unit-conversion.info/texttools/strip-tags/>
> ** Save as html [2] - to get the images. The bad news is that the images are 
> saved using some hash names and not in the order they are in the document
> ** Create the wiki page from [1]
> ** Fix up bad formatting (such as missing = ... =, == ... ==, etc.)
> ** Manually upload the images from [2]
> ** Add appropriate [[Category:...]] tags at the bottom of each page. At least 
> one for the wiki-book, e.g. [[Category:HelloXenProjectBook]] or something 
> similar. But of course further categories per topic can be added as needed.
> 
> Once we have all the content, create the common pages such as contents, 
> credits, etc. - and we should have a good starting point.
> 
> Best Regards
> Lars
> 
>   
> 
> 
> 
> 
> 
> 
> ___
> Xen-users mailing list
> xen-us...@lists.xen.org <mailto:xen-us...@lists.xen.org>
> https://lists.xen.org/xen-users <https://lists.xen.org/xen-users>
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] Add clang-format file for Xen Hypervisor format

[Lars Kurth]

> On 12 Apr 2017, at 12:51, Ishani  wrote:
> 
> Hello,
> 
> Thanks for the comments.
> I have created a markdown file for clang-format doc which I will add. 
> 
>> If so, you want to include this information in the cover letter (see above). 
>> Or you >could write little bash script which shows how this works
> 
> Can you elaborate on the task of bash script? Should it give the difference 
> between files?

Please don't top-post (see 
https://en.wikipedia.org/wiki/Posting_style#Top-posting) but use 
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style instead
Top-posting makes it really hard to follow a review on a mailing list. In this 
case I had to go down into the e-mail thread and identify the context of "If 
so, you want to include this information in the cover letter ..."

I added the reply in the right place below.

> Shall I create a new patch by incorporating the suggested changes?

Yes, basically, you want to follow 
https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Review.2C_Rinse_.26_Repeat

> Regards,
> Ishani
> 
> - Original Message -
> From: "Lars Kurth" 
> To: "Ishani Chugh" 
> Cc: "xen-devel" , "cardoe" 
> Sent: Wednesday, April 12, 2017 4:56:53 PM
> Subject: Re: [PATCH] Add clang-format file for Xen Hypervisor format
> 
> Hi Ishani,
> 
>> On 12 Apr 2017, at 11:49, Ishani Chugh  
>> wrote:
> 
> Here you would normally add a short description of the patch (also called 
> cover letter). In this case, you probably want to describe:
> a) What you have done
> b) Link to 
> https://wiki.xenproject.org/wiki/Outreach_Program_Projects#Code_Standards_Checking_using_clang-format,
>  
> https://docs.google.com/document/d/1tCcwZ9K38ToLGTPHZkfs2sS4s4YulrGoH8LIHwBMbg4/edit
>  and 
> https://docs.google.com/document/d/10NJn-QvO1TvyJJJGE2PD6FtElYCT3neBAffIqeWHdiE/edit
> c) You can also link to your github repo
> d) You may want to add the two command lines or some instructions showing how 
> to use the patch
> 
>> Signed-off-by: Ishani Chugh 
>> ---
>> tools/clang-format/tests/.clang-format | 88 
>> ++
>> tools/clang-format/tests/correct.c | 73 
>> tools/clang-format/tests/test.c| 69 ++
>> 3 files changed, 230 insertions(+)
>> create mode 100644 tools/clang-format/tests/.clang-format
>> create mode 100644 tools/clang-format/tests/correct.c
>> create mode 100644 tools/clang-format/tests/test.c
>> 
>> diff --git a/tools/clang-format/tests/.clang-format 
>> b/tools/clang-format/tests/.clang-format
>> new file mode 100644
>> index 000..2229910
>> --- /dev/null
>> +++ b/tools/clang-format/tests/.clang-format
>> @@ -0,0 +1,88 @@
>> +---
>> +Language:Cpp
>> +# BasedOnStyle:  LLVM
> 
> This would be a good place to list in a comment, which coding style patterns 
> the file covers. In other words, everything that has a yes in 
> https://docs.google.com/document/d/1tCcwZ9K38ToLGTPHZkfs2sS4s4YulrGoH8LIHwBMbg4/edit
> 
> The other thing, which is a little unclear is whether the file applies to 
> - http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=CODING_STYLE only
> - http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/CODING_STYLE 
> only
> - or both
> 
> I think it is OK to just cover one file.

...

>> diff --git a/tools/clang-format/tests/test.c 
>> b/tools/clang-format/tests/test.c
>> new file mode 100644
>> index 000..2e7eaf2
>> --- /dev/null
>> +++ b/tools/clang-format/tests/test.c
> 
> How is this file different from correct.c? 
> It looks to me that test.c is a wrongly formatted version of correct.c and 
> when you pass it into clang-format it is re-formatted to match correct.c
> 
> If so, you want to include this information in the cover letter (see above). 
> Or you could write little bash script which shows how this works

But to answer your question: this was just a suggestion to save you explaining 
in text what each file does. If I assume correctly, something like

#!/bin/sh
#
clang-format -i -style=.clang-format test.c > _test.c
if cmp -s _test.c correct.c"
then
   echo "test.c was transformed correctly"
else
   echo "test.c was transformed incorrectly"
fi

should explain what the two .c files do and are intended for. Note that I 
didn't test this (I am not familiar with the clang-format syntax). 

Cheers
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] Add clang-format file for Xen Hypervisor format

[Lars Kurth]

Hi Ishani,

> On 12 Apr 2017, at 11:49, Ishani Chugh  
> wrote:

Here you would normally add a short description of the patch (also called cover 
letter). In this case, you probably want to describe:
a) What you have done
b) Link to 
https://wiki.xenproject.org/wiki/Outreach_Program_Projects#Code_Standards_Checking_using_clang-format,
 
https://docs.google.com/document/d/1tCcwZ9K38ToLGTPHZkfs2sS4s4YulrGoH8LIHwBMbg4/edit
 and 
https://docs.google.com/document/d/10NJn-QvO1TvyJJJGE2PD6FtElYCT3neBAffIqeWHdiE/edit
c) You can also link to your github repo
d) You may want to add the two command lines or some instructions showing how 
to use the patch

> Signed-off-by: Ishani Chugh 
> ---
> tools/clang-format/tests/.clang-format | 88 ++
> tools/clang-format/tests/correct.c | 73 
> tools/clang-format/tests/test.c| 69 ++
> 3 files changed, 230 insertions(+)
> create mode 100644 tools/clang-format/tests/.clang-format
> create mode 100644 tools/clang-format/tests/correct.c
> create mode 100644 tools/clang-format/tests/test.c
> 
> diff --git a/tools/clang-format/tests/.clang-format 
> b/tools/clang-format/tests/.clang-format
> new file mode 100644
> index 000..2229910
> --- /dev/null
> +++ b/tools/clang-format/tests/.clang-format
> @@ -0,0 +1,88 @@
> +---
> +Language:Cpp
> +# BasedOnStyle:  LLVM

This would be a good place to list in a comment, which coding style patterns 
the file covers. In other words, everything that has a yes in 
https://docs.google.com/document/d/1tCcwZ9K38ToLGTPHZkfs2sS4s4YulrGoH8LIHwBMbg4/edit

The other thing, which is a little unclear is whether the file applies to 
- http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=CODING_STYLE only
- http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/CODING_STYLE 
only
- or both

I think it is OK to just cover one file.

> +AccessModifierOffset: 0
> +AlignAfterOpenBracket: Align
> +AlignConsecutiveAssignments: false
> +AlignConsecutiveDeclarations: false
> +AlignEscapedNewlinesLeft: true
> +AlignOperands:   true
> +AlignTrailingComments: true
> +AllowAllParametersOfDeclarationOnNextLine: true
> +AllowShortBlocksOnASingleLine: false
> +AllowShortCaseLabelsOnASingleLine: false
> +AllowShortFunctionsOnASingleLine: All
> +AllowShortIfStatementsOnASingleLine: false
> +AllowShortLoopsOnASingleLine: false
> +AlwaysBreakAfterDefinitionReturnType: None
> +AlwaysBreakAfterReturnType: None
> +AlwaysBreakBeforeMultilineStrings: false
> +AlwaysBreakTemplateDeclarations: true
> +BinPackArguments: false
> +BinPackParameters: false
> +BraceWrapping:   
> +  AfterClass:  true
> +  AfterControlStatement: true
> +  AfterEnum:   true
> +  AfterFunction:   true
> +  AfterNamespace:  true
> +  AfterObjCDeclaration: true
> +  AfterStruct: true
> +  AfterUnion:  true
> +  BeforeCatch: true
> +  BeforeElse:  true
> +  IndentBraces:false
> +BreakBeforeBinaryOperators: None
> +BreakBeforeBraces: Custom
> +BreakBeforeTernaryOperators: true
> +BreakConstructorInitializersBeforeComma: false
> +ColumnLimit: 80
> +CommentPragmas:  '^ IWYU pragma:'
> +ContinuationIndentWidth: 4
> +Cpp11BracedListStyle: true
> +DerivePointerAlignment: false
> +DisableFormat:   false
> +ExperimentalAutoDetectBinPacking: false
> +ForEachMacros:   [ foreach, Q_FOREACH, BOOST_FOREACH ]
> +IncludeCategories: 
> +  - Regex:   '^"(llvm|llvm-c|clang|clang-c)/'
> +Priority:2
> +  - Regex:   '^(<|"(gtest|isl|json)/)'
> +Priority:3
> +  - Regex:   '.*'
> +Priority:1
> +IndentCaseLabels: false
> +IndentWidth: 4
> +IndentWrappedFunctionNames: false
> +KeepEmptyLinesAtTheStartOfBlocks: false
> +MacroBlockBegin: ''
> +MacroBlockEnd:   ''
> +MaxEmptyLinesToKeep: 1
> +NamespaceIndentation: None
> +ObjCBlockIndentWidth: 2
> +ObjCSpaceAfterProperty: false
> +ObjCSpaceBeforeProtocolList: true
> +PenaltyBreakBeforeFirstCallParameter: 19
> +PenaltyBreakComment: 300
> +PenaltyBreakFirstLessLess: 120
> +PenaltyBreakString: 1000
> +PenaltyExcessCharacter: 100
> +PenaltyReturnTypeOnItsOwnLine: 60
> +PointerAlignment: Left
> +ReflowComments:  true
> +SortIncludes:false
> +SpaceAfterCStyleCast: false
> +SpaceBeforeAssignmentOperators: true
> +SpaceBeforeParens: ControlStatements
> +SpaceInEmptyParentheses: false
> +SpacesBeforeTrailingComments: 1
> +SpacesInAngles:  false
> +SpacesInContainerLiterals: true
> +SpacesInCStyleCastParentheses: true
> +SpacesInParentheses: true
> +SpacesInSquareBrackets: false
> +Standard:Cpp11
> +TabWidth:4
> +UseTab:  Never
> +...
> +
> diff --git a/tools/clang-format/tests/correct.c 
> b/tools/clang-format/tests/correct.c
> new file mode 100644
> index 000..882d50a
> --- /dev/null
> +++ b/tools/clang-format/tests/correct.c
> @@ -0,0 +1,73 @@

I think it would be extremely useful, to add a reference in a comment to the 
coding standard document for each tes

Re: [Xen-devel] [Outreachy] Interested in contribution: Code Standards Checking using clang-format

[Lars Kurth]

Ishani,

> On 11 Apr 2017, at 12:21, Ishani  wrote:
> 
> Hello,
> 
> I have created a clang-format file for specifications of format for Xen 
> hypervisor and incorporated all of the support which present clang-format 
> could provide. I have seen all the options provided by clang-format and have 
> some discrepancies regarding some of format specifications which are not 
> provided in the doc but is followed. I will commit the code and provide you 
> with full report by today.

Can you post this on any public repo for now (e.g. github gitlab bitbucket, 
whatever) and post an e-mail on this list with title "[RFC] Code Standards 
Checking using clang-format" or something like it with link to the repo. I 
think this is enough for the small task required for this project.

Ideally, because at some point you will need to do this, you would follow 
https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches - but we could 
waive this in this case. But it will make reviewing the output easier.

>> 4: a small script that checks that .clang-format works correctly on these 
>> pieces of code
> 
>> For 4, you probably want to run clang-format with -output-replacements-xml 
>> on the files in 1 >- 3 and then do some grep magic to see whether it does 
>> the right thing.
> 
> Can you elaborate on this a bit. What is the expected end product?

I think this is a stretch goal, given the time-frame. The idea here was to be 
able to verify that the clang-format file works as expected on some code 
snippets that we know adhere to the coding standards. This would in essence 
produce a stand-alone test for the .clang-format file on the coding style 
patterns that we know works. 

> Meanwhile, I have resumed on writing the proposal. I somehow missed previous 
> comments. Please have a look at it.
> https://docs.google.com/document/d/10NJn-QvO1TvyJJJGE2PD6FtElYCT3neBAffIqeWHdiE/edit?usp=sharing

I saw it. Thank you!

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Outreachy] Doubts regarding setup, git version and selection of micro task

[Lars Kurth]

On 05/04/2017 19:19, "Stefano Stabellini"  wrote:

>On Wed, 5 Apr 2017, Julien Grall wrote:
>> (CC Lars)
>> 
>> Hi,
>> 
>> On 05/04/17 12:51, Wei Liu wrote:
>> > On Wed, Apr 05, 2017 at 05:04:00PM +0530, Tejaswini Poluri wrote:
>> > > Hi Stefano and Julien,
>> > > 
>> > > This is Tejaswini. I had been working as a Senior Software
>>developer in
>> > > Samsung and Cavium networks in linux kernel domain previously. I
>>took a
>> > > break for one year for personal reasons. I am looking forward to
>>restart
>> > > my
>> > > career as a Linux engineer again with outreachy 2017 internships. I
>>have
>> > > experience of working with device trees and virtual filesystems.
>> 
>> Thank you for your interest on the project.
>> 
>> I think the deadline for Outreachy has been extended to 13th of April.
>>You
>> would need to fill out an application (see
>> https://wiki.xenproject.org/wiki/Outreachy/Apply).
>
>Indeed. Also see the following page (it says GSoC at the top but applies
>to Outreachy too):
>
>https://wiki.xenproject.org/wiki/GSoC_Student_Application_Template
>
>Please add as many details as you can to the "Implementation Plan".

I changed the page name to make it program independent: I must have missed
this. The old link will still work. I also added a mapping between
sections in 
https://wiki.xenproject.org/wiki/Internship_Application_Template and the
Outreachy application form.

Regards
Lars

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] How to migrate vCPUs based on Credit Scheduler

[Lars Kurth]

Adding George, Dario & Anshul
Lars

> On 3 Apr 2017, at 17:21, 甘清甜  wrote:
> 
> Hi,
> 
>  I'm now designing new vCPU scheduler in Xen, and trying to implement 
> the scheduler based on the Credit scheduler in Xen-4.5.1. But I encountered
>  come problems when debuging the code.
> 
> Most of the code modification is done in function csched_schedule() in 
> file: xen/common/csched_schedule.c . And the core code is as followed:
> 
> if( vcpu_runnable(current) )
> {
> if( match the migration contition )
> {
> cpu_affinity = pick_pcpu_runq();  // this function is defined by 
> myself
> 
> pcpulock = pcpu_schedule_lock_irqsave(cpu_affinity, 
>&pcpulock_flag);
> 
> TRACE_3D(TRC_CSCHED_STOLEN_VCPU, cpu_affinity  , domain_id,
>vcpu_id);
> SCHED_VCPU_STAT_CRANK(scurr, migrate_q);
> SCHED_STAT_CRANK(migrate_queued);
> WARN_ON(scurr->vcpu->is_urgent);
>   scurr->vcpu->processor = cpu_affinity;  
> 
>  __runq_insert(cpu_affinity, scurr);
> pcpu_schedule_unlock_irqrestore(pcpulock, pcpulock_flag, 
> cpu_affinity  );
> }
> else 
> __runq_insert(cpu, scurr);
> }
> else 
> BUG_ON( is_idle_vcpu(current) || list_empty(runq) );
> 
> 
> I try to run the modified Xen. But according to the log I found that, 
> although I insert the vCPU into the runqueue  of another pCPU, the 
> vCPU still appears at the old pCPU in the following scheduling period. 
> Now I have a few questions:
> 
> 1. Does the Xen scheduler framework support changing the pCPU of a 
> vCPU after using out the scheduling time slice, but not just to steal one 
> vCPU from runqueue of other pCPU in load_balance period?
> 
> 2. If yes, what status of the vCPU should be changed before inserting 
> the vCPU into the destination pCPU?
> 
> Thank you very much!
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Outreachy] Interested in contribution: Code Standards Checking using clang-format

[Lars Kurth]

Hi Ishani,

> On 3 Apr 2017, at 13:01, Ishani  wrote:
> 
> Thanks for the reply. I started with looking into format of Xen Hypervisor 
> files and creating a clang format file for the same. However, given the 
> present implementation of clang-format, it is not possible to incorporate 
> fine-grained custom coding format requirements. For example, although 
> clang-format provides a custom brace wrapping format specification, it is 
> still not sufficient to incorporate the do while loop exception for braces in 
> Xen Hypervisor format.

OK, this makes this project a lot harder. I don't think we are particularly 
tied to clang-format. 

> The implementations is presently on higher layers of abstraction and needs to 
> be fine grained to allow for different types of loops as well. I figured that 
> there are two ways of doing it, First , by creating a wrapper over 
> clang-format and using regexes to find appropriate partitions of code to 
> follow different format styles.
> However, this will be rather cumbersome with multiple heuristics and we will 
> lose the advantage of clang-format tool using the inbuilt parser.

Indeed.

> Second way is to extend the functionality of clang-format. I have experience 
> working with compilers. Going through the code of clang it seems possible to 
> include kw_do token for do while loop as a functionality for braces wrapping( 
> although it would require a very thorough reading through the codebase 
> first). 

One of the objectives behind using clang was that we wanted to be able to use a 
compiler front-end that has a plug-in model or a library which allows access to 
the AST. Clang-format appeared to fit the bill. 

> At present, I am working on creating a clang format file supporting subset of 
> coding formats for Xen Hypervisor.

I think that would fit with what I suggested.

> Can you provide me with some more insight into the project?

There are really two objectives:

A) A tool which can be run over submissions by new contributors. Right now, 
adherence to coding standards has to be manually checked. Having a tool which 
lists coding standard violations in a git-patch or in the files a patch touches 
would allow contributors to run the tool before submitting the code. Ideally, 
it could be run over a patch, but that may be harder than running it over the 
files the patch touches. In any case, it would improve the work-flow.

B) Be able to run the tool on the entire code base.

We also want to make this as easy to maintain in future as possible, which is 
why we originally suggested clang-format. It might in fact be easier or cleaner 
to use LibTooling rather than clang-format for this as base-line and use 
clang-format as an inspiration (looking at the source of clang-format, there 
does not seem to be that much).  Keeping as close as possibly to the 
clang-format config file. The downside is that we would have to maintain such a 
tool in the long-run (but then we would have to do the same if we used a 
stand-alone tool using script hackery). I don't think either Doug, Andy, or me 
could give you concrete advice on what the best way forward is here without 
further discussion though.

> clang-format formats the tool to specified format. However, there are many 
> standards which include a particular format for a variable name. It is not 
> feasible for a tool to automatically change the name of variable according to 
> given standards because of possible breaking build of the code.

That is not the intention: it is definitely not desirable to do that kind of 
refactoring automatically. 

> As a part of the project, are we required to create an auto-formatter or 
> something like pylint which suggests the necessary changes and gives an 
> overall score, or a combination of both?

We don't need an auto-formatter. Just a tool - such a pylint - which runs on 
our codebase and makes suggestions on how the code should look like. This 
should make things easier. 

> I wish to apply to GSOC as well for the same project. However, since the 
> deadline is today, I will be grateful if you could provide some suggestions 
> for the project.

I see you submitted already

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] "Hello Xen Project" Book.

[Lars Kurth]

> On 29 Mar 2017, at 05:51, Juergen Gross  wrote:
> 
> On 28/03/17 21:33, Mike Wright wrote:
>> On 03/28/2017 12:11 PM, Mohsen wrote:
>>> I'm using LibreOffice 4.3.3.2 on Debian amd64 and this version not
>>> have MediaWiki export function!!
>> 
>> There was a deb for the libreoffice extension
>> libreoffice-wiki-publisher.  Give that a try.
> 
> In the end you need that only once for the initial conversion. So
> instead of trying to find the correct package you could just use
> your Xen skills to create an openSUSE VM and do it there. :-)

Juergen, thanks for the tip. I installed an openSUSE VM and the 
libreoffice-wiki-publisher came as default, which is good. So I ran a few tests.

First, what I couldn't get to work: I tried Send > MediaWiki in the hope that 
this would allow transferring of images, but could not get it to work. There 
seems to be an issue with authentication on the XenProject wiki side. But File 
> Export [MediaWiki (.txt)] works. 

Also, I couldn't find any docs for the converter: the help links to pages which 
do not exist. But hey, we can live with that.

Here is what I learned:
===
* The export granularity is 1 LibreOffice document to 1 Wiki Page
* Most of the basic formatting such as lists and tables get correctly 
converted, but the converter introduces an awful lot of ... and ... attributes. Basically it 
does this every time, something slightly out of the ordinary has been done with 
text. These may have to be stripped with on-line tools such as 
http://www.unit-conversion.info/texttools/strip-tags/ or similar, otherwise the 
wiki pages become a nightmare to edit in future.
* URLs are correctly converted
* Headlines (aka text marked as "Heading 1", "Heading 2", etc. are not 
converted) to = ... =, == ... ==, etc.
* Images are not converted: when an image is found, "[[image:|top]]" is inserted
* I don't know how code snippets will come across in terms of formatting, as I 
don't have the ODT source of the book

What does this mean:

In principle, this means that should be doable with 1-2 days worth of work. 
However it's not going to be entirely trivial. What we would need to do is to:
* Break the original book ODT file into smaller sections (probably along the 
chapter structure as exposed in the Contents)
* Then take each of the ODT files and do the following
** Save as MediaWiki (.txt) [1]. If appropriate remove tags using 
http://www.unit-conversion.info/texttools/strip-tags/
** Save as html [2] - to get the images. The bad news is that the images are 
saved using some hash names and not in the order they are in the document
** Create the wiki page from [1]
** Fix up bad formatting (such as missing = ... =, == ... ==, etc.)
** Manually upload the images from [2]
** Add appropriate [[Category:...]] tags at the bottom of each page. At least 
one for the wiki-book, e.g. [[Category:HelloXenProjectBook]] or something 
similar. But of course further categories per topic can be added as needed.

Once we have all the content, create the common pages such as contents, 
credits, etc. - and we should have a good starting point.

Best Regards
Lars
  

 




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Outreachy] Interested in contribution: Code Standards Checking using clang-format

[Lars Kurth]

Hi Ishani, (+Doug, +Andy)

not that the deadline as been extended to April 13. I am also CC'ing Andy 
Cooper as Doug has not responded. 

The challenge that we will have is to satisfy the small task requirement while 
also making the task sensible in the context of the project. Now we do have 
some flexibility there: in other words, we can declare for example an RFC that 
everyone agrees with as such a contribution.

But as a basic idea, the first step of the project would be to try and come up 
with a .clang-format file that matches one of the coding styles in the git tree 
(either http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=CODING_STYLE or 
http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/CODING_STYLE) and 
prove that it works. I don't know whether this is doable as a small task: the 
number of options in clang-format do not seem to be huge. And there are tools 
such as https://clangformat.com/ to experiment with.

If it is possible to do the whole thing in a small task, the output would be a 
.clang-format file that is submitted for code review to a suitable location in 
the xen tree. I don't know what the right location should be, but 
xen.git:tools/clang-format would be sensible. 

If it is not possible to do this for one of the complete coding standards, I 
would develop for one or two subsets: 
1: a .clang-format settings snippet for a reasonably complex coding style 
pattern in the coding standard
2: a piece of test code that adheres to the coding standard 
3: a variant of the same code snippet that does not adhere to the coding 
standard 
4: a small script that checks that .clang-format works correctly on these 
pieces of code 

For 4, you probably want to run clang-format with -output-replacements-xml on 
the files in 1 - 3 and then do some grep magic to see whether it does the right 
thing.

Again this could go into xen.git:tools/clang-format - maybe under 
tools/clang-format/tests for now

@Andy, @Doug: does this sound sensible?

Regards
Lars


> On 31 Mar 2017, at 10:37, Ishani  wrote:
> 
> Hello Sir,
> 
> I am Ishani Chugh, a fourth year undergraduate student of Electronics and 
> Communications Engineering at International Institute of Information 
> Technology, Hyderabad. I am passionate about Open Source Development. I am 
> interested to contribute to your organization Xen Project as an Outreachy 
> intern . I am interested to work in the project: Code Standards Checking 
> using clang-format. I am proficient in C++ and have slight experience of 
> working in clang-format.
> Please guide me for initial contributions to prove my candidature.
> 
> Regards
> Ishani
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Outreachy project - Xen Code Review Dashboard

[Lars Kurth]

Hi,

> On 31 Mar 2017, at 02:48, Vaishnavi Ramesh Jayaraman 
>  wrote:
> 
> Hi,
> I am Vaishnavi, interested in contributing to the Xen Project as part of the 
> Outreachy Program. I am particularly interested in working on the Xen Code 
> Review Dashboard.
> 
> I have worked on the ElasticSearch - Logstash- Kibana (ELK) stack previously 
> and am comfortable with Javascript.
> 
> It would be great if you could give me pointers on how to get started!

You may want to look at http://markmail.org/message/7adkmords3imkswd 


Regards
Lars___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Outreachy Project

[Lars Kurth]

Nazira,
please check https://wiki.xenproject.org/wiki/2017-Summer-Internships 
 and then 
https://wiki.xenproject.org/wiki/Outreach_Program_Projects 
, which contains a 
list of possible project including their mentors. Please also check the 
Outreachy Eligibility requirements on https://wiki.gnome.org/Outreachy 
. If in doubt, please ask 
outreachy-l...@gnome.org   and CC me.
Best Regards
Lars

> On 31 Mar 2017, at 09:45, Nakka Ricci  wrote:
> 
> Hello! 
> 
> My name is Nazira and I am interested in the project called "Xen Hypervisor". 
> This mail is displayed as a contact information, so I wanted to ask how to 
> connect to a mentor of the project? 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] IMPORTANT INFO FOR MENTORS: Outreachy and GSoc deadlines (Outreachy March 30, GSoC April 4) + Next Steps

[Lars Kurth]

Added Thomas to the CC list
Lars
 
> On 28 Mar 2017, at 16:31, Mindy  wrote:
> 
> On 03/28/2017 10:29 AM, Lars Kurth wrote:
>> Hi everyone,
>> 
>> on the TO list: mailing lists where people have submitted queries and/or 
>> have projects and program coordinators
>> on the CC list: possible mentors I have seen engaging with applicants (for 
>> the Hypervisor I also added committers@)
>> 
>> @Mindy: could you please look at the Mirage OS section below and check 
>> whether I missed anyone. Also, can you confirm the mentor for the item 
>> marked in red below.
>> 
>> 
>> Mirage OS
>> * No Outreachy applicants : is this correct?
>> * GSoC: Mentor: Thomas Gazagnaire?; Applicant: David Udelson 
>> * GSoC: Mentor: Christiano F. Haesbaert, Applicant: Harshit Daga
>> * GSoC: Mentor: Rudi Grinberg and/or Nik Sultana , Applicant: Haiyu Yang 
> 
> It's correct that we have no Outreachy applicants this term.
> 
> Thomas Gazagnaire confirmed his availability in 
> https://github.com/mirage/irmin/issues/415 
> <https://github.com/mirage/irmin/issues/415> .
> 
> Thanks,
> Mindy

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] IMPORTANT INFO FOR MENTORS: Outreachy and GSoc deadlines (Outreachy March 30, GSoC April 4) + Next Steps

[Lars Kurth]

Hi everyone,

on the TO list: mailing lists where people have submitted queries and/or have 
projects and program coordinators
on the CC list: possible mentors I have seen engaging with applicants (for the 
Hypervisor I also added committers@)

@Mindy: could you please look at the Mirage OS section below and check whether 
I missed anyone. Also, can you confirm the mentor for the item marked in red 
below.

If I missed anyone, please reply and add them in the CC list. The lists are 
established by searching for GSoC, Outreachy in the mailing list archives. I 
would have missed mentors / applicants, which did not use these keywords.

A quick reminder that the application deadlines are coming up. Note that they 
are different for GSoC and Outreachy
* March 30: Application deadline for Outreachy (but applicants can edit 
applications submitted until April 28th and can still work on the required 
micro-tasks until the 28th) 
* April 3: Application deadline for GSoC. This is a hard deadline

== IMPORTANT ACTIONS/INFO for MENTORS ==
* Please remind possible applicants of these deadlines: the application system 
for Outreachy is at https://outreachy.gnome.org <https://outreachy.gnome.org/>, 
for GSoC at https://summerofcode.withgoogle.com/get-started/ 
<https://summerofcode.withgoogle.com/get-started/>
* Also we will only allow each MENTOR to mentor one applicant. If you have 
several, you should direct them to another project or ask them to make two 
submissions, such that they don't loose out. In situations where we have two 
people applying for the same project, we will require that you choose the best 
applicant.

This next step is important: we need to have mentors assigned to proposal to 
get them processed
* Outreachy: 
  1) Create an account on https://outreachy.gnome.org 
<https://outreachy.gnome.org/?q=view_projects&prg=8&a=apply&c=mentor>, 
  2) Sign up as mentor at 
https://outreachy.gnome.org/?q=view_projects&prg=8&a=apply&c=mentor 
<https://outreachy.gnome.org/?q=view_projects&prg=8&a=apply&c=mentor>, choose 
Xen Project

* GSoC: 
  1) I will invite everyone on the CC list as mentor to make things easier for 
you. If you know your applicant will NOT apply for GSoC, but Outreachy only, 
ignore the request. Otherwise accept.

Outreachy or GSoC?
This Outreachy round coincides with Google Summer of Code, and we have several 
candidates which may qualify for both. Please encourage any applicants you know 
are students to apply for GSoC. When you encourage them to apply, please let 
them know that GSoC stipends are now variable based on location, so they may 
get a lower GSoC stipend than the fixed Outreachy flat stipend of $5,500. 
Please make sure applicants aren't surprised by this change by pointing them to 
https://wiki.gnome.org/Outreachy#Is_Google_Summer_of_Code_right_for_you 
<https://wiki.gnome.org/Outreachy#Is_Google_Summer_of_Code_right_for_you>

GSoC risks
Note that we will be assigned a number of GSoC slots based on the number of 
applicants and mentors. We will not know until sometimes after April 4th, how 
many slots we will get assigned. This means that good applicants 

Too few Outreachy applicants?
I don't quite know whether we have anyone who will apply for Outreachy. Can 
those, who have been dealing with potential Outreachy applicants raise their 
hands by replying to this section!

== NEXT STEPS ==
I am aiming to set up a meeting amongst all mentors on APRIL 11 and APRIL 18 to 
assess the candidates. I intend to choose a 15:00 time-slot for each.
If you can't make it please scream.

== APPLICATIONS IN THE APPLICATION SYSTEM SO FAR ==
* GSoC: 3
* Outreachy: 0

== Conversations on lists regarding Outreachy / GSoC ==

Mirage OS
* No Outreachy applicants : is this correct?
* GSoC: Mentor: Thomas Gazagnaire?; Applicant: David Udelson 
* GSoC: Mentor: Christiano F. Haesbaert, Applicant: Harshit Daga
* GSoC: Mentor: Rudi Grinberg and/or Nik Sultana , Applicant: Haiyu Yang 

Infra
* GSoC (or Outreachy - may not qualify; TBD), Mentor: Jesus M. 
Gonzalez-Barahona & Lars Kurth, Applicant: Gayathri Menakath 

Hypervisor Team (including MiniOS)
* No Outreachy applicants : is this correct?
* GSoC: Mentor: Stefano Stabellini or Julien Grall, Applicant: Zhongze Liu
* GSoC: Mentor: Stefano Stabellini or Julien Grall, Applicant: Luca Miccio
* GSoC: Mentor: Stefano Stabellini or Julien Grall, Applicant: Methuku Karthik 
* GSoC: Mentor: Wei Liu, Applicant: Felix Schmoll (2 separate projects) 
* GSoC: Mentor: Wei Liu or Doug Goldstein (has not responded to applicant), 
Applicant: Saurav Sachidanand 

Best Regards
Lars



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] "Hello Xen Project" Book.

[Lars Kurth]

> On 16 Mar 2017, at 05:00, Juergen Gross  wrote:
> 
> On 15/03/17 19:05, Mohsen wrote:
>> Thank you so much Lars.
>> I used LibreOffice and I will test HTML format and inform you.
> 
> You are aware of the MediaWiki export function of LibreOffice?

Yes, but I have not been able to find the extension at 
https://extensions.libreoffice.org/
The extension seems to have been discontinued some time ago
I don't know whether there are any distros, where you can still get it from.
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [GSoC] GSoC Introduction : Fuzzing Xen hypercall interface

[Lars Kurth]

Hi all,

I wanted to add a few thoughts here, as this is clearly one of the harder tasks.

> On 27 Mar 2017, at 14:07, Felix Schmoll  wrote:
> 
> 2017-03-26 15:04 GMT+02:00 Wei Liu  >:
> On Sun, Mar 26, 2017 at 01:33:08PM +0200, Felix Schmoll wrote:
> [...]
> > > So just one last time to be clear about this: You can't just ignore
> > interrupts and write all other edges to a shared memory region, like the
> > KCOV feature the syzkaller uses does,
> 
> Yes, you can.
> 
> Since you mention that, let's break things down a bit more.
> 
> [snip]
> 
> Feel free to speak your thought. This project is meant to be beneficial
> to both you and the Xen project. I would be quite delighted to hear your
> understanding of the project.
>  
> Principally I would be fine with either the tracing or the prototype, I find 
> it however much more difficult to imagine what successfully implementing the 
> tracing would look like and how to write a good proposal that goes into 
> specifics. Writing a proof-of-concept/prototype is easier in that regard as 
> success would be just defined by "does it run".

I think there may be other possibilities to structure a proposal, e.g. a 
prototype (or set of experiments) followed by a design and/or gap analysis that 
could be community reviewed (and checked into our docs tree). We could also 
build in a blog post (or similar). The challenge is to come up with a structure 
that ensures that we make progress on understanding the problem space and that 
you have something to show and refer to at the end of the project. 

I am just throwing this in as a possibility, but obviously Wei would have to 
agree with it.

> What I'm having in my mind right now is still a rather vague notion of how 
> the tracing output looks like and an a bunch of ideas on what afl and 
> syzkaller do, combined with huge gaps in how Xen "really" works. That will 
> certainly start to clear up once I start really digging into it, but until 
> then I have to rely mostly on your intuition in terms of what is realistic in 
> what timeframe.

I would maybe suggest that you and Wei have a discussion on IRC to discuss the 
pro's and con's of the two different approaches and to see what is realistic. 

> Now if I have to decide between the two, I'd still prefer the tracing, since 
> on the one hand being the author of a hypercall seems to be pretty cool, and 
> on the other hand learning the actual contribution process and writing 
> something ready for deployment seems much more valuable.

It is also worth noting that the contribution process for a design or similar 
would be the same than for code (we tend to store such documents in [xen.git] / 
docs ).

Hope that helped

Regards
Lars___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Outreachy] Interested in open source contribution to Xen Project

[Lars Kurth]

Hi Sreya,
I had missed this mail as I wasn't CC'ed

> On 5 Mar 2017, at 11:34, Sreya Mittal  wrote:
> 
> Hi,
> I am Sreya Mittal pursuing my Btech in Computer Science at International 
> Institute of Information Technology, Hyderabad. 
> 
> I am interested in open-source contributions and looking forward to becoming 
> a part of Xen Project and Outreachy community. I am very  interested in the 
> following project ideas : 
>  
> 1) Xen Hypervisor - golang bindings for libxl
> 2) Xen Code Review Dashboard

Adding Jesus for #2

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] "Hello Xen Project" Book.

[Lars Kurth]

Hi Mohsen,

> On 15 Mar 2017, at 09:50, Mohsen  wrote:
> 
> Dear Xen Project community members,
> 
> I have written a Xen book recently (pdf attached) which is aimed at teaching 
> Xen newbies. I would like to make the book available to the Xen Project under 
> a CC-BY-SA-3.0 license. Ideally, I would like to publish the content on the 
> Xen Project wiki in an editable form, such that others can contribute and 
> build on it and it stays up-to-date. I also noticed that the Xen Wiki has the 
> https://www.mediawiki.org/wiki/Extension:Collection extension, which should 
> make it possible to create a PDF, ODF or DocBook from the pages for those who 
> want a manual rather than wiki pages.

Thank you for doing this. As far as I can tell the fact that you published the 
book under CC-BY-SA-3.0 would make it possible to move the content to the wiki. 

> I had a conversation with Lars to check whether this is possible and he 
> believes it is. He suggests that first we upload the book as pdf to the wiki 
> and as a second step, agree an information architecture and then convert the 
> book to mark-down. There are a number of conversion tools which should get us 
> there some of the way, with a bit of cleanup and beautification needed after 
> the initial import. I can make the source available in a format that makes 
> conversion to markdown easier.

We do need to find a way to convert the content into markdown format though, 
which may be quite a bit of work.

I have done this before for html pages, converting them into docman markdown. I 
have not checked whether there are online or command line tools which do that 
for mediawiki markdown. In any case, the conversion is fundamentally doable, 
although it will be somewhat tedious to do this. If anyone has more experience, 
please share and advise what the best way forward is.

The main problem that I faced when doing something similar were tables, figures 
and other more advanced formatting. Much of this may get lost or "corrupted" in 
some way and will have to be re-introduced post conversion.  

@Mohsen: as far as I recall, you used Word or LibreOffice to create the book? 
Is that correct? If so, it should be possible to save it in html, which would 
ensure that figures and so on are saved in some sensible way. We would probably 
need to find a temporary location where to store this. And we can start 
experimenting a little and maybe provide a quick guide on how to do this.

As for the information architecture, I was thinking about a structure such as 
...

https://wiki.xenproject.org/wiki/ 
https://wiki.xenproject.org/wiki//title_and_credits 
https://wiki.xenproject.org/wiki// 
https://wiki.xenproject.org/wiki/// 
... a separate article for each article in the book, such as "Virtualization 
and Security". As a first step, we would probably keep the original chapter 
structure. 

This would then look something like ...
https://wiki.xenproject.org/wiki/HelloXenProject
https://wiki.xenproject.org/wiki/HelloXenProject/0/Title
https://wiki.xenproject.org/wiki/HelloXenProject/0/Credits
https://wiki.xenproject.org/wiki/HelloXenProject/0/Licence
https://wiki.xenproject.org/wiki/HelloXenProject/1-Intro
https://wiki.xenproject.org/wiki/HelloXenProject/1-Intro/History
https://wiki.xenproject.org/wiki/HelloXenProject/1-Intro/TypesOfVirtualization

We may need some other extensible numbering scheme, which would make it easy to 
create PDF's with https://www.mediawiki.org/wiki/Extension:Collection - again, 
this is something I don't have experience with.

> What do people think? Is this a good idea? Would anyone be willing to help? I 
> am not very familiar with Markdown and would need someone else to help with 
> the wikification of the book. Lars already volunteered to help.

I will definitely help, but this would be an activity, which could easily be 
distributed. So help from others would be very highly appreciated.

Best Regards
Lars


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] 2017 Xen Project Developer and Design Summit: CfP open from now to April 14, Event is held July 11-13, 2017 in Budapest

[Lars Kurth]

Dear Community Members, 

I am excited to announce that registration and the call for proposals is open 
for Xen Project Developer and Design Summit 2017, which will be held in 
Budapest, Hungary from July 11-13, 2017. The Xen Project Developer and Design 
Summit combines the formats of Xen Project Developer Summits with Xen Project 
Hackathons, and brings together the Xen Project’s community. We will aim to 
have talks in the mornings and smaller interactive design and problem solving 
sessions in the afternoon. 

Note that the CfP period is quite short: if you need extra time, or you 
otherwise have difficulties with the CfP please contact me via 
community.mana...@xenproject.org.

Submit a Talk
=

Several formats are being accepted for speaking proposals, including: 

* Presentations and Panels: these are presentations and panels as we always had 
them at Developer Summits in the past
* Interactive design and problem solving sessions. These sessions can be 
submitted as part of the CFP, but we will reserve a number of design sessions 
to be allocated during the event. Proposers of design sessions are expected to 
host and moderate design sessions following the format we have used at Xen 
Project Hackathons. If you have not participated in these in the past, check 
out past event reports from 2016 
(https://blog.xenproject.org/2016/04/26/xen-project-hackathon-16-event-report), 
2015 
(https://blog.xenproject.org/2015/05/07/xen-project-hackathon-15-event-report) 
and 2013 
(https://blog.xenproject.org/2013/05/28/event-report-xen-hackathon-2013). 

Never talked at a conference before? Don’t worry! We encourage new speakers to 
submit for our events and have plenty of resources to help you prepare for your 
presentation. 

Here are some dates to remember for submissions and in general:

* CFP Close: April 14, 2017
* CFP Notifications: May 5, 2017
* Schedule Announced: May 16, 2017
* Event: July 11-13, 2017

What is different to past events?
=

* We are merging the 2 day Hackathons and 2 day Developer summit into a 
combined 3 day event.
* It is possible to submit Interactive design and problem solving sessions (or 
Hackathon sessions) via the CFP.
* Lunch will be provided 

Registration


Registration information is available at 
https://events.linuxfoundation.org/events/xen-developer-and-design-summit/attend/register

Travel stipends are only available for students or individuals that are not 
associated with a company. 

Accommodation and other information
===

General information about the event can be found on 
http://events.linuxfoundation.org/events/xen-developer-and-design-summit

Note that the following information is not yet in place: we will add this 
shortly
- visa letter requests
- hotel block booking information

If you have any questions, please contact me via 
community.mana...@xenproject.org 

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel