Re: [Xen-devel] [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Rajiv Ranganath]

On Tue, Sep 12 2017 at 04:38:19 AM, Andrew Cooper  
wrote:
> On 11/09/2017 21:20, Stefano Stabellini wrote:

[...]

>> My only concern is about diverging from the upstream Xen codebase. I
>> think the runit scripts should call xencommons underneath. If xencommons
>> cannot cope with being called from runit, we could make changes to
>> xencommon in xen.git to make it so.
>>
>> Otherwise, we will end up in a situation such as:
>> - xen.git changes xencommons
>> - we don't notice
>> - we upgrade Xen version
>> - stage1-xen doesn't work anymore
>>
>> If we used xencommons underneath we would avoid this, and it looks like
>> xencommons could be made to work well with runit.
>
> If possible, upstream Xen should be made to be compatible with runit
> (this would be the ideal case). If not, upstream Xen should contain
> different styles of these files, which are selected between by a
> ./configure option (this is suboptimal, but better than locally
> forking). This offers the greatest chance that updates to one don't
> cause the other to be stale.

I agree that it would be beneficial to have upstream Xen support for
runit. However, runit is packaged differently in every distro.

We work around this issue by packaging our own version of runit [1].
Fedora does not include runit in its repositories. That helps because we
don't have to worry about conflicting with distro packaged runit.

One option to consider is for xen project to package its own version of
runit for major distros (we will have one for Fedora in stage1-xen), and
use that as the basis for runit support.

Since stage1-xen is still under development, maybe we can use runit in
stage1-xen as a testing ground. If things work out well, we can then see
how best to integrate with xencommons or add a configure option.

By then we will also know if there is broader community interest in
having runit support in xen.

As to changes to xencommons breaking stage1-xen, as we get closer to
stable release, we probably will have integration tests to catch this
and many other things! :-)

Best,
Rajiv

[1]: https://github.com/lambda-linux-fedora/runit/tree/ver-2.1.2-1.1.fc25

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Andrew Cooper]

On 11/09/2017 21:20, Stefano Stabellini wrote:
> On Sat, 9 Sep 2017, Rajiv Ranganath wrote:
>> On Thu, Sep 07 2017 at 12:10:21 AM, Stefano Stabellini 
>>  wrote:
>>
>> [...]
>>
>>> The series is much better now thank you. One question: why did you write
>>> your own init scripts rather than reusing xencommons (with the caveat
>>> that you would have to add make sure to source_path.sh before running
>>> xencommons)?  Does it have something to do with systemd?
>> There are a few related reasons for this.
>>
>> 1. Using runit lets us abstract out our dependency on systemd and
>> upstart. We can use the same abstraction in containers [1], virtual
>> machines and on bare metal.
>>
>> 2. In Linux distributions, there is tight coupling between package
>> management system (rpm/deb), init systems (upstart/systemd), and service
>> daemons.
>>
>> With containers, if the expectation is that most service daemons and
>> apps would be containerized, and managed by a node agent then a natural
>> question to ask would be what should be the role of init systems like
>> systemd?
>>
>> By using runit (on systemd, upstart and within containers), we defer
>> answering this question. 
>>
>> 3. One of the use cases that we want to support is to have different
>> versions of xen co-exist on the same filesystem. Then a higher level
>> tool can do rolling updates and if required rollbacks.
>>
>> While it is possible to accomplish this on existing init systems,
>> depending on how xen is packaged and deployed, it might involve using
>> distro package and repository management tools.
>>
>> With runit, we can use regular docker tools, which is much more friendly
>> for mainstream developers and CI systems. We also abstract over init
>> systems, which is a desirable property to have.
>>
>> 4. I looked into xencommons script and systemd unit files when creating
>> runit scripts. Our runit scripts is straightforward translation of how
>> one would start xen manually.
>>
>> Perhaps the only part of the script that might need some explanation is
>> in `xen-init-dom0/run`.
>>
>> ```
>> exec chpst -b xen-init-dom0 runit-pause
>> ```
>>
>> This is a pattern used to build equivalent of "oneshot" service in
>> systemd. It was developed in Ignite (a Arch Linux project before they
>> switched to systemd) and later co-opted by Void Linux [2].
>>
>> I am not sure if I answered your question. Sometimes I feel, maybe we
>> should just let questions around init systems be like one of those
>> "unanswered questions" in theology. :-) [3]
>>
>> Best,
>> Rajiv
>>
>> [1] https://github.com/lambda-linux/baseimage-amzn#adding_additional_daemons
>>
>> [2] 
>> https://github.com/voidlinux/void-runit/commit/7aecf46ec589a5bc49ae2392137bcd0e7468dd08
> Thank you for the pointers. I have no opinions on the format of the init
> scripts. runit is fine by me in that respect, and I understand the
> advantages you pointed out.
>
> My only concern is about diverging from the upstream Xen codebase. I
> think the runit scripts should call xencommons underneath. If xencommons
> cannot cope with being called from runit, we could make changes to
> xencommon in xen.git to make it so.
>
> Otherwise, we will end up in a situation such as:
> - xen.git changes xencommons
> - we don't notice
> - we upgrade Xen version
> - stage1-xen doesn't work anymore
>
> If we used xencommons underneath we would avoid this, and it looks like
> xencommons could be made to work well with runit.

If possible, upstream Xen should be made to be compatible with runit
(this would be the ideal case).  If not, upstream Xen should contain
different styles of these files, which are selected between by a
./configure option (this is suboptimal, but better than locally
forking).  This offers the greatest chance that updates to one don't
cause the other to be stale.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Stefano Stabellini]

On Sat, 9 Sep 2017, Rajiv Ranganath wrote:
> On Thu, Sep 07 2017 at 12:10:21 AM, Stefano Stabellini 
>  wrote:
> 
> [...]
> 
> > The series is much better now thank you. One question: why did you write
> > your own init scripts rather than reusing xencommons (with the caveat
> > that you would have to add make sure to source_path.sh before running
> > xencommons)?  Does it have something to do with systemd?
> 
> There are a few related reasons for this.
> 
> 1. Using runit lets us abstract out our dependency on systemd and
> upstart. We can use the same abstraction in containers [1], virtual
> machines and on bare metal.
> 
> 2. In Linux distributions, there is tight coupling between package
> management system (rpm/deb), init systems (upstart/systemd), and service
> daemons.
> 
> With containers, if the expectation is that most service daemons and
> apps would be containerized, and managed by a node agent then a natural
> question to ask would be what should be the role of init systems like
> systemd?
> 
> By using runit (on systemd, upstart and within containers), we defer
> answering this question. 
> 
> 3. One of the use cases that we want to support is to have different
> versions of xen co-exist on the same filesystem. Then a higher level
> tool can do rolling updates and if required rollbacks.
> 
> While it is possible to accomplish this on existing init systems,
> depending on how xen is packaged and deployed, it might involve using
> distro package and repository management tools.
> 
> With runit, we can use regular docker tools, which is much more friendly
> for mainstream developers and CI systems. We also abstract over init
> systems, which is a desirable property to have.
> 
> 4. I looked into xencommons script and systemd unit files when creating
> runit scripts. Our runit scripts is straightforward translation of how
> one would start xen manually.
> 
> Perhaps the only part of the script that might need some explanation is
> in `xen-init-dom0/run`.
> 
> ```
> exec chpst -b xen-init-dom0 runit-pause
> ```
> 
> This is a pattern used to build equivalent of "oneshot" service in
> systemd. It was developed in Ignite (a Arch Linux project before they
> switched to systemd) and later co-opted by Void Linux [2].
> 
> I am not sure if I answered your question. Sometimes I feel, maybe we
> should just let questions around init systems be like one of those
> "unanswered questions" in theology. :-) [3]
> 
> Best,
> Rajiv
> 
> [1] https://github.com/lambda-linux/baseimage-amzn#adding_additional_daemons
> 
> [2] 
> https://github.com/voidlinux/void-runit/commit/7aecf46ec589a5bc49ae2392137bcd0e7468dd08

Thank you for the pointers. I have no opinions on the format of the init
scripts. runit is fine by me in that respect, and I understand the
advantages you pointed out.

My only concern is about diverging from the upstream Xen codebase. I
think the runit scripts should call xencommons underneath. If xencommons
cannot cope with being called from runit, we could make changes to
xencommon in xen.git to make it so.

Otherwise, we will end up in a situation such as:
- xen.git changes xencommons
- we don't notice
- we upgrade Xen version
- stage1-xen doesn't work anymore

If we used xencommons underneath we would avoid this, and it looks like
xencommons could be made to work well with runit.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Rajiv Ranganath]

On Thu, Sep 07 2017 at 12:10:21 AM, Stefano Stabellini  
wrote:

[...]

> The series is much better now thank you. One question: why did you write
> your own init scripts rather than reusing xencommons (with the caveat
> that you would have to add make sure to source_path.sh before running
> xencommons)?  Does it have something to do with systemd?

There are a few related reasons for this.

1. Using runit lets us abstract out our dependency on systemd and
upstart. We can use the same abstraction in containers [1], virtual
machines and on bare metal.

2. In Linux distributions, there is tight coupling between package
management system (rpm/deb), init systems (upstart/systemd), and service
daemons.

With containers, if the expectation is that most service daemons and
apps would be containerized, and managed by a node agent then a natural
question to ask would be what should be the role of init systems like
systemd?

By using runit (on systemd, upstart and within containers), we defer
answering this question. 

3. One of the use cases that we want to support is to have different
versions of xen co-exist on the same filesystem. Then a higher level
tool can do rolling updates and if required rollbacks.

While it is possible to accomplish this on existing init systems,
depending on how xen is packaged and deployed, it might involve using
distro package and repository management tools.

With runit, we can use regular docker tools, which is much more friendly
for mainstream developers and CI systems. We also abstract over init
systems, which is a desirable property to have.

4. I looked into xencommons script and systemd unit files when creating
runit scripts. Our runit scripts is straightforward translation of how
one would start xen manually.

Perhaps the only part of the script that might need some explanation is
in `xen-init-dom0/run`.

```
exec chpst -b xen-init-dom0 runit-pause
```

This is a pattern used to build equivalent of "oneshot" service in
systemd. It was developed in Ignite (a Arch Linux project before they
switched to systemd) and later co-opted by Void Linux [2].

I am not sure if I answered your question. Sometimes I feel, maybe we
should just let questions around init systems be like one of those
"unanswered questions" in theology. :-) [3]

Best,
Rajiv

[1] https://github.com/lambda-linux/baseimage-amzn#adding_additional_daemons

[2] 
https://github.com/voidlinux/void-runit/commit/7aecf46ec589a5bc49ae2392137bcd0e7468dd08

[3] https://en.wikipedia.org/wiki/The_unanswered_questions

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath 
> 
> Signed-off-by: Rajiv Ranganath 

The series is much better now thank you. One question: why did you write
your own init scripts rather than reusing xencommons (with the caveat
that you would have to add make sure to source_path.sh before running
xencommons)?  Does it have something to do with systemd?


> ---
>  build/fedora/xen-unstable-runit/setup.sh   |   18 
>  build/fedora/xen-unstable-runit/teardown.sh|   18 
>  .../xen-init-dom0-disk-backend/run |   11 ++
>  build/fedora/xen-unstable-runit/xen-init-dom0/run  |9 
>  build/fedora/xen-unstable-runit/xenconsoled/run|   13 +++
>  build/fedora/xen-unstable-runit/xenstored/run  |   23 
> 
>  6 files changed, 92 insertions(+)
>  create mode 100755 build/fedora/xen-unstable-runit/setup.sh
>  create mode 100755 build/fedora/xen-unstable-runit/teardown.sh
>  create mode 100755 
> build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
>  create mode 100755 build/fedora/xen-unstable-runit/xen-init-dom0/run
>  create mode 100755 build/fedora/xen-unstable-runit/xenconsoled/run
>  create mode 100755 build/fedora/xen-unstable-runit/xenstored/run
> 
> diff --git a/build/fedora/xen-unstable-runit/setup.sh 
> b/build/fedora/xen-unstable-runit/setup.sh
> new file mode 100755
> index 000..b5adf8c
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/setup.sh
> @@ -0,0 +1,18 @@
> +#!/bin/bash
> +
> +set -e
> +
> +# runit RPM creates `/etc/service` directory
> +if [ ! -d "/etc/service" ]; then
> +echo "/etc/service directory not found. Please install runit RPM."
> +exit 1
> +fi
> +
> +runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend 
> xenstored"
> +
> +for service in $runit_services; do
> +ln -sf /opt/xen-unstable-runit/$service /etc/service/$service
> +done
> +
> +echo "Successfully created symlinks in /etc/service directory."
> +exit 0
> diff --git a/build/fedora/xen-unstable-runit/teardown.sh 
> b/build/fedora/xen-unstable-runit/teardown.sh
> new file mode 100755
> index 000..d333807
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/teardown.sh
> @@ -0,0 +1,18 @@
> +#!/bin/bash
> +
> +set -e
> +
> +# runit RPM creates `/etc/service` directory
> +if [ ! -d "/etc/service" ]; then
> +echo "/etc/service directory not found."
> +exit 1
> +fi
> +
> +runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend 
> xenstored"
> +
> +for service in $runit_services; do
> +rm -f /etc/service/$service
> +done
> +
> +echo "Successfully deleted symlinks in /etc/service directory."
> +exit 0
> diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run 
> b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
> new file mode 100755
> index 000..6315d48
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
> @@ -0,0 +1,11 @@
> +#!/bin/bash
> +
> +set -e
> +
> +sv check xenstored >/dev/null || exit 1
> +sv check xenconsoled >/dev/null || exit 1
> +
> +# In case of failure, allow user to run teardown script
> +sleep 5s
> +
> +exec /opt/xen-unstable/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach 
> -name dom0 -nographic -M xenpv -monitor /dev/null -serial /dev/null -parallel 
> /dev/null -nodefaults -no-user-config
> diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0/run 
> b/build/fedora/xen-unstable-runit/xen-init-dom0/run
> new file mode 100755
> index 000..193ba19
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xen-init-dom0/run
> @@ -0,0 +1,9 @@
> +#!/bin/bash
> +
> +set -e
> +
> +sv check xenstored >/dev/null || exit 1
> +
> +/opt/xen-unstable/lib/xen/bin/xen-init-dom0
> +
> +exec chpst -b xen-init-dom0 runit-pause
> diff --git a/build/fedora/xen-unstable-runit/xenconsoled/run 
> b/build/fedora/xen-unstable-runit/xenconsoled/run
> new file mode 100755
> index 000..b5b7a9f
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xenconsoled/run
> @@ -0,0 +1,13 @@
> +#!/bin/bash
> +
> +set -e
> +
> +sv check xen-init-dom0 >/dev/null || exit 1
> +
> +[ ! -d /var/log/xen/console ] && mkdir -p /var/log/xen/console
> +
> +# In case of failure, allow user to run teardown script
> +sleep 5s
> +
> +# --log=[none|guest|hv|all]
> +exec /opt/xen-unstable/sbin/xenconsoled -i --log=none
> diff --git a/build/fedora/xen-unstable-runit/xenstored/run 
> b/build/fedora/xen-unstable-runit/xenstored/run
> new file mode 100755
> index 000..beb2a5f
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xenstored/run
> @@ -0,0 +1,23 @@
> +#!/bin/bash
> +
> +set -e
> +
> +[ ! -d /var/run/xen ] && mkdir -p /var/run/xen
> +[ ! -d /var/run/xenstored ] && mkdir -p /var/run/xenstored
> +[ ! -d /var/log/xen ] && mkdir -p /var/log/xen
> +[ ! -d /var/lib/xen ] && mkdir -p /var/lib/xen
> +[ ! -d 

[Xen-devel] [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Rajiv Ranganath]

From: Rajiv M Ranganath 

Signed-off-by: Rajiv Ranganath 
---
 build/fedora/xen-unstable-runit/setup.sh   |   18 
 build/fedora/xen-unstable-runit/teardown.sh|   18 
 .../xen-init-dom0-disk-backend/run |   11 ++
 build/fedora/xen-unstable-runit/xen-init-dom0/run  |9 
 build/fedora/xen-unstable-runit/xenconsoled/run|   13 +++
 build/fedora/xen-unstable-runit/xenstored/run  |   23 
 6 files changed, 92 insertions(+)
 create mode 100755 build/fedora/xen-unstable-runit/setup.sh
 create mode 100755 build/fedora/xen-unstable-runit/teardown.sh
 create mode 100755 
build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
 create mode 100755 build/fedora/xen-unstable-runit/xen-init-dom0/run
 create mode 100755 build/fedora/xen-unstable-runit/xenconsoled/run
 create mode 100755 build/fedora/xen-unstable-runit/xenstored/run

diff --git a/build/fedora/xen-unstable-runit/setup.sh 
b/build/fedora/xen-unstable-runit/setup.sh
new file mode 100755
index 000..b5adf8c
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/setup.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -e
+
+# runit RPM creates `/etc/service` directory
+if [ ! -d "/etc/service" ]; then
+echo "/etc/service directory not found. Please install runit RPM."
+exit 1
+fi
+
+runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend xenstored"
+
+for service in $runit_services; do
+ln -sf /opt/xen-unstable-runit/$service /etc/service/$service
+done
+
+echo "Successfully created symlinks in /etc/service directory."
+exit 0
diff --git a/build/fedora/xen-unstable-runit/teardown.sh 
b/build/fedora/xen-unstable-runit/teardown.sh
new file mode 100755
index 000..d333807
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/teardown.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -e
+
+# runit RPM creates `/etc/service` directory
+if [ ! -d "/etc/service" ]; then
+echo "/etc/service directory not found."
+exit 1
+fi
+
+runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend xenstored"
+
+for service in $runit_services; do
+rm -f /etc/service/$service
+done
+
+echo "Successfully deleted symlinks in /etc/service directory."
+exit 0
diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run 
b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
new file mode 100755
index 000..6315d48
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -e
+
+sv check xenstored >/dev/null || exit 1
+sv check xenconsoled >/dev/null || exit 1
+
+# In case of failure, allow user to run teardown script
+sleep 5s
+
+exec /opt/xen-unstable/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach 
-name dom0 -nographic -M xenpv -monitor /dev/null -serial /dev/null -parallel 
/dev/null -nodefaults -no-user-config
diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0/run 
b/build/fedora/xen-unstable-runit/xen-init-dom0/run
new file mode 100755
index 000..193ba19
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xen-init-dom0/run
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -e
+
+sv check xenstored >/dev/null || exit 1
+
+/opt/xen-unstable/lib/xen/bin/xen-init-dom0
+
+exec chpst -b xen-init-dom0 runit-pause
diff --git a/build/fedora/xen-unstable-runit/xenconsoled/run 
b/build/fedora/xen-unstable-runit/xenconsoled/run
new file mode 100755
index 000..b5b7a9f
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xenconsoled/run
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+set -e
+
+sv check xen-init-dom0 >/dev/null || exit 1
+
+[ ! -d /var/log/xen/console ] && mkdir -p /var/log/xen/console
+
+# In case of failure, allow user to run teardown script
+sleep 5s
+
+# --log=[none|guest|hv|all]
+exec /opt/xen-unstable/sbin/xenconsoled -i --log=none
diff --git a/build/fedora/xen-unstable-runit/xenstored/run 
b/build/fedora/xen-unstable-runit/xenstored/run
new file mode 100755
index 000..beb2a5f
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xenstored/run
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+set -e
+
+[ ! -d /var/run/xen ] && mkdir -p /var/run/xen
+[ ! -d /var/run/xenstored ] && mkdir -p /var/run/xenstored
+[ ! -d /var/log/xen ] && mkdir -p /var/log/xen
+[ ! -d /var/lib/xen ] && mkdir -p /var/lib/xen
+[ ! -d /var/lib/xen/dump ] && mkdir -p /var/lib/xen/dump
+[ ! -d /var/lib/xen/xenpaging ] && mkdir -p /var/lib/xen/paging
+[ ! -d /var/lib/xenstored ] && mkdir -p /var/lib/xenstored
+modprobe -q xen-evtchn || exit 1
+modprobe -q xen-gntdev || exit 1
+modprobe -q xen-gntalloc || exit 1
+mountpoint -q /proc/xen || mount -t xenfs xenfs /proc/xen
+mountpoint -q /var/lib/xenstored || mount -t tmpfs xenstored /var/lib/xenstored
+grep -q "control_d" /proc/xen/capabilities || exit 1
+
+# In case of failure, allow user to run teardown script
+sleep 5s
+
+# Use `--trace-file /var/log/xen/xenstored-trace.log` for logs