[yocto] [yocto-autobuilder2][RFC][PATCH] README-Guide.md: Add multi-node content, extra config info
Signed-off-by: Trevor Gamblin --- README-Guide.md | 94 + 1 file changed, 94 insertions(+) diff --git a/README-Guide.md b/README-Guide.md index 21dd7c1..8558c48 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -43,6 +43,16 @@ yocto-controller/yoctoabb yocto-worker ``` +Before proceeding, make sure that the following is added to the +pokybuild3 user's exports (e.g. in .bashrc), or builds will fail after +being triggered: + +``` +export LC_ALL=en_US.UTF-8 +export LANG=en_US.UTF-8 +export LANGUAGE=en_US.UTF-8 +``` + Next, we need to update the `yocto-controller/yoctoabb/master.cfg` towards the bottom where the `title`, `titleURL`, and `buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. Then, we need to update the `yocto-controller/yoctoabb/config.py` to include our worker. In that file, find the line where `workers` is set and add: ["example-worker"]. _NOTE:_ if your worker's name is different, use that here. Section 3.1 discusses how to further refine this list of workers. @@ -112,6 +122,90 @@ sudo /home/pokybuild3/yocto-worker/qemuarm/build/scripts/runqemu-gen-tapdevs \ In the above command, we assume the a build named qemuarm failed. The value of 8 is the number of tap interfaces to create on the worker. +### 1.3) Adding Dedicated Worker Nodes + +Running both the controller and the worker together on a single machine +can quickly result in long build times and an unresponsive web UI, +especially if you plan on running any of the more comprehensive builders +(e.g. a-full). Additional workers can be added to the cluster by +following the steps given above, except that the yocto-controller steps +do not need to be repeated. For example, to add a new worker +"ala-blade51" to an Autobuilder cluster with a yocto-controller at the +IP address 147.11.105.72: + +1. On the yocto-controller host, add the name of the new worker to a worker +list (or create a new one) e.g. 'workers_wrlx = ["ala-blade51"]' and +make sure that it is added to the "workers" list. + +2. On the new worker node: + +``` +sudo apt-get install gawk wget git-core diffstat unzip texinfo \ +gcc-multilib build-essential chrpath socat cpio python python3 \ +python3-pip python3-pexpect xz-utils debianutils iputils-ping \ +libsdl1.2-dev xterm + +sudo pip3 install buildbot buildbot-www buildbot-waterfall-view \ +buildbot-console-view buildbot-grid-view buildbot-worker + +useradd -m --system pokybuild3 +cd /home/pokybuild3 +mkdir -p git/trash +buildbot-worker create-worker -r --umask=0o22 yocto-worker 147.11.105.72 ala-blade51 pass +chown -R pokybuild3:pokybuild3 /home/pokybuild3 +``` + + > Note 1: The URL/IP given to the create-worker command must match the +host running the yocto-controller. + + > Note 2: The "pass" argument given to the create-worker command must +match the common "worker_pass" variable set in yocto-controller/yoctoabb/config.py. + + +### 1.4) Configuring NFS for the Autobuilder Cluster + +The Yocto Autobuilder relies on NFS to distribute a common sstate cache +and other outputs between nodes. A similar configuration can be +deployed by performing the steps given below, which were written for +Ubuntu 18.04.In order for both the controller and worker nodes to be able +to access the NFS share without issue, the "pokybuild3" user on all +systems must have the same UID/GID, or sufficient permissions must be +granted on the /srv/autobuilder path (or wherever you modified the config +files to point to). The following instructions assume a controller node +at 147.11.105.72 and a single worker node at 147.11.105.71, but +additional worker nodes can be added as needed (see the previous +section). + +1. On the NFS host: + +``` +sudo apt install -y nfs-kernel-server +sudo mkdir -p /srv/autobuilder/autobuilder.yoctoproject.org/pub/sstate +sudo chown -R pokybuild3:pokybuild3 /srv +``` +2. Add the following to /etc/exports, replacing the path and IP fields + as necessary for each client node: +``` +/srv/autobuilder/autobuilder.yoctoproject.org/pub/sstate 147.11.105.71(rw,sync,no_subtree_check) +``` + +3. Run +``` +sudo systemctl restart nfs-kernel-server +``` + +4. Adjust the firewall (if required). Example: +``` +sudo ufw allow from 147.11.105.71 to any port nfs +``` + +5. On the client node(s): +``` +sudo mkdir -p /srv/autobuilder/autobuilder.yoctoproject.org/pub/sstate +sudo chown -R pokybuild3:pokybuild3 /srv/autobuilder/ +sudo mount 147.11.105.72:/srv/autobuilder/autobuilder.yoctoproject.org/pub/sstate /srv/autobuilder/autobuilder.yoctoproject.org/pub/sstate +``` + ## 2) Basics This section is an overview of operation and a few basic configuration file relationships. See Section 3 for more detailed instructions. -- 2.30.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent
Re: [yocto] [yocto-autobuilder2][RFC][PATCH] README-Guide.md: Add multi-node content, extra config info
On 2021-04-05 2:55 p.m., Michael Halstead wrote: **[Please note: This e-mail is from an EXTERNAL e-mail address] On Mon, Apr 5, 2021 at 9:23 AM Trevor Gamblin mailto:trevor.gamb...@windriver.com>> wrote: Signed-off-by: Trevor Gamblin mailto:trevor.gamb...@windriver.com>> --- README-Guide.md | 94 + 1 file changed, 94 insertions(+) diff --git a/README-Guide.md b/README-Guide.md index 21dd7c1..8558c48 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -43,6 +43,16 @@ yocto-controller/yoctoabb yocto-worker ``` +Before proceeding, make sure that the following is added to the +pokybuild3 user's exports (e.g. in .bashrc), or builds will fail after +being triggered: + +``` +export LC_ALL=en_US.UTF-8 +export LANG=en_US.UTF-8 +export LANGUAGE=en_US.UTF-8 +``` On the AB at typhoon.yocto.io <http://typhoon.yocto.io> only LANG=en_US.UTF-8 is set. I don't know why LC_ALL or LANGUAGE need to be set on your cluster for builds to succeed. You're right, I don't need the others. Fixing this for the next revision. + Next, we need to update the `yocto-controller/yoctoabb/master.cfg` towards the bottom where the `title`, `titleURL`, and `buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. Then, we need to update the `yocto-controller/yoctoabb/config.py` to include our worker. In that file, find the line where `workers` is set and add: ["example-worker"]. _NOTE:_ if your worker's name is different, use that here. Section 3.1 discusses how to further refine this list of workers. @@ -112,6 +122,90 @@ sudo /home/pokybuild3/yocto-worker/qemuarm/build/scripts/runqemu-gen-tapdevs \ In the above command, we assume the a build named qemuarm failed. The value of 8 is the number of tap interfaces to create on the worker. +### 1.3) Adding Dedicated Worker Nodes + +Running both the controller and the worker together on a single machine +can quickly result in long build times and an unresponsive web UI, +especially if you plan on running any of the more comprehensive builders +(e.g. a-full). Additional workers can be added to the cluster by +following the steps given above, except that the yocto-controller steps +do not need to be repeated. For example, to add a new worker +"ala-blade51" to an Autobuilder cluster with a yocto-controller at the +IP address 147.11.105.72 <http://147.11.105.72>: + +1. On the yocto-controller host, add the name of the new worker to a worker +list (or create a new one) e.g. 'workers_wrlx = ["ala-blade51"]' and +make sure that it is added to the "workers" list. + +2. On the new worker node: + +``` +sudo apt-get install gawk wget git-core diffstat unzip texinfo \ +gcc-multilib build-essential chrpath socat cpio python python3 \ +python3-pip python3-pexpect xz-utils debianutils iputils-ping \ +libsdl1.2-dev xterm Should we link to https://docs.yoctoproject.org/ref-manual/system-requirements.html#ubuntu-and-debian <https://docs.yoctoproject.org/ref-manual/system-requirements.html#ubuntu-and-debian> for the current package set as well as listing this information here? The beginning of README-Guide.md mentions that the user should reference the Yocto Manual for the required packages, so maybe copying the list here is inconsistent. I'll put the link near the top of the doc and we can look at a better way to do this if/when a new version of this guide makes it into the Manual. + +sudo pip3 install buildbot buildbot-www buildbot-waterfall-view \ +buildbot-console-view buildbot-grid-view buildbot-worker + +useradd -m --system pokybuild3 +cd /home/pokybuild3 +mkdir -p git/trash +buildbot-worker create-worker -r --umask=0o22 yocto-worker 147.11.105.72 ala-blade51 pass +chown -R pokybuild3:pokybuild3 /home/pokybuild3 +``` + + > Note 1: The URL/IP given to the create-worker command must match the +host running the yocto-controller. + + > Note 2: The "pass" argument given to the create-worker command must +match the common "worker_pass" variable set in yocto-controller/yoctoabb/config.py. + + +### 1.4) Configuring NFS for the Autobuilder Cluster + +The Yocto Autobuilder relies on NFS to distribute a common sstate cache +and other outputs between nodes. A similar configuration can be +deployed by performing the steps given below, which were written for +Ubuntu 18.04.In <http://18.04.In> order for both the controller and worker nodes to be able +t
[yocto] [yocto-autobuilder2][PATCH] README-Guide.md: Add multi-node content, extra config info
The instructions in README-Guide.md are a good starting point, but there are some additional guidelines in this patch for setting up worker nodes which may be useful to others who want to run their own Autobuilder instance. Specifically, it adds: - Section 1.3 on adding additional worker nodes to a cluster - Section 1.4 on setting up an NFS share for the controller and workers to reference - A link to the Yocto Manual where the requirements to support running builds on Ubuntu/Debian systems is listed - A note to make sure that any new users (pokybuild3) created for the Autobuilder have LANG set in their bash profile Signed-off-by: Trevor Gamblin --- README-Guide.md | 86 - 1 file changed, 85 insertions(+), 1 deletion(-) diff --git a/README-Guide.md b/README-Guide.md index 21dd7c1..d976fdd 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -6,7 +6,8 @@ This guide will walk through how to install a stand-alone autobuilder controller The final outputs of this section are a controller and worker installed in the same server, ready for trimming back to an individual organization's needs. - > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting. Please refer to the Yocto manual for those packages. + > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting. Please refer to the Yocto manual for those packages: + https://docs.yoctoproject.org/ref-manual/system-requirements.html#ubuntu-and-debian The latest version of BuildBot is written in Python 3, so installation via pip3: @@ -43,6 +44,14 @@ yocto-controller/yoctoabb yocto-worker ``` +Before proceeding, make sure that the following is added to the +pokybuild3 user's exports (e.g. in .bashrc), or builds will fail after +being triggered: + +``` +export LANG=en_US.UTF-8 +``` + Next, we need to update the `yocto-controller/yoctoabb/master.cfg` towards the bottom where the `title`, `titleURL`, and `buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. Then, we need to update the `yocto-controller/yoctoabb/config.py` to include our worker. In that file, find the line where `workers` is set and add: ["example-worker"]. _NOTE:_ if your worker's name is different, use that here. Section 3.1 discusses how to further refine this list of workers. @@ -112,6 +121,81 @@ sudo /home/pokybuild3/yocto-worker/qemuarm/build/scripts/runqemu-gen-tapdevs \ In the above command, we assume the a build named qemuarm failed. The value of 8 is the number of tap interfaces to create on the worker. +### 1.3) Adding Dedicated Worker Nodes + +Running both the controller and the worker together on a single machine +can quickly result in long build times and an unresponsive web UI, +especially if you plan on running any of the more comprehensive builders +(e.g. a-full). Additional workers can be added to the cluster by +following the steps in Section 1, except that the yocto-controller steps +do not need to be repeated. For example, to add a new worker +"ala-blade51" to an Autobuilder cluster with a yocto-controller at the +IP address 147.11.105.72: + +1. On the yocto-controller host, add the name of the new worker to a worker +list (or create a new one) e.g. 'workers_wrlx = ["ala-blade51"]' and +make sure that it is added to the "workers" list. + +2. On the new worker node: + +``` +useradd -m --system pokybuild3 +cd /home/pokybuild3 +mkdir -p git/trash +buildbot-worker create-worker -r --umask=0o22 yocto-worker 147.11.105.72 ala-blade51 pass +chown -R pokybuild3:pokybuild3 /home/pokybuild3 +``` + + > Note 1: The URL/IP given to the create-worker command must match the +host running the yocto-controller. + + > Note 2: The "pass" argument given to the create-worker command must +match the common "worker_pass" variable set in yocto-controller/yoctoabb/config.py. + + +### 1.4) Configuring NFS for the Autobuilder Cluster + +The Yocto Autobuilder relies on NFS to distribute a common sstate cache +and other outputs between nodes. A similar configuration can be +deployed by performing the steps given below, which were written for +Ubuntu 18.04.In order for both the controller and worker nodes to be able +to access the NFS share without issue, the "pokybuild3" user on all +systems must have the same UID/GID, or sufficient permissions must be +granted on the /srv/autobuilder path (or wherever you modified the config +files to point to). The following instructions assume a controller node +at 147.11.105.72 and a single worker node at 147.11.105.71, but +additional worker nodes can be added as needed (see the previous +section). + +1. On the NFS host: + +``` +sudo apt install -
Re: [yocto] [yocto-autobuilder2][PATCH] README-Guide.md: Add multi-node content, extra config info
On 2021-04-06 9:44 a.m., Trevor Gamblin wrote: The instructions in README-Guide.md are a good starting point, but there are some additional guidelines in this patch for setting up worker nodes which may be useful to others who want to run their own Autobuilder instance. Specifically, it adds: Sending a v2 - adding an extra line or two about starting the worker(s). - Section 1.3 on adding additional worker nodes to a cluster - Section 1.4 on setting up an NFS share for the controller and workers to reference - A link to the Yocto Manual where the requirements to support running builds on Ubuntu/Debian systems is listed - A note to make sure that any new users (pokybuild3) created for the Autobuilder have LANG set in their bash profile Signed-off-by: Trevor Gamblin --- README-Guide.md | 86 - 1 file changed, 85 insertions(+), 1 deletion(-) diff --git a/README-Guide.md b/README-Guide.md index 21dd7c1..d976fdd 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -6,7 +6,8 @@ This guide will walk through how to install a stand-alone autobuilder controller The final outputs of this section are a controller and worker installed in the same server, ready for trimming back to an individual organization's needs. - > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting. Please refer to the Yocto manual for those packages. + > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting. Please refer to the Yocto manual for those packages: + https://docs.yoctoproject.org/ref-manual/system-requirements.html#ubuntu-and-debian The latest version of BuildBot is written in Python 3, so installation via pip3: @@ -43,6 +44,14 @@ yocto-controller/yoctoabb yocto-worker ``` +Before proceeding, make sure that the following is added to the +pokybuild3 user's exports (e.g. in .bashrc), or builds will fail after +being triggered: + +``` +export LANG=en_US.UTF-8 +``` + Next, we need to update the `yocto-controller/yoctoabb/master.cfg` towards the bottom where the `title`, `titleURL`, and `buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. Then, we need to update the `yocto-controller/yoctoabb/config.py` to include our worker. In that file, find the line where `workers` is set and add: ["example-worker"]. _NOTE:_ if your worker's name is different, use that here. Section 3.1 discusses how to further refine this list of workers. @@ -112,6 +121,81 @@ sudo /home/pokybuild3/yocto-worker/qemuarm/build/scripts/runqemu-gen-tapdevs \ In the above command, we assume the a build named qemuarm failed. The value of 8 is the number of tap interfaces to create on the worker. +### 1.3) Adding Dedicated Worker Nodes + +Running both the controller and the worker together on a single machine +can quickly result in long build times and an unresponsive web UI, +especially if you plan on running any of the more comprehensive builders +(e.g. a-full). Additional workers can be added to the cluster by +following the steps in Section 1, except that the yocto-controller steps +do not need to be repeated. For example, to add a new worker +"ala-blade51" to an Autobuilder cluster with a yocto-controller at the +IP address 147.11.105.72: + +1. On the yocto-controller host, add the name of the new worker to a worker +list (or create a new one) e.g. 'workers_wrlx = ["ala-blade51"]' and +make sure that it is added to the "workers" list. + +2. On the new worker node: + +``` +useradd -m --system pokybuild3 +cd /home/pokybuild3 +mkdir -p git/trash +buildbot-worker create-worker -r --umask=0o22 yocto-worker 147.11.105.72 ala-blade51 pass +chown -R pokybuild3:pokybuild3 /home/pokybuild3 +``` + + > Note 1: The URL/IP given to the create-worker command must match the +host running the yocto-controller. + + > Note 2: The "pass" argument given to the create-worker command must +match the common "worker_pass" variable set in yocto-controller/yoctoabb/config.py. + + +### 1.4) Configuring NFS for the Autobuilder Cluster + +The Yocto Autobuilder relies on NFS to distribute a common sstate cache +and other outputs between nodes. A similar configuration can be +deployed by performing the steps given below, which were written for +Ubuntu 18.04.In order for both the controller and worker nodes to be able +to access the NFS share without issue, the "pokybuild3" user on all +systems must have the same UID/GID, or sufficient permissions must be +granted on the /srv/autobuilder path (or wherever you modified the config +files to point to). The following instructions assume a controller node +at 147.11.105.72 and a single worker node at 147.11.105
[yocto] [yocto-autobuilder2][PATCH v2] README-Guide.md: Add multi-node content, extra config info
The instructions in README-Guide.md are a good starting point, but there are some additional guidelines in this patch for setting up worker nodes which may be useful to others who want to run their own Autobuilder instance. Specifically, it adds: - Section 1.3 on adding additional worker nodes to a cluster - Section 1.4 on setting up an NFS share for the controller and workers to reference - A link to the Yocto Manual where the requirements to support running builds on Ubuntu/Debian systems is listed - A note to make sure that any new users (pokybuild3) created for the Autobuilder have LANG set in their bash profile Signed-off-by: Trevor Gamblin --- README-Guide.md | 108 +++- 1 file changed, 106 insertions(+), 2 deletions(-) diff --git a/README-Guide.md b/README-Guide.md index 21dd7c1..832996f 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -6,7 +6,8 @@ This guide will walk through how to install a stand-alone autobuilder controller The final outputs of this section are a controller and worker installed in the same server, ready for trimming back to an individual organization's needs. - > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting. Please refer to the Yocto manual for those packages. + > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting. Please refer to the Yocto manual for those packages: + https://docs.yoctoproject.org/ref-manual/system-requirements.html#ubuntu-and-debian The latest version of BuildBot is written in Python 3, so installation via pip3: @@ -43,6 +44,14 @@ yocto-controller/yoctoabb yocto-worker ``` +Before proceeding, make sure that the following is added to the +pokybuild3 user's exports (e.g. in .bashrc), or builds will fail after +being triggered: + +``` +export LANG=en_US.UTF-8 +``` + Next, we need to update the `yocto-controller/yoctoabb/master.cfg` towards the bottom where the `title`, `titleURL`, and `buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. Then, we need to update the `yocto-controller/yoctoabb/config.py` to include our worker. In that file, find the line where `workers` is set and add: ["example-worker"]. _NOTE:_ if your worker's name is different, use that here. Section 3.1 discusses how to further refine this list of workers. @@ -61,7 +70,17 @@ Set `BASE_HOMEDIR` should be your build user's home directory. (There are she > NOTE: The way the build step is written, the worker will pull a fresh copy of the helper from the server. Therefore these configuration files must be committed to the `yocto-autobuilder-helper` repo location you have specified in `yoctoabb/config.py` because the worker is given a build step that pulls from that repo (see `yoctoabb/builders.py`). -Finally, as root, add the `yocto-*.service` files to `/lib/systemd/system` (See Appendix A). Run: `systemctl daemon-reload`. You should now be able to successfully start these services (e.g., `sudo systemctl start yocto-*`). The controller may take up to 15 seconds to start. +Finally, it is time to start the Autobuilder. There are two ways to do this: + +1. As the pokybuild3 user, run the following: + +``` +yocto-autobuilder-helper/janitor/ab-janitor& +buildbot start yocto-controller +buildbot-worker start yocto-worker +``` + +2. As root, add the `yocto-*.service` files to `/lib/systemd/system` (See Appendix A). Run: `systemctl daemon-reload`. You should now be able to successfully start these services (e.g., `sudo systemctl start yocto-*`). The controller may take up to 15 seconds to start. ### 1.1) Configuring the Worker's Hash Equivalency Server @@ -112,6 +131,91 @@ sudo /home/pokybuild3/yocto-worker/qemuarm/build/scripts/runqemu-gen-tapdevs \ In the above command, we assume the a build named qemuarm failed. The value of 8 is the number of tap interfaces to create on the worker. +### 1.3) Adding Dedicated Worker Nodes + +Running both the controller and the worker together on a single machine +can quickly result in long build times and an unresponsive web UI, +especially if you plan on running any of the more comprehensive builders +(e.g. a-full). Additional workers can be added to the cluster by +following the steps in Section 1, except that the yocto-controller steps +do not need to be repeated. For example, to add a new worker +"ala-blade51" to an Autobuilder cluster with a yocto-controller at the +IP address 147.11.105.72: + +1. On the yocto-controller host, add the name of the new worker to a worker +list (or create a new one) e.g. 'workers_wrlx = ["ala-blade51"]' and +make sure that it is added to the "workers" list. + +2. On the new worker node: + +``` +
Re: [yocto] bitbake controlling memory use
On 2021-06-05 9:35 a.m., Gmane Admin wrote: [Please note: This e-mail is from an EXTERNAL e-mail address] Op 14-04-2021 om 06:59 schreef Richard Purdie: On Tue, 2021-04-13 at 21:14 -0400, Randy MacLeod wrote: On 2021-04-11 12:19 p.m., Alexander Kanavin wrote: make already has -l option for limiting new instances if load average is too high, so it's only natural to add a RAM limiter too. -l [N], --load-average[=N], --max-load[=N] Don't start multiple jobs unless load is below N. In any case, patches welcome :) During today's Yocto technical call (1), we talked about approaches to limiting the system load and avoiding swap and/or OOM events. Here's what (little!) i recall from the discussion, 9 busy hours later. In the short run, instead of independently maintaining changes to configurations to limit parallelism or xz memory usage, etc, we could develop an optional common include file where such limits are shared across the community. In the longer run, changes to how bitbake schedules work may be needed. Richard says that there was a make/build server idea and maybe even a patch from a while ago. It may be in one of his poky-contrib branches. I took a few minutes to look but nothing popped up. A set of keywords to search for might help me find it. http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wipqueue4&id=d66a327fb6189db5de8bc489859235dcba306237 Cheers, Richard I like the idea. Unfortunately the patch doesn't apply to Gatesgarth, so I couldn't test it. Any chance you would be doing a refresh? I have reworked the patch and I'm doing some testing with it right now. Once I have collected some data (and possibly reworked it further, depending on results), perhaps I can have you test it out as well? That should be in the next day or two. - Trevor -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#53796): https://lists.yoctoproject.org/g/yocto/message/53796 Mute This Topic: https://lists.yoctoproject.org/mt/82015730/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] bitbake controlling memory use
On 2021-06-07 3:27 p.m., Gmane Admin wrote: [Please note: This e-mail is from an EXTERNAL e-mail address] Op 05-06-2021 om 15:35 schreef Gmane Admin: Op 14-04-2021 om 06:59 schreef Richard Purdie: On Tue, 2021-04-13 at 21:14 -0400, Randy MacLeod wrote: On 2021-04-11 12:19 p.m., Alexander Kanavin wrote: make already has -l option for limiting new instances if load average is too high, so it's only natural to add a RAM limiter too. -l [N], --load-average[=N], --max-load[=N] Don't start multiple jobs unless load is below N. In any case, patches welcome :) During today's Yocto technical call (1), we talked about approaches to limiting the system load and avoiding swap and/or OOM events. Here's what (little!) i recall from the discussion, 9 busy hours later. In the short run, instead of independently maintaining changes to configurations to limit parallelism or xz memory usage, etc, we could develop an optional common include file where such limits are shared across the community. In the longer run, changes to how bitbake schedules work may be needed. Richard says that there was a make/build server idea and maybe even a patch from a while ago. It may be in one of his poky-contrib branches. I took a few minutes to look but nothing popped up. A set of keywords to search for might help me find it. http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wipqueue4&id=d66a327fb6189db5de8bc489859235dcba306237 This patch resolves a starvation of a particular resource (execution cores), which is good. However, the problem I am facing is starvation of another resource (memory). Cheers, Richard I like the idea. Unfortunately the patch doesn't apply to Gatesgarth, so I couldn't test it. Any chance you would be doing a refresh? Ok so I refreshed this patch my self and it seems to be working nicely (3000 out of 4000 tasks complete), except for one thing: do_configure for cmake-native fails and I don't see why. From the log: loading initial cache file xx/out/linux64/build/tmp/work/x86_64-linux/cmake-native/3.18.2-r0/build/Bootstrap.cmk/InitialCacheFlags.cmake -- The C compiler identification is GNU 10.3.0 -- The CXX compiler identification is GNU 10.3.0 -- Detecting C compiler ABI info CMake Error: Generator: execution of make failed. Make command was: xx/out/linux64/poky/scripts/make-intercept/make cmTC_68352/fast && -- Detecting C compiler ABI info - failed -- Check for working C compiler: xx/out/linux64/build/tmp/hosttools/gcc CMake Error: Generator: execution of make failed. Make command was: xx/out/linux64/poky/scripts/make-intercept/make cmTC_f23a0/fast && -- Check for working C compiler: xx/out/linux64/build/tmp/hosttools/gcc - broken CMake Error at Modules/CMakeTestCCompiler.cmake:66 (message): The C compiler "xx/out/linux64/build/tmp/hosttools/gcc" is not able to compile a simple test program. It fails with the following output: Change Dir: xx/tmp/work/x86_64-linux/cmake-native/3.18.2-r0/build/CMakeFiles/CMakeTmp Run Build Command(s):xx/out/linux64/poky/scripts/make-intercept/make cmTC_f23a0/fast && Permission denied Generator: execution of make failed. Make command was: xx/out/linux64/poky/scripts/make-intercept/make cmTC_f23a0/fast && CMake will not be able to correctly generate this project. Call Stack (most recent call first): CMakeLists.txt:7 (project) Crazy. I don't see why making a complete recipe works fine, while making a test program during configure fails. Ideas? When I encountered this failure with the patch, it was because the scripts/make-intercept/make script was not marked as executable. However, there was another failure even after that was changed, claiming that an appropriate Makefile parser was not found on the system, which (strangely) seems to be fixed by correcting the script's hashbang to use python3 instead of python. Please see my other response in this thread for further action. - Trevor -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#53798): https://lists.yoctoproject.org/g/yocto/message/53798 Mute This Topic: https://lists.yoctoproject.org/mt/82015730/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] bitbake controlling memory use
On 2021-06-10 5:22 a.m., Ferry Toth wrote: **[Please note: This e-mail is from an EXTERNAL e-mail address] Hi Trevor, Gmane is really messing things up here, sorry about that. I need to create a new thread I'm afraid. I'd like to your reworked patch. But note, I reworked it too (but maybe wrongly). I builds like 90% of my image, but fails building cmake-native. Or more accurately it fails do_configure while trying to build a small test program. Hi, I've pushed the patch onto my fork of the poky repo at https://github.com/threexc/poky Let me know how your testing turns out - I am still running tests as well, but it would be good to know how others' attempts turn out, and more changes could still end up being necessary. - Trevor Ferry -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#53829): https://lists.yoctoproject.org/g/yocto/message/53829 Mute This Topic: https://lists.yoctoproject.org/mt/82015730/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper][PATCH] config.json: track system load with PARALLEL_MAKE
This adds the "-l" option to PARALLEL_MAKE in config.json with an initial testing value of 100 (100% system load). This option is supported by both Make and Ninja. However, we also require the "--debug=j" option to be passed to Make in order for the latter to report perceived system load in the do_compile logs, and since this option is not supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if the target load percentage needs to be adjusted. Signed-off-by: Trevor Gamblin --- config.json | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/config.json b/config.json index f54081b..7fc89ea 100644 --- a/config.json +++ b/config.json @@ -44,7 +44,7 @@ "PREMIRRORS = ''", "BB_GENERATE_MIRROR_TARBALLS = '1'", "BB_NUMBER_THREADS = '16'", -"PARALLEL_MAKE = '-j 16'", +"PARALLEL_MAKE = '-j 16 -l 100'", "XZ_MEMLIMIT = '5%'", "XZ_THREADS = '8'", "BB_TASK_NICE_LEVEL = '5'", @@ -61,7 +61,8 @@ "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'", "BB_HEARTBEAT_EVENT = '60'", "BB_LOG_HOST_STAT_ON_INTERVAL = '1'", -"BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'" +"BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'", +"EXTRA_OEMAKE = ' --debug=j'" ] }, "templates" : { -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54096): https://lists.yoctoproject.org/g/yocto/message/54096 Mute This Topic: https://lists.yoctoproject.org/mt/84151721/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Yocto Autobuilder: Latency Monitor and AB-INT - Meeting notes: July 8, 2021
On 2021-07-08 9:53 a.m., Randy MacLeod wrote: YP AB Intermittent failures meeting === July 1, 2021, 9 AM ET https://windriver.zoom.us/j/3696693975 Attendees: Tony, Richard, Trevor, Randy Summary: The autobuilder RCU hang is still fixed ;-), master branch builds greener. ptest failures are the top problem now. Add Michael Halstead, see questions below in section 4. If anyone wants to help, we could use more eyes on the logs, particularly the summary logs and understanding iostat # when the dd test times out. Plans for the week: === Richard: glibc upgrade, etc. Alex: ? Sakib: pub/non-release link upgrade, script clean-up. Trevor: make job server test. Try it on YP AB!!! What type of build? Tony: fix/work-around valgrind ptest bug: none/tests/amd64/fb_test_amd64 Saul: nothing this week for YP. Randy: vacation, then email catch-up! Meeting Notes: == 1, runqemu Tony having trouble with runqemu on some Wind River machine. Richard has a fix for a race in runqemu in master-next. These might be related but if not Tony should debug the issue/ collect logs. 2. job server - Trevor has conclusive evidence that the 'make' job server is useful. email summary to come. Need to fix some assumptoins in code that parses PARALLEL_MAKE then send patch to yocto-autobuilder-helper. - ninja will have to be done next. I've submitted an initial patch to yocto@lists.yoctoproject.org. My short notes to go along with it: MAKE JOBSERVERS, SYSTEM LOADS, AND THE YOCTO AUTOBUILDER INTRO - Yocto Autobuilder has been experiencing several issues with nightly builds, some intermittent - Previous discussions looking for ways to limit CPU/RAM loading (see " [yocto] bitbake controlling memory use" on yocto@lists.yoctoproject.org mailing list) - Source packages built as part of YP builds rely on various build tools, including Make, Ninja. Can we use them to limit system resource loading, and therefore reduce build failures that occur near/beyond these limits? MAKE JOBSERVER - When the "-j" option is provided (e.g. by PARALLEL_MAKE and/or EXTRA_OEMAKE in local.conf), this tells Make to use that many job slots, e.g. PARALLEL_MAKE = "-j 10" tells make to use at most 10 job slots during compilation - When the "-l" or "--max-load" option is provided (also via PARALLEL_MAKE or EXTRA_OEMAKE in local.conf), this tells Make not to start *more than one job* if the system load average (as perceived by Make) exceeds that value. If the system load exceeds the average specified with the "-l" argument and there is already a Make job running, no new ones are started until either the existing jobs finish, or the system load falls below the threshold. - No limit by default Sources: https://www.gnu.org/software/make/manual/make.html#Parallel http://make.mad-scientist.net/papers/jobserver-implementation/ PROPOSED SOLUTION - "Just do it" - put a patch in for use with the Autobuilder and see what shakes out - add 'PARALLEL_MAKE = "-j N -l M"' to the yocto-autobuilder-helper config.json default - Complicated slightly by Ninja not supporting the same "--debug" functionality as Make - this is what has been used to track the system load in the initial testing. To get around this, we can temporarily set EXTRA_OEMAKE. Without "--debug", the "-l" option still works, but there is no output in the do_compile logs to tell us how the actual system load compares to the percentage targeted by the "-l" option I did not see any issues in my manual testing with PARALLEL_MAKE and EXTRA_OEMAKE set in the same manner. It appears from a recursive grep through the poky repository that PARALLEL_MAKE gets referenced in various places, but it is usually either passed directly to the build tool of choice, or the argument to the "-j" flag is explicitly stripped out for use. An example of the former is meta/classes/meson.bbclass: ... meson_do_compile() { ninja -v ${PARALLEL_MAKE} } ... It's possible that we will run into some edge cases of course, but I haven't seen them in my manual testing. Builds are running now at https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/1478 , so we should have some initial results soon. - Trevor 3. AB status generally better but... ptests are having some recurring problems. - parted - only on arm? - valgrind - none/tests/amd64/fb_test_amd64 - gdb test failing again. - Randy! 4. Richard reported - something really flaky going on with serial ports. - particularly bad on qemuppc but also x86. - related to Saul's QMP data dump? 5. Sakib needs to send patch to make testimage failures generate summary logs. 6. Richard says that we may need to redesign the data collection system that Sakib's AB INT tests are based on. Still relevant parts of Previous Meeting Notes: === 1. The qemu RCU hang h
Re: [yocto] [yocto-autobuilder-helper][PATCH] config.json: track system load with PARALLEL_MAKE
On 2021-07-12 12:04 p.m., Alexander Kanavin wrote: **[Please note: This e-mail is from an EXTERNAL e-mail address] I seem to vaguely remember that -l is not actually taking a percentage, but an absolute value that is specific to the amount of CPU cores a system has. Have you verified your assumption? You are right that it's not actually a percentage and is instead tied to the number of cores. I'll correct the patch body to reflect this when I inevitably send a v2. With the latest testing on the AB, my suspicion that "-l 100" is too generous for most machines is being confirmed... Alex On Mon, 12 Jul 2021 at 14:11, Trevor Gamblin mailto:trevor.gamb...@windriver.com>> wrote: This adds the "-l" option to PARALLEL_MAKE in config.json with an initial testing value of 100 (100% system load). This option is supported by both Make and Ninja. However, we also require the "--debug=j" option to be passed to Make in order for the latter to report perceived system load in the do_compile logs, and since this option is not supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if the target load percentage needs to be adjusted. Signed-off-by: Trevor Gamblin mailto:trevor.gamb...@windriver.com>> --- config.json | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/config.json b/config.json index f54081b..7fc89ea 100644 --- a/config.json +++ b/config.json @@ -44,7 +44,7 @@ "PREMIRRORS = ''", "BB_GENERATE_MIRROR_TARBALLS = '1'", "BB_NUMBER_THREADS = '16'", - "PARALLEL_MAKE = '-j 16'", + "PARALLEL_MAKE = '-j 16 -l 100'", "XZ_MEMLIMIT = '5%'", "XZ_THREADS = '8'", "BB_TASK_NICE_LEVEL = '5'", @@ -61,7 +61,8 @@ "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'", "BB_HEARTBEAT_EVENT = '60'", "BB_LOG_HOST_STAT_ON_INTERVAL = '1'", - "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh <https://urldefense.com/v3/__http://oe-time-dd-test.sh__;!!AjveYdw8EvQ!IMv8JyMfE5mhi_q22gXmTjUg883rq2tckayxOn-9FWThX9U5fJRJRkJYL6L8m9A1NvmBBg$> 100'" + "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh <https://urldefense.com/v3/__http://oe-time-dd-test.sh__;!!AjveYdw8EvQ!IMv8JyMfE5mhi_q22gXmTjUg883rq2tckayxOn-9FWThX9U5fJRJRkJYL6L8m9A1NvmBBg$> 100'", + "EXTRA_OEMAKE = ' --debug=j'" ] }, "templates" : { -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54101): https://lists.yoctoproject.org/g/yocto/message/54101 Mute This Topic: https://lists.yoctoproject.org/mt/84151721/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper][PATCH] config.json: set max load in PARALLEL_MAKE
Add "-l 52" to PARALLEL_MAKE in config.json to limit Make and Ninja builds based on the detected system load. With this option added, if either tool has at least one job running and detects that the system load exceeds the given value, it will wait until either the system load average drops below that limit, or until all other jobs are finished before starting additional jobs. Since most autobuilder machines have 56 cores, this should help keep the system from being overloaded during builds. Reference: https://www.gnu.org/software/make/manual/html_node/Parallel.html Signed-off-by: Trevor Gamblin --- config.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config.json b/config.json index 7be0df7..3709b40 100644 --- a/config.json +++ b/config.json @@ -44,7 +44,7 @@ "PREMIRRORS = ''", "BB_GENERATE_MIRROR_TARBALLS = '1'", "BB_NUMBER_THREADS = '16'", -"PARALLEL_MAKE = '-j 16'", +"PARALLEL_MAKE = '-j 16 -l 52'", "XZ_MEMLIMIT = '5%'", "XZ_THREADS = '8'", "BB_TASK_NICE_LEVEL = '5'", -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54129): https://lists.yoctoproject.org/g/yocto/message/54129 Mute This Topic: https://lists.yoctoproject.org/mt/84225849/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper][dunfell][PATCH] config.json: track system load with PARALLEL_MAKE
Backport for dunfell since we have been running for ~1 week with this patch in master and the same load limiting may help reduce build failures for dunfell itself as well as when its builds are running alongside other branches. This adds the "-l" option to PARALLEL_MAKE in config.json with an initial testing value of 50 (the previous iteration of this patch both had it set to 100 and mistakenly stated that this was a percentage, rather than an absolute representation of system load ala top/htop). This option is supported by both Make and Ninja. However, we also require the "--debug=j" option to be passed to Make in order for the latter to report perceived system load in the do_compile logs, (which we need in order to track test values), and since this option is not supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if the target load percentage needs to be adjusted. Signed-off-by: Trevor Gamblin --- config.json | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/config.json b/config.json index 3cff363..8323864 100644 --- a/config.json +++ b/config.json @@ -43,7 +43,7 @@ "PREMIRRORS = ''", "BB_GENERATE_MIRROR_TARBALLS = '1'", "BB_NUMBER_THREADS = '16'", -"PARALLEL_MAKE = '-j 16'", +"PARALLEL_MAKE = '-j 16 -l 50'", "XZ_MEMLIMIT = '5%'", "XZ_THREADS = '8'", "BB_TASK_NICE_LEVEL = '5'", @@ -56,7 +56,8 @@ "SDK_EXT_TYPE = 'minimal'", "SDK_INCLUDE_TOOLCHAIN = '1'", "BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,100K ABORT,${TMPDIR},100M,1K ABORT,${DL_DIR},100M ABORT,${SSTATE_DIR},100M ABORT,/tmp,10M,1K'", -"BB_HASHSERVE = 'typhoon.yocto.io:8686'" +"BB_HASHSERVE = 'typhoon.yocto.io:8686'", +"EXTRA_OEMAKE = ' --debug=j'" ] }, "templates" : { -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54192): https://lists.yoctoproject.org/g/yocto/message/54192 Mute This Topic: https://lists.yoctoproject.org/mt/84378808/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper][gatesgarth][PATCH] config.json: track system load with PARALLEL_MAKE
Backport for gatesgarth since we have been running for ~1 week with this patch in master and the same load limiting may help reduce build failures for gatesgarth itself as well as when its builds are running alongside other branches. This adds the "-l" option to PARALLEL_MAKE in config.json with an initial testing value of 50 (the previous iteration of this patch both had it set to 100 and mistakenly stated that this was a percentage, rather than an absolute representation of system load ala top/htop). This option is supported by both Make and Ninja. However, we also require the "--debug=j" option to be passed to Make in order for the latter to report perceived system load in the do_compile logs, (which we need in order to track test values), and since this option is not supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if the target load percentage needs to be adjusted. Signed-off-by: Trevor Gamblin --- config.json | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/config.json b/config.json index 5dda653..8a4343a 100644 --- a/config.json +++ b/config.json @@ -43,7 +43,7 @@ "PREMIRRORS = ''", "BB_GENERATE_MIRROR_TARBALLS = '1'", "BB_NUMBER_THREADS = '16'", -"PARALLEL_MAKE = '-j 16'", +"PARALLEL_MAKE = '-j 16 -l 50'", "XZ_MEMLIMIT = '5%'", "XZ_THREADS = '8'", "BB_TASK_NICE_LEVEL = '5'", @@ -56,7 +56,8 @@ "SDK_EXT_TYPE = 'minimal'", "SDK_INCLUDE_TOOLCHAIN = '1'", "BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,100K ABORT,${TMPDIR},100M,1K ABORT,${DL_DIR},100M ABORT,${SSTATE_DIR},100M ABORT,/tmp,10M,1K'", -"BB_HASHSERVE = 'typhoon.yocto.io:8686'" +"BB_HASHSERVE = 'typhoon.yocto.io:8686'", +"EXTRA_OEMAKE = ' --debug=j'" ] }, "templates" : { -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54193): https://lists.yoctoproject.org/g/yocto/message/54193 Mute This Topic: https://lists.yoctoproject.org/mt/84378840/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper][hardknott][PATCH] config.json: track system load with PARALLEL_MAKE
Backport for hardknott since we have been running for ~1 week with this patch in master and the same load limiting may help reduce build failures for hardknott itself as well as when its builds are running alongside other branches. This adds the "-l" option to PARALLEL_MAKE in config.json with an initial testing value of 50 (the previous iteration of this patch both had it set to 100 and mistakenly stated that this was a percentage, rather than an absolute representation of system load ala top/htop). This option is supported by both Make and Ninja. However, we also require the "--debug=j" option to be passed to Make in order for the latter to report perceived system load in the do_compile logs, (which we need in order to track test values), and since this option is not supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if the target load percentage needs to be adjusted. Signed-off-by: Trevor Gamblin --- config.json | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/config.json b/config.json index 110b0d9..5689cb9 100644 --- a/config.json +++ b/config.json @@ -43,7 +43,7 @@ "PREMIRRORS = ''", "BB_GENERATE_MIRROR_TARBALLS = '1'", "BB_NUMBER_THREADS = '16'", -"PARALLEL_MAKE = '-j 16'", +"PARALLEL_MAKE = '-j 16 -l 50'", "XZ_MEMLIMIT = '5%'", "XZ_THREADS = '8'", "BB_TASK_NICE_LEVEL = '5'", @@ -57,7 +57,8 @@ "SDK_INCLUDE_TOOLCHAIN = '1'", "BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,100K ABORT,${TMPDIR},100M,1K ABORT,${DL_DIR},100M ABORT,${SSTATE_DIR},100M ABORT,/tmp,10M,1K'", "BB_HASHSERVE = 'typhoon.yocto.io:8686'", -"RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'" +"RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'", +"EXTRA_OEMAKE = ' --debug=j'" ] }, "templates" : { -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54194): https://lists.yoctoproject.org/g/yocto/message/54194 Mute This Topic: https://lists.yoctoproject.org/mt/84379026/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 7/29/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Diane, Joshua, MichaelO, Randy, Richard, Stephen, Steve, Tim, Tony, TrevorG *ARs:* - Tim to talk with Intel team about bug 14491 ([QA 3.4_M2.rc1] - stap.StapTest.test_stap failure) - Randy to discuss bug (save which recipe fetches which source) with Robert and come to a conclusion *Notes:* * * - Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance *Medium+ 3.4 Unassigned Enhancements/Bugs: *80 (Last week 73) *Medium+ 3.99 Unassigned Enhancements/Bugs: *28**(Last week 27) *AB-INT Bugs: *46 (Last week 49) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54258): https://lists.yoctoproject.org/g/yocto/message/54258 Mute This Topic: https://lists.yoctoproject.org/mt/84530185/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 8/5/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Diane, Joshua, MichaelO, Randy, Richard, Saul, Stephen, Steve, Tim, Tony, TrevorG *ARs:* N/A *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - No Future/3.99 bugs to triage this week (wow!) *Medium+ 3.4 Unassigned Enhancements/Bugs: *80 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *30**(Last week 28) *AB-INT Bugs: *46 (No change) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54327): https://lists.yoctoproject.org/g/yocto/message/54327 Mute This Topic: https://lists.yoctoproject.org/mt/84686898/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 8/19/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Armin, Bruce, Diane, Joshua, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Tony, TrevorG *ARs:* N/A *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *76 (Last week 80) *Medium+ 3.99 Unassigned Enhancements/Bugs: *36**(Last week 30) *AB-INT Bugs: *46 (No change) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54514): https://lists.yoctoproject.org/g/yocto/message/54514 Mute This Topic: https://lists.yoctoproject.org/mt/84999458/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 8/26/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Armin, Bruce, Joshua, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* N/A *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *77 (Last week 76) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 36) *AB-INT Bugs: *48 (Last week 46) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54573): https://lists.yoctoproject.org/g/yocto/message/54573 Mute This Topic: https://lists.yoctoproject.org/mt/85164281/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 9/2/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Bruce, Joshua, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* N/A *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *77 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(No change) *AB-INT Bugs: *48 (No change) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54641): https://lists.yoctoproject.org/g/yocto/message/54641 Mute This Topic: https://lists.yoctoproject.org/mt/85330365/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2][PATCH] add prioritizeBuilders
This prioritizeBuilders function sorts builders by the length of their associated worker lists, so that builders that can only be assigned to a small number of workers are assigned to those workers before other builds that don't have specific needs when resources are limited. An example might be when a slot is available on an Ubuntu-based worker, and "oe-selftest-ubuntu" and "genericx86-64" build requests exist in the queue. Since oe-selftest-ubuntu requires an Ubuntu-based worker and genericx86-64 does not, genericx86-64 will be assigned a higher value (lower priority) so that oe-selftest-ubuntu is assigned to that worker first. Signed-off-by: Trevor Gamblin --- builders.py | 13 + master.cfg | 6 ++ 2 files changed, 19 insertions(+) diff --git a/builders.py b/builders.py index 53c5f0e..94fb561 100644 --- a/builders.py +++ b/builders.py @@ -135,6 +135,19 @@ for builder in config.subbuilders: workernames=workers, nextWorker=nextWorker, nextBuild=nextBuild, factory=f, env=extra_env)) +# prioritize assigning builders to available workers based on the length +# of the worker lists they are associated with. Builders that have fewer +# valid worker options should always be assigned first +def prioritizeBuilders(buildmaster, builders): +# re-use the builder_to_workers list +builder_to_workers = config.builder_to_workers + +# sort builders by the length of their worker lists. Since not all +# builders are explicitly listed in builder_to_workers, make sure to +# default to the len() of the "default" value +builders.sort(key=lambda b: len(builder_to_workers.get(b.name)) if b.name in builder_to_workers.keys() else len(builder_to_workers.get("default"))) +return builders + def create_parent_builder_factory(buildername, waitname): factory = util.BuildFactory() # NOTE: Assumes that yocto-autobuilder repo has been cloned to home diff --git a/master.cfg b/master.cfg index a7c151f..4f7d74e 100644 --- a/master.cfg +++ b/master.cfg @@ -88,6 +88,12 @@ c['www'] = www.www # These items are specific to an individual AB deployment c['workers'] = workers.workers +# This enables our prioritizeBuilders function from builders.py. +# Builders such as buildperf-centos7, buildperf-ubuntu1604, +# oe-selftest-*, and reproducible-* will be assigned (if possible) +# before other builders since their possible worker lists are smaller +c['prioritizeBuilders'] = builders.prioritizeBuilders + c['title'] = "Yocto Autobuilder" c['titleURL'] = "https://autobuilder.yoctoproject.org/main/"; # visible location for internal web server -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54683): https://lists.yoctoproject.org/g/yocto/message/54683 Mute This Topic: https://lists.yoctoproject.org/mt/85483506/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 9/9/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Bruce, Diane, Kiran, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Randy to move AB defects to M4 after call - Tim to file a bug about improving ptest runner timeout warnings - Stephen to handle 3.4 M3 unassigned bugs (move to M4 or 3.5 M1 if enhancements) - Trevor to ask Alex Kanavin if he could help on 14518 - General reminder to review Old Milestone bugs and move as necessary *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *77 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(No change) *AB-INT Bugs: *49 (Last week 48) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54685): https://lists.yoctoproject.org/g/yocto/message/54685 Mute This Topic: https://lists.yoctoproject.org/mt/85486459/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 9/16/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Diane, Jon, Joshua, Michael, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Trevor to move old AB defects to M4 after call - Trevor to ping WR folks about moving Old Milestone bugs - Richard to add a comment to 7298 *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *68 (Last week 77) *Medium+ 3.5 Unassigned Enhancements/Bugs: *10 (new) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(No change) *AB-INT Bugs: *49 (Last week 48) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54741): https://lists.yoctoproject.org/g/yocto/message/54741 Mute This Topic: https://lists.yoctoproject.org/mt/85654473/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Minutes: Yocto Project Weekly Triage Meeting 9/16/2021
On 2021-09-16 11:33 a.m., Trevor Gamblin wrote: *Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Diane, Jon, Joshua, Michael, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Trevor to move old AB defects to M4 after call - Trevor to ping WR folks about moving Old Milestone bugs - Richard to add a comment to 7298 *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *68 (Last week 77) *Medium+ 3.5 Unassigned Enhancements/Bugs: *10 (new) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(No change) *AB-INT Bugs: *49 (Last week 48) Correction: AB-INT bug count is at 52. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54742): https://lists.yoctoproject.org/g/yocto/message/54742 Mute This Topic: https://lists.yoctoproject.org/mt/85654473/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Minutes: Yocto Project Weekly Triage Meeting 9/16/2021
On 2021-09-16 11:33 a.m., Trevor Gamblin wrote: On 2021-09-16 11:33 a.m., Trevor Gamblin wrote: *Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Diane, Jon, Joshua, Michael, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Trevor to move old AB defects to M4 after call - Trevor to ping WR folks about moving Old Milestone bugs I've moved Old Milestone AB-INT defects to M4 and pinged the group internally. - Richard to add a comment to 7298 *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *68 (Last week 77) *Medium+ 3.5 Unassigned Enhancements/Bugs: *10 (new) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(No change) *AB-INT Bugs: *49 (Last week 48) Correction: AB-INT bug count is at 52. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54761): https://lists.yoctoproject.org/g/yocto/message/54761 Mute This Topic: https://lists.yoctoproject.org/mt/85654473/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 9/23/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Bruce, Kiran, Randy, Richard, Ross, Saul, Stephen, Steve, Trevor *ARs:* - Steve to review failure logs for 14557 and post any that might be useful - Richard to have a look at bug 11801 *Notes:* * * - (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this! *Medium+ 3.4 Unassigned Enhancements/Bugs: *68 (No change) *Medium+ 3.5 Unassigned Enhancements/Bugs: *10 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(Last week 38) *AB-INT Bugs: *54 (Last week 49) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54812): https://lists.yoctoproject.org/g/yocto/message/54812 Mute This Topic: https://lists.yoctoproject.org/mt/85816965/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 9/30/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alex, Diane, Joshua, Michael, Oleksiy, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Trevor to revisit Make parallelization as part of look at bugs 14564 and 14565 (originally encountered errors such as https://errors.yoctoproject.org/Errors/Details/593109/while attempting to add load average checking to dunfell) *Notes:* * * - N/A *Medium+ 3.4 Unassigned Enhancements/Bugs: *69 (Last week 68) *Medium+ 3.5 Unassigned Enhancements/Bugs: *10 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB-INT Bugs: *62 (Last week 54) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54921): https://lists.yoctoproject.org/g/yocto/message/54921 Mute This Topic: https://lists.yoctoproject.org/mt/85976717/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 10/7/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Diane, Joshua, Kiran, Michael, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* *- *David Reyna to update Future/3.99 Toaster bugs ** *Notes:* * * - Stephen and Trevor will both be unavailable for the October 14th meeting *Medium+ 3.4 Unassigned Enhancements/Bugs: *68 (Last week 69) *Medium+ 3.5 Unassigned Enhancements/Bugs: *12 (Last week 10) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB Bugs: *64 (Last week 62) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#54985): https://lists.yoctoproject.org/g/yocto/message/54985 Mute This Topic: https://lists.yoctoproject.org/mt/86148326/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 10/21/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alexandre, Anuj, Bruce, Daiane, Joshua, Kiran, Michael, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* *- *Daiane to take ownership 14605 when a Bugzilla account is sorted out - Randy to move 3.4 AB-INT/Medium+ bugs to 3.5 M2 - Everyone to review Old Milestone bugs and move to 3.5 milestones as necessary *Notes:* *Medium+ 3.4 Unassigned Enhancements/Bugs: *70 (Last week 68) *Medium+ 3.5 Unassigned Enhancements/Bugs: *13 (Last week 12) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB Bugs: *63 (Last week 65) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55113): https://lists.yoctoproject.org/g/yocto/message/55113 Mute This Topic: https://lists.yoctoproject.org/mt/86492499/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 10/28/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alexandre, Anuj, Armin, Joshua, Kiran, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Everyone to review Old Milestone bugs and move to 3.5 milestones as necessary - Richard to add guidance to Bug # 14613 *Notes:* N/A *Medium+ 3.5 Unassigned Enhancements/Bugs: *81 (Last week 13; 3.4 bugs merged to this list) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB Bugs: *62 (Last week 63) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55180): https://lists.yoctoproject.org/g/yocto/message/55180 Mute This Topic: https://lists.yoctoproject.org/mt/86655281/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2][PATCH] builders.py: Add canStartBuild disk space check
We need a way to limit the builds for when a given worker has less than a certain amount of disk space available. This implements a canStartBuild method based on the example in the Buildbot docs and blocks a build if the worker has less than 60GB of disk space available. Unlike the example code, we want the stdout of the command so that we can calculate the amount of disk space, rather than just relying on the remote command's return code. Docs: https://docs.buildbot.net/latest/manual/customization.html#canstartbuild-functions [YOCTO #14591] Signed-off-by: Trevor Gamblin --- builders.py | 44 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/builders.py b/builders.py index 5773950..0d1facc 100644 --- a/builders.py +++ b/builders.py @@ -8,6 +8,7 @@ from yoctoabb import config from yoctoabb.steps.writelayerinfo import WriteLayerInfo from yoctoabb.steps.runconfig import get_publish_dest, get_publish_resultdir, get_publish_name, RunConfigCheckSteps, TargetPresent from buildbot.process.results import Results, SUCCESS, FAILURE, CANCELLED, WARNINGS, SKIPPED, EXCEPTION, RETRY +from buildbot.process.remotecommand import RemoteCommand from twisted.python import log from twisted.internet import defer @@ -45,6 +46,41 @@ def ensure_props_set(props): "publish_destination": props.getProperty("publish_destination", "") } +@defer.inlineCallbacks +def shell(command, worker, builder): +args = { +'command': command, +'workdir': worker.worker_basedir, +'logEnviron': False, +'want_stdout': True, +'want_stderr': False, +} + +cmd = RemoteCommand('shell', args, collectStdout=True, stdioLogName="stdio") +cmd.worker = worker +yield cmd.run(None, worker.conn, builder.name) +return cmd + +@defer.inlineCallbacks +def canStartBuild(builder, wfb, request): +log.msg("Checking available disk space...") + +cmd = yield shell("df -BG | grep $(findmnt -T . | awk '{print $2}' | sed -n 2p) | awk '{print $4}' | sed 's/[^0-9]*//g'", wfb.worker, builder) +threshold = 60 # GB of space +if int(cmd.stdout) < threshold: +log.msg("Detected {0} GB of space available, less than threshold of {1} GB. Can't start build".format(cmd.stdout, threshold)) +wfb.worker.putInQuarantine() +return False +else: +log.msg("Detected {0} GB of space available, more than threshold of {1} GB. OK to build".format(cmd.stdout, threshold)) + +wfb.worker.quarantine_timeout = 120 +wfb.worker.putInQuarantine() + +wfb.worker.resetQuarantine() + +return True + def create_builder_factory(): f = util.BuildFactory() @@ -136,7 +172,7 @@ for builder in config.subbuilders: if not workers: workers = config.builder_to_workers['default'] builders.append(util.BuilderConfig(name=builder, - workernames=workers, nextWorker=nextWorker, nextBuild=nextBuild, + workernames=workers, canStartBuild=canStartBuild, nextWorker=nextWorker, nextBuild=nextBuild, factory=f, env=extra_env)) # Prioritize assigning builders to available workers based on the length @@ -300,8 +336,8 @@ def create_parent_builder_factory(buildername, waitname): return factory -builders.append(util.BuilderConfig(name="a-quick", workernames=config.workers, factory=create_parent_builder_factory("a-quick", "wait-quick"), nextWorker=nextWorker, nextBuild=nextBuild, env=extra_env)) -builders.append(util.BuilderConfig(name="a-full", workernames=config.workers, factory=create_parent_builder_factory("a-full", "wait-full"), nextWorker=nextWorker, nextBuild=nextBuild, env=extra_env)) +builders.append(util.BuilderConfig(name="a-quick", workernames=config.workers, factory=create_parent_builder_factory("a-quick", "wait-quick"), canStartBuild=canStartBuild, nextWorker=nextWorker, nextBuild=nextBuild, env=extra_env)) +builders.append(util.BuilderConfig(name="a-full", workernames=config.workers, factory=create_parent_builder_factory("a-full", "wait-full"), canStartBuild=canStartBuild,nextWorker=nextWorker, nextBuild=nextBuild, env=extra_env)) def create_doc_builder_factory(): f = util.BuildFactory() @@ -345,4 +381,4 @@ def create_doc_builder_factory(): # Only run one docs build at a time docs_lock = util.MasterLock("docs_lock") -builders.append(util.BuilderConfig(name="docs", workernames=config.workers, factory=create_doc_builder_factory(), nextWorker=nextWorker, nextBuild=nextBuild, env=extra_env, locks=[docs_lock.access(
[yocto] [yocto-autobuilder2][PATCH] builders.py: fix bonus time calculation
Modify the builder_bonuses calculation so that it provides variable bonus time based on the builder, rather than a constant value. Signed-off-by: Trevor Gamblin --- builders.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builders.py b/builders.py index 5773950..f94d1dd 100644 --- a/builders.py +++ b/builders.py @@ -145,7 +145,7 @@ for builder in config.subbuilders: # Add 2 seconds * length as the weight so tightly constrained builders go first builder_bonuses = {} for builder in config.builder_to_workers: -bonus = (len(config.workers) - len(config.builder_to_workers)) * 2 +bonus = (len(config.workers) - len(config.builder_to_workers[builder])) * 2 builder_bonuses[builder] = timedelta(seconds=bonus) # Modified default algothirm from buildbot with a bonus mechanism (thanks tardyp!) -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55186): https://lists.yoctoproject.org/g/yocto/message/55186 Mute This Topic: https://lists.yoctoproject.org/mt/86679036/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2][PATCH] builders.py: subtract builder weight instead of add
Signed-off-by: Trevor Gamblin --- builders.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builders.py b/builders.py index f94d1dd..fc92e36 100644 --- a/builders.py +++ b/builders.py @@ -168,7 +168,7 @@ def prioritizeBuilders(master, builders): time = max_time else: if bldr.name in builder_bonuses: -time = time + builder_bonuses[bldr.name] +time = time - builder_bonuses[bldr.name] defer.returnValue((time, bldr)) transformed = yield defer.gatherResults( -- 2.31.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55188): https://lists.yoctoproject.org/g/yocto/message/55188 Mute This Topic: https://lists.yoctoproject.org/mt/86679559/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 11/4/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alexandre, Armin, Bruce, Jon, Joshua, Kiran, Michael, Randy, Richard, Stephen, Steve, Tim, Trevor *ARs:* N/A ** *Notes:* N/A *Medium+ 3.5 Unassigned Enhancements/Bugs: *79 (Last week 81) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB Bugs: *62 (No change) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55240): https://lists.yoctoproject.org/g/yocto/message/55240 Mute This Topic: https://lists.yoctoproject.org/mt/86818685/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 11/11/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alexandre, Armin, Bruce, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* N/A *Notes:* N/A *Medium+ 3.5 Unassigned Enhancements/Bugs: *73 (Last week 79) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB Bugs: *62 (No change) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55316): https://lists.yoctoproject.org/g/yocto/message/55316 Mute This Topic: https://lists.yoctoproject.org/mt/86986125/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 11/18/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees:* Alexandre, Armin, Daiane, Jon, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Richard to determine what infrastructure support/changes are needed for i586 tuning (see YP #14632) *Notes:* N/A *Medium+ 3.5 Unassigned Enhancements/Bugs: *73 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(No change) *AB Bugs: *61 (Last week 62) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55361): https://lists.yoctoproject.org/g/yocto/message/55361 Mute This Topic: https://lists.yoctoproject.org/mt/87147021/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 11/25/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Bruce, Michael, Stephen, Randy, Richard, Tim, Trevor *ARs:* N/A *Notes:* No meeting next week - YP Summit *Medium+ 3.5 Unassigned Enhancements/Bugs: *73 (No change) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 39) *AB Bugs: *61 (No change) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55422): https://lists.yoctoproject.org/g/yocto/message/55422 Mute This Topic: https://lists.yoctoproject.org/mt/87305300/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 12/9/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Anuj, Bruce, Daiane, Joshua, Michael, Ross, Saul, Stephen, Steve, Randy, Richard, Tim, Trevor *ARs:* N/A *Notes:* There was no meeting for 12/2/2021 due to Yocto Summit *Medium+ 3.5 Unassigned Enhancements/Bugs: *75 (Last week 73) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *67 (Last week 61) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55536): https://lists.yoctoproject.org/g/yocto/message/55536 Mute This Topic: https://lists.yoctoproject.org/mt/87615375/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 12/16/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Armin, Jon, Joshua, Michael, Randy, Richard, Ryan, Saul, Stephen, Steve, Trevor, Valerii *ARs:* - Randy to bump the M1s in the AB-INT list - Randy to remove AB-INT tempfs from the whiteboard field - Randy to move Medium+ M1s to later milestones - Everyone to review their Old Milestone bugs and move to new milestones or close *Notes:* No triage meeting on December 30th *Medium+ 3.5 Unassigned Enhancements/Bugs: *76 (Last week 75) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *73 (Last week 67) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55601): https://lists.yoctoproject.org/g/yocto/message/55601 Mute This Topic: https://lists.yoctoproject.org/mt/87770500/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 12/23/2021
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Randy, Richard, Saul, Stephen, Steve, Trevor *ARs:* Happy Holidays! *Notes:* No triage meeting on December 30th *Medium+ 3.5 Unassigned Enhancements/Bugs: *75 (Last week 76) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *70 (Last week 73) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55650): https://lists.yoctoproject.org/g/yocto/message/55650 Mute This Topic: https://lists.yoctoproject.org/mt/87920883/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 1/6/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Joshua, Michael, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Joshua to send a patch limiting the size of diffoscope output for reproducibility *Notes:* N/A *Medium+ 3.5 Unassigned Enhancements/Bugs: *75 (Last week 75) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *66 (Last week 70) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55751): https://lists.yoctoproject.org/g/yocto/message/55751 Mute This Topic: https://lists.yoctoproject.org/mt/88241068/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 1/13/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Bruce, Michael, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* N/A *Notes:* - ~50% of AB workers have been switched to SSDs. Still too early to determine what effect they are having on failure rates *Medium+ 3.5 Unassigned Enhancements/Bugs: *78 (Last week 75) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *74 (Last week 66) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55837): https://lists.yoctoproject.org/g/yocto/message/55837 Mute This Topic: https://lists.yoctoproject.org/mt/88400400/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 1/20/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Bruce, Diane, Jate, Joshua, Márton, Michael, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Randy/Stephen to move Medium+ 3.5 unassigned bugs to M3 - Stephen to make a note about prelink for next week's status *Notes:* - ~50% of AB workers have been switched to SSDs. Still too early to determine what effect they are having on failure rates *Medium+ 3.5 Unassigned Enhancements/Bugs: *80 (Last week 78) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *76 (Last week 74) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55934): https://lists.yoctoproject.org/g/yocto/message/55934 Mute This Topic: https://lists.yoctoproject.org/mt/88562784/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 1/27/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Armin, Daiane, Michael, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Randy to take a look AB-INT defects for M2s and update them to M3 - Everyone to review open bugs and move Old Milestone (M2) bugs to M3 or later *Notes:* - ~50% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD *Medium+ 3.5 Unassigned Enhancements/Bugs: *76 (Last week 80) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(Last week 38) *AB Bugs: *76 (Last week 76) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55998): https://lists.yoctoproject.org/g/yocto/message/55998 Mute This Topic: https://lists.yoctoproject.org/mt/88724693/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 2/3/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alejandro, Alexandre, Andrei, Armin, Bruce, Daiane, Michael, Pavel, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Armin/Bruce/Trevor/Saul to move Old Milestone bugs to M3 or later ** *Notes:* - ~50% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD *Medium+ 3.5 Unassigned Enhancements/Bugs: *76 (Last week 76) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(Last week 39) *AB Bugs: *73 (Last week 76) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56060): https://lists.yoctoproject.org/g/yocto/message/56060 Mute This Topic: https://lists.yoctoproject.org/mt/6608/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 2/10/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Andrei, Alexandre, Armin, Bruce, Daiane, Joshua, Pavel, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Saul to address NEEDINFO bugs from Robert Berger - Randy to search Bugzilla for prelink bugs to close *Notes:* - (Correction) ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD *Medium+ 3.5 Unassigned Enhancements/Bugs: *79 (Last week 76) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(Last week 39) *AB Bugs: *75 (Last week 73) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56153): https://lists.yoctoproject.org/g/yocto/message/56153 Mute This Topic: https://lists.yoctoproject.org/mt/89052723/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 2/17/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alejandro, Alexandre, Bruce, Daiane, Jan-Simon, Joshua, Michael, Pavel, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* N/A ** *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD *Medium+ 3.5 Unassigned Enhancements/Bugs: *78 (Last week 79) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(Last week 39) *AB Bugs: *73 (Last week 73) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56215): https://lists.yoctoproject.org/g/yocto/message/56215 Mute This Topic: https://lists.yoctoproject.org/mt/89212970/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 2/24/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Armin, Bruce, Daiane, Joshua, Pavel, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs: * ** - Randy to write an email list to the Yocto mailing lists to encourage users to check for existing bugs and add their input to those before reporting new issues *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD *Medium+ 3.5 Unassigned Enhancements/Bugs: *78 (Last week 78) *Medium+ 3.99 Unassigned Enhancements/Bugs: *39**(Last week 39) *AB Bugs: *75 (Last week 73) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56278): https://lists.yoctoproject.org/g/yocto/message/56278 Mute This Topic: https://lists.yoctoproject.org/mt/89368995/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 3/3/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Armin, Daiane, Joshua, Luca, Pavel, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs: * - Randy to talk to David Reyna about triaging his Toaster Future/3.99 bugs (25) *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: *77 (Last week 71) *Medium+ 3.5 Unassigned Enhancements/Bugs: *6 (new) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 39) *AB Bugs: *71 (Last week 75) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56356): https://lists.yoctoproject.org/g/yocto/message/56356 Mute This Topic: https://lists.yoctoproject.org/mt/89528206/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 3/10/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Joshua, Luca, Michael, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs: * - Randy to move AB, 3.5 Enhancement M3 bugs to M4 or later - All to review Old Milestone bugs and move to M4 or later *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: *73 (Last week 77) *Medium+ 3.6 Unassigned Enhancements/Bugs: *2 (new) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *71 (Last week 75) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56416): https://lists.yoctoproject.org/g/yocto/message/56416 Mute This Topic: https://lists.yoctoproject.org/mt/89689873/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 3/10/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Joshua, Luca, Michael, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs: * - Randy to move AB, 3.5 Enhancement M3 bugs to M4 or later - All to review Old Milestone bugs and move to M4 or later *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: *73 (Last week 77) *Medium+ 3.6 Unassigned Enhancements/Bugs: *2 *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *71 (Last week 75) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56417): https://lists.yoctoproject.org/g/yocto/message/56417 Mute This Topic: https://lists.yoctoproject.org/mt/89689873/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 3/17/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Daiane, Luca, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Randy to move remaining Medium+ M3s to M1 (and move to newcomer bugs category, where appropriate) - Stephen to create an issue for Michael run milestone naming script (3.6 to 4.1 and 3.99 to 4.99) - Everyone to review assigned Old Milestone M3 bugs and move to later milestones *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: * 68 (Last week 73) *Medium+ 3.6 Unassigned Enhancements/Bugs: *10 (Last week 2) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *72 (Last week 71) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56481): https://lists.yoctoproject.org/g/yocto/message/56481 Mute This Topic: https://lists.yoctoproject.org/mt/89847754/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 3/24/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Bruce, Jon, Joshua, Luca, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Randy to ask Sakib to update his Old Milestone bugs ** *Notes:* - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: * 18 (Last week 68) *Medium+ 3.6 Unassigned Enhancements/Bugs: *60 (Last week 10) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *65 (Last week 72) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56551): https://lists.yoctoproject.org/g/yocto/message/56551 Mute This Topic: https://lists.yoctoproject.org/mt/90002717/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 3/31/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Bruce, Luca, Michael, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Richard to triage bug 14770* * *Notes:* - Trevor is leaving WR/YP, so a new note-taker will be required for this meeting - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: * 16 (Last week 18) *Medium+ 3.6 Unassigned Enhancements/Bugs: *60 (Last week 60) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *61 (Last week 65) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56611): https://lists.yoctoproject.org/g/yocto/message/56611 Mute This Topic: https://lists.yoctoproject.org/mt/90157006/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 4/7/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Luca, Randy, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor *ARs:* - Randy to review AB-INT bugs and adjust/move as necessary - Randy to review 3.5+ bugs and move them as necessary - Stephen to update the reminder email with info about bug contributions/assignment ** *Notes:* - Trevor is leaving WR/YP, so a new note-taker will be required for this meeting - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: * 14 (Last week 16) *Medium+ 3.6 Unassigned Enhancements/Bugs: *61 (Last week 60) *Medium+ 3.99 Unassigned Enhancements/Bugs: *38**(Last week 38) *AB Bugs: *64 (Last week 61) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56682): https://lists.yoctoproject.org/g/yocto/message/56682 Mute This Topic: https://lists.yoctoproject.org/mt/90315565/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Minutes: Yocto Project Weekly Triage Meeting 4/14/2022
*Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Alexandre, Bruce, Daiane, Joshua, Luca, Pavel, Randy, Richard, Saul, Stephen, Steve, Tim, Trevor *ARs:* N/A *Notes:* - Trevor has left WR/YP, so a new note-taker will be required for this meeting. It was great working with you all. Take care and good luck on all of your future endeavors. - ~43% of AB workers have been switched to SSDs. Failure rate appears lower, but still TBD. More coming soon! *Medium+ 3.5 Unassigned Enhancements/Bugs: * 3 (Last week 14) *Medium+ 3.6 Unassigned Enhancements/Bugs: *74 (Last week 61) *Medium+ 3.99 Unassigned Enhancements/Bugs: *41**(Last week 38) *AB Bugs: *58 (Last week 64) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#56770): https://lists.yoctoproject.org/g/yocto/message/56770 Mute This Topic: https://lists.yoctoproject.org/mt/90467428/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 0/7] patchtest: update to work with latest Yocto changes
This is the first series of patches intended to help get patchtest up and running again. Specifically, these make core-image-patchtest build and run again (although one of the initscripts still shuts the image down shortly after boot, so testing it may seem confusing). In the longer-term, patchtest may be moved to oe-core, so right now the focus for this repo is to make sure it's in a working state so that it can be cleaned up and prepared for that change.There are some other additions that will be coming soon, namely updates to the pylint version and some updated documentation. In order to start core-image-patchtest with runqemu, I had to invoke it like so: runqemu kvm nographic qemuparams="-snapshot \ -fsdev local,id=test_dev,path=/workspace/yocto/manual/openembedded-core/build/test_share,security_model=mapped \ -device virtio-9p-pci,fsdev=test_dev,mount_tag=test_mount -m 2048" which is similar to the contents of the patchtest/scripts/guest script (but in this case I created the share directory "test_share" inside my build directory). The PACKAGECONFIG for qemu-system-native wasn't being set to properly support the required virtio-9p-pci device, hence the addition of the .bbappend file for that recipe. I've left the corresponding qemu .bbappend in place for now, but it might be removable after more testing. Trevor Gamblin (7): python3-pylint: update to use new override syntax qemu: update to use new override syntax initscripts: update to use new override syntax initscripts: use mkdir before mv with patchtest initscript linux-yocto: update version, fix override syntax qemu-system-native: add virtfs to PACKAGECONFIG README: Update maintainer list README| 1 + .../recipes-core/initscripts/initscripts/patchtest| 1 + .../recipes-core/initscripts/initscripts_1.0.bbappend | 8 .../recipes-devtools/python/python3-pylint_1.6.5.bb | 2 +- .../recipes-devtools/qemu/qemu-system-native_%.bbappend | 1 + meta-patchtest/recipes-devtools/qemu/qemu_%.bbappend | 2 +- .../recipes-kernel/linux/linux-yocto_4.%.bbappend | 2 -- .../recipes-kernel/linux/linux-yocto_6.%.bbappend | 2 ++ 8 files changed, 11 insertions(+), 8 deletions(-) create mode 100644 meta-patchtest/recipes-devtools/qemu/qemu-system-native_%.bbappend delete mode 100644 meta-patchtest/recipes-kernel/linux/linux-yocto_4.%.bbappend create mode 100644 meta-patchtest/recipes-kernel/linux/linux-yocto_6.%.bbappend -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59808): https://lists.yoctoproject.org/g/yocto/message/59808 Mute This Topic: https://lists.yoctoproject.org/mt/98558549/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 3/7] initscripts: update to use new override syntax
Signed-off-by: Trevor Gamblin --- .../recipes-core/initscripts/initscripts_1.0.bbappend | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/meta-patchtest/recipes-core/initscripts/initscripts_1.0.bbappend b/meta-patchtest/recipes-core/initscripts/initscripts_1.0.bbappend index d239865..057921f 100644 --- a/meta-patchtest/recipes-core/initscripts/initscripts_1.0.bbappend +++ b/meta-patchtest/recipes-core/initscripts/initscripts_1.0.bbappend @@ -1,13 +1,13 @@ -FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" +FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:" -SRC_URI_append =" file://patchtest" +SRC_URI:append =" file://patchtest" inherit useradd USERADD_PACKAGES = "${PN}" -USERADD_PARAM_${PN} = "-u 1200 -r -m -s /bin/sh patchtest" +USERADD_PARAM:${PN} = "-u 1200 -r -m -s /bin/sh patchtest" -do_install_append () { +do_install:append () { install -m 0755 ${WORKDIR}/patchtest ${D}${sysconfdir}/init.d update-rc.d -r ${D} patchtest start 99 3 5 . } -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59811): https://lists.yoctoproject.org/g/yocto/message/59811 Mute This Topic: https://lists.yoctoproject.org/mt/98558554/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 4/7] initscripts: use mkdir before mv with patchtest initscript
Signed-off-by: Trevor Gamblin --- meta-patchtest/recipes-core/initscripts/initscripts/patchtest | 1 + 1 file changed, 1 insertion(+) diff --git a/meta-patchtest/recipes-core/initscripts/initscripts/patchtest b/meta-patchtest/recipes-core/initscripts/initscripts/patchtest index 3b075af..1dc434f 100755 --- a/meta-patchtest/recipes-core/initscripts/initscripts/patchtest +++ b/meta-patchtest/recipes-core/initscripts/initscripts/patchtest @@ -53,6 +53,7 @@ if [ -d $REPO ]; then fi # Move results dir to MP as root +mkdir -p $RESULTS mv $RESULTS $MP/results # Poweroff machine -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59812): https://lists.yoctoproject.org/g/yocto/message/59812 Mute This Topic: https://lists.yoctoproject.org/mt/98558555/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 5/7] linux-yocto: update version, fix override syntax
Signed-off-by: Trevor Gamblin --- meta-patchtest/recipes-kernel/linux/linux-yocto_4.%.bbappend | 2 -- meta-patchtest/recipes-kernel/linux/linux-yocto_6.%.bbappend | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) delete mode 100644 meta-patchtest/recipes-kernel/linux/linux-yocto_4.%.bbappend create mode 100644 meta-patchtest/recipes-kernel/linux/linux-yocto_6.%.bbappend diff --git a/meta-patchtest/recipes-kernel/linux/linux-yocto_4.%.bbappend b/meta-patchtest/recipes-kernel/linux/linux-yocto_4.%.bbappend deleted file mode 100644 index 7a718f6..000 --- a/meta-patchtest/recipes-kernel/linux/linux-yocto_4.%.bbappend +++ /dev/null @@ -1,2 +0,0 @@ -FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" -SRC_URI += "file://virtio.cfg" diff --git a/meta-patchtest/recipes-kernel/linux/linux-yocto_6.%.bbappend b/meta-patchtest/recipes-kernel/linux/linux-yocto_6.%.bbappend new file mode 100644 index 000..3a69acb --- /dev/null +++ b/meta-patchtest/recipes-kernel/linux/linux-yocto_6.%.bbappend @@ -0,0 +1,2 @@ +FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:" +SRC_URI += "file://virtio.cfg" -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59813): https://lists.yoctoproject.org/g/yocto/message/59813 Mute This Topic: https://lists.yoctoproject.org/mt/98558556/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 6/7] qemu-system-native: add virtfs to PACKAGECONFIG
Signed-off-by: Trevor Gamblin --- .../recipes-devtools/qemu/qemu-system-native_%.bbappend | 1 + 1 file changed, 1 insertion(+) create mode 100644 meta-patchtest/recipes-devtools/qemu/qemu-system-native_%.bbappend diff --git a/meta-patchtest/recipes-devtools/qemu/qemu-system-native_%.bbappend b/meta-patchtest/recipes-devtools/qemu/qemu-system-native_%.bbappend new file mode 100644 index 000..4e5918b --- /dev/null +++ b/meta-patchtest/recipes-devtools/qemu/qemu-system-native_%.bbappend @@ -0,0 +1 @@ +PACKAGECONFIG:append:class-native = " virtfs" -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59814): https://lists.yoctoproject.org/g/yocto/message/59814 Mute This Topic: https://lists.yoctoproject.org/mt/98558557/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 7/7] README: Update maintainer list
Signed-off-by: Trevor Gamblin --- README | 1 + 1 file changed, 1 insertion(+) diff --git a/README b/README index 8e0ae71..b978855 100644 --- a/README +++ b/README @@ -44,6 +44,7 @@ Maintenance Maintainers: Paul Barker +Trevor Gamblin Links - -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59815): https://lists.yoctoproject.org/g/yocto/message/59815 Mute This Topic: https://lists.yoctoproject.org/mt/98558558/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 2/7] qemu: update to use new override syntax
Signed-off-by: Trevor Gamblin --- meta-patchtest/recipes-devtools/qemu/qemu_%.bbappend | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-patchtest/recipes-devtools/qemu/qemu_%.bbappend b/meta-patchtest/recipes-devtools/qemu/qemu_%.bbappend index 3ad9f03..4e5918b 100644 --- a/meta-patchtest/recipes-devtools/qemu/qemu_%.bbappend +++ b/meta-patchtest/recipes-devtools/qemu/qemu_%.bbappend @@ -1 +1 @@ -PACKAGECONFIG_append_pn-qemu-native = " virtfs" +PACKAGECONFIG:append:class-native = " virtfs" -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59810): https://lists.yoctoproject.org/g/yocto/message/59810 Mute This Topic: https://lists.yoctoproject.org/mt/98558552/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 1/7] python3-pylint: update to use new override syntax
Signed-off-by: Trevor Gamblin --- meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb b/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb index 269983b..0bb9e3f 100644 --- a/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb +++ b/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb @@ -22,7 +22,7 @@ RDEPENDS_${PN} += "${PYTHON_PN}-astroid \ ${PYTHON_PN}-netserver \ " -do_install_append(){ +do_install:append(){ rm ${D}${bindir}/pylint cat >> ${D}${bindir}/pylint < -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59809): https://lists.yoctoproject.org/g/yocto/message/59809 Mute This Topic: https://lists.yoctoproject.org/mt/98558551/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][yocto-autobuilder2][PATCH] README-Guide: fix typos, add hashserv setup
Added some instructions on setting up the hash equivalency server for an autobuilder instance that operates independently of the Yocto one. This all still assumes that the user is planning on running the controller, worker, and janitor on the same build host, although the hash equivalency server itself doesn't need to be. Also made some minor changes to the list of packages to install to support the autobuilder on Ubuntu, since there were some typos in the package names. Signed-off-by: Trevor Gamblin --- README-Guide.md | 42 ++ 1 file changed, 38 insertions(+), 4 deletions(-) diff --git a/README-Guide.md b/README-Guide.md index 70f1433..21dd7c1 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -11,8 +11,8 @@ The final outputs of this section are a controller and worker installed in the s The latest version of BuildBot is written in Python 3, so installation via??pip3: ``` -apt install python-pip3 git build-essential python3-pip virtualenv enchang npm -sudo pip3 install buildbot buildbot-www buildbot-waterfall-view buildbot-console-view buildbot-grid-view +apt install git build-essential python3-pip virtualenv enchant npm +sudo pip3 install buildbot buildbot-www buildbot-waterfall-view buildbot-console-view buildbot-grid-view buildbot-worker ``` It is recommended to also install??`testtools` and??`libccpunit-subunit-dev` (via apt, in this case) on the worker in order??for certain image tests to work correctly (e.g., `core-image-sato-sdk:do_testimage`). @@ -63,7 +63,41 @@ Set??`BASE_HOMEDIR` should be your build user's home directory.?? (There are she Finally, as root,??add the??`yocto-*.service` files to??`/lib/systemd/system` (See Appendix A).?? Run: `systemctl daemon-reload`.?? You should now be able to successfully start these services (e.g., `sudo systemctl start yocto-*`).?? The controller may take up to 15 seconds to start. -### 1.1) Special Notes for the Worker Environment +### 1.1) Configuring the Worker's Hash Equivalency Server + +A key part of the autobuilder setup is the use of a hash equivalency +server, which is started manually on the Yocto hosts. By default, the +config.json file that is pulled along with yocto-autobuilder-helper +contains "BB_HASHSERVE = 'typhoon.yocto.io:8686'", which will likely +cause a custom autobuilder instance to hang while trying to communicate +with the hash equivalency server. As mentioned in the previous section, +any configuration changes made need to be committed to the +yocto-autobuilder-helper repo that your worker will pull from. A simple +way to start the hash equivalency server and properly configure your +workers is done with the following steps: + +1. Clone a copy of the poky distribution repo from + https://git.yoctoproject.org/cgit/cgit.cgi/poky/ +2. Source the oe-init-build-env script into your build directory +3. Run "bitbake-hashserv --bind :" from the build directory +4. Clone a copy of + https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/, + then create config-local.json by copying config.json (and removing + sections that aren't required). Since the contents of config-local.json + are an overlay for those in config.json, this will preserve as much of + the initial configuration as possible. +5. Change the BB_HASHSERVE value in config-local.json to point to the IP + and port you specified in step 3 +6. Commit the changes in the local yocto-autobuilder-helper repository +7. In /yocto-controller/yoctoabb/config.py, change the + entry for "yocto-autobuilder-helper" under "repos" to point at your + local copy of the yocto-autobuilder-helper repository. + +After these steps have been taken, the workers that you start should +attach to your local hash equivalency server and be able to complete a +target build. + +### 1.2) Special Notes for the Worker Environment The QEMU??tap interfaces also need to be generated and owned by the worker's user (created above).?? One way to this is to compile the `meta/recipes-devtools/qemu/qemu-helper/tunctl.c`??file and run it _N_ times on the worker's host.?? See the related `qemu-helper-native` recipe for instructions.?? The resulting executable would be run _N_ times (e.g., 8), so for example:??`sudo??tunctl -u $(id -u pokybuild3) -g $(id -g pokybuild3) -t tun0`. @@ -281,4 +315,4 @@ Group=nogroup [Install] WantedBy=multi-user.target -``` \ No newline at end of file +``` -- 2.25.4 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#49346): https://lists.yoctoproject.org/g/yocto/message/49346 Mute This Topic: https://lists.yoctoproject.org/mt/74006635/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2][PATCH 0/2] Clarification and formatting of README-Guide.md
Long-overdue patches based on experiences trying to set up an autobuilder instance. I've split it into two patches because the second patch (the one that line wraps the majority of the document) may not be desired, if the doc is meant to be read in the pre-existing format. Patch 0001 cleans up the doc so that the sections can be more easily referenced. Trevor Gamblin (2): README-Guide: cleanup, clarify setup instructions README-Guide: wrap lines at 80 characters README-Guide.md | 489 1 file changed, 372 insertions(+), 117 deletions(-) -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#49717): https://lists.yoctoproject.org/g/yocto/message/49717 Mute This Topic: https://lists.yoctoproject.org/mt/75039158/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2][PATCH 1/2] README-Guide: cleanup, clarify setup instructions
Broke the document up into more sections, and added some info to make it clear that the same instructions for setting up the systemd autobuilder can be used to start it manually. Signed-off-by: Trevor Gamblin --- README-Guide.md | 33 ++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/README-Guide.md b/README-Guide.md index 21dd7c1..83e30d0 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -4,6 +4,8 @@ This guide will walk through how to install a stand-alone autobuilder controller ## 1) Stand-alone Installation +### 1.1) Basic Requirements + The final outputs of this section are a controller and worker installed in the same server, ready for trimming back to an individual organization's needs. > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting.?? Please refer to the Yocto manual for those packages. @@ -17,6 +19,8 @@ sudo pip3 install buildbot buildbot-www buildbot-waterfall-view buildbot-console It is recommended to also install??`testtools` and??`libccpunit-subunit-dev` (via apt, in this case) on the worker in order??for certain image tests to work correctly (e.g., `core-image-sato-sdk:do_testimage`). +### 1.2) Creating the Autobuilder User's Directory + For a new installation you will need a system user that can run Autobuilder2 as well as a couple of repositories: Autobuilder2 itself and a helper plugin. As root: ``` @@ -27,11 +31,14 @@ buildbot-worker create-worker -r --umask=0o22 yocto-worker localhost example-wor cd yocto-controller git clone??https://git.yoctoproject.org/git/yocto-autobuilder2??yoctoabb ln -rs yoctoabb/master.cfg master.cfg -cd ~ +cd - git clone??https://git.yoctoproject.org/git/yocto-autobuilder-helper chown -R pokybuild3:nogroup /home/pokybuild3 ``` +Note: depending on the configuration of your system, you may need to +replace "nogroup" with "pokybuild3" for the last command to work. + > **IMPORTANT:** In the above command you created a controller and a worker, which will attempt to join the controller using??`pass` as the password.?? Feel free to change this, knowing that if you do, you must change the controller's master configuration file to match. At the end of this, your build user's home directory (e.g., `/home/pokybuild3`) should look like this: @@ -43,6 +50,8 @@ yocto-controller/yoctoabb yocto-worker ``` +### 1.3) Customizing the Autobuilder Configuration + Next, we need to update the??`yocto-controller/yoctoabb/master.cfg` towards the bottom where the??`title`,??`titleURL`, and??`buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. Then, we need to update the??`yocto-controller/yoctoabb/config.py` to include our worker.?? In that file, find the line where??`workers` is set and add:??["example-worker"]._NOTE:_ if your worker's name is different, use that here.?? Section 3.1 discusses how to further refine this list of workers. @@ -61,9 +70,27 @@ Set??`BASE_HOMEDIR` should be your build user's home directory.?? (There are she > NOTE: The way the build step is written, the worker will pull a fresh copy of the helper from the server. Therefore??these configuration files must be committed to the??`yocto-autobuilder-helper`??repo location you have specified in??`yoctoabb/config.py`??because the worker is given a build step that pulls from that repo (see??`yoctoabb/builders.py`). -Finally, as root,??add the??`yocto-*.service` files to??`/lib/systemd/system` (See Appendix A).?? Run: `systemctl daemon-reload`.?? You should now be able to successfully start these services (e.g., `sudo systemctl start yocto-*`).?? The controller may take up to 15 seconds to start. +### 1.4) Starting the Autobuilder + +At this point, you can start your Autobuilder controller, worker, and +janitor in one of two ways: using buildbot manually, or by creating +systemd service files. + + 1.4.1) Manually Starting the Autobuilder + +Run the following from the pokybuild3 directory: + + 1. yocto-autobuilder-helper/janitor/ab-janitor& + 2. buildbot start yocto-controller + 3. buildbot-worker start yocto-worker + + 1.4.2) Creating Service Files + + 1. As root,??add the??`yocto-*.service` files to??`/lib/systemd/system` (See Appendix A).?? + 2. Run: `systemctl daemon-reload`.?? + 3. You should now be able to successfully start these services (e.g., `sudo systemctl start yocto-*`).??The controller may take up to 15 seconds to start. -### 1.1) Configuring the Worker's Hash Equivalency Server +### 1.5) Configuring the Worker's Hash Equivalency Server A key part of the autobuilder setup is the use of a hash equivalency server, which is started manually on the Yocto hosts. By default, the -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Lin
[yocto] [yocto-autobuilder2][PATCH 2/2] README-Guide: wrap lines at 80 characters
Signed-off-by: Trevor Gamblin --- README-Guide.md | 454 1 file changed, 341 insertions(+), 113 deletions(-) diff --git a/README-Guide.md b/README-Guide.md index 83e30d0..aaefd02 100644 --- a/README-Guide.md +++ b/README-Guide.md @@ -1,14 +1,20 @@ # Installing and Maintaining the Yocto Autobuilder 2 -This guide will walk through how to install a stand-alone autobuilder controller and worker, and then reconfigure it with new builds, etc.?? The guide was written with the help of Richard Purdie under the context of a headless Ubuntu 18.04.02 Server installation. +This guide will walk through how to install a stand-alone autobuilder controller +and worker, and then reconfigure it with new builds, etc.?? The guide was written +with the help of Richard Purdie under the context of a headless Ubuntu 18.04.02 +Server installation. ## 1) Stand-alone Installation ### 1.1) Basic Requirements -The final outputs of this section are a controller and worker installed in the same server, ready for trimming back to an individual organization's needs. +The final outputs of this section are a controller and worker installed in the +same server, ready for trimming back to an individual organization's needs. - > NOTE: The guide assumes that your host OS has the packages installed to support BitBake for the release(s) you are targeting.?? Please refer to the Yocto manual for those packages. + > NOTE: The guide assumes that your host OS has the packages installed to + > support BitBake for the release(s) you are targeting.?? Please refer to the + > Yocto manual for those packages. The latest version of BuildBot is written in Python 3, so installation via??pip3: @@ -17,11 +23,15 @@ apt install git build-essential python3-pip virtualenv enchant npm sudo pip3 install buildbot buildbot-www buildbot-waterfall-view buildbot-console-view buildbot-grid-view buildbot-worker ``` -It is recommended to also install??`testtools` and??`libccpunit-subunit-dev` (via apt, in this case) on the worker in order??for certain image tests to work correctly (e.g., `core-image-sato-sdk:do_testimage`). +It is recommended to also install??`testtools` and??`libccpunit-subunit-dev` (via +apt, in this case) on the worker in order??for certain image tests to work +correctly (e.g., `core-image-sato-sdk:do_testimage`). ### 1.2) Creating the Autobuilder User's Directory -For a new installation you will need a system user that can run Autobuilder2 as well as a couple of repositories: Autobuilder2 itself and a helper plugin. As root: +For a new installation you will need a system user that can run Autobuilder2 as +well as a couple of repositories: Autobuilder2 itself and a helper plugin. As +root: ``` useradd -m --system pokybuild3 @@ -39,9 +49,13 @@ chown -R pokybuild3:nogroup /home/pokybuild3 Note: depending on the configuration of your system, you may need to replace "nogroup" with "pokybuild3" for the last command to work. - > **IMPORTANT:** In the above command you created a controller and a worker, which will attempt to join the controller using??`pass` as the password.?? Feel free to change this, knowing that if you do, you must change the controller's master configuration file to match. + > **IMPORTANT:** In the above command you created a controller and a worker, + > which will attempt to join the controller using??`pass` as the password.?? Feel + > free to change this, knowing that if you do, you must change the controller's + > master configuration file to match. -At the end of this, your build user's home directory (e.g., `/home/pokybuild3`) should look like this: +At the end of this, your build user's home directory (e.g., `/home/pokybuild3`) +should look like this: ``` yocto-autobuilder-helper @@ -52,23 +66,50 @@ yocto-worker ### 1.3) Customizing the Autobuilder Configuration -Next, we need to update the??`yocto-controller/yoctoabb/master.cfg` towards the bottom where the??`title`,??`titleURL`, and??`buildbotURL` are all set. This is also where you would specify a different password for binding workers to the master. +Next, we need to update the??`yocto-controller/yoctoabb/master.cfg` towards the +bottom where the??`title`,??`titleURL`, and??`buildbotURL` are all set. This is +also where you would specify a different password for binding workers to the +master. -Then, we need to update the??`yocto-controller/yoctoabb/config.py` to include our worker.?? In that file, find the line where??`workers` is set and add:??["example-worker"]._NOTE:_ if your worker's name is different, use that here.?? Section 3.1 discusses how to further refine this list of workers. +Then, we need to update the??`yocto-controller/yoctoabb/config.py` to include our +worker.?? In that file, find the line where??`workers` is set and +add:??
Re: [yocto] [yocto-autobuilder2][PATCH 0/2] Clarification and formatting of README-Guide.md
On 6/22/20 10:18 AM, Trevor Gamblin wrote: Long-overdue patches based on experiences trying to set up an autobuilder instance. I've split it into two patches because the second patch (the one that line wraps the majority of the document) may not be desired, if the doc is meant to be read in the pre-existing format. Patch 0001 cleans up the doc so that the sections can be more easily referenced. Any issues with this patch set? Do I need to resend? Trevor Gamblin (2): README-Guide: cleanup, clarify setup instructions README-Guide: wrap lines at 80 characters README-Guide.md | 489 1 file changed, 372 insertions(+), 117 deletions(-) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#49904): https://lists.yoctoproject.org/g/yocto/message/49904 Mute This Topic: https://lists.yoctoproject.org/mt/75039158/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 1/2] packagegroup-patchtest-oe.bb: fix RDEPENDS syntax
Signed-off-by: Trevor Gamblin --- .../recipes-core/packagegroups/packagegroup-patchtest-oe.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-patchtest/recipes-core/packagegroups/packagegroup-patchtest-oe.bb b/meta-patchtest/recipes-core/packagegroups/packagegroup-patchtest-oe.bb index 3dd97bd..35758c5 100644 --- a/meta-patchtest/recipes-core/packagegroups/packagegroup-patchtest-oe.bb +++ b/meta-patchtest/recipes-core/packagegroups/packagegroup-patchtest-oe.bb @@ -6,7 +6,7 @@ DESCRIPTION = "Packages required to run patchtest-oe test cases using patchtest" inherit packagegroup -RDEPENDS_${PN} = "\ +RDEPENDS:${PN} = "\ python3-pyparsing \ python3-unidiff \ python3-pylint \ -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59890): https://lists.yoctoproject.org/g/yocto/message/59890 Mute This Topic: https://lists.yoctoproject.org/mt/98711879/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH 2/2] packagegroup-self-hosted.bbappend: fix RDEPENDS syntax
Signed-off-by: Trevor Gamblin --- .../packagegroups/packagegroup-self-hosted.bbappend | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta-patchtest/recipes-core/packagegroups/packagegroup-self-hosted.bbappend b/meta-patchtest/recipes-core/packagegroups/packagegroup-self-hosted.bbappend index 409d24d..e22948d 100644 --- a/meta-patchtest/recipes-core/packagegroups/packagegroup-self-hosted.bbappend +++ b/meta-patchtest/recipes-core/packagegroups/packagegroup-self-hosted.bbappend @@ -1,5 +1,5 @@ -RDEPENDS_packagegroup-self-hosted-extended += "\ +RDEPENDS:packagegroup-self-hosted-extended += "\ openssh-scp \ openssh-ssh \ perl-misc \ -" \ No newline at end of file +" -- 2.40.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#59891): https://lists.yoctoproject.org/g/yocto/message/59891 Mute This Topic: https://lists.yoctoproject.org/mt/98711880/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 03/11] python3-lazy-object-proxy: upgrade 1.3.1 -> 1.9.0
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-lazy-object-proxy_1.3.1.bb | 10 -- .../python/python3-lazy-object-proxy_1.9.0.bb | 11 +++ 2 files changed, 11 insertions(+), 10 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb b/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb deleted file mode 100644 index a876fa7..000 --- a/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb +++ /dev/null @@ -1,10 +0,0 @@ -SUMMARY = "A fast and thorough lazy object proxy" -HOMEPAGE = "https://python-lazy-object-proxy.readthedocs.io/"; -LICENSE = "BSD" -SECTION = "devel/python" -LIC_FILES_CHKSUM = "file://LICENSE;md5=ec680df42d8ad56427a83308817658cc" - -SRC_URI[md5sum] = "e128152b76eb5b9ba759504936139fd0" -SRC_URI[sha256sum] = "eb91be369f945f10d3a49f5f9be8b3d0b93a4c2be8f8a5b83b0571b8123e0a7a" - -inherit pypi setuptools3 diff --git a/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb b/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb new file mode 100644 index 000..74c5e9b --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb @@ -0,0 +1,11 @@ +SUMMARY = "A fast and thorough lazy object proxy" +HOMEPAGE = "https://python-lazy-object-proxy.readthedocs.io/"; +LICENSE = "BSD-2-Clause" +SECTION = "devel/python" +LIC_FILES_CHKSUM = "file://LICENSE;md5=d606e94f56c21c8e0cdde0b622dcdf57" + +DEPENDS += "${PYTHON_PN}-setuptools-scm-native ${PYTHON_PN}-pip-native" + +SRC_URI[sha256sum] = "659fb5809fa4629b8a1ac5106f669cfc7bef26fbb389dda53b3e010d1ac4ebae" + +inherit pypi setuptools3 -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60050): https://lists.yoctoproject.org/g/yocto/message/60050 Mute This Topic: https://lists.yoctoproject.org/mt/98956430/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 01/11] python3-astroid: upgrade 1.5.3 -> 2.15.4
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without needing that layer. Signed-off-by: Trevor Gamblin --- .../python/python3-astroid_1.5.3.bb | 20 --- .../python/python3-astroid_2.15.4.bb | 35 +++ 2 files changed, 35 insertions(+), 20 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb b/meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb deleted file mode 100644 index 104038a..000 --- a/meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb +++ /dev/null @@ -1,20 +0,0 @@ -SUMMARY = "An abstract syntax tree for Python with inference support." -HOMEPAGE = "https://pypi.python.org/pypi/astroid"; -SECTION = "devel/python" -LICENSE = "LGPL" -LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe" - -SRC_URI[md5sum] = "6f65e4ea8290ec032320460905afb828" -SRC_URI[sha256sum] = "492c2a2044adbf6a84a671b7522e9295ad2f6a7c781b899014308db25312dd35" - -inherit pypi setuptools3 - -RDEPENDS_${PN}_class-target += "\ -${PYTHON_PN}-distutils \ -${PYTHON_PN}-lazy-object-proxy \ -${PYTHON_PN}-six \ -${PYTHON_PN}-subprocess \ -${PYTHON_PN}-wrapt \ -${PYTHON_PN}-re \ -${PYTHON_PN}-setuptools \ -" diff --git a/meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb b/meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb new file mode 100644 index 000..6053728 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb @@ -0,0 +1,35 @@ +SUMMARY = "An abstract syntax tree for Python with inference support." +HOMEPAGE = "https://pypi.python.org/pypi/astroid"; +SECTION = "devel/python" +LICENSE = "LGPL-2.1-only" +LIC_FILES_CHKSUM = "file://LICENSE;md5=a70cf540abf41acb644ac3b621b2fad1" + +SRC_URI[sha256sum] = "c81e1c7fbac615037744d067a9bb5f9aeb655edf59b63ee8b59585475d6f80d8" + +inherit pypi python_setuptools_build_meta + +DEPENDS += "\ +${PYTHON_PN}-pytest-runner-native \ +${PYTHON_PN}-wheel-native \ +" + +PACKAGES =+ "${PN}-tests" + +FILES:${PN}-tests += " \ +${PYTHON_SITEPACKAGES_DIR}/astroid/test* \ +${PYTHON_SITEPACKAGES_DIR}/astroid/__pycache__/test* \ +" + +RDEPENDS:${PN}:class-target += "\ +${PYTHON_PN}-distutils \ +${PYTHON_PN}-lazy-object-proxy \ +${PYTHON_PN}-logging \ +${PYTHON_PN}-six \ +${PYTHON_PN}-wrapt \ +${PYTHON_PN}-setuptools \ +" + +RDEPENDS:${PN}-tests:class-target += "\ +${PYTHON_PN}-unittest \ +${PYTHON_PN}-xml \ +" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60048): https://lists.yoctoproject.org/g/yocto/message/60048 Mute This Topic: https://lists.yoctoproject.org/mt/98956427/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 00/11] patchtest: layer updates
This series includes several changes to upgrade existing recipes, add those that the new upgrades depend on, and remove what's no longer necessary to have in the layer. The objective is to make patchtest usable with only oe-core and bitbake, so almost all changes have been copied over from the meta-python layer. python3-pyparsing is in oe-core, so it's no longer needed in patchtest. Trevor Gamblin (11): python3-astroid: upgrade 1.5.3 -> 2.15.4 python3-isort: upgrade 4.2.15 -> 5.12.0 python3-lazy-object-proxy: upgrade 1.3.1 -> 1.9.0 python3-pylint: upgrade 1.6.5 -> 2.17.4 python3-pyparsing: remove recipe python3-unidiff: upgrade 0.5.4 -> 0.7.5 python3-wrapt: upgrade 1.10.10 -> 1.15.0 python3-dill: add recipe python3-mccabe: add recipe python3-platformdirs: add recipe python3-tomlkit: add recipe .../python/python3-astroid_1.5.3.bb | 20 .../python/python3-astroid_2.15.4.bb | 35 + .../python/python3-dill_0.3.6.bb | 17 +++ .../python/python3-isort_4.2.15.bb| 10 .../python/python3-isort_5.12.0.bb| 20 .../python/python3-lazy-object-proxy_1.3.1.bb | 10 .../python/python3-lazy-object-proxy_1.9.0.bb | 11 .../python/python3-mccabe_0.7.0.bb| 11 .../python/python3-platformdirs/run-ptest | 3 ++ .../python/python3-platformdirs_3.5.1.bb | 35 + .../python/python3-pylint/run-ptest | 3 ++ .../python/python3-pylint_1.6.5.bb| 33 .../python/python3-pylint_2.17.4.bb | 51 +++ .../python/python3-pyparsing_2.2.0.bb | 19 --- .../python/python3-tomlkit/run-ptest | 3 ++ .../python/python3-tomlkit_0.11.8.bb | 23 + .../python/python3-unidiff/run-ptest | 3 ++ .../python/python3-unidiff_0.5.4.bb | 9 .../python/python3-unidiff_0.7.5.bb | 26 ++ .../python/python3-wrapt_1.10.10.bb | 15 -- .../python/python3-wrapt_1.15.0.bb| 16 ++ 21 files changed, 257 insertions(+), 116 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_2.17.4.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60047): https://lists.yoctoproject.org/g/yocto/message/60047 Mute This Topic: https://lists.yoctoproject.org/mt/98956426/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 04/11] python3-pylint: upgrade 1.6.5 -> 2.17.4
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-pylint/run-ptest | 3 ++ .../python/python3-pylint_1.6.5.bb| 33 .../python/python3-pylint_2.17.4.bb | 51 +++ 3 files changed, 54 insertions(+), 33 deletions(-) create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_2.17.4.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest b/meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest new file mode 100644 index 000..7a5ceed --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest --benchmark-disable -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb b/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb deleted file mode 100644 index 0bb9e3f..000 --- a/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb +++ /dev/null @@ -1,33 +0,0 @@ -SUMMARY="Pylint is a Python source code analyzer" -HOMEPAGE= "http://www.pylint.org/"; -LICENSE = "GPLv2" -LIC_FILES_CHKSUM = "file://COPYING;md5=4325afd396febcb659c36b49533135d4" - -SRC_URI[md5sum] = "31da2185bf59142479e4fa16d8a9e347" -SRC_URI[sha256sum] = "a673984a8dd78e4a8b8cfdee5359a1309d833cf38405008f4a249994a8456719" - -inherit pypi setuptools3 - -SRC_URI += "file://0001-epylint-corrects-msg-template-object.patch" - -DEPENDS += "${PYTHON_PN}-pytest-runner-native" - -RDEPENDS_${PN} += "${PYTHON_PN}-astroid \ - ${PYTHON_PN}-isort \ - ${PYTHON_PN}-numbers \ - ${PYTHON_PN}-shell \ - ${PYTHON_PN}-json \ - ${PYTHON_PN}-pkgutil \ - ${PYTHON_PN}-difflib \ - ${PYTHON_PN}-netserver \ - " - -do_install:append(){ -rm ${D}${bindir}/pylint -cat >> ${D}${bindir}/pylint <http://www.pylint.org/"; +LICENSE = "GPL-2.0-only" +LIC_FILES_CHKSUM = "file://LICENSE;md5=c107cf754550e65755c42985a5d4e9c9" + +SRC_URI += " \ +git://github.com/pylint-dev/pylint;branch=main;protocol=https \ +file://run-ptest \ +file://0001-pyproject.toml-Remove-hard-coded-dependencies.patch \ +file://0001-pylint-remove-plugin-pickle-test.patch \ +" +SRCREV ?= "fc34a4b6abe56f3ac07ca15d846b1c1955545f85" + +inherit python_setuptools_build_meta ptest + +RDEPENDS:${PN} += "${PYTHON_PN}-astroid \ + ${PYTHON_PN}-dill \ + ${PYTHON_PN}-isort \ + ${PYTHON_PN}-mccabe \ + ${PYTHON_PN}-numbers \ + ${PYTHON_PN}-platformdirs \ + ${PYTHON_PN}-shell \ + ${PYTHON_PN}-json \ + ${PYTHON_PN}-pkgutil \ + ${PYTHON_PN}-difflib \ + ${PYTHON_PN}-netserver \ + ${PYTHON_PN}-tomlkit \ + " + +RDEPENDS:${PN}-ptest += " \ +${PYTHON_PN}-core \ +${PYTHON_PN}-git \ +${PYTHON_PN}-py \ +${PYTHON_PN}-pytest \ +${PYTHON_PN}-pytest-benchmark \ +${PYTHON_PN}-pytest-runner \ +${PYTHON_PN}-pytest-timeout \ +${PYTHON_PN}-pytest-xdist \ +${PYTHON_PN}-requests \ +${PYTHON_PN}-statistics \ +${PYTHON_PN}-tomllib \ +${PYTHON_PN}-typing-extensions \ +" + +S = "${WORKDIR}/git" + +do_install_ptest() { +install -d ${D}${PTEST_PATH}/tests +cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ +sed -i 's#/usr/bin/python$#/usr/bin/python3#g' ${D}${PTEST_PATH}/tests/data/ascript +} -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60051): https://lists.yoctoproject.org/g/yocto/message/60051 Mute This Topic: https://lists.yoctoproject.org/mt/98956431/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 05/11] python3-pyparsing: remove recipe
This is in oe-core now, so no need to carry it in patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-pyparsing_2.2.0.bb | 19 --- 1 file changed, 19 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb b/meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb deleted file mode 100644 index 033e0db..000 --- a/meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb +++ /dev/null @@ -1,19 +0,0 @@ -SUMMARY = "Python parsing module" -HOMEPAGE = "http://pyparsing.wikispaces.com/"; -LICENSE = "MIT" -LIC_FILES_CHKSUM = "file://LICENSE;md5=fb46329938e6bc829b256e37d5c1e31a" - -SRC_URI[md5sum] = "0214e42d63af850256962b6744c948d9" -SRC_URI[sha256sum] = "0832bcf47acd283788593e7a0f542407bd9550a55a8a8435214a1960e04bcb04" - -inherit pypi - -RDEPENDS_${PN} += " \ -${PYTHON_PN}-datetime \ -${PYTHON_PN}-debugger \ -${PYTHON_PN}-json \ -${PYTHON_PN}-netclient \ -${PYTHON_PN}-pprint \ -${PYTHON_PN}-stringold \ -${PYTHON_PN}-threading \ -" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60052): https://lists.yoctoproject.org/g/yocto/message/60052 Mute This Topic: https://lists.yoctoproject.org/mt/98956433/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 02/11] python3-isort: upgrade 4.2.15 -> 5.12.0
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-isort_4.2.15.bb| 10 -- .../python/python3-isort_5.12.0.bb| 20 +++ 2 files changed, 20 insertions(+), 10 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb b/meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb deleted file mode 100644 index fc98e7c..000 --- a/meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb +++ /dev/null @@ -1,10 +0,0 @@ -SUMMARY = "A Python utility / library to sort Python imports." -HOMEPAGE = "https://pypi.python.org/pypi/isort"; -LICENSE = "MIT" -SECTION = "devel/python" -LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=8227180126797a0148f94f483f3e1489" - -SRC_URI[md5sum] = "34915a2ce60e6fe3dbcbf5982deef9b4" -SRC_URI[sha256sum] = "79f46172d3a4e2e53e7016e663cc7a8b538bec525c36675fcfd2767df30b3983" - -inherit pypi setuptools3 diff --git a/meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb b/meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb new file mode 100644 index 000..6d7b20f --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb @@ -0,0 +1,20 @@ +SUMMARY = "A Python utility / library to sort Python imports." +HOMEPAGE = "https://pypi.python.org/pypi/isort"; +LICENSE = "MIT" +SECTION = "devel/python" +LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=6;endline=6;md5=8227180126797a0148f94f483f3e1489" + +SRC_URI[sha256sum] = "8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504" + +inherit pypi python_poetry_core + +RDEPENDS:${PN} += "\ +${PYTHON_PN}-datetime \ +${PYTHON_PN}-shell \ +${PYTHON_PN}-profile \ +${PYTHON_PN}-numbers \ +${PYTHON_PN}-pprint \ +${PYTHON_PN}-difflib \ +" + +BBCLASSEXTEND = "native nativesdk" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60049): https://lists.yoctoproject.org/g/yocto/message/60049 Mute This Topic: https://lists.yoctoproject.org/mt/98956428/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 06/11] python3-unidiff: upgrade 0.5.4 -> 0.7.5
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-unidiff/run-ptest | 3 +++ .../python/python3-unidiff_0.5.4.bb | 9 --- .../python/python3-unidiff_0.7.5.bb | 26 +++ 3 files changed, 29 insertions(+), 9 deletions(-) create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest b/meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest new file mode 100644 index 000..b63c4de --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb b/meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb deleted file mode 100644 index 5ad9bfc..000 --- a/meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb +++ /dev/null @@ -1,9 +0,0 @@ -SUMMARY = "Unified diff parsing/metadata extraction library" -HOMEPAGE = "http://github.com/matiasb/python-unidiff"; -LICENSE = "MIT" -LIC_FILES_CHKSUM = "file://LICENSE;md5=4c434b08ef42fea235bb019b5e5a97b3" - -SRC_URI[md5sum] = "c8099edf89d2e3bac16123766625a38b" -SRC_URI[sha256sum] = "a7baf71846a68c5305a7d6005dfb5eca8798f13861176355a285cbda3206824d" - -inherit pypi setuptools3 diff --git a/meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb b/meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb new file mode 100644 index 000..690acf8 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb @@ -0,0 +1,26 @@ +SUMMARY = "Unified diff parsing/metadata extraction library" +HOMEPAGE = "http://github.com/matiasb/python-unidiff"; +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=4c434b08ef42fea235bb019b5e5a97b3" + +SRC_URI[sha256sum] = "2e5f0162052248946b9f0970a40e9e124236bf86c82b70821143a6fc1dea2574" + +inherit pypi setuptools3 ptest + +SRC_URI += " \ +file://run-ptest \ +" + +RDEPENDS:${PN}-ptest += " \ + ${PYTHON_PN}-pytest \ +" + +do_install_ptest() { + install -d ${D}${PTEST_PATH}/tests +cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ +} + +RDEPENDS:${PN} += " \ +${PYTHON_PN}-codecs \ +${PYTHON_PN}-io \ +" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60053): https://lists.yoctoproject.org/g/yocto/message/60053 Mute This Topic: https://lists.yoctoproject.org/mt/98956434/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 08/11] python3-dill: add recipe
python3-pylint requires python3-dill, so copy it from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-dill_0.3.6.bb| 17 + 1 file changed, 17 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb b/meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb new file mode 100644 index 000..bec4643 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb @@ -0,0 +1,17 @@ +SUMMARY = "Serialize all of python" +HOMEPAGE = "https://pypi.org/project/dill/"; +LICENSE = "BSD-3-Clause" +LIC_FILES_CHKSUM = "file://LICENSE;md5=61f24e44fc855bde43ed5a1524a37bc4" + +SRC_URI[sha256sum] = "e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373" + +inherit pypi setuptools3 + +PYPI_PACKAGE_EXT = "tar.gz" + +RDEPENDS:${PN} += "\ +${PYTHON_PN}-multiprocessing \ +${PYTHON_PN}-logging \ +${PYTHON_PN}-profile \ +${PYTHON_PN}-core \ +" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60055): https://lists.yoctoproject.org/g/yocto/message/60055 Mute This Topic: https://lists.yoctoproject.org/mt/98956436/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 07/11] python3-wrapt: upgrade 1.10.10 -> 1.15.0
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-wrapt_1.10.10.bb | 15 --- .../python/python3-wrapt_1.15.0.bb | 16 2 files changed, 16 insertions(+), 15 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb b/meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb deleted file mode 100644 index cd987d3..000 --- a/meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb +++ /dev/null @@ -1,15 +0,0 @@ -SUMMARY = "A Python module for decorators, wrappers and monkey patching." -HOMEPAGE = "http://wrapt.readthedocs.org/"; -LICENSE = "BSD" -SECTION = "devel/python" -LIC_FILES_CHKSUM = "file://LICENSE;md5=82704725592991ea88b042d150a66303" - -SRC_URI[md5sum] = "97365e906afa8b431f266866ec4e2e18" -SRC_URI[sha256sum] = "42160c91b77f1bc64a955890038e02f2f72986c01d462d53cb6cb039b995cdd9" - -inherit pypi setuptools3 - -RDEPENDS_${PN}_class-target += "\ -${PYTHON_PN}-stringold \ -${PYTHON_PN}-threading \ -" diff --git a/meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb b/meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb new file mode 100644 index 000..e35e1f7 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb @@ -0,0 +1,16 @@ +SUMMARY = "A Python module for decorators, wrappers and monkey patching." +HOMEPAGE = "http://wrapt.readthedocs.org/"; +LICENSE = "BSD-2-Clause" +SECTION = "devel/python" +LIC_FILES_CHKSUM = "file://LICENSE;md5=dc34cbad60bc961452eb7ade801d25f7" + +SRC_URI[sha256sum] = "d06730c6aed78cee4126234cf2d071e01b44b915e725a6cb439a879ec9754a3a" + +inherit pypi setuptools3 + +RDEPENDS:${PN}:class-target += "\ +${PYTHON_PN}-stringold \ +${PYTHON_PN}-threading \ +" + +BBCLASSEXTEND = "native" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60054): https://lists.yoctoproject.org/g/yocto/message/60054 Mute This Topic: https://lists.yoctoproject.org/mt/98956435/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 09/11] python3-mccabe: add recipe
python3-pylint requires python3-mccabe, so copy it from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../recipes-devtools/python/python3-mccabe_0.7.0.bb | 11 +++ 1 file changed, 11 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb b/meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb new file mode 100644 index 000..a2c72fc --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb @@ -0,0 +1,11 @@ +DESCRIPTION = "McCabe checker, plugin for flake8" +HOMEPAGE = "https://github.com/PyCQA/mccabe"; +SECTION = "devel/python" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=a489dc62bacbdad3335c0f160a974f0f" + +SRC_URI[sha256sum] = "348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325" + +inherit pypi setuptools3 + +DEPENDS += "${PYTHON_PN}-pytest-runner-native" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60056): https://lists.yoctoproject.org/g/yocto/message/60056 Mute This Topic: https://lists.yoctoproject.org/mt/98956437/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 10/11] python3-platformdirs: add recipe
python3-pylint requires python3-platformdirs, so copy it from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-platformdirs/run-ptest | 3 ++ .../python/python3-platformdirs_3.5.1.bb | 35 +++ 2 files changed, 38 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest b/meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest new file mode 100644 index 000..b63c4de --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb b/meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb new file mode 100644 index 000..4fefe56 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb @@ -0,0 +1,35 @@ +SUMMARY = "A small Python module for determining appropriate platform-specific dirs" +HOMEPAGE = "https://github.com/platformdirs/platformdirs"; +LICENSE = "BSD-3-Clause" +LIC_FILES_CHKSUM = "file://LICENSE;md5=ea4f5a41454746a9ed111e3d8723d17a" + +SRC_URI = " \ +git://github.com/platformdirs/platformdirs.git;branch=main;protocol=https \ +file://run-ptest \ +" + +SRCREV ?= "b8c42ddca4def1fba38b9815a7d94ec2ac630b29" +SRC_URI[sha256sum] = "7954a68d0ba23558d753f73437c55f89027cf8f5108c19844d4b82e5af396335" + +inherit python_setuptools_build_meta ptest + +DEPENDS += " \ +python3-hatch-vcs-native \ +python3-setuptools-scm-native \ +python3-toml-native \ +" + +S = "${WORKDIR}/git" + +RDEPENDS:${PN}-ptest += " \ +${PYTHON_PN}-appdirs \ +${PYTHON_PN}-pytest \ +${PYTHON_PN}-pytest-mock \ +" + +do_install_ptest() { +install -d ${D}${PTEST_PATH}/Tests +cp -rf ${S}/tests ${D}${PTEST_PATH}/ +} + +BBCLASSEXTEND = "native" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60057): https://lists.yoctoproject.org/g/yocto/message/60057 Mute This Topic: https://lists.yoctoproject.org/mt/98956438/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [PATCH 11/11] python3-tomlkit: add recipe
python3-pylint requires python3-tomlkit, so copy the recipe from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-tomlkit/run-ptest | 3 +++ .../python/python3-tomlkit_0.11.8.bb | 23 +++ 2 files changed, 26 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest b/meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest new file mode 100644 index 000..b63c4de --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb b/meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb new file mode 100644 index 000..71a43c4 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb @@ -0,0 +1,23 @@ +SUMMARY = "Style preserving TOML library" +HOMEPAGE = "https://pypi.org/project/tomlkit/"; +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=31aac0dbc1babd278d5386dadb7f8e82" + +SRC_URI[sha256sum] = "9330fc7faa1db67b541b28e62018c17d20be733177d290a13b24c62d1614e0c3" + +inherit pypi python_poetry_core ptest + +SRC_URI += " \ +file://run-ptest \ +" + +RDEPENDS:${PN}-ptest += " \ +${PYTHON_PN}-poetry-core \ +${PYTHON_PN}-pytest \ +${PYTHON_PN}-pyyaml \ +" + +do_install_ptest() { +install -d ${D}${PTEST_PATH}/tests +cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ +} -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60058): https://lists.yoctoproject.org/g/yocto/message/60058 Mute This Topic: https://lists.yoctoproject.org/mt/98956440/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH v2 00/11] patchtest: layer updates
This series includes several changes to upgrade existing recipes, add those that the new upgrades depend on, and remove what's no longer necessary to have in the layer. The objective is to make patchtest usable with only oe-core and bitbake, so almost all changes have been copied over from the meta-python layer. python3-pyparsing is in oe-core, so it's no longer needed in patchtest. Resending series so that the proper labels are present. Trevor Gamblin (11): python3-astroid: upgrade 1.5.3 -> 2.15.4 python3-isort: upgrade 4.2.15 -> 5.12.0 python3-lazy-object-proxy: upgrade 1.3.1 -> 1.9.0 python3-pylint: upgrade 1.6.5 -> 2.17.4 python3-pyparsing: remove recipe python3-unidiff: upgrade 0.5.4 -> 0.7.5 python3-wrapt: upgrade 1.10.10 -> 1.15.0 python3-dill: add recipe python3-mccabe: add recipe python3-platformdirs: add recipe python3-tomlkit: add recipe .../python/python3-astroid_1.5.3.bb | 20 .../python/python3-astroid_2.15.4.bb | 35 + .../python/python3-dill_0.3.6.bb | 17 +++ .../python/python3-isort_4.2.15.bb| 10 .../python/python3-isort_5.12.0.bb| 20 .../python/python3-lazy-object-proxy_1.3.1.bb | 10 .../python/python3-lazy-object-proxy_1.9.0.bb | 11 .../python/python3-mccabe_0.7.0.bb| 11 .../python/python3-platformdirs/run-ptest | 3 ++ .../python/python3-platformdirs_3.5.1.bb | 35 + .../python/python3-pylint/run-ptest | 3 ++ .../python/python3-pylint_1.6.5.bb| 33 .../python/python3-pylint_2.17.4.bb | 51 +++ .../python/python3-pyparsing_2.2.0.bb | 19 --- .../python/python3-tomlkit/run-ptest | 3 ++ .../python/python3-tomlkit_0.11.8.bb | 23 + .../python/python3-unidiff/run-ptest | 3 ++ .../python/python3-unidiff_0.5.4.bb | 9 .../python/python3-unidiff_0.7.5.bb | 26 ++ .../python/python3-wrapt_1.10.10.bb | 15 -- .../python/python3-wrapt_1.15.0.bb| 16 ++ 21 files changed, 257 insertions(+), 116 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_2.17.4.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb delete mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60059): https://lists.yoctoproject.org/g/yocto/message/60059 Mute This Topic: https://lists.yoctoproject.org/mt/98956599/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 01/11] python3-astroid: upgrade 1.5.3 -> 2.15.4
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without needing that layer. Signed-off-by: Trevor Gamblin --- .../python/python3-astroid_1.5.3.bb | 20 --- .../python/python3-astroid_2.15.4.bb | 35 +++ 2 files changed, 35 insertions(+), 20 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb b/meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb deleted file mode 100644 index 104038a..000 --- a/meta-patchtest/recipes-devtools/python/python3-astroid_1.5.3.bb +++ /dev/null @@ -1,20 +0,0 @@ -SUMMARY = "An abstract syntax tree for Python with inference support." -HOMEPAGE = "https://pypi.python.org/pypi/astroid"; -SECTION = "devel/python" -LICENSE = "LGPL" -LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe" - -SRC_URI[md5sum] = "6f65e4ea8290ec032320460905afb828" -SRC_URI[sha256sum] = "492c2a2044adbf6a84a671b7522e9295ad2f6a7c781b899014308db25312dd35" - -inherit pypi setuptools3 - -RDEPENDS_${PN}_class-target += "\ -${PYTHON_PN}-distutils \ -${PYTHON_PN}-lazy-object-proxy \ -${PYTHON_PN}-six \ -${PYTHON_PN}-subprocess \ -${PYTHON_PN}-wrapt \ -${PYTHON_PN}-re \ -${PYTHON_PN}-setuptools \ -" diff --git a/meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb b/meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb new file mode 100644 index 000..6053728 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-astroid_2.15.4.bb @@ -0,0 +1,35 @@ +SUMMARY = "An abstract syntax tree for Python with inference support." +HOMEPAGE = "https://pypi.python.org/pypi/astroid"; +SECTION = "devel/python" +LICENSE = "LGPL-2.1-only" +LIC_FILES_CHKSUM = "file://LICENSE;md5=a70cf540abf41acb644ac3b621b2fad1" + +SRC_URI[sha256sum] = "c81e1c7fbac615037744d067a9bb5f9aeb655edf59b63ee8b59585475d6f80d8" + +inherit pypi python_setuptools_build_meta + +DEPENDS += "\ +${PYTHON_PN}-pytest-runner-native \ +${PYTHON_PN}-wheel-native \ +" + +PACKAGES =+ "${PN}-tests" + +FILES:${PN}-tests += " \ +${PYTHON_SITEPACKAGES_DIR}/astroid/test* \ +${PYTHON_SITEPACKAGES_DIR}/astroid/__pycache__/test* \ +" + +RDEPENDS:${PN}:class-target += "\ +${PYTHON_PN}-distutils \ +${PYTHON_PN}-lazy-object-proxy \ +${PYTHON_PN}-logging \ +${PYTHON_PN}-six \ +${PYTHON_PN}-wrapt \ +${PYTHON_PN}-setuptools \ +" + +RDEPENDS:${PN}-tests:class-target += "\ +${PYTHON_PN}-unittest \ +${PYTHON_PN}-xml \ +" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60060): https://lists.yoctoproject.org/g/yocto/message/60060 Mute This Topic: https://lists.yoctoproject.org/mt/98956600/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 02/11] python3-isort: upgrade 4.2.15 -> 5.12.0
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-isort_4.2.15.bb| 10 -- .../python/python3-isort_5.12.0.bb| 20 +++ 2 files changed, 20 insertions(+), 10 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb b/meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb deleted file mode 100644 index fc98e7c..000 --- a/meta-patchtest/recipes-devtools/python/python3-isort_4.2.15.bb +++ /dev/null @@ -1,10 +0,0 @@ -SUMMARY = "A Python utility / library to sort Python imports." -HOMEPAGE = "https://pypi.python.org/pypi/isort"; -LICENSE = "MIT" -SECTION = "devel/python" -LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=8227180126797a0148f94f483f3e1489" - -SRC_URI[md5sum] = "34915a2ce60e6fe3dbcbf5982deef9b4" -SRC_URI[sha256sum] = "79f46172d3a4e2e53e7016e663cc7a8b538bec525c36675fcfd2767df30b3983" - -inherit pypi setuptools3 diff --git a/meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb b/meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb new file mode 100644 index 000..6d7b20f --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-isort_5.12.0.bb @@ -0,0 +1,20 @@ +SUMMARY = "A Python utility / library to sort Python imports." +HOMEPAGE = "https://pypi.python.org/pypi/isort"; +LICENSE = "MIT" +SECTION = "devel/python" +LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=6;endline=6;md5=8227180126797a0148f94f483f3e1489" + +SRC_URI[sha256sum] = "8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504" + +inherit pypi python_poetry_core + +RDEPENDS:${PN} += "\ +${PYTHON_PN}-datetime \ +${PYTHON_PN}-shell \ +${PYTHON_PN}-profile \ +${PYTHON_PN}-numbers \ +${PYTHON_PN}-pprint \ +${PYTHON_PN}-difflib \ +" + +BBCLASSEXTEND = "native nativesdk" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60061): https://lists.yoctoproject.org/g/yocto/message/60061 Mute This Topic: https://lists.yoctoproject.org/mt/98956602/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 03/11] python3-lazy-object-proxy: upgrade 1.3.1 -> 1.9.0
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-lazy-object-proxy_1.3.1.bb | 10 -- .../python/python3-lazy-object-proxy_1.9.0.bb | 11 +++ 2 files changed, 11 insertions(+), 10 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb b/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb deleted file mode 100644 index a876fa7..000 --- a/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.3.1.bb +++ /dev/null @@ -1,10 +0,0 @@ -SUMMARY = "A fast and thorough lazy object proxy" -HOMEPAGE = "https://python-lazy-object-proxy.readthedocs.io/"; -LICENSE = "BSD" -SECTION = "devel/python" -LIC_FILES_CHKSUM = "file://LICENSE;md5=ec680df42d8ad56427a83308817658cc" - -SRC_URI[md5sum] = "e128152b76eb5b9ba759504936139fd0" -SRC_URI[sha256sum] = "eb91be369f945f10d3a49f5f9be8b3d0b93a4c2be8f8a5b83b0571b8123e0a7a" - -inherit pypi setuptools3 diff --git a/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb b/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb new file mode 100644 index 000..74c5e9b --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-lazy-object-proxy_1.9.0.bb @@ -0,0 +1,11 @@ +SUMMARY = "A fast and thorough lazy object proxy" +HOMEPAGE = "https://python-lazy-object-proxy.readthedocs.io/"; +LICENSE = "BSD-2-Clause" +SECTION = "devel/python" +LIC_FILES_CHKSUM = "file://LICENSE;md5=d606e94f56c21c8e0cdde0b622dcdf57" + +DEPENDS += "${PYTHON_PN}-setuptools-scm-native ${PYTHON_PN}-pip-native" + +SRC_URI[sha256sum] = "659fb5809fa4629b8a1ac5106f669cfc7bef26fbb389dda53b3e010d1ac4ebae" + +inherit pypi setuptools3 -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60062): https://lists.yoctoproject.org/g/yocto/message/60062 Mute This Topic: https://lists.yoctoproject.org/mt/98956604/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 04/11] python3-pylint: upgrade 1.6.5 -> 2.17.4
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-pylint/run-ptest | 3 ++ .../python/python3-pylint_1.6.5.bb| 33 .../python/python3-pylint_2.17.4.bb | 51 +++ 3 files changed, 54 insertions(+), 33 deletions(-) create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-pylint_2.17.4.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest b/meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest new file mode 100644 index 000..7a5ceed --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-pylint/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest --benchmark-disable -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb b/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb deleted file mode 100644 index 0bb9e3f..000 --- a/meta-patchtest/recipes-devtools/python/python3-pylint_1.6.5.bb +++ /dev/null @@ -1,33 +0,0 @@ -SUMMARY="Pylint is a Python source code analyzer" -HOMEPAGE= "http://www.pylint.org/"; -LICENSE = "GPLv2" -LIC_FILES_CHKSUM = "file://COPYING;md5=4325afd396febcb659c36b49533135d4" - -SRC_URI[md5sum] = "31da2185bf59142479e4fa16d8a9e347" -SRC_URI[sha256sum] = "a673984a8dd78e4a8b8cfdee5359a1309d833cf38405008f4a249994a8456719" - -inherit pypi setuptools3 - -SRC_URI += "file://0001-epylint-corrects-msg-template-object.patch" - -DEPENDS += "${PYTHON_PN}-pytest-runner-native" - -RDEPENDS_${PN} += "${PYTHON_PN}-astroid \ - ${PYTHON_PN}-isort \ - ${PYTHON_PN}-numbers \ - ${PYTHON_PN}-shell \ - ${PYTHON_PN}-json \ - ${PYTHON_PN}-pkgutil \ - ${PYTHON_PN}-difflib \ - ${PYTHON_PN}-netserver \ - " - -do_install:append(){ -rm ${D}${bindir}/pylint -cat >> ${D}${bindir}/pylint <http://www.pylint.org/"; +LICENSE = "GPL-2.0-only" +LIC_FILES_CHKSUM = "file://LICENSE;md5=c107cf754550e65755c42985a5d4e9c9" + +SRC_URI += " \ +git://github.com/pylint-dev/pylint;branch=main;protocol=https \ +file://run-ptest \ +file://0001-pyproject.toml-Remove-hard-coded-dependencies.patch \ +file://0001-pylint-remove-plugin-pickle-test.patch \ +" +SRCREV ?= "fc34a4b6abe56f3ac07ca15d846b1c1955545f85" + +inherit python_setuptools_build_meta ptest + +RDEPENDS:${PN} += "${PYTHON_PN}-astroid \ + ${PYTHON_PN}-dill \ + ${PYTHON_PN}-isort \ + ${PYTHON_PN}-mccabe \ + ${PYTHON_PN}-numbers \ + ${PYTHON_PN}-platformdirs \ + ${PYTHON_PN}-shell \ + ${PYTHON_PN}-json \ + ${PYTHON_PN}-pkgutil \ + ${PYTHON_PN}-difflib \ + ${PYTHON_PN}-netserver \ + ${PYTHON_PN}-tomlkit \ + " + +RDEPENDS:${PN}-ptest += " \ +${PYTHON_PN}-core \ +${PYTHON_PN}-git \ +${PYTHON_PN}-py \ +${PYTHON_PN}-pytest \ +${PYTHON_PN}-pytest-benchmark \ +${PYTHON_PN}-pytest-runner \ +${PYTHON_PN}-pytest-timeout \ +${PYTHON_PN}-pytest-xdist \ +${PYTHON_PN}-requests \ +${PYTHON_PN}-statistics \ +${PYTHON_PN}-tomllib \ +${PYTHON_PN}-typing-extensions \ +" + +S = "${WORKDIR}/git" + +do_install_ptest() { +install -d ${D}${PTEST_PATH}/tests +cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ +sed -i 's#/usr/bin/python$#/usr/bin/python3#g' ${D}${PTEST_PATH}/tests/data/ascript +} -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60063): https://lists.yoctoproject.org/g/yocto/message/60063 Mute This Topic: https://lists.yoctoproject.org/mt/98956605/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 06/11] python3-unidiff: upgrade 0.5.4 -> 0.7.5
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-unidiff/run-ptest | 3 +++ .../python/python3-unidiff_0.5.4.bb | 9 --- .../python/python3-unidiff_0.7.5.bb | 26 +++ 3 files changed, 29 insertions(+), 9 deletions(-) create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest delete mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest b/meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest new file mode 100644 index 000..b63c4de --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-unidiff/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb b/meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb deleted file mode 100644 index 5ad9bfc..000 --- a/meta-patchtest/recipes-devtools/python/python3-unidiff_0.5.4.bb +++ /dev/null @@ -1,9 +0,0 @@ -SUMMARY = "Unified diff parsing/metadata extraction library" -HOMEPAGE = "http://github.com/matiasb/python-unidiff"; -LICENSE = "MIT" -LIC_FILES_CHKSUM = "file://LICENSE;md5=4c434b08ef42fea235bb019b5e5a97b3" - -SRC_URI[md5sum] = "c8099edf89d2e3bac16123766625a38b" -SRC_URI[sha256sum] = "a7baf71846a68c5305a7d6005dfb5eca8798f13861176355a285cbda3206824d" - -inherit pypi setuptools3 diff --git a/meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb b/meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb new file mode 100644 index 000..690acf8 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-unidiff_0.7.5.bb @@ -0,0 +1,26 @@ +SUMMARY = "Unified diff parsing/metadata extraction library" +HOMEPAGE = "http://github.com/matiasb/python-unidiff"; +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=4c434b08ef42fea235bb019b5e5a97b3" + +SRC_URI[sha256sum] = "2e5f0162052248946b9f0970a40e9e124236bf86c82b70821143a6fc1dea2574" + +inherit pypi setuptools3 ptest + +SRC_URI += " \ +file://run-ptest \ +" + +RDEPENDS:${PN}-ptest += " \ + ${PYTHON_PN}-pytest \ +" + +do_install_ptest() { + install -d ${D}${PTEST_PATH}/tests +cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ +} + +RDEPENDS:${PN} += " \ +${PYTHON_PN}-codecs \ +${PYTHON_PN}-io \ +" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60065): https://lists.yoctoproject.org/g/yocto/message/60065 Mute This Topic: https://lists.yoctoproject.org/mt/98956608/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 05/11] python3-pyparsing: remove recipe
This is in oe-core now, so no need to carry it in patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-pyparsing_2.2.0.bb | 19 --- 1 file changed, 19 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb b/meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb deleted file mode 100644 index 033e0db..000 --- a/meta-patchtest/recipes-devtools/python/python3-pyparsing_2.2.0.bb +++ /dev/null @@ -1,19 +0,0 @@ -SUMMARY = "Python parsing module" -HOMEPAGE = "http://pyparsing.wikispaces.com/"; -LICENSE = "MIT" -LIC_FILES_CHKSUM = "file://LICENSE;md5=fb46329938e6bc829b256e37d5c1e31a" - -SRC_URI[md5sum] = "0214e42d63af850256962b6744c948d9" -SRC_URI[sha256sum] = "0832bcf47acd283788593e7a0f542407bd9550a55a8a8435214a1960e04bcb04" - -inherit pypi - -RDEPENDS_${PN} += " \ -${PYTHON_PN}-datetime \ -${PYTHON_PN}-debugger \ -${PYTHON_PN}-json \ -${PYTHON_PN}-netclient \ -${PYTHON_PN}-pprint \ -${PYTHON_PN}-stringold \ -${PYTHON_PN}-threading \ -" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60064): https://lists.yoctoproject.org/g/yocto/message/60064 Mute This Topic: https://lists.yoctoproject.org/mt/98956607/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 07/11] python3-wrapt: upgrade 1.10.10 -> 1.15.0
Copy the recipe from meta-python to handle the upgrade, so that patchtest can build it without requiring the layer. Signed-off-by: Trevor Gamblin --- .../python/python3-wrapt_1.10.10.bb | 15 --- .../python/python3-wrapt_1.15.0.bb | 16 2 files changed, 16 insertions(+), 15 deletions(-) delete mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb create mode 100644 meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb b/meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb deleted file mode 100644 index cd987d3..000 --- a/meta-patchtest/recipes-devtools/python/python3-wrapt_1.10.10.bb +++ /dev/null @@ -1,15 +0,0 @@ -SUMMARY = "A Python module for decorators, wrappers and monkey patching." -HOMEPAGE = "http://wrapt.readthedocs.org/"; -LICENSE = "BSD" -SECTION = "devel/python" -LIC_FILES_CHKSUM = "file://LICENSE;md5=82704725592991ea88b042d150a66303" - -SRC_URI[md5sum] = "97365e906afa8b431f266866ec4e2e18" -SRC_URI[sha256sum] = "42160c91b77f1bc64a955890038e02f2f72986c01d462d53cb6cb039b995cdd9" - -inherit pypi setuptools3 - -RDEPENDS_${PN}_class-target += "\ -${PYTHON_PN}-stringold \ -${PYTHON_PN}-threading \ -" diff --git a/meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb b/meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb new file mode 100644 index 000..e35e1f7 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-wrapt_1.15.0.bb @@ -0,0 +1,16 @@ +SUMMARY = "A Python module for decorators, wrappers and monkey patching." +HOMEPAGE = "http://wrapt.readthedocs.org/"; +LICENSE = "BSD-2-Clause" +SECTION = "devel/python" +LIC_FILES_CHKSUM = "file://LICENSE;md5=dc34cbad60bc961452eb7ade801d25f7" + +SRC_URI[sha256sum] = "d06730c6aed78cee4126234cf2d071e01b44b915e725a6cb439a879ec9754a3a" + +inherit pypi setuptools3 + +RDEPENDS:${PN}:class-target += "\ +${PYTHON_PN}-stringold \ +${PYTHON_PN}-threading \ +" + +BBCLASSEXTEND = "native" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60066): https://lists.yoctoproject.org/g/yocto/message/60066 Mute This Topic: https://lists.yoctoproject.org/mt/98956609/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 08/11] python3-dill: add recipe
python3-pylint requires python3-dill, so copy it from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-dill_0.3.6.bb| 17 + 1 file changed, 17 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb b/meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb new file mode 100644 index 000..bec4643 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-dill_0.3.6.bb @@ -0,0 +1,17 @@ +SUMMARY = "Serialize all of python" +HOMEPAGE = "https://pypi.org/project/dill/"; +LICENSE = "BSD-3-Clause" +LIC_FILES_CHKSUM = "file://LICENSE;md5=61f24e44fc855bde43ed5a1524a37bc4" + +SRC_URI[sha256sum] = "e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373" + +inherit pypi setuptools3 + +PYPI_PACKAGE_EXT = "tar.gz" + +RDEPENDS:${PN} += "\ +${PYTHON_PN}-multiprocessing \ +${PYTHON_PN}-logging \ +${PYTHON_PN}-profile \ +${PYTHON_PN}-core \ +" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60067): https://lists.yoctoproject.org/g/yocto/message/60067 Mute This Topic: https://lists.yoctoproject.org/mt/98956610/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 09/11] python3-mccabe: add recipe
python3-pylint requires python3-mccabe, so copy it from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../recipes-devtools/python/python3-mccabe_0.7.0.bb | 11 +++ 1 file changed, 11 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb b/meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb new file mode 100644 index 000..a2c72fc --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-mccabe_0.7.0.bb @@ -0,0 +1,11 @@ +DESCRIPTION = "McCabe checker, plugin for flake8" +HOMEPAGE = "https://github.com/PyCQA/mccabe"; +SECTION = "devel/python" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=a489dc62bacbdad3335c0f160a974f0f" + +SRC_URI[sha256sum] = "348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325" + +inherit pypi setuptools3 + +DEPENDS += "${PYTHON_PN}-pytest-runner-native" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60068): https://lists.yoctoproject.org/g/yocto/message/60068 Mute This Topic: https://lists.yoctoproject.org/mt/98956611/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 10/11] python3-platformdirs: add recipe
python3-pylint requires python3-platformdirs, so copy it from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-platformdirs/run-ptest | 3 ++ .../python/python3-platformdirs_3.5.1.bb | 35 +++ 2 files changed, 38 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest b/meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest new file mode 100644 index 000..b63c4de --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-platformdirs/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb b/meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb new file mode 100644 index 000..4fefe56 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-platformdirs_3.5.1.bb @@ -0,0 +1,35 @@ +SUMMARY = "A small Python module for determining appropriate platform-specific dirs" +HOMEPAGE = "https://github.com/platformdirs/platformdirs"; +LICENSE = "BSD-3-Clause" +LIC_FILES_CHKSUM = "file://LICENSE;md5=ea4f5a41454746a9ed111e3d8723d17a" + +SRC_URI = " \ +git://github.com/platformdirs/platformdirs.git;branch=main;protocol=https \ +file://run-ptest \ +" + +SRCREV ?= "b8c42ddca4def1fba38b9815a7d94ec2ac630b29" +SRC_URI[sha256sum] = "7954a68d0ba23558d753f73437c55f89027cf8f5108c19844d4b82e5af396335" + +inherit python_setuptools_build_meta ptest + +DEPENDS += " \ +python3-hatch-vcs-native \ +python3-setuptools-scm-native \ +python3-toml-native \ +" + +S = "${WORKDIR}/git" + +RDEPENDS:${PN}-ptest += " \ +${PYTHON_PN}-appdirs \ +${PYTHON_PN}-pytest \ +${PYTHON_PN}-pytest-mock \ +" + +do_install_ptest() { +install -d ${D}${PTEST_PATH}/Tests +cp -rf ${S}/tests ${D}${PTEST_PATH}/ +} + +BBCLASSEXTEND = "native" -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60069): https://lists.yoctoproject.org/g/yocto/message/60069 Mute This Topic: https://lists.yoctoproject.org/mt/98956612/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto][patchtest][PATCH 11/11] python3-tomlkit: add recipe
python3-pylint requires python3-tomlkit, so copy the recipe from meta-python to avoid requiring that layer for patchtest. Signed-off-by: Trevor Gamblin --- .../python/python3-tomlkit/run-ptest | 3 +++ .../python/python3-tomlkit_0.11.8.bb | 23 +++ 2 files changed, 26 insertions(+) create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest create mode 100644 meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb diff --git a/meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest b/meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest new file mode 100644 index 000..b63c4de --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-tomlkit/run-ptest @@ -0,0 +1,3 @@ +#!/bin/sh + +pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}' diff --git a/meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb b/meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb new file mode 100644 index 000..71a43c4 --- /dev/null +++ b/meta-patchtest/recipes-devtools/python/python3-tomlkit_0.11.8.bb @@ -0,0 +1,23 @@ +SUMMARY = "Style preserving TOML library" +HOMEPAGE = "https://pypi.org/project/tomlkit/"; +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=31aac0dbc1babd278d5386dadb7f8e82" + +SRC_URI[sha256sum] = "9330fc7faa1db67b541b28e62018c17d20be733177d290a13b24c62d1614e0c3" + +inherit pypi python_poetry_core ptest + +SRC_URI += " \ +file://run-ptest \ +" + +RDEPENDS:${PN}-ptest += " \ +${PYTHON_PN}-poetry-core \ +${PYTHON_PN}-pytest \ +${PYTHON_PN}-pyyaml \ +" + +do_install_ptest() { +install -d ${D}${PTEST_PATH}/tests +cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ +} -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60070): https://lists.yoctoproject.org/g/yocto/message/60070 Mute This Topic: https://lists.yoctoproject.org/mt/98956614/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [patchtest][PATCH] core-image-patchtest: Add safe git directories
patchtest's method of mounting a shared directory into the guest image means that the associated content may not be owned by the image's user. Add /home/patchtest/openembedded-core and /home/patchtest/openembedded-core/bitbake to the patchtest user's .gitconfig so that it can still run tests if the provided repo is owned by someone else. Signed-off-by: Trevor Gamblin --- meta-patchtest/recipes-core/images/core-image-patchtest.bb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/meta-patchtest/recipes-core/images/core-image-patchtest.bb b/meta-patchtest/recipes-core/images/core-image-patchtest.bb index 47e4041..4aa768c 100644 --- a/meta-patchtest/recipes-core/images/core-image-patchtest.bb +++ b/meta-patchtest/recipes-core/images/core-image-patchtest.bb @@ -43,6 +43,9 @@ EOF [user] name = patchtest email = patcht...@patchtest.com +[safe] +directory = /home/patchtest/openembedded-core +directory = /home/patchtest/openembedded-core/bitbake EOF } -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60113): https://lists.yoctoproject.org/g/yocto/message/60113 Mute This Topic: https://lists.yoctoproject.org/mt/99116431/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Minutes: Yocto Project Weekly Triage Meeting 02/06/2023
On 2023-06-01 11:17, Sakib Sajal wrote: *Wiki: *https://wiki.yoctoproject.org/wiki/Bug_Triage *Attendees: *Richard Purdie, Steve Sakoman, Stephen Jolley, Bruce Ashfield, Joshua Watt, Trevor Gamblin, Randy Macleod, Alexandre Belloni, Tim Orling, Thomas Roos, Yoann Congal, Michael Opdenacker, Ryan Eatmon *ARs:** * - Trevor to create followup bug to 15129 Done: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15130 - Nat to look at 6986 ** *Notes: * * * *Medium+ 4.3 Unassigned Enhancements/Bugs: *72 (Last week 72) *Medium+ 4.99 Unassigned Enhancements/Bugs: *44**(Last week 44) *AB Bugs: *70 (Last week 72) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60160): https://lists.yoctoproject.org/g/yocto/message/60160 Mute This Topic: https://lists.yoctoproject.org/mt/99267059/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper][PATCH] clobberdir: only move dirs more than 60s old
See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14952 Help avoid contention by adding a check to ensure targets are at least 60s old when attempting to move them into a trash directory. Signed-off-by: Trevor Gamblin --- janitor/clobberdir | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/janitor/clobberdir b/janitor/clobberdir index 16b019e..832173c 100755 --- a/janitor/clobberdir +++ b/janitor/clobberdir @@ -52,8 +52,12 @@ trashdir = utils.getconfig("TRASH_DIR", ourconfig) for x in [clobberdir]: if os.path.exists(x) and os.path.exists(trashdir): if (os.stat(trashdir).st_dev == os.stat(x).st_dev): -trashdest = trashdir + "/" + str(int(time.time())) + '-' + str(random.randrange(100, 10, 2)) -mkdir(trashdest) -subprocess.check_call(['mv', x, trashdest]) +x_age = time.time() - os.path.getmtime(x) +if x_age > 60: +trashdest = trashdir + "/" + str(int(time.time())) + '-' + str(random.randrange(100, 10, 2)) +mkdir(trashdest) +subprocess.check_call(['mv', x, trashdest]) +else: +print("Not moving '%s' - age is only %s seconds. There may be another process using it" % (x, str(int(x_age else: subprocess.check_call(['rm', "-rf", x]) -- 2.40.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60216): https://lists.yoctoproject.org/g/yocto/message/60216 Mute This Topic: https://lists.yoctoproject.org/mt/99392420/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Snort 2.9 into Snort 3 recipe
On 2023-06-09 10:05, Rostanin Gleb SBR DIRCS wrote: Hello OpenEmbedded Community, currently I am working on implementing the Snort IDS on an embedded system with the Yocto Linux OS. As OpenEmbedded already provides a recipe for Snort 2.9 it is easy to install snort into the OS. Unfortunately, it is not easy to change the recipe so that the new Snort 3 version is installed. I tried to change the git repo link and the checksum (as a simple solution try), but unfortunately, I ran into multiple errors, and I am not sure at which point I need to make further changes. As I could understand, also a few changes to the source code of Snort must be done to make it work on the Yocto OS. Was someone already successful in implementing a recipe to install the newer Snort 3 version directly on Yocto? What do I have to change in the Snort 2.9 recipe to install the Snort 3 version? The errors you're seeing when you try to build Snort 3 should provide a clue. If you post those, someone may be able to provide further suggestions. Trevor Of course, another solution could be cross compiling Snort for the embedded system, but maybe there is already a solution for this. I happy about any help or comment on this topic! Kind regards G. Rostanin -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60240): https://lists.yoctoproject.org/g/yocto/message/60240 Mute This Topic: https://lists.yoctoproject.org/mt/99428914/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Snort 2.9 into Snort 3 recipe
On 2023-06-12 07:33, Rostanin Gleb SBR DIRCS wrote: Hello, I did the following steps: * Rename recipe file name to snort_3.1.62.0.bb * Changed line 9 “SRC_URI = “…/snort/${BP}.tar.gz”” to https://www.snort.org/downloads/snortplus/snort3-3.1.62.0.tar.gz * Added correct snort3 sha256sum in line 17 * Added correct MD5 checksum of the snort tar ball in line 5 The following error occurs in “0001-libpcap … headers.patch” file: /ERROR: Applying patch '0001-libpcap-search-sysroot-for-headers.patch' on target directory '/home/user/fsl-auto-yocto-bsp/build_s32g274ardb2/tmp/work/cortexa53-crypto-fsl-linux/snort/3.1.62.0-r0/snort-3.1.62.0'/ /CmdError('quilt --quiltrc /home/user/fsl-auto-yocto-bsp/build_s32g274ardb2/tmp/work/cortexa53-crypto-fsl-linux/snort/3.1.62.0-r0/recipe-sysroot-native/etc/quiltrc push', 0, "stdout: Applying patch 0001-libpcap-search-sysroot-for-headers.patch/ /can't find file to patch at input line 21/ /Perhaps you used the wrong -p or --strip option?/ // As the patches are not written for snort 3 I tried again but deleted the patches from the “.bb” file. Now the following error occurs: /DEBUG: Executing python function autotools_aclocals DEBUG: SITE files ['endian-little', 'bit-64', 'arm-common', 'arm-64', 'common-linux', 'common-glibc', 'aarch64-linux', 'common'] DEBUG: Python function autotools_aclocals finished DEBUG: Executing shell function do_compile NOTE: make -j 6 make: *** No targets specified and no makefile found. Stop. ERROR: oe_runmake failed WARNING: exit code 1 from a shell command./// Now I am not sure where and at which point the make file should be created and why it is missing. Could you give some tips on how to transfer this recipe to an snort 3 recipe? It looks like snort 2.x uses Autotools for setup, while 3.x uses CMake instead. You would have to adjust the recipe so that it uses the latter and passes the correct build flags for the "default" snort configuration (I'm not sure what those would be). You may also need to spend some time making sure that the changes provided by those deleted patches aren't still required somewhere in the new source files. If you do 'grep -r "inherit cmake"' on openembedded-core/meta, you can find many examples of other recipes that use CMake for building that you could use as references. Thank you for reading G. Rostanin *Von:*Trevor Gamblin *Gesendet:* Freitag, 9. Juni 2023 18:25 *An:* Rostanin Gleb SBR DIRCS ; yocto@lists.yoctoproject.org *Betreff:* Re: [yocto] Snort 2.9 into Snort 3 recipe On 2023-06-09 10:05, Rostanin Gleb SBR DIRCS wrote: Hello OpenEmbedded Community, currently I am working on implementing the Snort IDS on an embedded system with the Yocto Linux OS. As OpenEmbedded already provides a recipe for Snort 2.9 it is easy to install snort into the OS. Unfortunately, it is not easy to change the recipe so that the new Snort 3 version is installed. I tried to change the git repo link and the checksum (as a simple solution try), but unfortunately, I ran into multiple errors, and I am not sure at which point I need to make further changes. As I could understand, also a few changes to the source code of Snort must be done to make it work on the Yocto OS. Was someone already successful in implementing a recipe to install the newer Snort 3 version directly on Yocto? What do I have to change in the Snort 2.9 recipe to install the Snort 3 version? The errors you're seeing when you try to build Snort 3 should provide a clue. If you post those, someone may be able to provide further suggestions. Trevor Of course, another solution could be cross compiling Snort for the embedded system, but maybe there is already a solution for this. I happy about any help or comment on this topic! Kind regards G. Rostanin -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60258): https://lists.yoctoproject.org/g/yocto/message/60258 Mute This Topic: https://lists.yoctoproject.org/mt/99428914/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-