Bug#1056592: debos: yaml: line X: did not find expected key
On 24/11/2023 17:50, Christopher Obbard wrote: Oh, turns out I completely forgot about that. I think it makes sense to release a new upstream Debos version, import that into Debian and drop your merge request. I will do that today. Thanks for the quick fix Christopher! Have a nice week-end, Arnaud
Bug#1056592: debos: yaml: line X: did not find expected key
Hi Arnaud, On Fri, 2023-11-24 at 09:48 +0700, Arnaud Rebillout wrote: > > I think this has something to do with the recent shell escaping patches. > > Perhaps there is some go module which isn't up to date in Debian causing > > the > > additional single quotation marks around the inner debos call ? > > > Indeed, the package golang-github-alessio-shellescape is not exactly at the > last version in Debian, we have 1.4.1, upstream is at 1.4.2. > > However, after updating it, and rebuilding the fakemachine package, and then > rebuilding debos, I still have the same error. > > After digging a bit more, I found that Sjoerd added a few commits to debos, > just after release 1.1.2. Those commits are needed if fakemachine 0.0.7 is > used, and I can confirm that the issue is fixed by importing those commits. > Cf: https://salsa.debian.org/go-team/packages/debos/-/merge_requests/5 Oh, turns out I completely forgot about that. I think it makes sense to release a new upstream Debos version, import that into Debian and drop your merge request. I will do that today. > Packaging sidenote: please push your pristine-tar branch (or disable it in > debian/gbp.conf), at the moment "gbp buildpackage" fails because of that. I did look on Salsa and the pristine-tar branches for both debos and golang- github-go-debos-fakemachine are up-to-date. As part of my update process I try to always call "gbp push" which pushes the pristine-tar branch, are you sure that is the problem here?
Bug#1056592: debos: yaml: line X: did not find expected key
Hello Christopher, On 23/11/2023 23:45, Christopher Obbard wrote: For me with debos 1.1.2-2, I seem to reproduce your issue: Running /debos --artifactdir debos-tests --template-var 'ospack:"debian"' debos-tests/test-key.yaml using kvm backend 2023/11/23 16:41:30 yaml: line 7: did not find expected key Good to hear, I'm not alone! But with a local checkout of debos upstream built with `go build`, your recipe runs just fine (notice there are now no single-quotes around the --template- var argument to the inner debos call): Running /debos --artifactdir debos-tests --template-var ospack:debdfsa debos-tests/test-key.yaml using kvm backend 2023/11/23 16:41:46 debootstrap I think this has something to do with the recent shell escaping patches. Perhaps there is some go module which isn't up to date in Debian causing the additional single quotation marks around the inner debos call ? Indeed, the package golang-github-alessio-shellescape is not exactly at the last version in Debian, we have 1.4.1, upstream is at 1.4.2. However, after updating it, and rebuilding the fakemachine package, and then rebuilding debos, I still have the same error. After digging a bit more, I found that Sjoerd added a few commits to debos, just after release 1.1.2. Those commits are needed if fakemachine 0.0.7 is used, and I can confirm that the issue is fixed by importing those commits. Cf: https://salsa.debian.org/go-team/packages/debos/-/merge_requests/5 Packaging sidenote: please push your pristine-tar branch (or disable it in debian/gbp.conf), at the moment "gbp buildpackage" fails because of that. Thanks! -- Arnaud Rebillout / OffSec / Kali Linux Developer
Bug#1056592: debos: yaml: line X: did not find expected key
Hi Arnaud,
On Thu, 2023-11-23 at 23:23 +0700, Arnaud Rebillout wrote:
> Package: debos
> Version: 1.1.2-2
> Severity: normal
> User: de...@kali.org
> Usertags: origin-kali
>
> Hi Christopher,
>
> Here's a surprising bug report. The version of debos currently in Debian
> unstable is a bit broken, in an odd way.
>
> Consider the following recipe:
>
> ```
> $ cat main.yaml
> {{ $ospack := or .ospack "debian" }}
> architecture: amd64
> actions:
> - action: debootstrap
> suite: sid
> - action: pack
> file: {{ $ospack }}.tar.gz
> ```
>
> If I run with `-t ospack:debian` it fails:
>
> ```
> $ debos --print-recipe -t ospack:debian main.yaml
> 2023/11/23 23:05:27 Recipe '/home/user/debos/main.yaml':
> 2023/11/23 23:05:27
> architecture: amd64
> actions:
> - action: debootstrap
> suite: sid
> - action: pack
> file: debian.tar.gz
> Running /debos --artifactdir /home/user/debos --template-var
> 'ospack:"debian"' /home/user/debos/main.yaml using kvm backend
> 2023/11/23 16:05:31 yaml: line 6: did not find expected key
> ```
For me with debos 1.1.2-2, I seem to reproduce your issue:
Running /debos --artifactdir debos-tests --template-var 'ospack:"debian"'
debos-tests/test-key.yaml using kvm backend
2023/11/23 16:41:30 yaml: line 7: did not find expected key
But with a local checkout of debos upstream built with `go build`, your recipe
runs just fine (notice there are now no single-quotes around the --template-
var argument to the inner debos call):
Running /debos --artifactdir debos-tests --template-var ospack:debdfsa
debos-tests/test-key.yaml using kvm backend
2023/11/23 16:41:46 debootstrap
I think this has something to do with the recent shell escaping patches.
Perhaps there is some go module which isn't up to date in Debian causing the
additional single quotation marks around the inner debos call ?
> However if I run without `-t`, it succeeds:
>
> ```
> $ debos --print-recipe main.yaml
> 2023/11/23 23:10:55 Recipe '/home/user/debos/main.yaml':
> 2023/11/23 23:10:55
> architecture: amd64
> actions:
> - action: debootstrap
> suite: sid
> - action: pack
> file: debian.tar.gz
> Running /debos --artifactdir /home/user/debos /home/user/debos/main.yaml
> using kvm backend
> 2023/11/23 16:10:59 debootstrap
> 2023/11/23 16:10:59 Debootstrap | I: Target architecture can be executed
> 2023/11/23 16:10:59 Debootstrap | I: Retrieving InRelease
> [...]
> ```
>
> Another surprise: this is a regression introduced by package 1.1.2-2. If
> I downgrade to version 1.1.2-1, everything works fine.
>
> ```
> sudo apt install --allow-downgrades ./debos_1.1.2-1_amd64.deb
> ```
>
> I compared the Built-Using fields between both packages:
>
> ```
> -golang-1.21 (= 1.21.1-1)
> +golang-1.21 (= 1.21.4-1)
> +golang-github-alessio-shellescape (= 1.4.1-3)
> golang-github-cespare-xxhash (= 2.1.1-2)
> golang-github-docker-go-units (= 0.4.0-4)
> -golang-github-go-debos-fakemachine (= 0.0.6-1)
> +golang-github-go-debos-fakemachine (= 0.0.7-1)
> golang-github-google-uuid (= 1.3.0-1)
> -golang-github-klauspost-compress (= 1.15.12+ds1-3)
> +golang-github-klauspost-compress (= 1.17.2+ds1-1)
> golang-github-sjoerdsimons-ostree-go (= 0.0~git20201014.8fae757-2)
> golang-github-surma-gocpio (= 1.1.0+git20160926.fcb6877-1.1)
> golang-github-ulikunitz-xz (= 0.5.6-2)
> golang-go-flags (= 1.4.0-6)
> -golang-golang-x-sys (= 0.9.0-1)
> +golang-golang-x-sys (= 0.13.0-1)
> golang-gopkg-freddierice-go-losetup.v1 (= 0.0~git20170407.fc9adea-1.1)
> golang-yaml.v2 (= 2.4.0-4)
> ```
>
> I suppose the regression was introduced by one of the dependencies that
> changed. I have no idea how to troubleshot that... Tomorrow I'll try to
> rebuild a package to see if that magically fixes it.
>
> Can you try to reproduce this issue on your side, just to confirm?
>
> Thanks in advance,
>
> Arnaud
>
>
> -- System Information:
> Debian Release: trixie/sid
> APT prefers unstable
> APT policy: (500, 'unstable')
> Architecture: amd64 (x86_64)
>
> Kernel: Linux 6.5.0-4-amd64 (SMP w/8 CPU threads; PREEMPT)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not
> set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
>
> Versions of packages debos depends on:
> ii busybox 1:1.36.1-4
> ii debootstrap 1.0.133
> ii libc6 2.37-12
> ii libglib2.0-0 2.78.1-4
> ii libostree-1-1 2023.7-3
> ii qemu-system-x86 1:8.1.2+ds-1
> ii qemu-user-static 1:8.1.2+ds-1
> ii systemd-container 255~rc2-1
>
> Versions of packages debos recommends:
> ii bmap-tools 3.7-1
> ii bzip2 1.0.8-5+b1
> ii dosfstools 4.2-1
> ii e2fsprogs 1.47.0-2+b1
> ii fdisk 2.39.2-6
> ii linux-image-amd64 6.5.10-1
> ii mount 2.39.2-6
> ii ovmf 2023.05-2
> ii parted 3.6-3
> ii systemd-resolved 255
Bug#1056592: debos: yaml: line X: did not find expected key
Package: debos
Version: 1.1.2-2
Severity: normal
User: de...@kali.org
Usertags: origin-kali
Hi Christopher,
Here's a surprising bug report. The version of debos currently in Debian
unstable is a bit broken, in an odd way.
Consider the following recipe:
```
$ cat main.yaml
{{ $ospack := or .ospack "debian" }}
architecture: amd64
actions:
- action: debootstrap
suite: sid
- action: pack
file: {{ $ospack }}.tar.gz
```
If I run with `-t ospack:debian` it fails:
```
$ debos --print-recipe -t ospack:debian main.yaml
2023/11/23 23:05:27 Recipe '/home/user/debos/main.yaml':
2023/11/23 23:05:27
architecture: amd64
actions:
- action: debootstrap
suite: sid
- action: pack
file: debian.tar.gz
Running /debos --artifactdir /home/user/debos --template-var 'ospack:"debian"'
/home/user/debos/main.yaml using kvm backend
2023/11/23 16:05:31 yaml: line 6: did not find expected key
```
However if I run without `-t`, it succeeds:
```
$ debos --print-recipe main.yaml
2023/11/23 23:10:55 Recipe '/home/user/debos/main.yaml':
2023/11/23 23:10:55
architecture: amd64
actions:
- action: debootstrap
suite: sid
- action: pack
file: debian.tar.gz
Running /debos --artifactdir /home/user/debos /home/user/debos/main.yaml using
kvm backend
2023/11/23 16:10:59 debootstrap
2023/11/23 16:10:59 Debootstrap | I: Target architecture can be executed
2023/11/23 16:10:59 Debootstrap | I: Retrieving InRelease
[...]
```
Another surprise: this is a regression introduced by package 1.1.2-2. If
I downgrade to version 1.1.2-1, everything works fine.
```
sudo apt install --allow-downgrades ./debos_1.1.2-1_amd64.deb
```
I compared the Built-Using fields between both packages:
```
-golang-1.21 (= 1.21.1-1)
+golang-1.21 (= 1.21.4-1)
+golang-github-alessio-shellescape (= 1.4.1-3)
golang-github-cespare-xxhash (= 2.1.1-2)
golang-github-docker-go-units (= 0.4.0-4)
-golang-github-go-debos-fakemachine (= 0.0.6-1)
+golang-github-go-debos-fakemachine (= 0.0.7-1)
golang-github-google-uuid (= 1.3.0-1)
-golang-github-klauspost-compress (= 1.15.12+ds1-3)
+golang-github-klauspost-compress (= 1.17.2+ds1-1)
golang-github-sjoerdsimons-ostree-go (= 0.0~git20201014.8fae757-2)
golang-github-surma-gocpio (= 1.1.0+git20160926.fcb6877-1.1)
golang-github-ulikunitz-xz (= 0.5.6-2)
golang-go-flags (= 1.4.0-6)
-golang-golang-x-sys (= 0.9.0-1)
+golang-golang-x-sys (= 0.13.0-1)
golang-gopkg-freddierice-go-losetup.v1 (= 0.0~git20170407.fc9adea-1.1)
golang-yaml.v2 (= 2.4.0-4)
```
I suppose the regression was introduced by one of the dependencies that
changed. I have no idea how to troubleshot that... Tomorrow I'll try to
rebuild a package to see if that magically fixes it.
Can you try to reproduce this issue on your side, just to confirm?
Thanks in advance,
Arnaud
-- System Information:
Debian Release: trixie/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 6.5.0-4-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages debos depends on:
ii busybox1:1.36.1-4
ii debootstrap1.0.133
ii libc6 2.37-12
ii libglib2.0-0 2.78.1-4
ii libostree-1-1 2023.7-3
ii qemu-system-x861:8.1.2+ds-1
ii qemu-user-static 1:8.1.2+ds-1
ii systemd-container 255~rc2-1
Versions of packages debos recommends:
ii bmap-tools 3.7-1
ii bzip2 1.0.8-5+b1
ii dosfstools 4.2-1
ii e2fsprogs 1.47.0-2+b1
ii fdisk 2.39.2-6
ii linux-image-amd64 6.5.10-1
ii mount 2.39.2-6
ii ovmf 2023.05-2
ii parted 3.6-3
ii systemd-resolved 255~rc2-1
ii udev 255~rc2-1
ii xz-utils 5.4.4-0.1
ii zip3.0-13
Versions of packages debos suggests:
pn libslirp-helper
pn user-mode-linux
-- no debconf information
