Bug#1010570: binaries in source without related source
On 2022-05-06 10:54:56, Tino Mettler wrote: > Hi Antoine, > > I just discussed this a bit on #debian-mentors in IRC. > > Conclusion: > > - autogenerated files in the upstream tarball might be okay, if they are > not used for the binary packages (upgrade.py, install.py) or if they > are regenerated during build time (qrc_resources.py) > > - the details about this should be documented in debian/copyright > > - the missing images to generate the qrc_resources.py file can be put > into debian/missing-sources > > - we can discuss this issue again with the upstream author once he has > recovered > > - it is unlikely that there will be a new upstream release > before this > > When we follow this, we don't need to repack the upstream source, just > add the images to debian/missing-sources and document all that in > debian/copyright. > > What do you thing about this? That makes total sense to me. -- If quantum mechanics hasn't profoundly shocked you, you haven't understood it yet. - Niels Bohr
Bug#1010570: binaries in source without related source
Hi Antoine, I just discussed this a bit on #debian-mentors in IRC. Conclusion: - autogenerated files in the upstream tarball might be okay, if they are not used for the binary packages (upgrade.py, install.py) or if they are regenerated during build time (qrc_resources.py) - the details about this should be documented in debian/copyright - the missing images to generate the qrc_resources.py file can be put into debian/missing-sources - we can discuss this issue again with the upstream author once he has recovered - it is unlikely that there will be a new upstream release before this When we follow this, we don't need to repack the upstream source, just add the images to debian/missing-sources and document all that in debian/copyright. What do you thing about this? Regards, Tino
Bug#1010570: binaries in source without related source
On 2022-05-05 21:51:16, Tino Mettler wrote:
> Hi Antoine,
>
> I'm currently thinking about missing bits for a 0.9.33 release. I
> didn't switch a package to dfsg source before, so I'm new to that.
>
> It looks like it should be documented in debian/copyright, e.g. why the
> source tarball was repacked and what is the reason.
>
> There should be a changelog entry, which also refers to this bug.
That all sounds about right to me!
> Furthermore, I guess that the procedure to create the upstream tarball
> should be described, by providing the script that I used. This could be
> done in a README.Packaging.
Typically, the script lives in debian/rules, as a target
("get-orig-source"). This has actually been deprecated by Debian policy
4.1.4, interestingly:
https://www.debian.org/doc/debian-policy/upgrading-checklist.html#version-4-1-4
... but is still mentioned in the developer's reference:
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#repackaged-upstream-source
In general, I think that means *maybe* we could try harder to get uscan
to do the right thing here, but I understand if it seems impossible. :)
The documentation about that stuff can also live in README.source:
https://www.debian.org/doc/debian-policy/ch-source.html#source-package-handling-debian-readme-source
> Please let me know if I am missing something.
I think that's about right, thanks for the hard work!
--
We will create a civilization of the Mind in Cyberspace. May it be more
humane and fair than the world your governments have made before.
- John Perry Barlow, 1996
A Declaration of Independence of Cyberspace
Bug#1010570: binaries in source without related source
Hi Antoine, I'm currently thinking about missing bits for a 0.9.33 release. I didn't switch a package to dfsg source before, so I'm new to that. It looks like it should be documented in debian/copyright, e.g. why the source tarball was repacked and what is the reason. There should be a changelog entry, which also refers to this bug. Furthermore, I guess that the procedure to create the upstream tarball should be described, by providing the script that I used. This could be done in a README.Packaging. Please let me know if I am missing something. Regards, Tino
Bug#1010570: binaries in source without related source
On 2022-05-05 10:37:41, Tino Mettler wrote: > Hi, > > regarding the comments in the code, the binary blob in install.py and > upgrade.py can be generated during build time using tasks/generate.py. > However, both files are not part of the binary package so I see no > point in doing that. So I think replacing them with empty files in > the dfsg source tarball is okay. Agreed. -- Music gives a soul to the universe, wings to the mind, flight to the imagination and life to everything - Plato
Bug#1010570: binaries in source without related source
Hi, I pushed a branch that uses the dfsg source generated by the above script. It contains one additional change to generate the qrc_resources.py file. https://salsa.debian.org/scorpi-guest/rapid-photo-downloader/-/commits/debian/unstable-0.9.33+dfsg
Bug#1010570: binaries in source without related source
Am Wed, May 04, 2022 at 22:12:20 +0200 schrieb Tino Mettler:
> Am Wed, May 04, 2022 at 15:17:05 -0400 schrieb Antoine Beaupré:
>
> [...]
>
> > maybe we could just use a +ds tarball then. add the files to redact in
> > debian/copyright and then uscan will just do the right thing for us, no
> > need to worry about the git stuff.
>
> We need the images/ and tips/ folders from the upstream git repository.
>
> This script would work. I already prepared a package using the +dfsg upstream
> source generated by the script. The resources are compiles during
> build, the result looks good.
I forgot to git add the upgrade.py. Furthermore, Damon mentioned
install.py. And yes, it contains another binary blob (the same ZIP
archive as upgrade.py, I guess).
Here is an improved script:
#!/bin/sh -ex
git reset --hard $1
VERSION=$(sed '/^-/,$d' CHANGES.md | tail -n 1 | cut -d' ' -f1)
echo > upgrade.py
echo > install.py
git add upgrade.py install.py
git rm raphodo/qrc_resources.py
git commit -m "Remove source files with embedded binary blobs"
git archive HEAD --prefix=rapid-photo-downloader-$VERSION/ -o
../rapid-photo-downloader_${VERSION}+dfsg.orig.tar.xz
Regards,
Tino
Bug#1010570: binaries in source without related source
Am Wed, May 04, 2022 at 15:17:05 -0400 schrieb Antoine Beaupré:
[...]
> maybe we could just use a +ds tarball then. add the files to redact in
> debian/copyright and then uscan will just do the right thing for us, no
> need to worry about the git stuff.
We need the images/ and tips/ folders from the upstream git repository.
This script would work. I already prepared a package using the +dfsg upstream
source generated by the script. The resources are compiles during
build, the result looks good.
#!/bin/sh -ex
git reset --hard $1
VERSION=$(sed '/^-/,$d' CHANGES.md | tail -n 1 | cut -d' ' -f1)
echo > upgrade.py
git rm raphodo/qrc_resources.py
git commit -m "Remove source files with embedded binary blobs"
git archive HEAD --prefix=rapid-photo-downloader-$VERSION/ -o
../rapid-photo-downloader_${VERSION}+dfsg.orig.tar.xz
Regards,
Tino
Bug#1010570: binaries in source without related source
On 2022-05-04 21:06:38, Tino Mettler wrote: >> Am 04.05.2022 um 20:17 schrieb Antoine Beaupré : >> >> On 2022-05-04 20:07:28, Tino Mettler wrote: >>> Hi, >>> >>> the upstream git repository has no release tags, so I guess that we >>> have to drop uscan support and create the upstream source archive with >>> a selfmade script. >> >> we could lay down release tags ourselves if we can figure out what those >> are... > > Hi, > > that would be an option. I'm currently crafting a script that creates a > tarball. Just removing upgrade.py doesn't work, because it is referenced in > the translations and that will cause a build errors. Replacing it with an > empty file works, and should not affect Debian users. maybe we could just use a +ds tarball then. add the files to redact in debian/copyright and then uscan will just do the right thing for us, no need to worry about the git stuff. -- La guerre, c'est le massacre d'hommes qui ne se connaissent pas, au profit d'hommes qui se connaissent mais ne se massacreront pas. - Paul Valéry
Bug#1010570: binaries in source without related source
> Am 04.05.2022 um 20:17 schrieb Antoine Beaupré : > > On 2022-05-04 20:07:28, Tino Mettler wrote: >> Hi, >> >> the upstream git repository has no release tags, so I guess that we >> have to drop uscan support and create the upstream source archive with >> a selfmade script. > > we could lay down release tags ourselves if we can figure out what those > are... Hi, that would be an option. I'm currently crafting a script that creates a tarball. Just removing upgrade.py doesn't work, because it is referenced in the translations and that will cause a build errors. Replacing it with an empty file works, and should not affect Debian users. Regards, Tino
Bug#1010570: binaries in source without related source
On 2022-05-04 20:07:28, Tino Mettler wrote: > Hi, > > the upstream git repository has no release tags, so I guess that we > have to drop uscan support and create the upstream source archive with > a selfmade script. we could lay down release tags ourselves if we can figure out what those are...
Bug#1010570: binaries in source without related source
Hi, the upstream git repository has no release tags, so I guess that we have to drop uscan support and create the upstream source archive with a selfmade script. Regards, Tino
Bug#1010570: binaries in source without related source
On 2022-05-04 19:29:24, Tino Mettler wrote: > Am Wed, May 04, 2022 at 13:03:19 -0400 schrieb Antoine Beaupré: > > [...] > >> Maybe a solution that wouldn't involve upstream would be to change the >> way we generate the tarball, from git instead of the upstream tarball. >> >> After all, if we're going to rebuild thing from scratch ourselves >> anyways... > > I also think that would be the way to go, as the upstream author is > currently fighting with health issues. > > https://github.com/damonlynch/rapid-photo-downloader/issues/73#issuecomment-1117546807 Oh dear yeah, screw that. Let's not bother that poor upstream already. :) Let's make a new tarball with the upstream git repo instead, can you try that? Thanks! a. -- Any sufficiently advanced technology is indistinguishable from magic. - Arthur C. Clarke
Bug#1010570: binaries in source without related source
On 2022-05-04 17:41:42, Tino Mettler wrote: > Hi, > > the images used to generate qrc_resources.py are present in the > upstream git repository, but not in the release tarballs. Once they are > copied into the source directory of the Debian package, > qrc_resources.py can be regenerated using pyrcc5 (the PyQT resource > compiler). > > I opened an issue in the upstream git repository and described the > nature of this bug. I also asked if it is possible to provide signed > release tarballs with the images included. Maybe a solution that wouldn't involve upstream would be to change the way we generate the tarball, from git instead of the upstream tarball. After all, if we're going to rebuild thing from scratch ourselves anyways...
Bug#1010570: binaries in source without related source
Package: rapid-photo-downloader Version: 0.9.26-2 Severity: normal I'm not sure about this, but it seems to me that `upgrade.py` and `qrc_resources.py` ship binary data as source code. This is normally a red flag in Debian packages, because source code should typically be text file that are humanly understandable, or at least usable with free software tools. Basing myself on the excellent investigation done by Tino Mettler: https://salsa.debian.org/debian/rapid-photo-downloader/-/merge_requests/2#note_307363 ... i t seems like the `upgrade.py` file is fairly innocuous: it's actually an encoded ZIP file that has .mo files generated from the provided .po files. I'm not sure that needs to be removed, as there is probably an obvious source for those. The other file, `qrc_resources.py`, is more problematic. It bundles binary data like images and those don't seem to have an associated source in the source code. It's unclear if that file could be redistributed as is, as it's clearly not modifiable, and would possibly be a license violation. It seems like the source images for that file are missing from the upstream source as well, crucially. Normally, we should be able to recompile that file from source, but because those images are missing, it's not possible. I'm not an expert on those issues as I used to be, so I'm not sure about all this, but it seemed important to flag this as an issue on the package. -- System Information: Debian Release: 11.3 APT prefers stable-security APT policy: (500, 'stable-security'), (500, 'stable-debug'), (500, 'stable'), (1, 'unstable'), (1, 'testing') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.10.0-13-amd64 (SMP w/4 CPU threads) Kernel taint flags: TAINT_FIRMWARE_WORKAROUND Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages rapid-photo-downloader depends on: ii gir1.2-gexiv2-0.10 0.12.1-1 ii gir1.2-glib-2.01.66.1-1+b1 ii gir1.2-gstreamer-1.0 1.18.4-2.1 ii gir1.2-gudev-1.0 234-1 ii gir1.2-notify-0.7 0.7.9-3 ii gir1.2-udisks-2.0 2.9.2-2+deb11u1 ii gstreamer1.0-libav 1.18.4-3 ii gstreamer1.0-plugins-good 1.18.4-2 ii libgphoto2-6 2.5.27-1 ii libimage-exiftool-perl 12.16+dfsg-2 ii libmediainfo0v520.09+dfsg-2 ii libqt5svg5 5.15.2-3 ii python33.9.2-3 ii python3-arrow 1.2.1-1 ii python3-babel 2.8.0+dfsg.1-7 ii python3-colour 0.1.5-2 ii python3-dateutil 2.8.1-6 ii python3-easygui0.98.1-1 ii python3-gi 3.38.0-2 ii python3-gphoto21.9.0-1+b2 ii python3-gphoto2cffi [python3-gphoto2] 0.4.3~a1-1.1+b1 ii python3-psutil 5.8.0-1 ii python3-pymediainfo5.0.3-1 ii python3-pyqt5 5.15.2+dfsg-3 ii python3-requests 2.25.1+dfsg-2 ii python3-sortedcontainers 2.1.0-2 ii python3-tenacity 6.2.0-4 ii python3-tornado6.1.0-1+b1 ii python3-xdg0.27-2 ii python3-zmq20.0.0-1+b1 ii qt5-image-formats-plugins 5.15.2-2 Versions of packages rapid-photo-downloader recommends: ii libraw-bin 0.20.2-1 rapid-photo-downloader suggests no packages. -- debconf-show failed
