On Tue, 08 Mar 2022 at 01:04:17 +0000, Simon McVittie wrote: > This update isn't finalized yet, but does it look like something of a > scale that you would be likely to accept in a point release?
I've tested and uploaded the proposed update (no changes except for d/changelog, updated diff attached), but please let me know if any of it needs to be reverted. > all are straightforward backports, either from upstream release 2022.2 > (in unstable, not in testing yet) The version I'm backporting from is now in testing. > I will give this some real-world testing via Flatpak on bullseye > before upload (in particular to confirm that the eCryptFS fix has been > successful), but so far it has only had automated tests. I confirm that the eCryptFS issue is indeed fixed by these changes. Thanks, smcv
diffstat for ostree-2020.8 ostree-2020.8 debian/changelog | 27 ++++ debian/control | 2 debian/gbp.conf | 2 debian/patches/Fall-back-if-copy_file_range-fails-with-EINVAL.patch | 31 +++++ debian/patches/Fix-marking-static-delta-commits-as-partial.patch | 60 ++++++++++ debian/patches/Fix-translation-of-file-URIs-into-paths.patch | 57 +++++++++ debian/patches/lib-Fix-a-bad-call-to-g_file_get_child.patch | 39 ++++++ debian/patches/libotutil-Avoid-infinite-recursion-during-error-unwinding.patch | 36 ++++++ debian/patches/series | 5 libglnx/glnx-fdio.c | 2 src/libostree/ostree-repo-checkout.c | 2 src/libostree/ostree-repo-pull.c | 7 - src/libotutil/ot-variant-builder.c | 2 13 files changed, 265 insertions(+), 7 deletions(-) diff -Nru ostree-2020.8/debian/changelog ostree-2020.8/debian/changelog --- ostree-2020.8/debian/changelog 2020-11-22 13:24:35.000000000 +0000 +++ ostree-2020.8/debian/changelog 2022-03-10 11:59:18.000000000 +0000 @@ -1,3 +1,30 @@ +ostree (2020.8-2+deb11u1) bullseye; urgency=medium + + * d/gbp.conf, d/control: Branch for Debian 11 updates + * Backport various bug fixes from newer libostree releases. + Each of these fixes an issue that was reported against Flatpak when + using the libostree from Debian 11, either via bullseye or + buster-backports. + - d/p/Fall-back-if-copy_file_range-fails-with-EINVAL.patch: + Add patch to fall back if copy_file_range fails with EINVAL. + This fixes an incompatibility with eCryptFS, in particular when + using Flatpak in an eCryptFS home directory. (Closes: #1004467) + - d/p/libotutil-Avoid-infinite-recursion-during-error-unwinding.patch: + Avoid infinite recursion when recovering from certain errors, in + particular the one that was a symptom of #1004467. + - d/p/Fix-marking-static-delta-commits-as-partial.patch: + Mark commits as partial before downloading, to avoid Flatpak and other + ostree users getting into a state where a failed download cannot be + resumed. + - d/p/lib-Fix-a-bad-call-to-g_file_get_child.patch: + Fix an assertion failure when using a backport or local build of + GLib >= 2.71 + - d/p/Fix-translation-of-file-URIs-into-paths.patch: + Fix the ability to fetch OSTree content from paths containing + non-URI characters (such as backslashes) or non-ASCII + + -- Simon McVittie <s...@debian.org> Thu, 10 Mar 2022 11:59:18 +0000 + ostree (2020.8-2) unstable; urgency=medium * d/p/test-pull-summary-sigs-Set-timestamps-to-serve-expected-f.patch: diff -Nru ostree-2020.8/debian/control ostree-2020.8/debian/control --- ostree-2020.8/debian/control 2020-11-22 13:24:35.000000000 +0000 +++ ostree-2020.8/debian/control 2022-03-10 11:59:18.000000000 +0000 @@ -49,7 +49,7 @@ Rules-Requires-Root: no Standards-Version: 4.5.0 Homepage: https://github.com/ostreedev/ostree/ -Vcs-Git: https://salsa.debian.org/debian/ostree.git +Vcs-Git: https://salsa.debian.org/debian/ostree.git -b debian/bullseye Vcs-Browser: https://salsa.debian.org/debian/ostree Package: gir1.2-ostree-1.0 diff -Nru ostree-2020.8/debian/gbp.conf ostree-2020.8/debian/gbp.conf --- ostree-2020.8/debian/gbp.conf 2020-11-22 13:24:35.000000000 +0000 +++ ostree-2020.8/debian/gbp.conf 2022-03-10 11:59:18.000000000 +0000 @@ -1,7 +1,7 @@ [DEFAULT] pristine-tar = True compression = xz -debian-branch = debian/master +debian-branch = debian/bullseye upstream-branch = upstream/latest patch-numbers = False upstream-vcs-tag = v%(version)s diff -Nru ostree-2020.8/debian/patches/Fall-back-if-copy_file_range-fails-with-EINVAL.patch ostree-2020.8/debian/patches/Fall-back-if-copy_file_range-fails-with-EINVAL.patch --- ostree-2020.8/debian/patches/Fall-back-if-copy_file_range-fails-with-EINVAL.patch 1970-01-01 01:00:00.000000000 +0100 +++ ostree-2020.8/debian/patches/Fall-back-if-copy_file_range-fails-with-EINVAL.patch 2022-03-10 11:59:18.000000000 +0000 @@ -0,0 +1,31 @@ +From: Olaf Leidinger <ol...@mescharet.de> +Date: Mon, 24 Jan 2022 16:12:57 +0100 +Subject: Fall back if copy_file_range fails with EINVAL + +Although EINVAL usually indicates a programming error, ecryptfs (and +possibly other stacked filesystems) returns EINVAL for attempts to +copy_file_range() or sendfile() between files on that filesystem. + +Bug: https://gitlab.gnome.org/GNOME/libglnx/-/issues/3 +Bug: https://github.com/ostreedev/ostree/issues/2525 +Bug: https://github.com/flatpak/flatpak/issues/4673 +Bug-Debian: https://bugs.debian.org/1004467 +Applied-upstream: libglnx commit:24231a956a4b849087fbf01173cdebb53e1bd60b +Applied-upstream: libostree 2022.2, commit:0ebf9d9f64647d0b8b95cda112c7854be6f58ed3 +--- + libglnx/glnx-fdio.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/libglnx/glnx-fdio.c b/libglnx/glnx-fdio.c +index 422bc2d..0bb26f4 100644 +--- a/libglnx/glnx-fdio.c ++++ b/libglnx/glnx-fdio.c +@@ -829,7 +829,7 @@ glnx_regfile_copy_bytes (int fdf, int fdt, off_t max_bytes) + have_cfr = 0; + try_cfr = false; + } +- else if (G_IN_SET (errno, EXDEV, EOPNOTSUPP)) ++ else if (G_IN_SET (errno, EXDEV, EINVAL, EOPNOTSUPP)) + /* We won't try cfr again for this run, but let's be + * conservative and not mark it as available/unavailable until + * we know for sure. diff -Nru ostree-2020.8/debian/patches/Fix-marking-static-delta-commits-as-partial.patch ostree-2020.8/debian/patches/Fix-marking-static-delta-commits-as-partial.patch --- ostree-2020.8/debian/patches/Fix-marking-static-delta-commits-as-partial.patch 1970-01-01 01:00:00.000000000 +0100 +++ ostree-2020.8/debian/patches/Fix-marking-static-delta-commits-as-partial.patch 2022-03-10 11:59:18.000000000 +0000 @@ -0,0 +1,60 @@ +From: Phaedrus Leeds <mwle...@protonmail.com> +Date: Sat, 19 Feb 2022 07:55:02 -0600 +Subject: Fix marking static delta commits as partial + +This patch makes it so that we mark the .commit file from a static delta +as partial before writing the commit to the staging directory. This +exactly mirrors what we do in meta_fetch_on_complete() when writing the +commit on that codepath, which should lend some credibility to the +correctness of this patch. + +I have checked that this fixes an issue Flatpak users have been +encountering (https://github.com/flatpak/flatpak/issues/3479) which +results in error messages like "error: Failed to install +org.freedesktop.Sdk.Extension.texlive: Failed to read commit +c7958d966cfa8b80a42877d1d6124831d7807f93c89461a2a586956aa28d438a: No +such metadata object +8bdaa943b957f3cf14d19301c59c7eec076e57389e0fbb3ef5d30082e47a178f.dirtree" + +Here's the sequence of events that lead to the error: +1. An install operation is started that fetches static deltas. +2. The fetch is interrupted for some reason such as network connectivity + dropping. +3. The .commit and .commitmeta files for the commit being pulled are + left in the staging dir, e.g. + "~/.local/share/flatpak/repo/tmp/staging-dfe862b2-13fc-49a2-ac92-5a59cc0d8e18-RURckd" +4. There is no `.commitpartial` file for the commit in + "~/.local/share/flatpak/repo/state/" +5. The next time the user attempts the install, libostree reuses the + existing staging dir, pulls the commit and commitmeta objects into + the repo from the staging dir on the assumption that it's a complete + commit. +6. Flatpak then tries to deploy the commit but fails in + ostree_repo_read_commit() in flatpak_dir_deploy(), leading to the + error message "Failed to read commit ..." +7. This happens again any subsequent time the user attempts the install, + until the incomplete commit is removed with "flatpak repair --user". + +I will try to also add a workaround in Flatpak so this is fixed even +when Flatpak links against affected versions of libostree. + +Bug: https://github.com/flatpak/flatpak/issues/3479 +Origin: upstream, 2022.2, commit:5d3b1ca37a508e9f80702b7ef7383fe95253ec6a +--- + src/libostree/ostree-repo-pull.c | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/src/libostree/ostree-repo-pull.c b/src/libostree/ostree-repo-pull.c +index 758c505..4f44191 100644 +--- a/src/libostree/ostree-repo-pull.c ++++ b/src/libostree/ostree-repo-pull.c +@@ -2214,6 +2214,9 @@ process_one_static_delta (OtPullData *pull_data, + ref, cancellable, error)) + return FALSE; + ++ if (!ostree_repo_mark_commit_partial (pull_data->repo, to_revision, TRUE, error)) ++ return FALSE; ++ + if (detached_data && !ostree_repo_write_commit_detached_metadata (pull_data->repo, + to_revision, + detached_data, diff -Nru ostree-2020.8/debian/patches/Fix-translation-of-file-URIs-into-paths.patch ostree-2020.8/debian/patches/Fix-translation-of-file-URIs-into-paths.patch --- ostree-2020.8/debian/patches/Fix-translation-of-file-URIs-into-paths.patch 1970-01-01 01:00:00.000000000 +0100 +++ ostree-2020.8/debian/patches/Fix-translation-of-file-URIs-into-paths.patch 2022-03-10 11:59:18.000000000 +0000 @@ -0,0 +1,57 @@ +From: Phaedrus Leeds <mwle...@endlessos.org> +Date: Wed, 10 Mar 2021 10:02:14 -0800 +Subject: Fix translation of file:// URIs into paths + +Currently if a file path contains a special character such as '\', and +that character is encoded into a file:// URI that is passed to +ostree_repo_pull_with_options(), the percent encoding will remain in the +path passed to g_file_new() (in the case of backslash %5C) and the pull +will then fail with a file not found error. This is an important edge +case to handle because by default on many Linux distributions a +filesystem with no label is mounted at a path based on its UUID, and +this is then passed to systemd-escape by Flatpak (when +--enable-auto-sideloading was used at compile time) to create a symbolic +link such as this which contains backslashes: + +$ ls -l /run/flatpak/sideload-repos/ +total 0 +lrwxrwxrwx 1 mwleeds mwleeds 55 Mar 9 14:21 +'automount-run-media-mwleeds-29419e8f\x2dc680\x2d4e95\x2d9a31\x2d2cc907d421cb' +-> /run/media/mwleeds/29419e8f-c680-4e95-9a31-2cc907d421cb + +And Flatpak then passes libostree a file:// URI containing that path, to +implement sideloading (pulling content from the USB drive). + +This results in an error like: + +Error: While pulling app/org.videolan.VLC/x86_64/stable from remote +flathub: +/run/flatpak/sideload-repos/automount-run-media-mwleeds-29419e8f%5Cx2dc680%5Cx2d4e95%5Cx2d9a31%5Cx2d2cc907d421cb/.ostree/repo: +opendir(/run/flatpak/sideload-repos/automount-run-media-mwleeds-29419e8f%5Cx2dc680%5Cx2d4e95%5Cx2d9a31%5Cx2d2cc907d421cb/.ostree/repo): +No such file or directory + +This patch avoids such errors by using g_file_new_for_uri() instead of +g_file_new_for_path(), so that GLib handles the %-decoding for us. + +Bug: https://community.endlessos.com/t/can-not-install-vlc-from-usb-drive-3-9-3/16353 +Bug: https://github.com/flatpak/flatpak/issues/4378 +Origin: upstream, 2021.1, commit:19577522f8eacd868cf25d53e1ac0e7f424e952b +--- + src/libostree/ostree-repo-pull.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/src/libostree/ostree-repo-pull.c b/src/libostree/ostree-repo-pull.c +index 4f44191..ac9e7a7 100644 +--- a/src/libostree/ostree-repo-pull.c ++++ b/src/libostree/ostree-repo-pull.c +@@ -4084,8 +4084,8 @@ ostree_repo_pull_with_options (OstreeRepo *self, + */ + if (g_str_equal (first_scheme, "file") && !pull_data->require_static_deltas) + { +- g_autofree char *path = _ostree_fetcher_uri_get_path (first_uri); +- g_autoptr(GFile) remote_repo_path = g_file_new_for_path (path); ++ g_autofree char *uri = _ostree_fetcher_uri_to_string (first_uri); ++ g_autoptr(GFile) remote_repo_path = g_file_new_for_uri (uri); + pull_data->remote_repo_local = ostree_repo_new (remote_repo_path); + if (!ostree_repo_open (pull_data->remote_repo_local, cancellable, error)) + goto out; diff -Nru ostree-2020.8/debian/patches/lib-Fix-a-bad-call-to-g_file_get_child.patch ostree-2020.8/debian/patches/lib-Fix-a-bad-call-to-g_file_get_child.patch --- ostree-2020.8/debian/patches/lib-Fix-a-bad-call-to-g_file_get_child.patch 1970-01-01 01:00:00.000000000 +0100 +++ ostree-2020.8/debian/patches/lib-Fix-a-bad-call-to-g_file_get_child.patch 2022-03-10 11:59:18.000000000 +0000 @@ -0,0 +1,39 @@ +From: Valentin David <m...@valentindavid.com> +Date: Tue, 2 Nov 2021 19:49:04 +0100 +Subject: lib: Fix a bad call to g_file_get_child + +In Glib, since commit 3a6e8bc8876e149c36b6b14c6a25a718edb581ed, +`g_file_get_child` does not accept absolute path as paramater anymore. + +The broken assertion was encountered during `ostree admin deploy` +command for the checkout of subpath `etc`. + +Example of error log: +``` +(ostree admin deploy:1640): GLib-GIO-CRITICAL **: 03:42:00.570: g_file_get_child: assertion '!g_path_is_absolute (name)' failed + +(ostree admin deploy:1640): GLib-GIO-CRITICAL **: 03:42:00.570: g_file_query_info: assertion 'G_IS_FILE (file)' failed +** +OSTree:ERROR:src/ostree/ot-main.c:232:ostree_run: assertion failed: (success || error) +Bail out! OSTree:ERROR:src/ostree/ot-main.c:232:ostree_run: assertion failed: (success || error) +``` + +Bug: https://github.com/flatpak/flatpak/pull/4707 +Origin: upstream, 2021.6, commit:adc097a2edb1b7aaf5604043b4b1d5bd6ef8a308 +--- + src/libostree/ostree-repo-checkout.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/libostree/ostree-repo-checkout.c b/src/libostree/ostree-repo-checkout.c +index 00c6a77..c6645d7 100644 +--- a/src/libostree/ostree-repo-checkout.c ++++ b/src/libostree/ostree-repo-checkout.c +@@ -1381,7 +1381,7 @@ ostree_repo_checkout_at (OstreeRepo *self, + g_autoptr(GFile) target_dir = NULL; + + if (strcmp (options->subpath, "/") != 0) +- target_dir = g_file_get_child (commit_root, options->subpath); ++ target_dir = g_file_resolve_relative_path (commit_root, options->subpath); + else + target_dir = g_object_ref (commit_root); + g_autoptr(GFileInfo) target_info = diff -Nru ostree-2020.8/debian/patches/libotutil-Avoid-infinite-recursion-during-error-unwinding.patch ostree-2020.8/debian/patches/libotutil-Avoid-infinite-recursion-during-error-unwinding.patch --- ostree-2020.8/debian/patches/libotutil-Avoid-infinite-recursion-during-error-unwinding.patch 1970-01-01 01:00:00.000000000 +0100 +++ ostree-2020.8/debian/patches/libotutil-Avoid-infinite-recursion-during-error-unwinding.patch 2022-03-10 11:59:18.000000000 +0000 @@ -0,0 +1,36 @@ +From: Simon McVittie <s...@collabora.com> +Date: Fri, 28 Jan 2022 11:08:00 +0000 +Subject: libotutil: Avoid infinite recursion during error unwinding + +When we clean up from an error, for example copy_file_range() failing +while we generate a static delta (perhaps caused by +https://gitlab.gnome.org/GNOME/libglnx/-/issues/3 or by a +genuine write error), we might free a variant builder that has a +non-null parent. Previously, this caused infinite recursion and a stack +overflow, repeatedly freeing the same object, but Luca Bruno suggested +that the intention here appears to have been to free the parent object. + +Partially resolves https://github.com/ostreedev/ostree/issues/2525 +(the other bug reported in that issue needs to be resolved by updating +libglnx to a version where libglnx#3 has been fixed). + +Signed-off-by: Simon McVittie <s...@collabora.com> +Bug: https://github.com/ostreedev/ostree/issues/2525 +Applied-upstream: 2022.2, commit:920f85cabc656e4a7c07574aa9af211b6153756d +--- + src/libotutil/ot-variant-builder.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/libotutil/ot-variant-builder.c b/src/libotutil/ot-variant-builder.c +index 6636068..afbdc68 100644 +--- a/src/libotutil/ot-variant-builder.c ++++ b/src/libotutil/ot-variant-builder.c +@@ -832,7 +832,7 @@ static void + ot_variant_builder_info_free (OtVariantBuilderInfo *info) + { + if (info->parent) +- ot_variant_builder_info_free (info); ++ ot_variant_builder_info_free (info->parent); + + g_variant_type_free (info->type); + g_array_unref (info->child_ends); diff -Nru ostree-2020.8/debian/patches/series ostree-2020.8/debian/patches/series --- ostree-2020.8/debian/patches/series 2020-11-22 13:24:35.000000000 +0000 +++ ostree-2020.8/debian/patches/series 2022-03-10 11:59:18.000000000 +0000 @@ -1,2 +1,7 @@ test-pull-summary-sigs-Set-timestamps-to-serve-expected-f.patch +Fall-back-if-copy_file_range-fails-with-EINVAL.patch +libotutil-Avoid-infinite-recursion-during-error-unwinding.patch +Fix-marking-static-delta-commits-as-partial.patch +lib-Fix-a-bad-call-to-g_file_get_child.patch +Fix-translation-of-file-URIs-into-paths.patch debian/Skip-test-pull-repeated-during-CI.patch diff -Nru ostree-2020.8/libglnx/glnx-fdio.c ostree-2020.8/libglnx/glnx-fdio.c --- ostree-2020.8/libglnx/glnx-fdio.c 2020-10-07 16:33:17.000000000 +0100 +++ ostree-2020.8/libglnx/glnx-fdio.c 2022-03-11 17:46:49.000000000 +0000 @@ -829,7 +829,7 @@ have_cfr = 0; try_cfr = false; } - else if (G_IN_SET (errno, EXDEV, EOPNOTSUPP)) + else if (G_IN_SET (errno, EXDEV, EINVAL, EOPNOTSUPP)) /* We won't try cfr again for this run, but let's be * conservative and not mark it as available/unavailable until * we know for sure. diff -Nru ostree-2020.8/src/libostree/ostree-repo-checkout.c ostree-2020.8/src/libostree/ostree-repo-checkout.c --- ostree-2020.8/src/libostree/ostree-repo-checkout.c 2020-10-07 16:33:08.000000000 +0100 +++ ostree-2020.8/src/libostree/ostree-repo-checkout.c 2022-03-11 17:46:49.000000000 +0000 @@ -1381,7 +1381,7 @@ g_autoptr(GFile) target_dir = NULL; if (strcmp (options->subpath, "/") != 0) - target_dir = g_file_get_child (commit_root, options->subpath); + target_dir = g_file_resolve_relative_path (commit_root, options->subpath); else target_dir = g_object_ref (commit_root); g_autoptr(GFileInfo) target_info = diff -Nru ostree-2020.8/src/libostree/ostree-repo-pull.c ostree-2020.8/src/libostree/ostree-repo-pull.c --- ostree-2020.8/src/libostree/ostree-repo-pull.c 2020-11-10 11:34:43.000000000 +0000 +++ ostree-2020.8/src/libostree/ostree-repo-pull.c 2022-03-11 17:46:49.000000000 +0000 @@ -2214,6 +2214,9 @@ ref, cancellable, error)) return FALSE; + if (!ostree_repo_mark_commit_partial (pull_data->repo, to_revision, TRUE, error)) + return FALSE; + if (detached_data && !ostree_repo_write_commit_detached_metadata (pull_data->repo, to_revision, detached_data, @@ -4081,8 +4084,8 @@ */ if (g_str_equal (first_scheme, "file") && !pull_data->require_static_deltas) { - g_autofree char *path = _ostree_fetcher_uri_get_path (first_uri); - g_autoptr(GFile) remote_repo_path = g_file_new_for_path (path); + g_autofree char *uri = _ostree_fetcher_uri_to_string (first_uri); + g_autoptr(GFile) remote_repo_path = g_file_new_for_uri (uri); pull_data->remote_repo_local = ostree_repo_new (remote_repo_path); if (!ostree_repo_open (pull_data->remote_repo_local, cancellable, error)) goto out; diff -Nru ostree-2020.8/src/libotutil/ot-variant-builder.c ostree-2020.8/src/libotutil/ot-variant-builder.c --- ostree-2020.8/src/libotutil/ot-variant-builder.c 2020-10-07 16:33:08.000000000 +0100 +++ ostree-2020.8/src/libotutil/ot-variant-builder.c 2022-03-11 17:46:49.000000000 +0000 @@ -832,7 +832,7 @@ ot_variant_builder_info_free (OtVariantBuilderInfo *info) { if (info->parent) - ot_variant_builder_info_free (info); + ot_variant_builder_info_free (info->parent); g_variant_type_free (info->type); g_array_unref (info->child_ends);