It's true. Do you think I should revert? I sanitize the type of file that we get in?
Le lun. 1 juin 2026 à 12:25, Jean-Baptiste Kempf via ffmpeg-devel < [email protected]> a écrit : > This is particularly dangerous. > This makes the CI prone to injection to files from random people. > > On Mon, 1 Jun 2026, at 17:41, Romain Beauxis via ffmpeg-cvslog wrote: > > This is an automated email from the git hooks/post-receive script. > > > > Git pushed a commit to branch master > > in repository ffmpeg. > > > > commit 78fff004f021fc9b5a3467317eaab7deb446c955 > > Author: Romain Beauxis <[email protected]> > > AuthorDate: Wed May 27 08:09:12 2026 -0500 > > Commit: Romain Beauxis <[email protected]> > > CommitDate: Mon Jun 1 10:40:57 2026 -0500 > > > > .forgejo: add support for ephemeral FATE samples via PR attachments > > > > Developers can attach sample files to a PR and list their target > paths > > within the fate-suite in a fate-samples block in the PR description: > > > > ```fate-samples > > vorbis/tos.ogg > > mov/some-new-sample.mov > > ``` > > > > A new inject-pr-samples.py script fetches the PR metadata from the > > Forgejo API, resolves each listed path to its matching attachment by > > filename, and downloads the files into the fate-suite directory > before > > FATE runs. > > > > The script validates that pr-number is an integer, that paths are > > relative, contain no '..', and are at most 3 components deep > (matching > > the deepest paths in the existing fate-suite). Attachment URLs are > > restricted to the code.ffmpeg.org domain. > > > > The script exports a new_samples=true/false output via > $FORGEJO_OUTPUT. > > After FATE completes, a final workflow step fails the run if any new > > sample was injected, reminding contributors to add their samples to > the > > official fate-suite before the PR can be merged. > > > > The script can also be used locally: > > SAMPLES=/path/to/fate-suite .forgejo/inject-pr-samples.py > <pr-number> > > --- > > .forgejo/inject-pr-samples.py | 174 > ++++++++++++++++++++++++++++++++++++++++++ > > .forgejo/workflows/test.yml | 18 +++++ > > 2 files changed, 192 insertions(+) > > > > diff --git a/.forgejo/inject-pr-samples.py > > b/.forgejo/inject-pr-samples.py > > new file mode 100755 > > index 0000000000..3f50067751 > > --- /dev/null > > +++ b/.forgejo/inject-pr-samples.py > > @@ -0,0 +1,174 @@ > > +#!/usr/bin/env python3 > > +# Copyright (c) 2026 Romain Beauxis <[email protected]> > > +# > > +# Redistribution and use in source and binary forms, with or without > > +# modification, are permitted provided that the following conditions > > are met: > > +# > > +# 1. Redistributions of source code must retain the above copyright > > notice, > > +# this list of conditions and the following disclaimer. > > +# 2. Redistributions in binary form must reproduce the above copyright > > notice, > > +# this list of conditions and the following disclaimer in the > > documentation > > +# and/or other materials provided with the distribution. > > +# > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > > "AS IS" > > +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED > > TO, THE > > +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > > PURPOSE > > +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR > > CONTRIBUTORS BE > > +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > > +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF > > +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR > > BUSINESS > > +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER > > IN > > +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR > > OTHERWISE) > > +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED > > OF THE > > +# POSSIBILITY OF SUCH DAMAGE. > > + > > +"""Inject PR attachment samples into the fate-suite directory. > > + > > +Usage: inject-pr-samples.py <pr-number> > > + > > +Reads SAMPLES from the environment (defaults to fate-suite). For each > > path > > +listed in a ```fate-samples``` block in the PR description, downloads > > the > > +matching PR attachment into $SAMPLES/<path>. > > + > > +The PR description should contain a block like: > > + > > + ```fate-samples > > + vorbis/tos.ogg > > + mov/some-new-sample.mov > > + ``` > > + > > +Each filename must match a file attached to the PR. > > +""" > > + > > +import hashlib > > +import json > > +import os > > +import re > > +import sys > > +import tempfile > > +import urllib.request > > +from pathlib import Path, PurePosixPath > > + > > +FORGEJO_API = > > "https://code.ffmpeg.org/api/v1/repos/ffmpeg/ffmpeg/issues"; > > +ATTACHMENT_BASE = "https://code.ffmpeg.org/attachments/"; > > + > > + > > +def fetch_json(url): > > + with urllib.request.urlopen(url) as r: > > + return json.load(r) > > + > > + > > +def parse_fate_samples(body): > > + paths = [] > > + in_block = False > > + for line in body.splitlines(): > > + if line == "```fate-samples": > > + in_block = True > > + elif line == "```" and in_block: > > + break > > + elif in_block: > > + parts = line.split() > > + if len(parts) == 1: > > + paths.append(parts[0]) > > + return paths > > + > > + > > +MAX_PATH_DEPTH = 3 > > + > > + > > +def validate_path(path): > > + p = PurePosixPath(path) > > + if p.is_absolute(): > > + raise ValueError(f"path must be relative: {path!r}") > > + if ".." in p.parts: > > + raise ValueError(f"path must not contain '..': {path!r}") > > + if not p.parts: > > + raise ValueError(f"empty path") > > + if len(p.parts) > MAX_PATH_DEPTH: > > + raise ValueError(f"path too deep (max {MAX_PATH_DEPTH} > > components): {path!r}") > > + > > + > > +def validate_url(url): > > + if not url.startswith(ATTACHMENT_BASE): > > + raise ValueError(f"unexpected attachment URL: {url!r}") > > + > > + > > +def digest(path): > > + h = hashlib.sha256() > > + with open(path, "rb") as f: > > + while chunk := f.read(1 << 16): > > + h.update(chunk) > > + return h.digest() > > + > > + > > +def download(url, dst): > > + dst.parent.mkdir(parents=True, exist_ok=True) > > + with tempfile.NamedTemporaryFile(dir=dst.parent, delete=False) as > > tmp: > > + tmp_path = Path(tmp.name) > > + try: > > + with urllib.request.urlopen(url) as r: > > + while chunk := r.read(1 << 16): > > + tmp.write(chunk) > > + if dst.exists() and digest(dst) != digest(tmp_path): > > + raise ValueError(f"already exists with different > > content: {dst}") > > + tmp_path.rename(dst) > > + except: > > + tmp_path.unlink(missing_ok=True) > > + raise > > + > > + > > +def main(): > > + if len(sys.argv) != 2 or not re.fullmatch(r"[0-9]+", sys.argv[1]): > > + print(f"Usage: {sys.argv[0]} <pr-number>", file=sys.stderr) > > + sys.exit(1) > > + > > + pr_number = sys.argv[1] > > + samples_dir = Path(os.environ.get("SAMPLES", "fate-suite")) > > + > > + pr = fetch_json(f"{FORGEJO_API}/{pr_number}") > > + assets = {a["name"]: a["browser_download_url"] for a in > > pr.get("assets", [])} > > + paths = parse_fate_samples(pr.get("body", "")) > > + > > + if not paths: > > + sys.exit(0) > > + > > + new_samples = False > > + > > + for path in paths: > > + try: > > + validate_path(path) > > + except ValueError as e: > > + print(f"fate-samples: {e}", file=sys.stderr) > > + sys.exit(1) > > + > > + name = PurePosixPath(path).name > > + url = assets.get(name) > > + if url is None: > > + print(f"fate-samples: no attachment named {name!r}", > > file=sys.stderr) > > + sys.exit(1) > > + > > + try: > > + validate_url(url) > > + except ValueError as e: > > + print(f"fate-samples: {e}", file=sys.stderr) > > + sys.exit(1) > > + > > + dst = samples_dir / path > > + is_new = not dst.exists() > > + try: > > + download(url, dst) > > + except ValueError as e: > > + print(f"fate-samples: {e}", file=sys.stderr) > > + sys.exit(1) > > + if is_new: > > + new_samples = True > > + print(f"Injected: {path}") > > + > > + output_file = os.environ.get("FORGEJO_OUTPUT") > > + if output_file: > > + with open(output_file, "a") as f: > > + print(f"new_samples={'true' if new_samples else 'false'}", > > file=f) > > + > > + > > +if __name__ == "__main__": > > + main() > > diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml > > index 342120188e..3af1522b88 100644 > > --- a/.forgejo/workflows/test.yml > > +++ b/.forgejo/workflows/test.yml > > @@ -58,11 +58,20 @@ jobs: > > with: > > path: fate-suite > > key: fate-suite-${{ steps.fate.outputs.hash }} > > + - name: Inject PR Samples > > + id: inject > > + if: ${{ forge.event_name == 'pull_request' }} > > + run: SAMPLES=$PWD/fate-suite .forgejo/inject-pr-samples.py ${{ > > forge.event.pull_request.number }} > > - name: Run Fate > > run: | > > LD_LIBRARY_PATH="$(printf "%s:" "$PWD"/lib*)$PWD" make fate > > fate-build SAMPLES="$PWD/fate-suite" -j$(nproc) || FATERES=$? > > find . -name "*.err" -exec printf '::group::%s\n' {} \; > > -exec cat {} \; -exec printf '::endgroup::\n' \; > > exit ${FATERES:-0} > > + - name: Fail if new samples were injected > > + if: ${{ steps.inject.outputs.new_samples == 'true' }} > > + run: | > > + echo "New FATE samples were injected from PR attachments. > > Please add them to the official fate-suite before merging." > > + exit 1 > > run_fate_full: > > name: Fate (Full, ${{ matrix.target_exec }}) > > strategy: > > @@ -110,6 +119,10 @@ jobs: > > with: > > path: fate-suite > > key: fate-suite-${{ steps.fate.outputs.hash }} > > + - name: Inject PR Samples > > + id: inject > > + if: ${{ forge.event_name == 'pull_request' }} > > + run: SAMPLES=$PWD/fate-suite > > ffmpeg/.forgejo/inject-pr-samples.py ${{ > > forge.event.pull_request.number }} > > - name: Run Fate > > run: | > > if [[ "${{ matrix.target_exec }}" == "wine" ]]; then > > @@ -119,3 +132,8 @@ jobs: > > LD_LIBRARY_PATH="$(printf "%s:" "$PWD"/lib*)$PWD" make -C > > build fate fate-build SAMPLES="$PWD/fate-suite" -j$(nproc) || FATERES=$? > > find . -name "*.err" -exec printf '::group::%s\n' {} \; > > -exec cat {} \; -exec printf '::endgroup::\n' \; > > exit ${FATERES:-0} > > + - name: Fail if new samples were injected > > + if: ${{ steps.inject.outputs.new_samples == 'true' }} > > + run: | > > + echo "New FATE samples were injected from PR attachments. > > Please add them to the official fate-suite before merging." > > + exit 1 > > > > _______________________________________________ > > ffmpeg-cvslog mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > -- > Jean-Baptiste Kempf - President > +33 672 704 734 > https://jbkempf.com/ > _______________________________________________ > ffmpeg-devel mailing list -- [email protected] > To unsubscribe send an email to [email protected] > _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
