https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109098
Bug ID: 109098 Summary: Encoding errors on SARIF output for non-UTF-8 source files Product: gcc Version: 13.0 Status: UNCONFIRMED Severity: normal Priority: P3 Component: analyzer Assignee: dmalcolm at gcc dot gnu.org Reporter: dmalcolm at gcc dot gnu.org Target Milestone: --- I've run into issues with -fanalyzer integration testing when a diagnostic occurs in a source file with non-UTF-8 bytes in it. Specifically, haproxy-2.7.1's include/import/ist.h has a comment within function "istalloc" that isn't quite UTF-8. When a diagnostic occurs on this, with -fdiagnostics-format=sarif-file, the file is quoted in the SARIF output, but the individual bytes are copied directly into the JSON, leading to e.g.: $ sarif ls integration-tests/haproxy-2.7.1/haproxy-2.7.1/http_client.c.sarif Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 57, in load_sarif_file data = json.load(file_in) File "/usr/lib64/python3.8/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/lib64/python3.8/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 96006: invalid start byte The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/bin/sarif", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/sarif/cmdline/main.py", line 40, in main exitcode = args.func(args) File "/usr/local/lib/python3.8/site-packages/sarif/cmdline/main.py", line 384, in _ls ls_op.print_ls(args.files_or_dirs, args.output) File "/usr/local/lib/python3.8/site-packages/sarif/operations/ls_op.py", line 17, in print_ls sarif_files = loader.load_sarif_files(path) File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 30, in load_sarif_files path_exists = _add_path_to_sarif_file_set(path, ret) File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 17, in _add_path_to_sarif_file_set sarif_file_set.add_file(load_sarif_file(path)) File "/usr/local/lib/python3.8/site-packages/sarif/loader.py", line 60, in load_sarif_file raise IOError(f"Cannot load {file_path}") from exception OSError: integration-tests/haproxy-2.7.1/haproxy-2.7.1/http_client.c.sarif I'm working on a fix.