[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Matthias Klose
** Also affects: python3-defaults (Ubuntu Noble) Importance: Undecided Status: New ** Also affects: dh-python (Ubuntu Noble) Importance: Undecided Status: New ** Changed in: python3-defaults (Ubuntu) Assignee: (unassigned) => Matthias Klose (doko) -- You received this b

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Proposed upstream fix: https://salsa.debian.org/cpython- team/python3-defaults/-/merge_requests/15 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaults in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: p

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Robie Basak
Any suggestions on how to identify SRUs that are affected so that we can ensure that the necessary Breaks are in there? (also, do we know that Breaks will work as needed for an issue in the prerm?) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, wh

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Run something like `grep -E 'py3clean|dpkg -L' $prerm` on all prerm files in the existing package. On my system: ``` $ grep -l -E 'py3clean|dpkg -L' /var/lib/dpkg/info/*.prerm | wc -l 411 ``` -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Dave Jones
Having spent some time digging into this (at the request of the SRU team), I'll summarise my findings: Firstly, the analysis is correct: py3clean is ultimately the issue; packages that both use py3clean and have diversions are affected (diversions are the only circumstances that produce localised

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Dave Jones
Attaching debdiff for proposed changes to python3-minimal in oracular ** Patch added: "oracular.debdiff" https://bugs.launchpad.net/ubuntu/+source/python3-defaults/+bug/2075337/+attachment/5802638/+files/oracular.debdiff -- You received this bug notification because you are a member of Ubunt

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
First I thought that the fallback dpkg -L call needs to be fixed as well, but then I noticed that the pipe consumers will use the user's locale as well. So that change will probably not be needed. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, whic

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Dave Jones
Attaching debdiff for proposed changes to python3-minimal in noble ** Patch added: "noble.debdiff" https://bugs.launchpad.net/ubuntu/+source/python3-defaults/+bug/2075337/+attachment/5802639/+files/noble.debdiff -- You received this bug notification because you are a member of Ubuntu Touch s

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
The dpkg call should set the language to enforce dpkg to output in UTF-8. I see no reason why the user's locale should be used internally. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaults in Ubuntu. https://bug

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
I wrote a small Python script that tries to install a package in a prepared chroot and looks for UnicodeDecodeError in the apt output. I let this script running. Let's see how many affected package can be found. -- You received this bug notification because you are a member of Ubuntu Touch seeded

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Robie Basak
Here's the buggy code that assumes UTF-8: https://git.launchpad.net/ubuntu/+source/python3-defaults/tree/debpython/files.py#n54 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaults in Ubuntu. https://bugs.launchpa

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Matthias Klose
ta, I'll merge that for oracular together with the python 3.12.5 updates -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: py3clean fails when using alternat

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Robie Basak
Setting C.UTF-8 would work internally I agree. But what does this mean for SRUs? Does every affected SRU now have to declare Breaks: python3-minimal (<< whatever)? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaul

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
That would probably the safest solution (as long as the Breaks is not forgotten in the SRU). -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaults in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: py3clea

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Dave, I included your additional change from oracular.debdiff in https://salsa.debian.org/cpython- team/python3-defaults/-/merge_requests/15 My analysis script is nearing its end. I'll post the results in some hours. -- You received this bug notification because you are a member of Ubuntu Touch

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Nice that such a fundamental package has zero tests. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaults in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: py3clean fails when using alternate character s

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Current analysis status: ``` $ sqlite3 affected.sqlite3 "SELECT count(package) FROM package_affected where affected is not NULL;" 1322 $ sqlite3 affected.sqlite3 "SELECT package FROM package_affected where affected = 1;" cinder-api cinder-backup cinder-scheduler cinder-volume cloud-init fglrx-px

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Ubuntu Foundations Team Bug Bot
** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python3-defaults in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: py3clean fails when using alternate character set Status in dh-python package

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Matthias Klose
the fallback dpkg -L call in dh-python's autoscripts/prerm-py3clean should also set the env var. ** Also affects: dh-python (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Good suggestion from the discussion with Ravi: Require all noble SRUs to check if the package is affected by this bug. Add an entry to the SRU template for noble with a test that the submitter has to check. -- You received this bug notification because you are a member of Ubuntu Touch seeded pack

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Robie Basak
I think it's fair to call this Critical since SRU releases generally are currently blocked by this. Looking at the definition, "Severely affects applications beyond the package responsible for the root cause" also applies. ** Changed in: python3-defaults (Ubuntu) Importance: High => Critical -

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-05 Thread Benjamin Drung
Analysis update: There are 72 affected packages so far. ``` $ sqlite3 affected.sqlite3 "SELECT section, COUNT(*) FROM package_affected WHERE install_affected = 1 AND (remove_affected = 1 OR remove_affected ISNULL) GROUP BY section;" main|10 multiverse|2 universe|60 ``` ``` $ sqlite3 affected.sq

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-06 Thread Benjamin Drung
Analysis result === Code and resulting database can be found on https://github.com/bdrung/bdrung-scripts/tree/lp-2075337-find-affected 122 packages failed to analyze: ``` $ sqlite3 affected.sqlite3 "SELECT section, count(*) from package_affected WHERE install_affected IS NULL or (re

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-06 Thread Robie Basak
Thanks all for working on this! Do we know if releases prior to Noble are affected? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: py3clean fails when us

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-06 Thread Benjamin Drung
I tested the affected packages on jammy. Result: Only sgt-launcher is affected on jammy. I'm running a full test on jammy (starting with main). -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python in Ubuntu. https://bugs

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Robie Basak
** Also affects: python3-defaults (Ubuntu Jammy) Importance: Undecided Status: New ** Also affects: dh-python (Ubuntu Jammy) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed t

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Dave Jones
** Patch added: "lp2075337-jammy.debdiff" https://bugs.launchpad.net/ubuntu/+source/python3-defaults/+bug/2075337/+attachment/5803631/+files/lp2075337-jammy.debdiff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Dave Jones
** Patch added: "lp2075337-noble.debdiff" https://bugs.launchpad.net/ubuntu/+source/python3-defaults/+bug/2075337/+attachment/5803632/+files/lp2075337-noble.debdiff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Dave Jones
Attaching debdiffs for sponsorship in noble and jammy. For the purposes of review, the jammy diff does include one additional change to avoid using shell=True on the invocation of dpkg -L. This can be excised if necessary, but I really didn't want to leave something that ultimately runs as root do

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Benjamin Drung
Sponsored python3-defaults 3.12.3-0ubuntu2 for noble (after adding the bug reference to the changelog) and uploaded python3-defaults 3.10.6-1~22.04.1 for jammy. ** Patch added: "python3-defaults_3.10.6-1~22.04.1.debdiff" https://bugs.launchpad.net/ubuntu/+source/python3-defaults/+bug/2075337/+

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Benjamin Drung
** Also affects: cloud-init (Ubuntu) Importance: Undecided Status: New ** Changed in: cloud-init (Ubuntu Jammy) Status: New => Invalid ** Also affects: ubuntu-advantage-tools (Ubuntu) Importance: Undecided Status: New ** Changed in: ubuntu-advantage-tools (Ubuntu Jammy

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Benjamin Drung
** Patch added: "cloud-init_24.2-0ubuntu1~24.04.2.debdiff" https://bugs.launchpad.net/ubuntu/jammy/+source/ubuntu-advantage-tools/+bug/2075337/+attachment/5803640/+files/cloud-init_24.2-0ubuntu1~24.04.2.debdiff -- You received this bug notification because you are a member of Ubuntu Touch see

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Benjamin Drung
Attached the debdiff for ubuntu-advantage-tools. I did not upload it, because the test TestExportGPGKey.test_export_single_key_from_keyring_dir failed during build. ** Patch added: "ubuntu-advantage-tools_33.2~24.04.1.debdiff" https://bugs.launchpad.net/ubuntu/jammy/+source/ubuntu-advantage-to

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-07 Thread Brian Murray
What is the plan for fixing this in Oracular? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python in Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: py3clean fails when using alternate character set Status in

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-08 Thread Benjamin Drung
For oracular: Matthias uploaded 3.12.5-1: https://tracker.debian.org/news/1553054/accepted-python3-defaults-3125-1-source-into-unstable/ -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to dh-python in Ubuntu. https://bugs.launchp

[Touch-packages] [Bug 2075337] Re: py3clean fails when using alternate character set

2024-08-08 Thread Ɓukasz Zemczak
There's a lot of history to follow here. Can someone SRUify the bug description to properly explain the situation? From what I see many packages are affected, but I hope no no-change rebuilds will be required for those as this is python? What is the test case, regression potential? How was this iss