Re: Ubuntu Focal update of broken Calibre package
Hi James! On Tue, Dec 22, 2020 at 06:11:15PM -0700, James Cuzella wrote: > Yes! Qt5 is indeed a very large project too! I'm sure there is a Lord of > the Rings meme that applies here like: "One does not simply backport Qt5" 😄 This is true :) > Simply using the backportpackage tool on qtwebengine-opensource-src started > a build dependency hunt that resulted in finding a massive set of Qt5 > packages that needed to be backported (without relaxing version > constraints). > A lot of these eventually resolved into being blocked on circular > dependency chains. > > I was able to spend a few hours on this to visually represent what I found > (See Gist linked below): > > https://gist.github.com/b005fa0ef6e600c6a6e0dfd22dd3e604 > > I'm not sure how the Debian or Ubuntu teams build and backport Qt5 packages > when so many build dependencies resolve circularly. > I suppose they must bootstrap this dependency chain somehow, but it seems > like more work than I have time to figure out at the moment. Yes, in Ubuntu we bootstrap the packages manually for every new Qt release. If you build packages locally, you can follow the instruction in qtbase's debian/README.source (I have just pushed a minor update for it): https://salsa.debian.org/qt-kde-team/qt/qtbase/-/blob/master/debian/README.source#L16 If you are pushing packages to a PPA, you can use the following procedure: - Push qtbase and qtdeclarative where the -doc and -doc-html packages, and also all Build-Depends-Indep are removed. - Push qttools with the same change, and additionally remove libqt5webkit5-dev build-dependency and make qwebview_archs empty in debian/rules. - Wait until qttools builds on amd64, then push the normal versions of all packages. In Debian this bootstrapping happens automatically because Debian does separate builds for "amd64" (arch-dependent) and "all" (arch-indep) architectures. Also keep in mind that if you update Qt, you will also need to do no-change rebuilds of packages that depend on qtbase-abi-5-x-y virtual package or on qtdeclarative-abi-5-x-y (usually these are packages that use private ABI). Otherwise they can stop working. -- Dmitry Shachnev signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Thanks Dmitry for pointing me in the right direction! I tried to backport the set of Qt5 and qtwebengine packages and found out what large set of build dependencies it was! So what I said is true: this requires simultaneous changes in many packages, > even if it's just a rebuild for some of them. > > Yes! Qt5 is indeed a very large project too! I'm sure there is a Lord of the Rings meme that applies here like: "One does not simply backport Qt5" 😄 Simply using the backportpackage tool on qtwebengine-opensource-src started a build dependency hunt that resulted in finding a massive set of Qt5 packages that needed to be backported (without relaxing version constraints). A lot of these eventually resolved into being blocked on circular dependency chains. I was able to spend a few hours on this to visually represent what I found (See Gist linked below): https://gist.github.com/b005fa0ef6e600c6a6e0dfd22dd3e604 I'm not sure how the Debian or Ubuntu teams build and backport Qt5 packages when so many build dependencies resolve circularly. I suppose they must bootstrap this dependency chain somehow, but it seems like more work than I have time to figure out at the moment. Any ideas for how to simplify this? Thanks, - James Cuzella On Sun, Dec 13, 2020 at 10:16 AM Dmitry Shachnev wrote: > Hi James! > > On Fri, Dec 11, 2020 at 08:43:16PM -0700, James Cuzella wrote: > > I would appreciate any help with figuring out how to get that > > last python3-pyqt5.qtwebengine package to be generated from the pyqt5 > > source package. I'm assuming this is where it comes from, given that all > > the other similarly named ones were generated from that one. Also, the > > Debian package site shows this as the source package: > > https://packages.debian.org/stretch/python3-pyqt5.qtwebengine > > It used to be built from pyqt5 source, but since then it got a new source > package, pyqt5webengine: > > https://launchpad.net/ubuntu/+source/pyqt5webengine > > https://packages.debian.org/sid/source/pyqt5webengine > > -- > Dmitry Shachnev > -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hi James! On Fri, Dec 11, 2020 at 08:43:16PM -0700, James Cuzella wrote: > I would appreciate any help with figuring out how to get that > last python3-pyqt5.qtwebengine package to be generated from the pyqt5 > source package. I'm assuming this is where it comes from, given that all > the other similarly named ones were generated from that one. Also, the > Debian package site shows this as the source package: > https://packages.debian.org/stretch/python3-pyqt5.qtwebengine It used to be built from pyqt5 source, but since then it got a new source package, pyqt5webengine: https://launchpad.net/ubuntu/+source/pyqt5webengine https://packages.debian.org/sid/source/pyqt5webengine -- Dmitry Shachnev signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hello, I recently wanted to test out Calibre with a newer Kindle Fire tablet. It turns out that I ran into many of the issues mentioned with the broken 4.99.4+dfsg+really4.12.0-1build1 package version. I was able to get this old version of Calibre working to identify the tablet on Focal with an updated version of libmtp. This is in my PPA here: https://launchpad.net/~trinitronx/+archive/ubuntu/calibre So, knowing some packaging tricks for Ubuntu & Debian, I decided to try and create an updated Calibre package in a PPA for Focal / 20.04.1. I ran into the issue of first needing to backport all the build dependencies. I started out by using the same version of Calibre ( 5.6.0+dfsg ) from the source package in Ubuntu groovy, and then after reading this thread trying to backport the versions from Debian bullseye & sid. So far I've been able to backport the following: - pyqt-builder 1.4.0+dfsg-1 and 1.6.0+dfsg-1 - sip4 4.19.24+dfsg-1 - sip5 5.3.0+dfsg-2 and 5.5.0+dfsg-1 - pyqt5-sip 12.8.1-1 Note: I also backported debhelper 13.2 in that PPA, but ended up not using it because strict build-depends on "debhelper (=12)" is on most of the other packages in Focal, and Soyuz / pbuilder will fail on that build dependency. Instead, I just switched back to "debhelper (=12)" in the source package's debian/control files. These are in this PPA: https://launchpad.net/~trinitronx/+archive/ubuntu/focal-backport-buildeps > Most packages I mentioned will need a rebuild and a patch to use a different > location for PyQt5 *.sip files. > > This explains exactly the issues I ran into for the main Calibre package! 😊 So far I have tried two routes to resolve the build dependency: - sip4 4.19.24+dfsg-1 + pyqt-builder 1.4.0+dfsg-1 - Patched Calibre's setup.py to use Python3 - Patched Calibre's setup/build.py to use sip-module = "sip" - Ran into issues with python3-sipbuilder - It could not find the package path, as you described: SIPing 3 files... /usr/bin/python3.8 -c import os; os.chdir('/media/terabyte/src/pub/calibre-debian/calibre/build/pyqt/pictureflow'); from sipbuild.tools.build import main; main(); --verbose --no-make --qmake /usr/bin/qmake Querying qmake about your Qt installation... /usr/bin/qmake -query These bindings will be built: pictureflow. Generating the pictureflow bindings... -c: Unable to find file "QtWidgets/QtWidgetsmod.sip" - There was a problem with how python3-sipbuilder's "get_package_dir" function was working. The "sip_parts" result of the split only had 1 array item: ["sip"] The code discards the last array item, assuming it's a filename extension. Therefore, it creates an empty array that gets passed to: os.path.join() - sip5 5.3.0+dfsg-2 + pyqt-builder 1.4.0+dfsg-1 + pyqt5-dev 5.14.1+dfsg-3build1 - This option required just patching Calibre's setup.py to use Python3 - Tried downgrading Build-Depends to pyqt5-dev (>= 5.14.1+dfsg-3build1) - I became blocked at first on backporting the build dependencies for the pyqt5-sip / PyQt5.sip and many other pyqt5 packages. Not sure what the issue was (I've never seen or used PyQt5 or SIP before this... seemed like a large rabbit hole) - sip5 5.5.0+dfsg-1 + pyqt-builder / python3-pyqtbuild 1.6.0+dfsg-1 + pyqt5-sip / python3-pyqt5.sip 12.8.1-1 + pyqt5 5.15.2+dfsg-1 (all backported) - Again, just added the patch Calibre's setup.py to use Python3 - This time it's missing a package: python3-pyqt5.qtwebengine dpkg-checkbuilddeps: error: Unmet build dependencies: python3-pyqt5.qtwebengine (>= 5.12.1-4+b1) You can see this in Debian: after I uploaded new PyQt5 (built with SIP 5) to > unstable, immediately some packages started to fail to build from source: > > - https://bugs.debian.org/971173 (krita) > - https://bugs.debian.org/971217 (python-poppler-qt5) > - https://bugs.debian.org/971172 (veusz) > > And packages that were not rebuilt got runtime errors: > > - https://bugs.debian.org/971538 (qgis) > - https://bugs.debian.org/970921 (calibre) > > So what I said is true: this requires simultaneous changes in many packages, > even if it's just a rebuild for some of them. > > I think if I or someone else here can figure out how to get python3-pyqt5.qtwebengine to build, then Calibre may also be able to build with this set of dependencies. Note, I'm not an "official" Debian or Ubuntu developer and have not gone through the onboarding / mentorship process to become one. So for now, these efforts are for a Focal targeted PPA only. However, with some help this could be the first step towards getting an official set into focal-backports. I would appreciate any help with figuring out how to get that last python3-pyqt5.qtwebengine package to be generated from the pyqt5 source package.
Re: Ubuntu Focal update of broken Calibre package
Hi Julian, thanks for the guidance, but I am not sure anymore what should be done and what Ubuntu devs prefer. There was the suggestion of only backporting the groovy version to focal instead of jumpint to v5. I guess most Ubuntu devs are busy with the release, but if at some point you have a bit of air to comment on your preferred option, please let me know, because I don't want to waste cycles (of both sides) with preparing a version that will not make it. Best Norbert On Tue, 20 Oct 2020, Julian Andres Klode wrote: > On Fri, Oct 16, 2020 at 01:18:49AM +0900, Norbert Preining wrote: > > Hi Lukasz, > > > > so, here are a few more answers to your questions: > > > > On Wed, 14 Oct 2020, Lukasz Zemczak wrote: > > > * How badly is calibre broken on focal right now? Is it really > > > unusable in its current state? Examples of how broken things are so > > > that we can understand the situation better > > > > Yes. I just installed a vm, updated to the latest version of the > > packages of focal, installed calibre, started calibre, and got > > > > norbert@ubuntu2004-vm:~$ calibre > > Traceback (most recent call last): > > File "/usr/bin/calibre", line 20, in > > sys.exit(calibre()) > > File "/usr/lib/calibre/calibre/gui_launch.py", line 73, in calibre > > main(args) > > File "/usr/lib/calibre/calibre/gui2/main.py", line 543, in main > > listener = create_listener() > > File "/usr/lib/calibre/calibre/gui2/main.py", line 514, in create_listener > > return Listener(address=gui_socket_address()) > > File "/usr/lib/calibre/calibre/utils/ipc/server.py", line 110, in __init__ > > self._listener._unlink.cancel() > > AttributeError: 'NoneType' object has no attribute 'cancel' > > norbert@ubuntu2004-vm:~$ vim > > > > So well, it is completely useless. > > > > This can also be seen by the list of upstream bugs reported > > https://bugs.launchpad.net/bugs/1899700 > > https://bugs.launchpad.net/bugs/1899674 > > https://bugs.launchpad.net/bugs/1899355 > > https://bugs.launchpad.net/bugs/1899035 > > https://bugs.launchpad.net/bugs/1899029 > > https://bugs.launchpad.net/bugs/1898940 > > https://bugs.launchpad.net/bugs/1898904 > > > > > > > * How does the automated test coverage on calibre look like? Do all > > > new features come with unit testing? What about autopkgtests (I don't > > > think I see any?)? > > > > There are not autopkgtests, but there is an extensive test suite built > > into calibre. > > > > > * What would be the acceptance criteria for the new version? What > > > testing should be performed to make sure the new version works as > > > expected and doesn't regress any existing users (assuming calibre in > > > focal right now is at least usable to some extent) > > > > Since it does not even start, I guess there is no regression for > > focal users, only for those upgrading from a previous release. > > > > > > Together with YOKOTA Hiroshi (in Cc), who has done most of the work on > > recent packaging, I have prepared a version for focal (SIP4, debhelper > > 12), built it on my focal machine, and successfully run it. Source and > > amd64 packages are available here: > > > > deb http://www.preining.info/debian focal main > > deb-src http://www.preining.info/debian focal main > > > > (signed with my gpg key https://www.preining.info/rsa.asc) > > > > What are the next steps you are expecting from me? > > > > - prepare a package for groovy and separately for focal? > > Yes. > > > - what are the version numbers you want to see? > > > If it's based on e.g. 5.3.0+dfsg-1, you want > > 5.3.0+dfsg-1ubuntu0.20.04.1 for focal > 5.3.0+dfsg-1ubuntu0.20.10.1 for groovy > > or > > 5.3.0+dfsg-1~ubuntu<20.04.1/20.10.1> would work > too, but I think the first version is better. > > > - how should we proceed? > > Open - or repurpose an existing bug - with the SRU template: > > https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template > > e.g. > > [Impact] > calibre does not start, is an unstable pre-release with a lots of bugs > [Test case] > calibre should start and be functional > [Regression potential] > It did not start before, so it can hardly regress > [Other info] > RT members agreed in ... that we can update to the > latest release. > > Attached are debdiffs against the version in Debian. > > with a bit nicer wording and more details :D > > And close it in the changelog with LP: #thatbugnumber. > > Then attach a debdiff against the debian version to the bug > report for each focal and groovy, and subscribe ~ubuntu-sponsors. > > -- > debian developer - deb.li/jak | jak-linux.org - free software dev > ubuntu core developer i speak de, en > -- PREINING Norbert https://www.preining.info Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13 -- ubuntu-devel mailing list ubuntu-
Re: Ubuntu Focal update of broken Calibre package
On Fri, Oct 16, 2020 at 01:18:49AM +0900, Norbert Preining wrote: > Hi Lukasz, > > so, here are a few more answers to your questions: > > On Wed, 14 Oct 2020, Lukasz Zemczak wrote: > > * How badly is calibre broken on focal right now? Is it really > > unusable in its current state? Examples of how broken things are so > > that we can understand the situation better > > Yes. I just installed a vm, updated to the latest version of the > packages of focal, installed calibre, started calibre, and got > > norbert@ubuntu2004-vm:~$ calibre > Traceback (most recent call last): > File "/usr/bin/calibre", line 20, in > sys.exit(calibre()) > File "/usr/lib/calibre/calibre/gui_launch.py", line 73, in calibre > main(args) > File "/usr/lib/calibre/calibre/gui2/main.py", line 543, in main > listener = create_listener() > File "/usr/lib/calibre/calibre/gui2/main.py", line 514, in create_listener > return Listener(address=gui_socket_address()) > File "/usr/lib/calibre/calibre/utils/ipc/server.py", line 110, in __init__ > self._listener._unlink.cancel() > AttributeError: 'NoneType' object has no attribute 'cancel' > norbert@ubuntu2004-vm:~$ vim > > So well, it is completely useless. > > This can also be seen by the list of upstream bugs reported > https://bugs.launchpad.net/bugs/1899700 > https://bugs.launchpad.net/bugs/1899674 > https://bugs.launchpad.net/bugs/1899355 > https://bugs.launchpad.net/bugs/1899035 > https://bugs.launchpad.net/bugs/1899029 > https://bugs.launchpad.net/bugs/1898940 > https://bugs.launchpad.net/bugs/1898904 > > > > * How does the automated test coverage on calibre look like? Do all > > new features come with unit testing? What about autopkgtests (I don't > > think I see any?)? > > There are not autopkgtests, but there is an extensive test suite built > into calibre. > > > * What would be the acceptance criteria for the new version? What > > testing should be performed to make sure the new version works as > > expected and doesn't regress any existing users (assuming calibre in > > focal right now is at least usable to some extent) > > Since it does not even start, I guess there is no regression for > focal users, only for those upgrading from a previous release. > > > Together with YOKOTA Hiroshi (in Cc), who has done most of the work on > recent packaging, I have prepared a version for focal (SIP4, debhelper > 12), built it on my focal machine, and successfully run it. Source and > amd64 packages are available here: > > deb http://www.preining.info/debian focal main > deb-src http://www.preining.info/debian focal main > > (signed with my gpg key https://www.preining.info/rsa.asc) > > What are the next steps you are expecting from me? > > - prepare a package for groovy and separately for focal? Yes. > - what are the version numbers you want to see? If it's based on e.g. 5.3.0+dfsg-1, you want 5.3.0+dfsg-1ubuntu0.20.04.1 for focal 5.3.0+dfsg-1ubuntu0.20.10.1 for groovy or 5.3.0+dfsg-1~ubuntu<20.04.1/20.10.1> would work too, but I think the first version is better. > - how should we proceed? Open - or repurpose an existing bug - with the SRU template: https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template e.g. [Impact] calibre does not start, is an unstable pre-release with a lots of bugs [Test case] calibre should start and be functional [Regression potential] It did not start before, so it can hardly regress [Other info] RT members agreed in ... that we can update to the latest release. Attached are debdiffs against the version in Debian. with a bit nicer wording and more details :D And close it in the changelog with LP: #thatbugnumber. Then attach a debdiff against the debian version to the bug report for each focal and groovy, and subscribe ~ubuntu-sponsors. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hi Lukasz, so, here are a few more answers to your questions: On Wed, 14 Oct 2020, Lukasz Zemczak wrote: > * How badly is calibre broken on focal right now? Is it really > unusable in its current state? Examples of how broken things are so > that we can understand the situation better Yes. I just installed a vm, updated to the latest version of the packages of focal, installed calibre, started calibre, and got norbert@ubuntu2004-vm:~$ calibre Traceback (most recent call last): File "/usr/bin/calibre", line 20, in sys.exit(calibre()) File "/usr/lib/calibre/calibre/gui_launch.py", line 73, in calibre main(args) File "/usr/lib/calibre/calibre/gui2/main.py", line 543, in main listener = create_listener() File "/usr/lib/calibre/calibre/gui2/main.py", line 514, in create_listener return Listener(address=gui_socket_address()) File "/usr/lib/calibre/calibre/utils/ipc/server.py", line 110, in __init__ self._listener._unlink.cancel() AttributeError: 'NoneType' object has no attribute 'cancel' norbert@ubuntu2004-vm:~$ vim So well, it is completely useless. This can also be seen by the list of upstream bugs reported https://bugs.launchpad.net/bugs/1899700 https://bugs.launchpad.net/bugs/1899674 https://bugs.launchpad.net/bugs/1899355 https://bugs.launchpad.net/bugs/1899035 https://bugs.launchpad.net/bugs/1899029 https://bugs.launchpad.net/bugs/1898940 https://bugs.launchpad.net/bugs/1898904 > * How does the automated test coverage on calibre look like? Do all > new features come with unit testing? What about autopkgtests (I don't > think I see any?)? There are not autopkgtests, but there is an extensive test suite built into calibre. > * What would be the acceptance criteria for the new version? What > testing should be performed to make sure the new version works as > expected and doesn't regress any existing users (assuming calibre in > focal right now is at least usable to some extent) Since it does not even start, I guess there is no regression for focal users, only for those upgrading from a previous release. Together with YOKOTA Hiroshi (in Cc), who has done most of the work on recent packaging, I have prepared a version for focal (SIP4, debhelper 12), built it on my focal machine, and successfully run it. Source and amd64 packages are available here: deb http://www.preining.info/debian focal main deb-src http://www.preining.info/debian focal main (signed with my gpg key https://www.preining.info/rsa.asc) What are the next steps you are expecting from me? - prepare a package for groovy and separately for focal? - what are the version numbers you want to see? - how should we proceed? Best Norbert -- PREINING Norbert https://www.preining.info Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
On Thu, 15 Oct 2020 08:35:36 +0900, Norbert Preining wrote: > Hi Lukasz, Dmitry, Eli, all > > thanks for your messages and interest and support. Sorry for the late > replies, time zones (I am Japan based). > > On Wed, 14 Oct 2020, Lukasz Zemczak wrote: > > With my SRU team hat on, after reading what you said about the status > > of current calibre, I would say it is possible to update the package > > version in focal to 5.2.0. But we would need to know a bit more, and > > there is some (important) paperwork to do. > > Sounds like a good idea. > > On Wed, 14 Oct 2020, Dmitry Shachnev wrote: > > Tomorrow is final freeze, so it is really bad timing for this. > > Indeed. > > > > - Revert code in calibre to make it build with Sip 4 again, and package > > > calibre 5.2.0: > > > > > > https://github.com/kovidgoyal/calibre/commit/7a4b3f61ff24f8c39c8d5cf86c54da9de9267025 > > > > > > I suspect that last option would be the easiest resolution. It should > > > work. > > > > That would be the easiest option, yes. > > > > I don't volunteer to work on this (no time, sorry), but I can sponsor an > > upload if someone gets a feature freeze exception for this and > > prepares/tests > > the upload. > > I think I can prepare a package based on the Debian calibre 5.2.0 > version in sid/testing, that reverts the Sip5 changes. > > I will install a focal vm and see if building works out of the box. > > If that is done, I will contact you again for further steps. > > Is this ok a plan for you or do you want/need anything else in this > area right now? > > All the best > > Norbert > Hi all, I had sent an email to ubuntu-devel-discuss (instead of here) and not sure if any of you read. Reposting here just to be sure. I checked Calibre in Groovy (version 4.99.12+dfsg+really4.23.0-1) and it works fine. As per bug report #1898904 (https://bugs.launchpad.net/ubuntu/+source/calibre/+bug/1898904/comments/7), Calibre >=4.15 has the required patch. So, instead of waiting for 5.2, can we backport current Groovy version of Calibre to Focal? Regards, Sai -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
On 10/14/20 7:29 AM, Dmitry Shachnev wrote: > Hi Norbert, Lukasz and all! > > Unfortunately, Calibre 5.x requires SIP 5 and PyQt5 that is built > against SIP 5. Moving PyQt5 to the new SIP is a major transition, > that happened in Debian recently (two weeks ago) and did not happen > in Ubuntu yet because of freeze. > > This requires changes in many packages simultaneously: at least > pyqt5, pyqt5charts, pyqt5webengine, qscintilla2, calibre, > python-poppler-qt5, veusz, krita and qgis. > > I am planning to land this change early in Groovy+1 cycle. Just for the record, this is untrue. Arch Linux has built python3 PyQt5 using Sip 5 via /usr/bin/sip-build, since Dec 13 18:01:34 2019 while continuing to build python2 PyQt5 using Sip 4 via python2 configure.py This worked well enough that even though on Nov 24 20:33:38 2019 I began shipping multiple repository packages for calibre -- one "calibre" package built with python2 and one "calibre-python3" package built with python3 -- both using Sip 4, they worked fine. The calibre-python3 package was, as expected, buggy due to being beta quality, but it never failed due to pyqt5/sip itself. There are still various packages in our distro archives building with Sip 4 but successfully using the pyqt5 bindings built using Sip 5. Old versions of some of those packages (at least krita, qgis) did need patches to change the location of the sip dir. qscintilla2 did need to be rebuilt with no changes, then a week later the 2.11.4 update moved to sip-build. It's plainly possible to mix them at least a little. From memory, we did not even need to rebuild (most of) the packages. YMMV, but it should definitely be feasible to update pyqt5/sip5, test everything that build-depends on them, and leave many of them alone if they're not ready to move. >> Is the above (update to 5.2.0) possible in Ubuntu Focal, and if >> yes, what kind if steps are necessary? > > So for Focal and Groovy we need a version of Calibre that still uses > SIP 4. Last such version in Debian was 4.99.12+dfsg+really4.23.0-1, > Groovy already has that. If you know some specific fixes, maybe they > can be applied on top of what Focal or Groovy has. Even if you could solve the immediate, critical crashes, calibre + python3 is a combination that is only beta quality until 5.0.0, as many issues were found during the final stages of the beta. I strongly advise Ubuntu Focal to NOT ship beta-quality code that upstream never released and refuses to sign off on, for the next 5 years as an LTS. It is a recipe for heartbreak and bad relationships with upstream. It was never intended to be present in a stable, frozen distro release anywhere. In recent times, the Debian package for calibre has gained a lot of polish, and gotten rid of legacy baggage from the "bad old days" where upstream simply told all users that Debian was the main reason users should never use distro packages. I'm kind of hoping that Ubuntu doesn't become a new reason for upstream to begrudge distros. ... Possible options going forward: very much not ideal: - reinstate a stable legacy python2 build for Ubuntu (needs various python2 deps re-added) - dropping calibre from the archives - doing nothing: keep a broken package that crashes on startup and drives upstream nuts - cherry-picking some specific fix: keep a partially broken beta package that crashes at odd, unpredictable times, and drives upstream occasionally nuts ideal: - getting a Sip 5 collection in place which still supports some packages stuck on Sip 4, and package calibre 5.2.0 not great, but workable: - Revert code in calibre to make it build with Sip 4 again, and package calibre 5.2.0: https://github.com/kovidgoyal/calibre/commit/7a4b3f61ff24f8c39c8d5cf86c54da9de9267025 ... I suspect that last option would be the easiest resolution. It should work. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hi Lukasz, Dmitry, Eli, all thanks for your messages and interest and support. Sorry for the late replies, time zones (I am Japan based). On Wed, 14 Oct 2020, Lukasz Zemczak wrote: > With my SRU team hat on, after reading what you said about the status > of current calibre, I would say it is possible to update the package > version in focal to 5.2.0. But we would need to know a bit more, and > there is some (important) paperwork to do. Sounds like a good idea. On Wed, 14 Oct 2020, Dmitry Shachnev wrote: > Tomorrow is final freeze, so it is really bad timing for this. Indeed. > > - Revert code in calibre to make it build with Sip 4 again, and package > > calibre 5.2.0: > > > > https://github.com/kovidgoyal/calibre/commit/7a4b3f61ff24f8c39c8d5cf86c54da9de9267025 > > > > I suspect that last option would be the easiest resolution. It should work. > > That would be the easiest option, yes. > > I don't volunteer to work on this (no time, sorry), but I can sponsor an > upload if someone gets a feature freeze exception for this and prepares/tests > the upload. I think I can prepare a package based on the Debian calibre 5.2.0 version in sid/testing, that reverts the Sip5 changes. I will install a focal vm and see if building works out of the box. If that is done, I will contact you again for further steps. Is this ok a plan for you or do you want/need anything else in this area right now? All the best Norbert -- PREINING Norbert https://www.preining.info Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hi Eli! On Wed, Oct 14, 2020 at 11:52:44AM -0400, Eli Schwartz wrote: > > This requires changes in many packages simultaneously: at least > > pyqt5, pyqt5charts, pyqt5webengine, qscintilla2, calibre, > > python-poppler-qt5, veusz, krita and qgis. > > > > I am planning to land this change early in Groovy+1 cycle. > > Just for the record, this is untrue. > > Arch Linux has built python3 PyQt5 using Sip 5 via /usr/bin/sip-build, > since Dec 13 18:01:34 2019 while continuing to build python2 PyQt5 using > Sip 4 via python2 configure.py > > This worked well enough that even though on Nov 24 20:33:38 2019 I began > shipping multiple repository packages for calibre -- one "calibre" > package built with python2 and one "calibre-python3" package built with > python3 -- both using Sip 4, they worked fine. The calibre-python3 > package was, as expected, buggy due to being beta quality, but it never > failed due to pyqt5/sip itself. > > There are still various packages in our distro archives building with > Sip 4 but successfully using the pyqt5 bindings built using Sip 5. > > Old versions of some of those packages (at least krita, qgis) did need > patches to change the location of the sip dir. > qscintilla2 did need to be rebuilt with no changes, then a week later > the 2.11.4 update moved to sip-build. > > It's plainly possible to mix them at least a little. From memory, we did > not even need to rebuild (most of) the packages. Most packages I mentioned will need a rebuild and a patch to use a different location for PyQt5 *.sip files. You can see this in Debian: after I uploaded new PyQt5 (built with SIP 5) to unstable, immediately some packages started to fail to build from source: - https://bugs.debian.org/971173 (krita) - https://bugs.debian.org/971217 (python-poppler-qt5) - https://bugs.debian.org/971172 (veusz) And packages that were not rebuilt got runtime errors: - https://bugs.debian.org/971538 (qgis) - https://bugs.debian.org/970921 (calibre) So what I said is true: this requires simultaneous changes in many packages, even if it's just a rebuild for some of them. > YMMV, but it should definitely be feasible to update pyqt5/sip5, test > everything that build-depends on them, and leave many of them alone if > they're not ready to move. Tomorrow is final freeze, so it is really bad timing for this. > not great, but workable: > > - Revert code in calibre to make it build with Sip 4 again, and package > calibre 5.2.0: > > https://github.com/kovidgoyal/calibre/commit/7a4b3f61ff24f8c39c8d5cf86c54da9de9267025 > > I suspect that last option would be the easiest resolution. It should work. That would be the easiest option, yes. I don't volunteer to work on this (no time, sorry), but I can sponsor an upload if someone gets a feature freeze exception for this and prepares/tests the upload. -- Dmitry Shachnev -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hi Norbert, Lukasz and all! On Wed, Oct 14, 2020 at 09:06:35AM +0900, Norbert Preining wrote: > Dear all, > > (please Cc) > > I am the Debian maintainer of Calibre, and unfortunately it seems that > for Focal Ubuntu has pulled a preliminary version of Calibre, which is > **seriously** broken and unusable, not even starting in most cases. > > We were forced by the Python3 transition to temporarily ship pre-release > versions of Calibre. In particular, Ubuntu Focal ships > 4.99.4+dfsg+really4.12.0-1build1 > which is version 4.12 with experimental Python3 patches on top of it. > This worked for a short time being until Calibre 5 was released with > proper Python3 support. > > Due to this unfortunate squeeze in release timing, Ubuntu Focal users > now have a seriously broken Calibre, and upstream is swamped with bug > reports. > > I would strongly suggest and support, and help preparing, an update to > Focal based on the current version in Debian/testing, 5.2.0+dfsg-1, > which has been out since quite some time and field-tested with Python3 > in various environments, due to upstream having switched to Py3, too. Unfortunately, Calibre 5.x requires SIP 5 and PyQt5 that is built against SIP 5. Moving PyQt5 to the new SIP is a major transition, that happened in Debian recently (two weeks ago) and did not happen in Ubuntu yet because of freeze. This requires changes in many packages simultaneously: at least pyqt5, pyqt5charts, pyqt5webengine, qscintilla2, calibre, python-poppler-qt5, veusz, krita and qgis. I am planning to land this change early in Groovy+1 cycle. > Is the above (update to 5.2.0) possible in Ubuntu Focal, and if yes, > what kind if steps are necessary? So for Focal and Groovy we need a version of Calibre that still uses SIP 4. Last such version in Debian was 4.99.12+dfsg+really4.23.0-1, Groovy already has that. If you know some specific fixes, maybe they can be applied on top of what Focal or Groovy has. -- Dmitry Shachnev signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Ubuntu Focal update of broken Calibre package
Hello Norbert, With my SRU team hat on, after reading what you said about the status of current calibre, I would say it is possible to update the package version in focal to 5.2.0. But we would need to know a bit more, and there is some (important) paperwork to do. First thing to remember is that for a new version to be releasable to a stable series it also has to be present in all the newer series as well - so groovy upwards. We are long past debian import freeze so groovy is still on 4.99.12+dfsg+really4.23.0-1 - and syncing 5.2.0 now, so late in the cycle, requires an approved Feature Freeze Exception [1], since we're also past feature freeze. It's a very very unfortunate time, since Final Freeze for groovy is tomorrow, and we're a bit reluctant with accepting risky pieces. The 'good' news is, calibre is only seeded in our Ubuntu Studio flavor, so the impact should be manageable. I would like the Ubuntu Studio flavor representatives to chip in and (maybe) help out with the paperwork there. Once we have it in groovy+ we can look into backporting it to focal. This requires filling in some SRU paperwork as per the policy [2]. What makes it a bit problematic from the SRU perspective is that the debdiff between 4.99.4+dfsg+really4.12.0-1build1 (in focal) and 5.2.0 is 7600433 lines long. Looking at the diff itself, most of it are translation changes, but still... it's quite a change nevertheless. The things that need to be thought of before performing the backport: * How badly is calibre broken on focal right now? Is it really unusable in its current state? Examples of how broken things are so that we can understand the situation better * How does the automated test coverage on calibre look like? Do all new features come with unit testing? What about autopkgtests (I don't think I see any?)? * What would be the acceptance criteria for the new version? What testing should be performed to make sure the new version works as expected and doesn't regress any existing users (assuming calibre in focal right now is at least usable to some extent) If needed, we can help a bit with some of the SRU bits. Cheers, [1] https://wiki.ubuntu.com/FreezeExceptionProcess#FeatureFreeze_for_new_upstream_versions [2] https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template On Wed, 14 Oct 2020 at 10:20, Norbert Preining wrote: > > Dear all, > > (please Cc) > > I am the Debian maintainer of Calibre, and unfortunately it seems that > for Focal Ubuntu has pulled a preliminary version of Calibre, which is > **seriously** broken and unusable, not even starting in most cases. > > We were forced by the Python3 transition to temporarily ship pre-release > versions of Calibre. In particular, Ubuntu Focal ships > 4.99.4+dfsg+really4.12.0-1build1 > which is version 4.12 with experimental Python3 patches on top of it. > This worked for a short time being until Calibre 5 was released with > proper Python3 support. > > Due to this unfortunate squeeze in release timing, Ubuntu Focal users > now have a seriously broken Calibre, and upstream is swamped with bug > reports. > > I would strongly suggest and support, and help preparing, an update to > Focal based on the current version in Debian/testing, 5.2.0+dfsg-1, > which has been out since quite some time and field-tested with Python3 > in various environments, due to upstream having switched to Py3, too. > > Is the above (update to 5.2.0) possible in Ubuntu Focal, and if yes, > what kind if steps are necessary? > > Note that I am not an Ubuntu developers, but Debian developer and > maintainer of Calibre. > > Thanks and all the best > > Norbert > > -- > PREINING Norbert https://www.preining.info > Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev > GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13 > > -- > ubuntu-devel mailing list > ubuntu-devel@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Łukasz 'sil2100' Zemczak Foundations Team lukasz.zemc...@canonical.com www.canonical.com -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Ubuntu Focal update of broken Calibre package
Dear all, (please Cc) I am the Debian maintainer of Calibre, and unfortunately it seems that for Focal Ubuntu has pulled a preliminary version of Calibre, which is **seriously** broken and unusable, not even starting in most cases. We were forced by the Python3 transition to temporarily ship pre-release versions of Calibre. In particular, Ubuntu Focal ships 4.99.4+dfsg+really4.12.0-1build1 which is version 4.12 with experimental Python3 patches on top of it. This worked for a short time being until Calibre 5 was released with proper Python3 support. Due to this unfortunate squeeze in release timing, Ubuntu Focal users now have a seriously broken Calibre, and upstream is swamped with bug reports. I would strongly suggest and support, and help preparing, an update to Focal based on the current version in Debian/testing, 5.2.0+dfsg-1, which has been out since quite some time and field-tested with Python3 in various environments, due to upstream having switched to Py3, too. Is the above (update to 5.2.0) possible in Ubuntu Focal, and if yes, what kind if steps are necessary? Note that I am not an Ubuntu developers, but Debian developer and maintainer of Calibre. Thanks and all the best Norbert -- PREINING Norbert https://www.preining.info Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel