Bug#894159: transition: icu
On 13/05/18 22:52, László Böszörményi (GCS) wrote: > Last but not least I've a different understanding (again) of package > transitions with Adrian Bunk. He reassigned #898465 [2] to ICU, when both > related bugreports of ncmpcpp [3] and viking due to mapnik [4] are due to > partial upgrades (shown in the package versions 0.8.1-1 and 3.0.20+ds-1 > respectively that these are _not_ the binNMUed ones). Checking the build > logs show mapnik and ncmpcpp rebuilt normally and I've even tested those > correctly. The bug reporters need to do a full packages upgrade, that's all. > Then the migration script knows about dependencies and will not let icu > enter testing by itself, only with the transitioned dependencies. icu should migrate to testing even before all the rdeps migrate. That could cause mapnik or ncmpcpp, for example, to stay at older versions, with boost1.62 and icu at the new ones, breaking those packages. Besides, we support partial upgrades. The solution I see here is to add Breaks in boost1.62 against mapnik and ncmpcpp (and other affected packages, if any). That should solve this bug for now, and while not ideal, we should have a boost transition eventually so I don't mind using Breaks in this situation. Cheers, Emilio
Bug#894159: transition: icu
On Thu, May 3, 2018 at 8:36 AM Emilio Pozuelo Monfort wrote: > Control: tags -1 confirmed > On 01/05/18 19:29, László Böszörményi (GCS) wrote: > > OK, the fresh transition testing is the following. > Alright, let's do this. I was off the grid on the weekend, but a quick summary how it goes. ICU -> icu-le-hb -> ICU with Paragraph Layout compiled successfully on all of our primary architectures. But harfbuzz had to be binNMUed as well. On sparc64 harfbuzz FTBFS for a while (since last October) and this architecture won't make the transition. :( Then openttd was binNMUed too quick, please reschedule that strictly with icu >= 60.2-6 package version. I don't see if texlive-bin was binNMUed, you might have missed that. Please start that. Unfortunately chromium browser FTBFS on armhf since this January and will not be able to migrate to testing due to this. haskell-pandoc-citeproc is an other package which will be unable to migrate as one of its build dependency, panadoc FTBFS on armel and armhf. frog FTBFS as it needs a transitioned ucto which is done now. Please reschedule its binNMU. Note that it will fail on hurd-i386 as while ucto is built on this architecture, it's not uploaded even to the archives. qt4-x11 doesn't show up in the automatic ICU transition list and it FTBFS with it. The bug reporter also supplied a working fix[1]. Maintainer may upload the fixed package version tomorrow. Last but not least I've a different understanding (again) of package transitions with Adrian Bunk. He reassigned #898465 [2] to ICU, when both related bugreports of ncmpcpp [3] and viking due to mapnik [4] are due to partial upgrades (shown in the package versions 0.8.1-1 and 3.0.20+ds-1 respectively that these are _not_ the binNMUed ones). Checking the build logs show mapnik and ncmpcpp rebuilt normally and I've even tested those correctly. The bug reporters need to do a full packages upgrade, that's all. Then the migration script knows about dependencies and will not let icu enter testing by itself, only with the transitioned dependencies. Please tell me if I'm might be wrong, otherwise I plan to close these bug reports. Cheers, Laszlo/GCS [1] https://bugs.debian.org/898542#27 [2] https://bugs.debian.org/898465#22 [3] https://bugs.debian.org/898369#5 [4] https://bugs.debian.org/898465#5
Bug#894159: transition: icu
Control: tags -1 confirmed On 01/05/18 19:29, László Böszörményi (GCS) wrote: > On Mon, Apr 23, 2018 at 11:31 PM Emilio Pozuelo Monfort > wrote: >> On 23/04/18 21:40, László Böszörményi (GCS) wrote: >>> I do agree and open to do an other compilation test of the dependent >>> packages with icu-config being available. > >> Thanks. Let me know how that goes, and I'll ack this transition as soon > as possible. > OK, the fresh transition testing is the following. Alright, let's do this. Cheers, Emilio
Bug#894159: transition: icu
On Wed, May 02, 2018 at 08:06:57PM +0200, Rene Engelhard wrote: > Yes, that's why various LibreOffice/Document Liberation libraries > (and LO also patched firebird) - parts of this reverted since > c++11-using projects apparently don't need it) did add a > -DUCHAR_TYPE=uint16_t to their defines. Explicitely backported in > various debian/rules of those libs Sorry, sent to early. From LOs configure.ac: if test "$ICU_MAJOR" -ge "59"; then # As of ICU 59 it defaults to typedef char16_t UChar; which is available # with -std=c++11 but not all external libraries can be built with that, # for those use a bit-compatible typedef uint16_t UChar; see # icu/source/common/unicode/umachine.h ICU_UCHAR_TYPE="-DUCHAR_TYPE=uint16_t" else ICU_UCHAR_TYPE="" fi Regards, Rene
Bug#894159: transition: icu
Hi, On Tue, May 01, 2018 at 05:29:33PM +, László Böszörményi wrote: > 5.10.1+dfsg-2) were uploaded to experimental. Now it FTBFS due to GCC > being picky. Recent ICU (59.1+) named variable types to be consistent for > different architectures. That is, UChar defaults to char16_t but can be > uint16_t if the architecture / compiler requires that. > Probably newer GCC versions no longer do the automatic type casts. I had to > add them explicitly. No code change was needed. With my patch, this package > builds correctly again. Yes, that's why various LibreOffice/Document Liberation libraries (and LO also patched firebird) - parts of this reverted since c++11-using projects apparently don't need it) did add a -DUCHAR_TYPE=uint16_t to their defines. Explicitely backported in various debian/rules of those libs Regards, Rene
Bug#894159: transition: icu
Hi, > pyicu FTBFS, seems unmaintained. Last upload was in 2016 and since then > there were seven upstream releases. Version 1.9.8+ builds fine with the new > ICU version. Maintainer also Cc-d, may I overtake it? unmaintained is the wrong word, I try to have it uptodate before the next release - and I'm almost sure that I've uploaded the current version before the last freeze. I'd be happy if you could maintain it - I'm missing the time for it. Thanks, Bernd -- Bernd ZeimetzDebian GNU/Linux Developer http://bzed.dehttp://www.debian.org GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
Bug#894159: transition: icu
On Mon, Apr 23, 2018 at 11:31 PM Emilio Pozuelo Monfort wrote: > On 23/04/18 21:40, László Böszörményi (GCS) wrote: > > I do agree and open to do an other compilation test of the dependent > > packages with icu-config being available. > Thanks. Let me know how that goes, and I'll ack this transition as soon as possible. OK, the fresh transition testing is the following. Level 1: I don't know how haskell-yi-rope directly related, but it builds with the rest of the packages. Level 1.5: I choose this intermediate step to build both Boost1.62 and Boost1.63 as well correctly. Level 2: dee FTBFS for an unrealated issue, #876594 [1]. libsimpleini fails only due to it needs library symbols update. Maintainer is Cc-d. pyicu FTBFS, seems unmaintained. Last upload was in 2016 and since then there were seven upstream releases. Version 1.9.8+ builds fine with the new ICU version. Maintainer also Cc-d, may I overtake it? Level 3: node-stringprep FTBFS for an unrelated issue, #895029 [2]. webkitgtk FTBFS, requested to be removed[3] due to unmaintained and many security bugs. That's why I ignored it. Probably variable casting problems is the FTBFS reason, see later. Level 4: cyrus-imapd FTBFS for an unrelated issue, #883951 [4]. haskell-blogliterately FTBFS for an unrelated issue, #897172 [5]. Level 5: kbibtex FTBFS for an unrelated issue, #893538 [6]. qtwebengine-opensource-src is strange. Previously it built for me, even built for the maintainers when previous package versions (5.10.1+dfsg-1 and 5.10.1+dfsg-2) were uploaded to experimental. Now it FTBFS due to GCC being picky. Recent ICU (59.1+) named variable types to be consistent for different architectures. That is, UChar defaults to char16_t but can be uint16_t if the architecture / compiler requires that. Probably newer GCC versions no longer do the automatic type casts. I had to add them explicitly. No code change was needed. With my patch, this package builds correctly again. Cheers, Laszlo/GCS [1] https://bugs.debian.org/876594 [2] https://bugs.debian.org/895029 [3] https://bugs.debian.org/893863 [4] https://bugs.debian.org/883951 [5] https://bugs.debian.org/897172 [6] https://bugs.debian.org/893538
Bug#894159: transition: icu
On Tue, Apr 24, 2018 at 6:46 PM, Rene Engelhard wrote: > On Mon, Apr 23, 2018 at 09:19:06PM +0200, László Böszörményi wrote: >> > I would appreciate a number of failing rdeps and how many are due to ICU >> > API >> > changes and how many are due to icu-config removal. >> [...] I do not >> recall any API change. In short, there are fifteen packages FTBFS and > > Interesting. I do. Let me rephrase my sentence. I do not remember any FTBFS reason which happens due to an ICU API change. I could build all dependent packages without any code change in those. Sorry for the bad wording. I'm in a run, but double checked my patches and all about ICU detection without icu-config installed - expect one. That is OpenTTD which need an additional build dependency on libicu-le-hb-dev. To be extra sure about the possible changes needed in the packages, I'll start the rebuild tests soon. Kind regards, Laszlo/GCS
Bug#894159: transition: icu
On Mon, Apr 23, 2018 at 09:19:06PM +0200, László Böszörményi wrote: > > I would appreciate a number of failing rdeps and how many are due to ICU API > > changes and how many are due to icu-config removal. > [...] I do not > recall any API change. In short, there are fifteen packages FTBFS and Interesting. I do. https://cgit.freedesktop.org/libreoffice/core/diff/i18npool/source/breakiterator/breakiterator_unicode.cxx?id=3e42714c76b1347babfdea0564009d8d82a83af4 Regards, Rene
Bug#894159: transition: icu
On 23/04/18 21:40, László Böszörményi (GCS) wrote: > On Mon, Apr 23, 2018 at 9:27 PM, Emilio Pozuelo Monfort > wrote: >> On 23/04/18 21:19, László Böszörményi (GCS) wrote: >>> As I remember, 99% of the FTBFS reasons were the icu-config removal, >>> others are the dependent package bugs I've already mentioned. I do not >>> recall any API change. In short, there are fifteen packages FTBFS and >>> all is due to the icu-config removal. This is true for the date of my >>> previous mail. There were several dependent packages upload since >>> then, but I think the situation remained the same. >> >> In that case I'm ok with removing icu-config, but please don't entangle that >> with the SONAME bump. That is, reintroduce icu-config so we can have an easy >> transition, and once the transition is finished, then you are free to drop >> icu-config. Sounds good? > I do agree and open to do an other compilation test of the dependent > packages with icu-config being available. Thanks. Let me know how that goes, and I'll ack this transition as soon as possible. Cheers, Emilio
Bug#894159: transition: icu
On Mon, Apr 23, 2018 at 9:27 PM, Emilio Pozuelo Monfort wrote: > On 23/04/18 21:19, László Böszörményi (GCS) wrote: >> As I remember, 99% of the FTBFS reasons were the icu-config removal, >> others are the dependent package bugs I've already mentioned. I do not >> recall any API change. In short, there are fifteen packages FTBFS and >> all is due to the icu-config removal. This is true for the date of my >> previous mail. There were several dependent packages upload since >> then, but I think the situation remained the same. > > In that case I'm ok with removing icu-config, but please don't entangle that > with the SONAME bump. That is, reintroduce icu-config so we can have an easy > transition, and once the transition is finished, then you are free to drop > icu-config. Sounds good? I do agree and open to do an other compilation test of the dependent packages with icu-config being available. Cheers, Laszlo/GCS
Bug#894159: transition: icu
On 23/04/18 21:19, László Böszörményi (GCS) wrote: > On Mon, Apr 23, 2018 at 7:57 PM, Emilio Pozuelo Monfort > wrote: >> First of all, sorry for the delay. I saw there were several issues here and >> decided to postpone this a bit. > Emilio! I already owe you a couple of beers. Hope we can meet again > in Hsinchu and have a chat about this. > >> On 26/03/18 22:29, László Böszörményi (GCS) wrote: >>> It will need a quick bootstrap. It needs to build without the Layout >>> Engine API, then the support library (icu-le-hb). Only then ICU can be >>> built with the Layout Engine API as the two libraries tied together. >> >> Can you explain that? Why can't you enable Layout Engine from the start? > The Layout Engine was always buggy and was abandoned for a while. It > was removed in ICU 58.1 [1]. Actually it was deprecated with ICU 54.1, > released in October, 2014 (three and a half years ago). > Someone started to re-implement the API using an external library, > HarfBuzz[2]. But it is also stalled, the last real code commit is from > 6th of May, 2016 [3]. Only one project still using it via the > Paragraph Layout and it's the OpenTTD game[4]. > To answer your question, as you see Layout Engine is an external > project by now. It needs to build with the actual ICU ABI, which > changes with major releases (hence the need of a transition). When the > Layout Engine is available for the current ICU ABI, then the > additional Paragraph Layout API / libraries of ICU can be built. This > is the cause of the ICU build -> icu-le-hb -> ICU build again steps. > I might bundle the two sources together into one source package, but > then every ICU build would do this three step compilation instead of > one. It's enough to do once IMHO for every ICU major releases. > In short, I should speak with the OpenTTD folks how / why they use > Paragraph Layout API and if they can migrate to an other solution. Ok, I didn't realise that icu-le-hb was a separate source package. >>> Some patches are simply adding '--with-icu=/usr' to the configure >>> invocation in debian/rules. Others are for ICU detection in the >>> configuration phase. No code change is needed in the packages. >>> If it matters, Ubuntu already transitioned with a bit different method. >> >> Are those changes and the large number of FTBFS because of the removal of >> icu-config? Can we keep it for now and file bugs at severity:important for >> the >> rdeps asking to fix the build when icu-config is removed? That should ease >> this >> transition. > Agree, keeping icu-config would help a lot with the transition. > Ubuntu did the transition this way. On the other hand, icu-config is a > simple script and don't know much about MultiArch and/or > cross-compilation. Upstream has pkg-config files for a while, but not > all dependent packages migrated to it yet. :-/ > OK, riscv64 already over the initial bootstrap and rebuild steps. > >> I would appreciate a number of failing rdeps and how many are due to ICU API >> changes and how many are due to icu-config removal. > As I remember, 99% of the FTBFS reasons were the icu-config removal, > others are the dependent package bugs I've already mentioned. I do not > recall any API change. In short, there are fifteen packages FTBFS and > all is due to the icu-config removal. This is true for the date of my > previous mail. There were several dependent packages upload since > then, but I think the situation remained the same. In that case I'm ok with removing icu-config, but please don't entangle that with the SONAME bump. That is, reintroduce icu-config so we can have an easy transition, and once the transition is finished, then you are free to drop icu-config. Sounds good? Cheers, Emilio
Bug#894159: transition: icu
On Mon, Apr 23, 2018 at 7:57 PM, Emilio Pozuelo Monfort wrote: > First of all, sorry for the delay. I saw there were several issues here and > decided to postpone this a bit. Emilio! I already owe you a couple of beers. Hope we can meet again in Hsinchu and have a chat about this. > On 26/03/18 22:29, László Böszörményi (GCS) wrote: >> It will need a quick bootstrap. It needs to build without the Layout >> Engine API, then the support library (icu-le-hb). Only then ICU can be >> built with the Layout Engine API as the two libraries tied together. > > Can you explain that? Why can't you enable Layout Engine from the start? The Layout Engine was always buggy and was abandoned for a while. It was removed in ICU 58.1 [1]. Actually it was deprecated with ICU 54.1, released in October, 2014 (three and a half years ago). Someone started to re-implement the API using an external library, HarfBuzz[2]. But it is also stalled, the last real code commit is from 6th of May, 2016 [3]. Only one project still using it via the Paragraph Layout and it's the OpenTTD game[4]. To answer your question, as you see Layout Engine is an external project by now. It needs to build with the actual ICU ABI, which changes with major releases (hence the need of a transition). When the Layout Engine is available for the current ICU ABI, then the additional Paragraph Layout API / libraries of ICU can be built. This is the cause of the ICU build -> icu-le-hb -> ICU build again steps. I might bundle the two sources together into one source package, but then every ICU build would do this three step compilation instead of one. It's enough to do once IMHO for every ICU major releases. In short, I should speak with the OpenTTD folks how / why they use Paragraph Layout API and if they can migrate to an other solution. >> Some patches are simply adding '--with-icu=/usr' to the configure >> invocation in debian/rules. Others are for ICU detection in the >> configuration phase. No code change is needed in the packages. >> If it matters, Ubuntu already transitioned with a bit different method. > > Are those changes and the large number of FTBFS because of the removal of > icu-config? Can we keep it for now and file bugs at severity:important for the > rdeps asking to fix the build when icu-config is removed? That should ease > this > transition. Agree, keeping icu-config would help a lot with the transition. Ubuntu did the transition this way. On the other hand, icu-config is a simple script and don't know much about MultiArch and/or cross-compilation. Upstream has pkg-config files for a while, but not all dependent packages migrated to it yet. :-/ OK, riscv64 already over the initial bootstrap and rebuild steps. > I would appreciate a number of failing rdeps and how many are due to ICU API > changes and how many are due to icu-config removal. As I remember, 99% of the FTBFS reasons were the icu-config removal, others are the dependent package bugs I've already mentioned. I do not recall any API change. In short, there are fifteen packages FTBFS and all is due to the icu-config removal. This is true for the date of my previous mail. There were several dependent packages upload since then, but I think the situation remained the same. Cheers, Laszlo/GCS
Bug#894159: transition: icu
Hi László, First of all, sorry for the delay. I saw there were several issues here and decided to postpone this a bit. Please see some questions below. On 26/03/18 22:29, László Böszörményi (GCS) wrote: > It will need a quick bootstrap. It needs to build without the Layout > Engine API, then the support library (icu-le-hb). Only then ICU can be > built with the Layout Engine API as the two libraries tied together. Can you explain that? Why can't you enable Layout Engine from the start? > Some patches are simply adding '--with-icu=/usr' to the configure > invocation in debian/rules. Others are for ICU detection in the > configuration phase. No code change is needed in the packages. > If it matters, Ubuntu already transitioned with a bit different method. Are those changes and the large number of FTBFS because of the removal of icu-config? Can we keep it for now and file bugs at severity:important for the rdeps asking to fix the build when icu-config is removed? That should ease this transition. I would appreciate a number of failing rdeps and how many are due to ICU API changes and how many are due to icu-config removal. Cheers, Emilio
Bug#894159: transition: icu
Package: release.debian.org Severity: normal User: release.debian@packages.debian.org Usertags: transition Hi RMs, While I've packaged new ICU releases, those were uploaded only to experimental. It's really time to change this, especially that some packages already need ICU 60.1+. It will need a quick bootstrap. It needs to build without the Layout Engine API, then the support library (icu-le-hb). Only then ICU can be built with the Layout Engine API as the two libraries tied together. Local rebuild of dependent packages[1] on amd64 revealed the good, the bad and the ugly. Level 1 rebuilt OK and as level 1.5 I've rebuilt boost1.62 and boost1.63 normally. For level 2 I could rebuilt the following packages with a small patch: 389-adminutil 389-ds-base an dwdiff ibus-qt open-vm-tools pyicu yaz Package aegisub FTBFS for a know issue, #873327 [2], fixed upstream but the maintainer doesn't respond for six months now. :( I've a patch for dee, but it FTBFS for a know problem, #876594 [3]. While I've a patch for grcompiler too and it builds, its self tests are failing. It's a highly outdated package. :( The first Debian revision of the current package version (v4.2) is uploaded on 2012, 26th of June. But its upstream is more or less still active[4] and has a much newer version (v5.0.2). The package libsimpleini compiles fine, but its symbols change with the new ICU version. Two packages, pyicu and yaz are highly outdated by the way, I may overtake the former. For level 3 two packages need a patch: 389-dsgw and libfolia. Others build correctly. Level 4 is a bit more complicated. I've a patch for the following packages: ucto php7.0 php7.1 php7.2 Two packages FTBFS for other reasons: cyrus-imapd for #883951 [5] which is fixed upstream, but maintainer doesn't respond for four months. yi is an other example, the problem is reported as #868637 [6] without any action for nine months. :( Upstream maybe solved the problem, at least there are more upstream releases out there. About level 5, only frog needs a patch. There's a failing package, kbibtex and it's reason is filed as #893538 [7]. Some patches are simply adding '--with-icu=/usr' to the configure invocation in debian/rules. Others are for ICU detection in the configuration phase. No code change is needed in the packages. If it matters, Ubuntu already transitioned with a bit different method. I hope this transition can be authorized and/or please tell me if any more information is needed. Thanks in advance, Laszlo/GCS [1] https://release.debian.org/transitions/html/auto-icu.html [2] https://bugs.debian.org/873327 [3] https://bugs.debian.org/876594 [4] https://github.com/silnrsi/grcompiler/commits/master [5] https://bugs.debian.org/883951 [6] https://bugs.debian.org/868637 [7] https://bugs.debian.org/893538