Re: [ANN] U-Boot v2024.07-rc5 released
On Thu, Jun 27, 2024 at 10:23:34AM +0200, Christophe Leroy wrote: > > > Le 26/06/2024 à 17:01, Tom Rini a écrit : > > On Tue, Jun 25, 2024 at 09:20:54PM +0200, Christophe Leroy wrote: > > > > > > > > > Le 25/06/2024 à 17:19, Tom Rini a écrit : > > > > On Tue, Jun 25, 2024 at 03:12:38PM +, LEROY Christophe wrote: > > > > > Hi All, > > > > > > > > > > Le 24/06/2024 à 21:30, Tom Rini a écrit : > > > > > > Hey all, > > > > > > > > > > > > Well, once again I need to check my calender reminders since, > > > > > > whoops, > > > > > > I'm a week late. That said, looking at the list of changes in this > > > > > > slightly elongated period, I think it's OK. At this point we're a > > > > > > week > > > > > > out from the release, so it's time for clear and obvious regression > > > > > > fixes. Ideally ones introduced this release cycle so there's less > > > > > > chances of unintended side-effects. > > > > > > > > > > > > In terms of a changelog, > > > > > > git log --merges v2024.07-rc4..v2024.07-rc5 > > > > > > contains what I've pulled but as always, better PR messages and tags > > > > > > will provide better results here. > > > > > > > > > > > > With that, the final release is planned for Monday, July 1st, 2024. > > > > > > Thanks all! > > > > > > > > > > > > > > > > For your information, I have started testing v2024.07 on my powerpc > > > > > boards in operational condition and I have a major problem : U-boot > > > > > doesn't get the Linux kernel started before the watchdog fires. I have > > > > > to disable watchdog to get it running. > > > > > > > > > > This problem doesn't occur with v2024.04. > > > > > > > > > > I bisected it to commit 340fdf1303dc ("zlib: Port fix for > > > > > CVE-2016-9841 > > > > > to U-Boot") and I also confirm that v2024.07-rc5 works well with that > > > > > commit reverted on top of it. > > > > > > > > > > Any idea of what is going wrong with that commit ? > > > > > > > > Are you able to check the zlib performance before/after that commit? > > > > > > > > > > Indeed there's a huge difference. > > > > > > Before : 12298074 timebase ticks to decompress kernel > > > After : 17145906 ticks, that is 40% more time. > > > > > > But behind that, we also need to understand why the watchdog is not kept > > > alive during that time. > > > > That is odd, I would have expected it to have been kept alive. > > > > I digged into it a bit more and really that commit 340fdf1303dc ("zlib: Port > fix for CVE-2016-9841 to U-Boot") is the issue. The watchdog is still kept > alive, but not fast enough. On powerpc 8xx, the CPU watchdog has a timeout > of slightly more than 1s, so it must be kept alive very often. > > schedule() is called in inflate() just before calling inflate_fast(), which > worked until now because the maximum time spent in inflate_fast() was about > 300ms. With the above commit, more than 1s is spent in inflate_fast() , for > the problematic block the time spent inside inflate_fast() is multiplied by > almost 4. > > The problem is that the commit does more than what it says and removes > an important optimisation that is using halfwords instead of bytes for the > copy. > > I'm going to send shortly a patch to partially revert that commit. But if > you feel it is better to revert the commit completely and provide a fix in > next cycle, it is ok as well, up to you but I think it is important to get > that fixed for v2024.07. > > I'm also sending a fix on the setup of 8xx watchdog, but that has almost no > impact in reality and doesn't fix the problem. Thanks for digging in to all of this, I'll have to think about what path is better at this point in the cycle. -- Tom signature.asc Description: PGP signature
Re: [ANN] U-Boot v2024.07-rc5 released
Le 26/06/2024 à 17:01, Tom Rini a écrit : On Tue, Jun 25, 2024 at 09:20:54PM +0200, Christophe Leroy wrote: Le 25/06/2024 à 17:19, Tom Rini a écrit : On Tue, Jun 25, 2024 at 03:12:38PM +, LEROY Christophe wrote: Hi All, Le 24/06/2024 à 21:30, Tom Rini a écrit : Hey all, Well, once again I need to check my calender reminders since, whoops, I'm a week late. That said, looking at the list of changes in this slightly elongated period, I think it's OK. At this point we're a week out from the release, so it's time for clear and obvious regression fixes. Ideally ones introduced this release cycle so there's less chances of unintended side-effects. In terms of a changelog, git log --merges v2024.07-rc4..v2024.07-rc5 contains what I've pulled but as always, better PR messages and tags will provide better results here. With that, the final release is planned for Monday, July 1st, 2024. Thanks all! For your information, I have started testing v2024.07 on my powerpc boards in operational condition and I have a major problem : U-boot doesn't get the Linux kernel started before the watchdog fires. I have to disable watchdog to get it running. This problem doesn't occur with v2024.04. I bisected it to commit 340fdf1303dc ("zlib: Port fix for CVE-2016-9841 to U-Boot") and I also confirm that v2024.07-rc5 works well with that commit reverted on top of it. Any idea of what is going wrong with that commit ? Are you able to check the zlib performance before/after that commit? Indeed there's a huge difference. Before : 12298074 timebase ticks to decompress kernel After : 17145906 ticks, that is 40% more time. But behind that, we also need to understand why the watchdog is not kept alive during that time. That is odd, I would have expected it to have been kept alive. I digged into it a bit more and really that commit 340fdf1303dc ("zlib: Port fix for CVE-2016-9841 to U-Boot") is the issue. The watchdog is still kept alive, but not fast enough. On powerpc 8xx, the CPU watchdog has a timeout of slightly more than 1s, so it must be kept alive very often. schedule() is called in inflate() just before calling inflate_fast(), which worked until now because the maximum time spent in inflate_fast() was about 300ms. With the above commit, more than 1s is spent in inflate_fast() , for the problematic block the time spent inside inflate_fast() is multiplied by almost 4. The problem is that the commit does more than what it says and removes an important optimisation that is using halfwords instead of bytes for the copy. I'm going to send shortly a patch to partially revert that commit. But if you feel it is better to revert the commit completely and provide a fix in next cycle, it is ok as well, up to you but I think it is important to get that fixed for v2024.07. I'm also sending a fix on the setup of 8xx watchdog, but that has almost no impact in reality and doesn't fix the problem. Thanks Christophe
Re: [ANN] U-Boot v2024.07-rc5 released
On Tue, Jun 25, 2024 at 09:20:54PM +0200, Christophe Leroy wrote: > > > Le 25/06/2024 à 17:19, Tom Rini a écrit : > > On Tue, Jun 25, 2024 at 03:12:38PM +, LEROY Christophe wrote: > > > Hi All, > > > > > > Le 24/06/2024 à 21:30, Tom Rini a écrit : > > > > Hey all, > > > > > > > > Well, once again I need to check my calender reminders since, whoops, > > > > I'm a week late. That said, looking at the list of changes in this > > > > slightly elongated period, I think it's OK. At this point we're a week > > > > out from the release, so it's time for clear and obvious regression > > > > fixes. Ideally ones introduced this release cycle so there's less > > > > chances of unintended side-effects. > > > > > > > > In terms of a changelog, > > > > git log --merges v2024.07-rc4..v2024.07-rc5 > > > > contains what I've pulled but as always, better PR messages and tags > > > > will provide better results here. > > > > > > > > With that, the final release is planned for Monday, July 1st, 2024. > > > > Thanks all! > > > > > > > > > > For your information, I have started testing v2024.07 on my powerpc > > > boards in operational condition and I have a major problem : U-boot > > > doesn't get the Linux kernel started before the watchdog fires. I have > > > to disable watchdog to get it running. > > > > > > This problem doesn't occur with v2024.04. > > > > > > I bisected it to commit 340fdf1303dc ("zlib: Port fix for CVE-2016-9841 > > > to U-Boot") and I also confirm that v2024.07-rc5 works well with that > > > commit reverted on top of it. > > > > > > Any idea of what is going wrong with that commit ? > > > > Are you able to check the zlib performance before/after that commit? > > > > Indeed there's a huge difference. > > Before : 12298074 timebase ticks to decompress kernel > After : 17145906 ticks, that is 40% more time. > > But behind that, we also need to understand why the watchdog is not kept > alive during that time. That is odd, I would have expected it to have been kept alive. -- Tom signature.asc Description: PGP signature
Re: [ANN] U-Boot v2024.07-rc5 released
Hi All, Le 24/06/2024 à 21:30, Tom Rini a écrit : > Hey all, > > Well, once again I need to check my calender reminders since, whoops, > I'm a week late. That said, looking at the list of changes in this > slightly elongated period, I think it's OK. At this point we're a week > out from the release, so it's time for clear and obvious regression > fixes. Ideally ones introduced this release cycle so there's less > chances of unintended side-effects. > > In terms of a changelog, > git log --merges v2024.07-rc4..v2024.07-rc5 > contains what I've pulled but as always, better PR messages and tags > will provide better results here. > > With that, the final release is planned for Monday, July 1st, 2024. > Thanks all! > For your information, I have started testing v2024.07 on my powerpc boards in operational condition and I have a major problem : U-boot doesn't get the Linux kernel started before the watchdog fires. I have to disable watchdog to get it running. This problem doesn't occur with v2024.04. I bisected it to commit 340fdf1303dc ("zlib: Port fix for CVE-2016-9841 to U-Boot") and I also confirm that v2024.07-rc5 works well with that commit reverted on top of it. Any idea of what is going wrong with that commit ? Christophe
Re: [ANN] U-Boot v2024.07-rc5 released
Le 25/06/2024 à 17:19, Tom Rini a écrit : On Tue, Jun 25, 2024 at 03:12:38PM +, LEROY Christophe wrote: Hi All, Le 24/06/2024 à 21:30, Tom Rini a écrit : Hey all, Well, once again I need to check my calender reminders since, whoops, I'm a week late. That said, looking at the list of changes in this slightly elongated period, I think it's OK. At this point we're a week out from the release, so it's time for clear and obvious regression fixes. Ideally ones introduced this release cycle so there's less chances of unintended side-effects. In terms of a changelog, git log --merges v2024.07-rc4..v2024.07-rc5 contains what I've pulled but as always, better PR messages and tags will provide better results here. With that, the final release is planned for Monday, July 1st, 2024. Thanks all! For your information, I have started testing v2024.07 on my powerpc boards in operational condition and I have a major problem : U-boot doesn't get the Linux kernel started before the watchdog fires. I have to disable watchdog to get it running. This problem doesn't occur with v2024.04. I bisected it to commit 340fdf1303dc ("zlib: Port fix for CVE-2016-9841 to U-Boot") and I also confirm that v2024.07-rc5 works well with that commit reverted on top of it. Any idea of what is going wrong with that commit ? Are you able to check the zlib performance before/after that commit? Indeed there's a huge difference. Before : 12298074 timebase ticks to decompress kernel After : 17145906 ticks, that is 40% more time. But behind that, we also need to understand why the watchdog is not kept alive during that time. Christophe
Re: [ANN] U-Boot v2024.07-rc5 released
On Tue, Jun 25, 2024 at 03:12:38PM +, LEROY Christophe wrote: > Hi All, > > Le 24/06/2024 à 21:30, Tom Rini a écrit : > > Hey all, > > > > Well, once again I need to check my calender reminders since, whoops, > > I'm a week late. That said, looking at the list of changes in this > > slightly elongated period, I think it's OK. At this point we're a week > > out from the release, so it's time for clear and obvious regression > > fixes. Ideally ones introduced this release cycle so there's less > > chances of unintended side-effects. > > > > In terms of a changelog, > > git log --merges v2024.07-rc4..v2024.07-rc5 > > contains what I've pulled but as always, better PR messages and tags > > will provide better results here. > > > > With that, the final release is planned for Monday, July 1st, 2024. > > Thanks all! > > > > For your information, I have started testing v2024.07 on my powerpc > boards in operational condition and I have a major problem : U-boot > doesn't get the Linux kernel started before the watchdog fires. I have > to disable watchdog to get it running. > > This problem doesn't occur with v2024.04. > > I bisected it to commit 340fdf1303dc ("zlib: Port fix for CVE-2016-9841 > to U-Boot") and I also confirm that v2024.07-rc5 works well with that > commit reverted on top of it. > > Any idea of what is going wrong with that commit ? Are you able to check the zlib performance before/after that commit? -- Tom signature.asc Description: PGP signature