Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/08/2018 12:33 PM, Peter Kjellerstedt wrote: FYI, the already applied patch only solves half the problem. I have an updated version that solves all of it that I am preparing for publication. I only need to make some verifications first... I just found that out as well. Lua is not relevant, we don't even enable or build it. Anyway, I've been fighting*scratch*convincing upstream all morning that fcntl bombs are a real problem under Docker, you are welcome to join :) https://bugzilla.redhat.com/show_bug.cgi?id=1537564 Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
> -Original Message- > From: yocto-boun...@yoctoproject.org [mailto:yocto- > boun...@yoctoproject.org] On Behalf Of Alexander Kanavin > Sent: den 7 maj 2018 12:12 > To: Mirza Krak <mirza.k...@gmail.com> > Cc: Yocto list discussion <yocto@yoctoproject.org> > Subject: Re: [yocto] ROOTFS_RPM_DEBUG undocumented > > On 05/07/2018 12:03 PM, Mirza Krak wrote: > > I have also been bitten by rpm performance issues in docker and glad > > that I found this. I am in the process of verifying above (on pyro > > branch). > > > > Is this patch on its way to pyro and rocko branches? > > As far as I know, no. You are welcome to backport and submit. > > Alex FYI, the already applied patch only solves half the problem. I have an updated version that solves all of it that I am preparing for publication. I only need to make some verifications first... //Peter -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/07/2018 12:03 PM, Mirza Krak wrote: I have also been bitten by rpm performance issues in docker and glad that I found this. I am in the process of verifying above (on pyro branch). Is this patch on its way to pyro and rocko branches? As far as I know, no. You are welcome to backport and submit. Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 4 May 2018 at 14:41, Alexander Kanavinwrote: > On 05/04/2018 03:42 PM, Paulo Neves wrote: >> >> I will propose a patch with a default bb.debug, and always have the >> verbose on dnf. Let's see how it affects the performance. I cannot >> test this in a docker container because of the problems described >> below: >> >> In the mean time I found what was happening with my slow do_rootfs. I >> was running bitbake with RPM packaging in a docker instance. It seems >> there is a problem with RPM/dnf in bitbake where it is extremely slow >> inside a docker container (1 file per second). After changing to IPK >> packages everything went smoothly. >> >> Others have had the same problem[1]. I will later report this to >> openembedded mailing list as this effectively makes RPM unusable in a >> container. I will also provide a Dockerfile to reproduce this problem. >> >> [1] https://github.com/moby/moby/issues/23137#issuecomment-359097008 > > > No need to report. We have already fixed this and reported upstream: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/rpm/files/0001-Revert-Set-FD_CLOEXEC-on-opened-files-before-exec-fr.patch I have also been bitten by rpm performance issues in docker and glad that I found this. I am in the process of verifying above (on pyro branch). Is this patch on its way to pyro and rocko branches? -- Med Vänliga Hälsningar / Best Regards Mirza Krak -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
Ouch you are right. Damn me for using pyro still? After applying this patch I will report on the remaining issues. On Fri, May 4, 2018 at 2:41 PM, Alexander Kanavinwrote: > On 05/04/2018 03:42 PM, Paulo Neves wrote: >> >> I will propose a patch with a default bb.debug, and always have the >> verbose on dnf. Let's see how it affects the performance. I cannot >> test this in a docker container because of the problems described >> below: >> >> In the mean time I found what was happening with my slow do_rootfs. I >> was running bitbake with RPM packaging in a docker instance. It seems >> there is a problem with RPM/dnf in bitbake where it is extremely slow >> inside a docker container (1 file per second). After changing to IPK >> packages everything went smoothly. >> >> Others have had the same problem[1]. I will later report this to >> openembedded mailing list as this effectively makes RPM unusable in a >> container. I will also provide a Dockerfile to reproduce this problem. >> >> [1] https://github.com/moby/moby/issues/23137#issuecomment-359097008 > > > No need to report. We have already fixed this and reported upstream: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/rpm/files/0001-Revert-Set-FD_CLOEXEC-on-opened-files-before-exec-fr.patch > > https://bugzilla.redhat.com/show_bug.cgi?id=1537564 > > > Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/04/2018 03:50 PM, Scott Rifenbark wrote: If ROOTFS_RPM_DEBUG should be a documented variable in the Yocto Project Reference Manual, could someone please provide me with some base explanation of the variable and any usage specifics? No need, we're going to remove it. Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
If ROOTFS_RPM_DEBUG should be a documented variable in the Yocto Project Reference Manual, could someone please provide me with some base explanation of the variable and any usage specifics? Thanks, Scott On Fri, May 4, 2018 at 5:41 AM, Alexander Kanavin < alexander.kana...@linux.intel.com> wrote: > On 05/04/2018 03:42 PM, Paulo Neves wrote: > >> I will propose a patch with a default bb.debug, and always have the >> verbose on dnf. Let's see how it affects the performance. I cannot >> test this in a docker container because of the problems described >> below: >> >> In the mean time I found what was happening with my slow do_rootfs. I >> was running bitbake with RPM packaging in a docker instance. It seems >> there is a problem with RPM/dnf in bitbake where it is extremely slow >> inside a docker container (1 file per second). After changing to IPK >> packages everything went smoothly. >> >> Others have had the same problem[1]. I will later report this to >> openembedded mailing list as this effectively makes RPM unusable in a >> container. I will also provide a Dockerfile to reproduce this problem. >> >> [1] https://github.com/moby/moby/issues/23137#issuecomment-359097008 >> > > No need to report. We have already fixed this and reported upstream: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/rec > ipes-devtools/rpm/files/0001-Revert-Set-FD_CLOEXEC-on-opene > d-files-before-exec-fr.patch > > https://bugzilla.redhat.com/show_bug.cgi?id=1537564 > > > > Alex > -- > ___ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/04/2018 03:41 PM, Alexander Kanavin wrote: https://bugzilla.redhat.com/show_bug.cgi?id=1537564 Obviously you and anyone else affected need to make noise in the redhat bugzilla, otherwise they're likely to keep this low priority. Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/04/2018 03:42 PM, Paulo Neves wrote: I will propose a patch with a default bb.debug, and always have the verbose on dnf. Let's see how it affects the performance. I cannot test this in a docker container because of the problems described below: In the mean time I found what was happening with my slow do_rootfs. I was running bitbake with RPM packaging in a docker instance. It seems there is a problem with RPM/dnf in bitbake where it is extremely slow inside a docker container (1 file per second). After changing to IPK packages everything went smoothly. Others have had the same problem[1]. I will later report this to openembedded mailing list as this effectively makes RPM unusable in a container. I will also provide a Dockerfile to reproduce this problem. [1] https://github.com/moby/moby/issues/23137#issuecomment-359097008 No need to report. We have already fixed this and reported upstream: http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/rpm/files/0001-Revert-Set-FD_CLOEXEC-on-opened-files-before-exec-fr.patch https://bugzilla.redhat.com/show_bug.cgi?id=1537564 Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
I will propose a patch with a default bb.debug, and always have the verbose on dnf. Let's see how it affects the performance. I cannot test this in a docker container because of the problems described below: In the mean time I found what was happening with my slow do_rootfs. I was running bitbake with RPM packaging in a docker instance. It seems there is a problem with RPM/dnf in bitbake where it is extremely slow inside a docker container (1 file per second). After changing to IPK packages everything went smoothly. Others have had the same problem[1]. I will later report this to openembedded mailing list as this effectively makes RPM unusable in a container. I will also provide a Dockerfile to reproduce this problem. [1] https://github.com/moby/moby/issues/23137#issuecomment-359097008 Paulo Neves On Thu, May 3, 2018 at 3:03 PM, Alexander Kanavinwrote: > On 05/03/2018 01:42 PM, Paulo Neves wrote: >> >> # Add ROOTFS_RPM_DEBUG to the documentation; > > > I'd rather get rid of it, the less variables the better :) > >> # Detect if we are running with debug output and enable the debugging >> output. This is the most elegant solution but I do not know how to >> detect if debug log level is turned on; > > > I don't think you are meant to detect that; bitbake's debug level is not > meant to affect what is being printed, but merely whether it's printed or > discarded. > >> # Have dnf always print in verbose mode and print the output to >> bb.debug instead of bb.note. > > > I think this is the best solution actually. > >> I am happy to provide a patch upon decision or suggestions. > > > Thank you :) Do measure the performance before and after as we don't want to > introduce a significant slowdown here. > > > Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/03/2018 04:37 PM, Peter Kjellerstedt wrote: # Have dnf always print in verbose mode and print the output to bb.debug instead of bb.note. I think this is the best solution actually. I disagree with changing bb.note to bb.debug for this. It is very good to be able to look in log.do_rootfs to see what dnf has done (we have ROOTFS_RPM_DEBUG set by default). Changing the output to debug level would mean having to run bitbake with -D, which is anything but desirable. If I'm reading the bitbake --help right, bb.debug (and bb.note) are always written to the log; setting -D level or -v only affects what is printed on stdout. Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
> -Original Message- > From: yocto-boun...@yoctoproject.org [mailto:yocto- > boun...@yoctoproject.org] On Behalf Of Alexander Kanavin > Sent: den 3 maj 2018 15:04 > To: Paulo Neves <ptsne...@gmail.com>; Yocto list discussion > <yocto@yoctoproject.org> > Subject: Re: [yocto] ROOTFS_RPM_DEBUG undocumented > > On 05/03/2018 01:42 PM, Paulo Neves wrote: > > # Add ROOTFS_RPM_DEBUG to the documentation; > > I'd rather get rid of it, the less variables the better :) > > > # Detect if we are running with debug output and enable the debugging > > output. This is the most elegant solution but I do not know how to > > detect if debug log level is turned on; > > I don't think you are meant to detect that; bitbake's debug level is > not > meant to affect what is being printed, but merely whether it's printed > or discarded. > > > # Have dnf always print in verbose mode and print the output to > > bb.debug instead of bb.note. > > I think this is the best solution actually. I disagree with changing bb.note to bb.debug for this. It is very good to be able to look in log.do_rootfs to see what dnf has done (we have ROOTFS_RPM_DEBUG set by default). Changing the output to debug level would mean having to run bitbake with -D, which is anything but desirable. > > I am happy to provide a patch upon decision or suggestions. > > Thank you :) Do measure the performance before and after as we don't > want to introduce a significant slowdown here. > > Alex //Peter -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] ROOTFS_RPM_DEBUG undocumented
On 05/03/2018 01:42 PM, Paulo Neves wrote: # Add ROOTFS_RPM_DEBUG to the documentation; I'd rather get rid of it, the less variables the better :) # Detect if we are running with debug output and enable the debugging output. This is the most elegant solution but I do not know how to detect if debug log level is turned on; I don't think you are meant to detect that; bitbake's debug level is not meant to affect what is being printed, but merely whether it's printed or discarded. # Have dnf always print in verbose mode and print the output to bb.debug instead of bb.note. I think this is the best solution actually. I am happy to provide a patch upon decision or suggestions. Thank you :) Do measure the performance before and after as we don't want to introduce a significant slowdown here. Alex -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] ROOTFS_RPM_DEBUG undocumented
Hello all, I recently had the problem where the do_rootfs task seemed to hang. The output where it hang looked like [1] and stayed that way for the better part of an hour. When I ran bitbake with -vD arguments not more relevant output was given [2], also hanging for a lot of time. As noted in another mailing list post about rootfs hanging[3], the culprit was the enormous amount of kernel modules which were being built. I only discovered this because I was able to strace and see that actually bitbake was not hanging but actually running dnf install. When I searched for the code responsible for the invocation of dnf I notice there is an undocumented variable called ROOTFS_RPM_DEBUG that enables verbosity of the dnf and prints it out. This would have been very helpful output for my 'bitbake -vD' invocation, and truly debugging information would have helped. What I ask is a decision on which option would better suite this issue: # Add ROOTFS_RPM_DEBUG to the documentation; # Detect if we are running with debug output and enable the debugging output. This is the most elegant solution but I do not know how to detect if debug log level is turned on; # Have dnf always print in verbose mode and print the output to bb.debug instead of bb.note. I am happy to provide a patch upon decision or suggestions. Paulo Neves [1] NOTE: recipe drotag-cloud-image-debug-1.0-r0: task do_rootfs: Started [2] DEBUG: drotag-cloud-image-debug-1.0-r0 do_rootfs: Executing python function do_rootfs [3] https://lists.yoctoproject.org/pipermail/yocto/2017-May/036293.html -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto