Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On Thursday 12 of April 2012, Matúš Kukan wrote: Maybe it is better to just let check target depend on build target ? In general. Do you mean making the toplevel targets depend on each other, or also in each module? If this is meant to be also for all modules, wouldn't this possibly still miss some problems? I.e. if e.g. sw links the i18npool library, does or does not sw depend on i18npool/ build target? If sw depends only on the library itself, then localedata_en still might not be built by the time sw unittests are run (in this example sw is far too late after i18npool for this to be realistic, but consider whatever is more realistic). Requiring everything be built before running any checks would avoid the problem, but I doubt anybody would want to rebuild everything just in order to run checks in sal/ . So unless the first case would work reliably, there does not seem to be any way other than finding all such dependencies, as much as doing that is painful, with such problems being random race conditions. Or do people use make check with demand to build only what is necessary for tests and nothing more ? I would expect that most of what the build target builds in each module is already a prerequisite for the module's checks (and it'd definitely be so in an ideal world where everything is tested). -- Lubos Lunak l.lu...@suse.cz ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On Thursday 12 of April 2012, Rene Engelhard wrote: On Thu, Apr 12, 2012 at 09:13:30PM +0200, Matúš Kukan wrote: Or do people use make check with demand to build only what is necessary for tests and nothing more ? Well, make check is for after a successful build to check the result, no? That's correct. So the next build will be almost a no-op, right? And there's not much point in checking something that is not up to date, so if something needs rebuilding, it should be rebuilt (and finally, make has this -o option). What especially is bad is when make install (thankfully gone) or make check rebuild unneeded stuff which gets rebuilt on every pass. What especially is bad is when unneeded stuff gets rebuilt on every pass. Fixed that for ya :). -- Lubos Lunak l.lu...@suse.cz ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On Fri, Apr 13, 2012 at 04:35:27PM +0200, Lubos Lunak wrote: On Thursday 12 of April 2012, Rene Engelhard wrote: On Thu, Apr 12, 2012 at 09:13:30PM +0200, Matúš Kukan wrote: Or do people use make check with demand to build only what is necessary for tests and nothing more ? Well, make check is for after a successful build to check the result, no? That's correct. So the next build will be almost a no-op, right? And there's Not right now :) Especially bad on slow archs. not much point in checking something that is not up to date, so if something True. But make check is ran after a normal make (build), so... Fixed that for ya :). When/how? (I don't see anything obvious from you on libreoffice-commits, but I might be blind) Regards, Rene ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On 13 April 2012 16:32, Lubos Lunak l.lu...@suse.cz wrote: On Thursday 12 of April 2012, Matúš Kukan wrote: Maybe it is better to just let check target depend on build target ? In general. Do you mean making the toplevel targets depend on each other, or also in each module? I mean to add into gbuild/Module.mk: unitcheck slowcheck : build So if you run make in sal it's for sal only. If you run make in tail_build, first everything is built and then tested. Note that the modules' dependencies (in the sense: first build sal then cppu) are written in module/prj/build.lst and build.pl reads them. If this is meant to be also for all modules, wouldn't this possibly still miss some problems? I.e. if e.g. sw links the i18npool library, does or does not sw depend on i18npool/ build target? If sw depends only on the library itself, then localedata_en still might not be built by the time sw unittests are run (in this example sw is far too late after i18npool for this to be realistic, but consider whatever is more realistic). Now i18npool is not yet in tail_build so when the time comes for sw everything is there. When everything will be in tail_build (or Makefile.post) sw/ won't depend on i18npool/ but with *check : build dependency all tests from all modules will be executed after everything is built. Requiring everything be built before running any checks would avoid the problem, but I doubt anybody would want to rebuild everything just in order to run checks in sal/ You can still do: cd sal/ make check Only sal is considered in that case, nothing before (solenv..) nor after (sw..) I would expect that most of what the build target builds in each module is already a prerequisite for the module's checks (and it'd definitely be so in an ideal world where everything is tested). Right, so it is good. Bjoern, what do you think about attached patch ? I left subsequentcheck untouched for now. (Sorry to involve you here but you can best understand) All the best, Matus 0001-gbuild-add-check-build-dependency.patch Description: Binary data ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On Friday 13 of April 2012, Rene Engelhard wrote: On Fri, Apr 13, 2012 at 04:35:27PM +0200, Lubos Lunak wrote: On Thursday 12 of April 2012, Rene Engelhard wrote: On Thu, Apr 12, 2012 at 09:13:30PM +0200, Matúš Kukan wrote: Or do people use make check with demand to build only what is necessary for tests and nothing more ? Well, make check is for after a successful build to check the result, no? That's correct. So the next build will be almost a no-op, right? And there's Not right now :) Especially bad on slow archs. There's always the -o option to make. And, assuming this would avoid all the build breakages because of race conditions, I'd assume the time spent on this still worth the time not spent hunting random breakages. Fixed that for ya :). When/how? (I don't see anything obvious from you on libreoffice-commits, but I might be blind) = What especially is bad is when make install (thankfully gone) or make check rebuild unneeded stuff which gets rebuilt on every pass. What especially is bad is when unneeded stuff gets rebuilt on every pass. = -- Lubos Lunak l.lu...@suse.cz ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On Friday 13 of April 2012, Matúš Kukan wrote: On 13 April 2012 16:32, Lubos Lunak l.lu...@suse.cz wrote: On Thursday 12 of April 2012, Matúš Kukan wrote: Maybe it is better to just let check target depend on build target ? In general. Do you mean making the toplevel targets depend on each other, or also in each module? I mean to add into gbuild/Module.mk: unitcheck slowcheck : build So if you run make in sal it's for sal only. If you run make in tail_build, first everything is built and then tested. Note that the modules' dependencies (in the sense: first build sal then cppu) are written in module/prj/build.lst and build.pl reads them. Does this mean that if there is a build race condition in check because of a missing runtime dependency, the problem is always inside the module? If that's the case, it does not seem it should be that hard to find such problems, I've assumed the broken dependency can be on anything. -- Lubos Lunak l.lu...@suse.cz ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On 13 April 2012 18:38, Lubos Lunak l.lu...@suse.cz wrote: On Friday 13 of April 2012, Matúš Kukan wrote: Note that the modules' dependencies (in the sense: first build sal then cppu) are written in module/prj/build.lst and build.pl reads them. Does this mean that if there is a build race condition in check because of a missing runtime dependency, the problem is always inside the module? In the sense that also tail_build is module, yes. But in tail_build it can be missing from any module listed in Module_tail_build.mk So far I've seen these runtime dependency problems only in sal and i18npool. I don't remember any from taill_build. I just think they are not necessary and we need to build things before testing anyway. The difference is: if you remove not tested library from module and run make check there now: nothing happens, only tests are executed after: it will get build first and then the tests are executed Best, Matus ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
Hi all, On 13 April 2012 18:21, Matúš Kukan matus.ku...@gmail.com wrote: Bjoern, what do you think about attached patch ? I left subsequentcheck untouched for now. Oh no, it won't work in new gbuild world where we can do: make -rs -j 4 $(WORKDIR)/Module/check/sw or anything, from clean. That's something as Lubos was trying to tell me but I did not see it this way. So if we want to be able to do that (I think yes) this *check : build dependency thing is nonsense. I am really sorry for this thread. It now looks to me as wasted. I did not realize that before. All the best, Matus ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: check : build dependency (was: Re: what exactly needs localedata_en library ?)
On Thu, Apr 12, 2012 at 09:13:30PM +0200, Matúš Kukan wrote: Maybe it is better to just let check target depend on build target ? No? Or do people use make check with demand to build only what is necessary for tests and nothing more ? Well, make check is for after a successful build to check the result, no? What especially is bad is when make install (thankfully gone) or make check rebuild unneeded stuff which gets rebuilt on every pass. IMHO it should just do what it does. Run the tests (yes, I know this involves some other round of going over all modules, but...) Regards, Rene ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: what exactly needs localedata_en library ?
On 04/12/2012 04:14 PM, Eike Rathke wrote: On Thursday, 2012-04-12 13:33:38 +0200, Lubos Lunak wrote: As far as I can see, it is the i18npool library that dlopens localedata_en (i18npool/source/localedata/localedata.cxx), so I think the best way to express this dependency is to make i18npool depend on localedata_en (and localedata_es, localedata_euro and localedata_others). Might be an approach. While at it, dependencies on the various libdict_* and lib*_data* that are dlopen'ed by i18npool could be added as well. ...as long as it does not take away too much opportunity for parallelization... Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: what exactly needs localedata_en library ?
On Wednesday 11 of April 2012, Matúš Kukan wrote: Hi, I was investigating which dependency is missing for sometimes failing i18npool_test_breakiterator and it is localedata_en. The dependency was often explicitly written in sc's makefiles (but i18npool is not yet in tail_build, so not needed) and I think: gb_CppunitTest_use_ure,foo = foo requires localedata_en is true, so I created attached patch where the dependency is moved to gb_CppunitTest_use_ure definition. But I don't know if also the other implication is true: gb_CppunitTest_use_ure,foo = foo requires localedata_en I did not push yet but if no one is against I will do so soon because it is safe and maybe also the right thing to do ? As far as I can see, it is the i18npool library that dlopens localedata_en (i18npool/source/localedata/localedata.cxx), so I think the best way to express this dependency is to make i18npool depend on localedata_en (and localedata_es, localedata_euro and localedata_others). While this is not 100% technically correct, as it is only a runtime dependency and not a build dependency, there's no cycle and it should make sure that anything that does or will need localedata_en will have it ready. -- Lubos Lunak l.lu...@suse.cz ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: what exactly needs localedata_en library ?
Hi, On Thursday, 2012-04-12 13:33:38 +0200, Lubos Lunak wrote: As far as I can see, it is the i18npool library that dlopens localedata_en (i18npool/source/localedata/localedata.cxx), so I think the best way to express this dependency is to make i18npool depend on localedata_en (and localedata_es, localedata_euro and localedata_others). Might be an approach. While at it, dependencies on the various libdict_* and lib*_data* that are dlopen'ed by i18npool could be added as well. Eike -- LibreOffice Calc developer. Number formatter stricken i18n transpositionizer. GnuPG key 0x293C05FD : 997A 4C60 CE41 0149 0DB3 9E96 2F1A D073 293C 05FD pgplfG7IEcaDa.pgp Description: PGP signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
check : build dependency (was: Re: what exactly needs localedata_en library ?)
On 12 April 2012 16:14, Eike Rathke er...@redhat.com wrote: On Thursday, 2012-04-12 13:33:38 +0200, Lubos Lunak wrote: As far as I can see, it is the i18npool library that dlopens localedata_en (i18npool/source/localedata/localedata.cxx), so I think the best way to express this dependency is to make i18npool depend on localedata_en (and localedata_es, localedata_euro and localedata_others). Might be an approach. While at it, dependencies on the various libdict_* and lib*_data* that are dlopen'ed by i18npool could be added as well. Hmm, for now I created: http://cgit.freedesktop.org/libreoffice/core/commit/?id=5dce272584cc79da727bdd4887d60cec04cbce11 I did not put there libdict_* and lib*_data* Maybe it is better to just let check target depend on build target ? In general. It could prevent more missing dependencies in future. In tail_build, all tests would run in the end, so no chance to stop the build process if there is something wrong and _maybe_ it would take more time because of less effective ordering. But there is no guarantee for ordering anyway and I can't see any problem with this. Does it make sense to add: check : build dependency ? Or do people use make check with demand to build only what is necessary for tests and nothing more ? -- Matus ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
what exactly needs localedata_en library ?
Hi, I was investigating which dependency is missing for sometimes failing i18npool_test_breakiterator and it is localedata_en. The dependency was often explicitly written in sc's makefiles (but i18npool is not yet in tail_build, so not needed) and I think: gb_CppunitTest_use_ure,foo = foo requires localedata_en is true, so I created attached patch where the dependency is moved to gb_CppunitTest_use_ure definition. But I don't know if also the other implication is true: gb_CppunitTest_use_ure,foo = foo requires localedata_en I did not push yet but if no one is against I will do so soon because it is safe and maybe also the right thing to do ? Best, Matus 0001-add-dependency-on-localedata_en-to-gb_CppunitTest_us.patch Description: Binary data ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: what exactly needs localedata_en library ?
On 04/11/2012 05:17 PM, Matúš Kukan wrote: I was investigating which dependency is missing for sometimes failing i18npool_test_breakiterator and it is localedata_en. The dependency was often explicitly written in sc's makefiles (but i18npool is not yet in tail_build, so not needed) and I think: gb_CppunitTest_use_ure,foo = foo requires localedata_en is true, so I created attached patch where the dependency is moved to gb_CppunitTest_use_ure definition. But I don't know if also the other implication is true: gb_CppunitTest_use_ure,foo= foo requires localedata_en I did not push yet but if no one is against I will do so soon because it is safe and maybe also the right thing to do ? Safe: at least as long as people do not want to use the other features of gb_CppunitTest_use_ure (like specifying UNO_TYPES and UNO_SERVICES) in a test that should be runnable before i18npool is built. The right thing: probably not. Rather, a consequence of too fat (unit) test dependencies hidden with a sledgehammer. At any rate, it makes gb_CppunitTest_use_ure even more of a misnomer. That said, probably a sufficiently pragmatic solution, anyway. ;) Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
