Just to followup since the required packages are finally available, the patches have been updated and are passing CI now.
https://review.openstack.org/#/c/202763/ -Alex On Fri, Jul 24, 2015 at 8:32 AM, Alex Schultz <aschu...@mirantis.com> wrote: > Unfortunately we got stuck with package availability issues. It was > successful without using librarian packages for building, but since we need > to leverage packages the work to get packaged versions of the code has > unfortunately blocked this. I've got the packages built, but the time to > get them to an publicly available mirror has killed this effort. Since it's > now past FF, I don't think this will be for 7.0 but we should be in a good > spot to start early for 8.0. Also I've switched to librarian-puppet-simple > since it has far fewer dependencies and we only want to support git > references for packages. Once that makes it to the mirror and can pass CI, > this change would in theory be ready. I'm continuing to keep an eye on the > changes so that we'll be ready to start running with them when the time > comes. > > -Alex > > On Fri, Jul 24, 2015 at 1:41 AM, Mike Scherbakov <mscherba...@mirantis.com > > wrote: > >> Do we have any success here.. ? >> >> On Mon, Jul 20, 2015 at 8:32 AM Alex Schultz <aschu...@mirantis.com> >> wrote: >> >>> Vladimir, >>> >>> Thanks. Can you point me to the error for perestroika? I'd be happy to >>> take a look as well. I spent most of Friday throwing various options at the >>> CI system to try and figure out how to get the spec to work with the CI >>> fuel-library package building so perhaps there's a different way to handle >>> this in the spec. >>> >>> -Alex >>> >>> On Mon, Jul 20, 2015 at 10:02 AM, Vladimir Kozhukalov < >>> vkozhuka...@mirantis.com> wrote: >>> >>>> Alex, >>>> >>>> As I've just found out this package available here [1] is not actually >>>> build with your patch (instead it is from previous successful build). Looks >>>> like "perestroika" can not build this package due to some environment >>>> related issues. I've poked Dmitry Burmistrov to check it out. >>>> However, your patch is OK, make system can build this package and ISO >>>> passes BVT tests. >>>> >>>> >>>> [1] >>>> http://172.18.160.74/osci/review/CR-202763/mos/7.0/fuel/base/centos6/Packages/fuel-library7.0-7.0.0-1.mos6888.git.bac86fe.noarch.rpm >>>> >>>> Vladimir Kozhukalov >>>> >>>> On Mon, Jul 20, 2015 at 4:04 PM, Vladimir Kozhukalov < >>>> vkozhuka...@mirantis.com> wrote: >>>> >>>>> Alex, >>>>> >>>>> Yes, spec is much better place to introduce this. BTW, "perestroika" >>>>> builds new package for every commit and patch set and publishes them via >>>>> HTTP. Please check here >>>>> http://172.18.160.74/osci/review/CR-202763/mos/7.0/fuel/base/centos6/Packages/fuel-library7.0-7.0.0-1.mos6888.git.bac86fe.noarch.rpm >>>>> if the package contains all necessary upstream modules. >>>>> >>>>> Vladimir Kozhukalov >>>>> >>>>> On Sat, Jul 18, 2015 at 3:28 AM, Alex Schultz <aschu...@mirantis.com> >>>>> wrote: >>>>> >>>>>> Not until we start using it then any ci that tests with that module >>>>>> will validate the modules inclusion. You can check the output of the jobs >>>>>> as we are printing what modules are managed by librarian. >>>>>> >>>>>> -Alex >>>>>> On Jul 17, 2015 6:17 PM, "Andrew Woodward" <xar...@gmail.com> wrote: >>>>>> >>>>>>> Fantastic, do we have some way to validate that the module was >>>>>>> pulled in properly as part of fuel-library CI? >>>>>>> >>>>>>> On Fri, Jul 17, 2015 at 2:48 PM Alex Schultz <aschu...@mirantis.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey All, >>>>>>>> >>>>>>>> I've figured it out without having to modify the fuel-main build >>>>>>>> code. I've updated the fuel-library spec with a build action that >>>>>>>> invokes >>>>>>>> the script to pull down external modules. Please take some time to >>>>>>>> review >>>>>>>> the two reviews out there for this change to see if there are any >>>>>>>> issues >>>>>>>> with the way it is implemented. >>>>>>>> >>>>>>>> https://review.openstack.org/#/c/202763/ >>>>>>>> https://review.openstack.org/#/c/202767/ >>>>>>>> >>>>>>>> This is a first step towards being able to pull in unmodified >>>>>>>> external puppet modules. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -Alex >>>>>>>> >>>>>>>> On Fri, Jul 17, 2015 at 4:23 PM, Andrew Woodward < >>>>>>>> awoodw...@mirantis.com> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Jul 17, 2015 at 12:24 PM Vladimir Kozhukalov < >>>>>>>>> vkozhuka...@mirantis.com> wrote: >>>>>>>>> >>>>>>>>>> Alex, >>>>>>>>>> >>>>>>>>>> Great that you did this. Now I think I can prepare fuel-main >>>>>>>>>> patch to invoke this script right before building fuel-library >>>>>>>>>> package. >>>>>>>>>> I'll add you to review it. Is it ok if I do this monday morning? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Keep in minde we agreeded to a deadline to get this sorted and in >>>>>>>>> shape to land by EOD monday or we will have to retain the old, and >>>>>>>>> crappy >>>>>>>>> fork method. If possible please work out how this needs to work as >>>>>>>>> early as >>>>>>>>> possible so Alex can continue. >>>>>>>>> >>>>>>>>> >>>>>>>>>> Vladimir Kozhukalov >>>>>>>>>> >>>>>>>>>> On Fri, Jul 17, 2015 at 5:51 PM, Alex Schultz < >>>>>>>>>> aschu...@mirantis.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hey Vladimir, >>>>>>>>>>> >>>>>>>>>>> On Fri, Jul 17, 2015 at 7:33 AM, Vladimir Kozhukalov < >>>>>>>>>>> vkozhuka...@mirantis.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Alex, >>>>>>>>>>>> >>>>>>>>>>>> Gathering upstream modules certainly should be implemented as a >>>>>>>>>>>> separate script so as to make it possible to use it wherever we >>>>>>>>>>>> need this >>>>>>>>>>>> (tests, builds, etc.) According to builds there are two things >>>>>>>>>>>> >>>>>>>>>>>> 1) We have so called "perestroika" package build system (Dmitry >>>>>>>>>>>> Burmistrov is a main contributor here). By the end of next week we >>>>>>>>>>>> are >>>>>>>>>>>> going to switch building all the packages to "perestroika". And in >>>>>>>>>>>> order to >>>>>>>>>>>> gather upstream modules right before building fuel-library >>>>>>>>>>>> package, we need >>>>>>>>>>>> to change perestroika build scripts. >>>>>>>>>>>> >>>>>>>>>>>> 2) Currently we build packages using make system and you are >>>>>>>>>>>> right about the place where you need to make changes. >>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82 >>>>>>>>>>>> If you create shell script, I'll help you to add it to make code. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I have updated my review[0] to extract the update logic to it's >>>>>>>>>>> own bash script that can be invoked by the build scripts. Let me >>>>>>>>>>> know what >>>>>>>>>>> would be the best way to wedge this in there. I think for the >>>>>>>>>>> "perestroika" this would also be needed for the fuel-library build, >>>>>>>>>>> so if >>>>>>>>>>> you point me at that I can see if I can help make that change as >>>>>>>>>>> well. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> -Alex >>>>>>>>>>> >>>>>>>>>>> [0] https://review.openstack.org/#/c/202763/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Vladimir Kozhukalov >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Jul 17, 2015 at 2:56 PM, Aleksandr Didenko < >>>>>>>>>>>> adide...@mirantis.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I believe build_repo function is the best way to do this [0]. >>>>>>>>>>>>> So for fuel-library we'll need to run a shell script right from >>>>>>>>>>>>> the repo >>>>>>>>>>>>> before 'touch $$@'. We can make it either conditional ( test -f >>>>>>>>>>>>> ./path/additional_build_script.sh && bash >>>>>>>>>>>>> ./path/additional_build_script.sh >>>>>>>>>>>>> ) or as additional parameter to function and add it in >>>>>>>>>>>>> fuel-library call [1] >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Alex >>>>>>>>>>>>> >>>>>>>>>>>>> [0] >>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L16-L37 >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L45 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Jul 17, 2015 at 2:37 PM, Alex Schultz < >>>>>>>>>>>>> aschu...@mirantis.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hey Alex, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Jul 17, 2015 4:32 AM, "Aleksandr Didenko" < >>>>>>>>>>>>>> adide...@mirantis.com> wrote: >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > Hi, >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > I think that we should provide a separate script that will >>>>>>>>>>>>>> fetch the upstream modules into fuel-library/deployment/puppet/ >>>>>>>>>>>>>> directory. >>>>>>>>>>>>>> It will allow us to have everything in a single place and use >>>>>>>>>>>>>> this script >>>>>>>>>>>>>> in ISO build process and CI jobs. >>>>>>>>>>>>>> > >>>>>>>>>>>>>> >>>>>>>>>>>>>> Right. That is what I'm going for. The issue I need help with >>>>>>>>>>>>>> is the best way to execute this as part of the build process. >>>>>>>>>>>>>> From what i >>>>>>>>>>>>>> understand of the build process is that we are using git archive >>>>>>>>>>>>>> for all >>>>>>>>>>>>>> pieces so I'm not sure how to wedge in an extra script execution >>>>>>>>>>>>>> to do the >>>>>>>>>>>>>> module fetch. The creation of the script isn't the issue, the >>>>>>>>>>>>>> issue is how >>>>>>>>>>>>>> can I properly run it as part of the build process. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> > Regards, >>>>>>>>>>>>>> > Alex >>>>>>>>>>>>>> > >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> -Alex >>>>>>>>>>>>>> >>>>>>>>>>>>>> > On Thu, Jul 16, 2015 at 11:17 PM, Alex Schultz < >>>>>>>>>>>>>> aschu...@mirantis.com> wrote: >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> Hello everyone, >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> I have committed the initial configuration required to >>>>>>>>>>>>>> start leveraging librarian-puppet as part of the way we pull in >>>>>>>>>>>>>> upstream >>>>>>>>>>>>>> puppet modules[0]. Additionally, I have also committed a change >>>>>>>>>>>>>> that would >>>>>>>>>>>>>> pull in the openstack-ironic module[1]. The one piece that is >>>>>>>>>>>>>> missing from >>>>>>>>>>>>>> this being a complete solution is the ability to run >>>>>>>>>>>>>> librarian-puppet as >>>>>>>>>>>>>> part of our build process for the fuel-library. I've looked >>>>>>>>>>>>>> into the >>>>>>>>>>>>>> fuel-main build scripts and I think it's over my head to figure >>>>>>>>>>>>>> this out >>>>>>>>>>>>>> just by looking. Can anyone explain to me or assist me in how I >>>>>>>>>>>>>> could go >>>>>>>>>>>>>> about modifying the existing build system to be able to run >>>>>>>>>>>>>> librarian-puppet to prepare the source for the package? In my >>>>>>>>>>>>>> initial >>>>>>>>>>>>>> investigation, it looks like it would be a modification of the >>>>>>>>>>>>>> fuel-main/packages/module.mk[3] file. I basically need to >>>>>>>>>>>>>> do the prepare_library[3] function from the 202763 review[0] >>>>>>>>>>>>>> after we've >>>>>>>>>>>>>> pulled all the sources together to fetch the upstream modules. >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> Thanks, >>>>>>>>>>>>>> >> -Alex >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> [0] https://review.openstack.org/202763 >>>>>>>>>>>>>> >> [1] https://review.openstack.org/202767 >>>>>>>>>>>>>> >> [2] >>>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82 >>>>>>>>>>>>>> >> [3] >>>>>>>>>>>>>> https://review.openstack.org/#/c/202763/1/utils/jenkins/fuel_noop_tests.rb >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> __________________________________________________________________________ >>>>>>>>>>>>>> >> OpenStack Development Mailing List (not for usage >>>>>>>>>>>>>> questions) >>>>>>>>>>>>>> >> Unsubscribe: >>>>>>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> __________________________________________________________________________ >>>>>>>>>>>>>> > OpenStack Development Mailing List (not for usage questions) >>>>>>>>>>>>>> > Unsubscribe: >>>>>>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>>>>>> > >>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>>>>>> > >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> __________________________________________________________________________ >>>>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>>>>>>> Unsubscribe: >>>>>>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> __________________________________________________________________________ >>>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>>>>>> Unsubscribe: >>>>>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>>>>> >>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> __________________________________________________________________________ >>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>>>>> Unsubscribe: >>>>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>>>> >>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> __________________________________________________________________________ >>>>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>>>> Unsubscribe: >>>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> __________________________________________________________________________ >>>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>>> Unsubscribe: >>>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> Andrew Woodward >>>>>>>>> Mirantis >>>>>>>>> Fuel Community Ambassador >>>>>>>>> Ceph Community >>>>>>>>> >>>>>>>>> >>>>>>>>> __________________________________________________________________________ >>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>> Unsubscribe: >>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> __________________________________________________________________________ >>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>> Unsubscribe: >>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>> >>>>>>> -- >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Andrew Woodward >>>>>>> >>>>>>> Mirantis >>>>>>> >>>>>>> Fuel Community Ambassador >>>>>>> >>>>>>> Ceph Community >>>>>>> >>>>>>> >>>>>>> __________________________________________________________________________ >>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>> Unsubscribe: >>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>> >>>>>>> >>>>>> >>>>>> __________________________________________________________________________ >>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>> Unsubscribe: >>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>> >>>>>> >>>>> >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> -- >> Mike Scherbakov >> #mihgen >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev