[OpenStack-Infra] [openstack_infra] setup openstack ci logserver questions on CentOS 7
Hi I am try to setup openstack ci in our company by follow below doc urlhttp://docs.openstack.org/infra/openstackci/index.html Seemed this doc was only tested on Ubuntu 14.04 Questions1. Is that mean CentOS not supported as I always hit error during run puppet installation on Centos? 2. If we want to enable CentOS, how to do? Or any guidance? Thanks in advance! BRs/Xu Gang ===Below is running log on CentOS [root@pmo03 puppet]# puppet apply --verbose ./manifests/site.ppInfo: Loading external facts from /etc/puppet/modules/openstacklib/facts.dInfo: Loading facts in /etc/puppet/modules/mysql/lib/facter/mysql_version.rbInfo: Loading facts in /etc/puppet/modules/mysql/lib/facter/mysql_server_id.rbInfo: Loading facts in /etc/puppet/modules/gnupg/lib/facter/gnupg_installed.rbInfo: Loading facts in /etc/puppet/modules/gnupg/lib/facter/gnupg_command.rbInfo: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_updates.rbInfo: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_update_last_succes s.rbInfo: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_reboot_required.rbInfo: Loading facts in /etc/puppet/modules/firewall/lib/facter/iptables_version. rbInfo: Loading facts in /etc/puppet/modules/firewall/lib/facter/iptables_persiste nt_version.rbInfo: Loading facts in /etc/puppet/modules/firewall/lib/facter/ip6tables_version .rbInfo: Loading facts in /etc/puppet/modules/stdlib/lib/facter/package_provider.rbInfo: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rbInfo: Loading facts in /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rbInfo: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rbInfo: Loading facts in /etc/puppet/modules/stdlib/lib/facter/pe_version.rbInfo: Loading facts in /etc/puppet/modules/stdlib/lib/facter/service_provider.rbInfo: Loading facts in /etc/puppet/modules/concat/lib/facter/concat_basedir.rbInfo: Loading facts in /etc/puppet/modules/archive/lib/facter/archive_windir.rbInfo: Loading facts in /etc/puppet/modules/selinux/lib/facter/selinux_custom_pol icy.rbInfo: Loading facts in /etc/puppet/modules/jenkins/lib/facter/dpkg.rbInfo: Loading facts in /etc/puppet/modules/jenkins/lib/facter/memorytotalbytes.r bInfo: Loading facts in /etc/puppet/modules/haproxy/lib/facter/haproxy_version.rbInfo: Loading facts in /etc/puppet/modules/python/lib/facter/python_version.rbInfo: Loading facts in /etc/puppet/modules/python/lib/facter/pip_version.rbInfo: Loading facts in /etc/puppet/modules/python/lib/facter/virtualenv_version. rbInfo: Loading facts in /etc/puppet/modules/openstacklib/lib/facter/os_service_de fault.rbInfo: Loading facts in /etc/puppet/modules/openstacklib/lib/facter/os_package_ty pe.rbInfo: Loading facts in /etc/puppet/modules/staging/lib/facter/staging_windir.rbInfo: Loading facts in /etc/puppet/modules/staging/lib/facter/staging_http_get.r bError: NetworkManager is not running.Notice: Compiled catalog for pmo03.chinac.com in environment production in 1.17 secondsWarning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/share/ruby/vendor_ruby/puppet/type.rb:816:in `set_default')Warning: Found multiple default providers for vcsrepo: bzr, cvs, svn, git, hg; using bzrInfo: Applying configuration version '1463834944'Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list build-essential' returned 1: Error: No matching Packages to listError: /Stage[main]/Openstackci::Logserver/Package[build-essential]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list build-essential' returned 1: Error: No matching Packages to listError: Cannot create /srv
Re: [OpenStack-Infra] [openstack_infra] setup openstack ci logserver questions on CentOS 7
>From looking on the code, it looks like centos/rhel should be supported[1] The error is: "Cannot create /srv/static; parent directory /srv does not exist" It assumes '/srv' exists. As far as I know it should be true for common centos/rhel instillation, but maybe this assumption isn't true for all centos installation types. Possible solution can be ensuring both '/srv' and '/srv/static' are present. [1] https://github.com/openstack-infra/system-config/blob/master/install_puppet.sh#L30 On Sat, May 21, 2016 at 4:04 PM, wrote: > Hi > > > I am try to setup openstack ci in our company by follow below doc url > > http://docs.openstack.org/infra/openstackci/index.html > > > Seemed this doc was only tested on Ubuntu 14.04 > > > Questions > > 1. Is that mean CentOS not supported as I always hit error during run puppet > installation on Centos? > > > 2. If we want to enable CentOS, how to do? Or any guidance? > > > Thanks in advance! > > > BRs/Xu Gang > > > === > > Below is running log on CentOS > > > [root@pmo03 puppet]# puppet apply --verbose ./manifests/site.pp > > Info: Loading external facts from /etc/puppet/modules/openstacklib/facts.d > > Info: Loading facts in /etc/puppet/modules/mysql/lib/facter/mysql_version.rb > > Info: Loading facts in > /etc/puppet/modules/mysql/lib/facter/mysql_server_id.rb > > Info: Loading facts in > /etc/puppet/modules/gnupg/lib/facter/gnupg_installed.rb > > Info: Loading facts in /etc/puppet/modules/gnupg/lib/facter/gnupg_command.rb > > Info: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_updates.rb > > Info: Loading facts in > /etc/puppet/modules/apt/lib/facter/apt_update_last_succes > s.rb > > Info: Loading facts in > /etc/puppet/modules/apt/lib/facter/apt_reboot_required.rb > > Info: Loading facts in > /etc/puppet/modules/firewall/lib/facter/iptables_version. > rb > > Info: Loading facts in > /etc/puppet/modules/firewall/lib/facter/iptables_persiste > nt_version.rb > > Info: Loading facts in > /etc/puppet/modules/firewall/lib/facter/ip6tables_version > .rb > > Info: Loading facts in > /etc/puppet/modules/stdlib/lib/facter/package_provider.rb > > Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb > > Info: Loading facts in > /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb > > Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rb > > Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/pe_version.rb > > Info: Loading facts in > /etc/puppet/modules/stdlib/lib/facter/service_provider.rb > > Info: Loading facts in > /etc/puppet/modules/concat/lib/facter/concat_basedir.rb > > Info: Loading facts in > /etc/puppet/modules/archive/lib/facter/archive_windir.rb > > Info: Loading facts in > /etc/puppet/modules/selinux/lib/facter/selinux_custom_pol > icy.rb > > Info: Loading facts in /etc/puppet/modules/jenkins/lib/facter/dpkg.rb > > Info: Loading facts in > /etc/puppet/modules/jenkins/lib/facter/memorytotalbytes.r > b > > Info: Loading facts in > /etc/puppet/modules/haproxy/lib/facter/haproxy_version.rb > > Info: Loading facts in > /etc/puppet/modules/python/lib/facter/python_version.rb > > Info: Loading facts in /etc/puppet/modules/python/lib/facter/pip_version.rb > > Info: Loading facts in > /etc/puppet/modules/python/lib/facter/virtualenv_version. > rb > > Info: Loading facts in > /etc/puppet/modules/openstacklib/lib/facter/os_service_de > fault.rb > > Info: Loading facts in > /etc/puppet/modules/openstacklib/lib/facter/os_package_ty > pe.rb > > Info: Loading facts in > /etc/puppet/modules/staging/lib/facter/staging_windir.rb > > Info: Loading facts in > /etc/puppet/modules/staging/lib/facter/staging_http_get.r > b > > Error: NetworkManager is not running. > > Notice: Compiled catalog for pmo03.chinac.com in environment production in > 1.17 seconds > > Warning: The package type's allow_virtual parameter will be changing its > default value from false to true in a future release. If you do not want to > allow virtual packages, please explicitly set allow_virtual to false. > >(at /usr/share/ruby/vendor_ruby/puppet/type.rb:816:in `set_default') > > Warning: Found multiple default providers for vcsrepo: bzr, cvs, svn, git, > hg; using bzr > > Info: Applying configuration version '1463834944' > > Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list build-essential' > returned 1: Error: No matching Packages to list > > Error: /Stage[main]/Openstackci::Logserver/Package[build-essential]/ensure: > change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 > -y list build-essential' returned 1: Error: No matching Packages to list > > Error: Cannot create /srv/static; parent directory /srv does not exist > > Error: /Stage[main]/Openstackci::Logserver/File[/srv/static]/ensure: change > from absent to directory failed: Cannot create /srv/static; parent directory > /srv does not exist > > Error: Execution of '/usr/bin/yum -d
Re: [OpenStack-Infra] [openstack_infra] setup openstack ci logserver questions on CentOS 7
On Sat, May 21, 2016 at 04:28:18PM +0300, Arie Bregman wrote: > From looking on the code, it looks like centos/rhel should be supported[1] > This only means our install_puppet.sh script support CentOS. We use it as an entry point for the Jenkins slaves we run in the gate. 99% of the services we run for OpenStack only support Ubuntu as the OS (minus the git servers they run CentOS). > The error is: "Cannot create /srv/static; parent directory /srv does not > exist" > > It assumes '/srv' exists. As far as I know it should be true for > common centos/rhel instillation, but maybe this assumption isn't true > for all centos installation types. > > Possible solution can be ensuring both '/srv' and '/srv/static' are present. > puppet-openstackci really is only tested using Ubuntu Trusty (eventually Ubuntu Xenial). So, you're likely better defaulting to that OS to use the puppet modules from openstack-infra. Some of the modules do support CentOS, however they are not regularly installed upstream. We do have basic testing coverage from beaker but that is not 100% based on your error below. > > [1] > https://github.com/openstack-infra/system-config/blob/master/install_puppet.sh#L30 > > On Sat, May 21, 2016 at 4:04 PM, wrote: > > Hi > > > > > > I am try to setup openstack ci in our company by follow below doc url > > > > http://docs.openstack.org/infra/openstackci/index.html > > > > > > Seemed this doc was only tested on Ubuntu 14.04 > > > > > > Questions > > > > 1. Is that mean CentOS not supported as I always hit error during run puppet > > installation on Centos? > > > > > > 2. If we want to enable CentOS, how to do? Or any guidance? > > > > > > Thanks in advance! > > > > > > BRs/Xu Gang > > > > > > === > > > > Below is running log on CentOS > > > > > > [root@pmo03 puppet]# puppet apply --verbose ./manifests/site.pp > > > > Info: Loading external facts from /etc/puppet/modules/openstacklib/facts.d > > > > Info: Loading facts in /etc/puppet/modules/mysql/lib/facter/mysql_version.rb > > > > Info: Loading facts in > > /etc/puppet/modules/mysql/lib/facter/mysql_server_id.rb > > > > Info: Loading facts in > > /etc/puppet/modules/gnupg/lib/facter/gnupg_installed.rb > > > > Info: Loading facts in /etc/puppet/modules/gnupg/lib/facter/gnupg_command.rb > > > > Info: Loading facts in /etc/puppet/modules/apt/lib/facter/apt_updates.rb > > > > Info: Loading facts in > > /etc/puppet/modules/apt/lib/facter/apt_update_last_succes > > s.rb > > > > Info: Loading facts in > > /etc/puppet/modules/apt/lib/facter/apt_reboot_required.rb > > > > Info: Loading facts in > > /etc/puppet/modules/firewall/lib/facter/iptables_version. > > rb > > > > Info: Loading facts in > > /etc/puppet/modules/firewall/lib/facter/iptables_persiste > > nt_version.rb > > > > Info: Loading facts in > > /etc/puppet/modules/firewall/lib/facter/ip6tables_version > > .rb > > > > Info: Loading facts in > > /etc/puppet/modules/stdlib/lib/facter/package_provider.rb > > > > Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb > > > > Info: Loading facts in > > /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb > > > > Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rb > > > > Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/pe_version.rb > > > > Info: Loading facts in > > /etc/puppet/modules/stdlib/lib/facter/service_provider.rb > > > > Info: Loading facts in > > /etc/puppet/modules/concat/lib/facter/concat_basedir.rb > > > > Info: Loading facts in > > /etc/puppet/modules/archive/lib/facter/archive_windir.rb > > > > Info: Loading facts in > > /etc/puppet/modules/selinux/lib/facter/selinux_custom_pol > > icy.rb > > > > Info: Loading facts in /etc/puppet/modules/jenkins/lib/facter/dpkg.rb > > > > Info: Loading facts in > > /etc/puppet/modules/jenkins/lib/facter/memorytotalbytes.r > > b > > > > Info: Loading facts in > > /etc/puppet/modules/haproxy/lib/facter/haproxy_version.rb > > > > Info: Loading facts in > > /etc/puppet/modules/python/lib/facter/python_version.rb > > > > Info: Loading facts in /etc/puppet/modules/python/lib/facter/pip_version.rb > > > > Info: Loading facts in > > /etc/puppet/modules/python/lib/facter/virtualenv_version. > > rb > > > > Info: Loading facts in > > /etc/puppet/modules/openstacklib/lib/facter/os_service_de > > fault.rb > > > > Info: Loading facts in > > /etc/puppet/modules/openstacklib/lib/facter/os_package_ty > > pe.rb > > > > Info: Loading facts in > > /etc/puppet/modules/staging/lib/facter/staging_windir.rb > > > > Info: Loading facts in > > /etc/puppet/modules/staging/lib/facter/staging_http_get.r > > b > > > > Error: NetworkManager is not running. > > > > Notice: Compiled catalog for pmo03.chinac.com in environment production in > > 1.17 seconds > > > > Warning: The package type's allow_virtual parameter will be changing its > > default value from false to true in a future releas
Re: [OpenStack-Infra] An idea to help browse and troubleshoot Ansible Playbook runs
Hi again openstack-infra, We've worked quite a bit on ARA since I first told you guys about it here. We're pretty much out of PoC and Alpha territory by now and I've made a formal public announcement about it [1]. Following up on the questions from the thread now that we've progressed a bit: - Usability with things like ElasticSearch: The ARA CLI client uses Cliff and thus has a very similar look and feel as python-openstackclient. Cliff provides several output formats out of the box such as YAML or JSON. You could use the client to output the information you want in JSON format and then send that off to ElasticSearch with already-indexed fields. - Static file generation This is very high on the list of things to do. We will make it possible for the ARA CLI client to generate a static version of the interface for a single playbook run. So far we're aiming more towards a html "tree" rather than just a single page to keep the same browsing experience. I see a great opportunity for ARA to be used by the OpenStack community. The project officially needs a home for code reviews and I think openstack or openstack-infra with the usual gerrit workflow would be a great fit if you believe it makes sense. What do you think ? Please let me know ! Thanks, [1]: https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/ David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Tue, May 10, 2016 at 9:24 PM, David Moreau Simard wrote: > Hi openstack-infra, > > I've been hacking on an idea since last friday and I was thinking it > could be of interest for you guys as Ansible users, but also > considering the next generation of Zuul jobs will be powered by > Ansible. > > The project I'm working on is called ARA [1]. It is very much an alpha > at this point but it is hopefully fleshed out enough at this point for > you to see where I am going with this. > I could go on about what the project looks like but I've actually > built a sane amount of docs [2] and even put up a video of what the > interface looks like on YouTube [3]. > > ARA was born out of necessity because the vast majority of RDO CI [4] > is driven by Ansible, either through projects like TripleO-Quickstart > [5] or WeIRDO [6]. > > We are equipped in a fairly similar way than upstream, we have Jenkins > console logs that are large enough to crash your browser and logs that > are collected and made browsable like on logs.o.o. > In a nutshell, we're running CI jobs that install and test OpenStack > deployments through various installers: TripleO, Packstack, > Puppet-OpenStack, and soon: Kolla and Chef-OpenStack. > > The sad reality is that the RDO community has limited resources and > ARA aims at making it easier and faster to see what failed, where and > why to make everyone more efficient. > Another sad truth is that I am much more a system administrator than a > developer and as such I am not exactly awesome in either Python or > UI/UX frontends. > > As such, if you are interested in the project, I would certainly > welcome not only feedback but contributions if you feel you could use > this. > > Let me know if you have any questions ! > > [1]: https://github.com/dmsimard/ara > [2]: http://ara.readthedocs.io/en/latest/ > [3]: https://www.youtube.com/watch?v=K3jTqgm2YuY > [4]: https://ci.centos.org/view/rdo/view/promotion-pipeline/ > [5]: https://github.com/openstack/tripleo-quickstart > [6]: https://github.com/redhat-openstack/weirdo > > David Moreau Simard > Senior Software Engineer | Openstack RDO > > dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] jjb-2.0.0-api rebase onto master & next steps
Howdy folks! About a year ago I announced my intent to put serious effort into improving JJB's Python API: http://lists.openstack.org/pipermail/openstack-infra/2015-May/002761.html The currently-approved spec can be seen here: https://specs.openstack.org/openstack-infra/infra-specs/specs/jenkins-job-builder_2.0.0-api-changes.html Related changes can be seen using the "jjb-2.0.0-api" Gerrit topic: https://review.openstack.org/#/q/topic:jjb-2.0.0-api Last month after lots of hard work by the core JJB team the final "1.x" version (1.5.0) of JJB was released. Shoutout in particular to Darragh Bailey, who took on the most difficult role of keeping track of which patches were highest priority for 1.5.0, coordinating the necessary reviews, and managing the release--thanks! Now that JJB 1.5.0 is released, I have rebased the 19 commits in the jjb-2.0.0-api topic onto master branch and am about to push those up to gerrit for review. But first I wanted to send this email with next steps and some notes about my approach in refactoring the API. Notes about the refactor approach: * While there are a few necessarily large commits, most commits are small and single-purpose. The small single-purpose commits serve two purposes--they made it easier to incrementally change and test the API (I was reading a book called The Mikado Method at the time, it's definitely worth checking out); they should also make it easier to understand and review. * Because much of the work was split across 19 commits, I am anticipating there may be feedback on the earlier commits that is addressed in later commits--rather than shuffing changes around between commits or squashing the whole line into one uber-commit (+2170,-1411), I am hoping it will suffice for me to point out where this is the case to satisfy reviewers. Next steps: * Remove the feature/2.0.x branch from the jenkins-job-builder repo. * Review the patches! * Merge them when they're ready. * Write some API unit tests and documentation. * Release 2.0.0! Happy reviewing! ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] jjb-2.0.0-api rebase onto master & next steps
On Sat, May 21, 2016 at 7:18 PM, Wayne Warren wrote: > Howdy folks! > > About a year ago I announced my intent to put serious effort into > improving JJB's Python API: > http://lists.openstack.org/pipermail/openstack-infra/2015-May/002761.html > > The currently-approved spec can be seen here: > https://specs.openstack.org/openstack-infra/infra-specs/specs/jenkins-job-builder_2.0.0-api-changes.html > > Related changes can be seen using the "jjb-2.0.0-api" Gerrit topic: > https://review.openstack.org/#/q/topic:jjb-2.0.0-api Probably everyone on this list is more gerrit-savvy than myself but here is a better search link anyway (excludes all the abandoned feature/2.0.0 branch changes): https://review.openstack.org/#/q/project:openstack-infra/jenkins-job-builder+topic:jjb-2.0.0-api+status:open > > Last month after lots of hard work by the core JJB team the final > "1.x" version (1.5.0) of JJB was released. Shoutout in particular to > Darragh Bailey, who took on the most difficult role of keeping track > of which patches were highest priority for 1.5.0, coordinating the > necessary reviews, and managing the release--thanks! > > Now that JJB 1.5.0 is released, I have rebased the 19 commits in the > jjb-2.0.0-api topic onto master branch and am about to push those up > to gerrit for review. But first I wanted to send this email with next > steps and some notes about my approach in refactoring the API. > > Notes about the refactor approach: > * While there are a few necessarily large commits, most commits are > small and single-purpose. The small single-purpose commits serve two > purposes--they made it easier to incrementally change and test the API > (I was reading a book called The Mikado Method at the time, it's > definitely worth checking out); they should also make it easier to > understand and review. > * Because much of the work was split across 19 commits, I am > anticipating there may be feedback on the earlier commits that is > addressed in later commits--rather than shuffing changes around > between commits or squashing the whole line into one uber-commit > (+2170,-1411), I am hoping it will suffice for me to point out where > this is the case to satisfy reviewers. > > Next steps: > * Remove the feature/2.0.x branch from the jenkins-job-builder repo. > * Review the patches! > * Merge them when they're ready. > * Write some API unit tests and documentation. > * Release 2.0.0! > > Happy reviewing! ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
