Re: [OpenStack-Infra] How to display the result of tests on the upper right of a gerrit page

2018-08-22 Thread Mikhail Medvedev 
On Wed, Aug 22, 2018 at 12:50 AM, Rikimaru Honjo
 wrote:
> Hi Mikhail,
>
> Thank you for replying!
>
> On 2018/08/22 2:20, Mikhail Medvedev wrote:
>>
>> On Sat, Aug 18, 2018 at 2:06 AM, Rikimaru Honjo
>>  wrote:
>>>
>>>
>>> Hello,
>>>
>>> I have a question about third-party CI.
>>>
>>> I'm preparing my third-party CI. So I read the following document.
>>>
>>> https://docs.openstack.org/infra/system-config/third_party.html
>>>
>>> But, I couldn't understand how to display the result of tests in the
>>> upper right of a gerrit page.
>>
>>
>> Hi,
>>
>> The table on the gerrit page is created by the script:
>>
>> https://github.com/openstack-infra/system-config/blob/d148f5f8efe9b66329600b445e9a6f4f46223896/modules/openstack_project/files/gerrit/hideci.js
>> . It parses all the comments posted on a patch to generate the list.
>> For your third-party CI to be listed on that table it should post a
>> comment adhering to a certain format. Press the "Toggle CI" to see
>> other CI comments for examples of formatting.
>
> I got it.
> I confirm the comment format of my CI.
>
>>
>>>
>>> e.g.
>>> I'd like to display the result of tests like IBM PowerKVM CI,INFINIDAT
>>> CI...
>>> of this page:
>>> https://review.openstack.org/#/c/593202/1
>>>
>>> Should I add my CI account to -ci group?
>>
>>
>> For your CI to be able to post a comment and to be added to the table
>> it does not have to be added to -ci group. You only need to
>> add your CI to -ci group to vote +/-1 Verified on the
>> project's changes. The Verified vote would show up at the top, e.g.
>> "Verified +1 Zuul".
>>
>> What is the name of the third-party CI you are setting up? I can add
>> it to the email filter list
>> https://review.openstack.org/#/admin/groups/270
>
> My third-party CI is "Networking-spp Integration CI".
> This is the wiki page about it.
> https://wiki.openstack.org/wiki/ThirdPartySystems/Networking-spp_Integration_CI
>

Thanks Rikimaru, I've added networking-spp-integration-ci
 to the mail filter list.

>
>>>
>>> Best regards,
>>> --
>>> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
>>> Rikimaru Honjo
>>> E-mail:honjo.rikim...@po.ntt-tx.co.jp
>>>
>>>
>>> ___
>>> OpenStack-Infra mailing list
>>> OpenStack-Infra@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>
>>
>> --
>> Mikhail Medvedev
>> IBM
>>
>
> --
> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
> Rikimaru Honjo
> E-mail:honjo.rikim...@po.ntt-tx.co.jp
>
>
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] How to display the result of tests on the upper right of a gerrit page

2018-08-21 Thread Mikhail Medvedev 
On Sat, Aug 18, 2018 at 2:06 AM, Rikimaru Honjo
 wrote:
>
> Hello,
>
> I have a question about third-party CI.
>
> I'm preparing my third-party CI. So I read the following document.
>
> https://docs.openstack.org/infra/system-config/third_party.html
>
> But, I couldn't understand how to display the result of tests in the
> upper right of a gerrit page.

Hi,

The table on the gerrit page is created by the script:
https://github.com/openstack-infra/system-config/blob/d148f5f8efe9b66329600b445e9a6f4f46223896/modules/openstack_project/files/gerrit/hideci.js
. It parses all the comments posted on a patch to generate the list.
For your third-party CI to be listed on that table it should post a
comment adhering to a certain format. Press the "Toggle CI" to see
other CI comments for examples of formatting.


>
> e.g.
> I'd like to display the result of tests like IBM PowerKVM CI,INFINIDAT CI...
> of this page:
> https://review.openstack.org/#/c/593202/1
>
> Should I add my CI account to -ci group?

For your CI to be able to post a comment and to be added to the table
it does not have to be added to -ci group. You only need to
add your CI to -ci group to vote +/-1 Verified on the
project's changes. The Verified vote would show up at the top, e.g.
"Verified +1 Zuul".

What is the name of the third-party CI you are setting up? I can add
it to the email filter list
https://review.openstack.org/#/admin/groups/270

>
> Best regards,
> --
> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
> Rikimaru Honjo
> E-mail:honjo.rikim...@po.ntt-tx.co.jp
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

--
Mikhail Medvedev
IBM

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Merging Zuul v3 into master

2017-11-28 Thread Mikhail Medvedev 
On Mon, Nov 27, 2017 at 5:31 PM, James E. Blair <cor...@inaugust.com> wrote:
> ...
>
> * If you are able to contribute to making the necessary changes to
>   puppet-openstackci, please reply here to volunteer to do so.
> ...

I am working on the puppet-openstackci update. Any other third-party
CI operators using the module are welcome to chime in.

Mikhail Medvedev (mmedvede)

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra] [jenkins-job-builder] run `jenkins-jobs test` failed

2017-05-04 Thread Mikhail Medvedev 
On Thu, May 4, 2017 at 4:56 AM,  <dong.wenj...@zte.com.cn> wrote:
>
> Hi folks,
>
> I use "puppet-jenkins" to setuped a jenkins node, and then run the command
> `jenkins-jobs test /etc/jenkins_jobs/config/`
>
> to parser the jobs which are from the "project-config/jenkins/jobs".
>
> But it raised a error, here is the log:
>
> Does anyone know how to resolve the issue? Do I need to install some tools?
>
> Thanks for help~
>
>
> INFO:jenkins_jobs.local_yaml:Including file 'include/run-project-guide.sh'
> from path '.'
>
> WARNING:root:logrotate is deprecated on jenkins＞=1.637, use the property
> build-discarder on newer jenkins instead
>
> Traceback (most recent call last):
>
>   File "/usr/local/bin/jenkins-jobs", line 10, in ＜module＞
>
> sys.exit(main())
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/cmd.py", line
> 191, in main
>
> execute(options, config)
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/cmd.py", line
> 380, in execute
>
> n_workers=1)
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/builder.py",
> line 350, in update_jobs
>
> self.parser.generateXML()
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/parser.py", line
> 342, in generateXML
>
> self.xml_jobs.append(self.getXMLForJob(job))
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/parser.py", line
> 352, in getXMLForJob
>
> self.gen_xml(xml, data)
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/parser.py", line
> 359, in gen_xml
>
> module.gen_xml(self, xml, data)
>
>   File
> "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/modules/publishers.py",
> line 6158, in gen_xml
>
> self.registry.dispatch('publisher', parser, publishers, action)
>
>   File "/usr/local/lib/python2.7/dist-packages/jenkins_jobs/registry.py",
> line 249, in dispatch
>
> format(name, component_type))
>
> jenkins_jobs.errors.JenkinsJobsException: Unknown entry point or macro 'afs'
> for component type: 'publisher'.
>

The problem is exactly what it says it is - jjb can not find a
publisher macro named 'afs'. If you look into project-config/tox.ini
's testenv:jjb section, you'll see that it does 'pip install -U
jenkins/modules/jjb_afs', which I assume installs the missing macro.
So you can either install that missing macro, or you can remove all
uses of the macro from you configuration.

A more important question is why you want to use
project-config/jenkins jobs as is for your deployment of Jenknis? It
is a configuration specific to OpenStack Infra. You would be better
off starting with a small subset of jobs you are interested in using.

---
Mikhail Medvedev (mmedvede)Now there is another question
IBM

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra] how to config gerrit to get a summary of all jobs in gerrit page

2016-12-21 Thread Mikhail Medvedev 
On Wed, Dec 21, 2016 at 1:49 AM, <dong.wenj...@zte.com.cn> wrote:
>
>
> Hi all,
>
> I setup a CI system, but after all the jobs run, the summary of all jobs
don't show in gerrit page.
> Like this:
>
>
> Does anybody know how to config it? Thanks~
>

OpenStack infra uses custom js script to create that table.

If you're asking how to configure your own gerrit instance to format CI
results like a table: you can use something similar to hideci.js script [1]
in your gerrit instance UI. To see how hideci.js is setup on OpenStack
gerrit instance, see puppet configuration at [2].

If you're asking how to get your third-party CI to show up on OpenStack
gerrit: your CI's comment needs to adhere to what hideci.js expects when it
parses CI comments.


[1]
https://github.com/openstack-infra/system-config/blob/dcacf61853fb5e4e2e2f15df8ce1fbab79a8494e/modules/openstack_project/files/gerrit/hideci.js
[2]
https://github.com/openstack-infra/system-config/blob/dcacf61853fb5e4e2e2f15df8ce1fbab79a8494e/modules/openstack_project/manifests/gerrit.pp

>
>
> BR,
> dwj
>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

--
Mikhail Medvedev
IBM
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] About setting up CI

2016-11-18 Thread Mikhail Medvedev 
There is also puppet-openstackci module that aims to provide most of
configuration necessary to setup an OpenStack CI. See the
documentation at
https://github.com/openstack-infra/puppet-openstackci/blob/master/doc/source/third_party_ci.rst

---
Mikhail Medvedev
IBM OpenStack CI for KVM on Power

On Fri, Nov 18, 2016 at 3:37 AM, Joshua Hesketh
<joshua.hesk...@gmail.com> wrote:
> Hey,
>
> Yes, that blog post is likely outdated.
>
> The official documentation is published here:
> http://docs.openstack.org/infra/system-config/
> It includes a tutorial on running your own CI but admits it may also be out
> of date. I haven't checked in detail what parts are missing, but it's likely
> closer to what you are after. Nevertheless reading through that and the
> system-config and related repos would be a good place to start.
>
> Cheers,
> Josh
>
> On Fri, Nov 18, 2016 at 4:06 PM, Wang Shilong <ddn_openst...@163.com> wrote:
>>
>> Hello folks,
>>
>>We are trying to setting up CI testing system for new Lustre cinder
>> Drivers.
>> I try to do as following URL In centos7:
>>
>>
>> http://www.joinfu.com/2014/02/setting-up-an-external-openstack-testing-system/
>>
>> But hit the following problem:
>>
>> [root@vm7-1 ~]# ./install_master.sh
>> Using upstream Gerrit user: ddn_openstack
>> Using Jenkins SSH key path: /root/data/jenkins_key
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Info: Loading facts
>> Error: Could not find template 'openstack_project/puppet.conf.erb' at
>> /root/os-ext-testing/puppet/modules/os_ext_testing/manifests/base.pp:132 on
>> node vm7-1
>> Error: Could not find template 'openstack_project/puppet.conf.erb' at
>> /root/os-ext-testing/puppet/modules/os_ext_testing/manifests/base.pp:132 on
>> node vm7-1
>>
>> I guess this Doc might be out of date, could you please give me some hints
>> how
>> I can setup latest CI?
>>
>> Thanks for your suggestion!
>> Shilong
>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [Zuul] can not get zuul env variables, such as 'ZUUL_URL'

2016-09-05 Thread Mikhail Medvedev 
Hi Li,

On Sun, Sep 4, 2016 at 10:18 AM,  <li.yuanz...@zte.com.cn> wrote:
> Hi all,
> For deploy CI env, I need a help for Zuul.
>
> In the slave node, I can not get the "zuul"
> environments variables using 'env' shell command,
> such as 'ZUUL_URL'.
>
> The following is the config and log info
> *) /etc/zuul/layout/zuul.conf:
> http://paste.ubuntu.com/23132730/
> *) /etc/jenkins_jobs/config/citest.yaml
> http://paste.ubuntu.com/23132723/
> *) jenkins job result:
> http://paste.ubuntu.com/23132737/
> *) /var/log/zuul/zuul.log
> http://paste.ubuntu.com/23132745/
> *) /var/log/zuul/merger-debug.log
> http://paste.ubuntu.com/23132748/
> *) /var/log/zuul/debug.log
> http://paste.ubuntu.com/23132762/
>
>I check the zuul code and it has been got the
> 'zuul_url' variables in zuul-mergerr, but in slave nodes,
>  thesevariables can not be passed by zuul-scheduler.
>
> Is there any wrong in my zuul config or jenkins jobs?

You might be affected by the fix to SECURITY-170 [1] in new Jenkins
you installed. It prevents Zuul from passing paramters that are not
white listed. Which version of Jenkins are you using? Last version
without the security fix is 1.651.1.

If you have later version, you would need to either white list all
zuul variables (
-Dhudson.model.ParametersAction.safeParameters=ZUUL_URL,ZUUL_COMMIT ),
or disable that security feature (
-Dhudson.model.ParametersAction.keepUndefinedParameters=true ).

[1] 
https://wiki.jenkins-ci.org/display/JENKINS/Plugins+affected+by+fix+for+SECURITY-170

>
> Thank you
>
> BRs
> liyuanzhen
>
>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>

Mikhail Medvedev (mmedvede)
IBM

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Request to disable IBM-GPFS-CINDER-CI account temparary

2016-03-21 Thread Mikhail Medvedev 
Digvijay,

Thanks for the follow up on this. I am not able to disable review.o.o
accounts, as I am not an infra root. But that might not be necessary.
Can you login on review.openstack.org using IBM GPFS CI account
(rsand...@in.ibm.com)? If so, try removing existing ssh public keys
associated with the account. I think that would prevent other zuul
server from using the account in the future to post comments.

Also, please attend one of third-party CI meeting
(https://wiki.openstack.org/wiki/Meetings/ThirdParty). It is a good
place to get your questions answered quickly. They are held Mondays at
1500 UTC and Tuesdays at 0800 UTC.

Anita - would you able to temporary disable the IBM GPFS CI
(rsand...@in.ibm.com) in case flushing account ssh keys does not work?

Thanks,

Mikhail Medvedev

On Mon, Mar 21, 2016 at 5:52 AM, Digvijay B Ukirde
<digvijay.uki...@in.ibm.com> wrote:
> Hi Mikhail,
>
> Currently IBM-GPFS-CINDER-CI is facing some internal issue because of which
> it is resulting negative report in CI dashboard.
> We are trying to disable our cinder-ci but some other server(older ci setup)
> is taking patches because of which we are unable to disable the CI or may be
> Gerrit cache is the issue (We are not sure about it)
>
> Since,
> We noticed that there are two recheck keywords for IBM GPFS CI in Cinder.
> One is recheck-gpfs and the result always shows NOT_REGISTERED.
> One is recheck ibmgpfs and the result is always good. [ this is in our ci
> setup - layout.yaml ]
>
>
> So till we fix the issue can you please disable the IBM-GPFS-CINDER-CI
> account.
>
> Once we fix the issue we will get back to you.
>
>
> Regards,
> Digvijay
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [Third-party CI] - Running CI setup failing due to NOT_REGISTERED issue

2016-03-19 Thread Mikhail Medvedev 
On Wed, Mar 16, 2016 at 12:01 AM, Digvijay B Ukirde
<digvijay.uki...@in.ibm.com> wrote:
>
> Hi Mikhail,
>
> > Search your zuul logs for NOT_REGISTERED for clues
> although there are not many statement with NOT_REGISTERED in it..
> pasting relevant part : http://paste.openstack.org/show/490634/

Is there anything interesting in the log before the error?

>
>
> > I have noticed that you are voting, and reporting on cinder.
> I was not aware of these restrictions & was trying to check whether I am able 
> to trigger patch by comment.
> I will take precaution now onwards.

Make sure you read and follow guidelines for third-party CIs at
http://docs.openstack.org/infra/system-config/third_party.html#requirements

>
>
> Thanks,
> Digvijay
> Mikhail Medvedev ---03/16/2016 03:43:36 AM---Digvijay, I can not see anything 
> obviously wrong with your configuration yet to cause
>
> From: Mikhail Medvedev <mihail...@gmail.com>
> To: Digvijay B Ukirde/India/IBM@IBMIN
> Cc: "openstack-infra@lists.openstack.org" 
> <openstack-infra@lists.openstack.org>
> Date: 03/16/2016 03:43 AM
>
>
> Subject: Re: [OpenStack-Infra] [Third-party CI] - Running CI setup failing 
> due to NOT_REGISTERED issue
> 
>
>
>
> Digvijay,
>
> I can not see anything obviously wrong with your configuration yet to cause 
> NOT_REGISTERED. Your check pipeline does use dsvm-tempest-full job, and there 
> is a build with that name registered on gearman. Search your zuul logs for 
> NOT_REGISTERED for clues, and post relevant parts to 
> http://paste.openstack.org/.
>
> But I have noticed that you are voting, and reporting on cinder. Did you ask 
> cinder devs if it is ok to start posting comments? Otherwise, please disable 
> reports until your CI becomes reasonably stable. Use openstack-dev/ci-sandbox 
> project to test your CI. You should remove/comment out the following from 
> check pipeline in zuul layout:
>
> success:
>   gerrit: {}
> failure:
>   gerrit: {}
> merge-failure:
>   gerrit: {}
>
> ---
> Mikhail
>
> On Tue, Mar 15, 2016 at 2:00 AM, Digvijay B Ukirde 
> <digvijay.uki...@in.ibm.com> wrote:
>
> Hi Mikhail,
> Thanks for quick response.
>
>
> > It would be useful if you could paste your zuul pipeline
>
> > configuration. It shows up in zuul.log when you restart zuul-serve
>
> 2016-03-15 12:14:00,482 INFO zuul.Gerrit: Updating information for 292730,1
> 2016-03-15 12:14:11,903 INFO zuul.Gerrit: Updating information for 282392,10
> 2016-03-15 12:14:21,520 INFO zuul.Server: Starting scheduler
> 2016-03-15 12:14:21,543 INFO zuul.IndependentPipelineManager: Configured 
> Pipeline Manager check
> 2016-03-15 12:14:21,543 INFO zuul.IndependentPipelineManager: Source: 
> 
> 2016-03-15 12:14:21,543 INFO zuul.IndependentPipelineManager: Requirements:
> 2016-03-15 12:14:21,543 INFO zuul.IndependentPipelineManager: Events:
> 2016-03-15 12:14:21,543 INFO zuul.IndependentPipelineManager:  types: patchset-created>
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager:  types: change-restored>
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager:  types: comment-added comments: (?i)^(Patch Set [0-9]+:)?( 
> [\w\\+-]*)*(\n\n)?\s*recheck(( (?:bug|lp)[\s#:]*(\d+))|( no bug))\s*gt;
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager:  types: comment-added comments: (?i)^(Patch Set [0-9]+:)?( 
> [\w\\+-]*)*(\n\n)?\s*recheck(( (?:bug|lp)[\s#:]*(\d+))|( ibmgpfs))\s*gt;
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager: Projects:
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager: openstack/cinder
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager:  dsvm-tempest-full>
> 2016-03-15 12:14:21,544 INFO zuul.IndependentPipelineManager: On start:
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: []
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: On success:
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: , {'verified': 
> 1}>
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: On failure:
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: , {'verified': 
> -1}>
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: On 
> merge-failure:
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: , {'verified': 
> -1}>
> 2016-03-15 12:14:21,545 INFO zuul.IndependentPipelineManager: Configured 
> Pipeline Manager silent
> 2016-03-15 12:14:21,546 INFO zuul.IndependentPipelineManager: Source: 
> 
> 2016-03-15 12:14:21,546 INFO zuul.IndependentPipelineManager: Requirements:
> 2016-03-15 12:14:21,546 IN

Re: [OpenStack-Infra] [Third-party CI] - New third party CI setup, Gerrit patches not getting displayed in Zuul UI and jenkins job not running.

2016-03-15 Thread Mikhail Medvedev 
On Tue, Mar 15, 2016 at 6:45 AM, Brijnandan <brijnan...@cloudbyte.com> wrote:
> Thanks Mikhail, with your suggestion I was able to proceed.
>
> When I logged in as zuul user and executed commands,
> $ sudo su zuul
> $ git clone ...
> It said no permissions.
>
> So I had again created a ssh-key pair with zuul user and which I had placed 
> in puppets common.yaml file and gerrit account.
> After running puppet when I logged in with  sudo su zuul I was able to clone 
> manually the repo to /var/lib/zuul/git/openstack/cinder .
>
> When I clone the repo manually to path it was successful.
> But when the new patches comes it again says t "Please make sure you have the 
> correct access rights "(log attached below).
> Merger and merger-debug logs: http://paste.openstack.org/show/490483/

"Host key verification failed" tells me that you need to add gerrit
host key properly. If you have used
https://github.com/openstack-infra/puppet-openstackci modules, it
should have been setup for you. Or you can consult that module to see
how the key is supposed to be added. In theory, if you try to clone or
update repository from gerrit manually (under zuul user), it should
ask you if you want to add the gerrit host key. And once you do this,
merger should work.

It has been awhile since I had to reinstall zuul from scratch. So
someone with more recent experience please pitch in.

>
> I had tried creating the ssh-keys with root and zuul user, It works manually  
> but not automatically via zuul as the new patch comes.
>
> Thanks,
> Brij
>
> -Original Message-
> From: Mikhail Medvedev [mailto:mihail...@gmail.com]
> Sent: 14 March 2016 20:04
> To: Brijnandan
> Cc: openstack-infra@lists.openstack.org; openstack-...@cloudbyte.com
> Subject: Re: [OpenStack-Infra] [Third-party CI] - New third party CI setup, 
> Gerrit patches not getting displayed in Zuul UI and jenkins job not running.
>
> Brijnandan,
>
> On Mon, Mar 14, 2016 at 6:12 AM, Brijnandan <brijnan...@cloudbyte.com> wrote:
>> Hi Team,
>>
>>
>> I am in a process of setting up new third party CI. I am running zuul,
>> zuul merger and Jenkins in one Ubuntu  VERSION=14.04.4 LTS, Trusty
>> Tahr
>>
>> I have configured zuul and zuul merger, when zuul starts listening to
>> patches they are not displayed in the zuul UI. However I can see them
>> adding in  zuul.log as below.
>>
>> http://paste.openstack.org/show/490364/
>>
>>
>>
>> Test connection for Jenkins gearman plugin also runs successfully.
>>
>> I  have specified test job (noop-check-communication) under silent
>> queue in nodepool.yaml but no jobs are getting triggered.
>>
>>
>>
>> debug.log:http://paste.openstack.org/show/490367/
>>
>> merger.log:  http://paste.openstack.org/show/490368/
>>
>> merger-debug.log: http://paste.openstack.org/show/490369/
>>
>
> You merger logs show an error, so you need to investigate that further. To 
> start, try manually cloning the repository you error out
> on:
>
> sudo su zuul
> git clone ssh://cloudbyt...@review.openstack.org:29418/openstack/cinder
> /var/lib/zuul/git/openstack/cinder  # taken that from log
>
> If that does not work, try figuring out why. Maybe zuul-merger fails for that 
> same reason.
>
>>
>>
>> I have added the public key of the root user to gerrit account and
>> puppet common.yaml and tried cloning the repo manually, it works fine.
>>
>> Please suggest how I should proceed further. Let me know if any more
>> inputs are required. Thanks in Advance.
>>
>>
>>
>> Regards,
>>
>> Brij
>>
>
> Mikhail Medvedev,
> IBM
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [Third-party CI] - Running CI setup failing due to NOT_REGISTERED issue

2016-03-15 Thread Mikhail Medvedev 
ndentPipelineManager:  types: comment-added comments: (?i)^(Patch Set [0-9]+:)?(
> [\w\\+-]*)*(\n\n)?\s*check(( (?:bug|lp)[\s#:]*(\d+))|( patch))\s*$>
> 2016-03-15 12:14:21,548 INFO zuul.IndependentPipelineManager: Projects:
> 2016-03-15 12:14:21,548 INFO zuul.IndependentPipelineManager: On start:
> 2016-03-15 12:14:21,548 INFO zuul.IndependentPipelineManager: []
> 2016-03-15 12:14:21,548 INFO zuul.IndependentPipelineManager: On success:
> 2016-03-15 12:14:21,548 INFO zuul.IndependentPipelineManager: []
> 2016-03-15 12:14:21,549 INFO zuul.IndependentPipelineManager: On failure:
> 2016-03-15 12:14:21,549 INFO zuul.IndependentPipelineManager: []
> 2016-03-15 12:14:21,549 INFO zuul.IndependentPipelineManager: On
> merge-failure:
> 2016-03-15 12:14:21,549 INFO zuul.IndependentPipelineManager: []
> 2016-03-15 12:14:21,550 INFO zuul.Server: Starting Webapp
> 2016-03-15 12:14:21,552 INFO zuul.Server: Starting RPC
> 2016-03-15 12:14:29,358 INFO zuul.Gerrit: Updating information for 264157,5
> 2016-03-15 12:14:45,383 INFO zuul.Gerrit: Updating information for 292673,2
> 2016-03-15 12:14:46,189 INFO zuul.Gerrit: Updating information for 292213,1
> 2016-03-15 12:14:48,244 INFO zuul.Gerrit: Updating information for 266803,6
> 2016-03-15 12:14:50,327 ERROR zuul.GerritEventConnector: Received
> unrecognized event type 'ref-replicated' from Gerrit. Can not get account
> information.
> 2016-03-15 12:14:50,330 ERROR zuul.GerritEventConnector: Received
> unrecognized event type 'ref-replicated' from Gerrit. Can not get account
> information.
> 2016-03-15 12:14:50,330 ERROR zuul.GerritEventConnector: Received
> unrecognized event type 'ref-replicated' from Gerrit. Can not get account
> information.
> 2016-03-15 12:14:50,330 ERROR zuul.GerritEventConnector: Received
> unrecognized event type 'ref-replicated' from Gerrit. Can not get account
> information.
>
>
> > NOT_REGISTERED means that there is mismatch between the job zuul wants
> > to start, and what jenkins registered (if it did) with zuul gearman
> > server. Check what got registered on gearman server with:
> > echo status | nc  4730  # assuming you use standard
> > port, and gearman is on zuul
>
> root@master:~# echo status | nc  4730
> build:dsvm-tempest-full 2   0   0
> zuul:enqueue 0   0   1
> zuul:promote 0   0   1
> merger:update   0   0   1
> stop:master.in.ibm.com   0   0   1
> build:dsvm-tempest-full:devstack_slave   0   0   0
> zuul:get_running_jobs   0   0   1
> build:noop-check-communication   0   0   2
> build:noop-check-communication:master   0   0   2
> set_description:master.in.ibm.com   0   0   1
> merger:merge 0   0   1
> .
>
>
> What is wrong with the configuration?
>
>
>
> Thanks,
> Digivjay
> [image: Inactive hide details for Mikhail Medvedev ---03/14/2016 08:12:59
> PM---Hi Gigvijay, On Mon, Mar 14, 2016 at 7:00 AM, Digvijay B]Mikhail
> Medvedev ---03/14/2016 08:12:59 PM---Hi Gigvijay, On Mon, Mar 14, 2016 at
> 7:00 AM, Digvijay B Ukirde
>
> From: Mikhail Medvedev <mihail...@gmail.com>
> To: Digvijay B Ukirde/India/IBM@IBMIN
> Cc: "openstack-infra@lists.openstack.org" <
> openstack-infra@lists.openstack.org>
> Date: 03/14/2016 08:12 PM
> Subject: Re: [OpenStack-Infra] [Third-party CI] - Running CI setup
> failing due to NOT_REGISTERED issue
> --
>
>
>
> Hi Gigvijay,
>
> On Mon, Mar 14, 2016 at 7:00 AM, Digvijay B Ukirde
> <digvijay.uki...@in.ibm.com> wrote:
> > Hi Team,
> >
> > I am running zuul, zuul merger and Jenkins in one Ubuntu.
> > I have configured zuul and zuul merger, Test connection for Jenkins
> gearman
> > plugin also runs successfully.
> >
> > But whenever patch come, CI is building job dsvm-tempest-full with
> > NOT_REGISTERED result at start before it actually get processed by CI.
> > So when it actually go through the CI, it gives result with SUCCESS but
> with
> > 1 recheck.
>
> It would be useful if you could paste your zuul pipeline
> configuration. It shows up in zuul.log when you restart zuul-server.
> NOT_REGISTERED means that there is mismatch between the job zuul wants
> to start, and what jenkins registered (if it did) with zuul gearman
> server. Check what got registered on gearman server with:
>
> echo status | nc  4730  # assuming you use standard
> port, and gearman is on zuul
>
> Would be helpful if you paste that as well.
>
> >
> > Besides when I checked my logs [debug.log] my Gearman is working fine
> > DEBUG zuul.Gearman: Function build:dsvm-tempest-full is registered
> > while I am gett

Re: [OpenStack-Infra] [Third-party CI] - Running CI setup failing due to NOT_REGISTERED issue

2016-03-14 Thread Mikhail Medvedev 
Hi Gigvijay,

On Mon, Mar 14, 2016 at 7:00 AM, Digvijay B Ukirde
 wrote:
> Hi Team,
>
> I am running zuul, zuul merger and Jenkins in one Ubuntu.
> I have configured zuul and zuul merger, Test connection for Jenkins gearman
> plugin also runs successfully.
>
> But whenever patch come, CI is building job dsvm-tempest-full with
> NOT_REGISTERED result at start before it actually get processed by CI.
> So when it actually go through the CI, it gives result with SUCCESS but with
> 1 recheck.

It would be useful if you could paste your zuul pipeline
configuration. It shows up in zuul.log when you restart zuul-server.
NOT_REGISTERED means that there is mismatch between the job zuul wants
to start, and what jenkins registered (if it did) with zuul gearman
server. Check what got registered on gearman server with:

echo status | nc  4730  # assuming you use standard
port, and gearman is on zuul

Would be helpful if you paste that as well.

>
> Besides when I checked my logs [debug.log] my Gearman is working fine
> DEBUG zuul.Gearman: Function build:dsvm-tempest-full is registered
> while I am getting error of Gerrit
> ERROR zuul.GerritEventConnector: Received unrecognized event type
> 'ref-replicated' from Gerrit. Can not get account information.
>

Ignore that error.

> If you know this issue with solution, can you please help me.
>
>
>
> Thanks,
> Digvijay
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [Third-party CI] - New third party CI setup, Gerrit patches not getting displayed in Zuul UI and jenkins job not running.

2016-03-14 Thread Mikhail Medvedev 
Brijnandan,

On Mon, Mar 14, 2016 at 6:12 AM, Brijnandan <brijnan...@cloudbyte.com> wrote:
> Hi Team,
>
>
> I am in a process of setting up new third party CI. I am running zuul, zuul
> merger and Jenkins in one Ubuntu  VERSION=14.04.4 LTS, Trusty Tahr
>
> I have configured zuul and zuul merger, when zuul starts listening to
> patches they are not displayed in the zuul UI. However I can see them adding
> in  zuul.log as below.
>
> http://paste.openstack.org/show/490364/
>
>
>
> Test connection for Jenkins gearman plugin also runs successfully.
>
> I  have specified test job (noop-check-communication) under silent queue in
> nodepool.yaml but no jobs are getting triggered.
>
>
>
> debug.log:http://paste.openstack.org/show/490367/
>
> merger.log:  http://paste.openstack.org/show/490368/
>
> merger-debug.log: http://paste.openstack.org/show/490369/
>

You merger logs show an error, so you need to investigate that
further. To start, try manually cloning the repository you error out
on:

sudo su zuul
git clone ssh://cloudbyt...@review.openstack.org:29418/openstack/cinder
/var/lib/zuul/git/openstack/cinder  # taken that from log

If that does not work, try figuring out why. Maybe zuul-merger fails
for that same reason.

>
>
> I have added the public key of the root user to gerrit account and puppet
> common.yaml and tried cloning the repo manually, it works fine.
>
> Please suggest how I should proceed further. Let me know if any more inputs
> are required. Thanks in Advance.
>
>
>
> Regards,
>
> Brij
>

Mikhail Medvedev,
IBM

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Zuul memory leak

2016-03-08 Thread Mikhail Medvedev 
On Mon, Mar 7, 2016 at 6:04 PM, Joshua Hesketh <joshua.hesk...@gmail.com> wrote:
> Hi Mikhail,
>
> Okay thanks, that's helpful.
>
> You mentioned that you might try restarting zuul periodically to see if that
> helps. Perhaps instead you could do a reload (or HUP) first to see if that
> clears the cache and alleviates the issue for you?

SIGHUP (kill -1) does get configuration reloaded (according to logs), but
I saw no immediate effect on memory footprint. At the time of test,
zuul-server was at
3GB (while 24 hours earlier it was at 2GB). Unfortunately I had to restart
zuul-server due to unrelated problems, so now I need to wait some time before
being able to test again. I would definitely go the periodic SIGHUP
route if it proves to
work, that is a good idea.

>
> Cheers,
> Josh
>
> On Tue, Mar 8, 2016 at 10:53 AM, Mikhail Medvedev <mihail...@gmail.com>
> wrote:
>>
>> Hi Josh,
>>
>> On Mon, Mar 7, 2016 at 5:25 PM, Joshua Hesketh <joshua.hesk...@gmail.com>
>> wrote:
>> > Hi Mikhail,
>> >
>> > Thank you for the extra details. I'll continue to look into this.
>> >
>> > With the daily bumps when you do the log rotation, I assume you aren't
>> > reloading zuul at that point and the freed memory is likely due to
>> > another
>> > process?
>>
>> I was puzzled by the bumps, and checked the syslog. They are definitely
>> due to
>> "run-parts --report /etc/cron.daily" being triggered at 06:25, and not
>> zuul reloads.
>> The memory bumps could be due to any of the cron jobs. logrotate seemed
>> likely.
>> For the record:
>>
>> root@zuul:~# ls /etc/cron.daily
>> apache2  apport  apt  aptitude  bsdmainutils  dpkg  exim4-base
>> logrotate  man-db  mlocate  ntp  passwd  update-notifier-common
>> upstart
>>
>> I have also confirmed there were no changes to zuul layout for the
>> interval that
>> the graph shows.
>>
>> >
>> > Cheers,
>> > Josh
>> >
>> > On Tue, Mar 8, 2016 at 10:17 AM, Mikhail Medvedev <mihail...@gmail.com>
>> > wrote:
>> >>
>> >> On Wed, Feb 10, 2016 at 10:57 AM, James E. Blair <cor...@inaugust.com>
>> >> wrote:
>> >> > Michael Still <mi...@stillhq.com> writes:
>> >> >
>> >> >> On Tue, Feb 9, 2016 at 4:59 AM, Joshua Hesketh
>> >> >> <joshua.hesk...@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> On Thu, Feb 4, 2016 at 2:44 AM, James E. Blair
>> >> >>> <cor...@inaugust.com>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> On the subject of clearing the cache more often, I think we may
>> >> >>>> not
>> >> >>>> want
>> >> >>>> to wipe out the cache more often than we do now -- in fact, I
>> >> >>>> think
>> >> >>>> we
>> >> >>>> may want to look into ways to keep from doing even that, because
>> >> >>>> whenever we reload now, Zuul slows down considerably as it has to
>> >> >>>> query
>> >> >>>> Gerrit again for all of the data previously in its cache.
>> >> >>>>
>> >> >>>
>> >> >>> I can see a lot of 3rd parties or simpler CI's not needing to
>> >> >>> reload
>> >> >>> zuul
>> >> >>> very often so this cache would never get cleared. Perhaps cached
>> >> >>> objects
>> >> >>> should have an expiry time (of a day or so) and can be cleaned up
>> >> >>> periodically? Additionally if clearing the cache on a reload is
>> >> >>> causing
>> >> >>> pain maybe we should move the cache into the scheduler and keep it
>> >> >>> between
>> >> >>> reloads?
>> >> >>>
>> >> >>
>> >> >> Do you guys use oslo at all? I ask because the olso memcache stuff
>> >> >> does
>> >> >> exactly this, so it should be trivial to implement if you don't mind
>> >> >> depending on oslo.
>> >> >
>> >> > One of the main things we use the cache for is to ensure that every
>> >> > change is represented by a single Change object in Zuul's memory.
>> >> > The
>> >> > graph of enqueued It

Re: [OpenStack-Infra] Zuul memory leak

2016-03-07 Thread Mikhail Medvedev 
Hi Josh,

On Mon, Mar 7, 2016 at 5:25 PM, Joshua Hesketh <joshua.hesk...@gmail.com> wrote:
> Hi Mikhail,
>
> Thank you for the extra details. I'll continue to look into this.
>
> With the daily bumps when you do the log rotation, I assume you aren't
> reloading zuul at that point and the freed memory is likely due to another
> process?

I was puzzled by the bumps, and checked the syslog. They are definitely due to
"run-parts --report /etc/cron.daily" being triggered at 06:25, and not
zuul reloads.
The memory bumps could be due to any of the cron jobs. logrotate seemed likely.
For the record:

root@zuul:~# ls /etc/cron.daily
apache2  apport  apt  aptitude  bsdmainutils  dpkg  exim4-base
logrotate  man-db  mlocate  ntp  passwd  update-notifier-common
upstart

I have also confirmed there were no changes to zuul layout for the interval that
the graph shows.

>
> Cheers,
> Josh
>
> On Tue, Mar 8, 2016 at 10:17 AM, Mikhail Medvedev <mihail...@gmail.com>
> wrote:
>>
>> On Wed, Feb 10, 2016 at 10:57 AM, James E. Blair <cor...@inaugust.com>
>> wrote:
>> > Michael Still <mi...@stillhq.com> writes:
>> >
>> >> On Tue, Feb 9, 2016 at 4:59 AM, Joshua Hesketh
>> >> <joshua.hesk...@gmail.com>
>> >> wrote:
>> >>
>> >>> On Thu, Feb 4, 2016 at 2:44 AM, James E. Blair <cor...@inaugust.com>
>> >>> wrote:
>> >>>>
>> >>>> On the subject of clearing the cache more often, I think we may not
>> >>>> want
>> >>>> to wipe out the cache more often than we do now -- in fact, I think
>> >>>> we
>> >>>> may want to look into ways to keep from doing even that, because
>> >>>> whenever we reload now, Zuul slows down considerably as it has to
>> >>>> query
>> >>>> Gerrit again for all of the data previously in its cache.
>> >>>>
>> >>>
>> >>> I can see a lot of 3rd parties or simpler CI's not needing to reload
>> >>> zuul
>> >>> very often so this cache would never get cleared. Perhaps cached
>> >>> objects
>> >>> should have an expiry time (of a day or so) and can be cleaned up
>> >>> periodically? Additionally if clearing the cache on a reload is
>> >>> causing
>> >>> pain maybe we should move the cache into the scheduler and keep it
>> >>> between
>> >>> reloads?
>> >>>
>> >>
>> >> Do you guys use oslo at all? I ask because the olso memcache stuff does
>> >> exactly this, so it should be trivial to implement if you don't mind
>> >> depending on oslo.
>> >
>> > One of the main things we use the cache for is to ensure that every
>> > change is represented by a single Change object in Zuul's memory.  The
>> > graph of enqueued Items link to their respective Changes which may link
>> > to each other due to dependencies.  When something changes in Gerrit, we
>> > want that reflected immediately and consistently in all of the objects
>> > in that graph.  Using the cache means that every time we add a new
>> > Change object to that graph, we use the same object for a given change.
>> >
>> > This is why we can't use time-based expiry -- we must not drop objects
>> > from the cache if they are still in the graph.  Otherwise we will create
>> > new duplicative objects and the ones still in the graph will not be
>> > updated.
>> >
>> > Perhaps we should change these objects to something more ephemeral that
>> > can proxy for some other mechanism that can operate more like a
>> > traditional cache (with time-based expiry).  But I think changes to this
>> > system should happen in Zuulv3 -- it works well enough for Zuulv2 for
>> > now.
>> >
>> > -Jim
>> >
>>
>> We are one of third-party CIs and using "Zuul version: 2.1.1.dev123",
>> which is one commit after [1]. That one commit after is not in tree - I am
>> applying [2] on top.
>>
>> The VM has 8GB of RAM. zuul-server memory footprint goes up consistently
>> over
>> the course of a week. Normally it takes about 3-4 days to get over to 3Gb.
>> About a week ago I witnessed zuul-server get to 95% of RAM, at which point
>> kernel started killing other processes. The graph [3] memory [3], and it
>> reflects zuul-server consumption. The daily bumps on the graph are daily
>> cron
>> doing log rotation etc, possibly flushing caches.
>>
>> I can not say 100% that it is still the leak. Could simply be that
>> zuul-server
>> requires more ram now.
>>
>> [1]
>> https://review.openstack.org/#q,I81ee47524cda71a500c55a95a2280f491b1b63d9,n,z
>> [2]
>> https://review.openstack.org/#q,If3a418fa2d4993a149d454e02a9b26529e4b6825,n,z
>> [3] http://imgur.com/SzqSA1H
>>
>> Mikhail Medvedev (mmedvede)
>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Zuul memory leak

2016-03-07 Thread Mikhail Medvedev 
On Wed, Feb 10, 2016 at 10:57 AM, James E. Blair <cor...@inaugust.com> wrote:
> Michael Still <mi...@stillhq.com> writes:
>
>> On Tue, Feb 9, 2016 at 4:59 AM, Joshua Hesketh <joshua.hesk...@gmail.com>
>> wrote:
>>
>>> On Thu, Feb 4, 2016 at 2:44 AM, James E. Blair <cor...@inaugust.com>
>>> wrote:
>>>>
>>>> On the subject of clearing the cache more often, I think we may not want
>>>> to wipe out the cache more often than we do now -- in fact, I think we
>>>> may want to look into ways to keep from doing even that, because
>>>> whenever we reload now, Zuul slows down considerably as it has to query
>>>> Gerrit again for all of the data previously in its cache.
>>>>
>>>
>>> I can see a lot of 3rd parties or simpler CI's not needing to reload zuul
>>> very often so this cache would never get cleared. Perhaps cached objects
>>> should have an expiry time (of a day or so) and can be cleaned up
>>> periodically? Additionally if clearing the cache on a reload is causing
>>> pain maybe we should move the cache into the scheduler and keep it between
>>> reloads?
>>>
>>
>> Do you guys use oslo at all? I ask because the olso memcache stuff does
>> exactly this, so it should be trivial to implement if you don't mind
>> depending on oslo.
>
> One of the main things we use the cache for is to ensure that every
> change is represented by a single Change object in Zuul's memory.  The
> graph of enqueued Items link to their respective Changes which may link
> to each other due to dependencies.  When something changes in Gerrit, we
> want that reflected immediately and consistently in all of the objects
> in that graph.  Using the cache means that every time we add a new
> Change object to that graph, we use the same object for a given change.
>
> This is why we can't use time-based expiry -- we must not drop objects
> from the cache if they are still in the graph.  Otherwise we will create
> new duplicative objects and the ones still in the graph will not be
> updated.
>
> Perhaps we should change these objects to something more ephemeral that
> can proxy for some other mechanism that can operate more like a
> traditional cache (with time-based expiry).  But I think changes to this
> system should happen in Zuulv3 -- it works well enough for Zuulv2 for
> now.
>
> -Jim
>

We are one of third-party CIs and using "Zuul version: 2.1.1.dev123",
which is one commit after [1]. That one commit after is not in tree - I am
applying [2] on top.

The VM has 8GB of RAM. zuul-server memory footprint goes up consistently over
the course of a week. Normally it takes about 3-4 days to get over to 3Gb.
About a week ago I witnessed zuul-server get to 95% of RAM, at which point
kernel started killing other processes. The graph [3] memory [3], and it
reflects zuul-server consumption. The daily bumps on the graph are daily cron
doing log rotation etc, possibly flushing caches.

I can not say 100% that it is still the leak. Could simply be that zuul-server
requires more ram now.

[1] 
https://review.openstack.org/#q,I81ee47524cda71a500c55a95a2280f491b1b63d9,n,z
[2] 
https://review.openstack.org/#q,If3a418fa2d4993a149d454e02a9b26529e4b6825,n,z
[3] http://imgur.com/SzqSA1H

Mikhail Medvedev (mmedvede)

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-dev] [cinder] How to configure the third party CI to be triggered only when jenkins +1

2016-02-23 Thread Mikhail Medvedev 
On Mon, Feb 22, 2016 at 7:32 PM, liuxinguo  wrote:
> Hi,
>
>
>
> There is no need to trigger third party CI if a patch does not pass Jenkins
> Verify.
>
> I think there is a way to reach this but I’m not sure how.
>
>
>
> So is there any reference or suggestion to configure the third party CI to
> be triggered only when jenkins +1?
>
>

If you are using zuul, then you should look into 'approval' setting in
layout. E.g. check the current layout that Infra uses for gate
pipeline: 
https://github.com/openstack-infra/project-config/blob/6b71e8cac676e04141839eeecce3462df3a04848/zuul/layout.yaml#L41-L46.

>
> Thanks for any input!
>
>
>
> Regards,
>
> Wilson Liu
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Issue with setup_workspace in safe-vm-gate-wrap.sh

2015-12-02 Thread Mikhail Medvedev 
Is OVERRIDE_ZUUL_BRANCH being set to anything, or is it empty? The
error could be explained by it not being set.

On Tue, Dec 1, 2015 at 3:10 PM, Bob Hansen  wrote:
> This showed up today after I ran a job to continue to test my 3rd party CI
> system. It appears that the $2 argument is not being properly passed to
> setup_workspace. The failure shows up when in the scirpt when the argument
> is actuall used.
>
> Here's snippets form the relevant log. In the jenkins console I see this, I
> added a few trace statements. This all looks good to me, the command
> invocation looks correct; at lest on the tsfilter invocation;
> setup_workspace $OVERRIDE_ZUUL_BRANCH $BASE/new, $BASE is set to /opt/stack
>
> Setting up the workspace BASE is set to /opt/stack
> + echo '... this takes 3 - 5 minutes (logs at
> logs/devstack-gate-setup-workspace-new.txt.gz)'
> ... this takes 3 - 5 minutes (logs at
> logs/devstack-gate-setup-workspace-new.txt.gz)
> ++ date +%s
> + start=1448995816
> ++ run_command setup_workspace '$OVERRIDE_ZUUL_BRANCH' '$BASE/new'
> ++ local 'fn=setup_workspace $OVERRIDE_ZUUL_BRANCH $BASE/new'
> ++ local cmd=
> ++ read -r -d '' cmd
> ++ echo 'source
> '\''/var/lib/jenkins/workspace/zvm-ci-test-slave/test_env.sh'\''
> source
> '\''/var/lib/jenkins/workspace/zvm-ci-test-slave/devstack-gate/functions.sh'\''
> set -o errexit
> tsfilter setup_workspace $OVERRIDE_ZUUL_BRANCH $BASE/new
> executable=/bin/bash'
> + /tmp/ansible/bin/ansible all -f 5 -i
> /var/lib/jenkins/workspace/zvm-ci-test-slave/inventory -m shell -a 'source
> '\''/var/lib/jenkins/workspace/zvm-ci-test-slave/test_env.sh'\''
> source
> '\''/var/lib/jenkins/workspace/zvm-ci-test-slave/devstack-gate/functions.sh'\''
> set -o errexit
> tsfilter setup_workspace $OVERRIDE_ZUUL_BRANCH $BASE/new
> executable=/bin/bash'
> ++ date +%s
> + end=1448995817
> + took=0
> + [[ 0 -gt 10 ]]
>
> However, if I look the log file for the workspace setup, or
> devstack-gate-setup-workspace-new.txt, (wtih a few more traces added as
> well), I see that $2 is passed as being empty, I would have expected it to
> be the value of $BASE/new as show in the tsfilter line above. $BASE should
> be /opt/stack as show in the trace above.
>
> localhost | FAILED | rc=1 >>
> 2015-12-01 20:48:10.373 | DEST is set to or In setup_workspace
> 2015-12-01 20:48:10.387 | BASE is set to /opt/stack is set globally.. In
> setup_workspace
> 2015-12-01 20:48:10.398 | base_branch is set to /opt/stack/new or
> /opt/stack/new
>
> It fails when the script does the mkdir -p $BASE a few lines down in
> functions.sh
>
> function setup_workspace {
> local base_branch=$1
> local DEST=$2
> local xtrace=$(set +o | grep xtrace)
> local cache_dir=$BASE/cache/files/
>
> echo "DEST is set to $DEST or $2 In setup_workspace"
> echo "BASE is set to $BASE is set globally.. In setup_workspace"
> echo "base_branch is set to $base_branch or $1"
>
> DEST=$BASE/new
>
> ...
>
> sudo mkdir -p $DEST
> sudo chown -R $USER:$USER $DEST
>
> The script looks like this
>
>
>
> I'm at a loss as to what could be going on. I've had to hack the script to
> make my gate jobs run, don't want to live with this, as this has been
> running pretty well for me for a while now without the ahck.
>
> Bob Hansen
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Request to add members to gerrit group for new infra project

2015-12-01 Thread Mikhail Medvedev 
Hi Infra team,

Please add <ramy.asse...@hpe.com> and <mihail...@gmail.com> to
puppet-ciwatch group [1]. The infra-core group should also be added to [1].

The project was created with [2] and governance change added with [3].


[1] https://review.openstack.org/#/admin/groups/1138,members
[2] https://review.openstack.org/#/c/231699/
[3] https://review.openstack.org/#/c/231698/

Thank you,

Mikhail Medvedev (mmedvede)
IBM
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] IBM XIV CI account

2015-08-13 Thread Mikhail Medvedev 
Hi Isaac,

  I have added your new account (ibmxi...@il.ibm.com) to the Third-Party CI
gerrit group and have removed the old one. Someone from the Infra team
would still need to deactivate your old gerrit account.

Mikhail Medvedev (irc:mmedvede)
IBM PowerKVM CI


On Wed, Aug 12, 2015 at 12:58 AM, Isaac Beckman isa...@il.ibm.com wrote:

 Dear Infra team,

 A new CI account was created for IBM XIV as we could not edit the ssh
 public key that was assigned to the original account.
 We did this after we saw the following thread
 http://openstack-infra.openstack.narkive.com/DHJ0JILf/change-of-public-key

 The new account is:
 Account full name:* IBM XIV CI*
 Account email:* ibmxi...@il.ibm.com ibmxi...@il.ibm.com*

 The old account was connected to a private mail address (alo...@il.ibm.com)
 rather then having a dedicated mail address for it
 Please take care of cleaning up the old account. Need to make sure that
 you are not cleaning the personal gerrit account of Alon Marx which is also
 connected to the address: alo...@il.ibm.com

 Thanks,
 Isaac Beckman

 Office: +972-3-6897874
 Fax: +972-3-6897755
 Mobile: +972-50-2680180
 Email: isa...@il.ibm.com

 IBM XIV, Cloud Storage Solutions (previously HSG)
 www.ibm.com/storage/disk/xiv

 ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra][devstack-gate][CI] Downloading only required projects in CI.

2015-07-21 Thread Mikhail Medvedev 
Abnishek,

I think you can use DEVSTACK_GATE_PROJECTS_OVERRIDE variable to limit the
projects you want.

Mikhail Medvedev (mmedvede)

On Tue, Jul 21, 2015 at 11:24 AM, Abhishek Shrivastava 
abhis...@cloudbyte.com wrote:

 Hi Folks,

 I have setup the CI using Ramy's guide and found a lot of ways to setup
 configs of Cinder, Tempest and DevStack using the Devstack Gate scripts
 mentioned and used in the guide.

 Even though we are able to set the Devstack services according to our
 needs, but still we have to download a lot of *openstack-projects *out
 of which only few are of our use.

 So, if anyone knows anything about how to download only required projects
 in CI please do reply.


- The project list mentioned in the *devstack-vm-gate-wrap*
script[1].
- Downloaded using *functions.sh* script through function *setup
workspace*[2]

 [1]
 https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L35
 [2]
 https://github.com/openstack-infra/devstack-gate/blob/master/functions.sh#L416

 --


 *Thanks  Regards,*
 *Abhishek*
 *Cloudbyte Inc. http://www.cloudbyte.com*

 ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra][devstack-gate][CI] Downloading only required projects in CI.

2015-07-21 Thread Mikhail Medvedev 
Abnishek,

 If you use jenkins job builder and have something similar to
infrastructure team's jjb configuration [1] , you would need to add an
export there. E.g.

...
export DEVSTACK_GATE_PROJECTS_OVERRIDE=openstack/cinder openstack/nova
openstack/neutron
...

Obviously, change the project list to the projects you want to use. This
should force devstack-vm-gate-wrap.sh only to checkout those projects.

Sidenote: if you use nodepool and do use snapshot image builds based on
using prepare_node_devstack.sh scripts, you can also limit which project
repositories are cached to that image by changing your cache_git_repos.py
[2].

[1]
https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L16-L20
[2]
https://github.com/openstack-infra/project-config/blob/master/nodepool/scripts/cache_git_repos.py

cheers,

Mikhail Medvedev (mmedvede)

On Tue, Jul 21, 2015 at 12:50 PM, Doug Wiegley doug...@parksidesoftware.com
 wrote:

 You can also override ENABLED_SERVICES with the minus syntax. Example:


 https://github.com/openstack/neutron-lbaas/blob/master/neutron_lbaas/tests/contrib/gate_hook.sh

 Thanks,
 doug


 On Jul 21, 2015, at 11:43 AM, Abhishek Shrivastava abhis...@cloudbyte.com
 wrote:

 Hi Mikhail,

 Thanks for the suggestion, can you tell me one more thing that how to use
 this, I mean can you show me some examples regarding its syntax.

 On Tue, Jul 21, 2015 at 10:20 PM, Mikhail Medvedev mihail...@gmail.com
 wrote:

 Abnishek,

 I think you can use DEVSTACK_GATE_PROJECTS_OVERRIDE variable to limit the
 projects you want.

 Mikhail Medvedev (mmedvede)

 On Tue, Jul 21, 2015 at 11:24 AM, Abhishek Shrivastava 
 abhis...@cloudbyte.com wrote:

 Hi Folks,

 I have setup the CI using Ramy's guide and found a lot of ways to setup
 configs of Cinder, Tempest and DevStack using the Devstack Gate scripts
 mentioned and used in the guide.

 Even though we are able to set the Devstack services according to our
 needs, but still we have to download a lot of *openstack-projects *out
 of which only few are of our use.

 So, if anyone knows anything about how to download only required
 projects in CI please do reply.


- The project list mentioned in the *devstack-vm-gate-wrap*
script[1].
- Downloaded using *functions.sh* script through function *setup
workspace*[2]

 [1]
 https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L35
 [2]
 https://github.com/openstack-infra/devstack-gate/blob/master/functions.sh#L416

 --


 *Thanks  Regards,*
 *Abhishek*
 *Cloudbyte Inc. http://www.cloudbyte.com/*

 ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra





 --


 *Thanks  Regards,*
 *Abhishek*
 *Cloudbyte Inc. http://www.cloudbyte.com/*
  ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra



___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra][devstack-gate][CI] Downloading only required projects in CI.

2015-07-21 Thread Mikhail Medvedev 
Abnishek,

I think you can use DEVSTACK_GATE_PROJECTS_OVERRIDE variable to limit the
projects you want.

Mikhail Medvedev (mmedvede)


On Tue, Jul 21, 2015 at 11:24 AM, Abhishek Shrivastava 
abhis...@cloudbyte.com wrote:

 Hi Folks,

 I have setup the CI using Ramy's guide and found a lot of ways to setup
 configs of Cinder, Tempest and DevStack using the Devstack Gate scripts
 mentioned and used in the guide.

 Even though we are able to set the Devstack services according to our
 needs, but still we have to download a lot of *openstack-projects *out
 of which only few are of our use.

 So, if anyone knows anything about how to download only required projects
 in CI please do reply.


- The project list mentioned in the *devstack-vm-gate-wrap*
script[1].
- Downloaded using *functions.sh* script through function *setup
workspace*[2]

 [1]
 https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L35
 [2]
 https://github.com/openstack-infra/devstack-gate/blob/master/functions.sh#L416

 --


 *Thanks  Regards,*
 *Abhishek*
 *Cloudbyte Inc. http://www.cloudbyte.com*

 ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Change of Public Key

2015-07-21 Thread Mikhail Medvedev 
Hi Xicheng,

  Let me know your new gerrit account name and I can add to third-party CI
gerrit group.

thanks,

Mikhail Medvedev (irc:mmedvede)

On Tue, Jul 21, 2015 at 2:05 PM, Anita Kuno ante...@anteaya.info wrote:

 On 07/17/2015 12:55 PM, Xicheng Chang wrote:
  Hi Lyz,
 
  The third party testing doc recommends that 3rd party CI accounts
 should
  be added to the Third Party CI mail filter list
  https://review.openstack.org/#/admin/groups/270 by contacting
  coordinators. Is there a group email address for those coordinators or
  should I just send an email to all of them?
 
 
  Thanks,
  Xicheng

 Hi Xicheng:

 Anyone on this list can help you if you provide the name of the gerrit
 ci account: https://review.openstack.org/#/admin/groups/440,members

 I am on this list and can help you if you provide the name of the gerrit
 ci account.

 I can't speak for the others on this list but for something like this I
 prefer if someone talks to me on irc.

 Thanks,
 Anita.
 irc nick: anteaya

 
  On Wed, Jul 15, 2015 at 8:48 PM, Elizabeth K. Joseph 
 l...@princessleia.com
  wrote:
 
  On Wed, Jul 15, 2015 at 8:39 PM, Jerry Zhao xyzje...@gmail.com wrote:
  Hi
  I am afraid this account was created long ago through email request
  instead of the procedure mentioned in the doc, so I don't have the
 password
  to login Gerrit with.
 
  I see. Since we switched to self-service accounts the only process we
  have for editing information is for you to create a new CI account
  using the self-service process I linked to in the last email. You'll
  want to go ahead and do that and update your system to use the new
  account.
 
  Let us know when you've done this so we can then take care of cleaning
  up the old account.
 
  --
  Elizabeth Krumbach Joseph || Lyz || pleia2
 
 
 
 
 
 
  ___
  OpenStack-Infra mailing list
  OpenStack-Infra@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
 


 ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-dev] [third-party]Time for Additional Meeting for third-party

2014-12-02 Thread Mikhail Medvedev 
Hi,

For me:

 - would not work: 03:00 UTC -- 12:00 UTC
 + better: 20:00 -- 01:00 UTC

As for having alternating meetings vs. two different meeting times on the
same week, I am leaning towards alternating. I have no strong opinion on
this though.

Thanks,

Mikhail Medvedev (mmedvede)

On Tue, Dec 2, 2014 at 12:09 PM, trinath.soman...@freescale.com 
trinath.soman...@freescale.com wrote:

 Hi Anita-

 I'm okay/happy with your proposal on 08:00 AM UTC on Tuesdays.

 +1

 --
 Trinath Somanchi - B39208
 trinath.soman...@freescale.com | extn: 4048

 -Original Message-
 From: Anita Kuno [mailto:ante...@anteaya.info]
 Sent: Tuesday, December 02, 2014 7:48 PM
 To: openstack-infra@lists.openstack.org
 Subject: Re: [OpenStack-Infra] [openstack-dev] [third-party]Time for
 Additional Meeting for third-party

 Thanks for all the responses so far.

 This is to find an *additional* meeting time. This is not to have meeting
 times that switch from week to week. That would only confuse people. We
 don't need that. Trying to find a meeting time that would work for
 everybody doesn't work, hence the direction to create an
 *additional* meeting time. That way if you miss one, you can attend
 another.

 So far we have:
 Josh's suggestion of 00:00am Tuesday UTC Nurit's suggestion of Mondays
 anytime between 8:00 - 16:00 Salvadore's seconding of Josh's proposal
 Trinath's suggestion of 05:00AM UTC on Tuesdays

 1400 and 2200 doesn't give us anything better than what we have.

 Based on conversations at the summit, I would really like to see if Nurit
 and Josh can convene on an agreed time, I would really like to see both of
 them being able to attend on a regular basis.

 If we tried to work with an 8:00 utc time, either Monday or Tuesday, would
 that work for you, Josh and Nurit? (That would be 7pm in the evening for
 you, Josh - I think - is that workable?)

 Thanks I appreciate the interest,
 Anita.


 On 12/02/2014 08:12 AM, Salvatore Orlando wrote:
  1800UTC should generally work for Europe. The only issue is that it
  falls right around dinner time.
  It is however a good timing since most of the night hours fall over
  the pacific ocean.
 
  Therefore I tend to agree with Joshua's proposal since with that time
  range most of the night hours would fall over the atlantic.
  Ideally 0600UTC is perfectly symmetric to the other meeting time, but
  it might be a bit tricky for western and central Europe, especially
  during winter.
  Anytime between 0700UTC and 0900UTC would be better for Europe, but
  might fall towards dinner time for Australia and be a bit
  uncomfortable for New Zealand during their summer.
 
  Anyway, this proposal would make the meeting time prohibitive for
  eastern/central US  Canada as well as South America. I don't know if
  that's acceptable considering that, from what I gather, most of the
  regular attendees come from those time zones.
 
  Regards,
  Salvatore
 
  On 2 December 2014 at 12:44, Kurt Taylor kurt.r.tay...@gmail.com
 wrote:
 
  Thanks for starting this discussion Anita.
 
  The existing meeting time on Monday has never worked well for me. We
  could follow what other working groups have done, having alternating
  meeting times to accommodate everyone.
 
  I propose that we have 2 meetings of Third-party CI Ops, alternating
 weeks:
  Wednesday 1400 UTC in #openstack-meeting-3 Wednesday 2200 UTC in
  #openstack-meeting-3
 
  Kurt Taylor (krtaylor)
 
 
  On Tue, Dec 2, 2014 at 2:46 AM, Nurit Vilosny nur...@mellanox.com
 wrote:
 
  HI,
  Thanks Anita for pushing it. We will be able to be much more
  involved if meetings would be earlier.
  We're located in Israel, so Mondays anytime between 8:00 - 16:00,
  will be ideal for us.
 
  Nurit
 
  -Original Message-
  From: trinath.soman...@freescale.com [mailto:
  trinath.soman...@freescale.com]
  Sent: Tuesday, December 02, 2014 10:32 AM
  To: Anita Kuno; openstack Development Mailing List;
  openstack-infra@lists.openstack.org
  Subject: Re: [openstack-dev] [OpenStack-Infra] [third-party]Time for
  Additional Meeting for third-party
 
  Hi-
 
  Its nice to have CI operators meetings.
 
  I'm from India, Its okay for me for 05:00AM UTC on Tuesdays.
 
  --
  Trinath Somanchi - B39208
  trinath.soman...@freescale.com | extn: 4048
 
  -Original Message-
  From: Anita Kuno [mailto:ante...@anteaya.info]
  Sent: Tuesday, December 02, 2014 1:37 AM
  To: openstack Development Mailing List;
  openstack-infra@lists.openstack.org
  Subject: [OpenStack-Infra] [third-party]Time for Additional Meeting
  for third-party
 
  One of the actions from the Kilo Third-Party CI summit session was
  to start up an additional meeting for CI operators to participate
  from non-North American time zones.
 
  Please reply to this email with times/days that would work for you.
  The current third party meeting is on Mondays at 1800 utc which
  works well since Infra meetings are on Tuesdays. If we could find a
  time that works for Europe and APAC