Hi Tang,
On Mon, Jul 20, 2015 at 11:18 AM, Tang Chen <tangc...@cn.fujitsu.com> wrote: > Hi Asselin, Abhishek, > > Thanks for the reply. :) > > > On 07/19/2015 02:41 AM, Asselin, Ramy wrote: > > HI Abhi & Tang, > > > > Sorry I missed this thread. Let me know if you’ve resolved your issues. > > > > My repo is undergoing migrations to reuse components in > openstack-infra/puppet-openstackci. > > > > For single-use-nodes, the file you need has been removed here [1]: But I > see now that it is still needed, or a different function is needed based on > this version used by infra: [2]. I will explore a solution. > > > > A couple other notes, please use ci-sandbox [3] instead of sandbox. > > > OK. > > > > Zuul use behind a proxy: seems you got past this? Could you share your > solution? > > > The root cause is that zuul uses a library named paramiko to create > connection > with low level socket APIs. And it doesn't provide proxy functionary. > > I tried to use tools like proxychain, to redirect the zuul connection to > go through > my proxy. But it doesn't work. If I use proxychain to run zuul service, it > doesn't > output anything to the log file, and the service will die soon. > > I think, there are two solutions: > 1. Add proxy functionality to paramiko. > 2. Add proxy functionality to zuul. That means, maybe, > zuul does not use paramiko to create connection, but by itself. > > Solution 1 is much simpler, so for now, I just modified the source code of > paramiko. > > I'm using python-socksipy package, and modified > /usr/local/lib/python2.7/dist-packages/paramiko/client.py like this: > > diff --git a/client.py b/client.py > index 15ff696..d7225ed 100644 > --- a/client.py > +++ b/client.py > @@ -24,6 +24,7 @@ from binascii import hexlify > import getpass > import os > import socket > +import socks > import warnings > import pdb > > @@ -235,6 +236,7 @@ class SSHClient (ClosingContextManager): > ``gss_deleg_creds`` and ``gss_host`` arguments. > """ > > + """ > if not sock: > for (family, socktype, proto, canonname, sockaddr) in > socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM): > if socktype == socket.SOCK_STREAM: > @@ -251,6 +253,13 @@ class SSHClient (ClosingContextManager): > except: > pass > retry_on_signal(lambda: sock.connect(addr)) > + """ > + > + if not sock: > + sock = socks.socksocket() > + sock.setproxy(socks.PROXY_TYPE_SOCKS5, "MY_PROXY_IP", > MY_PROXY_PORT, username="XXX", password="XXX") > > # This is review.openstack.org > + addr = ('104.130.159.134', 29418) > + retry_on_signal(lambda: sock.connect(addr)) > > t = self._transport = Transport(sock, gss_kex=gss_kex, > gss_deleg_creds=gss_deleg_creds) > t.use_compression(compress=compress) > > > Quite a nice explanation +1 for all the things you have done. Just one more thing to ask, have you made your slave node static? > Of course, this is just a draft. It is only for my case, not for all. > > > BTW, I'm now working on Fujitsu CI System, and really want to join into > the development of openstack-infra. > I think the proxy functionality is necessary for many companies. So if you > are planing to add the proxy support, > I think I can help. > > Thanks. :) > > > > Also, feel free to join 3rd party ci IRC meetings on freenode [4]. It’s > a great place to ask questions and meet others setting up or maintaining > these systems. > > > > Thanks, > > Ramy > > IRC: asselin > > > > [1] > https://github.com/rasselin/os-ext-testing/commit/dafe822be7813522a6c7361993169da20b37ffb7 > > [2] > https://github.com/openstack-infra/project-config/blob/master/zuul/openstack_functions.py > > [3] http://git.openstack.org/cgit/openstack-dev/ci-sandbox/ > > [4] http://eavesdrop.openstack.org/#Third_Party_Meeting > > > > > > > > *From:* Abhishek Shrivastava [mailto:abhis...@cloudbyte.com > <abhis...@cloudbyte.com>] > *Sent:* Monday, July 13, 2015 11:51 PM > *To:* OpenStack Development Mailing List (not for usage questions) > *Subject:* Re: [openstack-dev] [CI] How to set a proxy for zuul. > > > > Also if you want to change it you will need to talk with Asselin Ramy who > is the owner of the repo you followed. > > > > On Tue, Jul 14, 2015 at 12:18 PM, Abhishek Shrivastava < > abhis...@cloudbyte.com> wrote: > > Basically it is not required, and if you see /etc/jenkins_jobs/config > folder you will find one dsvm-cinder-tempest.yaml which is to be used > basically not examples.yaml. So its not an issue. > > > > On Tue, Jul 14, 2015 at 12:07 PM, Tang Chen <tangc...@cn.fujitsu.com> > wrote: > > > > On 07/14/2015 01:46 PM, Abhishek Shrivastava wrote: > > Instead of it use reusable_node option. > > > Thanks. Problem resolved. :) > > BTW, single_use_node is written in layout.yaml by default. > If it doesn't exist anymore, do we need a patch to fix it ? > > For someone who uses CI for the first time, it is really a problem...... > > And also, if I want to post patch for zuul, where should I post the patch ? > > Thanks. > > > > > > > On Tue, Jul 14, 2015 at 9:12 AM, Tang Chen <tangc...@cn.fujitsu.com> > wrote: > > Hi Abhishek, All, > > I found the problem. > > My /etc/zuul/layout/layout.yaml has the following config: > > jobs: > - name: ^dsvm-tempest.*$ > parameter-function: single_use_node > > But in _parseConfig() in zuul/scheduler.py, it failed to find > single_use_node(). > > fname = config_job.get('parameter-function', None) > if fname: > func = config_env.get(fname, None) > if not func: > raise Exception("Unable to find function %s" % fname) > > So projects section was not parsed. > > Does anyone know why ? > > Thanks. > > > > > On 07/14/2015 10:54 AM, Tang Chen wrote: > > Hi Abhishek, > > I printed the self.layout.projects in zuul/scheduler.py, it is empty. > So the project was not found. > > But I did do the *jenkins-jobs --flush-cache update* > */etc/jenkins_jobs/config/* > And I did configure openstack-dev/sandbox in layout.yaml. > > Do you have any idea what's wrong here ? > > Thanks. > > On 07/13/2015 05:58 PM, Tang Chen wrote: > > > > On 07/13/2015 04:35 PM, Abhishek Shrivastava wrote: > > Updating jobs using "*sudo jenkins-jobs --flush-cache update > /etc/jenkins_jobs/config/". *Also update the myvendor in examples.yaml > > > > > Sorry, I updated the jobs, restart the whole machine. But it still doesn't > work. > > By the way, there is no vendor in examples.yaml. > > It is still this error: Project openstack-dev/sandbox not found > > Anything else should I pay attention to? > > Thanks. > > > > > On Mon, Jul 13, 2015 at 1:45 PM, Tang Chen <tangc...@cn.fujitsu.com> > wrote: > > > > On 07/13/2015 03:50 PM, Abhishek Shrivastava wrote: > > Use tester or something, also are you updating the jobs or not? > > > I used tester as my vendor. It doesn't work. > > And what do you mean by updating the jobs ? I built the > noop-check-cimmunitication > job once in Jenkins UI. Does it matter ? All the others are not touched. > > And referring to the error, "Project openstack-dev/sandbox not found", it > seems like > somewhere the project name was wrong. > > right ? > > > > Thanks. > > > > > On Mon, Jul 13, 2015 at 1:16 PM, Tang Chen <tangc...@cn.fujitsu.com> > wrote: > > Hi Abhishek, > > Thanks for the quick reply. > > On 07/13/2015 03:16 PM, Abhishek Shrivastava wrote: > > Also check that Gearman is connecting or not through Jenkins UI. > > > > On Mon, Jul 13, 2015 at 12:45 PM, Abhishek Shrivastava < > abhis...@cloudbyte.com> wrote: > > First of all, change the "vendor" to your vendor name in > /etc/jenkins_jobs/config/projects.yaml > file. Also, restart the zuul and zuul merger. > > > I did the check. Gearman plugin works find. In Jenkins UI, I tested the > connection, and it succeeded. > > And also, I restarted zuul and zuul merger every time I modified the yaml > files. > > But it doesn't work. > > And the vendor, does that matter ? And what vendor name should I provide ? > I cannot find any vendor info in my Gerrit service account profile. > For example, is XXX OK ? > > Thanks. > > > > > > > > > On Mon, Jul 13, 2015 at 12:29 PM, Tang Chen <tangc...@cn.fujitsu.com> > wrote: > > Hi all, > > I have constructed my CI system. > When I tested it with sandbox project, I posted a patch to Gerrit. > > https://review.openstack.org/201002 > > But I got this error in zuul scheduler: > > 2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Run handler awake > 2015-07-14 14:07:24,921 DEBUG zuul.Scheduler: Fetching trigger event > 2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Processing trigger event > <TriggerEvent comment-added openstack-dev/sandbox master 201002,1 > Verified:1> > 2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Project > openstack-dev/sandbox not found > 2015-07-14 14:07:24,922 DEBUG zuul.Scheduler: Run handler sleeping > > > My /etc/zuul/layout/layout.yaml looks like this: > > projects: > - name: openstack-dev/sandbox > check: > - noop-check-communication > > > My /etc/jenkins_jobs/config/projects.yaml looks like this: > > - project: > name: sandbox > github-org: openstack-dev > node: master > vendor: myvendor > > jobs: > - noop-check-communication > - dsvm-tempest-full: > node: 'devstack_slave || devstack-precise-check || d-p-c' > > > And Jenkins master works fine. > > > Does anyone know what is going on here ? > > > > Thanks. > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > > > > -- > > *Thanks & Regards,* > > *Abhishek* > > *Cloudbyte Inc. <http://www.cloudbyte.com>* > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- *Thanks & Regards,* *Abhishek* *Cloudbyte Inc. <http://www.cloudbyte.com>*
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev