Hi Asselin, Abhishek,
Thanks for the reply. :)
On 07/19/2015 02:41 AM, Asselin, Ramy wrote:
HI Abhi & Tang,
Sorry I missed this thread. Let me know if you've resolved your issues.
My repo is undergoing migrations to reuse components in
openstack-infra/puppet-openstackci.
For single-use-nodes, the file you need has been removed here [1]: But
I see now that it is still needed, or a different function is needed
based on this version used by infra: [2]. I will explore a solution.
A couple other notes, please use ci-sandbox [3] instead of sandbox.
OK.
Zuul use behind a proxy: seems you got past this? Could you share your
solution?
The root cause is that zuul uses a library named paramiko to create
connection
with low level socket APIs. And it doesn't provide proxy functionary.
I tried to use tools like proxychain, to redirect the zuul connection to
go through
my proxy. But it doesn't work. If I use proxychain to run zuul service,
it doesn't
output anything to the log file, and the service will die soon.
I think, there are two solutions:
1. Add proxy functionality to paramiko.
2. Add proxy functionality to zuul. That means, maybe,
zuul does not use paramiko to create connection, but by itself.
Solution 1 is much simpler, so for now, I just modified the source code
of paramiko.
I'm using python-socksipy package, and modified
/usr/local/lib/python2.7/dist-packages/paramiko/client.py like this:
diff --git a/client.py b/client.py
index 15ff696..d7225ed 100644
--- a/client.py
+++ b/client.py
@@ -24,6 +24,7 @@ from binascii import hexlify
import getpass
import os
import socket
+import socks
import warnings
import pdb
@@ -235,6 +236,7 @@ class SSHClient (ClosingContextManager):
``gss_deleg_creds`` and ``gss_host`` arguments.
"""
+ """
if not sock:
for (family, socktype, proto, canonname, sockaddr) in
socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM):
if socktype == socket.SOCK_STREAM:
@@ -251,6 +253,13 @@ class SSHClient (ClosingContextManager):
except:
pass
retry_on_signal(lambda: sock.connect(addr))
+ """
+
+ if not sock:
+ sock = socks.socksocket()
+ sock.setproxy(socks.PROXY_TYPE_SOCKS5, "MY_PROXY_IP",
MY_PROXY_PORT, username="XXX", password="XXX")
# This is review.openstack.org
+ addr = ('104.130.159.134', 29418)
+ retry_on_signal(lambda: sock.connect(addr))
t = self._transport = Transport(sock, gss_kex=gss_kex,
gss_deleg_creds=gss_deleg_creds)
t.use_compression(compress=compress)
Of course, this is just a draft. It is only for my case, not for all.
BTW, I'm now working on Fujitsu CI System, and really want to join into
the development of openstack-infra.
I think the proxy functionality is necessary for many companies. So if
you are planing to add the proxy support,
I think I can help.
Thanks. :)
Also, feel free to join 3^rd party ci IRC meetings on freenode [4].
It's a great place to ask questions and meet others setting up or
maintaining these systems.
Thanks,
Ramy
IRC: asselin
[1]
https://github.com/rasselin/os-ext-testing/commit/dafe822be7813522a6c7361993169da20b37ffb7
[2]
https://github.com/openstack-infra/project-config/blob/master/zuul/openstack_functions.py
[3] http://git.openstack.org/cgit/openstack-dev/ci-sandbox/
[4] http://eavesdrop.openstack.org/#Third_Party_Meeting
*From:*Abhishek Shrivastava [mailto:abhis...@cloudbyte.com]
*Sent:* Monday, July 13, 2015 11:51 PM
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* Re: [openstack-dev] [CI] How to set a proxy for zuul.
Also if you want to change it you will need to talk with Asselin Ramy
who is the owner of the repo you followed.
On Tue, Jul 14, 2015 at 12:18 PM, Abhishek Shrivastava
<abhis...@cloudbyte.com <mailto:abhis...@cloudbyte.com>> wrote:
Basically it is not required, and if you see
/etc/jenkins_jobs/config folder you will find one
dsvm-cinder-tempest.yaml which is to be used basically not
examples.yaml. So its not an issue.
On Tue, Jul 14, 2015 at 12:07 PM, Tang Chen
<tangc...@cn.fujitsu.com <mailto:tangc...@cn.fujitsu.com>> wrote:
On 07/14/2015 01:46 PM, Abhishek Shrivastava wrote:
Instead of it use reusable_node option.
Thanks. Problem resolved. :)
BTW, single_use_node is written in layout.yaml by default.
If it doesn't exist anymore, do we need a patch to fix it ?
For someone who uses CI for the first time, it is really a
problem......
And also, if I want to post patch for zuul, where should I
post the patch ?
Thanks.
On Tue, Jul 14, 2015 at 9:12 AM, Tang Chen
<tangc...@cn.fujitsu.com <mailto:tangc...@cn.fujitsu.com>>
wrote:
Hi Abhishek, All,
I found the problem.
My /etc/zuul/layout/layout.yaml has the following config:
jobs:
- name: ^dsvm-tempest.*$
parameter-function: single_use_node
But in _parseConfig() in zuul/scheduler.py, it failed
to find single_use_node().
fname =
config_job.get('parameter-function', None)
if fname:
func = config_env.get(fname, None)
if not func:
raise Exception("Unable to find
function %s" % fname)
So projects section was not parsed.
Does anyone know why ?
Thanks.
On 07/14/2015 10:54 AM, Tang Chen wrote:
Hi Abhishek,
I printed the self.layout.projects in
zuul/scheduler.py, it is empty.
So the project was not found.
But I did do the *jenkins-jobs --flush-cache
update* */etc/jenkins_jobs/config/*
And I did configure openstack-dev/sandbox in
layout.yaml.
Do you have any idea what's wrong here ?
Thanks.
On 07/13/2015 05:58 PM, Tang Chen wrote:
On 07/13/2015 04:35 PM, Abhishek Shrivastava
wrote:
Updating jobs using "*sudo jenkins-jobs
--flush-cache update
/etc/jenkins_jobs/config/". *Also update
the myvendor in examples.yaml
Sorry, I updated the jobs, restart the whole
machine. But it still doesn't work.
By the way, there is no vendor in examples.yaml.
It is still this error: Project
openstack-dev/sandbox not found
Anything else should I pay attention to?
Thanks.
On Mon, Jul 13, 2015 at 1:45 PM, Tang Chen
<tangc...@cn.fujitsu.com
<mailto:tangc...@cn.fujitsu.com>> wrote:
On 07/13/2015 03:50 PM, Abhishek
Shrivastava wrote:
Use tester or something, also are
you updating the jobs or not?
I used tester as my vendor. It doesn't
work.
And what do you mean by updating the
jobs ? I built the
noop-check-cimmunitication
job once in Jenkins UI. Does it matter
? All the others are not touched.
And referring to the error, "Project
openstack-dev/sandbox not found", it
seems like
somewhere the project name was wrong.
right ?
Thanks.
On Mon, Jul 13, 2015 at 1:16 PM,
Tang Chen <tangc...@cn.fujitsu.com
<mailto:tangc...@cn.fujitsu.com>>
wrote:
Hi Abhishek,
Thanks for the quick reply.
On 07/13/2015 03:16 PM,
Abhishek Shrivastava wrote:
Also check that Gearman is
connecting or not through
Jenkins UI.
On Mon, Jul 13, 2015 at
12:45 PM, Abhishek
Shrivastava
<abhis...@cloudbyte.com
<mailto:abhis...@cloudbyte.com>>
wrote:
First of all, change
the "vendor" to your
vendor name in
/etc/jenkins_jobs/config/projects.yaml
file. Also, restart
the zuul and zuul merger.
I did the check. Gearman
plugin works find. In Jenkins
UI, I tested the connection,
and it succeeded.
And also, I restarted zuul and
zuul merger every time I
modified the yaml files.
But it doesn't work.
And the vendor, does that
matter ? And what vendor name
should I provide ?
I cannot find any vendor info
in my Gerrit service account
profile.
For example, is XXX OK ?
Thanks.
On Mon, Jul 13, 2015
at 12:29 PM, Tang Chen
<tangc...@cn.fujitsu.com
<mailto:tangc...@cn.fujitsu.com>>
wrote:
Hi all,
I have constructed
my CI system.
When I tested it
with sandbox
project, I posted
a patch to Gerrit.
https://review.openstack.org/201002
But I got this
error in zuul
scheduler:
2015-07-14
14:07:24,921 DEBUG
zuul.Scheduler:
Run handler awake
2015-07-14
14:07:24,921 DEBUG
zuul.Scheduler:
Fetching trigger event
2015-07-14
14:07:24,922 DEBUG
zuul.Scheduler:
Processing trigger
event
<TriggerEvent
comment-added
openstack-dev/sandbox
master 201002,1
Verified:1>
2015-07-14
14:07:24,922 DEBUG
zuul.Scheduler:
Project
openstack-dev/sandbox
not found
2015-07-14
14:07:24,922 DEBUG
zuul.Scheduler:
Run handler sleeping
My
/etc/zuul/layout/layout.yaml
looks like this:
projects:
- name:
openstack-dev/sandbox
check:
-
noop-check-communication
My
/etc/jenkins_jobs/config/projects.yaml
looks like this:
- project:
name: sandbox
github-org:
openstack-dev
node: master
vendor: myvendor
jobs:
-
noop-check-communication
-
dsvm-tempest-full:
node:
'devstack_slave ||
devstack-precise-check
|| d-p-c'
And Jenkins master
works fine.
Does anyone know
what is going on
here ?
Thanks.
__________________________________________________________________________
OpenStack
Development
Mailing List (not
for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc.
<http://www.cloudbyte.com>_/
--
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc.
<http://www.cloudbyte.com>_/
__________________________________________________________________________
OpenStack Development Mailing List
(not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing
List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc.
<http://www.cloudbyte.com>_/
__________________________________________________________________________
OpenStack Development Mailing List (not for
usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List
(not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
**
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc. <http://www.cloudbyte.com>_/
__________________________________________________________________________
OpenStack Development Mailing List (not for usage
questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage
questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage
questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
**
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc. <http://www.cloudbyte.com>_/
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
**
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc. <http://www.cloudbyte.com>_/
--
**
*Thanks & Regards,*
*Abhishek*
/_Cloudbyte Inc. <http://www.cloudbyte.com>_/
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
