Hi Ruben, I left a comment on one of the changes; after you take care of that I'll take a closer look at the code. Let me know if you have questions.
Tim On Tue, Nov 29, 2016 at 4:06 AM Ruben <r.manganiel...@studenti.unisa.it> wrote: > Hi Tim, > I've added the code of magnum_driver and its unit test to review. > It seems everything works. > > Ruben > > ----- Original Message ----- > From: "Tim Hinrichs" <t...@styra.com> > To: "Ruben" <r.manganiel...@studenti.unisa.it> > Cc: openstack-dev@lists.openstack.org, "timothy l hinrichs" < > timothy.l.hinri...@gmail.com> > Sent: Saturday, November 26, 2016 12:48:12 AM > Subject: Re: [Congress] Magnum_driver > > Definitely push that code up into Gerrit so we can all take a look. Data > like pods and containers is probably the most valuable data from Magnum, so > I'd definitely recommend adding that. But push the code you have to Gerrit > first. (As long as you leave the ChangeId the same each time you push to > Gerrit, Gerrit will keep all of the versions you pushed organized together, > yet keep the versions separate.) > > Tim > > On Fri, Nov 25, 2016 at 3:06 PM Ruben <r.manganiel...@studenti.unisa.it> > wrote: > > > Hi Tim, > > You are great. It works! Thanks a lot! > > I've also solved the problem with py27. The unit test seems to work. > > The only thing that seems not to work is populate the 'clusters_links' > and > > 'cluster_templates_links' tables: they are empty. > > Also, the 'labels' table is empty. > > I've no errors anyway. > > Are these problems according to you? > > > > Should I to try to add the translation of pods, containers and services? > > > > I've add the code to review. > > > > Ruben > > ----- Original Message ----- > > From: "Tim Hinrichs" <t...@styra.com> > > To: "Ruben" <r.manganiel...@studenti.unisa.it> > > Cc: openstack-dev@lists.openstack.org, "timothy l hinrichs" < > > timothy.l.hinri...@gmail.com> > > Sent: Friday, November 25, 2016 10:36:29 PM > > Subject: Re: [Congress] Magnum_driver > > > > Hi Ruben, > > > > Glad you got that worked out. Once in a while I end up deleting my .tox > > dir because it gets out of date. I guess that's what the --recreate > option > > to tox does. So I learned something. :) > > > > The new error message looks to be saying that an object of type 'Cluster' > > cannot be iterated. Congress's _translate_clusters method assumes that > > what you give it is basically JSON (arrays, dictionaries, numbers, > strings, > > bools). In this case Congress is trying to iterate over the keys of a > > dictionary but is getting a 'Cluster' Python object instead. What's > > happening is that when you do the cluster.list() (or whatever it's > called) > > on the magnum-python you're getting back a list of Cluster Python objects > > instead of a list of dictionaries. > > > > To fix the problem, you'll want to take the results of cluster.list() and > > translate it into a list of dictionaries, and then hand that off to > > _translate_clusters. Probably you'll need to do the same for the > > cluster_template. So instead of ... > > > > clusters_method = lambda: self._translate_clusters( > > {'clusters': self.magnum.clusters.list()}) > > > > You'll want to do something like ... > > > > clusters_method = lambda: self._translate_clusters( > > {'clusters': [x.__dict__ for x in > self.magnum.clusters.list()] > > }) > > > > Or at least, that's what I'd start trying. __dict__ grabs all the fields > > of an object and returns a dictionary. (If some of the values of that > > dictionary are also Python objects, then you might need something that's > > more complicated to recursively walk the result of clusters.list() and > > translate everything into lists and dictionaries. And I'm not sure that > > __dict__ will give you exactly what you want, but it's worth a try.). > > > > Tim > > > > > > On Thu, Nov 24, 2016 at 12:11 PM Ruben <r.manganiel...@studenti.unisa.it > > > > wrote: > > > > > Hi Tim, > > > I solved the problem with: > > > > > > tox --recreate -e py27 > > > > > > Now I no have the error on the import. The unit test still has some > > > errors, but I don't know why.. > > > > > > Anyway, I've this error when I try to add the magnum_driver: > > > > > > 2016-11-24 20:56:27.191 INFO congress.datasources.datasource_driver [-] > > > magnum:: polling > > > 2016-11-24 20:56:27.192 DEBUG congress.datasources.datasource_driver > [-] > > > update table clusters. from (pid=18720) update_from_datasource > > > /opt/stack/congress/congress/datasources/datasource_driver.py:1370 > > > 2016-11-24 20:56:27.427 DEBUG congress.datasources.magnum_driver [-] > > > CLUSTERS: {'clusters': [<Clusters {u'status': u'CREATE_FAILED', > > > u'cluster_template_id': u'8d25a1ed-faa6-4305-a6a1-6559708c805b', > u'uuid': > > > u'1634beb9-25de-4cdd-bafa-67537069f0cc', u'links': [{u'href': u' > > > http://10.0.2.15:9511/v1/clusters/1634beb9-25de-4cdd-bafa-67537069f0cc > ', > > > u'rel': u'self'}, {u'href': u' > > > http://10.0.2.15:9511/clusters/1634beb9-25de-4cdd-bafa-67537069f0cc', > > > u'rel': u'bookmark'}], u'stack_id': > > > u'17f1248d-286a-4fd4-9639-af5773670f03', u'master_count': 1, > u'keypair': > > > u'testkey', u'node_count': 1, u'create_timeout': 60, u'name': > > > u'k8s-cluster'}>]} from (pid=18720) _translate_clusters > > > /opt/stack/congress/congress/datasources/magnum_driver.py:165 > > > 2016-11-24 20:56:27.435 ERROR congress.datasources.datasource_driver > [-] > > > Datasource driver raised exception > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > Traceback (most recent call last): > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 1384, in poll > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > self.update_from_datasource() # sets self.state > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 1371, in update_from_datasource > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > self.update_methods[registered_table]() > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/magnum_driver.py", line > > > 150, in <lambda> > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > {'clusters': self.magnum.clusters.list()}) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_utils.py", > > line > > > 57, in inner > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > result = f(self, raw_data, *args, **kw) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/magnum_driver.py", line > > > 167, in _translate_clusters > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > obj['clusters'], MagnumDriver.clusters_translator) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 1048, in convert_objs > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > rows, _ = DataSourceDriver.convert_obj(o, translator) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 1021, in convert_obj > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > parent_row_dict) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 994, in _populate_translator_data_hdict > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > return cls._populate_hdict(translator, obj, parent_row_dict) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 924, in _populate_hdict > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > cls._get_value(obj, field, selector), extract_fn) > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > File "/opt/stack/congress/congress/datasources/datasource_driver.py", > > line > > > 693, in _get_value > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > if field in o: > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > TypeError: argument of type 'Cluster' is not iterable > > > 2016-11-24 20:56:27.435 TRACE congress.datasources.datasource_driver > > > 2016-11-24 20:56:27.435 INFO congress.datasources.datasource_driver [-] > > > magnum:: finished polling > > > > > > > > > Any suggestions to solve it? > > > I've add the code to review. > > > > > > Ruben > > > > > > ----- Original Message ----- > > > From: "Ruben" <r.manganiel...@studenti.unisa.it> > > > To: "Tim Hinrichs" <t...@styra.com> > > > Cc: openstack-dev@lists.openstack.org, "timothy l hinrichs" < > > > timothy.l.hinri...@gmail.com> > > > Sent: Thursday, November 24, 2016 12:44:38 AM > > > Subject: Re: [Congress] Magnum_driver > > > > > > Hi Tim, > > > I already have 'pyhton-magnumclient' in the requirements.txt file, but > I > > > still have the error. > > > > > > This is the /opt/stack/congress/requirements.txt file: > > > > > > "# The order of packages is significant, because pip processes them in > > the > > > order > > > # of appearance. Changing the order has an impact on the overall > > > integration > > > # process, which may cause wedges in the gate later. > > > > > > Babel>=2.3.4 # BSD > > > eventlet!=0.18.3,>=0.18.2 # MIT > > > PuLP>=1.4.1 # MIT > > > keystoneauth1>=2.14.0 # Apache-2.0 > > > keystonemiddleware!=4.5.0,>=4.2.0 # Apache-2.0 > > > Paste # MIT > > > PasteDeploy>=1.5.0 # MIT > > > pbr>=1.6 # Apache-2.0 > > > python-keystoneclient>=3.6.0 # Apache-2.0 > > > python-heatclient>=1.5.0 # Apache-2.0 > > > python-magnumclient > > > python-muranoclient>=0.8.2 # Apache-2.0 > > > python-novaclient!=2.33.0,>=2.29.0 # Apache-2.0 > > > python-neutronclient>=5.1.0 # Apache-2.0 > > > python-ceilometerclient>=2.5.0 # Apache-2.0 > > > python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0 > > > python-swiftclient>=2.2.0 # Apache-2.0 > > > python-ironicclient>=1.6.0 # Apache-2.0 > > > alembic>=0.8.4 # MIT > > > python-dateutil>=2.4.2 # BSD > > > python-glanceclient>=2.5.0 # Apache-2.0 > > > Routes!=2.0,!=2.1,!=2.3.0,>=1.12.3;python_version=='2.7' # MIT > > > Routes!=2.0,!=2.3.0,>=1.12.3;python_version!='2.7' # MIT > > > six>=1.9.0 # MIT > > > oslo.concurrency>=3.8.0 # Apache-2.0 > > > oslo.config!=3.18.0,>=3.14.0 # Apache-2.0 > > > oslo.context>=2.9.0 # Apache-2.0 > > > oslo.db!=4.13.1,!=4.13.2,>=4.11.0 # Apache-2.0 > > > oslo.messaging>=5.2.0 # Apache-2.0 > > > oslo.policy>=1.15.0 # Apache-2.0 > > > oslo.serialization>=1.10.0 # Apache-2.0 > > > oslo.service>=1.10.0 # Apache-2.0 > > > oslo.utils>=3.18.0 # Apache-2.0 > > > oslo.middleware>=3.0.0 # Apache-2.0 > > > oslo.vmware>=2.11.0 # Apache-2.0 > > > oslo.log>=3.11.0 # Apache-2.0 > > > WebOb>=1.6.0 # MIT" > > > > > > ----- Original Message ----- > > > From: "Tim Hinrichs" <t...@styra.com> > > > To: "Ruben" <r.manganiel...@studenti.unisa.it>, > > > openstack-dev@lists.openstack.org > > > Cc: "timothy l hinrichs" <timothy.l.hinri...@gmail.com> > > > Sent: Wednesday, November 23, 2016 11:44:40 PM > > > Subject: Re: [Congress] Magnum_driver > > > > > > Ruben, > > > > > > All the software that gets imported by your code needs to be listed in > > > requirements.txt so that when tox runs, it installs that software. > When > > I > > > did some debugging for you I made the necessary change, so if you add > > that > > > back into your change, that error should disappear. > > > > > > https://review.openstack.org/#/c/397150/ > > > > > > Tim > > > > > > > > > On Wed, Nov 23, 2016 at 12:29 PM Ruben < > r.manganiel...@studenti.unisa.it > > > > > > wrote: > > > > > > > Hi everybody, > > > > I've trying to run the unit test of the magnum_driver. > > > > > > > > I make: > > > > -cd /opt/stack/congress/ > > > > -tox -epy27 > > > > > > > > but I have error with the import. > > > > This is the output: > > > > > > > > "py27 develop-inst-noop: /opt/stack/congress > > > > py27 installed: > > > > > > > > > > alabaster==0.7.9,alembic==0.8.8,amqp==1.4.9,anyjson==0.3.3,appdirs==1.4.0,Babel==2.3.4,cachetools==2.0.0,cffi==1.9.1,cliff==2.2.0,cmd2==0.6.9,-e > > > > git+ > > > > > > > > > > http://git.openstack.org/openstack/congress@b2d96b56f721c941e85db565d203df008c455b19#egg=congress,contextlib2==0.5.4,coverage==4.2,cryptography==1.5.3,debtcollector==1.9.0,decorator==4.0.10,docutils==0.12,dogpile.cache==0.6.2,enum34==1.1.6,eventlet==0.19.0,extras==1.0.0,fasteners==0.14.1,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,functools32==3.2.3.post2,futures==3.0.5,futurist==0.19.0,greenlet==0.4.10,hacking==0.10.2,idna==2.1,ipaddress==1.0.17,iso8601==0.1.11,Jinja2==2.8,jsonpatch==1.14,jsonpointer==1.10,jsonschema==2.5.1,keystoneauth1==2.15.0,keystonemiddleware==4.10.0,kombu==3.0.37,linecache2==1.0.0,lxml==3.6.4,Mako==1.0.6,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,monotonic==1.2,mox3==0.18.0,msgpack-python==0.4.8,netaddr==0.7.18,netifaces==0.10.5,openstacksdk==0.9.9,os-client-config==1.22.0,osc-lib==1.2.0,oslo.concurrency==3.15.0,oslo.config==3.19.0,oslo.context==2.10.0,oslo.db==4.14.0,oslo.i18n==3.10.0,oslo.log==3.17.0,oslo.messaging==5.12.0,oslo.middleware==3.20.0,oslo.policy==1.16.0,oslo.serialization==2.14.0,oslo.service==1.17.0,oslo.utils==3.18.0,oslo.vmware==2.15.0,oslosphinx==4.8.0,oslotest==2.11.0,Paste==2.0.3,PasteDeploy==1.5.2,pbr==1.10.0,pep8==1.5.7,pika==0.10.0,pika-pool==0.1.3,ply==3.9,positional==1.1.1,prettytable==0.7.2,PuLP==1.6.1,pyasn1==0.1.9,pycadf==2.4.0,pycparser==2.17,pyflakes==0.8.1,Pygments==2.1.3,pyinotify==0.9.6,pyOpenSSL==16.2.0,pyparsing==1.5.7,python-ceilometerclient==2.7.0,python-cinderclient==1.9.0,python-dateutil==2.6.0,python-editor==1.0.1,python-glanceclient==2.5.0,python-heatclient==1.5.0,python-ironicclient==1.8.0,python-keystoneclient==3.6.0,python-mimeparse==1.6.0,python-muranoclient==0.11.1,python-neutronclient==6.0.0,python-novaclient==6.0.0,python-openstackclient==3.3.0,python-subunit==1.2.0,python-swiftclient==3.1.0,pytz==2016.7,PyYAML==3.12,reno==1.8.0,repoze.lru==0.6,requests==2.11.1,requests-mock==1.1.0,requestsexceptions==1.1.3,retrying==1.3.3,rfc3986==0.4.1,Routes==2.3.1,simplejson==3.10.0,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,SQLAlchemy==1.1.3,sqlalchemy-migrate==0.10.0,sqlparse==0.2.2,stevedore==1.18.0,suds-jurko==0.6,Tempita==0.5.2,tenacity==3.3.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unicodecsv==0.14.1,unittest2==1.1.0,urllib3==1.19,warlock==1.2.0,WebOb==1.6.2,wrapt==1.10.8,yaql==1.1.1 > > > > py27 > > > > < > > > > > > http://git.openstack.org/openstack/congress@b2d96b56f721c941e85db565d203df008c455b19#egg=congress,contextlib2==0.5.4,coverage==4.2,cryptography==1.5.3,debtcollector==1.9.0,decorator==4.0.10,docutils==0.12,dogpile.cache==0.6.2,enum34==1.1.6,eventlet==0.19.0,extras==1.0.0,fasteners==0.14.1,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,functools32==3.2.3.post2,futures==3.0.5,futurist==0.19.0,greenlet==0.4.10,hacking==0.10.2,idna==2.1,ipaddress==1.0.17,iso8601==0.1.11,Jinja2==2.8,jsonpatch==1.14,jsonpointer==1.10,jsonschema==2.5.1,keystoneauth1==2.15.0,keystonemiddleware==4.10.0,kombu==3.0.37,linecache2==1.0.0,lxml==3.6.4,Mako==1.0.6,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,monotonic==1.2,mox3==0.18.0,msgpack-python==0.4.8,netaddr==0.7.18,netifaces==0.10.5,openstacksdk==0.9.9,os-client-config==1.22.0,osc-lib==1.2.0,oslo.concurrency==3.15.0,oslo.config==3.19.0,oslo.context==2.10.0,oslo.db==4.14.0,oslo.i18n==3.10.0,oslo.log==3.17.0,oslo.messaging==5.12.0,oslo.middleware==3.20.0,oslo.policy==1.16.0,oslo.serialization==2.14.0,oslo.service==1.17.0,oslo.utils==3.18.0,oslo.vmware==2.15.0,oslosphinx==4.8.0,oslotest==2.11.0,Paste==2.0.3,PasteDeploy==1.5.2,pbr==1.10.0,pep8==1.5.7,pika==0.10.0,pika-pool==0.1.3,ply==3.9,positional==1.1.1,prettytable==0.7.2,PuLP==1.6.1,pyasn1==0.1.9,pycadf==2.4.0,pycparser==2.17,pyflakes==0.8.1,Pygments==2.1.3,pyinotify==0.9.6,pyOpenSSL==16.2.0,pyparsing==1.5.7,python-ceilometerclient==2.7.0,python-cinderclient==1.9.0,python-dateutil==2.6.0,python-editor==1.0.1,python-glanceclient==2.5.0,python-heatclient==1.5.0,python-ironicclient==1.8.0,python-keystoneclient==3.6.0,python-mimeparse==1.6.0,python-muranoclient==0.11.1,python-neutronclient==6.0.0,python-novaclient==6.0.0,python-openstackclient==3.3.0,python-subunit==1.2.0,python-swiftclient==3.1.0,pytz==2016.7,PyYAML==3.12,reno==1.8.0,repoze.lru==0.6,requests==2.11.1,requests-mock==1.1.0,requestsexceptions==1.1.3,retrying==1.3.3,rfc3986==0.4.1,Routes==2.3.1,simplejson==3.10.0,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,SQLAlchemy==1.1.3,sqlalchemy-migrate==0.10.0,sqlparse==0.2.2,stevedore==1.18.0,suds-jurko==0.6,Tempita==0.5.2,tenacity==3.3.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unicodecsv==0.14.1,unittest2==1.1.0,urllib3==1.19,warlock==1.2.0,WebOb==1.6.2,wrapt==1.10.8,yaql==1.1.1py27 > > > > > > > > runtests: PYTHONHASHSEED='4235171505 <(423)%20517-1505> > <(423)%20517-1505> > > <(423)%20517-1505> > > > <(423)%20517-1505>' > > > > py27 runtests: commands[0] | find . -type f -name *.py[c|o] -delete > > > > py27 runtests: commands[1] | python setup.py testr --slowest > > > --testr-args= > > > > running testr > > > > running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \ > > > > OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \ > > > > OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \ > > > > ${PYTHON:-python} -m subunit.run discover -t ./ ./congress/tests > --list > > > > --- import errors --- > > > > Failed to import test module: > > > congress.tests.datasources.test_magnum_driver > > > > Traceback (most recent call last): > > > > File > > > > > > > > > > "/opt/stack/congress/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", > > > > line 456, in _find_test_path > > > > module = self._get_module_from_name(name) > > > > File > > > > > > > > > > "/opt/stack/congress/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", > > > > line 395, in _get_module_from_name > > > > __import__(name) > > > > File "congress/tests/datasources/test_magnum_driver.py", line 18, > in > > > > <module> > > > > from congress.datasources import magnum_driver > > > > File "congress/datasources/magnum_driver.py", line 16, in <module> > > > > from magnumclient import client as magnum_client > > > > ImportError: No module named magnumclient > > > > Non-zero exit code (2) from test listing. > > > > error: testr failed (3) > > > > ERROR: InvocationError: '/opt/stack/congress/.tox/py27/bin/python > > > setup.py > > > > testr --slowest --testr-args=' > > > > > > > > > > __________________________________________________________________________________________________ > > > > summary > > > > > > > > > > __________________________________________________________________________________________________ > > > > ERROR: py27: commands failed" > > > > > > > > > > > > Pep8 is ok. > > > > What should I do to solve the error above? > > > > > > > > Anyway I've errors with the translators.. > > > > I've add the code to review. > > > > > > > > Ruben > > > > > > > > > > > > > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev