Re: [openstack-dev] [All] Fixed recent gate issues
> https://review.openstack.org/74451 doesn't solve the issue completely, we have > SKIP_EXERCISES=boot_from_volume,bundle,client-env,euca,swift,client-args > but failure is now in Grenade's Javelin script: > > + swift upload javelin /etc/hosts > ...(same Traceback)... > [ERROR] /opt/stack/new/grenade/setup-javelin:151 Swift upload failed What about just removing that test from Javelin in stable/havana? https://review.openstack.org/76058 Cheers, Alan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Fixed recent gate issues
2014-02-23 10:52 GMT+01:00 Gary Kotton : > It looks like this does not solve the issue. Yeah https://review.openstack.org/74451 doesn't solve the issue completely, we have SKIP_EXERCISES=boot_from_volume,bundle,client-env,euca,swift,client-args but failure is now in Grenade's Javelin script: + swift upload javelin /etc/hosts ...(same Traceback)... [ERROR] /opt/stack/new/grenade/setup-javelin:151 Swift upload failed > I wonder if we need the same change for stable/havana. devstack-gate master branch handles all projects branches, patch above was inside if "stable/grizzly" Cheers, Alan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Fixed recent gate issues
On 2/19/14 10:31 PM, "Alan Pevec" wrote: >> Yeah it's pip weirdness where things falls apart because of version >>cap. It's >> basically installing bin/swift from 1.9 when it sees the version >>requirement >> but it leaves everything in python-swiftclient namespace from master. >> >> So I've actually been looking at this since late yesterday the >>conclusion we've >> reached is to just skip the exercises on grizzly. Removing the version >>cap isn't >> going to be simple on grizzly because there global requirements wasn't >>enforced >> back in grizzly. We'd have to change the requirement for both glance, >>horizon, >> and swift and being ~3 weeks away from eol for grizzly I don't think we >>should >> mess with that. This failure is only an issue with cli swiftclient on >>grizzly >> (and one swift functional test) which as it sits now is just the >>devstack >> exercises on grenade. So if we just don't run those exercises on the >>grizzly >> side of a grenade run there shouldn't be an issue. I've got 2 patches >>to do >> this here: >> >> >>https://urldefense.proofpoint.com/v1/url?u=https://review.openstack.org/% >>23/c/74419/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=eH0pxTUZo8NPZyF6hgoMQu% >>2BfDtysg45MkPhCZFxPEq8%3D%0A&m=PlTk6Hx5SymtM%2BlXLUIqyNGMz3z9JFKMcy7XJRvm >>k6s%3D%0A&s=591c1b105d86204158b21bd572a7f589f8ad032359282b84da9bc2b649d1a >>3f9 >> >> >>https://urldefense.proofpoint.com/v1/url?u=https://review.openstack.org/% >>23/c/74451/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=eH0pxTUZo8NPZyF6hgoMQu% >>2BfDtysg45MkPhCZFxPEq8%3D%0A&m=PlTk6Hx5SymtM%2BlXLUIqyNGMz3z9JFKMcy7XJRvm >>k6s%3D%0A&s=d531e1933102045d5b77138fc42bd8e68ddc3d5b25dd281d94bfb0e9a22b9 >>a64 > >Looks like only the latter is needed, devstack-gate core please >approve it to unblock stable/havana. It looks like this does not solve the issue. I wonder if we need the same change for stable/havana. > >Cheers, >Alan > >___ >OpenStack-dev mailing list >OpenStack-dev@lists.openstack.org >https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi- >bin/mailman/listinfo/openstack-dev&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=e >H0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0A&m=PlTk6Hx5SymtM%2BlXLUI >qyNGMz3z9JFKMcy7XJRvmk6s%3D%0A&s=8a9c304e78b852d9eaa9063697282ccd38a303300 >54aebe635797a8bce793bbe ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Fixed recent gate issues
> Yeah it's pip weirdness where things falls apart because of version cap. It's > basically installing bin/swift from 1.9 when it sees the version requirement > but it leaves everything in python-swiftclient namespace from master. > > So I've actually been looking at this since late yesterday the conclusion > we've > reached is to just skip the exercises on grizzly. Removing the version cap > isn't > going to be simple on grizzly because there global requirements wasn't > enforced > back in grizzly. We'd have to change the requirement for both glance, horizon, > and swift and being ~3 weeks away from eol for grizzly I don't think we should > mess with that. This failure is only an issue with cli swiftclient on grizzly > (and one swift functional test) which as it sits now is just the devstack > exercises on grenade. So if we just don't run those exercises on the grizzly > side of a grenade run there shouldn't be an issue. I've got 2 patches to do > this here: > > https://review.openstack.org/#/c/74419/ > > https://review.openstack.org/#/c/74451/ Looks like only the latter is needed, devstack-gate core please approve it to unblock stable/havana. Cheers, Alan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Fixed recent gate issues
On Tue, Feb 18, 2014 at 08:27:16PM +0100, Alan Pevec wrote: > Hi John, > > thanks for the summary. > I've noticed one more fall out from swiftclient update in Grenade jobs > running on stable/havana changes e.g. > http://logs.openstack.org/02/73402/1/check/check-grenade-dsvm/a5650ac/console.html > ... > 2014-02-18 13:00:02.103 | Test Swift > 2014-02-18 13:00:02.103 | + swift --os-tenant-name=demo > --os-username=demo --os-password=secret > --os-auth-url=http://127.0.0.1:5000/v2.0 stat > 2014-02-18 13:00:02.284 | Traceback (most recent call last): > 2014-02-18 13:00:02.284 | File "/usr/local/bin/swift", line 35, in > 2014-02-18 13:00:02.284 | from swiftclient import Connection, > HTTPException > 2014-02-18 13:00:02.285 | ImportError: cannot import name HTTPException > 2014-02-18 13:00:02.295 | + STATUS_SWIFT=Failed > ... > > Grenade job installs swiftclient from git master but then later due to > python-swiftclient>=1.2,<2 requirement in Grizzly, older version 1.9.0 > is pulled from pypi and then half-installed or something, producing > above conflict between swift CLI binary and libs. > Solution could be to remove swiftclient cap in Grizzly, any other suggestions? Yeah it's pip weirdness where things falls apart because of version cap. It's basically installing bin/swift from 1.9 when it sees the version requirement but it leaves everything in python-swiftclient namespace from master. So I've actually been looking at this since late yesterday the conclusion we've reached is to just skip the exercises on grizzly. Removing the version cap isn't going to be simple on grizzly because there global requirements wasn't enforced back in grizzly. We'd have to change the requirement for both glance, horizon, and swift and being ~3 weeks away from eol for grizzly I don't think we should mess with that. This failure is only an issue with cli swiftclient on grizzly (and one swift functional test) which as it sits now is just the devstack exercises on grenade. So if we just don't run those exercises on the grizzly side of a grenade run there shouldn't be an issue. I've got 2 patches to do this here: https://review.openstack.org/#/c/74419/ https://review.openstack.org/#/c/74451/ -Matt Treinish ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Fixed recent gate issues
Hi John, thanks for the summary. I've noticed one more fall out from swiftclient update in Grenade jobs running on stable/havana changes e.g. http://logs.openstack.org/02/73402/1/check/check-grenade-dsvm/a5650ac/console.html ... 2014-02-18 13:00:02.103 | Test Swift 2014-02-18 13:00:02.103 | + swift --os-tenant-name=demo --os-username=demo --os-password=secret --os-auth-url=http://127.0.0.1:5000/v2.0 stat 2014-02-18 13:00:02.284 | Traceback (most recent call last): 2014-02-18 13:00:02.284 | File "/usr/local/bin/swift", line 35, in 2014-02-18 13:00:02.284 | from swiftclient import Connection, HTTPException 2014-02-18 13:00:02.285 | ImportError: cannot import name HTTPException 2014-02-18 13:00:02.295 | + STATUS_SWIFT=Failed ... Grenade job installs swiftclient from git master but then later due to python-swiftclient>=1.2,<2 requirement in Grizzly, older version 1.9.0 is pulled from pypi and then half-installed or something, producing above conflict between swift CLI binary and libs. Solution could be to remove swiftclient cap in Grizzly, any other suggestions? Cheers, Alan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Fixed recent gate issues
On 02/14/2014 02:15 PM, John Dickinson wrote: > As many of you surely noticed, we had some significant significant > gate issues in the last day. It's fixed now, and I've got the > details below. > > The root cause of the issue was a lack of proper testing in > python- swiftclient. We've made some improvements here in the last > few hours, but improving this will be a longer-term effort (and one > that is being prioritized). > > Here's what happened: In order to get support for TLS certificate > validation, swiftclient was ported to use the Python requests > library. This is a good change, overall, but it was merged with a > bug where the object data was uploaded as a multipart/form-data > instead of as the raw data itself. This issue was resolved with > patch https://review.openstack.org/#/c/73585/. The gate is > currently stable, everyone should be unblocked by this issue now. > If you have a patch that failed a check or gate run, you should > recheck/reverify with bug #1280072. > > This, of course, raises the question of how this was allowed to > happen. First, there is a lack of functional testing in > swiftclient itself. (And, mea culpa, I should have done better > testing before I merged the initial breaking patch.) These tests > are being prioritized and worked on now. > > Second, python-swiftclient did not have a symmetric gate with the > other projects that depend upon it. Although the gate change to > make this happen was proposed quite a while ago, it wasn't merged > until just this morning (https://review.openstack.org/#/c/70378/). > Having these tests earlier should have caught the issues in the > original python-swiftclient patch. Now that it has landed, there is > much less risk of such a problem happening again. > > I want to thank Jeremy Stanley on the infra team for helping > getting these patches landed quickly. I'd also like to thank > Tristan Cacqueray for helping get the fixes written for > python-swiftclient. > > --John > > > > ___ OpenStack-dev > mailing list OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > Thank you for the comprehensive write up, John. Anita. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
