Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Thomas Goirand 
Hi Monty,

Thanks for your reply (and your intention to fix this).

On 06/30/2013 03:26 AM, Monty Taylor wrote:
> Are you seeing problems with anything other than
> glance-manage?

As much as I can see, the only other unit tests that failed for b1 in
Debian are:
glance.tests.unit.test_notifier.TestRabbitContentType.test_content_type_passed
glance.tests.unit.v1.test_api.TestGlanceAPI.test_delete_pending_delete_image

The first one might be because I'm not up-to-date with python modules on
my build env, and I'm currently working on fixing that. kombu comes to
mind, but this one is waiting for python-amqp to pass the FTP masters
NEW queue (and this might take up to a month). In the mean time, I have
more python module updates to do for unit testings (FYI, I have just
uploaded flake8 after the maintainer of pyflakes uploaded 0.7 in Sid:
flake8 is also waiting for approval in the NEW queue).

On 06/30/2013 08:05 AM, Monty Taylor wrote:
> I looked at it for a bit this afternoon before realizing it was
> Saturday. :)

I wasn't expecting an answer before Monday! :)

Thomas

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Monty Taylor 


On 06/29/2013 08:02 PM, John Bresnahan wrote:
>>
>> I believe we should fix the unittests to not shell out to glance-manage
>> in that manner. One of the nice things about moving the code from bin/
>> to glance.cmd is that it's available inside of the source tree for
>> unittests! :) What we want to do is call the glance.cmd.manage:main()
>> function directly. So, for instance, in:
>>
>> glance/tests/functional/__init__.py
>> and
>> glance/tests/functional/test_glance_manage.py
>>
>> We can call the function itself, rather than the shell wrapper.
> 
> That was the original intention and major motivation for the patch.
> However the configuration information is globally shared in the process
> space so pushing this through gave me pause.

The global config context is pissing me off enough that I might have to
break down and fix oslo.config too.

>> I'll make a patch. 
> 
> excellent!  thanks!

My pleasure!

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Monty Taylor 


On 06/29/2013 06:15 PM, Mark Washenberger wrote:
> Munging together some replies:
> 
> 
> On Sat, Jun 29, 2013 at 12:26 PM, Monty Taylor  > wrote:
> 
> On 06/29/2013 02:14 PM, Thomas Goirand wrote:
> > On 06/30/2013 12:16 AM, Mark Washenberger wrote:
> >> Hi Thomas,
> >>
> >> We switched to using entrypoints for creating our binaries.
> >
> > Can you explain this a bit more?
> 
> 
> Very
> specifically, https://github.com/openstack/glance/blob/master/setup.py#L55
> 
> This means rather than keeping a /bin directory in our project, we're
> asking setup.py to generate scripts and install them in an appropriate
> location when setup.py install is run.
>  
> 
> >
> >> Unfortunately, this means you need to have the glance version
> >> "installed" in some sense in your path when you want to run those
> tests.
> >> This felt like a no-no when we made the change, but it worked fine
> >> without any changes to our normal tox and run_tests.sh (with
> virtualenv)
> >> so it made it through. Please let me know if this is really hard
> to deal
> >> with on your side; I can imagine a few other possible fixes we
> might pursue.
> >>
> >> Thanks
> >
> > Using virtualenv is indeed not an option on the packaging side. Having
> > glance build-depends: on itself is also not an option.
> >
> > What can I do?
> 
> 
> Just for future reference for me, what kinds of dependencies can you
> establish for your "test" step in maintaining your package repos? Do you
> pretty much only have access to a source checkout / tarball? And is the
> "test" step completely contained within the "build" step?
>  
> 
> 
> I believe we should fix the unittests to not shell out to glance-manage
> in that manner. One of the nice things about moving the code from bin/
> to glance.cmd is that it's available inside of the source tree for
> unittests! :) What we want to do is call the glance.cmd.manage:main()
> function directly. So, for instance, in:
> 
> glance/tests/functional/__init__.py
> and
> glance/tests/functional/test_glance_manage.py
> 
> We can call the function itself, rather than the shell wrapper.
> 
> I'll make a patch. Are you seeing problems with anything other than
> glance-manage?
> 
> 
> Thanks Monty! This was exactly what I had in mind as an alternative
> approach. I believe glance-manage is the only binary where glance had
> such an install dependency for testing--but I can't answer for Thomas.

I looked at it for a bit this afternoon before realizing it was
Saturday. :) As best I can tell, glance-manage should be the only problem.

While you're waiting on that patch Thomas:

http://bazaar.launchpad.net/~openstack-ubuntu-testing/glance/havana/view/head:/debian/rules

The ubuntu package is dealing with it like this:

ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
override_dh_auto_test:
#   nosetests -d glance/tests/unit || true
endif

Not idea long term- but given the MASSIVE amount of automated unittest
running that happens on the glance tree, it's probably not the _worst_
workaround.

Monty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread John Bresnahan 
> 
> I believe we should fix the unittests to not shell out to glance-manage
> in that manner. One of the nice things about moving the code from bin/
> to glance.cmd is that it's available inside of the source tree for
> unittests! :) What we want to do is call the glance.cmd.manage:main()
> function directly. So, for instance, in:
> 
> glance/tests/functional/__init__.py
> and
> glance/tests/functional/test_glance_manage.py
> 
> We can call the function itself, rather than the shell wrapper.

That was the original intention and major motivation for the patch.
However the configuration information is globally shared in the process
space so pushing this through gave me pause.

> I'll make a patch. 

excellent!  thanks!

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Mark Washenberger 
Munging together some replies:


On Sat, Jun 29, 2013 at 12:26 PM, Monty Taylor  wrote:

> On 06/29/2013 02:14 PM, Thomas Goirand wrote:
> > On 06/30/2013 12:16 AM, Mark Washenberger wrote:
> >> Hi Thomas,
> >>
> >> We switched to using entrypoints for creating our binaries.
> >
> > Can you explain this a bit more?
>

Very specifically,
https://github.com/openstack/glance/blob/master/setup.py#L55

This means rather than keeping a /bin directory in our project, we're
asking setup.py to generate scripts and install them in an appropriate
location when setup.py install is run.


> >
> >> Unfortunately, this means you need to have the glance version
> >> "installed" in some sense in your path when you want to run those tests.
> >> This felt like a no-no when we made the change, but it worked fine
> >> without any changes to our normal tox and run_tests.sh (with virtualenv)
> >> so it made it through. Please let me know if this is really hard to deal
> >> with on your side; I can imagine a few other possible fixes we might
> pursue.
> >>
> >> Thanks
> >
> > Using virtualenv is indeed not an option on the packaging side. Having
> > glance build-depends: on itself is also not an option.
> >
> > What can I do?
>

Just for future reference for me, what kinds of dependencies can you
establish for your "test" step in maintaining your package repos? Do you
pretty much only have access to a source checkout / tarball? And is the
"test" step completely contained within the "build" step?


>
> I believe we should fix the unittests to not shell out to glance-manage
> in that manner. One of the nice things about moving the code from bin/
> to glance.cmd is that it's available inside of the source tree for
> unittests! :) What we want to do is call the glance.cmd.manage:main()
> function directly. So, for instance, in:
>
> glance/tests/functional/__init__.py
> and
> glance/tests/functional/test_glance_manage.py
>
> We can call the function itself, rather than the shell wrapper.
>
> I'll make a patch. Are you seeing problems with anything other than
> glance-manage?
>

Thanks Monty! This was exactly what I had in mind as an alternative
approach. I believe glance-manage is the only binary where glance had such
an install dependency for testing--but I can't answer for Thomas.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Monty Taylor 
On 06/29/2013 02:14 PM, Thomas Goirand wrote:
> On 06/30/2013 12:16 AM, Mark Washenberger wrote:
>> Hi Thomas,
>>
>> We switched to using entrypoints for creating our binaries.
> 
> Can you explain this a bit more?
> 
>> Unfortunately, this means you need to have the glance version
>> "installed" in some sense in your path when you want to run those tests.
>> This felt like a no-no when we made the change, but it worked fine
>> without any changes to our normal tox and run_tests.sh (with virtualenv)
>> so it made it through. Please let me know if this is really hard to deal
>> with on your side; I can imagine a few other possible fixes we might pursue.
>>
>> Thanks
> 
> Using virtualenv is indeed not an option on the packaging side. Having
> glance build-depends: on itself is also not an option.
> 
> What can I do?

I believe we should fix the unittests to not shell out to glance-manage
in that manner. One of the nice things about moving the code from bin/
to glance.cmd is that it's available inside of the source tree for
unittests! :) What we want to do is call the glance.cmd.manage:main()
function directly. So, for instance, in:

glance/tests/functional/__init__.py
and
glance/tests/functional/test_glance_manage.py

We can call the function itself, rather than the shell wrapper.

I'll make a patch. Are you seeing problems with anything other than
glance-manage?

Monty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Thomas Goirand 
On 06/30/2013 12:16 AM, Mark Washenberger wrote:
> Hi Thomas,
> 
> We switched to using entrypoints for creating our binaries.

Can you explain this a bit more?

> Unfortunately, this means you need to have the glance version
> "installed" in some sense in your path when you want to run those tests.
> This felt like a no-no when we made the change, but it worked fine
> without any changes to our normal tox and run_tests.sh (with virtualenv)
> so it made it through. Please let me know if this is really hard to deal
> with on your side; I can imagine a few other possible fixes we might pursue.
> 
> Thanks

Using virtualenv is indeed not an option on the packaging side. Having
glance build-depends: on itself is also not an option.

What can I do?

Thomas

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Build-time unit tests failing when building glance 2013.2.b1

2013-06-29 Thread Mark Washenberger 
Hi Thomas,

We switched to using entrypoints for creating our binaries. Unfortunately,
this means you need to have the glance version "installed" in some sense in
your path when you want to run those tests. This felt like a no-no when we
made the change, but it worked fine without any changes to our normal tox
and run_tests.sh (with virtualenv) so it made it through. Please let me
know if this is really hard to deal with on your side; I can imagine a few
other possible fixes we might pursue.

Thanks


On Sat, Jun 29, 2013 at 3:07 AM, Thomas Goirand  wrote:

> Hi,
>
> I got a bunch of unit test failures with the output:
> STDERR: /bin/sh: 1: glance-manage: not found
>
> glance-manage used to be in bin/ in the sources (at least until
> Grizzly), though this changed for Havana. I used "find" to search for
> it, and couldn't find it in the source code, even though it really is
> installed at the correct place after my Debian package is built.
>
> Where has it been moved? How to fix the unit tests?
>
> Cheers,
>
> Thomas Goirand (zigo)
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev