Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Duncan McGreggor]

Adam, you've outdone yourself :-) That was a phenomenal reply with
tons of good information.

You've provided our team with all the info we need now -- thanks!

d

On Mon, Feb 27, 2012 at 10:36 PM, Adam Gandelman  wrote:
> On 02/27/2012 09:22 AM, Duncan McGreggor wrote:
>>
>> It's been an invaluable source for not only information, but also
>> planning for the cloud work here at DreamHost.
>
>
> This is great to hear!  I'm happy to hear other people are benefiting from
> this as much as Ubuntu
>
>
>>
>> To the point of this email, though, I have a question for you that I
>> wasn't able to parse an explicit answer to from your post: For the
>> packages that are built in the PPA linked above, are they only built
>> after all the components of OpenStack have been confirmed working as a
>> whole? Or are they built just after individual testing?
>
>
> The jenkins trunk build jobs are the ones responsible for uploading to the
> PPA.  It basically stuffs the newly built package in the local repository
> and uploads it to the PPA (if it fails to build, it doesnt upload).  The
> nova trunk build also triggers the deployment + testing.   So, to answer,
> those PPA packages are *pre* deployment + smoke testing.  The idea with that
> is that we can use the PPA externally to debug specific issues that might
> turn up during testing without blocking the deployment/smoke testing queue.
>  We're limited by the number of machines we have and can not currently run
> deployment testing in parallel.
>
>
>> My question comes from this concern: if we're building out a product
>> based on this PPA, (before Precise is delivered) we want to make sure
>> that when we bring up new systems by installing the packages from the
>> PPA, all of those work together properly. If the latest code from
>> keystone, for example, hasn't been building due to testing errors, we
>> want to make sure that the presence of the older keystone package in
>> the PPA won't be causing issues with the newer builds of the rest of
>> OpenStack.
>
>
> So you might have noticed the version of Keystone we are testing is getting
> a bit dusty.  We decided to freeze the version of Keystone we're testing
> just before the KSL/redux branch was merged.   There are still some features
> that need to land before we can modify our Juju charms to deploy the new
> version (specifically SQL persistence for the service catalog).  I'm hoping
> these will land before e4 (tomorrow) so we can begin testing the new branch
> this week.
>
>
>>
>> To clarify: in your blog post, you explicitly mention the validation
>> process per component, starting with the upstream git repos. In the
>> deploy phase, you verify that the system as a whole (all of OpenStack)
>> works as expected. But what happens when one or more of those
>> components don't work? Are packages rolled back in the PPA until the
>> PPA only provides packages that will result in, once installed, a
>> complete working system?
>
>
> So far, we haven't rolled anything back in the PPA or the local repository.
>  I believe Keystone is the only package we've had to freeze while things
> stabilize upstream.  Fortunately, we've found the CI (even with the limited
> test coverage we currently run) is picking up new bugs almost
> instantaneously.  In most cases we're able to either get a fix into gerrit
> same day or find a proposed fix that we can cherry pick into our
> debian/patches/ repository and carry temporarily in our
> lp:~openstack-ubuntu-testing packages branches until its been fixed proper
> upstream.
>
> Please understand the PPA is for testing purposes only, and we're not making
> any promise that what installs from there is stable/usable/working.  We're
> using the work around that PPA and this CI to essentially gate what goes
> into the Ubuntu archive.  This cycle, Chuck has been uploading a weekly
> snapshot of Openstack (usually every Friday).  With the CI in place, we can
> essentially verify that those packages build clean and install okay, and
> provide something usable given the last weeks Gerrit churn.
>
> Keep in mind this is still a developing process.  I expect this will evolve
> over the next few months.  We've just begun trigger pre-commit testing to
> the Diablo stable branch against Oneiric.  Still working out some kinks, but
> we'll hopefully be going into Folsom+Precise with coverage of both trunk and
> stable updates to Essex/Precise LTS.
>
> Also note that, at this point in the Ubuntu development cycle, it can often
> take *a long time* before an upload to a PPA is built and ready for use.
>  Its probably safe to assume that what you're using from the PPA is what we
> were testing yesterday (check the version of the package for a timestamp)
>
>
>>
>> Keep up the great work, guys -- you have fans out here in the wild,
>> wild world of OpenStack :-)
>
>
> :)
>
>
> Cheers,
> Adam
>
>
>
> --
> Ubuntu-cloud mailing list
> ubuntu-cl...@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lis

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Adam Gandelman]

On 02/27/2012 09:22 AM, Duncan McGreggor wrote:

It's been an invaluable source for not only information, but also
planning for the cloud work here at DreamHost.


This is great to hear!  I'm happy to hear other people are benefiting 
from this as much as Ubuntu




To the point of this email, though, I have a question for you that I
wasn't able to parse an explicit answer to from your post: For the
packages that are built in the PPA linked above, are they only built
after all the components of OpenStack have been confirmed working as a
whole? Or are they built just after individual testing?


The jenkins trunk build jobs are the ones responsible for uploading to 
the PPA.  It basically stuffs the newly built package in the local 
repository and uploads it to the PPA (if it fails to build, it doesnt 
upload).  The nova trunk build also triggers the deployment + testing.   
So, to answer, those PPA packages are *pre* deployment + smoke testing.  
The idea with that is that we can use the PPA externally to debug 
specific issues that might turn up during testing without blocking the 
deployment/smoke testing queue.  We're limited by the number of machines 
we have and can not currently run deployment testing in parallel.



My question comes from this concern: if we're building out a product
based on this PPA, (before Precise is delivered) we want to make sure
that when we bring up new systems by installing the packages from the
PPA, all of those work together properly. If the latest code from
keystone, for example, hasn't been building due to testing errors, we
want to make sure that the presence of the older keystone package in
the PPA won't be causing issues with the newer builds of the rest of
OpenStack.


So you might have noticed the version of Keystone we are testing is 
getting a bit dusty.  We decided to freeze the version of Keystone we're 
testing just before the KSL/redux branch was merged.   There are still 
some features that need to land before we can modify our Juju charms to 
deploy the new version (specifically SQL persistence for the service 
catalog).  I'm hoping these will land before e4 (tomorrow) so we can 
begin testing the new branch this week.




To clarify: in your blog post, you explicitly mention the validation
process per component, starting with the upstream git repos. In the
deploy phase, you verify that the system as a whole (all of OpenStack)
works as expected. But what happens when one or more of those
components don't work? Are packages rolled back in the PPA until the
PPA only provides packages that will result in, once installed, a
complete working system?


So far, we haven't rolled anything back in the PPA or the local 
repository.  I believe Keystone is the only package we've had to freeze 
while things stabilize upstream.  Fortunately, we've found the CI (even 
with the limited test coverage we currently run) is picking up new bugs 
almost instantaneously.  In most cases we're able to either get a fix 
into gerrit same day or find a proposed fix that we can cherry pick into 
our debian/patches/ repository and carry temporarily in our 
lp:~openstack-ubuntu-testing packages branches until its been fixed 
proper upstream.


Please understand the PPA is for testing purposes only, and we're not 
making any promise that what installs from there is 
stable/usable/working.  We're using the work around that PPA and this CI 
to essentially gate what goes into the Ubuntu archive.  This cycle, 
Chuck has been uploading a weekly snapshot of Openstack (usually every 
Friday).  With the CI in place, we can essentially verify that those 
packages build clean and install okay, and provide something usable 
given the last weeks Gerrit churn.


Keep in mind this is still a developing process.  I expect this will 
evolve over the next few months.  We've just begun trigger pre-commit 
testing to the Diablo stable branch against Oneiric.  Still working out 
some kinks, but we'll hopefully be going into Folsom+Precise with 
coverage of both trunk and stable updates to Essex/Precise LTS.


Also note that, at this point in the Ubuntu development cycle, it can 
often take *a long time* before an upload to a PPA is built and ready 
for use.  Its probably safe to assume that what you're using from the 
PPA is what we were testing yesterday (check the version of the package 
for a timestamp)




Keep up the great work, guys -- you have fans out here in the wild,
wild world of OpenStack :-)


:)

Cheers,
Adam



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Duncan McGreggor]

On Wed, Feb 8, 2012 at 6:57 PM, Adam Gandelman  wrote:
> As promised for anyone who was interested when we announced to the last last
> week, here is a blog post James Page and I put together describing our
> Openstack testing efforts and infrastructure in greater detail:
>
> http://javacruft.wordpress.com/2012/02/08/automating-openstack-testing-on-ubuntu/

Adam, thanks for this great write up :-)

Part of my morning ritual involves hitting these pages every day when I get up:
 * https://jenkins.qa.ubuntu.com/view/Precise%20OpenStack%20Testing/
 * 
https://launchpad.net/~openstack-ubuntu-testing/+archive/openstack-trunk-testing

It's been an invaluable source for not only information, but also
planning for the cloud work here at DreamHost.

To the point of this email, though, I have a question for you that I
wasn't able to parse an explicit answer to from your post: For the
packages that are built in the PPA linked above, are they only built
after all the components of OpenStack have been confirmed working as a
whole? Or are they built just after individual testing?

My question comes from this concern: if we're building out a product
based on this PPA, (before Precise is delivered) we want to make sure
that when we bring up new systems by installing the packages from the
PPA, all of those work together properly. If the latest code from
keystone, for example, hasn't been building due to testing errors, we
want to make sure that the presence of the older keystone package in
the PPA won't be causing issues with the newer builds of the rest of
OpenStack.

To clarify: in your blog post, you explicitly mention the validation
process per component, starting with the upstream git repos. In the
deploy phase, you verify that the system as a whole (all of OpenStack)
works as expected. But what happens when one or more of those
components don't work? Are packages rolled back in the PPA until the
PPA only provides packages that will result in, once installed, a
complete working system?

So there's that practical side of it, but to be honest, it's also
simply an interesting question :-) I find the logistics of automated
testing a great source of interest and fascination...

Keep up the great work, guys -- you have fans out here in the wild,
wild world of OpenStack :-)

d

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Chris Fordham]

On Thu, 09 Feb 2012 07:57:31 -0800, Robbie Williamson   
wrote:



On 02/09/2012 05:21 AM, Ghe Rivero wrote:

Wow! Openstack is really getting to much love from Ubuntu, nice!  I wish
we have so many resources to the same with Debian. Keep the good work  
guys!

Ghe Rivero



I could be 100% wrong here, but I think Debian has agreed (or are in
talks) to use Ubuntu as the upstream for OpenStack, thus allowing our
testing to "trickle" down.


Can you please cite a reference for this?
It would be a significant change to Debian policy and packaging.
Afaik, the package(s) would still need to be mentored and sponsored - in  
this case who does this, Debian or Ubuntu or?


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Thomas Goirand]

- Original message -
> On Thu, Feb 9, 2012 at 4:57 PM, Robbie Williamson 
> wrote:
> > On 02/09/2012 05:21 AM, Ghe Rivero wrote:
> > > Wow! Openstack is really getting to much love from Ubuntu, nice!   I
> > > wish we have so many resources to the same with Debian. Keep the
> > > good work
> > guys!
> > > Ghe Rivero
> > > 
> > 
> > I could be 100% wrong here, but I think Debian has agreed (or are in
> > talks) to use Ubuntu as the upstream for OpenStack, thus allowing our
> > testing to "trickle" down.

There's not really such thing anymore. Our Debian
packages aren't the same as from Ubuntu now, and
we take the code from Github. So openstack on
github is our upstream (even though many developers
sending code there are from Ubuntu).

Thomas


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Ghe Rivero]

Yes, we all agree that Ubuntu is the reference platform for OpenStack. And
I wasn't complaining about it, just making a wish trying to get some help
:). Sorry about the misunderstanding.
Ghe Rivero

On Thu, Feb 9, 2012 at 4:57 PM, Robbie Williamson  wrote:

> On 02/09/2012 05:21 AM, Ghe Rivero wrote:
> > Wow! Openstack is really getting to much love from Ubuntu, nice!  I wish
> > we have so many resources to the same with Debian. Keep the good work
> guys!
> > Ghe Rivero
> >
>
> I could be 100% wrong here, but I think Debian has agreed (or are in
> talks) to use Ubuntu as the upstream for OpenStack, thus allowing our
> testing to "trickle" down.
>
>
> --
> Robbie Williamson 
> robbiew[irc.freenode.net]
>
> "Don't make me angry...you wouldn't like me when I'm angry."
>  -Bruce Banner
>



-- 
Ghe Rivero
*OpenStack & Distribution Engineer
**www.stackops.com | * ghe.riv...@stackops.com 
** | +34 625 63 45 23 | skype:ghe.rivero*
* 
*

*

 ADVERTENCIA LEGAL 
Le informamos, como destinatario de este mensaje, que el correo electrónico
y las comunicaciones por medio de Internet no permiten asegurar ni
garantizar la confidencialidad de los mensajes transmitidos, así como
tampoco su integridad o su correcta recepción, por lo que STACKOPS
TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias.
Si no consintiese en la utilización del correo electrónico o de las
comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro
conocimiento de manera inmediata. Este mensaje va dirigido, de manera
exclusiva, a su destinatario y contiene información confidencial y sujeta
al secreto profesional, cuya divulgación no está permitida por la ley. En
caso de haber recibido este mensaje por error, le rogamos que, de forma
inmediata, nos lo comunique mediante correo electrónico remitido a nuestra
atención y proceda a su eliminación, así como a la de cualquier documento
adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o
utilización de este mensaje, o de cualquier documento adjunto al mismo,
cualquiera que fuera su finalidad, están prohibidas por la ley.

* PRIVILEGED AND CONFIDENTIAL 
We hereby inform you, as addressee of this message, that e-mail and
Internet do not guarantee the confidentiality, nor the completeness or
proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L.
does not assume any liability for those circumstances. Should you not agree
to the use of e-mail or to communications via Internet, you are kindly
requested to notify us immediately. This message is intended exclusively
for the person to whom it is addressed and contains privileged and
confidential information protected from disclosure by law. If you are not
the addressee indicated in this message, you should immediately delete it
and any attachments and notify the sender by reply e-mail. In such case,
you are hereby notified that any dissemination, distribution, copying or
use of this message or any attachments, for any purpose, is strictly
prohibited by law.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Robbie Williamson]

On 02/09/2012 05:21 AM, Ghe Rivero wrote:
> Wow! Openstack is really getting to much love from Ubuntu, nice!  I wish
> we have so many resources to the same with Debian. Keep the good work guys!
> Ghe Rivero
> 

I could be 100% wrong here, but I think Debian has agreed (or are in
talks) to use Ubuntu as the upstream for OpenStack, thus allowing our
testing to "trickle" down.


-- 
Robbie Williamson 
robbiew[irc.freenode.net]

"Don't make me angry...you wouldn't like me when I'm angry."
 -Bruce Banner

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Ghe Rivero]

Wow! Openstack is really getting to much love from Ubuntu, nice!  I wish we
have so many resources to the same with Debian. Keep the good work guys!
Ghe Rivero

On Thu, Feb 9, 2012 at 12:09 PM, Mark Shuttleworth  wrote:

> On 08/02/12 23:57, Adam Gandelman wrote:
>
>> As promised for anyone who was interested when we announced to the last
>> last week, here is a blog post James Page and I put together describing our
>> Openstack testing efforts and infrastructure in greater detail:
>>
>> http://javacruft.wordpress.**com/2012/02/08/automating-**
>> openstack-testing-on-ubuntu/
>>
>
> Awesome work guys!
>
>
> __**_
> Mailing list: 
> https://launchpad.net/~**openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : 
> https://launchpad.net/~**openstack
> More help   : 
> https://help.launchpad.net/**ListHelp
>



-- 
Ghe Rivero
*OpenStack & Distribution Engineer
**www.stackops.com | * ghe.riv...@stackops.com 
** | +34 625 63 45 23 | skype:ghe.rivero*
* 
*

*

 ADVERTENCIA LEGAL 
Le informamos, como destinatario de este mensaje, que el correo electrónico
y las comunicaciones por medio de Internet no permiten asegurar ni
garantizar la confidencialidad de los mensajes transmitidos, así como
tampoco su integridad o su correcta recepción, por lo que STACKOPS
TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias.
Si no consintiese en la utilización del correo electrónico o de las
comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro
conocimiento de manera inmediata. Este mensaje va dirigido, de manera
exclusiva, a su destinatario y contiene información confidencial y sujeta
al secreto profesional, cuya divulgación no está permitida por la ley. En
caso de haber recibido este mensaje por error, le rogamos que, de forma
inmediata, nos lo comunique mediante correo electrónico remitido a nuestra
atención y proceda a su eliminación, así como a la de cualquier documento
adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o
utilización de este mensaje, o de cualquier documento adjunto al mismo,
cualquiera que fuera su finalidad, están prohibidas por la ley.

* PRIVILEGED AND CONFIDENTIAL 
We hereby inform you, as addressee of this message, that e-mail and
Internet do not guarantee the confidentiality, nor the completeness or
proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L.
does not assume any liability for those circumstances. Should you not agree
to the use of e-mail or to communications via Internet, you are kindly
requested to notify us immediately. This message is intended exclusively
for the person to whom it is addressed and contains privileged and
confidential information protected from disclosure by law. If you are not
the addressee indicated in this message, you should immediately delete it
and any attachments and notify the sender by reply e-mail. In such case,
you are hereby notified that any dissemination, distribution, copying or
use of this message or any attachments, for any purpose, is strictly
prohibited by law.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ubuntu-cloud] Update on Ubuntu automated testing and CI of Openstack

[Mark Shuttleworth]

On 08/02/12 23:57, Adam Gandelman wrote:
As promised for anyone who was interested when we announced to the 
last last week, here is a blog post James Page and I put together 
describing our Openstack testing efforts and infrastructure in greater 
detail:


http://javacruft.wordpress.com/2012/02/08/automating-openstack-testing-on-ubuntu/ 



Awesome work guys!

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp