Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Joshua Harlow]

very tag in kolla repository.
These are static and will not be revised.


Yes, this is fine, but please keep in mind that this change[1] could be
merged without changing these published 4.0.0 style image tags, with the
added advantage of locally built images with a git checkout of kolla
have a less ambiguous default tag.
[1] https://review.openstack.org/#/c/448380/

Different scenerios can be handled this way
1. Autopushing to dockerhub will query freshest built registry
(tarballs, source) and and dockerhub (dest), it will create
image:branchname (nova-api:ocata) for HEAD of stable branch every run
and image:branchname-revision with revision increase
2. Users will have easy time managing their local registry - dockerhub
(source) and local (dest), if nova-api:ocata on dockerhub is newer
than local, pull it to local and increase local tip and revision

Thoughts?


Generally positive :)


[1]

http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-19.log.html#t2017-04-19T19:10:25

<http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-19.log.html#t2017-04-19T19:10:25>

On 19 April 2017 at 10:45, Fox, Kevin M <kevin@pnnl.gov
<mailto:kevin@pnnl.gov>> wrote:
 > That works for detecting changes in the build system.
 >
 > It does not solve the issue of how to keep containers atomic on
    end user systems.
 >
 > All images in a k8s deployment should be the same image. This is
done by specifying the same tag. When a new update is done, the
updated deployment should specify a new tag to distinguish it from
the old tag so that roll forwards/roll backs work atomically and as
designed. Otherwise, roll back can actually break or roll forward
wont actually grab newer images.
 >
 > Thanks,
 > Kevin
 >
 > 
 > From: Michał Jastrzębski [inc...@gmail.com <mailto:inc...@gmail.com>]
 > Sent: Wednesday, April 19, 2017 8:32 AM
 > To: OpenStack Development Mailing List (not for usage questions)
 > Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub
 >
 > I think LABEL is great idea for all the "informative" stuff. In fact
 > if we could somehow abuse LABEL to fill it up after we get packages
 > installed, we could use it for version manifesto. That will make
logic
 > around "if version changed" much easier since we'll have easy access
 > to this information on both image and container.
 >
 > Our autopushing mechanism will work with tags and HEAD of stable
 > branch in this case.
 >
 > Kevin, then your use case would be done like that:
 > 1. pull container nova-compute:ocata, tag it locally to
 > nova-compute:ocata-deployed, deploy it
 > 2. every now and then pull fresh nova-compute:ocata from dockerhub
 > 3. compare versions in LABELs to see whether you want to upgrade
or not
 > 4. if you do, retag :ocata-deployed to :ocata-old, :ocata to
 > :ocata-deployed and run upgrade
 > 5. keep ocata-old, revision it, backup it for as long as you want
 >
 > I also think that we can ship utils to do this in kolla, so people
 > won't need to write these themselves.
 >
 > Does that work?
 >
 > Cheers,
 > Michal
 >
 > On 19 April 2017 at 05:02, Flavio Percoco <fla...@redhat.com
<mailto:fla...@redhat.com>> wrote:
 >> On 19/04/17 11:20 +0100, Paul Bourke wrote:
 >>>
 >>> I'm wondering if moving to using docker labels is a better way
of solving
 >>> the various issue being raised here.
 >>>
 >>> We can maintain a tag for each of master/ocata/newton/etc, and
on each
 >>> image have a LABEL with info such as 'pbr of service/pbr of
kolla/link to CI
 >>> of build/etc'. I believe this solves all points Kevin mentioned
except
 >>> rollback, which afaik, OpenStack doesn't support anyway. It
also solves
 >>> people's concerns with what is actually in the images, and is a
standard
 >>> Docker mechanism.
 >>>
 >>> Also as Michal mentioned, if users are concerned about keeping
images,
 >>> they can tag and stash them away themselves. It is overkill to
maintain
 >>> hundreds of (imo meaningless) tags in a registry, the majority
of which
 >>> people don't care about - they only want the latest of the
branch they're
 >>> deploying.
 >>>
 >>> Every detail of a running Kolla system can be easily deduced by
scanning
 >>

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Steve Baker]

On Thu, Apr 20, 2017 at 8:12 AM, Michał Jastrzębski <inc...@gmail.com>
wrote:

> So after discussion started here [1] we came up with something like that:
>
> 1. Docker build will create "fingerprint" - manifesto of versions
> saved somewhere (LABEL?)
>

This would be great, especially a full package version listing in an image
label. However I don't see an easy way of populating a label from data
inside the image. Other options could be:
- have a script inside the image in a known location which generates the
package manifest on the fly, do a docker run whenever you need to get a
manifest to compare with another image.
- write out the package list during image build to a known location, do a
docker run to cat out its contents when needed

As for the format, taking a yum only image as an example would we need
anything more than the output of "rpm -qa | sort"?


> 2. We create new CLI tool kolla-registry for easier management of
> pushing and versioning
> 3. kolla-registry will be able to query existing source docker
> registry (incl. dockerhub) for latest tag-revision and it's version
> manifesto, also dest registry for tags-revisions and manifesto
> 4. if source image manifesto != dest image manifesto -> push source
> image to dest registry and increase tag-revision by 1
> 5. kolla-registry will output latest list of images:tags-revisions
> available for kolla-k8s/ansible to consume
> 6. we keep :4.0.0 style images for every tag in kolla repository.
> These are static and will not be revised.
>
>
Yes, this is fine, but please keep in mind that this change[1] could be
merged without changing these published 4.0.0 style image tags, with the
added advantage of locally built images with a git checkout of kolla have a
less ambiguous default tag.

[1] https://review.openstack.org/#/c/448380/

Different scenerios can be handled this way
> 1. Autopushing to dockerhub will query freshest built registry
> (tarballs, source) and and dockerhub (dest), it will create
> image:branchname (nova-api:ocata) for HEAD of stable branch every run
> and image:branchname-revision with revision increase
> 2. Users will have easy time managing their local registry - dockerhub
> (source) and local (dest), if nova-api:ocata on dockerhub is newer
> than local, pull it to local and increase local tip and revision
>
> Thoughts?
>

Generally positive :)


>
> [1] http://eavesdrop.openstack.org/irclogs/%23openstack-
> kolla/%23openstack-kolla.2017-04-19.log.html#t2017-04-19T19:10:25
>
> On 19 April 2017 at 10:45, Fox, Kevin M <kevin@pnnl.gov> wrote:
> > That works for detecting changes in the build system.
> >
> > It does not solve the issue of how to keep containers atomic on end user
> systems.
> >
> > All images in a k8s deployment should be the same image. This is done by
> specifying the same tag. When a new update is done, the updated deployment
> should specify a new tag to distinguish it from the old tag so that roll
> forwards/roll backs work atomically and as designed. Otherwise, roll back
> can actually break or roll forward wont actually grab newer images.
> >
> > Thanks,
> > Kevin
> >
> > ________________
> > From: Michał Jastrzębski [inc...@gmail.com]
> > Sent: Wednesday, April 19, 2017 8:32 AM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub
> >
> > I think LABEL is great idea for all the "informative" stuff. In fact
> > if we could somehow abuse LABEL to fill it up after we get packages
> > installed, we could use it for version manifesto. That will make logic
> > around "if version changed" much easier since we'll have easy access
> > to this information on both image and container.
> >
> > Our autopushing mechanism will work with tags and HEAD of stable
> > branch in this case.
> >
> > Kevin, then your use case would be done like that:
> > 1. pull container nova-compute:ocata, tag it locally to
> > nova-compute:ocata-deployed, deploy it
> > 2. every now and then pull fresh nova-compute:ocata from dockerhub
> > 3. compare versions in LABELs to see whether you want to upgrade or not
> > 4. if you do, retag :ocata-deployed to :ocata-old, :ocata to
> > :ocata-deployed and run upgrade
> > 5. keep ocata-old, revision it, backup it for as long as you want
> >
> > I also think that we can ship utils to do this in kolla, so people
> > won't need to write these themselves.
> >
> > Does that work?
> >
> > Cheers,
> > Michal
> >
> > On 19 April 2017 at 05:02, Flavio Percoco <fla...@redhat.com> wrot

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

So after discussion started here [1] we came up with something like that:

1. Docker build will create "fingerprint" - manifesto of versions
saved somewhere (LABEL?)
2. We create new CLI tool kolla-registry for easier management of
pushing and versioning
3. kolla-registry will be able to query existing source docker
registry (incl. dockerhub) for latest tag-revision and it's version
manifesto, also dest registry for tags-revisions and manifesto
4. if source image manifesto != dest image manifesto -> push source
image to dest registry and increase tag-revision by 1
5. kolla-registry will output latest list of images:tags-revisions
available for kolla-k8s/ansible to consume
6. we keep :4.0.0 style images for every tag in kolla repository.
These are static and will not be revised.

Different scenerios can be handled this way
1. Autopushing to dockerhub will query freshest built registry
(tarballs, source) and and dockerhub (dest), it will create
image:branchname (nova-api:ocata) for HEAD of stable branch every run
and image:branchname-revision with revision increase
2. Users will have easy time managing their local registry - dockerhub
(source) and local (dest), if nova-api:ocata on dockerhub is newer
than local, pull it to local and increase local tip and revision

Thoughts?
Michal

[1] 
http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-19.log.html#t2017-04-19T19:10:25

On 19 April 2017 at 10:45, Fox, Kevin M <kevin@pnnl.gov> wrote:
> That works for detecting changes in the build system.
>
> It does not solve the issue of how to keep containers atomic on end user 
> systems.
>
> All images in a k8s deployment should be the same image. This is done by 
> specifying the same tag. When a new update is done, the updated deployment 
> should specify a new tag to distinguish it from the old tag so that roll 
> forwards/roll backs work atomically and as designed. Otherwise, roll back can 
> actually break or roll forward wont actually grab newer images.
>
> Thanks,
> Kevin
>
> 
> From: Michał Jastrzębski [inc...@gmail.com]
> Sent: Wednesday, April 19, 2017 8:32 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub
>
> I think LABEL is great idea for all the "informative" stuff. In fact
> if we could somehow abuse LABEL to fill it up after we get packages
> installed, we could use it for version manifesto. That will make logic
> around "if version changed" much easier since we'll have easy access
> to this information on both image and container.
>
> Our autopushing mechanism will work with tags and HEAD of stable
> branch in this case.
>
> Kevin, then your use case would be done like that:
> 1. pull container nova-compute:ocata, tag it locally to
> nova-compute:ocata-deployed, deploy it
> 2. every now and then pull fresh nova-compute:ocata from dockerhub
> 3. compare versions in LABELs to see whether you want to upgrade or not
> 4. if you do, retag :ocata-deployed to :ocata-old, :ocata to
> :ocata-deployed and run upgrade
> 5. keep ocata-old, revision it, backup it for as long as you want
>
> I also think that we can ship utils to do this in kolla, so people
> won't need to write these themselves.
>
> Does that work?
>
> Cheers,
> Michal
>
> On 19 April 2017 at 05:02, Flavio Percoco <fla...@redhat.com> wrote:
>> On 19/04/17 11:20 +0100, Paul Bourke wrote:
>>>
>>> I'm wondering if moving to using docker labels is a better way of solving
>>> the various issue being raised here.
>>>
>>> We can maintain a tag for each of master/ocata/newton/etc, and on each
>>> image have a LABEL with info such as 'pbr of service/pbr of kolla/link to CI
>>> of build/etc'. I believe this solves all points Kevin mentioned except
>>> rollback, which afaik, OpenStack doesn't support anyway. It also solves
>>> people's concerns with what is actually in the images, and is a standard
>>> Docker mechanism.
>>>
>>> Also as Michal mentioned, if users are concerned about keeping images,
>>> they can tag and stash them away themselves. It is overkill to maintain
>>> hundreds of (imo meaningless) tags in a registry, the majority of which
>>> people don't care about - they only want the latest of the branch they're
>>> deploying.
>>>
>>> Every detail of a running Kolla system can be easily deduced by scanning
>>> across nodes and printing the labels of running containers, functionality
>>> which can be shipped by Kolla. There are also methods for fetching labels of
>>> remote images[0][1] for users wishing to ins

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

That works for detecting changes in the build system.

It does not solve the issue of how to keep containers atomic on end user 
systems.

All images in a k8s deployment should be the same image. This is done by 
specifying the same tag. When a new update is done, the updated deployment 
should specify a new tag to distinguish it from the old tag so that roll 
forwards/roll backs work atomically and as designed. Otherwise, roll back can 
actually break or roll forward wont actually grab newer images.

Thanks,
Kevin


From: Michał Jastrzębski [inc...@gmail.com]
Sent: Wednesday, April 19, 2017 8:32 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I think LABEL is great idea for all the "informative" stuff. In fact
if we could somehow abuse LABEL to fill it up after we get packages
installed, we could use it for version manifesto. That will make logic
around "if version changed" much easier since we'll have easy access
to this information on both image and container.

Our autopushing mechanism will work with tags and HEAD of stable
branch in this case.

Kevin, then your use case would be done like that:
1. pull container nova-compute:ocata, tag it locally to
nova-compute:ocata-deployed, deploy it
2. every now and then pull fresh nova-compute:ocata from dockerhub
3. compare versions in LABELs to see whether you want to upgrade or not
4. if you do, retag :ocata-deployed to :ocata-old, :ocata to
:ocata-deployed and run upgrade
5. keep ocata-old, revision it, backup it for as long as you want

I also think that we can ship utils to do this in kolla, so people
won't need to write these themselves.

Does that work?

Cheers,
Michal

On 19 April 2017 at 05:02, Flavio Percoco <fla...@redhat.com> wrote:
> On 19/04/17 11:20 +0100, Paul Bourke wrote:
>>
>> I'm wondering if moving to using docker labels is a better way of solving
>> the various issue being raised here.
>>
>> We can maintain a tag for each of master/ocata/newton/etc, and on each
>> image have a LABEL with info such as 'pbr of service/pbr of kolla/link to CI
>> of build/etc'. I believe this solves all points Kevin mentioned except
>> rollback, which afaik, OpenStack doesn't support anyway. It also solves
>> people's concerns with what is actually in the images, and is a standard
>> Docker mechanism.
>>
>> Also as Michal mentioned, if users are concerned about keeping images,
>> they can tag and stash them away themselves. It is overkill to maintain
>> hundreds of (imo meaningless) tags in a registry, the majority of which
>> people don't care about - they only want the latest of the branch they're
>> deploying.
>>
>> Every detail of a running Kolla system can be easily deduced by scanning
>> across nodes and printing the labels of running containers, functionality
>> which can be shipped by Kolla. There are also methods for fetching labels of
>> remote images[0][1] for users wishing to inspect what they are upgrading to.
>>
>> [0] https://github.com/projectatomic/skopeo
>> [1] https://github.com/docker/distribution/issues/1252
>
>
>
> You beat me to it, Paul.
>
> I think using lables to communicate the version of each openstack software
> installed in the image is the way to go here. We're looking into doing this
> ourselves as part of the RDO pipeline and it'd be awesome to have it being
> part
> of kolla-build itself. Steve Baker, I believe, was working on this.
>
> The more explicit we are about the contents of the image, the better. People
> want to know what's in there, rather than assuming based on the tag.
>
> Flavio
>
>
>> -Paul
>>
>> On 18/04/17 22:10, Michał Jastrzębski wrote:
>>>
>>> On 18 April 2017 at 13:54, Doug Hellmann <d...@doughellmann.com> wrote:
>>>>
>>>> Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>>>>>
>>>>> On 18 April 2017 at 12:41, Doug Hellmann <d...@doughellmann.com> wrote:
>>>>>>
>>>>>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>>>>>>>
>>>>>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann
>>>>>>> <d...@doughellmann.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34
>>>>>>>> -0700:
>>>>>>>>>
>>>>>>>>> My dear Kollegues,
>>>>>>>>>
>>>>>>>>> Today we had discussion about how to properly 

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

one other thing. checksums also do not relay any information about 
newer/older'ness. Only that a change happened.


From: Britt Houser (bhouser) [bhou...@cisco.com]
Sent: Wednesday, April 19, 2017 3:39 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I agree with Paul here.  I like the idea of solving this with labels instead of 
tags.  A label is imbedded into the docker image, and if it changes, the 
checksum of the image changes.  A tag is kept in the image manifest, and can be 
altered w/o changing the underlying image.  So to me a label is better IMHO, 
b/c it preserves this data within the image itself in a manner which is easy to 
detect if its been altered.

thx,
britt


From: Paul Bourke <paul.bou...@oracle.com>
Sent: Apr 19, 2017 6:28 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I'm wondering if moving to using docker labels is a better way of
solving the various issue being raised here.

We can maintain a tag for each of master/ocata/newton/etc, and on each
image have a LABEL with info such as 'pbr of service/pbr of kolla/link
to CI of build/etc'. I believe this solves all points Kevin mentioned
except rollback, which afaik, OpenStack doesn't support anyway. It also
solves people's concerns with what is actually in the images, and is a
standard Docker mechanism.

Also as Michal mentioned, if users are concerned about keeping images,
they can tag and stash them away themselves. It is overkill to maintain
hundreds of (imo meaningless) tags in a registry, the majority of which
people don't care about - they only want the latest of the branch
they're deploying.

Every detail of a running Kolla system can be easily deduced by scanning
across nodes and printing the labels of running containers,
functionality which can be shipped by Kolla. There are also methods for
fetching labels of remote images[0][1] for users wishing to inspect what
they are upgrading to.

[0] https://github.com/projectatomic/skopeo
[1] https://github.com/docker/distribution/issues/1252

-Paul

On 18/04/17 22:10, Michał Jastrzębski wrote:
> On 18 April 2017 at 13:54, Doug Hellmann <d...@doughellmann.com> wrote:
>> Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>>> On 18 April 2017 at 12:41, Doug Hellmann <d...@doughellmann.com> wrote:
>>>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>>>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann <d...@doughellmann.com>
>>>>> wrote:
>>>>>
>>>>>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>>>>>>> My dear Kollegues,
>>>>>>>
>>>>>>> Today we had discussion about how to properly name/tag images being
>>>>>>> pushed to dockerhub. That moved towards general discussion on revision
>>>>>>> mgmt.
>>>>>>>
>>>>>>> Problem we're trying to solve is this:
>>>>>>> If you build/push images today, your tag is 4.0
>>>>>>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>>>>>> we tag new release.
>>>>>>>
>>>>>>> But image built today is not equal to image built tomorrow, so we
>>>>>>> would like something like 4.0.0-1, 4.0.0-2.
>>>>>>> While we can reasonably detect history of revisions in dockerhub,
>>>>>>> local env will be extremely hard to do.
>>>>>>>
>>>>>>> I'd like to ask you for opinions on desired behavior and how we want
>>>>>>> to deal with revision management in general.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Michal
>>>>>>>
>>>>>>
>>>>>> What's in the images, kolla? Other OpenStack components?
>>>>>
>>>>>
>>>>> Yes, each image will typically contain all software required for one
>>>>> OpenStack service, including dependencies from OpenStack projects or the
>>>>> base OS. Installed via some combination of git, pip, rpm, deb.
>>>>>
>>>>>> Where does the
>>>>>> 4.0.0 come from?
>>>>>>
>>>>>>
>>>>> Its the python version string from the kolla project itself, so ultimately
>>>>> I think pbr. I'm suggesting that we switch to using the
>>>>> version.release_st

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

K8s cant pull containers based on labels. I think labels are maybe a good way 
of storing container fingerprints though.

Thanks,
Kevin

From: Britt Houser (bhouser) [bhou...@cisco.com]
Sent: Wednesday, April 19, 2017 3:39 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I agree with Paul here.  I like the idea of solving this with labels instead of 
tags.  A label is imbedded into the docker image, and if it changes, the 
checksum of the image changes.  A tag is kept in the image manifest, and can be 
altered w/o changing the underlying image.  So to me a label is better IMHO, 
b/c it preserves this data within the image itself in a manner which is easy to 
detect if its been altered.

thx,
britt


From: Paul Bourke <paul.bou...@oracle.com>
Sent: Apr 19, 2017 6:28 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I'm wondering if moving to using docker labels is a better way of
solving the various issue being raised here.

We can maintain a tag for each of master/ocata/newton/etc, and on each
image have a LABEL with info such as 'pbr of service/pbr of kolla/link
to CI of build/etc'. I believe this solves all points Kevin mentioned
except rollback, which afaik, OpenStack doesn't support anyway. It also
solves people's concerns with what is actually in the images, and is a
standard Docker mechanism.

Also as Michal mentioned, if users are concerned about keeping images,
they can tag and stash them away themselves. It is overkill to maintain
hundreds of (imo meaningless) tags in a registry, the majority of which
people don't care about - they only want the latest of the branch
they're deploying.

Every detail of a running Kolla system can be easily deduced by scanning
across nodes and printing the labels of running containers,
functionality which can be shipped by Kolla. There are also methods for
fetching labels of remote images[0][1] for users wishing to inspect what
they are upgrading to.

[0] https://github.com/projectatomic/skopeo
[1] https://github.com/docker/distribution/issues/1252

-Paul

On 18/04/17 22:10, Michał Jastrzębski wrote:
> On 18 April 2017 at 13:54, Doug Hellmann <d...@doughellmann.com> wrote:
>> Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>>> On 18 April 2017 at 12:41, Doug Hellmann <d...@doughellmann.com> wrote:
>>>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>>>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann <d...@doughellmann.com>
>>>>> wrote:
>>>>>
>>>>>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>>>>>>> My dear Kollegues,
>>>>>>>
>>>>>>> Today we had discussion about how to properly name/tag images being
>>>>>>> pushed to dockerhub. That moved towards general discussion on revision
>>>>>>> mgmt.
>>>>>>>
>>>>>>> Problem we're trying to solve is this:
>>>>>>> If you build/push images today, your tag is 4.0
>>>>>>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>>>>>> we tag new release.
>>>>>>>
>>>>>>> But image built today is not equal to image built tomorrow, so we
>>>>>>> would like something like 4.0.0-1, 4.0.0-2.
>>>>>>> While we can reasonably detect history of revisions in dockerhub,
>>>>>>> local env will be extremely hard to do.
>>>>>>>
>>>>>>> I'd like to ask you for opinions on desired behavior and how we want
>>>>>>> to deal with revision management in general.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Michal
>>>>>>>
>>>>>>
>>>>>> What's in the images, kolla? Other OpenStack components?
>>>>>
>>>>>
>>>>> Yes, each image will typically contain all software required for one
>>>>> OpenStack service, including dependencies from OpenStack projects or the
>>>>> base OS. Installed via some combination of git, pip, rpm, deb.
>>>>>
>>>>>> Where does the
>>>>>> 4.0.0 come from?
>>>>>>
>>>>>>
>>>>> Its the python version string from the kolla project itself, so ultimately
>>>>> I think pbr. I'm suggesting that we switch to using the
>>>>> versio

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

I'm not saying we keep around hundreds of tags, just that our tags have enough 
meaning that user systems can distinguish when something changes.

We can keep the most N revisions of the container around for some value of N. 
Older containers just get deleted from the hub. No reason to keep them around.

kubernetes uses tags with a deployment to do atomic rolling upgrades/rollbacks. 
There isn't very many other mechanisms it supports.

When I'm talking about roll forward/back, I'm not talking between major 
versions, as I do realize those are unsupported. What I'm talking about is roll 
forward/back of containers within the same single version. User applies 
security updates. stuff breaks. user rolls back to the container right before 
the security update, then works with upstream to fix the breakage before 
rolling forward again.

Thanks,
Kevin


From: Paul Bourke [paul.bou...@oracle.com]
Sent: Wednesday, April 19, 2017 3:20 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I'm wondering if moving to using docker labels is a better way of
solving the various issue being raised here.

We can maintain a tag for each of master/ocata/newton/etc, and on each
image have a LABEL with info such as 'pbr of service/pbr of kolla/link
to CI of build/etc'. I believe this solves all points Kevin mentioned
except rollback, which afaik, OpenStack doesn't support anyway. It also
solves people's concerns with what is actually in the images, and is a
standard Docker mechanism.

Also as Michal mentioned, if users are concerned about keeping images,
they can tag and stash them away themselves. It is overkill to maintain
hundreds of (imo meaningless) tags in a registry, the majority of which
people don't care about - they only want the latest of the branch
they're deploying.

Every detail of a running Kolla system can be easily deduced by scanning
across nodes and printing the labels of running containers,
functionality which can be shipped by Kolla. There are also methods for
fetching labels of remote images[0][1] for users wishing to inspect what
they are upgrading to.

[0] https://github.com/projectatomic/skopeo
[1] https://github.com/docker/distribution/issues/1252

-Paul

On 18/04/17 22:10, Michał Jastrzębski wrote:
> On 18 April 2017 at 13:54, Doug Hellmann <d...@doughellmann.com> wrote:
>> Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>>> On 18 April 2017 at 12:41, Doug Hellmann <d...@doughellmann.com> wrote:
>>>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>>>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann <d...@doughellmann.com>
>>>>> wrote:
>>>>>
>>>>>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>>>>>>> My dear Kollegues,
>>>>>>>
>>>>>>> Today we had discussion about how to properly name/tag images being
>>>>>>> pushed to dockerhub. That moved towards general discussion on revision
>>>>>>> mgmt.
>>>>>>>
>>>>>>> Problem we're trying to solve is this:
>>>>>>> If you build/push images today, your tag is 4.0
>>>>>>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>>>>>> we tag new release.
>>>>>>>
>>>>>>> But image built today is not equal to image built tomorrow, so we
>>>>>>> would like something like 4.0.0-1, 4.0.0-2.
>>>>>>> While we can reasonably detect history of revisions in dockerhub,
>>>>>>> local env will be extremely hard to do.
>>>>>>>
>>>>>>> I'd like to ask you for opinions on desired behavior and how we want
>>>>>>> to deal with revision management in general.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Michal
>>>>>>>
>>>>>>
>>>>>> What's in the images, kolla? Other OpenStack components?
>>>>>
>>>>>
>>>>> Yes, each image will typically contain all software required for one
>>>>> OpenStack service, including dependencies from OpenStack projects or the
>>>>> base OS. Installed via some combination of git, pip, rpm, deb.
>>>>>
>>>>>> Where does the
>>>>>> 4.0.0 come from?
>>>>>>
>>>>>>
>>>>> Its the python version string from the kolla project itself, so ultimately
>>>>> I think pbr. I'm suggesting that we s

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

I think LABEL is great idea for all the "informative" stuff. In fact
if we could somehow abuse LABEL to fill it up after we get packages
installed, we could use it for version manifesto. That will make logic
around "if version changed" much easier since we'll have easy access
to this information on both image and container.

Our autopushing mechanism will work with tags and HEAD of stable
branch in this case.

Kevin, then your use case would be done like that:
1. pull container nova-compute:ocata, tag it locally to
nova-compute:ocata-deployed, deploy it
2. every now and then pull fresh nova-compute:ocata from dockerhub
3. compare versions in LABELs to see whether you want to upgrade or not
4. if you do, retag :ocata-deployed to :ocata-old, :ocata to
:ocata-deployed and run upgrade
5. keep ocata-old, revision it, backup it for as long as you want

I also think that we can ship utils to do this in kolla, so people
won't need to write these themselves.

Does that work?

Cheers,
Michal

On 19 April 2017 at 05:02, Flavio Percoco  wrote:
> On 19/04/17 11:20 +0100, Paul Bourke wrote:
>>
>> I'm wondering if moving to using docker labels is a better way of solving
>> the various issue being raised here.
>>
>> We can maintain a tag for each of master/ocata/newton/etc, and on each
>> image have a LABEL with info such as 'pbr of service/pbr of kolla/link to CI
>> of build/etc'. I believe this solves all points Kevin mentioned except
>> rollback, which afaik, OpenStack doesn't support anyway. It also solves
>> people's concerns with what is actually in the images, and is a standard
>> Docker mechanism.
>>
>> Also as Michal mentioned, if users are concerned about keeping images,
>> they can tag and stash them away themselves. It is overkill to maintain
>> hundreds of (imo meaningless) tags in a registry, the majority of which
>> people don't care about - they only want the latest of the branch they're
>> deploying.
>>
>> Every detail of a running Kolla system can be easily deduced by scanning
>> across nodes and printing the labels of running containers, functionality
>> which can be shipped by Kolla. There are also methods for fetching labels of
>> remote images[0][1] for users wishing to inspect what they are upgrading to.
>>
>> [0] https://github.com/projectatomic/skopeo
>> [1] https://github.com/docker/distribution/issues/1252
>
>
>
> You beat me to it, Paul.
>
> I think using lables to communicate the version of each openstack software
> installed in the image is the way to go here. We're looking into doing this
> ourselves as part of the RDO pipeline and it'd be awesome to have it being
> part
> of kolla-build itself. Steve Baker, I believe, was working on this.
>
> The more explicit we are about the contents of the image, the better. People
> want to know what's in there, rather than assuming based on the tag.
>
> Flavio
>
>
>> -Paul
>>
>> On 18/04/17 22:10, Michał Jastrzębski wrote:
>>>
>>> On 18 April 2017 at 13:54, Doug Hellmann  wrote:

 Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>
> On 18 April 2017 at 12:41, Doug Hellmann  wrote:
>>
>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>>>
>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann
>>> 
>>> wrote:
>>>
 Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34
 -0700:
>
> My dear Kollegues,
>
> Today we had discussion about how to properly name/tag images being
> pushed to dockerhub. That moved towards general discussion on
> revision
> mgmt.
>
> Problem we're trying to solve is this:
> If you build/push images today, your tag is 4.0
> if you do it tomorrow, it's still 4.0, and will keep being 4.0
> until
> we tag new release.
>
> But image built today is not equal to image built tomorrow, so we
> would like something like 4.0.0-1, 4.0.0-2.
> While we can reasonably detect history of revisions in dockerhub,
> local env will be extremely hard to do.
>
> I'd like to ask you for opinions on desired behavior and how we
> want
> to deal with revision management in general.
>
> Cheers,
> Michal
>

 What's in the images, kolla? Other OpenStack components?
>>>
>>>
>>>
>>> Yes, each image will typically contain all software required for one
>>> OpenStack service, including dependencies from OpenStack projects or
>>> the
>>> base OS. Installed via some combination of git, pip, rpm, deb.
>>>
 Where does the
 4.0.0 come from?


>>> Its the python version string from the kolla project itself, so
>>> ultimately
>>> I think pbr. I'm suggesting that we switch to using the

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Flavio Percoco]

On 19/04/17 11:20 +0100, Paul Bourke wrote:
I'm wondering if moving to using docker labels is a better way of 
solving the various issue being raised here.


We can maintain a tag for each of master/ocata/newton/etc, and on each 
image have a LABEL with info such as 'pbr of service/pbr of kolla/link 
to CI of build/etc'. I believe this solves all points Kevin mentioned 
except rollback, which afaik, OpenStack doesn't support anyway. It 
also solves people's concerns with what is actually in the images, and 
is a standard Docker mechanism.


Also as Michal mentioned, if users are concerned about keeping images, 
they can tag and stash them away themselves. It is overkill to 
maintain hundreds of (imo meaningless) tags in a registry, the 
majority of which people don't care about - they only want the latest 
of the branch they're deploying.


Every detail of a running Kolla system can be easily deduced by 
scanning across nodes and printing the labels of running containers, 
functionality which can be shipped by Kolla. There are also methods 
for fetching labels of remote images[0][1] for users wishing to 
inspect what they are upgrading to.


[0] https://github.com/projectatomic/skopeo
[1] https://github.com/docker/distribution/issues/1252



You beat me to it, Paul.

I think using lables to communicate the version of each openstack software
installed in the image is the way to go here. We're looking into doing this
ourselves as part of the RDO pipeline and it'd be awesome to have it being part
of kolla-build itself. Steve Baker, I believe, was working on this.

The more explicit we are about the contents of the image, the better. People
want to know what's in there, rather than assuming based on the tag.

Flavio


-Paul

On 18/04/17 22:10, Michał Jastrzębski wrote:

On 18 April 2017 at 13:54, Doug Hellmann  wrote:

Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:

On 18 April 2017 at 12:41, Doug Hellmann  wrote:

Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:

On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
wrote:


Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:

My dear Kollegues,

Today we had discussion about how to properly name/tag images being
pushed to dockerhub. That moved towards general discussion on revision
mgmt.

Problem we're trying to solve is this:
If you build/push images today, your tag is 4.0
if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
we tag new release.

But image built today is not equal to image built tomorrow, so we
would like something like 4.0.0-1, 4.0.0-2.
While we can reasonably detect history of revisions in dockerhub,
local env will be extremely hard to do.

I'd like to ask you for opinions on desired behavior and how we want
to deal with revision management in general.

Cheers,
Michal



What's in the images, kolla? Other OpenStack components?



Yes, each image will typically contain all software required for one
OpenStack service, including dependencies from OpenStack projects or the
base OS. Installed via some combination of git, pip, rpm, deb.


Where does the
4.0.0 come from?



Its the python version string from the kolla project itself, so ultimately
I think pbr. I'm suggesting that we switch to using the
version.release_string[1] which will tag with the longer version we use for
other dev packages.

[1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py


Why are you tagging the artifacts containing other projects with the
version number of kolla, instead of their own version numbers and some
sort of incremented build number?


This is what we do in Kolla and I'd say logistics and simplicity of
implementation. Tags are more than just information for us. We have to


But for a user consuming the image, they have no idea what version of
nova is in it because the version on the image is tied to a different
application entirely.


That's easy enough to check tho (just docker exec into container and
do pip freeze). On the other hand you'll have information that "this
set of various versions was tested together" which is arguably more
important.


deploy these images and we have to know a tag. Combine that with clear
separation of build phase from deployment phase (really build phase is
entirely optional thanks to dockerhub), you'll end up with either
automagical script that will have to somehow detect correct version
mix of containers that works with each other, or hand crafted list
that will have 100+ versions hardcoded.

Incremental build is hard because builds are atomic and you never
really know how many times images were rebuilt (also local rebuilt vs
dockerhub-pushed rebuild will cause collisions in tags).


Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Bogdan Dobrelya]

On 19.04.2017 12:39, Britt Houser (bhouser) wrote:
> I agree with Paul here.  I like the idea of solving this with labels instead 
> of tags.  A label is imbedded into the docker image, and if it changes, the 
> checksum of the image changes.  A tag is kept in the image manifest, and can 
> be altered w/o changing the underlying image.  So to me a label is better 
> IMHO, b/c it preserves this data within the image itself in a manner which is 
> easy to detect if its been altered.
> 
> thx,
> britt

+1, very good idea. Binding released artifacts with a checksum is indeed
a way batter than unreliable tags!

-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Britt Houser (bhouser)]

I agree with Paul here.  I like the idea of solving this with labels instead of 
tags.  A label is imbedded into the docker image, and if it changes, the 
checksum of the image changes.  A tag is kept in the image manifest, and can be 
altered w/o changing the underlying image.  So to me a label is better IMHO, 
b/c it preserves this data within the image itself in a manner which is easy to 
detect if its been altered.

thx,
britt


From: Paul Bourke <paul.bou...@oracle.com>
Sent: Apr 19, 2017 6:28 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I'm wondering if moving to using docker labels is a better way of
solving the various issue being raised here.

We can maintain a tag for each of master/ocata/newton/etc, and on each
image have a LABEL with info such as 'pbr of service/pbr of kolla/link
to CI of build/etc'. I believe this solves all points Kevin mentioned
except rollback, which afaik, OpenStack doesn't support anyway. It also
solves people's concerns with what is actually in the images, and is a
standard Docker mechanism.

Also as Michal mentioned, if users are concerned about keeping images,
they can tag and stash them away themselves. It is overkill to maintain
hundreds of (imo meaningless) tags in a registry, the majority of which
people don't care about - they only want the latest of the branch
they're deploying.

Every detail of a running Kolla system can be easily deduced by scanning
across nodes and printing the labels of running containers,
functionality which can be shipped by Kolla. There are also methods for
fetching labels of remote images[0][1] for users wishing to inspect what
they are upgrading to.

[0] https://github.com/projectatomic/skopeo
[1] https://github.com/docker/distribution/issues/1252

-Paul

On 18/04/17 22:10, Michał Jastrzębski wrote:
> On 18 April 2017 at 13:54, Doug Hellmann <d...@doughellmann.com> wrote:
>> Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>>> On 18 April 2017 at 12:41, Doug Hellmann <d...@doughellmann.com> wrote:
>>>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>>>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann <d...@doughellmann.com>
>>>>> wrote:
>>>>>
>>>>>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>>>>>>> My dear Kollegues,
>>>>>>>
>>>>>>> Today we had discussion about how to properly name/tag images being
>>>>>>> pushed to dockerhub. That moved towards general discussion on revision
>>>>>>> mgmt.
>>>>>>>
>>>>>>> Problem we're trying to solve is this:
>>>>>>> If you build/push images today, your tag is 4.0
>>>>>>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>>>>>> we tag new release.
>>>>>>>
>>>>>>> But image built today is not equal to image built tomorrow, so we
>>>>>>> would like something like 4.0.0-1, 4.0.0-2.
>>>>>>> While we can reasonably detect history of revisions in dockerhub,
>>>>>>> local env will be extremely hard to do.
>>>>>>>
>>>>>>> I'd like to ask you for opinions on desired behavior and how we want
>>>>>>> to deal with revision management in general.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Michal
>>>>>>>
>>>>>>
>>>>>> What's in the images, kolla? Other OpenStack components?
>>>>>
>>>>>
>>>>> Yes, each image will typically contain all software required for one
>>>>> OpenStack service, including dependencies from OpenStack projects or the
>>>>> base OS. Installed via some combination of git, pip, rpm, deb.
>>>>>
>>>>>> Where does the
>>>>>> 4.0.0 come from?
>>>>>>
>>>>>>
>>>>> Its the python version string from the kolla project itself, so ultimately
>>>>> I think pbr. I'm suggesting that we switch to using the
>>>>> version.release_string[1] which will tag with the longer version we use 
>>>>> for
>>>>> other dev packages.
>>>>>
>>>>> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
>>>>
>>>> Why are you tagging the artifacts containing other projects with the
>>>> version number of kolla, inst

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Paul Bourke]

I'm wondering if moving to using docker labels is a better way of 
solving the various issue being raised here.


We can maintain a tag for each of master/ocata/newton/etc, and on each 
image have a LABEL with info such as 'pbr of service/pbr of kolla/link 
to CI of build/etc'. I believe this solves all points Kevin mentioned 
except rollback, which afaik, OpenStack doesn't support anyway. It also 
solves people's concerns with what is actually in the images, and is a 
standard Docker mechanism.


Also as Michal mentioned, if users are concerned about keeping images, 
they can tag and stash them away themselves. It is overkill to maintain 
hundreds of (imo meaningless) tags in a registry, the majority of which 
people don't care about - they only want the latest of the branch 
they're deploying.


Every detail of a running Kolla system can be easily deduced by scanning 
across nodes and printing the labels of running containers, 
functionality which can be shipped by Kolla. There are also methods for 
fetching labels of remote images[0][1] for users wishing to inspect what 
they are upgrading to.


[0] https://github.com/projectatomic/skopeo
[1] https://github.com/docker/distribution/issues/1252

-Paul

On 18/04/17 22:10, Michał Jastrzębski wrote:

On 18 April 2017 at 13:54, Doug Hellmann  wrote:

Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:

On 18 April 2017 at 12:41, Doug Hellmann  wrote:

Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:

On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
wrote:


Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:

My dear Kollegues,

Today we had discussion about how to properly name/tag images being
pushed to dockerhub. That moved towards general discussion on revision
mgmt.

Problem we're trying to solve is this:
If you build/push images today, your tag is 4.0
if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
we tag new release.

But image built today is not equal to image built tomorrow, so we
would like something like 4.0.0-1, 4.0.0-2.
While we can reasonably detect history of revisions in dockerhub,
local env will be extremely hard to do.

I'd like to ask you for opinions on desired behavior and how we want
to deal with revision management in general.

Cheers,
Michal



What's in the images, kolla? Other OpenStack components?



Yes, each image will typically contain all software required for one
OpenStack service, including dependencies from OpenStack projects or the
base OS. Installed via some combination of git, pip, rpm, deb.


Where does the
4.0.0 come from?



Its the python version string from the kolla project itself, so ultimately
I think pbr. I'm suggesting that we switch to using the
version.release_string[1] which will tag with the longer version we use for
other dev packages.

[1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py


Why are you tagging the artifacts containing other projects with the
version number of kolla, instead of their own version numbers and some
sort of incremented build number?


This is what we do in Kolla and I'd say logistics and simplicity of
implementation. Tags are more than just information for us. We have to


But for a user consuming the image, they have no idea what version of
nova is in it because the version on the image is tied to a different
application entirely.


That's easy enough to check tho (just docker exec into container and
do pip freeze). On the other hand you'll have information that "this
set of various versions was tested together" which is arguably more
important.


deploy these images and we have to know a tag. Combine that with clear
separation of build phase from deployment phase (really build phase is
entirely optional thanks to dockerhub), you'll end up with either
automagical script that will have to somehow detect correct version
mix of containers that works with each other, or hand crafted list
that will have 100+ versions hardcoded.

Incremental build is hard because builds are atomic and you never
really know how many times images were rebuilt (also local rebuilt vs
dockerhub-pushed rebuild will cause collisions in tags).


Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

On 18 April 2017 at 13:54, Doug Hellmann  wrote:
> Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
>> On 18 April 2017 at 12:41, Doug Hellmann  wrote:
>> > Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>> >> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
>> >> wrote:
>> >>
>> >> > Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>> >> > > My dear Kollegues,
>> >> > >
>> >> > > Today we had discussion about how to properly name/tag images being
>> >> > > pushed to dockerhub. That moved towards general discussion on revision
>> >> > > mgmt.
>> >> > >
>> >> > > Problem we're trying to solve is this:
>> >> > > If you build/push images today, your tag is 4.0
>> >> > > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>> >> > > we tag new release.
>> >> > >
>> >> > > But image built today is not equal to image built tomorrow, so we
>> >> > > would like something like 4.0.0-1, 4.0.0-2.
>> >> > > While we can reasonably detect history of revisions in dockerhub,
>> >> > > local env will be extremely hard to do.
>> >> > >
>> >> > > I'd like to ask you for opinions on desired behavior and how we want
>> >> > > to deal with revision management in general.
>> >> > >
>> >> > > Cheers,
>> >> > > Michal
>> >> > >
>> >> >
>> >> > What's in the images, kolla? Other OpenStack components?
>> >>
>> >>
>> >> Yes, each image will typically contain all software required for one
>> >> OpenStack service, including dependencies from OpenStack projects or the
>> >> base OS. Installed via some combination of git, pip, rpm, deb.
>> >>
>> >> > Where does the
>> >> > 4.0.0 come from?
>> >> >
>> >> >
>> >> Its the python version string from the kolla project itself, so ultimately
>> >> I think pbr. I'm suggesting that we switch to using the
>> >> version.release_string[1] which will tag with the longer version we use 
>> >> for
>> >> other dev packages.
>> >>
>> >> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
>> >
>> > Why are you tagging the artifacts containing other projects with the
>> > version number of kolla, instead of their own version numbers and some
>> > sort of incremented build number?
>>
>> This is what we do in Kolla and I'd say logistics and simplicity of
>> implementation. Tags are more than just information for us. We have to
>
> But for a user consuming the image, they have no idea what version of
> nova is in it because the version on the image is tied to a different
> application entirely.

That's easy enough to check tho (just docker exec into container and
do pip freeze). On the other hand you'll have information that "this
set of various versions was tested together" which is arguably more
important.

>> deploy these images and we have to know a tag. Combine that with clear
>> separation of build phase from deployment phase (really build phase is
>> entirely optional thanks to dockerhub), you'll end up with either
>> automagical script that will have to somehow detect correct version
>> mix of containers that works with each other, or hand crafted list
>> that will have 100+ versions hardcoded.
>>
>> Incremental build is hard because builds are atomic and you never
>> really know how many times images were rebuilt (also local rebuilt vs
>> dockerhub-pushed rebuild will cause collisions in tags).
>>
>> > Doug
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Doug Hellmann]

Excerpts from Michał Jastrzębski's message of 2017-04-18 13:37:30 -0700:
> On 18 April 2017 at 12:41, Doug Hellmann  wrote:
> > Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
> >> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
> >> wrote:
> >>
> >> > Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
> >> > > My dear Kollegues,
> >> > >
> >> > > Today we had discussion about how to properly name/tag images being
> >> > > pushed to dockerhub. That moved towards general discussion on revision
> >> > > mgmt.
> >> > >
> >> > > Problem we're trying to solve is this:
> >> > > If you build/push images today, your tag is 4.0
> >> > > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> >> > > we tag new release.
> >> > >
> >> > > But image built today is not equal to image built tomorrow, so we
> >> > > would like something like 4.0.0-1, 4.0.0-2.
> >> > > While we can reasonably detect history of revisions in dockerhub,
> >> > > local env will be extremely hard to do.
> >> > >
> >> > > I'd like to ask you for opinions on desired behavior and how we want
> >> > > to deal with revision management in general.
> >> > >
> >> > > Cheers,
> >> > > Michal
> >> > >
> >> >
> >> > What's in the images, kolla? Other OpenStack components?
> >>
> >>
> >> Yes, each image will typically contain all software required for one
> >> OpenStack service, including dependencies from OpenStack projects or the
> >> base OS. Installed via some combination of git, pip, rpm, deb.
> >>
> >> > Where does the
> >> > 4.0.0 come from?
> >> >
> >> >
> >> Its the python version string from the kolla project itself, so ultimately
> >> I think pbr. I'm suggesting that we switch to using the
> >> version.release_string[1] which will tag with the longer version we use for
> >> other dev packages.
> >>
> >> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
> >
> > Why are you tagging the artifacts containing other projects with the
> > version number of kolla, instead of their own version numbers and some
> > sort of incremented build number?
> 
> This is what we do in Kolla and I'd say logistics and simplicity of
> implementation. Tags are more than just information for us. We have to

But for a user consuming the image, they have no idea what version of
nova is in it because the version on the image is tied to a different
application entirely.

> deploy these images and we have to know a tag. Combine that with clear
> separation of build phase from deployment phase (really build phase is
> entirely optional thanks to dockerhub), you'll end up with either
> automagical script that will have to somehow detect correct version
> mix of containers that works with each other, or hand crafted list
> that will have 100+ versions hardcoded.
> 
> Incremental build is hard because builds are atomic and you never
> really know how many times images were rebuilt (also local rebuilt vs
> dockerhub-pushed rebuild will cause collisions in tags).
> 
> > Doug
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

On 18 April 2017 at 12:41, Doug Hellmann  wrote:
> Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
>> wrote:
>>
>> > Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>> > > My dear Kollegues,
>> > >
>> > > Today we had discussion about how to properly name/tag images being
>> > > pushed to dockerhub. That moved towards general discussion on revision
>> > > mgmt.
>> > >
>> > > Problem we're trying to solve is this:
>> > > If you build/push images today, your tag is 4.0
>> > > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>> > > we tag new release.
>> > >
>> > > But image built today is not equal to image built tomorrow, so we
>> > > would like something like 4.0.0-1, 4.0.0-2.
>> > > While we can reasonably detect history of revisions in dockerhub,
>> > > local env will be extremely hard to do.
>> > >
>> > > I'd like to ask you for opinions on desired behavior and how we want
>> > > to deal with revision management in general.
>> > >
>> > > Cheers,
>> > > Michal
>> > >
>> >
>> > What's in the images, kolla? Other OpenStack components?
>>
>>
>> Yes, each image will typically contain all software required for one
>> OpenStack service, including dependencies from OpenStack projects or the
>> base OS. Installed via some combination of git, pip, rpm, deb.
>>
>> > Where does the
>> > 4.0.0 come from?
>> >
>> >
>> Its the python version string from the kolla project itself, so ultimately
>> I think pbr. I'm suggesting that we switch to using the
>> version.release_string[1] which will tag with the longer version we use for
>> other dev packages.
>>
>> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
>
> Why are you tagging the artifacts containing other projects with the
> version number of kolla, instead of their own version numbers and some
> sort of incremented build number?

This is what we do in Kolla and I'd say logistics and simplicity of
implementation. Tags are more than just information for us. We have to
deploy these images and we have to know a tag. Combine that with clear
separation of build phase from deployment phase (really build phase is
entirely optional thanks to dockerhub), you'll end up with either
automagical script that will have to somehow detect correct version
mix of containers that works with each other, or hand crafted list
that will have 100+ versions hardcoded.

Incremental build is hard because builds are atomic and you never
really know how many times images were rebuilt (also local rebuilt vs
dockerhub-pushed rebuild will cause collisions in tags).

> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Doug Hellmann]

Excerpts from Steve Baker's message of 2017-04-18 10:46:43 +1200:
> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
> wrote:
> 
> > Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
> > > My dear Kollegues,
> > >
> > > Today we had discussion about how to properly name/tag images being
> > > pushed to dockerhub. That moved towards general discussion on revision
> > > mgmt.
> > >
> > > Problem we're trying to solve is this:
> > > If you build/push images today, your tag is 4.0
> > > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> > > we tag new release.
> > >
> > > But image built today is not equal to image built tomorrow, so we
> > > would like something like 4.0.0-1, 4.0.0-2.
> > > While we can reasonably detect history of revisions in dockerhub,
> > > local env will be extremely hard to do.
> > >
> > > I'd like to ask you for opinions on desired behavior and how we want
> > > to deal with revision management in general.
> > >
> > > Cheers,
> > > Michal
> > >
> >
> > What's in the images, kolla? Other OpenStack components?
> 
> 
> Yes, each image will typically contain all software required for one
> OpenStack service, including dependencies from OpenStack projects or the
> base OS. Installed via some combination of git, pip, rpm, deb.
> 
> > Where does the
> > 4.0.0 come from?
> >
> >
> Its the python version string from the kolla project itself, so ultimately
> I think pbr. I'm suggesting that we switch to using the
> version.release_string[1] which will tag with the longer version we use for
> other dev packages.
> 
> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py

Why are you tagging the artifacts containing other projects with the
version number of kolla, instead of their own version numbers and some
sort of incremented build number?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

Yes. within a stable branch, the kolla code hardly ever changes. but nova may 
put out a new release, or openssl may put out a new release.

From: Michał Jastrzębski [inc...@gmail.com]
Sent: Tuesday, April 18, 2017 9:19 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

Our issue is a bit complex tho. Dockerhub-pushed images are less
affected by version of our code than versions of everyone else's code.

On 18 April 2017 at 07:36, Flavio Percoco <fla...@redhat.com> wrote:
> On 13/04/17 13:48 +1200, Steve Baker wrote:
>>
>> On Thu, Apr 13, 2017 at 10:59 AM, Michał Jastrzębski <inc...@gmail.com>
>> wrote:
>>
>>> My dear Kollegues,
>>>
>>> Today we had discussion about how to properly name/tag images being
>>> pushed to dockerhub. That moved towards general discussion on revision
>>> mgmt.
>>>
>>> Problem we're trying to solve is this:
>>> If you build/push images today, your tag is 4.0
>>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>> we tag new release.
>>>
>>> But image built today is not equal to image built tomorrow, so we
>>> would like something like 4.0.0-1, 4.0.0-2.
>>> While we can reasonably detect history of revisions in dockerhub,
>>> local env will be extremely hard to do.
>>>
>>> I'd like to ask you for opinions on desired behavior and how we want
>>> to deal with revision management in general.
>>>
>>>
>> I already have a change which proposes tagging images with a pbr built
>> version [1]. I think if users want tags which are stable for the duration
>> of a major release they should switch to using the tag specified by
>> kolla-build.conf base_tag, which can be set to latest, ocata, pike, etc.
>> This would leave the version tag to at least track changes to the kolla
>> repo itself. Since the contents of upstream kolla images come from such
>> diverse sources, all I could suggest to ensure unique tags are created for
>> unique images is to append a datestamp to [1] (or have an extra datestamp
>> based tag). Bonus points for only publishing a new datestamp tag if the
>> contents of the image really changes.
>>
>> In the RDO openstack-kolla package we now tag images with the
>> {Version}-{Release} of the openstack-kolla package which built it[2]. I
>> realise this doesn't solve the problem of the tag needing to change when
>> other image contents need to be updated, but I believe this can be solved
>> within the RDO image build pipeline by incrementing the {Release} whenever
>> a new image needs to be published.
>>
>> [1] https://review.openstack.org/#/c/448380/
>> [2] https://review.rdoproject.org/r/#/c/5923/1/openstack-kolla.spec
>
>
> I like this option better because it's more consistent with how things are
> done
> elsewhere in OpenStack.
>
> Flavio
>
> --
> @flaper87
> Flavio Percoco
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Stephen Hindle]

On Tue, Apr 18, 2017 at 9:27 AM, Fox, Kevin M  wrote:
> Kolla's general target is in the node range of 1 - 100 nodes. A lot of
> smaller clusters can't afford to do ci/cd of their own and so Kolla
> currently pushes all the work to the site, and the site can't afford to do
> it right, and so just doesn't deal with security updates.
>

I agree - as more and more of the '10' end of the '1-100' nodes start
using kolla to experiment with OpenStack, having pre-baked images with
security/patches/etc. becomes more important...

-- 
Stephen Hindle - Senior Systems Engineer
480.807.8189 480.807.8189
www.limelight.com Delivering Faster Better

Join the conversation

at Limelight Connect

-- 
The information in this message may be confidential.  It is intended solely 
for
the addressee(s).  If you are not the intended recipient, any disclosure,
copying or distribution of the message, or any action or omission taken by 
you
in reliance on it, is prohibited and may be unlawful.  Please immediately
contact the sender if you have received this message in error.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

Kolla's general target is in the node range of 1 - 100 nodes. A lot of smaller 
clusters can't afford to do ci/cd of their own and so Kolla currently pushes 
all the work to the site, and the site can't afford to do it right, and so just 
doesn't deal with security updates.

This is akin to a site forcing super secure passwords and so all the users just 
write the password on a sticky note and put it under their keyboard. The ideal 
is extra security, but the reality is less security.

Keeping up to date containers with security updates applied  and trackable 
helps solve a real issue. I agree, it is better to do it at your own site, if 
you can afford it, but like most things security, its not a bool, secure or 
not, its a spectrum. Providing tagged up to date images on the hub strengthens 
the security of a lot of openstacks.

It also lowers the hugely high bar of getting a production system workable. You 
don't have to build all the containers from the get go to get a system going. 
You can do that as you get time to do so.

Thanks,
Kevin


From: Jeffrey Zhang [zhang.lei@gmail.com]
Sent: Monday, April 17, 2017 7:43 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

I think we have two topics and improvements here

1. images in https://hub.docker.com/r/kolla/
2. tag in end-user env.

# images in hub.docker.com<http://hub.docker.com>

we are building kolla tag image and push them into 
hub.docker.com<http://hub.docker.com>. After this,
we do nothing for these images.

The issue is

1. any security update is not included in these images.
   solution: I do not think use 4.0.0-1 4.0.0-2 in 
hub.docker.com<http://hub.docker.com> is a good idea.
   if so, we need mark what 4.0.0-1 container and what's the difference with 
4.0.0-2.
   This will make another chaos.
   And any prod env shouldn't depend on hub.docker.com<http://hub.docker.com>'s 
images, which is vulnerable
   to attack and is mutable.

2. branch images are not pushed.
   solution: we can add a job to push branch images into 
hub.docker.com<http://hub.docker.com> like inc0
   said. For example:
   centos-source-nova-api:4.0.0
   centos-source-nova-api:ocata
   centos-source-nova-api:pike
   centos-source-nova-api:master
   But branch tag images is not stable ( even its name is stable/ocata ), users 
are
   not recommended to use these images

# images in end-user env

I recommended end user should build its own image rather then use 
hub.docker.com<http://hub.docker.com> directly.
in my env, I build images with following tag rule.

when using 4.0.0 to build multi time, i use different tag name. For example
   1st: 4.0.0.1
   2nd: 4.0.0.2
   3rd: 4.0.0.3
   ...

The advantage in this way is: keep each tag as immutable ( never override )

On Tue, Apr 18, 2017 at 6:46 AM, Steve Baker 
<sba...@redhat.com<mailto:sba...@redhat.com>> wrote:


On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
<d...@doughellmann.com<mailto:d...@doughellmann.com>> wrote:
Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
> My dear Kollegues,
>
> Today we had discussion about how to properly name/tag images being
> pushed to dockerhub. That moved towards general discussion on revision
> mgmt.
>
> Problem we're trying to solve is this:
> If you build/push images today, your tag is 4.0
> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> we tag new release.
>
> But image built today is not equal to image built tomorrow, so we
> would like something like 4.0.0-1, 4.0.0-2.
> While we can reasonably detect history of revisions in dockerhub,
> local env will be extremely hard to do.
>
> I'd like to ask you for opinions on desired behavior and how we want
> to deal with revision management in general.
>
> Cheers,
> Michal
>

What's in the images, kolla? Other OpenStack components?

Yes, each image will typically contain all software required for one OpenStack 
service, including dependencies from OpenStack projects or the base OS. 
Installed via some combination of git, pip, rpm, deb.

Where does the
4.0.0 come from?


Its the python version string from the kolla project itself, so ultimately I 
think pbr. I'm suggesting that we switch to using the version.release_string[1] 
which will tag with the longer version we use for other dev packages.

[1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Regards,
Jeffrey Zh

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

I think we need to meet the following:

 1. provide up to date containers including if the dependencies changed without 
the main package changed.
 2. trackable changes. If a dep in a container changed, the tag should be 
different. This allows users to know that something changed.
 3. atomic changes. The version on their system now vs the new version should 
be easily separable so roll forward/rollback is supported in case something 
goes wrong.
 4. minimal changes. The revision should be bumped only if something did 
change, not just was rebuilt but nothing changed.
 5. tested changes. The gate jobs should be run to successful completion before 
released.

With this arrangement, and with a similar arrangement for helm package 
revisioning, the user can:

helm upgrade --name nova-compute kolla/nova-compute

And if any of the containers changed, it will perform a rolling upgrade.

Something that can be easily put in a cron job and should have minimal impact.

:ocata or 4.0.0 doesn't meet #2 and #3. I'm ok with having those tags in 
addition to revisions to support folks that dont want to follow the pattern 
above. In that case, they would just point to the newest revision.

Thanks,
Kevin


From: Michał Jastrzębski [inc...@gmail.com]
Sent: Monday, April 17, 2017 8:34 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

So, while I agree that everyone should use images built locally, I
also see what is value of downloading dockerhub-hosted images (besides
speed). Images at dockerhub passed our gates and are CI tested. Which
means quality of these images is ensured by our CI. Not everyone can
afford to have CI/CD system in their infra, so for small/medium
installation this actually might be more stable than local builds.

Given that both local and dockerhub hosted images have valid
production use case, I'd like that we keep our tagging mechanism same
for both.
That makes revisions per se impossible to handle (4.0.0-3 on dockerhub
doesn't mean 4.0.0-3 locally). Also how often we push 4.0.0-x? Daily
for quickest update on security patches (preferably for me), that
would mean that our dockerhub registry would grow extremely fast if
we'd like to retain every revision. One idea would be to put a little
of this weight on users (sorry!). We could upload daily :ocata images
and delete old ones. I think not many people does daily openstack
upgrades (cool use case tho), that means they will have some stale
:ocata image locally. We can just put an expectation to backup your
images locally, ones that you actually use. Just tar.gz
/var/lib/docker/volumes/registry/_data and save it somewhere safe.
Then you can always come back to it.

Bottom line, my suggestion for now is to have schema like that:

image-name:4.0.0 -> corresponding docker tag and image built (pref
automatically) close to tagging date
image-name:ocata -> tip of ocata branch build daily - the freshest
code that past gates
image-name:master -> tip of master branch

To achieve fully repeatable builds, we would need to have 4.0.0 type
tagging (based on pbr+git tag), version manifesto generation (as
discussed on PTG) and mechanism to consume existing manifestos and
rebuild images with these exact versions (baring issues with repo
removing previous version...). That is quite a project in it's own...

Thoughts?

Cheers,
Michal

On 17 April 2017 at 19:43, Jeffrey Zhang <zhang.lei@gmail.com> wrote:
> I think we have two topics and improvements here
>
> 1. images in https://hub.docker.com/r/kolla/
> 2. tag in end-user env.
>
> # images in hub.docker.com
>
> we are building kolla tag image and push them into hub.docker.com. After
> this,
> we do nothing for these images.
>
> The issue is
>
> 1. any security update is not included in these images.
>solution: I do not think use 4.0.0-1 4.0.0-2 in hub.docker.com is a good
> idea.
>if so, we need mark what 4.0.0-1 container and what's the difference with
> 4.0.0-2.
>This will make another chaos.
>And any prod env shouldn't depend on hub.docker.com's images, which is
> vulnerable
>to attack and is mutable.
>
> 2. branch images are not pushed.
>solution: we can add a job to push branch images into hub.docker.com like
> inc0
>said. For example:
>centos-source-nova-api:4.0.0
>centos-source-nova-api:ocata
>centos-source-nova-api:pike
>centos-source-nova-api:master
>But branch tag images is not stable ( even its name is stable/ocata ),
> users are
>not recommended to use these images
>
> # images in end-user env


> I recommended end user should build its own image rather then use
> hub.docker.com directly.
> in my env, I build images with following tag rule.
>
> when using 4.0.0 to build multi time, i

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

Our issue is a bit complex tho. Dockerhub-pushed images are less
affected by version of our code than versions of everyone else's code.

On 18 April 2017 at 07:36, Flavio Percoco  wrote:
> On 13/04/17 13:48 +1200, Steve Baker wrote:
>>
>> On Thu, Apr 13, 2017 at 10:59 AM, Michał Jastrzębski 
>> wrote:
>>
>>> My dear Kollegues,
>>>
>>> Today we had discussion about how to properly name/tag images being
>>> pushed to dockerhub. That moved towards general discussion on revision
>>> mgmt.
>>>
>>> Problem we're trying to solve is this:
>>> If you build/push images today, your tag is 4.0
>>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>> we tag new release.
>>>
>>> But image built today is not equal to image built tomorrow, so we
>>> would like something like 4.0.0-1, 4.0.0-2.
>>> While we can reasonably detect history of revisions in dockerhub,
>>> local env will be extremely hard to do.
>>>
>>> I'd like to ask you for opinions on desired behavior and how we want
>>> to deal with revision management in general.
>>>
>>>
>> I already have a change which proposes tagging images with a pbr built
>> version [1]. I think if users want tags which are stable for the duration
>> of a major release they should switch to using the tag specified by
>> kolla-build.conf base_tag, which can be set to latest, ocata, pike, etc.
>> This would leave the version tag to at least track changes to the kolla
>> repo itself. Since the contents of upstream kolla images come from such
>> diverse sources, all I could suggest to ensure unique tags are created for
>> unique images is to append a datestamp to [1] (or have an extra datestamp
>> based tag). Bonus points for only publishing a new datestamp tag if the
>> contents of the image really changes.
>>
>> In the RDO openstack-kolla package we now tag images with the
>> {Version}-{Release} of the openstack-kolla package which built it[2]. I
>> realise this doesn't solve the problem of the tag needing to change when
>> other image contents need to be updated, but I believe this can be solved
>> within the RDO image build pipeline by incrementing the {Release} whenever
>> a new image needs to be published.
>>
>> [1] https://review.openstack.org/#/c/448380/
>> [2] https://review.rdoproject.org/r/#/c/5923/1/openstack-kolla.spec
>
>
> I like this option better because it's more consistent with how things are
> done
> elsewhere in OpenStack.
>
> Flavio
>
> --
> @flaper87
> Flavio Percoco
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Flavio Percoco]

On 13/04/17 13:48 +1200, Steve Baker wrote:

On Thu, Apr 13, 2017 at 10:59 AM, Michał Jastrzębski 
wrote:


My dear Kollegues,

Today we had discussion about how to properly name/tag images being
pushed to dockerhub. That moved towards general discussion on revision
mgmt.

Problem we're trying to solve is this:
If you build/push images today, your tag is 4.0
if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
we tag new release.

But image built today is not equal to image built tomorrow, so we
would like something like 4.0.0-1, 4.0.0-2.
While we can reasonably detect history of revisions in dockerhub,
local env will be extremely hard to do.

I'd like to ask you for opinions on desired behavior and how we want
to deal with revision management in general.



I already have a change which proposes tagging images with a pbr built
version [1]. I think if users want tags which are stable for the duration
of a major release they should switch to using the tag specified by
kolla-build.conf base_tag, which can be set to latest, ocata, pike, etc.
This would leave the version tag to at least track changes to the kolla
repo itself. Since the contents of upstream kolla images come from such
diverse sources, all I could suggest to ensure unique tags are created for
unique images is to append a datestamp to [1] (or have an extra datestamp
based tag). Bonus points for only publishing a new datestamp tag if the
contents of the image really changes.

In the RDO openstack-kolla package we now tag images with the
{Version}-{Release} of the openstack-kolla package which built it[2]. I
realise this doesn't solve the problem of the tag needing to change when
other image contents need to be updated, but I believe this can be solved
within the RDO image build pipeline by incrementing the {Release} whenever
a new image needs to be published.

[1] https://review.openstack.org/#/c/448380/
[2] https://review.rdoproject.org/r/#/c/5923/1/openstack-kolla.spec


I like this option better because it's more consistent with how things are done
elsewhere in OpenStack.

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

So, while I agree that everyone should use images built locally, I
also see what is value of downloading dockerhub-hosted images (besides
speed). Images at dockerhub passed our gates and are CI tested. Which
means quality of these images is ensured by our CI. Not everyone can
afford to have CI/CD system in their infra, so for small/medium
installation this actually might be more stable than local builds.

Given that both local and dockerhub hosted images have valid
production use case, I'd like that we keep our tagging mechanism same
for both.
That makes revisions per se impossible to handle (4.0.0-3 on dockerhub
doesn't mean 4.0.0-3 locally). Also how often we push 4.0.0-x? Daily
for quickest update on security patches (preferably for me), that
would mean that our dockerhub registry would grow extremely fast if
we'd like to retain every revision. One idea would be to put a little
of this weight on users (sorry!). We could upload daily :ocata images
and delete old ones. I think not many people does daily openstack
upgrades (cool use case tho), that means they will have some stale
:ocata image locally. We can just put an expectation to backup your
images locally, ones that you actually use. Just tar.gz
/var/lib/docker/volumes/registry/_data and save it somewhere safe.
Then you can always come back to it.

Bottom line, my suggestion for now is to have schema like that:

image-name:4.0.0 -> corresponding docker tag and image built (pref
automatically) close to tagging date
image-name:ocata -> tip of ocata branch build daily - the freshest
code that past gates
image-name:master -> tip of master branch

To achieve fully repeatable builds, we would need to have 4.0.0 type
tagging (based on pbr+git tag), version manifesto generation (as
discussed on PTG) and mechanism to consume existing manifestos and
rebuild images with these exact versions (baring issues with repo
removing previous version...). That is quite a project in it's own...

Thoughts?

Cheers,
Michal

On 17 April 2017 at 19:43, Jeffrey Zhang  wrote:
> I think we have two topics and improvements here
>
> 1. images in https://hub.docker.com/r/kolla/
> 2. tag in end-user env.
>
> # images in hub.docker.com
>
> we are building kolla tag image and push them into hub.docker.com. After
> this,
> we do nothing for these images.
>
> The issue is
>
> 1. any security update is not included in these images.
>solution: I do not think use 4.0.0-1 4.0.0-2 in hub.docker.com is a good
> idea.
>if so, we need mark what 4.0.0-1 container and what's the difference with
> 4.0.0-2.
>This will make another chaos.
>And any prod env shouldn't depend on hub.docker.com's images, which is
> vulnerable
>to attack and is mutable.
>
> 2. branch images are not pushed.
>solution: we can add a job to push branch images into hub.docker.com like
> inc0
>said. For example:
>centos-source-nova-api:4.0.0
>centos-source-nova-api:ocata
>centos-source-nova-api:pike
>centos-source-nova-api:master
>But branch tag images is not stable ( even its name is stable/ocata ),
> users are
>not recommended to use these images
>
> # images in end-user env


> I recommended end user should build its own image rather then use
> hub.docker.com directly.
> in my env, I build images with following tag rule.
>
> when using 4.0.0 to build multi time, i use different tag name. For example
>1st: 4.0.0.1
>2nd: 4.0.0.2
>3rd: 4.0.0.3
>...
>
> The advantage in this way is: keep each tag as immutable ( never override )
>
> On Tue, Apr 18, 2017 at 6:46 AM, Steve Baker  wrote:
>>
>>
>>
>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
>> wrote:
>>>
>>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>>> > My dear Kollegues,
>>> >
>>> > Today we had discussion about how to properly name/tag images being
>>> > pushed to dockerhub. That moved towards general discussion on revision
>>> > mgmt.
>>> >
>>> > Problem we're trying to solve is this:
>>> > If you build/push images today, your tag is 4.0
>>> > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>>> > we tag new release.
>>> >
>>> > But image built today is not equal to image built tomorrow, so we
>>> > would like something like 4.0.0-1, 4.0.0-2.
>>> > While we can reasonably detect history of revisions in dockerhub,
>>> > local env will be extremely hard to do.
>>> >
>>> > I'd like to ask you for opinions on desired behavior and how we want
>>> > to deal with revision management in general.
>>> >
>>> > Cheers,
>>> > Michal
>>> >
>>>
>>> What's in the images, kolla? Other OpenStack components?
>>
>>
>> Yes, each image will typically contain all software required for one
>> OpenStack service, including dependencies from OpenStack projects or the
>> base OS. Installed via some combination of git, pip, rpm, deb.
>>
>>>
>>> Where does the
>>> 4.0.0 come from?
>>>
>>
>> Its the 

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Jeffrey Zhang]

I think we have two topics and improvements here

1. images in https://hub.docker.com/r/kolla/
2. tag in end-user env.

# images in hub.docker.com

we are building kolla tag image and push them into hub.docker.com. After
this,
we do nothing for these images.

The issue is

1. any security update is not included in these images.
   solution: I do not think use 4.0.0-1 4.0.0-2 in hub.docker.com is a good
idea.
   if so, we need mark what 4.0.0-1 container and what's the difference
with 4.0.0-2.
   This will make another chaos.
   And any prod env shouldn't depend on hub.docker.com's images, which is
vulnerable
   to attack and is mutable.

2. branch images are not pushed.
   solution: we can add a job to push branch images into hub.docker.com
like inc0
   said. For example:
   centos-source-nova-api:4.0.0
   centos-source-nova-api:ocata
   centos-source-nova-api:pike
   centos-source-nova-api:master
   But branch tag images is not stable ( even its name is stable/ocata ),
users are
   not recommended to use these images

# images in end-user env

I recommended end user should build its own image rather then use
hub.docker.com directly.
in my env, I build images with following tag rule.

when using 4.0.0 to build multi time, i use different tag name. For example
   1st: 4.0.0.1
   2nd: 4.0.0.2
   3rd: 4.0.0.3
   ...

The advantage in this way is: keep each tag as immutable ( never override )

On Tue, Apr 18, 2017 at 6:46 AM, Steve Baker  wrote:

>
>
> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
> wrote:
>
>> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
>> > My dear Kollegues,
>> >
>> > Today we had discussion about how to properly name/tag images being
>> > pushed to dockerhub. That moved towards general discussion on revision
>> > mgmt.
>> >
>> > Problem we're trying to solve is this:
>> > If you build/push images today, your tag is 4.0
>> > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>> > we tag new release.
>> >
>> > But image built today is not equal to image built tomorrow, so we
>> > would like something like 4.0.0-1, 4.0.0-2.
>> > While we can reasonably detect history of revisions in dockerhub,
>> > local env will be extremely hard to do.
>> >
>> > I'd like to ask you for opinions on desired behavior and how we want
>> > to deal with revision management in general.
>> >
>> > Cheers,
>> > Michal
>> >
>>
>> What's in the images, kolla? Other OpenStack components?
>
>
> Yes, each image will typically contain all software required for one
> OpenStack service, including dependencies from OpenStack projects or the
> base OS. Installed via some combination of git, pip, rpm, deb.
>
>
>> Where does the
>> 4.0.0 come from?
>>
>>
> Its the python version string from the kolla project itself, so ultimately
> I think pbr. I'm suggesting that we switch to using the
> version.release_string[1] which will tag with the longer version we use for
> other dev packages.
>
> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Steve Baker]

On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann 
wrote:

> Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
> > My dear Kollegues,
> >
> > Today we had discussion about how to properly name/tag images being
> > pushed to dockerhub. That moved towards general discussion on revision
> > mgmt.
> >
> > Problem we're trying to solve is this:
> > If you build/push images today, your tag is 4.0
> > if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> > we tag new release.
> >
> > But image built today is not equal to image built tomorrow, so we
> > would like something like 4.0.0-1, 4.0.0-2.
> > While we can reasonably detect history of revisions in dockerhub,
> > local env will be extremely hard to do.
> >
> > I'd like to ask you for opinions on desired behavior and how we want
> > to deal with revision management in general.
> >
> > Cheers,
> > Michal
> >
>
> What's in the images, kolla? Other OpenStack components?


Yes, each image will typically contain all software required for one
OpenStack service, including dependencies from OpenStack projects or the
base OS. Installed via some combination of git, pip, rpm, deb.


> Where does the
> 4.0.0 come from?
>
>
Its the python version string from the kolla project itself, so ultimately
I think pbr. I'm suggesting that we switch to using the
version.release_string[1] which will tag with the longer version we use for
other dev packages.

[1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Doug Hellmann]

Excerpts from Michał Jastrzębski's message of 2017-04-12 15:59:34 -0700:
> My dear Kollegues,
> 
> Today we had discussion about how to properly name/tag images being
> pushed to dockerhub. That moved towards general discussion on revision
> mgmt.
> 
> Problem we're trying to solve is this:
> If you build/push images today, your tag is 4.0
> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> we tag new release.
> 
> But image built today is not equal to image built tomorrow, so we
> would like something like 4.0.0-1, 4.0.0-2.
> While we can reasonably detect history of revisions in dockerhub,
> local env will be extremely hard to do.
> 
> I'd like to ask you for opinions on desired behavior and how we want
> to deal with revision management in general.
> 
> Cheers,
> Michal
> 

What's in the images, kolla? Other OpenStack components? Where does the
4.0.0 come from?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

Does reversion of deps get handled? Thats the primary concern I have. I don't
think pip/pbr handle it as deps are external to the package. Containers have
deps bundled into the actual build, so need to be accounted for somehow in the 
tag.

Thanks,
Kevin

From: Jay Pipes [jaypi...@gmail.com]
Sent: Friday, April 14, 2017 1:47 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

Or you could use pbr's strategy, which tags based on git tags and
revision history:

https://docs.openstack.org/developer/pbr/#version

"If the currently checked out revision is not tagged, then we take the
last tagged version number and increment it to get a minimum target
version."

Best,
-jay

On 04/13/2017 04:14 AM, Bogdan Dobrelya wrote:
> On 13.04.2017 0:59, Michał Jastrzębski wrote:
>> My dear Kollegues,
>>
>> Today we had discussion about how to properly name/tag images being
>> pushed to dockerhub. That moved towards general discussion on revision
>> mgmt.
>>
>> Problem we're trying to solve is this:
>> If you build/push images today, your tag is 4.0
>> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
>> we tag new release.
>
> +1 that this should be improved, indeed.
>
>>
>> But image built today is not equal to image built tomorrow, so we
>> would like something like 4.0.0-1, 4.0.0-2.
>
> Please take a look the build flow for a neighbour project Kubernetes
> [0]. I believe it addresses each of the possible versioning aspects,
> like stable branch builds, master builds, infrastructure vs local builds
> and so on. Kolla could reuse some build scripts and follow the similar
> versioning scheme from that project and save time as well.
>
> [0] https://github.com/kubernetes/kubernetes/tree/master/build#basic-flow
>
>> While we can reasonably detect history of revisions in dockerhub,
>> local env will be extremely hard to do.
>>
>> I'd like to ask you for opinions on desired behavior and how we want
>> to deal with revision management in general.
>>
>> Cheers,
>> Michal
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Jay Pipes]

Or you could use pbr's strategy, which tags based on git tags and 
revision history:


https://docs.openstack.org/developer/pbr/#version

"If the currently checked out revision is not tagged, then we take the 
last tagged version number and increment it to get a minimum target 
version."


Best,
-jay

On 04/13/2017 04:14 AM, Bogdan Dobrelya wrote:

On 13.04.2017 0:59, Michał Jastrzębski wrote:

My dear Kollegues,

Today we had discussion about how to properly name/tag images being
pushed to dockerhub. That moved towards general discussion on revision
mgmt.

Problem we're trying to solve is this:
If you build/push images today, your tag is 4.0
if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
we tag new release.


+1 that this should be improved, indeed.



But image built today is not equal to image built tomorrow, so we
would like something like 4.0.0-1, 4.0.0-2.


Please take a look the build flow for a neighbour project Kubernetes
[0]. I believe it addresses each of the possible versioning aspects,
like stable branch builds, master builds, infrastructure vs local builds
and so on. Kolla could reuse some build scripts and follow the similar
versioning scheme from that project and save time as well.

[0] https://github.com/kubernetes/kubernetes/tree/master/build#basic-flow


While we can reasonably detect history of revisions in dockerhub,
local env will be extremely hard to do.

I'd like to ask you for opinions on desired behavior and how we want
to deal with revision management in general.

Cheers,
Michal

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Bogdan Dobrelya]

On 13.04.2017 0:59, Michał Jastrzębski wrote:
> My dear Kollegues,
> 
> Today we had discussion about how to properly name/tag images being
> pushed to dockerhub. That moved towards general discussion on revision
> mgmt.
> 
> Problem we're trying to solve is this:
> If you build/push images today, your tag is 4.0
> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> we tag new release.

+1 that this should be improved, indeed.

> 
> But image built today is not equal to image built tomorrow, so we
> would like something like 4.0.0-1, 4.0.0-2.

Please take a look the build flow for a neighbour project Kubernetes
[0]. I believe it addresses each of the possible versioning aspects,
like stable branch builds, master builds, infrastructure vs local builds
and so on. Kolla could reuse some build scripts and follow the similar
versioning scheme from that project and save time as well.

[0] https://github.com/kubernetes/kubernetes/tree/master/build#basic-flow

> While we can reasonably detect history of revisions in dockerhub,
> local env will be extremely hard to do.
> 
> I'd like to ask you for opinions on desired behavior and how we want
> to deal with revision management in general.
> 
> Cheers,
> Michal
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Steve Baker]

On Thu, Apr 13, 2017 at 10:59 AM, Michał Jastrzębski 
wrote:

> My dear Kollegues,
>
> Today we had discussion about how to properly name/tag images being
> pushed to dockerhub. That moved towards general discussion on revision
> mgmt.
>
> Problem we're trying to solve is this:
> If you build/push images today, your tag is 4.0
> if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
> we tag new release.
>
> But image built today is not equal to image built tomorrow, so we
> would like something like 4.0.0-1, 4.0.0-2.
> While we can reasonably detect history of revisions in dockerhub,
> local env will be extremely hard to do.
>
> I'd like to ask you for opinions on desired behavior and how we want
> to deal with revision management in general.
>
>
I already have a change which proposes tagging images with a pbr built
version [1]. I think if users want tags which are stable for the duration
of a major release they should switch to using the tag specified by
kolla-build.conf base_tag, which can be set to latest, ocata, pike, etc.
This would leave the version tag to at least track changes to the kolla
repo itself. Since the contents of upstream kolla images come from such
diverse sources, all I could suggest to ensure unique tags are created for
unique images is to append a datestamp to [1] (or have an extra datestamp
based tag). Bonus points for only publishing a new datestamp tag if the
contents of the image really changes.

In the RDO openstack-kolla package we now tag images with the
{Version}-{Release} of the openstack-kolla package which built it[2]. I
realise this doesn't solve the problem of the tag needing to change when
other image contents need to be updated, but I believe this can be solved
within the RDO image build pipeline by incrementing the {Release} whenever
a new image needs to be published.

[1] https://review.openstack.org/#/c/448380/
[2] https://review.rdoproject.org/r/#/c/5923/1/openstack-kolla.spec
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Tags, revisions, dockerhub

[Fox, Kevin M]

+1 to revision information in tags. If you don't have it, depending on when 
containers start, you could have different versions of software with/without 
bug fixes running on the same cluster without an easy way to know.

We've seen this with libvirt versions being 1.x in some builds and 2.x in 
fresher builds.

We can still keep around the unrevisioned tags too for folks that don't want to 
deal with the revision numbers. in which case, the unrevisioned 4.0.0 tag would 
map to the newest revisioned tag, say 4.0.0-2 and would have the same behavior 
then as is today.

Thanks,
Kevin

From: Michał Jastrzębski [inc...@gmail.com]
Sent: Wednesday, April 12, 2017 3:59 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [kolla] Tags, revisions, dockerhub

My dear Kollegues,

Today we had discussion about how to properly name/tag images being
pushed to dockerhub. That moved towards general discussion on revision
mgmt.

Problem we're trying to solve is this:
If you build/push images today, your tag is 4.0
if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
we tag new release.

But image built today is not equal to image built tomorrow, so we
would like something like 4.0.0-1, 4.0.0-2.
While we can reasonably detect history of revisions in dockerhub,
local env will be extremely hard to do.

I'd like to ask you for opinions on desired behavior and how we want
to deal with revision management in general.

Cheers,
Michal

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla] Tags, revisions, dockerhub

[Michał Jastrzębski]

My dear Kollegues,

Today we had discussion about how to properly name/tag images being
pushed to dockerhub. That moved towards general discussion on revision
mgmt.

Problem we're trying to solve is this:
If you build/push images today, your tag is 4.0
if you do it tomorrow, it's still 4.0, and will keep being 4.0 until
we tag new release.

But image built today is not equal to image built tomorrow, so we
would like something like 4.0.0-1, 4.0.0-2.
While we can reasonably detect history of revisions in dockerhub,
local env will be extremely hard to do.

I'd like to ask you for opinions on desired behavior and how we want
to deal with revision management in general.

Cheers,
Michal

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev