Ok so then next iteration of proposal: The only doubt I have myself is the "master" vs. "master-prod". Maybe we should rather have "master" for "production-ready" image and use a different name for the "small-ci" image". Maybe "trimci" or "ci-lite" or "liteci" ? What do you think?
*Versions from master (development use only):* - Main non-CI images (small) *: airflow:master-python3.5, airflow:**master-python3.6, airflow:master*==airflow:master-python3.6 - CI images (big) *: airflow:master-ci-python3.5, airflow:**master-ci-python3.6, airflow:master-ci*==airflow:master-ci-python3.6 - Production optimised images: (future): *airflow:master-prod-python3.5, airflow:**master-prod-python3.6, airflow:master-prod* ==airflow:master-prod-python3.6 *Release versions (future):* - Main non-CI images (small): *airflow:1.10.4-python3.5, * *airflow:1.10.4**-python3.6, airflow:latest*==airflow:1.10.4-python3.5 - CI images (big): *airflow:1.10.4-ci-python3.5, **airflow:1.10.4**-ci-python3.6, airflow:latest-ci*==airflow:1.10.4-ci-python3.6 - Production optimised images: *airflow:1.10.4-prod-python3.5, * *airflow:1.10.4**-prod-python3.6, airflow:latest-prod* ==airflow:1.10.4-prod-python3.6 On Mon, Jun 17, 2019 at 10:31 PM Philippe Gagnon <[email protected]> wrote: > That makes sense. The reason I had doubts is because of the way docker hub > lists image tags together -- there's no real differentiation between > pre-release and release builds. But then I suppose that if the tagging > scheme is explicit enough it shouldn't be an issue. > > +1 on `:latest` being an official release. > > On Mon, Jun 17, 2019 at 10:25 AM Ash Berlin-Taylor <[email protected]> wrote: > > > That page does mention "Nightly" builds which is close to what building > > master would be. The other thing that matters is what we actual call A > > Release. > > > > > Do not include any links on the project website that might encourage > > non-developers to download and use nightly builds, snapshots, release > > candidates, or any other similar package > > > > I think we're find so long as we don't do that -- or in this case, since > > we will probably want to link to the docker hub page once we have > versioned > > images there if we make it clear that `:master` is not intended for end > > users, and by the same argument if we have anything as `:latest` it > should > > be a docker image relating to an official Release. > > > > Jarek: no `latest` pointing at CI images please. > > > > -a > > > > > On 17 Jun 2019, at 15:04, Philippe Gagnon <[email protected]> > wrote: > > > > > > One thing: we talked about releasing images under a "master" tag > > (perhaps in another thread?), we should check if this is compatible with > > Apache's release policy [1]. It's not clear to me if this is allowable or > > not after a cursory reading. > > > > > > [1] http://www.apache.org/legal/release-policy.html#what > > > > > > On Mon, Jun 17, 2019 at 9:48 AM Jarek Potiuk <[email protected] > > > > wrote: > > > Anyone has more comments. I think prevailing opnion is: > > > 1) To keep all images in one repo (apache/airflow) > > > 2) I am not sure about labelling but I might try to document all cases > > in a > > > "production" image proposal that I would like to start as soon as we > > merge > > > the current CI image (which I think is quite close to finalisation). > > > > > > J. > > > > > > On Tue, Jun 11, 2019 at 10:59 PM Jarek Potiuk < > [email protected]> > > > wrote: > > > > > > > It's super easy to do :) > > > > > > > > On Tue, Jun 11, 2019 at 10:33 PM Ash Berlin-Taylor <[email protected]> > > wrote: > > > > > > > >> I'm fine with us just publishing release images using the newest > > python > > > >> release (i.e. 3.7) as the main reason we support older python > > versions is > > > >> to support distros thats ship those versions.(i.e. Deb stable), but > I > > don't > > > >> think we need to support that in docker. > > > >> > > > >> (But if it's easy to do since we want them for ci then sure) > > > >> > > > >> -ash > > > >> > > > >> On 11 June 2019 21:21:28 BST, Jarek Potiuk < > [email protected]> > > > >> wrote: > > > >>> > > > >>> Yeah Kamil - python 3.5 is the default one for now. I think we > > should have > > > >>> another discussion here - how many versions to support. There is > this > > > >>> ticket opened today : > > https://issues.apache.org/jira/browse/AIRFLOW-4762 about > > > >>> supporting python 3.6 and 3.7 in tests. Anyone has a strong opinion > > on > > > >>> this? I am for testing on all 3.5, 3.6 and 3.7 even if it increases > > the > > > >>> build/test time on Travis. There are a number of differences > between > > those > > > >>> major versions (I have a blog post about it in writing ) but I > think > > there > > > >>> is concern about eating Apache Travis time. > > > >>> > > > >>> Anyone against those three ? > > > >>> > > > >>> On Tue, Jun 11, 2019 at 8:38 PM Kamil BreguĊa < > > [email protected]> > > > >>> wrote: > > > >>> > > > >>> 1) I would prefer to use one repository. > > > >>>> +1 > > > >>>> > > > >>>> 2) The presented schema looks logical to me. I had doubts whether > > > >>>> Python 3.5 was a good choice for "latest" version, but I checked > > that > > > >>>> travis uses only this version. > > > >>>> > > > >>>> On Tue, Jun 11, 2019 at 3:04 PM Jarek Potiuk < > > [email protected]> > > > >>>> wrote: > > > >>>> > > > >>>>> > > > >>>>> Hello everyone, > > > >>>>> > > > >>>>> We are close to finish AIP-10 (Airlfow image for CI) and seems > > that we > > > >>>>> > > > >>>> will > > > >>>> > > > >>>>> start working soon on an official image AIP, but in the meantime > > we have > > > >>>>> 1.10.4 release coming and we would like to agree tagging scheme > > used for > > > >>>>> the current CI images. We discussed it a bit on Slack, but it's > > time to > > > >>>>> bring it here. I created a JIRA issue for it: > > > >>>>> https://issues.apache.org/jira/browse/AIRFLOW-4764 and my > > proposals > > > >>>>> > > > >>>> after > > > >>>> > > > >>>>> the initial discussion are those: > > > >>>>> > > > >>>>> First of all we have different images that we can talk about : > > > >>>>> > > > >>>>> 1. "base" one - with bare development-ready airflow with > > minimum set > > > >>>>> > > > >>>> of > > > >>>> > > > >>>>> dependencies > > > >>>>> 2. "CI" with all the tools packages that are needed for CI > > tests > > > >>>>> 3. Soon we will likely have an "official" one which might be > > used in > > > >>>>> similar fashion as the "puckel" one. > > > >>>>> > > > >>>>> There are two decisions to make: > > > >>>>> > > > >>>>> 1) How to keep those images - in one repository or whether we > > should have > > > >>>>> separate repos. > > > >>>>> > > > >>>>> It is easier for now to keep all of them within apache/airflow > > > >>>>> < > > https://cloud.docker.com/u/apache/repository/docker/apache/airflow> > > > >>>>> > > > >>>> repository > > > >>>> > > > >>>>> it seems and use a labelling scheme to separate those (there is > > nothing > > > >>>>> wrong with that but it might seem a bit hacky). It's a bit > easier > > to > > > >>>>> maintain with access and CI. > > > >>>>> > > > >>>>> We could also think about separate apache/airflow-ci, > > apache/airflow-dev, > > > >>>>> apache/airflow-prod or smth similar - that would require some > > > >>>>> infrastructure tickets and is not very common. > > > >>>>> > > > >>>>> 2) What labelling scheme to use(apache/airflow:label). My > > proposal is > > > >>>>> similar to this (if we keep everything in the airflow > repository) > > > >>>>> > > > >>>>> - *latest* = latest released version (python 3.5) = * > > > >>>>> > > > >>>> v1.10.3-python3.5* > > > >>>> > > > >>>>> - *master* = latest master version (python 3.5) = > > > >>>>> > > > >>>> *v2.0.0dev0-python3.5* > > > >>>> > > > >>>>> - *v1.10.3-python3.5,v1.10.3-python3.6* - released 1.10.3 > > with python > > > >>>>> 3.5/3.6 > > > >>>>> - *latest-ci *= latest released version of CI variant (python > > 3.5) > > > >>>>> *v1.10.3-ci-python3.5* > > > >>>>> - *master-ci* = latest master version of CI variant (python > > 3.5) > > > >>>>> *v2.0.0dev0-ci-python3.5* > > > >>>>> - *v1.10.3-ci-python3.5, v1.10.3-ci-python3.6* - released > > 1.10.3 with > > > >>>>> python 3.5/3.6 > > > >>>>> > > > >>>>> > > > >>>>> My preference is to keep all the images in one repo and use > > labelling > > > >>>>> scheme as above, > > > >>>>> but I am open to discuss this. > > > >>>>> > > > >>>>> J, > > > >>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> > > > >>>>> Jarek Potiuk > > > >>>>> Polidea <https://www.polidea.com/> | Principal Software > Engineer > > > >>>>> > > > >>>>> M: +48 660 796 129 <+48660796129> > > > >>>>> [image: Polidea] <https://www.polidea.com/> > > > >>>>> > > > >>>> > > > >>>> > > > >>> > > > > > > > > -- > > > > > > > > Jarek Potiuk > > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > > > M: +48 660 796 129 <+48660796129> > > > > [image: Polidea] <https://www.polidea.com/> > > > > > > > > > > > > > > -- > > > > > > Jarek Potiuk > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > M: +48 660 796 129 <+48660796129> > > > [image: Polidea] <https://www.polidea.com/> > > > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
