OK. The next build should prepare "master" tags for all the "master" images: Would such revised labelling make sense?
V*ersions from master (development use only):* - Main non-CI images (small) *: airflow:master-v2.0.0dev0-python3.5, airflow:**master-v2.0.0dev0-python3.6, airflow:master* ==airflow:master-v2.0.0dev0-python3.5 - CI images (big) *: airflow:master-v2.0.0dev0-ci-python3.5, airflow:**master-v2.0.0dev0-ci-python3.6, airflow:master-ci*==airflow:master-v2.0.0dev0-ci-python3.5 - Production optimised images: (future): *airflow:master-v2.0.0dev0-prod-python3.5, airflow:**master-v2.0.0dev0-prod-python3.6, airflow:master-prod* ==airflow:master-v2.0.0dev0-prod-python3.5 *Release versions (future):* - Main non-CI images (small): *airflow:v1.10.4-python3.5, * *airflow:v1.10.4**-python3.6, airflow:latest*==*airflow:v1.10.4 (should we have them ? I think it might be useful to have a reference image for tests)* - CI images (big): *airflow:v1.10.4-ci-python3.5, **airflow:v1.10.4**-ci-python3.6, airflow:latest-ci*==*airflow:v1.10.4-ci **(should we have them ? I think it might be useful to have a reference image for tests)* - Production optimised images: *airflow:v1.10.4-prod-python3.5, * *airflow:v1.10.4**-prod-python3.6, airflow:latest-prod*== *airflow:v1.10.4-pro* *J.* On Mon, Jun 17, 2019 at 4:56 PM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > Sure. I agree "latest" might be misleading until we work out how we > release it so I am fine with changing to master :). It's super easy - > merely changing tags in DockerHub. > > On Mon, Jun 17, 2019 at 4:25 PM Ash Berlin-Taylor <a...@apache.org> wrote: > >> That page does mention "Nightly" builds which is close to what building >> master would be. The other thing that matters is what we actual call A >> Release. >> >> > Do not include any links on the project website that might encourage >> non-developers to download and use nightly builds, snapshots, release >> candidates, or any other similar package >> >> I think we're find so long as we don't do that -- or in this case, since >> we will probably want to link to the docker hub page once we have versioned >> images there if we make it clear that `:master` is not intended for end >> users, and by the same argument if we have anything as `:latest` it should >> be a docker image relating to an official Release. >> >> Jarek: no `latest` pointing at CI images please. >> >> -a >> >> > On 17 Jun 2019, at 15:04, Philippe Gagnon <philgagn...@gmail.com> >> wrote: >> > >> > One thing: we talked about releasing images under a "master" tag >> (perhaps in another thread?), we should check if this is compatible with >> Apache's release policy [1]. It's not clear to me if this is allowable or >> not after a cursory reading. >> > >> > [1] http://www.apache.org/legal/release-policy.html#what >> > >> > On Mon, Jun 17, 2019 at 9:48 AM Jarek Potiuk <jarek.pot...@polidea.com> >> wrote: >> > Anyone has more comments. I think prevailing opnion is: >> > 1) To keep all images in one repo (apache/airflow) >> > 2) I am not sure about labelling but I might try to document all cases >> in a >> > "production" image proposal that I would like to start as soon as we >> merge >> > the current CI image (which I think is quite close to finalisation). >> > >> > J. >> > >> > On Tue, Jun 11, 2019 at 10:59 PM Jarek Potiuk <jarek.pot...@polidea.com >> > >> > wrote: >> > >> > > It's super easy to do :) >> > > >> > > On Tue, Jun 11, 2019 at 10:33 PM Ash Berlin-Taylor <a...@apache.org> >> wrote: >> > > >> > >> I'm fine with us just publishing release images using the newest >> python >> > >> release (i.e. 3.7) as the main reason we support older python >> versions is >> > >> to support distros thats ship those versions.(i.e. Deb stable), but >> I don't >> > >> think we need to support that in docker. >> > >> >> > >> (But if it's easy to do since we want them for ci then sure) >> > >> >> > >> -ash >> > >> >> > >> On 11 June 2019 21:21:28 BST, Jarek Potiuk <jarek.pot...@polidea.com >> > >> > >> wrote: >> > >>> >> > >>> Yeah Kamil - python 3.5 is the default one for now. I think we >> should have >> > >>> another discussion here - how many versions to support. There is >> this >> > >>> ticket opened today : >> https://issues.apache.org/jira/browse/AIRFLOW-4762 about >> > >>> supporting python 3.6 and 3.7 in tests. Anyone has a strong opinion >> on >> > >>> this? I am for testing on all 3.5, 3.6 and 3.7 even if it increases >> the >> > >>> build/test time on Travis. There are a number of differences >> between those >> > >>> major versions (I have a blog post about it in writing ) but I >> think there >> > >>> is concern about eating Apache Travis time. >> > >>> >> > >>> Anyone against those three ? >> > >>> >> > >>> On Tue, Jun 11, 2019 at 8:38 PM Kamil Breguła < >> kamil.breg...@polidea.com> >> > >>> wrote: >> > >>> >> > >>> 1) I would prefer to use one repository. >> > >>>> +1 >> > >>>> >> > >>>> 2) The presented schema looks logical to me. I had doubts whether >> > >>>> Python 3.5 was a good choice for "latest" version, but I checked >> that >> > >>>> travis uses only this version. >> > >>>> >> > >>>> On Tue, Jun 11, 2019 at 3:04 PM Jarek Potiuk < >> jarek.pot...@polidea.com> >> > >>>> wrote: >> > >>>> >> > >>>>> >> > >>>>> Hello everyone, >> > >>>>> >> > >>>>> We are close to finish AIP-10 (Airlfow image for CI) and seems >> that we >> > >>>>> >> > >>>> will >> > >>>> >> > >>>>> start working soon on an official image AIP, but in the meantime >> we have >> > >>>>> 1.10.4 release coming and we would like to agree tagging scheme >> used for >> > >>>>> the current CI images. We discussed it a bit on Slack, but it's >> time to >> > >>>>> bring it here. I created a JIRA issue for it: >> > >>>>> https://issues.apache.org/jira/browse/AIRFLOW-4764 and my >> proposals >> > >>>>> >> > >>>> after >> > >>>> >> > >>>>> the initial discussion are those: >> > >>>>> >> > >>>>> First of all we have different images that we can talk about : >> > >>>>> >> > >>>>> 1. "base" one - with bare development-ready airflow with >> minimum set >> > >>>>> >> > >>>> of >> > >>>> >> > >>>>> dependencies >> > >>>>> 2. "CI" with all the tools packages that are needed for CI >> tests >> > >>>>> 3. Soon we will likely have an "official" one which might be >> used in >> > >>>>> similar fashion as the "puckel" one. >> > >>>>> >> > >>>>> There are two decisions to make: >> > >>>>> >> > >>>>> 1) How to keep those images - in one repository or whether we >> should have >> > >>>>> separate repos. >> > >>>>> >> > >>>>> It is easier for now to keep all of them within apache/airflow >> > >>>>> < >> https://cloud.docker.com/u/apache/repository/docker/apache/airflow> >> > >>>>> >> > >>>> repository >> > >>>> >> > >>>>> it seems and use a labelling scheme to separate those (there is >> nothing >> > >>>>> wrong with that but it might seem a bit hacky). It's a bit >> easier to >> > >>>>> maintain with access and CI. >> > >>>>> >> > >>>>> We could also think about separate apache/airflow-ci, >> apache/airflow-dev, >> > >>>>> apache/airflow-prod or smth similar - that would require some >> > >>>>> infrastructure tickets and is not very common. >> > >>>>> >> > >>>>> 2) What labelling scheme to use(apache/airflow:label). My >> proposal is >> > >>>>> similar to this (if we keep everything in the airflow repository) >> > >>>>> >> > >>>>> - *latest* = latest released version (python 3.5) = * >> > >>>>> >> > >>>> v1.10.3-python3.5* >> > >>>> >> > >>>>> - *master* = latest master version (python 3.5) = >> > >>>>> >> > >>>> *v2.0.0dev0-python3.5* >> > >>>> >> > >>>>> - *v1.10.3-python3.5,v1.10.3-python3.6* - released 1.10.3 >> with python >> > >>>>> 3.5/3.6 >> > >>>>> - *latest-ci *= latest released version of CI variant (python >> 3.5) >> > >>>>> *v1.10.3-ci-python3.5* >> > >>>>> - *master-ci* = latest master version of CI variant (python >> 3.5) >> > >>>>> *v2.0.0dev0-ci-python3.5* >> > >>>>> - *v1.10.3-ci-python3.5, v1.10.3-ci-python3.6* - released >> 1.10.3 with >> > >>>>> python 3.5/3.6 >> > >>>>> >> > >>>>> >> > >>>>> My preference is to keep all the images in one repo and use >> labelling >> > >>>>> scheme as above, >> > >>>>> but I am open to discuss this. >> > >>>>> >> > >>>>> J, >> > >>>>> >> > >>>>> >> > >>>>> -- >> > >>>>> >> > >>>>> Jarek Potiuk >> > >>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >> > >>>>> >> > >>>>> M: +48 660 796 129 <+48660796129> >> > >>>>> [image: Polidea] <https://www.polidea.com/> >> > >>>>> >> > >>>> >> > >>>> >> > >>> >> > > >> > > -- >> > > >> > > Jarek Potiuk >> > > Polidea <https://www.polidea.com/> | Principal Software Engineer >> > > >> > > M: +48 660 796 129 <+48660796129> >> > > [image: Polidea] <https://www.polidea.com/> >> > > >> > > >> > >> > -- >> > >> > Jarek Potiuk >> > Polidea <https://www.polidea.com/> | Principal Software Engineer >> > >> > M: +48 660 796 129 <+48660796129> >> > [image: Polidea] <https://www.polidea.com/> >> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
