"slim" is common amongst docker, so that sounds good. I think the "primary" docker images should be production focused, and anything else tagged (i.e. it is prod unless it says otherwise.) Since master is not meant for end use we could also _only_ have the CI versions of those images.
*Versions from master (development use only):* - CI images (big) *: airflow:master-ci-python3.5, airflow:**master-ci-python3.6, airflow:master-ci*==airflow:master-ci-python3.6 *Release versions (future):* - Main non-CI images (small): *airflow:1.10.4-python3.5-slim, * *airflow:1.10.4**-python3.6-slim, airflow:latest-slim*==airflow:1.10.4-python3.5-slim - CI images (big): *airflow:1.10.4-ci-python3.5, **airflow:1.10.4**-ci-python3.6, airflow:latest-ci*==airflow:1.10.4-ci-python3.6 - Production optimised images: *airflow:1.10.4-python3.5, * *airflow:1.10.4**-python3.6, airflow:latest* ==airflow:1.10.4-python3.6 > On 18 Jun 2019, at 10:16, James Coder <[email protected]> wrote: > > Just my 2 cents, at work we tend to use “slim” to denote things that are > pared down. > How about “ci-slim”? > > James Coder > >> On Jun 18, 2019, at 4:06 AM, Jarek Potiuk <[email protected]> wrote: >> >> Ok so then next iteration of proposal: The only doubt I have myself is the >> "master" vs. "master-prod". Maybe we should rather have "master" for >> "production-ready" image and use a different name for the "small-ci" >> image". Maybe "trimci" or "ci-lite" or "liteci" ? >> What do you think? >> >> *Versions from master (development use only):* >> >> - Main non-CI images (small) *: airflow:master-python3.5, >> airflow:**master-python3.6, >> airflow:master*==airflow:master-python3.6 >> - CI images (big) *: airflow:master-ci-python3.5, >> airflow:**master-ci-python3.6, >> airflow:master-ci*==airflow:master-ci-python3.6 >> - Production optimised images: (future): *airflow:master-prod-python3.5, >> airflow:**master-prod-python3.6, airflow:master-prod* >> ==airflow:master-prod-python3.6 >> >> *Release versions (future):* >> >> - Main non-CI images (small): *airflow:1.10.4-python3.5, * >> *airflow:1.10.4**-python3.6, airflow:latest*==airflow:1.10.4-python3.5 >> - CI images (big): *airflow:1.10.4-ci-python3.5, >> **airflow:1.10.4**-ci-python3.6, >> airflow:latest-ci*==airflow:1.10.4-ci-python3.6 >> - Production optimised images: *airflow:1.10.4-prod-python3.5, * >> *airflow:1.10.4**-prod-python3.6, airflow:latest-prod* >> ==airflow:1.10.4-prod-python3.6 >> >> >> >> On Mon, Jun 17, 2019 at 10:31 PM Philippe Gagnon <[email protected]> >> wrote: >> >>> That makes sense. The reason I had doubts is because of the way docker hub >>> lists image tags together -- there's no real differentiation between >>> pre-release and release builds. But then I suppose that if the tagging >>> scheme is explicit enough it shouldn't be an issue. >>> >>> +1 on `:latest` being an official release. >>> >>>> On Mon, Jun 17, 2019 at 10:25 AM Ash Berlin-Taylor <[email protected]> wrote: >>>> >>>> That page does mention "Nightly" builds which is close to what building >>>> master would be. The other thing that matters is what we actual call A >>>> Release. >>>> >>>>> Do not include any links on the project website that might encourage >>>> non-developers to download and use nightly builds, snapshots, release >>>> candidates, or any other similar package >>>> >>>> I think we're find so long as we don't do that -- or in this case, since >>>> we will probably want to link to the docker hub page once we have >>> versioned >>>> images there if we make it clear that `:master` is not intended for end >>>> users, and by the same argument if we have anything as `:latest` it >>> should >>>> be a docker image relating to an official Release. >>>> >>>> Jarek: no `latest` pointing at CI images please. >>>> >>>> -a >>>> >>>>> On 17 Jun 2019, at 15:04, Philippe Gagnon <[email protected]> >>> wrote: >>>>> >>>>> One thing: we talked about releasing images under a "master" tag >>>> (perhaps in another thread?), we should check if this is compatible with >>>> Apache's release policy [1]. It's not clear to me if this is allowable or >>>> not after a cursory reading. >>>>> >>>>> [1] http://www.apache.org/legal/release-policy.html#what >>>>> >>>>> On Mon, Jun 17, 2019 at 9:48 AM Jarek Potiuk <[email protected] >>>> >>>> wrote: >>>>> Anyone has more comments. I think prevailing opnion is: >>>>> 1) To keep all images in one repo (apache/airflow) >>>>> 2) I am not sure about labelling but I might try to document all cases >>>> in a >>>>> "production" image proposal that I would like to start as soon as we >>>> merge >>>>> the current CI image (which I think is quite close to finalisation). >>>>> >>>>> J. >>>>> >>>>> On Tue, Jun 11, 2019 at 10:59 PM Jarek Potiuk < >>> [email protected]> >>>>> wrote: >>>>> >>>>>> It's super easy to do :) >>>>>> >>>>>> On Tue, Jun 11, 2019 at 10:33 PM Ash Berlin-Taylor <[email protected]> >>>> wrote: >>>>>> >>>>>>> I'm fine with us just publishing release images using the newest >>>> python >>>>>>> release (i.e. 3.7) as the main reason we support older python >>>> versions is >>>>>>> to support distros thats ship those versions.(i.e. Deb stable), but >>> I >>>> don't >>>>>>> think we need to support that in docker. >>>>>>> >>>>>>> (But if it's easy to do since we want them for ci then sure) >>>>>>> >>>>>>> -ash >>>>>>> >>>>>>> On 11 June 2019 21:21:28 BST, Jarek Potiuk < >>> [email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> Yeah Kamil - python 3.5 is the default one for now. I think we >>>> should have >>>>>>>> another discussion here - how many versions to support. There is >>> this >>>>>>>> ticket opened today : >>>> https://issues.apache.org/jira/browse/AIRFLOW-4762 about >>>>>>>> supporting python 3.6 and 3.7 in tests. Anyone has a strong opinion >>>> on >>>>>>>> this? I am for testing on all 3.5, 3.6 and 3.7 even if it increases >>>> the >>>>>>>> build/test time on Travis. There are a number of differences >>> between >>>> those >>>>>>>> major versions (I have a blog post about it in writing ) but I >>> think >>>> there >>>>>>>> is concern about eating Apache Travis time. >>>>>>>> >>>>>>>> Anyone against those three ? >>>>>>>> >>>>>>>> On Tue, Jun 11, 2019 at 8:38 PM Kamil Breguła < >>>> [email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> 1) I would prefer to use one repository. >>>>>>>>> +1 >>>>>>>>> >>>>>>>>> 2) The presented schema looks logical to me. I had doubts whether >>>>>>>>> Python 3.5 was a good choice for "latest" version, but I checked >>>> that >>>>>>>>> travis uses only this version. >>>>>>>>> >>>>>>>>> On Tue, Jun 11, 2019 at 3:04 PM Jarek Potiuk < >>>> [email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hello everyone, >>>>>>>>>> >>>>>>>>>> We are close to finish AIP-10 (Airlfow image for CI) and seems >>>> that we >>>>>>>>>> >>>>>>>>> will >>>>>>>>> >>>>>>>>>> start working soon on an official image AIP, but in the meantime >>>> we have >>>>>>>>>> 1.10.4 release coming and we would like to agree tagging scheme >>>> used for >>>>>>>>>> the current CI images. We discussed it a bit on Slack, but it's >>>> time to >>>>>>>>>> bring it here. I created a JIRA issue for it: >>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-4764 and my >>>> proposals >>>>>>>>>> >>>>>>>>> after >>>>>>>>> >>>>>>>>>> the initial discussion are those: >>>>>>>>>> >>>>>>>>>> First of all we have different images that we can talk about : >>>>>>>>>> >>>>>>>>>> 1. "base" one - with bare development-ready airflow with >>>> minimum set >>>>>>>>>> >>>>>>>>> of >>>>>>>>> >>>>>>>>>> dependencies >>>>>>>>>> 2. "CI" with all the tools packages that are needed for CI >>>> tests >>>>>>>>>> 3. Soon we will likely have an "official" one which might be >>>> used in >>>>>>>>>> similar fashion as the "puckel" one. >>>>>>>>>> >>>>>>>>>> There are two decisions to make: >>>>>>>>>> >>>>>>>>>> 1) How to keep those images - in one repository or whether we >>>> should have >>>>>>>>>> separate repos. >>>>>>>>>> >>>>>>>>>> It is easier for now to keep all of them within apache/airflow >>>>>>>>>> < >>>> https://cloud.docker.com/u/apache/repository/docker/apache/airflow> >>>>>>>>>> >>>>>>>>> repository >>>>>>>>> >>>>>>>>>> it seems and use a labelling scheme to separate those (there is >>>> nothing >>>>>>>>>> wrong with that but it might seem a bit hacky). It's a bit >>> easier >>>> to >>>>>>>>>> maintain with access and CI. >>>>>>>>>> >>>>>>>>>> We could also think about separate apache/airflow-ci, >>>> apache/airflow-dev, >>>>>>>>>> apache/airflow-prod or smth similar - that would require some >>>>>>>>>> infrastructure tickets and is not very common. >>>>>>>>>> >>>>>>>>>> 2) What labelling scheme to use(apache/airflow:label). My >>>> proposal is >>>>>>>>>> similar to this (if we keep everything in the airflow >>> repository) >>>>>>>>>> >>>>>>>>>> - *latest* = latest released version (python 3.5) = * >>>>>>>>>> >>>>>>>>> v1.10.3-python3.5* >>>>>>>>> >>>>>>>>>> - *master* = latest master version (python 3.5) = >>>>>>>>>> >>>>>>>>> *v2.0.0dev0-python3.5* >>>>>>>>> >>>>>>>>>> - *v1.10.3-python3.5,v1.10.3-python3.6* - released 1.10.3 >>>> with python >>>>>>>>>> 3.5/3.6 >>>>>>>>>> - *latest-ci *= latest released version of CI variant (python >>>> 3.5) >>>>>>>>>> *v1.10.3-ci-python3.5* >>>>>>>>>> - *master-ci* = latest master version of CI variant (python >>>> 3.5) >>>>>>>>>> *v2.0.0dev0-ci-python3.5* >>>>>>>>>> - *v1.10.3-ci-python3.5, v1.10.3-ci-python3.6* - released >>>> 1.10.3 with >>>>>>>>>> python 3.5/3.6 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> My preference is to keep all the images in one repo and use >>>> labelling >>>>>>>>>> scheme as above, >>>>>>>>>> but I am open to discuss this. >>>>>>>>>> >>>>>>>>>> J, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Jarek Potiuk >>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software >>> Engineer >>>>>>>>>> >>>>>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Jarek Potiuk >>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>>>>> >>>>>> M: +48 660 796 129 <+48660796129> >>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> Jarek Potiuk >>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>>>> >>>>> M: +48 660 796 129 <+48660796129> >>>>> [image: Polidea] <https://www.polidea.com/> >>>> >>>> >>> >> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/>
