I’d rather we do it all at once and not need to maintain two builds. Most/all of our CI/CD is dockerized at this point so there shouldn’t be a huge difference.
On Fri, Dec 13, 2019 at 1:23 AM, Tomasz Urbaszek <tomasz.urbas...@polidea.com> wrote: Hi all, I have to solve a problem with our kubernetes test but for your information I have never experienced a flaky or timeouted build on Github Actions. Maybe I am lucky or maybe there's something different. If we agree to move to Github Actions, would we like to migrate incrementally or fully? Bests, Tomek On Wed, Dec 11, 2019 at 1:06 AM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > No problem on that side - Tomek is using the same scripts we have on Travis > (and they generally work - I think the last step is to make all the > tests pass. We have still a number of dependencies between tests and some > environmental flakiness so that some tests consistently fail in Github > Actions where they did not fail in Travis. From latest try by Tomek it > looks like we are 1 test to go (plus some cleanup/setup of project and > making sure all is stable): > > ==== 1 failed, 4030 passed, 119 skipped, 16 warnings in 1207.96s (0:20:07) > ===== > > We discussed with Tomek and Kamil that in order to speed it up we might > want to run our own workers on the GCP account we have - just to test > quickly how much we can optimise it and I am inclined to agree. If we do it > this way, the transition might be rather fast. > > If we want to use auto-scalable GKE cluster as we originally planned it > might take more time to setup (similar setup to what I tried with GitLab). > There we might need to use docker-in-docker to build the CI image with > latest as first step of build and then use that built image by all > subsequent steps. But we can do it as the next step - optimising the > experience. > > J. > > On Tue, Dec 10, 2019 at 11:34 PM Daniel Imberman < > daniel.imber...@gmail.com> > wrote: > > > +1 on my end as well. > > > > @jarek does this affect anything involving breeze? Do GitHub actions have > > an easy docker/docker-compose based workflow? > > > > via Newton Mail [ > > > https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.32&pv=10.14.5&source=email_footer_2 > > ] > > On Tue, Dec 10, 2019 at 5:28 AM, Ash Berlin-Taylor <a...@apache.org> > wrote: > > Legal: no I don't think so. > > > > Infra: possibly yes to get secrets in there to run things on our own > > "hardware" - > > > https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-and-using-encrypted-secrets > > < > > > https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-and-using-encrypted-secrets > > > > needs someone with Admin rights to create, and I don't see the Settings > tab > > at all. > > > > -ash > > > > > On 10 Dec 2019, at 02:46, Deng Xiaodong <xd.den...@gmail.com> wrote: > > > > > > +1 for GitHub Actions. I have been using it for months for my side > > > projects, and it’s working very well. I believe most of us are quite > > tired > > > of the waiting time using Travis CI. > > > > > > The only point I would like to remind is whether we need to communicate > > > with Infra/Legal team for this. > > > > > > > > > XD > > > > > > On Tue, Dec 10, 2019 at 06:49 Kaxil Naik <kaxiln...@gmail.com> wrote: > > > > > >> +1 for Github actions > > >> > > >> On Mon, Dec 9, 2019, 22:16 Ash Berlin-Taylor <a...@apache.org> wrote: > > >> > > >>> Happy with any thing that gives a more seamless CI experience - > faster > > is > > >>> good too! > > >>> > > >>> -a > > >>> > > >>> On 9 December 2019 22:12:05 GMT, Aizhamal Nurmamat kyzy < > > >>> aizha...@apache.org> wrote: > > >>>> +1 on GitHub Actions. > > >>>> > > >>>> On Mon, Dec 9, 2019 at 2:10 PM Jarek Potiuk < > jarek.pot...@polidea.com > > > > > >>>> wrote: > > >>>> > > >>>>> I am all for it! GitLab has been less-than helpful so far and > > >>>> recently it > > >>>>> seems that running PRs from forks will only be run in Enterrprise > > >>>> Edition, > > >>>>> which is less than welcome. I am quite a bit disappointed with the > > >>>> pace and > > >>>>> attitude. Github Actions seems to be much better choice - > especially > > >>>> that > > >>>>> they are closely integrated with Github repo and seem to get > > >>>>> attention/focus from Github/Microsoft. > > >>>>> > > >>>>> And they added self-hosted runners as well, which makes it possible > > >>>> for us > > >>>>> to optimise the experience. > > >>>>> > > >>>>> J. > > >>>>> > > >>>>> > > >>>>> J. > > >>>>> > > >>>>> On Mon, Dec 9, 2019 at 10:57 PM Tomasz Urbaszek < > > >>>>> tomasz.urbas...@polidea.com> > > >>>>> wrote: > > >>>>> > > >>>>>> Hi all, > > >>>>>> > > >>>>>> It sometime since we last discussed using other CI than Travis. > One > > >>>> of > > >>>>> the > > >>>>>> main reasons behind considering Gitlab CI was its ability to work > > >>>> on > > >>>>>> self-hosted runner. However, over time of few long weeks Github > > >>>> Actions > > >>>>>> matured enough to allow using self-hosted runners! > > >>>>>> > > >>>>>> Github Actions are still growing but using them have few big > > >>>> advantages: > > >>>>>> - they are Github natives > > >>>>>> - forking repo and enabling actions will run CI on your fork > > >>>>> automatically > > >>>>>> - variety of actions (PR checks, greetings, etc) > > >>>>>> > > >>>>>> I put together a PoC of CI in our internal repo: > > >>>>>> https://github.com/PolideaInternal/airflow/pull/542 > > >>>>>> My impression is quite good. I like information about steps > > >>>> successes at > > >>>>>> the PR level (no need to go to CI to check which step failed). The > > >>>> build > > >>>>>> log view is a little bit clumsy but it works. > > >>>>>> > > >>>>>> Does any of you have any experience with Github Actions? Any > > >>>> thoughts > > >>>>>> about using it? > > >>>>>> > > >>>>>> Best, > > >>>>>> Tomek > > >>>>>> > > >>>>>> On 2019/08/09 13:55:11, Jarek Potiuk <jarek.pot...@polidea.com> > > >>>> wrote: > > >>>>>>> FYI: Interesting article about the history behind GitLabCI > > >>>> (featuring > > >>>>>>> Kamil, my friend). > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > https://about.gitlab.com/2019/08/08/built-in-ci-cd-version-control-secret/?fbclid=IwAR2tEfqLaDXTCd1mD6XUZMX7hGYBfZcohPtI2BP3-oK_Yk_EHIXF4zLDixk > > >>>>>>> > > >>>>>>> On Mon, Aug 5, 2019 at 7:14 PM Jarek Potiuk > > >>>> <jarek.pot...@polidea.com> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Some update on my GitLab experiences so far: > > >>>>>>>> > > >>>>>>>> TL;DR; I think the POC has shown that we can fairly easily > > >>>> replicate > > >>>>>> the > > >>>>>>>> CI in GitLab + Kubernetes. I think i can say - it generally > > >>>> works, I > > >>>>>> can > > >>>>>>>> plug it in for master/v1-10-test builds in the main Airflow > > >>>> project > > >>>>>> for a > > >>>>>>>> few weeks to see how it is doing (while I am no holidays) and > > >>>> once we > > >>>>>> see > > >>>>>>>> it running and get the support for PRs from GitLab we can > > >>>> switch to > > >>>>> it. > > >>>>>>>> > > >>>>>>>> What do you think ? Should i call a vote or just try to set it > > >>>> up ? > > >>>>>>>> > > >>>>>>>> Some details > > >>>>>>>> > > >>>>>>>> - I manged to get full working builds in GitLabCI + > > >>>> kubernetes - > > >>>>>>>> without the kubernetes-specific tests yet, but this should > > >>>> be > > >>>>>> rather easy > > >>>>>>>> with kind (looking at it next): > > >>>>>>>> - Working example here - you can take a look and compare the > > >>>>> UI/how > > >>>>>> it > > >>>>>>>> is to navigate, comparing to Travis etc: > > >>>>>>>> https://gitlab.com/Jarek.Potiuk/airflow/pipelines/74625817 > > >>>>>>>> - Per-job it is a bit slower than Travis so far (still > > >>>> around 35 > > >>>>>>>> minutes in total), but I plan to optimise it further. I can > > >>>> play > > >>>>>> with > > >>>>>>>> memory/cpu settings of individual workers (Got some > > >>>> reasonable > > >>>>>> values now), > > >>>>>>>> I can use local SSD disk as Docker storage/logs/etc > > >>>>>>>> - I got an approval for 72vCPU quota (up for initial 24) - > > >>>> that > > >>>>>> should > > >>>>>>>> let us build 3 builds in parallel independently from each > > >>>> other. > > >>>>>>>> - I managed to get Preemptible nodes working (we have built > > >>>> in > > >>>>> retry > > >>>>>>>> mechanism in GitLab to work in case of system failures like > > >>>> that > > >>>>>>>> - Current spending with > 120 builds is 40 USD. We should be > > >>>> way > > >>>>>> below > > >>>>>>>> 500 USD/month according to my back-of-the-envelope > > >>>> calculations. > > >>>>>> Likely > > >>>>>>>> well below > > >>>>>>>> - The current setup does not use GCR as cache and Kaniko as > > >>>> I > > >>>>>>>> originally planned. GCR would require custom authentication > > >>>> (and > > >>>>>>>> easy-to-steal secrets) and Kaniko does not yet well handle > > >>>>>> multi-staging > > >>>>>>>> builds (cache does not work > > >>>>>>>> https://github.com/GoogleContainerTools/kaniko/issues/682). > > >>>> I > > >>>>>> updated > > >>>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-23+Migrate+out+of+Travis+CI > > >>>>>> to > > >>>>>>>> reflect that. > > >>>>>>>> - We only use GCR as mirroring of DockerHub - so that we can > > >>>> have > > >>>>>>>> reliable downloads not depending on DockerHub's stability > > >>>> (it has > > >>>>>> problems > > >>>>>>>> sometimes) > > >>>>>>>> - All in-all, it's GCP-independent. It could be run in any > > >>>>>> Kubernetes > > >>>>>>>> cluster (some optimisations like local volumes mounting for > > >>>> docker > > >>>>>> engine > > >>>>>>>> might have GCP-specific assumptions, but should be generally > > >>>>>> replicable). > > >>>>>>>> - You can take a look at the current source code in > > >>>>>>>> https://github.com/potiuk/airflow/commits/test-gitlab-ci > > >>>>>>>> - There will be some updates (I will get rid of custom > > >>>> builder > > >>>>>> Docker, > > >>>>>>>> simplify it a bit and implement kubernetes tests) - it's > > >>>> mostly > > >>>>> some > > >>>>>>>> cleanups + removal of Travis-Specific variables + gitlab.ci > > >>>> yaml > > >>>>>> with > > >>>>>>>> job definitions. > > >>>>>>>> > > >>>>>>>> J. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Wed, Jul 31, 2019 at 10:57 AM Jarek Potiuk < > > >>>>>> jarek.pot...@polidea.com> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> So GitLab already works on automatically running builds from > > >>>> for PRs > > >>>>>> :). > > >>>>>>>>> > > >>>>>>>>> Kamil got involved and will be out advocate on it: > > >>>>>>>>> https://gitlab.com/gitlab-org/gitlab-ce/issues/65139 > > >>>>>>>>> J. > > >>>>>>>>> > > >>>>>>>>> Principal Software Engineer > > >>>>>>>>> Phone: +48660796129 <+48%20660%20796%20129> > > >>>>>>>>> > > >>>>>>>>> pt., 26 lip 2019, 18:12 użytkownik Jarek Potiuk < > > >>>>>> jarek.pot...@polidea.com> > > >>>>>>>>> napisał: > > >>>>>>>>> > > >>>>>>>>>> Update: I added appropriate comment in the GitLab CI issue > > >>>> about > > >>>>> PRs > > >>>>>> and > > >>>>>>>>>> we are getting attention of Jason Lenny - director of Product > > >>>>>> Management @ > > >>>>>>>>>> GitLab. Let's hope they prioritise it quickly enough. > > >>>>>>>>>> > > >>>>>>>>>> Speaking of potential complexity/Maintenance - in order to > > >>>>> alleviate > > >>>>>> any > > >>>>>>>>>> maintenance worries, I think about setting up the whole > > >>>> system on > > >>>>>> GitLab > > >>>>>>>>>> CI + GKE and running it in parallel to Travis for quite some > > >>>> time > > >>>>>> (even > > >>>>>>>>>> months) so that we can switch it at any time. Then we will be > > >>>> able > > >>>>>> to tune > > >>>>>>>>>> it according to real use cases and compare the experience of > > >>>> both > > >>>>>> systems. > > >>>>>>>>>> > > >>>>>>>>>> Also I am going for holidays in two weeks and I will make > > >>>> sure that > > >>>>>>>>>> there will be someone with GitLab + Kubernetes experience > > >>>> (from my > > >>>>>> company) > > >>>>>>>>>> who can take over and make sure there will be no problems. > > >>>> However > > >>>>> I > > >>>>>> am > > >>>>>>>>>> quite confident :D nothing is going to happen while I am > > >>>> away. I > > >>>>>> would also > > >>>>>>>>>> invite whoever from committers who would like to join the > > >>>> project > > >>>>> and > > >>>>>>>>>> gitlab instance (once I setup POC) to learn and see how easy > > >>>> it is > > >>>>>> and how > > >>>>>>>>>> maintenance free it is going to be. > > >>>>>>>>>> > > >>>>>>>>>> J. > > >>>>>>>>>> > > >>>>>>>>>> On Fri, Jul 26, 2019 at 2:56 PM Kamil Breguła < > > >>>>>> kamil.breg...@polidea.com> > > >>>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> GKE and its own CI will allow us to solve other problems - > > >>>>> building > > >>>>>>>>>>> and publishing documentation from the master branch. > > >>>> Currently, > > >>>>>>>>>>> building is done using the RTD service. Unfortunately, our > > >>>> project > > >>>>>> is > > >>>>>>>>>>> too large and often the documentation is not built properly. > > >>>>>>>>>>> https://readthedocs.org/projects/airflow/builds/ > > >>>>>>>>>>> We should think about another way to build documentation. In > > >>>> the > > >>>>>> ideal > > >>>>>>>>>>> world, building documentation should use the same > > >>>> environment as > > >>>>>>>>>>> checking documentation on CI. Adding this step to Travis can > > >>>>> further > > >>>>>>>>>>> reduce our development opportunities. > > >>>>>>>>>>> Discussion on Slack about it: > > >>>>>>>>>>> > > >>>>>> > > >>>> > https://apache-airflow.slack.com/archives/CJ1LVREHX/p1561756652021900 > > >>>>>>>>>>> > > >>>>>>>>>>> It is worth thinking about the fact that our project will > > >>>> soon > > >>>>> have > > >>>>>> a > > >>>>>>>>>>> website and our documentation will also be available in many > > >>>>>>>>>>> languages. Currently, talks are taking place with the design > > >>>>> studio > > >>>>>>>>>>> and developers who can make these websites ;-) > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > https://lists.apache.org/thread.html/982c7baa06742ad722f2baa0db53ad99aea6c26b14b7d6d4aa522677@%3Cdev.airflow.apache.org%3E > > >>>>>>>>>>> We should provide an environment that will allow you to > > >>>> build a > > >>>>>>>>>>> website and documentation. At best, these tasks should be > > >>>>> combined. > > >>>>>> I > > >>>>>>>>>>> hope that we will be able to create a website that will be a > > >>>> real > > >>>>>>>>>>> support for the community on current events, so it will be > > >>>> updated > > >>>>>>>>>>> frequently. > > >>>>>>>>>>> > > >>>>>>>>>>> It seems to me that the project will grow. If we now have > > >>>> problems > > >>>>>>>>>>> with Travis, then the significance of these problems in the > > >>>> future > > >>>>>> can > > >>>>>>>>>>> only grow. Now we have a chance to provide a stable > > >>>> infrastructure > > >>>>>> for > > >>>>>>>>>>> the project for a long time. > > >>>>>>>>>>> > > >>>>>>>>>>> I would like to share another situation which was not > > >>>> pleasant for > > >>>>>> me. > > >>>>>>>>>>> Recently I wanted to send >10 PR, but because of Travis, I > > >>>> had to > > >>>>>> wait > > >>>>>>>>>>> for the weekend to send changes. If I would send my changes > > >>>> in a > > >>>>>> week, > > >>>>>>>>>>> I would block the queue for a few hours. Although I did it > > >>>> over > > >>>>> the > > >>>>>>>>>>> weekend, I got the message that the queue is blocked on > > >>>> Travis by > > >>>>> my > > >>>>>>>>>>> jobs. > > >>>>>>>>>>> > > >>>>>>>>>>> On Tue, Jul 23, 2019 at 6:12 PM Jarek Potiuk < > > >>>>>> jarek.pot...@polidea.com> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>> Hello Everyone, > > >>>>>>>>>>>> > > >>>>>>>>>>>> I prepared a short docs where I described general > > >>>> architecture > > >>>>> of > > >>>>>> the > > >>>>>>>>>>>> solution I imagine we can deploy fairly quickly - having > > >>>> GitLab > > >>>>> CI > > >>>>>>>>>>> support > > >>>>>>>>>>>> and Google provided funding for GCP resources. > > >>>>>>>>>>>> > > >>>>>>>>>>>> I am going to start working on Proof-Of-Concept soon but > > >>>> before > > >>>>> I > > >>>>>>>>>>> start > > >>>>>>>>>>>> doing it, I would like to get some comments and opinions > > >>>> on the > > >>>>>>>>>>> proposed > > >>>>>>>>>>>> approach. I discussed the basic approach with my friend > > >>>> Kamil > > >>>>> who > > >>>>>>>>>>> works at > > >>>>>>>>>>>> GitLab and he is a CI maintainer and this is what we think > > >>>> will > > >>>>> be > > >>>>>>>>>>>> achievable in fairly short time. > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-23+Migrate+out+of+Travis+CI > > >>>>>>>>>>>> > > >>>>>>>>>>>> I am happy to discuss details and make changes to the > > >>>> proposal - > > >>>>>> we > > >>>>>>>>>>> can > > >>>>>>>>>>>> discuss it here or as comments in the document. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Let's see what people think about it and if we get to some > > >>>>>> consensus > > >>>>>>>>>>> we > > >>>>>>>>>>>> might want to cast a vote (or maybe go via lasy consensus > > >>>> as > > >>>>> this > > >>>>>> is > > >>>>>>>>>>>> something we should have rather quickly) > > >>>>>>>>>>>> > > >>>>>>>>>>>> Looking forward to your comments! > > >>>>>>>>>>>> > > >>>>>>>>>>>> J. > > >>>>>>>>>>>> > > >>>>>>>>>>>> -- > > >>>>>>>>>>>> > > >>>>>>>>>>>> Jarek Potiuk > > >>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software > > >>>>> Engineer > > >>>>>>>>>>>> > > >>>>>>>>>>>> M: +48 660 796 129 <+48%20660%20796%20129> <+48660796129 > > >>>>> <+48%20660%20796%20129>> > > >>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> -- > > >>>>>>>>>> > > >>>>>>>>>> Jarek Potiuk > > >>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software > > >>>> Engineer > > >>>>>>>>>> > > >>>>>>>>>> M: +48 660 796 129 <+48%20660%20796%20129> <+48660796129 > > >>>>> <+48%20660%20796%20129>> > > >>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> > > >>>>>>>> Jarek Potiuk > > >>>>>>>> Polidea <https://www.polidea.com/> | Principal Software > > >>>> Engineer > > >>>>>>>> > > >>>>>>>> M: +48 660 796 129 <+48%20660%20796%20129> <+48660796129 > > >>>>> <+48%20660%20796%20129>> > > >>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> > > >>>>>>> Jarek Potiuk > > >>>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer > > >>>>>>> > > >>>>>>> M: +48 660 796 129 <+48%20660%20796%20129> <+48660796129 > > >>>>> <+48%20660%20796%20129>> > > >>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> > > >>>>> Jarek Potiuk > > >>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer > > >>>>> > > >>>>> M: +48 660 796 129 <+48%20660%20796%20129> <+48660796129 > > >>>>> <+48%20660%20796%20129>> > > >>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>> > > >>> > > >> > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > -- Tomasz Urbaszek Polidea <https://www.polidea.com/> | Junior Software Engineer M: +48 505 628 493 <+48505628493> E: tomasz.urbas...@polidea.com <tomasz.urbasz...@polidea.com> Unique Tech Check out our projects! <https://www.polidea.com/our-work>
