Re: Travis builds in a queue for hours

Hello everyone, Great news! I've been able to provision some Google Cloud credits to start trying out a CI solution (~3k USD). They should last for a couple months (let's optimize it so it lasts as long as possible :)) while Gris and I work on a longer-term solution to provision the credits. The

Re: Travis builds in a queue for hours

Just FYI - I opened a ticket to get stats of the machine usage for Travis to infra: https://issues.apache.org/jira/browse/INFRA-18742 On Thu, Jul 11, 2019 at 1:48 PM Jarek Potiuk wrote: > Absolutely! I thought about it today and GKE cluster would be perfect for > us - especially that we can also

Re: Travis builds in a queue for hours

Absolutely! I thought about it today and GKE cluster would be perfect for us - especially that we can also use it to run Kubernetes tests on it ! That's still a major pain having to setup minikube for the tests and having a GKE cluster that we can simply use would simplify this part a LOT. Princip

Re: Travis builds in a queue for hours

Yes, Gitlab works very well with GCP. A Kubernetes cluster with autoscaling for the runners would be perfect, and will also minimize the resources provided by Google. Cheers, Fokko Op do 11 jul. 2019 om 07:13 schreef Jarek Potiuk > Since more than few people (including myself) are in favour of

Re: Travis builds in a queue for hours

Since more than few people (including myself) are in favour of GitLab CI, and since Apache Infra is talking to GitLab CI, I will make sure to check if we can combine the two approaches - workers from Google and managed, central GitlabCI interface to manage it (likely managed by the Infra team). Air

Re: Travis builds in a queue for hours

Regardings the numbers, I believe that INFRA has an overview of the usage per project. I think they are happy to share these numbers with you. Also, it seems like there is also a queue in Jenkins: https://status.apache.org/ Talking about Jenkins. I'm not a big fan of it. For example, Spark uses it

Re: Travis builds in a queue for hours

Hi all, I am still working on trying to get approvals for this, so this is not yet a done deal. I'll keep y'all updated. As for the CI solution to use, we have no particular inclination. As long as the community supports it, and it is consistent with any Apache guidelines for CI from their projec

Re: Travis builds in a queue for hours

Yeah. Gitlab CI is definitely what I would prefer as well from the "modernity" point of view (and one of my very close friends is Gitlab CI maintainer and actually The person who introduced CI to GitLab offering). I also actually already catalysed discussion between GitLab and Apache infrastructure

Re: Travis builds in a queue for hours

It does not need to be Jenkins (we will hopefully get VMs that we can put whatever on) - but it seems that (If it works out of course which is something we need to finalise) from "process" point of view following the footsteps of Apache Beam and using the same setup might be one of the best choices

Re: Travis builds in a queue for hours

If you need an alternative why not use a couple of gitlab-ci runners? Much easier to maintain, light weight, and much closer to what we use now. B. Verstuurd vanaf mijn iPad > Op 10 jul. 2019 om 23:27 heeft Bolke de Bruin het > volgende geschreven: > > Awesome! But I hope you are not serious

Re: Travis builds in a queue for hours

Awesome! But I hope you are not serious about using Jenkins right? If I need to start a Holy War it would be against Jenkins. B. Verstuurd vanaf mijn iPad > Op 10 jul. 2019 om 22:55 heeft Jarek Potiuk het > volgende geschreven: > > Hello Everyone, > > I have some really good news. I just ha

Re: Travis builds in a queue for hours

Hello Everyone, I have some really good news. I just had a call with Google OSS team (Gris, Aizhamal) and they are willing to donate VMs on Google Cloud Platform to run CI for Airflow. In order to simplify the setup (and make sure it is ok according to Apache regulations) we think we should go exa

Re: Travis builds in a queue for hours

Thanks Aizhamal! I spoke already to Gris and she confirmed that as well and the 8th of July date is ok for us as we will have to evaluate and prepare as well. Have a nice trip. J. On Fri, Jun 28, 2019 at 4:25 PM Aizhamal Nurmamat kyzy wrote: > Hi all, > > On Thu, Jun 27, 2019 at 15:28 Jarek Pot

Re: Travis builds in a queue for hours

Hi all, On Thu, Jun 27, 2019 at 15:28 Jarek Potiuk wrote: > Yeah. I also have a working version of Cloud build configuration and we can > run the tests on cloud build if we can get some credits from Google. I can look into getting a small amount of credits approved for this, to see if it’s use

Re: Travis builds in a queue for hours

I had numerous issues related to particular backends. If you go deeper, i even had Postgres-related issue which only occurred on python3.6 (python 3.5 worked fine. It was related to parsing DB URL :) ). For sure testing all matrix combination is an overkill, but we need to test all backends. I also

Re: Travis builds in a queue for hours

I concur. There are subtle differences in SQLAlchemy behavior depending on the backend... In that context testing against multiple DBMS makes sense and adds value. @Conrad: I do see your point though, testing libraries should not be Airflow's responsibility but from a pragmatic point of view we sh

Re: Travis builds in a queue for hours

Yes Conrad, this is actually true. However I think it is important to test different backends, as an example, we had a lot of issues with MySQL's timezone behavior, besides that, it would be possible to test specific database evolution steps for certain backends. Cheers, Fokko Op vr 28 jun. 2019

Re: Travis builds in a queue for hours

Regarding the matrix of backends to test -- is this testing specific platform integrations, or if everything is routed through SQLAlchemy's ORM does this test suite just end up testing SQLAlchemy? It makes sense to have the ability to swap backends out for the sake of reproducing issues when somet

Re: Travis builds in a queue for hours

Yeah. I also have a working version of Cloud build configuration and we can run the tests on cloud build if we can get some credits from Google. And the changes from the upcoming CI image will make it much easier to run tests on any CI provider. Except Kubernetes tests they are pretty much CI-agnos

Re: Travis builds in a queue for hours

I think the combinations that you are proposing are sensible for pre-merge checks. I am working on a proposal to offload extra combinations to another CI provider (Azure DevOps specifically seems like a good candidate), either pre or post merge. Ideally I'd like to run more combinations pre-merge

Re: Travis builds in a queue for hours

Agree that we should be thoughtful about others as well: In the latest push (few minutes ago) of the upcoming official CI image i implemented the change we discussed in the Github where we limit the number of combinations we test: You can see it yourself: https://travis-ci.org/apache/airflow/build

Re: Travis builds in a queue for hours

We got this message last year: > Hello, Airflow PPMC. > While going through the usage statistics for our Travis CI service, I > have noticed that the Airflow project is using an abnormally large > amount of resources, 2600 hours per month or the equivalent of having > almost 4 machines building ai

Re: Travis builds in a queue for hours

I think we should really involve infra to increase the slot number or maybe even somehow allocate slots per project. The problem is that we cannot control what other apache projects are doing, so even if we decrease our runtime, it's the other projects that might hold us in the queue :( J. On Thu

Re: Travis builds in a queue for hours

I've noticed this at other Apache projects as well, sometimes it takes up to 7-8 hours. The only thing we can do, is reduce the runtime of the jobs so we take less slots :-) Cheers, Fokko Op wo 26 jun. 2019 om 21:59 schreef Jarek Potiuk : > Yep. That's what I suggested as the reason in the ticke

Re: Travis builds in a queue for hours

Yep. That's what I suggested as the reason in the ticket - I guess INFRA are the only people who can do anything about it (increase concurrency ? pay more for Travis :)? ). On Wed, Jun 26, 2019 at 9:51 PM Ash Berlin-Taylor wrote: > I asked Travis on twitter and they said it was due to the Apache

Re: Travis builds in a queue for hours

I asked Travis on twitter and they said it was due to the Apache other projects build queues https://twitter.com/travisci/status/1143893051460526080 -ash On 26 June 2019 20:48:33 BST, Jarek Potiuk wrote: >Hello everyone, > >For the last few days the Travis builds for apache/airflow project are

Travis builds in a queue for hours

Hello everyone, For the last few days the Travis builds for apache/airflow project are waiting in a queue for hours. This is not a normal situation. I've opened INFRA ticket for that: https://issues.apache.org/jira/browse/INFRA-18657 J. -- Jarek Potiuk Polidea | Prin