Agree that we should be thoughtful about others as well: In the latest push (few minutes ago) of the upcoming official CI image i implemented the change we discussed in the Github where we limit the number of combinations we test:
You can see it yourself: https://travis-ci.org/apache/airflow/builds/551305240 Those are the combinations I propose: Python: 3.6 BACKEND=mysql ENV=docker Python: 3.6 BACKEND=postgres ENV=docker Python: 3.5 BACKEND=sqlite ENV=docker Python: 3.6 BACKEND=postgres ENV=kubernetes KUBERNETES_VERSION=v1.13.0 J, On Thu, Jun 27, 2019 at 11:00 AM Driesprong, Fokko <fo...@driesprong.frl> wrote: > We got this message last year: > > > Hello, Airflow PPMC. > > While going through the usage statistics for our Travis CI service, I > > have noticed that the Airflow project is using an abnormally large > > amount of resources, 2600 hours per month or the equivalent of having > > almost 4 machines building airflow non-stop 24/7. As this is not free, > > but rather costing us money, I'm contacting you with the intention of > > figuring out ways to reduce the use of Travis for the project. > > > We would greatly prefer that the project itself comes up with a solution > > to lower the usage of Travis, as we'd hate to simply turn it off for > > you, but the usage is at a rather severe level, totaling more than 21% > > of the total build time of all projects using Travis, so something > > actionable should be decided upon and (preferably) completed by the end > > of May that will reduce the consumption of Travis resources. > > > Alternately, if you are unable to lower the pressure on Travis, the > > podling and/or IPMC may ask the board of directors for a separate budget > > for additional build nodes to cope with the added load - I'll leave this > > for the podling and IPMC to decide on. > > > Please let us know when you have decided on a plan to remedy this > situation. > > > With regards, > > Daniel on behalf of ASF Infrastructure. > > I think more and more projects are still migrating to the ASF Travis, so I > think natural that there is more load. However, this still leaves the > question if we have to run the full matrix. > > Cheers, Fokko > > > > Op do 27 jun. 2019 om 10:56 schreef Jarek Potiuk <jarek.pot...@polidea.com > >: > > > I think we should really involve infra to increase the slot number or > maybe > > even somehow allocate slots per project. > > The problem is that we cannot control what other apache projects are > doing, > > so even if we decrease our runtime, it's the other projects that might > hold > > us in the queue :( > > > > J. > > > > On Thu, Jun 27, 2019 at 10:19 AM Driesprong, Fokko <fo...@driesprong.frl > > > > wrote: > > > > > I've noticed this at other Apache projects as well, sometimes it takes > up > > > to 7-8 hours. The only thing we can do, is reduce the runtime of the > jobs > > > so we take less slots :-) > > > > > > Cheers, Fokko > > > > > > Op wo 26 jun. 2019 om 21:59 schreef Jarek Potiuk < > > jarek.pot...@polidea.com > > > >: > > > > > > > Yep. That's what I suggested as the reason in the ticket - I guess > > INFRA > > > > are the only people who can do anything about it (increase > concurrency > > ? > > > > pay more for Travis :)? ). > > > > > > > > On Wed, Jun 26, 2019 at 9:51 PM Ash Berlin-Taylor <a...@apache.org> > > > wrote: > > > > > > > > > I asked Travis on twitter and they said it was due to the Apache > > other > > > > > projects build queues > > > > > > > > > > https://twitter.com/travisci/status/1143893051460526080 > > > > > > > > > > -ash > > > > > > > > > > On 26 June 2019 20:48:33 BST, Jarek Potiuk < > jarek.pot...@polidea.com > > > > > > > > wrote: > > > > >> > > > > >> Hello everyone, > > > > >> > > > > >> For the last few days the Travis builds for apache/airflow project > > are > > > > >> waiting in a queue for hours. This is not a normal situation. I've > > > > opened > > > > >> INFRA ticket for that: > > > > https://issues.apache.org/jira/browse/INFRA-18657 > > > > >> > > > > >> J. > > > > >> > > > > >> > > > > > > > > -- > > > > > > > > Jarek Potiuk > > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > > > M: +48 660 796 129 <+48660796129> > > > > [image: Polidea] <https://www.polidea.com/> > > > > > > > > > > > > > -- > > > > Jarek Potiuk > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > M: +48 660 796 129 <+48660796129> > > [image: Polidea] <https://www.polidea.com/> > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
