Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- On 03/02/2021 17:50, Francois-Xavier Le Bail via tcpdump-workers wrote: > These scripts can be used locally for build tests, used on other CI and > easily be updated to run new tests (32 bits builds, sanitizers, coverage, > etc). > > Next step is doing similar setup for tcpdump... Done with commit 973e9c1c9c12bdab5e038c364e62bd099d3645ec. Some updates after. --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- Francois-Xavier Le Bail via tcpdump-workers wrote: > To save CI runtime, I have committed > a063c2d21417345ee583551ef2c07a0be6b32696 for libpcap. > This will currently run only five builders (amd64, arm64, ppc64le, > s390x and osx) and do the matrix processing with scripts. Thanks. --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- On 01/02/2021 18:08, Denis Ovsienko via tcpdump-workers wrote: > On Mon, 18 Jan 2021 22:29:21 -0800 > Guy Harris via tcpdump-workers > wrote: > >> I guess we meet those requirements, although I'm not too keen on >> having to keep going hat-in-hand to them every time we run out of >> credits; hopefully, we can just get a renewable amount. > I had requested a renewable OSS allowance on 29 January, got the > template response and confirmed the details. Let's see where it goes. > The account is at 3790/1 credits as of today, in other words, three > more builds of libpcap or at most one tcpdump build, if/when the latter > migrates. To save CI runtime, I have committed a063c2d21417345ee583551ef2c07a0be6b32696 for libpcap. This will currently run only five builders (amd64, arm64, ppc64le, s390x and osx) and do the matrix processing with scripts. We can build with ~ half the time (Total time ~26 mins). These scripts can be used locally for build tests, used on other CI and easily be updated to run new tests (32 bits builds, sanitizers, coverage, etc). Next step is doing similar setup for tcpdump... --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- On Mon, 18 Jan 2021 22:29:21 -0800 Guy Harris via tcpdump-workers wrote: > I guess we meet those requirements, although I'm not too keen on > having to keep going hat-in-hand to them every time we run out of > credits; hopefully, we can just get a renewable amount. I had requested a renewable OSS allowance on 29 January, got the template response and confirmed the details. Let's see where it goes. The account is at 3790/1 credits as of today, in other words, three more builds of libpcap or at most one tcpdump build, if/when the latter migrates. -- Denis Ovsienko --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- Denis Ovsienko via tcpdump-workers wrote: > My last commit to libpcap had waited in the queue for 2 days to get > tested. Obviously, the old free service is in the process of > disappearing, as advertised. I am migrating tcpslice and libpcap to the > new free service today, and tcpdump migration is imminent. If anybody > has useful input to make, please do not delay. I moved other open source projects without incident. So please don't hesitate, it was just unclear to me how to do it at first, and now I found the "migrate" button. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works|IoT architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- On Mon, 18 Jan 2021 22:29:21 -0800 Guy Harris via tcpdump-workers wrote: > So we can either migrate to travis-ci.com or use other providers. Travis has been downsizing the .org server pool for a couple weeks, so far it has went from 370 to 250 with apparent effect on the backlog: https://www.traviscistatus.com/#month My last commit to libpcap had waited in the queue for 2 days to get tested. Obviously, the old free service is in the process of disappearing, as advertised. I am migrating tcpslice and libpcap to the new free service today, and tcpdump migration is imminent. If anybody has useful input to make, please do not delay. -- Denis Ovsienko --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- On Thu, 21 Jan 2021 08:15:33 +0100 Dagobert Michelsen wrote: > Hi folks, > > Am 21.01.2021 um 04:24 schrieb Denis Ovsienko via tcpdump-workers > : > >>s390x (a/k/a z/Architecture, i.e. S/3x0-64) - a big-endian > >> platform, so we can do some testing of operation on big-endian > >> machines. > > Just as a tiny comment: big-endian is also checked on Solaris Sparc > with the OpenCSW farm buildbot already hooked up in the Github CI. OpenCSW buildbot has been building tcpdump and libpcap since at least 2014, and many times it proved to be useful in detecting and fixing bugs that were less obvious with other architectures, operating systems and compilers. The shell access is especially convenient for debugging as it provides a much shorter feedback loop. Thank you for providing this service to the community, Dagobert. -- Denis Ovsienko --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- Hi folks, Am 21.01.2021 um 04:24 schrieb Denis Ovsienko via tcpdump-workers : >> s390x (a/k/a z/Architecture, i.e. S/3x0-64) - a big-endian >> platform, so we can do some testing of operation on big-endian >> machines. Just as a tiny comment: big-endian is also checked on Solaris Sparc with the OpenCSW farm buildbot already hooked up in the Github CI. Best regards — Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- On Mon, 18 Jan 2021 22:29:21 -0800 Guy Harris via tcpdump-workers wrote: > Travis CI is announcing on the travis-ci.org site that "... > travis-ci.org will be shutting down in several weeks, with all > accounts migrating to travis-ci.com. Please stay tuned here for more > information." > > They don't provide any information there. However, at Thank you for collating a comprehensive problem statement. I had noticed the banner a few weeks ago, but before raising this decided to take some time to practise the migration on repositories other than the main tcpdump/libpcap repositories. Some information is missing and other information is inconsistent or difficult to match with the web control panel, but after a while I managed to get the hang of it. [...] > * For those of you who have been building on public > repositories (on travis-ci.com, with no paid subscription), we will > upgrade you to our trial (free) plan with a 10K credit allotment > (which allows around 1000 minutes in a Linux environment). [...] > We haven't been building on travis-ci.com, so presumably the first > item in the list doesn't apply. If the "We will be offering an > allotment..." part applies, the "should your run out of credits again > you can repeat the process to request more or discuss a renewable > amount" seems like a pain. In practical terms it means that all new accounts on travis-ci.com default to the free plan with 1 credits gratis one-off. The credits burn down at different rates depending on the OS. The tcpdump group is at 9970/1 now. [...] > I guess we meet those requirements, although I'm not too keen on > having to keep going hat-in-hand to them every time we run out of > credits; hopefully, we can just get a renewable amount. Re requirements: yes. Re renewable: it would be reasonable to expect it. In the worst case it will be $70/month for the cheapest plan. In the case of a monthly FOSS allowance it may be not large enough to accommodate the current setup, which is a full round of the following work for every master push and pull request: tcpdump: 108 jobs, of which 12 are macOS libpcap: 36 jobs, of which 4 are macOS Which in practical terms would mean that committers need to remember to skip CI when appropriate (to skip all three CI systems put "[skip ci]" on the git commit summary line, see my message from 21/08/2020 for details). Also it may make sense to make the default CI matrix smaller, and to run the full CI round manually (same as Coverity) or once a week/month automatically. In any case, it is practicable. > Frankly, I agree with the reply to that comment where they say > > I wonder if travis thinks they have something to gain in > reputation by letting this info out in dribs and drabs, making it > very hard to figure out what is actually going on. > > So we can either migrate to travis-ci.com or use other providers. To me it would make the most sense to migrate to the hello-real-world version of Travis CI for now if and where possible, and to take time to consider any long-term changes. > We're currently using AppVeyor for Windows builds and Cirrus CI for > FreeBSD builds. Beware that Cirrus documentation and feature set are nowhere near Travis. I had added Cirrus CI because at the time it was the only way to get FreeBSD. Travis documentation now has a minimal notion of experimental FreeBSD support, but no particulars yet. > AppVeyor offers Windows, Linux, and macOS; Cirrus offers those plus > FreeBSD. Neither of those have, as far as I know, decided to > complicate the process of using them for free software projects. Maybe they have deeper pockets, or fewer free workloads to piggy-back, or both. > The only thing I see that Travis offers that others don't is non-x86 > builds - they support: > > ppc64le - useful for tcpdump given that at least one bug > popped up only there (we were "cheating" in the use of a crypto > library, and only the ppc64le build of that library relied on the > program not cheating); > > arm64 for Linux - that may become a more common platform over > time (I don't know whether it requires strict alignment for 2-byte > and 4-byte loads - as I remember from reading the ARMv8-A > documentation, there's a control register bit to indicate whether to > fault or allow unaligned accesses, and I haven't checked whether > Linux enables them or not); > > s390x (a/k/a z/Architecture, i.e. S/3x0-64) - a big-endian > platform, so we can do some testing of operation on big-endian > machines. > > I don't know whether any other CI products that offer free service to > free-software projects support non-x86 platforms. Just that alone puts Travis way ahead of most other services. -- Denis Ovsienko --- End Message --- ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] Stick with Travis for continuous integration, or switch?
--- Begin Message --- Travis CI is announcing on the travis-ci.org site that "... travis-ci.org will be shutting down in several weeks, with all accounts migrating to travis-ci.com. Please stay tuned here for more information." They don't provide any information there. However, at https://travis-ci.community/t/build-delays-for-open-source-project/10272/26 they say As was pointed out in Builds hang in queued state 3 linked to earlier in this topic, Travis is moving workers from travis-ci.org to travis-ci.com 1 in preparation to fully close .org (or rather, make it read-only) around the New Year. ... So you need to migrate to .com to stop experiencing delays. Note the caveats: ... They claim that they'll still offer free service for free software: Q. Will Travis CI be getting rid of free users? # A. Travis CI will continue to offer a free tier for public or open-source repositories on travis-ci.com and will not be affected by the migration. They also say here: https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing that The upcoming pricing change will not affect those of you who are: * Building on the Travis CI 1, 2 and 5 concurrency job plans who are building on Linux, Windows and experimental FreeBSD environments. * GitHub Marketplace plans * Grouped Accounts * Enterprise customers (not building in our cloud environments) * Builders on our premium or manual plans. Contact the Travis CI support team for more information. but they also say that The upcoming pricing change will affect those of you who are: Building on the macOS environment macOS builds need special care and attention. We want to make sure that builders on Mac have the highest quality experience at the fastest possible speeds. Therefore, we are separating out macOS usage from other build usage and offering a distinct add-on plan that will correlate directly to your macOS usage. Purchase only the credits you need and use them until you run out. * $15 will buy you 25 000 credits (1 minute of mac build time costs 50 credits) * Use your credits for macOS builds only when you need to run these * Replenish your credits as you need them * More special build environments that fall into this category will be available soon which may mean that their "free tier" doesn't include macOS. They also say: Building on a public repositories only We love our OSS teams who choose to build and test using TravisCI and we fully want to support that community. However, in recent months we have encountered significant abuse of the intention of this offering (increased activity of cryptocurrency miners, TOR nodes operators etc.). Abusers have been tying up our build queues and causing performance reductions for everyone. In order to bring the rules back to fair playing grounds, we are implementing some changes for our public build repositories. * For those of you who have been building on public repositories (on travis-ci.com, with no paid subscription), we will upgrade you to our trial (free) plan with a 10K credit allotment (which allows around 1000 minutes in a Linux environment). * You will not need to change your build definitions when you are pointed to the new plan * When your credit allotment runs out - we’d love for you to consider which of our plans will meet your needs. * We will be offering an allotment of OSS minutes that will be reviewed and allocated on a case by case basis. Should you want to apply for these credits please open a request with Travis CI support stating that you’d like to be considered for the OSS allotment. Please include: * Your account name and VCS provider (like travis-ci.com/github/[your account name] ) * How many credits (build minutes) you’d like to request (should your run out of credits again you can repeat the process to request more or discuss a renewable amount) * Usage will be tracked under your account information so that you can better understand how many credits/minutes are being used We haven't been building on travis-ci.com, so presumably the first item in the list doesn't apply. If the "We will be offering an allotment..." part applies, the "should your run out of credits again you can repeat the process to request more or discuss a renewable amount" seems like a pain. See also this comment: https://travis-ci.community/t/org-com-migration-unexpectedly-comes-with-a-plan-change-for-oss-what-exactly-is-the-new-deal/10567/15 where the commenter says: When I emailed support for credits, they gave this list of requirements for the so-called “
