A few of us got together with some fine folks in Apache Infra and discussed getting builds going in AWS using an instance type that would give us the under 2-hour build time. Turns out this one was also on Infra's wishlist and just needed someone to volunteer to step up and own the task. Cesar volunteered to lead an effort. :)
Big thank you to Cesar!! If anyone wants to help, join this list and keep an eye out for related emails. - https://lists.apache.org/list.html?bui...@apache.org -- David Blevins http://twitter.com/dblevins http://www.tomitribe.com > On Oct 16, 2022, at 11:50 AM, David Blevins <david.blev...@gmail.com> wrote: > > Here's an updated set of data with some new EC2 instance types: > > c1.xlarge-gp2 : [INFO] Total time: 03:59 h > c1.xlarge-gp3 : [INFO] Total time: 04:00 h > c6i.2xlarge : [INFO] Total time: 01:40 h > c6i.8xlarge : [INFO] Total time: 01:34 h > c7g.2xlarge : [INFO] Total time: 02:02 h > c7g.4xlarge : [INFO] Total time: 01:55 h > c7g.8xlarge : [INFO] Total time: 01:51 h > t2.medium : [INFO] Total time: 05:05 h > t2.xlarge : [INFO] Total time: 02:30 h > t3.medium : [INFO] Total time: 03:21 h > > These all ran with MAVEN_OPTS="-Xmx4096m". I did a compare of the gp2 and > gp3 using the c1.xlarge, they look pretty identical. All of the c6i and c7g > instances also ran with gp3 storage. > > FYI the c6i types are "3rd Generation Intel Xeon Scalable processors with an > all-core turbo frequency of 3.5 GHz" > The c7g types are "AWS Graviton3" processors > > Looks like for our build the c6i.2xlarge is just fine. It's $0.34 per hour. > With 2 builds a day and 2 hours a build, that's $248.2 US/year. Let's assume > we build every PR and call it $1k or $2k / year, it's still reasonable. > > I'll chat with infra to see what kind of options we have for getting or > donating what is needed. > > > -David > > >> On Oct 13, 2022, at 4:13 PM, David Blevins <david.blev...@gmail.com> wrote: >> >> Alright, here are some build times for the runs I've done. Results are >> quite varied. >> >> c1.xlarge : [INFO] Total time: 03:59 h >> t2.medium : [INFO] Total time: 05:05 h >> t2.xlarge : [INFO] Total time: 02:30 h >> t3.medium : [INFO] Total time: 03:21 h >> my.laptop : [INFO] Total time: 01:56 h >> >> Specs for the AWS instances are public. Here are the stats for my laptop: >> >> MacBook Pro (16-inch, 2019) >> 2.4 GHz 8-Core Intel Core i9 >> Memory 64 GB 2667 MHz DDR4 >> >> I did try some other AWS instance types, but anything with 2GB or less >> failed due to out of memory issues. I'd be interested to try some other >> types, particularly the AMD-based types. >> >> Anyway long story short: yes, our build could be faster with better >> hardware. Just not very clear what that hardware should be. >> >> >> -David >> >> >>> On Oct 11, 2022, at 3:05 PM, David Blevins <david.blev...@gmail.com> wrote: >>> >>> All, >>> >>> I'm collecting some stats on how long it takes to run our full build >>> exactly as Jenkins does. The goal is to work with them to see if we can >>> get some better hardware -- I assume that will require donations, etc. >>> >>> If you'd like to help in collecting data, here's the script I'm running: >>> >>> - curl >>> https://gist.githubusercontent.com/dblevins/b39cc3300bcdd89b426ca33b87b5452b/raw/7c68d4df71e9246c8bf2d0a741f8b145ca5d0820/buildtime.sh >>> | bash >>> >>> Send the time reported in the build.log along with your system information >>> (os, number of cores, if you disk is an SSD, etc) >>> >>> >>> -- >>> David Blevins >>> http://twitter.com/dblevins >>> http://www.tomitribe.com >>> >> >
smime.p7s
Description: S/MIME cryptographic signature
