Hi Adam, Thanks for your feedback! For the DIY sources of dependecies I think there's a mix of benifits and drawbacks AFA practical security. One risk with snake oil security that I noticed when looking at windows package databases seems to be that specific builds of binaries sometimes get marked as false positives if a virus writer uses them for their dependencies.
With the "coucholddeps" I am pulling the otp and moz&icu binaries from a past builds so they can start building couchdb itself right away, the regular couchdb jobs aren't very practical to debug since they are waiting for the dependencies to be built in the current run. >From what documentation I could find, there's no way to have a conditional "needs" dependency so they are mostly a cut and paste.. Will Am Do., 9. Dez. 2021 um 16:42 Uhr schrieb Adam Kocoloski <kocol...@apache.org>: > > Hi Will! > > I’ve been happy with the GitHub Actions approach to Windows CI for erlfdb. > The submodule-based installation of third party actions hasn’t proven to be a > significant contributor to the total runtime. I think adding this workflow to > the glazier repo is a great idea. > > I remain interested in a simplified option for Windows builds that uses > pre-built packages for Erlang/OTP instead of relying on our own glazier-based > artifacts. I’m under the impression that we build Erlang ourselves mostly for > historical reasons, but I could be wrong. In the CI world we could simplify a > few steps by using the setup-beam action if we went that route as well. But > we don’t have to address that here. > > I have a few questions about the workflow — what’s the thinking behind the > separate “coucholddeps” and “couchdb” jobs? — but we can save that for a PR > if you like. Cheers, > > Adam > > > On Dec 9, 2021, at 4:06 AM, Will Young <lostnetwork...@gmail.com> wrote: > > > > Hi, > > > > I've been looking into github workflows primarily with an interest > > in being able to run ci tests on some of the supported platforms on > > changes to test a potential contribution isn't going to break them. > > > > In the case of windows everything is unbundled so glazier's process > > is providing a couchdb package with spidermonkey+icu, otp+openssl, > > where most of this is built from source. > > > > I've put put this process into a flow as several separate jobs that > > deliver intermediate results using some workarounds from discussions > > on slack, and could avoid many of the older workarounds in glazier by > > using newer versions of the dependencies. > > > > An example of the resulting release binaries are here: > > https://github.com/lostnet/couchdb/releases/tag/msdeps12 > > > > And the workflow that is generating them is here: > > https://github.com/lostnet/couchdb/blob/msdeps12/.github/workflows/mslegacy.yml > > > > I think it would make sense to integrate this as a flow on the > > couchdb-glazier repo to create an automated reference build and > > binaries. > > > > Issues: > > > > 1. versioning for reliability and supply side attacks (as discussed on > > slack): > > > > The runners get updated to new versions of things which break builds. > > It is helpful though to be easily able to attempt to reproduce older > > builds again to eliminate theories of the source of new build > > failures. > > Versions of things should also be locked down as should uses of third > > party actions like erlfdb's flows do via git submodule pinning. There > > is an inherent tradeoff that the install time of anything locked down > > instead of from a runner's defaults adds to the runtime. > > > > 2. Using the flow for release binaries: > > > > Aside from (1) it would not be wise to use gpg keys on a cloud > > service. Releases can be modified to add binaries so a couchdb member > > could add signed binaries ideally using binaries built in the flow > > itself if they can verify they were correctly built (or possibly > > running the flow on their own clone or manually follow the flow > > actions.) > > The versions of dependencies and compilers used in the build are > > newer than past builds. Where possible it would be best to verify > > these versions as older versions usually have more quirks for > > automating the build on modern runners. > > > > 3. For CI: > > If glazier repo had these flows and releases, I would like to add > > the ability to run the couchdbolddeps job as a manual action in the > > couchdb repo similar to the ubuntu ci added with esr91pr. > > > > 4. There are still quite a few problems in the tests to investigate as > > well as whether the resulting packages are valid (and whether > > foundationdb is intended to be external or a bundled deliverable on > > couch4?) > > > > At Any rate, I would appreciate any thoughts on automating the windows > > builds. > > > > Thanks, > > Will Young >
