hartmannathan commented on PR #18576: URL: https://github.com/apache/nuttx/pull/18576#issuecomment-4252021089
@lupyuen Sure, we can try it in its current form for now. Meanwhile I had an idea that might be worth exploring: In this idea, we would create a special repository on GitHub; suggested name: nuttx-ci-deps, meaning NuttX CI Dependencies. In that repository, we would place a copy of every dependency we are currently downloading from third parties. This would be used for CI builds only, not by normal developers or users. How would it work? Our GitHub CI scripts could pass a special command line argument to the NuttX build scripts (make or cmake). In the NuttX build scripts, the special argument would cause the download logic to get files from nuttx-ci-deps instead of downloading from third parties. The rationale behind this idea is that downloading from third parties introduces a failure mode that is separate from GitHub. It could be, for example, that some third parties limit downloads originating from GitHub for the same reasons as GMP: too many downloads. That might cause some of our download failures. Downloading from third parties again and again for every build is also unfair to them. Furthermore, I would assume that GitHub probably has a more efficient and reliable path to get data from itself than from third parties. One more idea: Putting the retry logic in the GitHub CI scripts has the (known) disadvantage of retrying builds that fail due to compiler errors. In those cases we want to stop the build. We might accomplish that in the following way: A script could be implemented at tools/download.sh. This script would encapsulate all download-related logic for the NuttX build system. It would know how to run curl, wget, fetch, and git, to retrieve and validate files when called for by the config. This script could also encapsulate the retry logic. Finally, this script could check the special argument and override the URL to get the files from nuttx-ci-deps instead of the normal download location. Thoughts? Just to clarify, yes, we should try this PR as-is and see if it improves our GHA usage. If we aren't satisfied, the ideas above are additional avenues for exploration... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
