Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
github-actions[bot] commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-2076286592 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch) If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
clayburn commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1910737138 > Looks like Jenkins doesn't have permission (by default at least) to write entries to the ASF Gradle Enterprise Build Cache: This is correct. The reason for this is that we do not currently intend for projects to use the built-in cache node (which is what this PR configures). If used this way, the node would be shared by all projects, which is not desirable because we would not want project A to inadvertently read a cache entry from project B. In the future, we will support such a use case without that risk, but for today, the correct option is to stand up a remote cache node specifically for Kafka, as requested in [INFRA-25336](https://issues.apache.org/jira/browse/INFRA-25336) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
divijvaidya commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1875475685 https://github.com/apache/beam/pull/27015 is not Apache Beam enabled this. Seems like we need to request a build node from Apache Infra and change our jenkins to pick up creds similar to how beam did it (from env vars which are deployed by infra). I have created a ticket to talk to Infra at https://issues.apache.org/jira/browse/INFRA-25336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
nicktelford commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1875405401 Looks like Jenkins doesn't have permission (by default at least) to write entries to the ASF Gradle Enterprise Build Cache: ``` Could not store entry 744bd685bdd1d149ec93934b276b7698 in remote build cache: Storing entry at 'https://ge.apache.org/cache/744bd685bdd1d149ec93934b276b7698' response status 403: Forbidden ``` It might require some explicit authentication. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
nicktelford commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1875356669 @ijuma I've temporarily reconfigured it to enable the cache for this branch (`gradle-remote-build-cache`) instead of `trunk`, so we can test that it works as expected using the CI builds on this PR. Pushing to the cache from these builds should be fine, since there are no code changes, so the task outputs should be the same as in `trunk` anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
ijuma commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1874508632 I guess one way would be to enable the cache only for CI & trunk by default at first. Then we could manually enable the cache locally and for one PR to test the behavior. If it all looks good, we would then enable the cache for everyone (but leave pushes for trunk && CI only). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
ijuma commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1874507122 Could we do a test with a new branch (not trunk) where we validate the behavior before we roll it out to everyone else? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
nicktelford commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1874500874 Hi @divijvaidya , this enables the _remote_ build cache, which allows different machines to share the cached output of tasks. Thanks for the links to the builds, I haven't had a chance to look at them yet, but the thing that requires access to ge would be too simulate a trunk CI run and push to the cache, which I won't be able to do locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
divijvaidya commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1874483500 Hey @nicktelford , you don't need committer permissions to look at ASF gradle. Try using this link: https://ge.apache.org, it should be publicly accessible. For this PR, here's the build - https://ge.apache.org/s/bdwopqcoxwge2 which shows `Build Cache` as `On` which is unlike this build on trunk https://ge.apache.org/s/ywmt7vaghr62u where `Build Cache` is `Off`. I am not sure how useful this will be since CI for trunk may build on different boxes but it's definitely worth keeping it on. @ijuma is our gradle expert in the community. I will wait to hear his thoughts regarding this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Enable Gradle Remote Build Cache [kafka]
nicktelford commented on PR #15109: URL: https://github.com/apache/kafka/pull/15109#issuecomment-1874134447 I don't have access to the ASF Gradle Enterprise, so a committer will need to verify that this works as intended. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] MINOR: Enable Gradle Remote Build Cache [kafka]
nicktelford opened a new pull request, #15109: URL: https://github.com/apache/kafka/pull/15109 We enable the remote build cache, hosted by the ASF Gradle Enterprise instance. We only cache tasks during builds on `trunk`, to ensure that pushes to PRs always re-run all Tasks needed by that PR. PRs will read from the build cache, enabling tasks for modules that haven't been changed in the PR to be read from the cache. This should ensure that changes "leaf" modules (e.g. connect, streams, etc.) don't run the entire test suite from dependent modules (i.e. core, clients, storage, etc.), which can be very costly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org