Hi Calcite and Drill devs
Here's an idea that, while it doesn't equate to the upstreaming of Drill
tests to the Calcite test suite, could mean that all of Drill's tests
would automatically and frequently be run with the HEAD of the Calcite
main branch.
Something Drill started doing last year is the continuous publication of
SNAPSHOT artefacts to the Apache snapshots repo [1]. We wanted to offer
Drill plugin developers targeting the upcoming version of Drill the
ability to pull in up to date libraries without having to build Drill
from its master branch themselves.
Being concerned about causing an explosion of artefact versions we asked
Infra whether it would be okay for Drill to republish every time a
commit is merged into master and they told us that we are not the first
and can publish there as often as we like. The only tricky bit to
setting this up was obtaining Nexus credentials from GitHub Secrets and
using them in a new GitHub workflow but we do now have a working example
in Drill [2] and I'd happy to help if Calcite is interested in doing the
same.
If Drill then bases its master branch on these proposed SNAPSHOT
artefacts then Drill's normal CI runs would continuously test it with
Calcite as at its most recent commit and we'd be made aware of
compatibility problems early. If declaring a dependency on a SNAPSHOT
version in master is too much malpractice, instability or extra process
[3] for Drill devs then I expect that the same thing could still be
achieved in a new "calcite-next" branch. In any event we'd continue to
have the stable Drill branch which would of course only ever depend on a
released version of Calcite.
Regards
James
[1]
https://repository.apache.org/content/repositories/snapshots/org/apache/drill/
[2]
https://github.com/apache/drill/blob/master/.github/workflows/publish-snapshot.yml
[3] If we did this in Drill master then I guess a new step in the Drill
release process would need to see us push a commit that pins the Calcite
dependency to the latest released version. We'd probably also become
motivated to time Drill releases so that they happen just after Calcite
releases. Personally I'd be prepared to try this out for a while.
