Hi Vihang, +1, We were even exploring publishing the docker images of the snapshot version as well per commit or maybe weekly, so just shoot 2 docker commands and you get a Hive cluster running with master code.
Sai, I think to spin up an env via Docker with all these things should be doable for sure, but would require someone with real good expertise with docker as well as setting up these services with Hive. Obviously, I am not that guy :-) @Simhadri has a PR which publishes docker images once a release tag is pushed, you can explore to have similar stuff for the Snapshot version, maybe if that sounds cool -Ayush On Tue, 23 May 2023 at 04:26, Sai Hemanth Gantasala <saihema...@cloudera.com.invalid> wrote: > Hi Vihang, > > +1 on the idea. > > This is a great idea to quickly test if a certain feature is working as > expected on a certain branch. > This way we test data loss, correctness, or any other unexpected scenarios > that are Hive specific only. However, I'm wondering if it is possible to > deploy/test in a kerberized environment or issues involving authorization > services like sentry/ranger. > > Thanks, > Sai. > > On Mon, May 22, 2023 at 11:15 AM vihang karajgaonkar <vihan...@apache.org> > wrote: > > > Hello Team, > > > > I have observed that it is a common use-case where users would like to > test > > out unreleased features/bug fixes either to unblock them or test out if > the > > bug fixes really work as intended in their environments. Today in the > case > > of Apache Hive, this is not very user friendly because it requires the > end > > user to build the binaries directly from the hive source code. > > > > I found that Apache Spark has a very useful infrastructure [1] which > > deploys nightly snapshots [2] [3] from the branch using github actions. > > This is super useful for any user who wants to try out the latest and > > greatest using the nightly builds. > > > > I was wondering if we should also adopt this. We can use github actions > to > > upload the snapshot jars to the public repository (e.g github packages) > and > > schedule it as a nightly job. > > > > [1] https://issues.apache.org/jira/browse/INFRA-21167 > > [2] https://github.com/apache/spark/pkgs/container/apache-spark-ci-image > > [3] https://github.com/apache/spark/pull/30623 > > > > I can take a stab at this if the community thinks that this is a nice > thing > > to have. > > > > Thanks, > > Vihang > > >
