Ah cool. Didn't know Cloudera has this :) Yes I'm looking for Docker environments just for integration tests. It's especially important when using non-HDFS storage.
On Tue, Nov 12, 2024 at 11:30 PM Istvan Toth <st...@cloudera.com.invalid> wrote: > What would be the scope of these images ? > > Having a production kubernetized Hbase is hard, and a lot of tradeoff > decisions are involved. > On the other hand, having an image with a pseudo-distributed cluster for > integration testing has been done many times, > and would be very useful for projects wishing to test integrations with > HBase. > > One example is this: https://github.com/cloudera/opdb-docker > > Istvan > > On Wed, Nov 13, 2024 at 1:56 AM Daniel Roudnitsky (BLOOMBERG/ 919 3RD A) < > droudnits...@bloomberg.net> wrote: > > > Sent from Bloomberg Professional for iPhone > > > > ----- Original Message ----- > > From: Nick Dimiduk <dev@hbase.apache.org> > > To: dev@hbase.apache.org > > At: 11/12/24 19:48:00 UTC-05:00 > > > > > > There are also dockerfiles in my PRs against the hbase-kubernetes repo, > for > > both HDFS and HBase. > > > > Running HBase from a container is not straightforward when you consider > the > > needs of running it colocated with a datanode. It can be done, as I > > demonstrate in those PRs, but only when there is sympathy between the > > images from the two projects. I believe that this is the biggest unsolved > > problem in terms of barriers between the community offering an image. > > Another hurdle is the practicalities of building and publishing an image > > from ASF infra, but I think this is more of a solved problem these days. > > > > Networking is also an issue with running HBase from containers. Because > > HBase exposes the addresses of region servers to the client, client > > processes need to be on the same network as region servers, be able to > > resolve hostnames to the same IPs. All of this is very runtime > environment > > dependent. I haven’t looked at it recently, so maybe these issues are > > solved problems, but I doubt it. > > > > Because of these unsolved issues, we have not come together with a > cohesive > > story around maintaining an official image for the project. It would be > > great to have an out-of-the-box experience that provides containerized > > HBase. That was one of the goals of the hbase-customize repo. I would be > > happy to collaborate with the Hadoop community on such an effort. > > > > Thanks, > > Nick > > > > On Tue, 12 Nov 2024 at 13:08, Wei-Chiu Chuang <weic...@apache.org> > wrote: > > > > > Forgot to mention, I'm looking at this as part of the HBase on Ozone > > > project, and trying to make it work in the Docker Compose environment. > > > > > > It looks like we don't have an official Apache HBase docker image. > There > > > are many (outdated) Docker images made by other contributors though. > > Would > > > folks be interested in maintaining one? Hadoop and Ozone have them so > it > > > should be relatively easy for me to make one. > > > > > > On Tue, Nov 12, 2024 at 10:03 AM Wei-Chiu Chuang <weic...@apache.org> > > > wrote: > > > > > > > Hi I'm looking for a little explanation about the various docker > files > > in > > > > the HBase project. > > > > > > > > There are currently 6 Dockerfiles in the codebase: > > > > > > > > % find . -name Dockerfile > > > > ./dev-support/create-release/mac-sshd-gpg-agent/Dockerfile > > > > ./dev-support/create-release/hbase-rm/Dockerfile > > > > ./dev-support/docker/Dockerfile > > > > ./dev-support/hbase_docker/m1/Dockerfile > > > > ./dev-support/hbase_docker/Dockerfile > > > > ./dev-support/flaky-tests/Dockerfile > > > > > > > > I can understand the first two are for creating releases, and the > last > > > one > > > > is used by nightly Jenkins job. But what about the rest? Trying to > > > > understand the purpose of them, and update the HBase reference guide. > > > > > > > > BTW the ones under dev-support/hbase_docker/ don't work on the master > > > > branch due to JDK version. I'll open a jira for them. > > > > > > > > > > > > -- > *István Tóth* | Sr. Staff Software Engineer > *Email*: st...@cloudera.com > cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera > on LinkedIn] <https://www.linkedin.com/company/cloudera> > ------------------------------ > ------------------------------ >
