Dev - I had a conversation IRL with Todd recently, cc'd here - not on the project but willing to help out. He has offered some advice for the project to get the Docker distro of Apache Fineract working again. I would like to have either a push back or we should restore the docker file asap.
To recap: The DockerHub Image is two years old, and the process to pull from our Dev branch has been broken that entire time. It broke when we removed the docker-build file with this ticket https://issues.apache.org/jira/browse/FINERACT-1469. With a Million downloads of fineract from DockerHUB, where that version has multiple CVEs (security issues), we should not be continuing to keep that there. So, we need to fix the docker pipeline. Credentials will be required from infra. Todd's comments: Extended Summary > > The problem for the internal Fineract development pipeline is that changes > were made to the build process that > removed the expected Dockerfile > added an external dependency to the code repo (mifos community-app web UI) > does not publish a public Fineract Docker image to Docker Hub > > At first glance, the lack of a Dockerfile in the code might seem to be the > reason that no containers have been pushed to Docker Hub. A Dockerfile is > the standard way of creating images. This is very confusing for many people > (including me), however this is not the actual problem because JIB (Java > Image Builder) is set up to build the image during testing directly from > java source code by Gradle in two places: > > build-docker-postgresql.yml > build-docker-mariadb.yml > > The problem is that JIB does not seem to be configured to actually push > the container image to Docker Hub. It only seems to be configured to build > the image for testing. > > To solve this, two things need to be done: > > > - It needs to be decided when to push the image (and possibly create a > new GitHub Action to do it) > - Code needs to be added to configure JIB to know where to push the > image on Docker Hub (see this example) > - Credentials need to be supplied to the GitHub Action to allow it > actually push the image > > > Additional Open Source Observations (Optics) > > Dockerfile > > The removal of the Dockerfile from the repo is confusing (especially > coupled with the existence of a docker-compose.yml file) and also makes it > harder for potential contributors to set up and run Fineract because now > dependencies need to be installed locally, rather than running them all in > containers. > > The lack of a Dockerfile in the repository is nonstandard from an Open > Source perspective. Regardless of whether it is needed by the Fineract > build process or not, most open source projects include a Dockerfile, and > most open source users expect one to exist in the repo so they can easily > build / run / test the project locally. Adding the Dockerfile back to the > repo should be trivial (and removes the need for JIB entirely). > > General Setup > > The current Fineract process for building and running using containers > makes it significantly harder for developers to get started with Fineract > because a local Java environment needs to be installed. More disappointing, > a completely different public set of instructions exist on Docker Hub . > These instructions do not work because they are out of date, but are > significantly easier for developers to use. Having two sets of different > install instructions is confusing, but having the simpler set of > instructions that do not work is a very bad developer experience. > > > > > On Sun, Feb 18, 2024 at 8:46 PM VICTOR MANUEL ROMERO RODRIGUEZ < > victor.rom...@fintecheando.mx> wrote: > >> Hello, >> >> Another way to have the Docker Hub image published (just like Apache >> Tomcat): >> >> https://github.com/docker-library/official-images >> >> https://github.com/docker-library/tomcat >> >> Regards >> >> >> >> El dom, 18 feb 2024 a las 10:05, James Dailey (<jdai...@apache.org>) >> escribió: >> >>> Is there an easy thing to request? >>> >>> ---------- Forwarded message --------- >>> From: Gavin McDonald <gmcdon...@apache.org> >>> Date: Sun, Feb 18, 2024 at 12:24 AM >>> Subject: Re: Docker help >>> To: James Dailey <jdai...@apache.org> >>> CC: Users <us...@infra.apache.org> >>> >>> >>> Hi James. >>> >>> >>> >>> On Sun, Feb 18, 2024 at 3:00 AM James Dailey <jdai...@apache.org> wrote: >>> >>>> Infra - >>>> >>>> Can you confirm that we can use other processes to push to >>>> apache DockerHUB? >>>> >>> >>> Current supported methods are via Github Actions or Jenkins or locally >>> via your own credentials. >>> >>> For Github Actions we can use a role account and attach the secrets to >>> your repository, or you >>> can provide your own secrets for us to add to your repository >>> >>> For Jenkins we have a role account that we provide access to push to >>> your repository. >>> >>> Committers could also use a settings.xml with this plugin and use their >>> own credentials, we just need >>> to ensure they have push access to Dockerhub. >>> >>> There may also be other methods not explored. >>> >>> See also: >>> https://github.com/GoogleContainerTools/jib/tree/master/jib-maven-plugin#authentication-methods >>> >>> HTH >>> >>>> >>>> When I opened a ticket about this, I was told we need a dockerfile at >>>> the root. >>>> >>>> Can we use "jib-maven-plugin to publish the image to Dockerhub". ? >>>> Can we get credentials ? >>>> >>>> James >>>> >>>> >>>> ---------- Forwarded message --------- >>>> From: Arnold Galovics <arn...@apache.org> >>>> Date: Sun, Feb 11, 2024 at 10:45 PM >>>> Subject: Re: Docker help >>>> To: <dev@fineract.apache.org> >>>> >>>> >>>> James, >>>> >>>> This is the out-of-the box solution from DockerHub which definitely >>>> won't work without a Dockerfile. Though that doesn't mean it's the only way >>>> to build a docker image; as I stated in my previous email. >>>> >>>> Best, >>>> Arnold >>>> >>>> On Mon, Feb 12, 2024 at 7:43 AM James Dailey <jamespdai...@gmail.com> >>>> wrote: >>>> >>>>> On DockerHUB the build fails because there is no dockerfile. >>>>> https://hub.docker.com/r/apache/fineract >>>>> >>>>> 2024-02-08T13:12:27Z Building in Docker Cloud's infrastructure... >>>>> 2024-02-08T13:12:28Z Cloning into '.'... >>>>> 2024-02-08T13:12:28Z Warning: Permanently added the RSA host key for >>>>> IP address '140.82.114.4' to the list of known hosts. >>>>> 2024-02-08T13:12:48Z Reset branch 'develop' >>>>> 2024-02-08T13:12:48Z Your branch is up to date with 'origin/develop'. >>>>> 2024-02-08T13:12:48Z Dockerfile not found at ./Dockerfile >>>>> >>>>> >>>>> Let's discuss on slack and revert back here. >>>>> >>>>> My intention is to either DELETE the DockerHUB repo or to get this >>>>> working. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Sun, Feb 11, 2024 at 10:14 PM Arnold Galovics <arn...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi Zoltan, James, >>>>>> >>>>>> Just to reflect on your points: >>>>>> 1) Let's not do such a radical change unless we absolutely need to >>>>>> 2) I'm not sure what's the issue here, please explain. We already >>>>>> have docker builds in our pipeline via GitHub Actions (using their >>>>>> runners), the only missing piece is to do a docker push. >>>>>> >>>>>> We need the credentials to be able to do a docker push, alter the >>>>>> pipeline and that's all. >>>>>> >>>>>> If the only thing preventing us from doing this is to keep asking the >>>>>> infra team for the creds, let's pursue them instead of making such an >>>>>> unnecessary change. >>>>>> >>>>>> Arnold >>>>>> >>>>>> On Mon, Feb 12, 2024 at 3:30 AM James Dailey <jamespdai...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks Zoltan >>>>>>> >>>>>>> Micheal - can you please comment on this discussion? As this >>>>>>> relates to the Google deployment that you put in place? Question! >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Feb 11, 2024 at 6:27 PM Zoltan Mezei <zoltan.me...@zz-it.hu> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I think the real issue here is that we use GoogleContainerTools's >>>>>>>> Jib as the build mechanism. It works entirely without a Dockerfile. And >>>>>>>> unfortunately Dockerhub's Automated Builds doesn't support building >>>>>>>> without >>>>>>>> a Dockerfile. :-( >>>>>>>> >>>>>>>> We have two ways to move forward: >>>>>>>> >>>>>>>> 1. Replace the Jib build with a more traditional, Dockerfile-based >>>>>>>> approach. This would be a quite large change of how Fineract is built >>>>>>>> and >>>>>>>> the consequences need to be explored - but it's definitely doable. >>>>>>>> 2. Stick with the Jib build, but don't rely on >>>>>>>> Dockerhub's Automated Builds, but some other build tools like >>>>>>>> jib-maven-plugin to publish the image to Dockerhub. This could also >>>>>>>> work, >>>>>>>> but it requires a build server that I'm not sure we have. >>>>>>>> >>>>>>>> I can try to create a traditional Dockerfile, but it will be >>>>>>>> different from what Jib can produce, so this might lead to regressions. >>>>>>>> >>>>>>>> Want me to try this approach next week? >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> Zoltan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Feb 11, 2024 at 8:16 AM James Dailey < >>>>>>>> jamespdai...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Victor - my read of the docs is that the default “build rule “ >>>>>>>>> points to master or main but we can also use dev. In fact that’s what >>>>>>>>> is >>>>>>>>> already there in dockerHUB for our project. >>>>>>>>> >>>>>>>>> I think a proper dockerfile in dev branch should be fine. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> James >>>>>>>>> >>>>>>>>> On Fri, Feb 9, 2024 at 7:47 PM VICTOR MANUEL ROMERO RODRIGUEZ < >>>>>>>>> victor.rom...@fintecheando.mx> wrote: >>>>>>>>> >>>>>>>>>> Reading the dockerhub docs, I think we can do the following: >>>>>>>>>> >>>>>>>>>> 1. Create a master branch from develop branch >>>>>>>>>> 2. Add the Dockerfile (and some scripting on it for handling the >>>>>>>>>> versions) on master branch >>>>>>>>>> 3. Dockerhub will use the dockerfile (and its scripts) from the >>>>>>>>>> master branch >>>>>>>>>> 4. Create github action for keeping in sync develop with master, >>>>>>>>>> so then it will push the changes to the master branch everytime the >>>>>>>>>> develop >>>>>>>>>> branch has a commit on it, then the dockerhub will publish it as the >>>>>>>>>> latest >>>>>>>>>> version. >>>>>>>>>> >>>>>>>>>> Or... we can be more standard >>>>>>>>>> >>>>>>>>>> 1. Rename develop to master >>>>>>>>>> 2. Add a Dockerfile template (and some scripting on it for >>>>>>>>>> handling the versions) on master branch >>>>>>>>>> 3. Dockerhub will use the dockerfile (and its scripts) from the >>>>>>>>>> master branch >>>>>>>>>> 4. Everytime a new commit or tag is created, the dockerhub will >>>>>>>>>> publish it as the latest/specific version. >>>>>>>>>> >>>>>>>>>> What do you think? >>>>>>>>>> >>>>>>>>>> Dockerhub automated builds info: >>>>>>>>>> https://docs.docker.com/docker-hub/builds >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> El vie, 9 feb 2024 a las 20:34, James Dailey (< >>>>>>>>>> jamespdai...@gmail.com>) escribió: >>>>>>>>>> >>>>>>>>>>> Victor - I was trying to go down that path as well, as that is >>>>>>>>>>> the error thrown and the suggestion at DockerHUB. However, to add >>>>>>>>>>> the key >>>>>>>>>>> to the git hub requires access and the git is controlled by Apache >>>>>>>>>>> Infra. >>>>>>>>>>> I asked infra@a.o. about that since, again, that is what >>>>>>>>>>> DockerHUB had documented. Unfortunately, I think infra has it >>>>>>>>>>> setup a >>>>>>>>>>> specific way to allow all of the projects to publish to the Apache >>>>>>>>>>> DockerHUB so that route would appear to be blocked. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Feb 9, 2024 at 4:04 PM VICTOR MANUEL ROMERO RODRIGUEZ < >>>>>>>>>>> victor.rom...@fintecheando.mx> wrote: >>>>>>>>>>> >>>>>>>>>>>> For making it work without a Dockerfile the credentials of the >>>>>>>>>>>> docker hub account are requiered. >>>>>>>>>>>> >>>>>>>>>>>> If they are set in the git repository, a github action can be >>>>>>>>>>>> enabled for this task. >>>>>>>>>>>> >>>>>>>>>>>> Regards >>>>>>>>>>>> >>>>>>>>>>>> El vie., 9 de febrero de 2024 4:45 p. m., < >>>>>>>>>>>> jamespdai...@gmail.com> escribió: >>>>>>>>>>>> >>>>>>>>>>>>> I've re-opened >>>>>>>>>>>>> https://issues.apache.org/jira/browse/FINERACT-1164 >>>>>>>>>>>>> >>>>>>>>>>>>> This ticket is to enable the build at DockerHUB to work. For >>>>>>>>>>>>> the past two years ++ the Build has failed. >>>>>>>>>>>>> >>>>>>>>>>>>> https://hub.docker.com/r/apache/fineract >>>>>>>>>>>>> This docker account is held by Apache and the Fineract project >>>>>>>>>>>>> is responsible for the content. >>>>>>>>>>>>> >>>>>>>>>>>>> The dockerHUB has an "auto build" concept so that every >>>>>>>>>>>>> committed change on Dev leads to a new deployment. >>>>>>>>>>>>> >>>>>>>>>>>>> The build is actually failing or not running because we >>>>>>>>>>>>> have removed the dockerbuild file from the root. That is as far >>>>>>>>>>>>> as I've >>>>>>>>>>>>> gotten. I suspect we had good reasons for that at the time. >>>>>>>>>>>>> >>>>>>>>>>>>> Anyway, I would also say that if we cannot get the Docker >>>>>>>>>>>>> build to work THEN we should take this down. Our standard is to >>>>>>>>>>>>> only >>>>>>>>>>>>> support and distribute publicly the last two releases. This build >>>>>>>>>>>>> is really >>>>>>>>>>>>> old, has unfixed CVEs, and is being downloaded in large numbers. >>>>>>>>>>>>> (no idea >>>>>>>>>>>>> why) >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> James >>>>>>>>>>>>> >>>>>>>>>>>> >>> >>> -- >>> >>> >>> *Gavin McDonald - * >>> Systems Administrator, ASF Infrastructure Team >>> V.P Travel Assistance Committee >>> >>> https://tac.apache.org - Applications now open for Community Over Code >>> 2024 >>> in Bratislava, Slovakia. Don't delay, apply today! >>> >>>
