Resent the email at the mail thread. On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <[email protected]> wrote:
> Hello everyone, > > I am trying to implement xwiki helm chart in High availability but facing > some problems. > > How we can implement High Availability(HA) in Kubernetes:- > > 1. Through Deployment:- In this approach we would use a shared file > storage, that would get attached to multiple pods(or containers) > 2. Through StatefulSet:- In this approach every pod(or container) would > have it’s own different volume and state, which should be synchronised by > an consensus algorithm > > Problem faced in xwiki case:- > > 1. In deployment case we need a locking mechanism for our filesystem, > moreover caching and states should also be maintained outside the container. > 2. In StatefulSet the consensus algorithm is not present so it is > currently not possible. > > Could anyone help me through it. > > Thanks, > Ashish > > On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <[email protected]> > wrote: > >> Hey Ashish, >> >> I think Vincent/Thomas can help with the PR on the xwiki-docker project. >> >> Regarding adding docs for High-Availability setup using shared data >> volumes, I think you can use could create a docs folder in the GitHub repo, >> and add the docs in Markdown format. I think you can also document it in >> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart >> should support enabling HA, and then passing a storage class which supports >> shareable volumes to Pods, and then you can document examples for using >> this with something like Rook+GKE setup. >> >> Best, >> Shubham. >> >> >> > On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[email protected]> >> wrote: >> > >> > Hello everyone, >> > >> > I require the xwiki solr docker container to do progress with my work. I >> > have made a pull request >> > <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same. >> > Please look into it. >> > >> > Moreover, I was working with Rook(Ceph Operator) for a shared file >> system >> > which would be a better approach to deploy our app on production. So I >> > think we should document it somewhere. Can anyone help me where should >> I do >> > that. It may contain the details to configure rook-ceph, and then >> > connecting it. >> > >> > P.S. I am writing a blog at Medium on how to configure rook on Google >> > Kubernetes Engine. We could also use that? >> > >> > Thanks, >> > Ashish >> > >> > On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma < >> [email protected]> >> > wrote: >> > >> >> Hi Shubham, >> >> >> >> Thank you Shubham, I will start working on adding test cases. >> >> >> >> Thanks, >> >> Ashish >> >> >> >> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[email protected]> >> >> wrote: >> >> >> >>> Hey Ashish, >> >>> >> >>> Thanks for the update. >> >>> >> >>> Adding our chart to incubator section of helm community charts >> repository >> >>> sounds great. Although (unlike Dockerhub) this repo is for community >> charts >> >>> and not really official charts of the respective installations, but >> >>> supporting this chart in the community chart repo should help with the >> >>> visibility. >> >>> >> >>> Lets also work on adding tests to the helm chart to ensure things are >> >>> going fine. >> >>> >> >>> Best, >> >>> Shubham. >> >>> >> >>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma < >> [email protected]> >> >>> wrote: >> >>>> >> >>>> Hi everyone, >> >>>> >> >>>> Update on HELM project:- >> >>>> >> >>>> We have added some features to the chart that includes:- >> >>>> * Making chart configurable with both mysql and postgres database >> >>>> * We are now using configmaps and secrets for storing database >> >>> credentials >> >>>> * We use helm dependency to manage the dependencies of the chart >> >>>> >> >>>> I think our chart is ready to be deployed in the incubator section of >> >>> helm >> >>>> community repo. <https://github.com/helm/charts> If you guys agree I >> >>> would >> >>>> make the necessary changes required to deploy it there. >> >>>> >> >>>> Thanks >> >>>> - Ashish >> >>>> >> >>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[email protected]> >> >>> wrote: >> >>>> >> >>>>> Hi Ashish, >> >>>>> >> >>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <[email protected] >> > >> >>>>> wrote: >> >>>>>> >> >>>>>> Hi everyone, >> >>>>>> >> >>>>>> The first week update of helm chart xwiki project. We currently >> have a >> >>>>>> chart that would deploy xwiki mysql stateful service with a >> persistent >> >>>>>> volume. The link to the repo:- >> >>>>>> >> >>>>>> https://github.com/xwiki-contrib/xwiki-helm >> >>>>> >> >>>>> Thanks for the update. >> >>>>> >> >>>>> Would be great if you could update the README file to explain who to >> >>> use >> >>>>> it. >> >>>>> >> >>>>> Thanks >> >>>>> -Vincent >> >>>>> >> >>>>> >> >>>>>> >> >>>>>> Thank You >> >>>>>> Ashish >> >>>>>> >> >>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma < >> >>>>> [email protected]> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol < >> [email protected]> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Hi Ashish, >> >>>>>>>> >> >>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma < >> >>> [email protected]> >> >>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>> Hi everyone, >> >>>>>>>>> Thanks for having me here >> >>>>>>>>> >> >>>>>>>>> About Me >> >>>>>>>>> >> >>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of >> >>> Code. I >> >>>>>>>> am >> >>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha >> >>>>>>>> University, >> >>>>>>>>> Delhi. I am a resident of India. >> >>>>>>>>> >> >>>>>>>>> Profiles >> >>>>>>>>> GitHub - https://github.com/ashish932/xwiki-helm-chart/ >> >>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/ >> >>>>>>>>> Riot - @ashish932:matrix.org >> >>>>>>>>> >> >>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all of >> >>> you. >> >>>>>>>>> Following >> >>>>>>>>> are the relevant details. >> >>>>>>>>> >> >>>>>>>>> Helm Chart for XWiki >> >>>>>>>>> >> >>>>>>>>> Mentors: Shubham Jain, Neha Gupta >> >>>>>>>>> >> >>>>>>>>> Technologies: Kubernetes, Docker, other if required >> >>>>>>>>> >> >>>>>>>>> Overview >> >>>>>>>>> The proposed project is a helm chart that would deploy xwiki as >> >>> highly >> >>>>>>>>> available and reliable. It should be configurable with different >> >>>>>>>>> databases(either a standalone database or a clustered one) that >> are >> >>>>>>>>> configurable with xwiki. It would give the option to either >> >>> configure >> >>>>>>>> solr >> >>>>>>>>> externally (standalone or clustered) or managed within the >> >>> container. >> >>>>>>>> >> >>>>>>>> It also needs to be deployed on various Servlet Containers and >> with >> >>> a >> >>>>>>>> LibreOffice server configured. >> >>>>>>>> >> >>>>>>>> In short, it should support all the configurations supported by >> the >> >>>>>>>> official XWiki Docker images :) >> >>>>>>>> >> >>>>>>>>> It >> >>>>>>>>> should deploy the app on a shared file system like a rook. It >> >>> should >> >>>>>>>>> support Istio virtual services, istio matrix, and istio >> distributed >> >>>>>>>> tracing >> >>>>>>>>> and should be a secured system with RBAC and security credential >> >>>>>>>> rotation. >> >>>>>>>> >> >>>>>>>> Ok I have no knowledge about this… The shared file system sounds >> >>> good >> >>>>> to >> >>>>>>>> support clustering XWiki for the permanent directory sharing. >> >>>>>>>> >> >>>>>>>>> The chart should be easily deployed on GKE and amazon EKS. >> >>>>>>>> >> >>>>>>>> On locally on minikube for testing I guess? >> >>>>>>>> >> >>>>>>> >> >>>>>>> Yes we can test using minikube one >> >>>>>>> >> >>>>>>>> >> >>>>>>>>> Features >> >>>>>>>>> >> >>>>>>>>> -> Support for different Databases >> >>>>>>>>> -> Choice between using an external database, a single node DB >> or >> >>> a >> >>>>>>>>> multi-cluster DB setup >> >>>>>>>>> -> Support for shared file system >> >>>>>>>>> -> Support for istio and it's services >> >>>>>>>>> -> RBAC, SSL and other security methods >> >>>>>>>>> >> >>>>>>>>> If you have any features in mind that should be added please >> feel >> >>> free >> >>>>>>>> to >> >>>>>>>>> reply to this mail. >> >>>>>>>>> >> >>>>>>>>> Some Design Questions? >> >>>>>>>>> -> Which Databases should be supported? >> >>>>>>>> >> >>>>>>>> See >> >>>>>>>> >> >>>>> >> >>> >> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy >> >>>>>>>> >> >>>>>>>> More generally see the XWiki Docker image documentation and >> >>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy >> >>>>>>>> >> >>>>>>>>> -> As we have to detach solr out of the docker container(run it >> in >> >>> an >> >>>>>>>>> independent container) would be there a requirement for a code >> >>> change, >> >>>>>>>> and >> >>>>>>>>> we should approach it? >> >>>>>>>> >> >>>>>>>> See the doc for Solr which explains how to configure an external >> >>> SOLR. >> >>>>>>>> There are some XWiki code/config to deploy in the external SOLR >> >>> AFAIR. >> >>>>>>>> comunity controls chart >> >>>>>>>>> -> Apart from solr is there any other stateful service that >> could >> >>> or >> >>>>>>>> should >> >>>>>>>>> be detached from the docker container? >> >>>>>>>> >> >>>>>>>> Which docker container are you referring to? The Servlet >> Container >> >>> one? >> >>>>>>>> >> >>>>>>>> xwiki:lts-mysql-tomcat >> >>>>>>> >> >>>>>>> >> >>>>>>>>> Here is my current repository which deploys XWiki for MySQL >> >>> database >> >>>>>>>> using >> >>>>>>>>> official XWiki docker container:- >> >>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/ >> >>>>>>>> >> >>>>>>>> Will need to have a look. Some additional questions: >> >>>>>>>> >> >>>>>>>> 1) Is there an official place for publishing helm charts, >> similar to >> >>>>>>>> dockerhub for docker images? How do you provide the helm charts >> to >> >>>>> XWiki >> >>>>>>>> users in term of packaging? In other words, what will we/you >> >>> document >> >>>>> on >> >>>>>>>> >> >>>>> >> >>> >> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/ >> >>>>>>>> ? >> >>>>>>>> >> >>>>>>>> Yes, There is a centralized community control chart management >> >>>>>>> repository. Helm charts are hosted there. Organizations like >> mysql, >> >>>>>>> wordpress uses it. >> >>>>>>> https://github.com/helm/charts >> >>>>>>> But in addition to that, we can build our own charts repository >> >>> locally, >> >>>>>>> and host it. Similar to docker run command, we have helm update >> >>> command >> >>>>>>> >> >>>>>>> >> >>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm >> charts, as >> >>>>> part >> >>>>>>>> of the build, to verify that they work fine? How can we execute >> >>> XWik’s >> >>>>>>>> functional tests based on TestContainers using Helm Charts? >> >>>>>>>> >> >>>>>>> >> >>>>>>> There are some tools available >> >>>>>>> chart-testing(started by helm) -98 commits >> >>>>>>> terratest - 1039 commits (It looks a good tool to use) >> >>>>>>> >> >>>>>>> >> >>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it >> be >> >>> done >> >>>>>>>> with Maven? Since we’ll be offering several I guess, how do you >> >>> avoid >> >>>>> code >> >>>>>>>> duplication and make maintenance of them as simple as possible? >> >>>>>>>> >> >>>>>>> >> >>>>>>> No, the charts don’t need to be built externally, though it gets >> >>>>>>> compressed when pushed f we build to chart repository(this is an >> >>>>> automatic >> >>>>>>> step in case of official chart repo and we can automate using ci >> our >> >>> own >> >>>>>>> chart repo) >> >>>>>>> >> >>>>>>> 4) Could you provide a roadmap with various milestones and dates? >> See >> >>>>> the >> >>>>>>>> last roadmap email I sent on the devs list and please reply to >> it. >> >>>>>>>> >> >>>>>>> >> >>>>>>> Okay I would work on that and update the progress on the mail >> thread >> >>>>>>> >> >>>>>>> >> >>>>>>>> 5) It seems you missed the bonding period so you’ll need to catch >> >>> up on >> >>>>>>>> that. Could you pick some jira issue and work on them quickly? >> This >> >>> is >> >>>>> to >> >>>>>>>> make sure you use the proper xwiki communication channels, learn >> to >> >>>>> work >> >>>>>>>> with the xwiki team and start understanding the xwiki >> architecture, >> >>>>> and how >> >>>>>>>> extensions are developed for XWiki. Since you’re a bit late, >> you’ll >> >>>>> need to >> >>>>>>>> catch up and achieve this very quickly now and be present full >> time >> >>> on >> >>>>> the >> >>>>>>>> xwiki chat :) >> >>>>>>>> >> >>>>>>> I would start doing that >> >>>>>>> >> >>>>>>>> >> >>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s >> >>> jira. >> >>>>>>>> Maybe you can help with those too as part of onboarding/bonding. >> It >> >>>>> should >> >>>>>>>> also give you additional use cases for the XWiki Helm Charts IMO. >> >>>>>>>> >> >>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts >> in >> >>>>> action. >> >>>>>>>> >> >>>>>>>> -Vincent >> >>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Thank You >> >>>>>>>>> Ashish Sharma >> >>>>>>>> >> >>>>>>>> Thank You >> >>>>>>> Ashish >> >>>>>>> >> >>>>> >> >>>>> >> >>> >> >>> >> >>
