Hi, Today, I had a meeting with Michaël and had discussions about work items from the last week which basically includes: - Connect local-sugarizer-chart with SSP. - Add feature to add create an admin in Sugarizer-Server deployment through SSP. - Look into implementing the DNS feature in SSP.
Problems experienced last week: Faced issue while using the cluster with Cloud DNS Admin Service Account credential. However, we were able to fix this issue. Here is a list of tasks that we have decided for this week: - Do research on how to implement HTTPS on the cluster. - Deploy the SSP into Kubernetes PFA slack logs for reference. Next meeting is scheduled for next Monday. Regards, Nikhil
Michaël Ohayon 3:30 PM Hello Nikhil :wave: ! Nikhil 3:30 PM Hi @Michaël Ohayon Michaël Ohayon 3:30 PM Everything’s fine on your side? :smile: Nikhil 3:30 PM Yeah 3:30 What about you? Michaël Ohayon 3:30 PM Fine, no issue nor so much news. (edited) Nikhil 3:31 PM About the last week tasks: - Connect local-sugarizer-chart with SSP. - Add feature to add create an admin in Sugarizer-Server deployment through SSP. - Look into implementing the DNS feature in SSP. 3:32 The Sugarizer-Chart is now connected with Sugarizer-School-Portal 3:32 You can install Sugarizer-Server chart by clicking on the Deploy button 3:33 You mentioned about some typo while testing SSP on GKE. Where was the typo exactly? Michaël Ohayon 3:33 PM Yes, It’s working great congrats, it’s some nice work done here :slightly_smiling_face: 3:34 Minor issue: On the chart repo there is a typo in the folder name 3:34 enviornment or something like that 3:34 So I guessed my conf was bad but the folder name was :upside_down_face: 3:35 I’ve encountered an issue with the releases tab, the last updated indicates Last Updated 3:35 Except for the last one I deployed 3:35 The others releases were done already on the GKE instance so that may have a link 3:37 I’ve guessed I’ve made a mistake in translation also 3:37 Because Releases and Deployments have the same name in menu :smile: 3:37 So I’ll need to fix that Nikhil 3:37 PM I’ve encountered an issue with the releases tab, the last updated indicates Last Updated Actually I have not added translations for the Releases view Michaël Ohayon 3:37 PM Oh so that’s the same key maybe Nikhil 3:37 PM I was not sure we needed Releases view in the final product 3:37 Do we need that? Michaël Ohayon 3:38 PM Not sure Nikhil 3:38 PM Since it just shows all the installed Helm releases Michaël Ohayon 3:38 PM But that’s helpful since there was already some helm releases in place 3:38 And that allows me to kick them out 3:38 If needed 3:38 So maybe keep that under a more advanced menu or something like that Nikhil 3:38 PM Yeah... Then I'll add translations and improvements to that view too. Michaël Ohayon 3:39 PM Or maybe we could merge all those datas in one view 3:39 I dunno Nikhil 3:40 PM Yeah... For now maybe we can keep the releases view, when some UI/UX designer/contributor suggests some good way to display it, we can switch to more presentable view. Michaël Ohayon 3:41 PM Ok that’s not really a big issue since we won’t have multiple ssp instances connected to the same k8s cluster Nikhil 3:43 PM Add feature to add create an admin in Sugarizer-Server deployment through SSP. The feature to create Admin account has been added. However since it does not uses service to connect with the deployment, instead it runs add_admin.sh script to create the admin, we can't show the status of the request -- That was it success or not. We can just run add_admin.sh script Michaël Ohayon 3:43 PM I’ve tested it, that is nice :ok_hand: It worked great (edited) Nikhil 3:43 PM Okay Michaël Ohayon 3:44 PM I did not tried to run it without waiting for the instance to exist and with an user already existing yet 3:44 But for the basic workflow it worked Nikhil 3:46 PM Yeah. It will just run the script and give output Deployment Updated. It cannot check for existing user currently. The reason was that we use a curl request in add_admin.sh script. It does not gives the correct response in stdout of Node Child Process because of the delay between request and response. Michaël Ohayon 3:47 PM Yes that’s sufficent I think 3:47 And that’s because of the script limitations so that is understandable Nikhil 3:48 PM Look into implementing the DNS feature in SSP. Now the SSP creates a DNS entry for each deployment with the school-short-name as the subdomain. 3:49 We used external-dns to implement the feature to create a DNS entry. https://github.com/kubernetes-sigs/external-dns 3:50 Also, this repository https://github.com/NikhilM98/sugarizer-chart contains the Helm Chart for Sugarizer-Server. Michaël Ohayon 3:50 PM Yes, that’s working too :ok_hand: 3:50 The only issue is the dns propagation time Nikhil 3:51 PM Yeah. It's roughly 2 minutes. Michaël Ohayon 3:51 PM Oh nice, I would have thought more Nikhil 3:52 PM External DNS runs a check every 1 minute, that's why it's roughly 2 minutes. You can see the logs of the External-DNS pod to monitor DNS change. Michaël Ohayon 3:53 PM For the propagation I meant the time for the end user to resolve correctly 3:53 On Google’s side it’s quite fast 3:54 I have a question though. Is every deployemnt exposing it’s own service ip ? 3:54 I guessed that Nginx would be a core http redirecter and only that would be exposed outside 3:54 I’m not sure about where traffic goes 3:55 Is there a front nginx ? :thinking_face: Nikhil 3:58 PM I'm not sure. This is the service config I'm using: https://github.com/NikhilM98/sugarizer-chart/blob/master/gke-enviornment/sugarizer-chart/templates/service.yaml I used this config to set-up external-dns external_dns_gke.yaml Michaël Ohayon 3:58 PM Mm I will need to take a deeper look after the meeting 3:59 My idea would be to use nginx ingress to spread http dispatch across all nodes 3:59 each node would then listen on :80 and :443 3:59 gcloud dns would bind nodes to the internet 4:00 then nginx would re-dispatch using host to the correct nodes 4:00 so there won’t be any delay for deploy 4:00 The other idea is about the https certificates 4:01 I suggest having it on a upper level 4:01 But that means we need some nginx-ish on front Nikhil 4:01 PM Oh... Michaël Ohayon 4:01 PM But it’s still a WIP in my head so I need to think about it :thinking_face: 4:02 Anyway what’s done now is great and it’s working Nikhil 4:02 PM Instead of Nginx Ingress Controller, current implementation used Google's Default Ingress Controller. Maybe using Nginx Ingress Controller may show some significant improvements. Michaël Ohayon 4:02 PM We will need to add that https thing at some point yes Nikhil 4:02 PM Yeah Michaël Ohayon 4:02 PM And I would better like to have it cloud agnostic ready :smile: 4:02 But maybe it would work as-it on Azure aswell 4:03 With only a few tweaks Nikhil 4:04 PM Having it cloud agnostic sounds tricky :sweat_smile: Michaël Ohayon 4:04 PM If dns entry is only about targeting main ip of the cluster 4:04 we could also do it manually Nikhil 4:04 PM Yeah. Right Michaël Ohayon 4:04 PM Nginx afterwards would do traffic splitting without headache I guess Nikhil 4:05 PM Yeah Michaël Ohayon 4:07 PM It’s all for my thought I guess 4:07 Did you had any particular issue that may slow you down? :thinking_face: Nikhil 4:08 PM So you're thinking about adding a Nginx Load Balancer and applying HTTPS certificates on top of it. Right? (edited) Michaël Ohayon 4:08 PM Yes I guess :smile: 4:08 Or any other load traffic tool 4:08 like Haproxy or other 4:08 But Nginx seems nicely supported 4:08 In my experiments I had success with nginx + tags for hostname Nikhil 4:09 PM Oh nice 4:10 So what are the things remaining in this project? Do we need more features in Sugarizer-School-Portal-Server? I think we can add more unit tests to the app, make some minor improvements. What else do we need? Michaël Ohayon 4:10 PM Let’s break down the HTTPS maybe 4:11 because that may have some impacts with the websocket 4:11 And then I suggest we have a test with Lionel and gather feedbacks Nikhil 4:11 PM Okay Michaël Ohayon 4:11 PM Two things that would be nice also : chart to deploy ssp on kube maybe 4:11 or some ansible things 4:11 To automate what we achieved 4:12 because there are more and more scripts to run and things to install :smile: 4:12 I think my nginx thing may be there https://github.com/mikklfr/sugarizer-chart https://github.com/mikklfr/sugarizer-chart/blob/master/templates/sugarizer-so-expose.yaml Nikhil 4:15 PM Okay. But this won't add DNS entry automatically. So we have to depend upon manually adding DNS entries? Michaël Ohayon 4:15 PM For the wildcard I guess 4:15 But we could use your work based on clouddns to have something 4:15 like nginx ingress would be able to create the entry 4:16 host: {{required “hostName not defined” .Values.hostName}} 4:16 Nginx ingress will automatically balance to the right container based on the host spec 4:16 But yes we need to first have some bind between dns and the nginx Nikhil 4:17 PM Do you have any documentation which shows how this can be done? Michaël Ohayon 4:19 PM One the former link clouddns is used to create the https certs 4:19 but the dns entry was just pointing to the nginx ingress service ip 4:19 manually 4:20 https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/dnsimple.md 4:20 I guess adding annotation external-dns.alpha.kubernetes.io/hostname: validate-external-dns.example.com 4:20 to the nginx ingress may be sufficent 4:21 Or maybe some dummy container especially for that purpose 4:22 Maybe we can discuss it more outside of the meeting :thinking_face: 4:22 Because there are some researches to do Nikhil 4:23 PM Alright! 4:24 Do you think that if we use Google's Ingress Controller as it is then we would have load balancing issues? (edited) Michaël Ohayon 4:24 PM I guess it works fine 4:24 But we would still need that HTTPS Nikhil 4:25 PM So if we find some way to use HTTPS over Google's Ingress Controller then it would be easier Michaël Ohayon 4:25 PM So we would in the end go back to some nginx, embedded in the deployment itself or using a cluster wide way 4:25 Yes but that may be good only if this is cloud agnostic :upside_down_face: 4:26 If having the same conf works on Azure I would say go Nikhil 4:26 PM Okay Michaël Ohayon 4:26 PM I let you have a look since I have no idea at what they suggest 4:26 But if their kube implementation makes it easy for us then go for sure :slightly_smiling_face: Nikhil 4:27 PM Yeah. Since so many people use GKE, maybe they'll have some simpler way to implement HTTPS. Not sure (edited) Michaël Ohayon 4:28 PM I guess they have, atleast I hope :smile: 4:28 But I guess they charge more for that too 4:28 But I’m curious about their proposal Nikhil 4:28 PM Oh. Right 4:29 Do they charge extra for DNS entries? Michaël Ohayon 4:29 PM .40$ per million request 4:29 + .20$ per dns zone 4:30 That’s not much :smile: 4:30 I guess external endpoint cost Nikhil 4:30 PM Yeah. We need only one Zone per subdomain Michaël Ohayon 4:30 PM Oh Google updated their conditions 4:31 .10$ per cluster per hour for GKE 4:31 It was free before 4:31 72$ per month increase :face_with_raised_eyebrow: 4:32 On Azure it’s free 4:32 (the management) 4:32 the nodes are always to pay Nikhil 4:32 PM Is this additional charges for only having GKE? Extra charges for the Nodes Michaël Ohayon 4:32 PM yes 4:32 Nodes are priced as compute instances 4:33 before the cluster management was free 4:33 but as it requires some other machines on top I guess they want their money Nikhil 4:33 PM 72$ for just using Kubernetes sounds expensive Michaël Ohayon 4:33 PM On AWS it’s almost the same issue Nikhil 4:34 PM Oh 4:34 So let's discuss the tasks for this week 4:35 Do research on how to implement HTTPS on the cluster. 4:35 What else should we do? Write unit tests for the SSP? Michaël Ohayon 4:36 PM You may but let’s focus on the essentials part 4:36 Like check that helm is correctly trigger from some request 4:37 Let’s forget about the display of informations for now as it will vary and does not bring that much safety 4:37 One cool thing would be to be able to deploy the ssp onto kube 4:37 manually using helm 4:38 So it would be very convenient to host it. 4:38 That will bring some issues like how to pass kubeconfig etc 4:38 But in the end we will need that :thinking_face: Nikhil 4:38 PM Like check that helm is correctly trigger from some request What do you mean by this? I did't understand this point clearly. 4:39 Deploy the ssp onto kube Yeah, this sounds good. We can work on this. Michaël Ohayon 4:39 PM I guess the key parts for our test would be : check if only admin can access / requests admin parts Ensure that triggering a deploy from admin with some informations actually triggers helm with the correct informations 4:39 But I would not go onto time consuming tests for UI and stuff right now Nikhil 4:40 PM Yeah. Right. It's more clear now. I was also thinking the same 4:41 Adding tests for only the API part which includes -> Role based tests, tests for deployment using helm, and other tests (edited) Michaël Ohayon 4:43 PM Perfect 4:43 If we have time it would be great to have a look at Azure Nikhil 4:43 PM So, tasks for this week are: Do research on how to implement HTTPS on the cluster. Deploy the SSP into Kubernetes Michaël Ohayon 4:44 PM I will see with Lionel how to get credentials and stuff like that 4:44 Because with this princing maybe the public free instance would go on Azure 4:44 It’s almost 6months nodes cost Nikhil 4:45 PM Yeah Michaël Ohayon 4:48 PM That’s good for me :slightly_smiling_face: Nikhil 4:48 PM About the Deploy the SSP into Kubernetes part. For this we'll have to move SSP into a Docker Container. This task may take more time than expected since I'm not sure how to configure Kubectl and Helm to access GKE cluster. Michaël Ohayon 4:48 PM Yes, it should only be a matter of giving the container the good kubectl file (edited) 4:48 “In theory” 4:49 So there may be surprise Nikhil 4:49 PM https://github.com/dunefro/helm-web-api 4:50 They somehow managed to access it from inside the container Michaël Ohayon 4:50 PM how yes using rbac and sutff Nikhil 4:50 PM This is the Dockerfile: https://github.com/dunefro/helm-web-api/blob/master/Dockerfile Michaël Ohayon 4:50 PM maybe it just targets the default kube ip 4:50 by default 4:50 and since it has inherited rights it works Nikhil 4:50 PM That's what I was thinking Michaël Ohayon 4:51 PM That’s a better way actually 4:51 Because the secret is not accesible 4:51 If that works that would be better :smile: 4:51 But I don’t mind going the other way if we get stuck in a loop with RBAC issues 4:51 Let’s try that first and see 4:52 If rbac and stuff is too hard we’ll go the easy way then back at it in the end Nikhil 4:52 PM Do you think that can the modified version of this Dockerfile to allow SSP access Kubectl and Helm from inside the cluster? Michaël Ohayon 4:53 PM I dont see anything on the Dockerfile 4:53 Regarding that rights things Nikhil 4:54 PM Exactly, does that means it inherits the configs? Michaël Ohayon 4:54 PM I guess 4:54 I looking for the IP or dns name they are calling 4:56 https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ (edited) kubernetes.io Configure Service Accounts for Pods(25 kB) https://kubernetes.io/images/favicon.png 4:57 Seems really related to what’s going on Nikhil 4:57 PM Oh Michaël Ohayon 4:59 PM I will have a look on my side aswell :thinking_face: Nikhil 5:00 PM Alright. Thanks :smile: 5:01 I think they created some service account using this config: https://github.com/dunefro/helm-web-api/blob/master/chart/on-demand-micro-services-deployment/templates/helm-service-account.yaml Michaël Ohayon 5:01 PM Nice catch Nikhil 5:02 PM Okay, so now I have basic idea what we need to do to move SSP inside a pod. Michaël Ohayon 5:03 PM :smile: Nikhil 5:04 PM So we have two tasks: Do research on how to implement HTTPS on the cluster. Deploy the SSP into Kubernetes I think the Second task might take more time than expected but I'll try to finish it by the end of the week. 5:06 Problem faced last week: Faced issue while using cluster with Cloud DNS Admin Service Account credential. However, we were able to fix this issue. :+1: 1 5:07 Should we end this meeting? Michaël Ohayon 5:07 PM Yes ! 5:07 Thanks for you time Nikhil 5:08 PM Thanks Michaël Ohayon 5:08 PM Do not hesitate if you’re stuck at some point Nikhil 5:08 PM Okay!
_______________________________________________ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
