Thanks for that Sebastian. I really appreciate you taking the time to get back to me on this.
Sent from my iPhone > On Sep 1, 2021, at 3:32 AM, "seba.wag...@gmail.com" <seba.wag...@gmail.com> > wrote: > > Hi Denis, > > I have been running tests with a single KMS instance doing 4x4 audio video > sessions. > The test results are here: > https://cwiki.apache.org/confluence/display/OPENMEETINGS/Performance+Testing > > Those are all based on a single KMS instance. I don't know where you have > your numbers and there are logical limits, but it is by far not as how you > suggest. > > The reason why BBB introduced 3 different KMS servers (optional feature > afaik) is to _seperate_ audio, video and screensharing stream and have a > dedicated KMS instance for each one. So that if there are bandwidth issues on > e.g. the screensharing stream (cause it generally is for example more > bandwidth intensive) it will not affect the audio stream. > However this is more a theory from my point of view. There is no evidence > that that would suggest splitting streams like this would actually use the > available bandwidth and hardware better. Especially if the 3 KMS would be > installed on the same physical hardware!! > > So in short, the 3 KMS servers they have is NOT intended for scaling, its for > separating audio, video and screensharing in separated streams. > > When it comes to scaling the thing that help is (A) performance optimising > your application & configuration (B) upgrade the hardware [aka scale > vertically] (C) adding additional hardware [aka scale horizontally] > > And it makes sense to look at those things in this order. Jumping to (C) > while (A) and (B) are not optimised is expensive and frustrating. > > But once (A) and (B) are exhausted, the only thing is scaling horizontally. > Meaning: Adding additional servers. > > Translated into OpenMeetings (C) has two pathways: > Either by having more OpenMeetings instances (each instance with both > OpenMeetings/Tomcat _And_ Kurento/KMS on the same server) (like OpenMeetings > clustering) or by having some advanced clustering features that enable > scaling of Kurento/KMS instances independently from the OpenMeetings/Tomcat > Server on separated hardware (like https://om-hosting.com 's Private Elastic > Cloud ). > One thing to keep in mind is that OpenMeetings/Tomcat and Kurento/KMS scale > quite differently. OpenMeetings/Tomcat consumes memory, but at a relatively > linear profile. Kurento/KMS consumes CPU. And it is very hungry :) So putting > OpenMeetings and KMS on the same hardware, especially also on the same > hardware _type_ (eg CPU type) is probably not helpful, it works fine in a > scenario with 100 participants. But if you need to scale to 1000+ the > difference in hardware usage makes separating and scaling it independently > quite a lot cheaper and more effective. > > As for your crash scenarios the most likely cause is that you are probably > still not using either still having enough things to optimise in (A) and (B). > Eg you havn't performance optimised the settings (eg memory settings > allocated, uLimits et cetera). > > But Performance Optimisation is not something where a quick diagnosis or a > few steps will help. Especially since without an actual performance test of > your new settings or setup all those things you do are not validated. So you > wouldn't know if it has any effect or not. > > Hope that makes sense, > > Thanks > Sebastian > > Sebastian Wagner > Director Arrakeen Solutions, OM-Hosting.com > http://arrakeen-solutions.co.nz/ > https://om-hosting.com - Cloud & Server Hosting for HTML5 Video-Conferencing > OpenMeetings > > > >> On Wed, 1 Sept 2021 at 18:41, Denis Noctor <denisnoc...@gmail.com> wrote: >> Hi there Maxim, >> >> Thanks for getting back to me. I've already sent an email to Sebastian. >> >> The clustering approach "looks" relatively straight forward as per the OM >> site recommendations (and is something I am going to look at in more detail >> over the weekend).... but I am REALLY interested in trying to implement the >> three different types of media streams: listen only, webcams, and screen >> share over 3 KMS servers, as suggested in BBB (which as you may know uses >> Kurento / WebRTC). Apologies for bringing up BBB but it appears to be the >> only alternative that uses similar technologies that has addressed this >> issue. >> >> KMS limitations have been talked about over the last year or so - for >> example... From what I recall, a standard OM / KMS setup can deal with >> around 300 media server connections. Therefore a group of 5 people sharing >> audio and video would in theory use 125 of these connections. (5 audio x 5 >> video x 5 people) but that will grow exponentially if you have 7 users >> (sharing audio and video... (7x7x7= 343). You might recall last year that I >> reported on a physical test on one of the OM servers and I got about 9 users >> (audio and video) working in the same room before things got a little shaky >> - but let's be honest - having 9 or 10 users sharing audio and video at the >> same time can be a little messy given the fact that their video pods might >> unnecessarily take up a lot of "landscape" on the screen also... but I would >> feel more relaxed knowing that if I have 7 - 9 students in one room and a >> teacher teaching 7 - 9 students in another room that the server isn't going >> to go down... irrespective of how good an AWS instance I set up. >> >> Last year, there was a time last year when I had 3 classes running >> concurrently and unfortunately everything crashed. The one time only. Memory >> usage spiked... but it all boiled down to KMS, it seems. >> >> This led me to start looking into everything in more detail, for obvious >> reasons. After all the time and effort I've spent in experimenting and >> giving some casual feedback regarding OM, I think simply moving to a >> commercially OM hosted environment would defeat the purpose of the learning >> curve I have been on (apologies Sebastian, I mean no disrespect). >> >> Anyway, if there is anyone out there that can give me any pointers on how to >> spread KMS activity over 3 servers (over a single OM instance), I'd >> appreciate it, paid or otherwise, let me know. This has been a great >> experience and I'd like to continue my learning curve on this as much as >> possible. I think this would open a gateway to a lot more conversation and >> discussion.... a lot more ideas and a solid way forward with users and for >> OM. Over the years I've seen users achieve what they need to achieve and >> then disappear from the User List once they've accomplished (or not). >> Anyway, I digress. >> >> For reference the following links are the main ones I've come across that >> have sparked this interest: >> >> https://docs.bigbluebutton.org/admin/customize.html#run-three-parallel-kurento-media-servers >> >> and >> https://github.com/bigbluebutton/bigbluebutton/blob/master/bigbluebutton-config/bin/apply-lib.sh >> >> Any suggestions from anybody would be appreciated. >> >> I look forward to ANY suggestions on this matter. Please drop me a line >> either via the forum or directly. >> >> All the best. >> >> Denis. >> >>> On Tue, Jul 27, 2021 at 11:16 AM Maxim Solodovnik <solomax...@gmail.com> >>> wrote: >>> Hello Denis, >>> >>> On Tue, 27 Jul 2021 at 12:26, Denis Noctor <denisnoc...@gmail.com> wrote: >>> > >>> > Hi there Maxim, >>> > >>> > I hope this email finds you well. >>> >>> Everything is great :) >>> Hope you are well and healthy :) >>> >>> > >>> > I’ve been using OM over the last few years as mentioned in previous >>> > posts. Apologies for not commenting on previous updates and upgrades of >>> > recent. >>> > >>> > I’ve been using OM 5.1.0 for some time now (on an AWS EC2 instance ) and >>> > is running well as expected but now I need to explore the issue of >>> > clustering due to KMS limitations, number of concurrent users etc per >>> > room (and instance) as has been previously documented in previous posts. >>> > I’ve been avoiding the issue as some of the classes I am running are >>> > scheduled at different times... but now I am faced with the issue of a >>> > handful of classes that run at the same time. >>> >>> Hopefully you can try the latest release :) >>> it should be much more stable :) >>> >>> And the UI was improved a lot :) >>> >>> > >>> > Is there anyone out there who has successfully got clustering working in >>> > an AWS environment (using Ubuntu 18.04)? >>> > >>> > I would really appreciate it if anyone could give me some pointers. At >>> > present I have my AWS instance set up using Amazon’s Route 53, AWS >>> > certificate for https etc. If someone out there could give suggestions >>> > regarding setting up at least 2 instances using the same domain name, I’d >>> > be grateful and I could experiment from there. >>> > >>> > I’ve included Ali in this email (hope you don’t mind... as I’ve been >>> > following your progress and contributions to this forum since this whole >>> > pandemic began). >>> >>> @Seba was able to implement "real" clustering (so you can add more >>> users to the room >>> AFAIK this is commercial feature: >>> https://openmeetings.apache.org/commercial-support.html#cloud-based-hosting-for-openmeetings >>> >>> > >>> > All the best and thanks in advance. Denis. >>> > >>> > Sent from my iPhone >>> >>> >>> >>> -- >>> Best regards, >>> Maxim
