Hi Wei, Sorry I'm not sure if we're keeping 512 or not, referring to Lucian's point ""but the slightest memory leak or a temporarily spiky conntrack table could derail it." - for this reason it's better to keep 512 even if 384 may be good enough.
Regards. ________________________________ From: Wei ZHOU <ustcweiz...@gmail.com> Sent: Monday, February 12, 2024 13:00 To: Rohit Yadav <rohit.ya...@shapeblue.com> Cc: dev <dev@cloudstack.apache.org> Subject: Re: Discussion: CloudStack upgrade to JRE17 and Debian12/python3 Hi Rohit, Thanks. I will revert the changes then. If the users think 512MB will lead to large memory consumption, they can - Create a System Offering - Update global setting router.service.offering to the uuid of the new offering. -Wei On Fri, 9 Feb 2024 at 16:15, Rohit Yadav <rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>> wrote: I would advise we should move to 512 MiB. Regards. ________________________________ From: Wei ZHOU <ustcweiz...@gmail.com<mailto:ustcweiz...@gmail.com>> Sent: Friday, February 9, 2024 4:14:06 PM To: Rohit Yadav <rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>>; dev <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Subject: Re: Discussion: CloudStack upgrade to JRE17 and Debian12/python3 Hi Rohit, I tested the VRs of a VPC (with 2 tiers, LB, PF, etc) with different memory sizes. 384MB memory looks ok. (1) 384 MB root@r-601-VM:~# free -m total used free shared buff/cache available Mem: 331 222 39 0 144 109 Swap: 242 0 242 (2) 320MB root@r-603-VM:~# free -m total used free shared buff/cache available Mem: 267 214 45 0 80 53 Swap: 242 0 242 (3) 512MB root@r-604-VM:~# free -m total used free shared buff/cache available Mem: 457 214 63 0 256 242 Swap: 242 0 242 On Thu, 8 Feb 2024 at 15:15, Rohit Yadav <rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>> wrote: Thanks Wei. I think VR should be set to 512 (if memory consuming is above 300MB). SSVM and CPVM aren’t they already at 512-1024 MiB? Regards. ________________________________ From: Nux <n...@li.nux.ro<mailto:n...@li.nux.ro>> Sent: Thursday, February 8, 2024 2:50:11 PM To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Cc: Wei ZHOU <ustcweiz...@gmail.com<mailto:ustcweiz...@gmail.com>> Subject: Re: Discussion: CloudStack upgrade to JRE17 and Debian12/python3 +1 option 2 (Debian12). I think we need to do a bit longer term testing before we decide on the new RAM size for the VR. 384MB might be good to boot, but the slightest memory leak or a temporarily spiky conntrack table could derail it. ..But yeah, need to be conscious of the overall memory overhead in large clouds. On 2024-02-08 07:44, Wei ZHOU wrote: > Hi Rohit, > > Thanks for your reply. > > Indeed the memory consumption could be an issue, especially for the > users > who have thousands of virtual routers. > > I have tested the debian12 systemvm template several times. Every time > I > get an error "kernel panic" if the memory is 256MiB (the current > memory > size of the default system offering for virtual routers). > I have tested 320MiB, 384MiB, and 512 MiB. The VRs seem good. For > stability, I have updated the PR to change the memory size of the > default > offering of virtual routers from 256 MiB to 384 MiB. > > The current default memory size of system vms (CPVM, SSVM) is 512 MiB, > so > they will not be impacted. > Hope it has less impact on users. > > > -Wei > > > On Thu, 8 Feb 2024 at 08:30, Rohit Yadav > <rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>> > wrote: > >> Hi Wei, all, >> >> Thanks for the thread, I've no objections to either of the options, >> but >> option2 is preferred as Debian 11 security will EOL mid of this year. >> So, >> if not in 4.20, eventually we'll need to migrate systemvmtemplate base >> OS >> to Debian 12 at some point in future. >> >> The bigger implication for users will be doubling of their memory >> consumption in their env for VRs. >> >> >> Regards. >> >> >> >> >> ________________________________ >> From: Wei ZHOU <ustcweiz...@gmail.com<mailto:ustcweiz...@gmail.com>> >> Sent: Tuesday, February 6, 2024 17:33 >> To: dev <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>; users >> <us...@cloudstack.apache.org<mailto:us...@cloudstack.apache.org>> >> Subject: Discussion: CloudStack upgrade to JRE17 and Debian12/python3 >> >> Hi all, >> >> My colleagues and I are working on upgrading CloudStack to support the >> more >> recent JRE version, python version and Debian version for the systemvm >> template. >> We already have made some changes, if you are interested, please >> review the >> pull request: https://github.com/apache/cloudstack/pull/8497 >> >> Here are what we want to achieve in CloudStack 4.20 >> >> *1. Upgrade CloudStack to run with JRE17.* >> >> Currently we use JRE11 to build the CloudStack packages starting in >> 2020. >> CloudStack mgmt servers/usage/kvm agents run with JRE11 as well. >> However, >> JRE11 is EOL in September 2023 [1]. >> In CloudStack 4.20, we will build CloudStack using JRE11 (because old >> 4.17/4.18/4/19 systemvm template do not have JRE17 installed) , but >> enforce >> user to install JRE17 on mgmt server and kvm hosts when upgrade to >> CloudStack 4.20 >> It requires a lot of changes to build CloudStack using JRE17, so it >> will be >> mostly like done in CloudStack 4.21 or later. >> >> >> *2. Upgrade CloudStack VR to use python3* >> >> python2 is currently used in CloudStack VR, which is already EOL in >> 2020 >> [2]. We must migrate to python3 as soon as possible. >> All python scripts used in CloudStack VR will be migrated to python3. >> If you use a Debian11 systemvm template (4.17/4.18/4.19), you need to >> recreate or patch the System VMs and Virtual routers after upgrading >> to >> CloudStack 4.20. >> If you choose to patch a System VM and Virtual router, two packages >> will be >> installed: python-is-python3 and python3-netaddr >> >> >> *3. Upgrade CloudStack VR to Debian12* >> >> The more recent operating system provides more features and security. >> This >> is not urgent as Debian11 will be supported until 2026 [3]. >> We have tested the Debian12 systemvm templates, and found only two >> issues >> - OpenSSL has been upgraded from 1.1.0 to 3.0 in Debian12. Some >> algorithms are deprecated. We have to set "@SECLEVEL=0" in apache2 >> config >> and "PubkeyAcceptedAlgorithms=+ssh-rsa" to sshd config to support some >> old >> SSH keys and certificates. >> - The current default memory size (256MB) of virtual routers is not >> big >> enough. The Debian document says 780MB memory is required [4]. We have >> tested that 512MB/384MB memory is enough. It also works if memory is >> 320MB. >> But with 256MB, we got "kernel panic" when system VMs/VRs start. >> >> The memory upgrade should not be a problem for most users. If users >> have >> thousands of virtual routers, they might need to add more memory. >> The new debian12 template might have an impact on CKS (cloudstack >> kubernetes service). Until now, we have not found any issue in our >> testing >> with multiple hypervisors (ubuntu22/20, rocky8/alma8/ol8, alma9/ol9, >> xenserver-71, vmware 67u3/70u3/80) >> >> >> What's your opinion on the following two options ? >> >> - Option 1: Upgrade to JRE17 and python3 (still use Debian11) >> - Option 2: Upgrade to JRE17 and python3 and Debian12 >> >> Thank you ! >> >> >> >> [1] >> https://www.oracle.com/be/java/technologies/java-se-support-roadmap.html >> [2] https://www.python.org/doc/sunset-python-2/ >> [3] https://wiki.debian.org/LTS >> [4] https://www.debian.org/releases/bookworm/amd64/ch02s05.en.html >>
