Thanks Daniel, My point is that I'm not sure that the VMware jars _need_ to be noredist, which would simplify things for everyone.
Kind Regards Paul Angus -----Original Message----- From: Daniel Augusto Veronezi Salvador <dvsalvador...@gmail.com> Sent: Wednesday, October 20, 2021 6:20 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Paul, Noredist packages are only necessary to VMWare. KVM/XenServer infrastructures don't use noredist packages. Best regards, Daniel Salvador On 20/10/2021 12:29, Paul Angus wrote: > I vaguely thought that the VMware jars come under a compatible license these > days, so don't need to be noredist anymore? > > > -----Original Message----- > From: Daniel Augusto Veronezi Salvador <dvsalvador...@gmail.com> > Sent: Wednesday, October 20, 2021 2:53 PM > To: dev@cloudstack.apache.org > Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 > > Hi Nicolas, thanks for the work. > > I started testing this RC and unfortunately I ran into some problems related > to building/upgrading ACS. > > From the less to the more complex: > > - When building ".deb", there is a problem with the Debian changelog: > invalid weekday 'lun'; > > - Regarding auto upgrading "systemvms-templates", I analyzed the code and > concluded that "systemvms-templates" will only "autoupgrade" if we use the > profile "noredist" when building the packages, which is necessary only to > VMWare. Some people, might not want the noredist JARs, and therefore will not > build with this profile. With that said, there are some points regarding it > that I think are important: > - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, > when using "noredist" while executing the 4.16.0.0 build. It happens due to > "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, > and VMware "systemvms-templates". Every time that we build the packages, it > will download all these systemvms-templates; we are unable to decide which > one is necessary or not. If we do not use XenServer, or other hypervisors, > why download it? > - If we build the packages with "noredist", when we are first running > ACS after upgrading it to 4.16, it will try to autoupgrade the > systemvms-templates, however, some KVM/XenServer infrastructures don't allow > the management server to access the secondary storage directly (except for > the first systemvm-template seed). This will cause > "mount.nfs: access denied by server while mounting <path_to_secondary>". > After generating this error, ACS continues the upgrade and updates the > database; however, the new systemvm-template will not be in the secondary > storage. If we try to deploy a system (like virtual router), it will fail (as > expected). > - I rolled back my environment to 4.15, did a manual seed of > systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" > and tried to upgrade the environment to 4.16. Then, ACS automatically tried > to upgrade the systemvm-templates; even though the system VM template for > 4.16 is already there, it throws some errors: > > 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) > (logid:) Updating System Vm template IDs > 2021-10-19 23:08:02,904 DEBUG [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Updating System Vm template IDs > 2021-10-19 23:08:02,910 DEBUG [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Updating VMware System Vms > 2021-10-19 23:08:02,915 WARN [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) 4.16.0 VMware SystemVm template not found. Cannot > upgrade system Vms hypervisor is not used, so not failing upgrade > 2021-10-19 23:08:02,962 DEBUG [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Updating KVM System Vms > 2021-10-19 23:08:02,966 INFO [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Grabbing lock to register templates. > 2021-10-19 23:08:02,976 ERROR [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Failed to calculate template checksum > java.nio.file.NoSuchFileException: > /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.16.0-kvm.qcow2.bz2 > at > java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) > at > java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) > at java.base/java.nio.file.Files.newByteChannel(Files.java:371) > at java.base/java.nio.file.Files.newByteChannel(Files.java:422) > at > java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) > at java.base/java.nio.file.Files.newInputStream(Files.java:156) > at > com.cloud.upgrade.SystemVmTemplateRegistration.calculateChecksum(SystemVmTemplateRegistration.java:330) > at > com.cloud.upgrade.SystemVmTemplateRegistration.validateTemplates(SystemVmTemplateRegistration.java:690) > at > com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplates(SystemVmTemplateRegistration.java:713) > at > com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:824) > at > com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > at > com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) > at > com.cloud.upgrade.SystemVmTemplateRegistration.updateSystemVmTemplates(SystemVmTemplateRegistration.java:802) > at > com.cloud.upgrade.dao.Upgrade41520to41600.updateSystemVmTemplates(Upgrade41520to41600.java:105) > at > com.cloud.upgrade.DatabaseUpgradeChecker.updateSystemVmTemplates(DatabaseUpgradeChecker.java:258) > at > com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:347) > at > com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:384) > ... > 2021-10-19 23:08:02,994 ERROR [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) updateSystemVmTemplates:Exception while getting ids of > templates > com.cloud.utils.exception.CloudRuntimeException: 4.16.0 KVM SystemVm template > not found. Cannot upgrade system Vms > at > com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:827) > at > com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > at > com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) > ... > > > Due to these points, my vote for this RC is -1. > > My proposals to work around these issues are the following: > - Separate the build/downloading of systemVM template with different profiles > to be activated, for instance, "systemVmTemplateXenServer" (for people that > want XenServer system VM template with the auto seed process), and so on for > the other hypervisors. > - Allow people to pre-seed the system VM template for the upgrade. > Therefore, one can generate ACS packages without the system VM templates, and > then, during the upgrade, if the system VM template is already there, the > auto seed code does not need to do anything. > > I can work on both of these issues if you all agree to address this situation > with the proposals I described. > > Best regards, > Daniel Salvador > > > On 18/10/2021 15:25, Nicolas Vazquez wrote: >> Hi All, >> >> I have created a 4.16.0.0 release (RC1) with the following artefacts up for >> testing and a vote: >> >> Git Branch and Commit SH: >> https://github.com/apache/cloudstack/tree/4.16.0.0-RC20211018T1454 >> Commit: 506f8a8269551acb9b07b029da0f9e4fe858d150 >> >> Source release (checksums and signatures are available at the same location): >> https://dist.apache.org/repos/dist/dev/cloudstack/4.16.0.0/ >> >> PGP release keys (signed using 656E1BCC8CB54F84): >> https://dist.apache.org/repos/dist/release/cloudstack/KEYS >> >> The vote will be open until 21st October 2021, 21.00 CET (72h). >> >> For sanity in tallying the vote, can PMC members please be sure to indicate >> "(binding)" with their vote? >> >> [ ] +1 approve >> [ ] +0 no opinion >> [ ] -1 disapprove (and reason why) >> >> 4.16.0.0 System VM templates are available from here: >> https://download.cloudstack.org/systemvm/4.16/ >> >> >> Regards, >> >> Nicolas Vazquez >> >> >> >>
