[DISCUSS] Way to fix a problem with KVM snapshots for non-managed storages

[Slavka Peleva]

Hi all,

There is a problem with snapshots kept only on primary storage for
non-managed drivers, and I've opened an issue request #5433

The problem is when we delete a volume with more than one snapshot kept
only on primary with the option snapshot.backup.to.secondary to false. And
the behaviour is different for most of the non-managed drivers.
My question is which is the best way to handle this?
For me the better option is each driver to handle this (if there is need to
delete the snapshots from DB/storage or they could live without the volume)

Thanks for the help!


Best regards,
Slavka

Re: 4.16.0.0 release

[Andrija Panic]

Hi Ahmed,

please do not take me wrong - you can not "order" things - this is a
community project - people will implement new features when they need it
(or if somebody pays for it) - so it's "developed when needed".

I do understand your concerns, some improvements are possible - but you
should also know that your NETWORK infrastructure should be the one
blocking DDOS and other network attacks - NOT the end-point
servers/software, this is what every serious company/provider will do.

Best,
Andrija

On Mon, 13 Sept 2021 at 23:42, ahmed jabbar  wrote:

> Hi Nicolas,
> It will be better if you add 2FA function in this version due cyber
> security issues,
> And reCapcha connector also.
> CVM & SSVM need to be secured from DDOS attack with auto block ip sources
> configuration ,
> One of our CVM faced many times DDOS attacks with no configuration
> possibilities to auto block sources ip of attacker,
> Management UI can be secured by Cloudflare also ip table of management can
> be configured for various types of actions but CVM can’t secured by
> Cloudflare proxy.
> BR
> Ahmed.
>
> > On 13 Sep 2021, at 10:44 PM, Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
> >
> > Hi All,
> >
> > We are looking forward to cutting RC1 soon. Kindly share or ping me this
> week if there are any issues or pull requests that we should include in
> 4.16.0.0.
> >
> >
> > Regards,
> >
> > Nicolas Vazquez
> >
> >
> >
>


-- 

Andrija Panić

Re: [VOTE] Standard string lib

[Rohit Yadav]

Daniel - can you explain what are we exactly voting for?

I get that your vote thread is primarily about moving to commons-lang3 but it 
does not explain the plan and logistics, for example what about:

  *   Creating a utility facade under cloud-api and using that throughout the 
codebase; or is it find-replace all usage of google's Strings with common-lang3?
  *   Introducing specific checks via checkstyle plugin to enforce developers 
(https://github.com/apache/cloudstack/tree/main/tools/checkstyle)
  *   Updating the code conventions wiki/docs

Regards.


From: Pearl d'Silva 
Sent: Tuesday, September 14, 2021 09:27
To: dev@cloudstack.apache.org 
Subject: Re: [VOTE] Standard string lib

+1. Sounds like a good plan.

From: Gabriel Br?scher 
Sent: Monday, September 13, 2021 9:15 PM
To: dev 
Subject: Re: [VOTE] Standard string lib

+1

On Mon, Sep 13, 2021, 12:40 Sadi  wrote:

> +1
>
> Good idea.
>
> On 13/09/2021 12:02, Daniel Augusto Veronezi Salvador wrote:
> > Hi All,
> >
> > We had a discussion about standardizing the string libs we're using (
> https://lists.apache.org/thread.html/r806cd10b3de645c150e5e0e3d845c5a380a700197143f57f0834d758%40%3Cdev.cloudstack.apache.org%3E
> ).
> >
> > As I proposed, I'm opening this voting thread to see if all are in favor
> of using "commons.lang3" as the String standard library and for String
> operations not convered on "commons.lang3", we use our StringUtils. Then,
> if the vote passes, I will create the PR to address this change in the code
> base by removing unnecessary libraries, and changing the code to use
> "commons.lang3".
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Best regards,
> > Daniel
> >
>




 

Re: 4.16.0.0 release

[Rohit Yadav]

Hi Ahmed,

Our 4.16 RM Nicolas is simply reminding (mainly devs) about the possibility of 
cutting a RC soon (as he has previously shared his tentative/soft plan of 
cutting RC around the end of Sept) and asking if there are any specific 
issues/pull requests that should be included towards the 4.16.0.0 milestone. 
Here the "issues" are generally interpreted as something 
blocker/critical/essential towards the release, rather than a new feature/idea 
that someone in the community wants.

The process of requesting new features/ideas with the project is to raise a 
ticket here: https://github.com/apache/cloudstack/issues/new and hope it will 
be picked by somebody in the future if you're not going to work on it yourself. 
Any pull requests or discussions are also welcome on dev@ ML.


Regards.


From: Andrija Panic 
Sent: Tuesday, September 14, 2021 14:04
To: users 
Cc: dev 
Subject: Re: 4.16.0.0 release

Hi Ahmed,

please do not take me wrong - you can not "order" things - this is a
community project - people will implement new features when they need it
(or if somebody pays for it) - so it's "developed when needed".

I do understand your concerns, some improvements are possible - but you
should also know that your NETWORK infrastructure should be the one
blocking DDOS and other network attacks - NOT the end-point
servers/software, this is what every serious company/provider will do.

Best,
Andrija


 

On Mon, 13 Sept 2021 at 23:42, ahmed jabbar  wrote:

> Hi Nicolas,
> It will be better if you add 2FA function in this version due cyber
> security issues,
> And reCapcha connector also.
> CVM & SSVM need to be secured from DDOS attack with auto block ip sources
> configuration ,
> One of our CVM faced many times DDOS attacks with no configuration
> possibilities to auto block sources ip of attacker,
> Management UI can be secured by Cloudflare also ip table of management can
> be configured for various types of actions but CVM can’t secured by
> Cloudflare proxy.
> BR
> Ahmed.
>
> > On 13 Sep 2021, at 10:44 PM, Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
> >
> > Hi All,
> >
> > We are looking forward to cutting RC1 soon. Kindly share or ping me this
> week if there are any issues or pull requests that we should include in
> 4.16.0.0.
> >
> >
> > Regards,
> >
> > Nicolas Vazquez
> >
> >
> >
>


--

Andrija Panić

Re: [DISCUSS] Way to fix a problem with KVM snapshots for non-managed storages

[Daan Hoogland]

Slavka,
By your description it seems best to either backup those snapshots to
secondary on deletion of the VM or to delete them with the VM. leaving it
to the driver is ok, as long as the functionality is clear to the user. In
addition, we would want to keep behaviour as consistent as possible across
drivers. In this case there is a limitation to that. I'd say make an extra
option that defines behaviour on delete so the operator can decide.

On Tue, Sep 14, 2021 at 9:20 AM Slavka Peleva 
wrote:

> Hi all,
>
> There is a problem with snapshots kept only on primary storage for
> non-managed drivers, and I've opened an issue request #5433
> 
> The problem is when we delete a volume with more than one snapshot kept
> only on primary with the option snapshot.backup.to.secondary to false. And
> the behaviour is different for most of the non-managed drivers.
> My question is which is the best way to handle this?
> For me the better option is each driver to handle this (if there is need to
> delete the snapshots from DB/storage or they could live without the volume)
>
> Thanks for the help!
>
>
> Best regards,
> Slavka
>


-- 
Daan

Re: [VOTE] Standard string lib

[Daniel Augusto Veronezi Salvador]

Rohit, sure.

About the points:

1. The objective of the vote is to see if all are in favor of using 
"commons.lang3" as the String standard library and for String operations 
not covered on "commons.lang3", we use our StringUtils (as we discussed 
in the discussion thread - 
https://lists.apache.org/thread.html/r806cd10b3de645c150e5e0e3d845c5a380a700197143f57f0834d758%40%3Cdev.cloudstack.apache.org%3E). 
Then, if the vote passes, I will create the PR to address this change in 
the code base by removing unnecessary libraries, and changing the code 
to use "commons.lang3"'. Te proposal is to use "lang3" as the standard 
String library; therefore, I will replace every occurrence of others 
String libraries by "lang3" (and update "lang" to "lang3"). Our (facade) 
StringUtils will be only to specific methods that "lang3" doesn't cover, 
like "csvTagsToList", "areTagsEqual" and others.


2. As there are many libraries, what I could do is to add the module 
"IllegalImport" to the checkstyle and verify the libraries I will remove 
in the refactor.


3. I will update the code conventions wiki/docs with the outcome of this 
vote, and then we will be able to use it as a guideline in our reviews.


Best regards,
Daniel

On 14/09/2021 05:35, Rohit Yadav wrote:

Daniel - can you explain what are we exactly voting for?

I get that your vote thread is primarily about moving to commons-lang3 but it 
does not explain the plan and logistics, for example what about:

   *   Creating a utility facade under cloud-api and using that throughout the 
codebase; or is it find-replace all usage of google's Strings with common-lang3?
   *   Introducing specific checks via checkstyle plugin to enforce developers 
(https://github.com/apache/cloudstack/tree/main/tools/checkstyle)
   *   Updating the code conventions wiki/docs

Regards.


From: Pearl d'Silva 
Sent: Tuesday, September 14, 2021 09:27
To: dev@cloudstack.apache.org 
Subject: Re: [VOTE] Standard string lib

+1. Sounds like a good plan.

From: Gabriel Br?scher 
Sent: Monday, September 13, 2021 9:15 PM
To: dev 
Subject: Re: [VOTE] Standard string lib

+1

On Mon, Sep 13, 2021, 12:40 Sadi  wrote:


+1

Good idea.

On 13/09/2021 12:02, Daniel Augusto Veronezi Salvador wrote:

Hi All,

We had a discussion about standardizing the string libs we're using (

https://lists.apache.org/thread.html/r806cd10b3de645c150e5e0e3d845c5a380a700197143f57f0834d758%40%3Cdev.cloudstack.apache.org%3E
).

As I proposed, I'm opening this voting thread to see if all are in favor

of using "commons.lang3" as the String standard library and for String
operations not convered on "commons.lang3", we use our StringUtils. Then,
if the vote passes, I will create the PR to address this change in the code
base by removing unnecessary libraries, and changing the code to use
"commons.lang3".

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Best regards,
Daniel





  

Re: [VOTE] Standard string lib

[Nicolas Vazquez]

+1


Regards,

Nicolas Vazquez


From: Daniel Augusto Veronezi Salvador 
Sent: Monday, September 13, 2021 12:02 PM
To: dev@cloudstack.apache.org 
Subject: [VOTE] Standard string lib

Hi All,

We had a discussion about standardizing the string libs we're using 
(https://lists.apache.org/thread.html/r806cd10b3de645c150e5e0e3d845c5a380a700197143f57f0834d758%40%3Cdev.cloudstack.apache.org%3E).

As I proposed, I'm opening this voting thread to see if all are in favor of 
using "commons.lang3" as the String standard library and for String operations 
not convered on "commons.lang3", we use our StringUtils. Then, if the vote 
passes, I will create the PR to address this change in the code base by 
removing unnecessary libraries, and changing the code to use "commons.lang3".

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Best regards,
Daniel


 

Unable to add host

[technologyrss.mail]

*Hi, *

Please see my log, I can't understand why not added host on my Advanced 
zone. All servers are CentOS Linux release 7.9.2009 (Core).


|ca.plugin.root.auth.strictness|is set to|false
|

for ssh setting below config

PubkeyAcceptedKeyTypes=+ssh-dss
HostKeyAlgorithms=+ssh-dss
KexAlgorithms=+diffie-hellman-group1-sha1

please see my log error from ACS mgmt server

2021-09-14 21:43:01,298 DEBUG [c.c.u.s.SSHCmdHelper] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) SSH 
command: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup 
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/

SSH command output:
cat: /etc/cloudstack/agent/cloud.csr: No such file or directory
chmod: cannot access ‘/etc/cloudstack/agent/cloud.csr’: No such file or 
directory


2021-09-14 21:43:01,299 DEBUG [c.c.u.s.SSHCmdHelper] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) Executing 
cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup 
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
2021-09-14 21:43:03,881 DEBUG [c.c.s.StatsCollector] 
(StatsCollector-6:ctx-f2675358) (logid:2fb31c5c) StorageCollector is 
running...
2021-09-14 21:43:03,894 DEBUG [c.c.s.StatsCollector] 
(StatsCollector-3:ctx-240201ac) (logid:a7801c31) AutoScaling Monitor is 
running...
2021-09-14 21:43:03,947 DEBUG [c.c.s.StatsCollector] 
(StatsCollector-5:ctx-c7e24e1f) (logid:d16881a3) HostStatsCollector is 
running...
2021-09-14 21:43:06,826 DEBUG [c.c.u.s.SSHCmdHelper] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) SSH 
command: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup 
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/

SSH command output:
cat: /etc/cloudstack/agent/cloud.csr: No such file or directory
chmod: cannot access ‘/etc/cloudstack/agent/cloud.csr’: No such file or 
directory


2021-09-14 21:43:06,826 DEBUG [c.c.u.s.SSHCmdHelper] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) Executing 
cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup 
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
2021-09-14 21:43:09,204 DEBUG [c.c.u.s.SSHCmdHelper] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) SSH 
command: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup 
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/

SSH command output:
cat: /etc/cloudstack/agent/cloud.csr: No such file or directory
chmod: cannot access ‘/etc/cloudstack/agent/cloud.csr’: No such file or 
directory


2021-09-14 21:43:09,205 WARN  [c.c.h.k.d.LibvirtServerDiscoverer] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) can't 
setup agent, due to com.cloud.utils.exception.CloudRuntimeException: 
Failed to setup keystore on the KVM host: 10.66.100.50 - Failed to setup 
keystore on the KVM host: 10.66.100.50
2021-09-14 21:43:09,206 WARN  [c.c.r.ResourceManagerImpl] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) Unable to 
find the server resources at http://10.66.100.50
2021-09-14 21:43:09,206 INFO  [c.c.u.e.CSExceptionErrorCode] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) Could not 
find exception: com.cloud.exception.DiscoveryException in error code 
list for exceptions
2021-09-14 21:43:09,212 WARN  [o.a.c.a.c.a.h.AddHostCmd] 
(qtp1136768342-268:ctx-9737fb53 ctx-4efc5a53) (logid:83b994d6) Exception:

com.cloud.exception.DiscoveryException: Unable to add the host
    at 
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:825)
    at 
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:611)
    at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
    at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)

    at com.sun.proxy.$Proxy192.discoverHosts(Unknown Source)
    at 
org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)

    at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.jav