Re: Apache CloudStack Generic Key-Value and Communication Plug-in

[Ivan Kudryavtsev]

Hello, Daan, Rafael.

Ok, let's discover some points.

Rafael:
All the stuff is managed in Elasticsearch, clustered ES is supported as
well. As for now, value datatype is 'keyword' and doesn't fit well for
blobs. But, actually, there are no practical limitations except we don't
want press the CS server with streaming for blobs as it's design is not
well designed for long lasting jobs like that.

To adress the case we also plan to develop artifact storage which is
intended to keep blobs with versioning and managed security. Probably will
use Ceph under the hood.

Daan:
Let's go with the cases. Actually, in the past I published the announcement
for log management plugin which intention is to provide access to VM logs
from the API and other interfaces. Now we develop CLI utility for it. That
plugin helps to observe what logs are generate during VM deployment.

This plugin (KV) helps to deploy complex topologies, like Docker Swarm,
Kubernetes, MySQL Galera replicaset, DC/OS, monitor the state of the
deployments and do trivial things:

Observe current VM load, RAM availability, disk space. All these parameters
can be observed in historical perspective or last values.

A quorum example. Say, you deploy two VMs programmatically and they have to
decide who is Docker master and who is worker, so you create shared storage
and both VMs place they UUID in 'order' key. They check the history for
'order' and find who is the first. Next, the first generates cluster
initialization and places secret key to 'secret'. After, worker reads it
and joins the cluster. Want to add a new VM to the cluster, just pass that
KV to it as well. Of cource, proper templates with bootstrap code are
required.

Actually, there are tons of very sophisticated deployments can be
implemented in a very generic way with theese two plugins.

Another piece of software to be developed is one-time accessor plugin which
is important for secure sharing of various credentials when multi tenant
VMs are used.

Have a good day.


пт, 21 сент. 2018 г., 2:21 Rafael Weingärtner :

> That is very interesting Ivan!
>
> I have a few questions though:
>
>- Do you have a limit on ”value” size? Can it be a binary of a file?
>- Where do you guys store these “keys” and ”values”?
>
>
> On Wed, Sep 19, 2018 at 4:02 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com>
> wrote:
>
> > Hello, community.
> >
> > We glad to announce a new Open Source plug-in for Apache CloudStack which
> > helps to maintain generic key-value storages. There are three types are
> > supported right now:
> >
> > - temporary KV with TTL (many per account);
> > - persistent KV bound to VM (one per VM);
> > - persistent KV bound to account (many per account).
> >
> > KV storages support a commit-log based approach where old values are not
> > erased with new ones and it provides the access to the last key/values
> and
> > to the history which helps to maintain things like quorums and other
> > operations which require serialized access.
> >
> > The plugin uses Elasticsearch. I
> >
> > If you are interested to know more, contact our website for the full
> > announcement and download details:
> > https://bitworks.software/en/2018-09-19-acs-vm-kv-storage-plugin.html
> >
> > All questions, improvements and considerations are welcome!
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks LLC
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ 
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: Apache CloudStack Generic Key-Value and Communication Plug-in

[Rafael Weingärtner]

That is very interesting Ivan!

I have a few questions though:

   - Do you have a limit on ”value” size? Can it be a binary of a file?
   - Where do you guys store these “keys” and ”values”?


On Wed, Sep 19, 2018 at 4:02 AM, Ivan Kudryavtsev 
wrote:

> Hello, community.
>
> We glad to announce a new Open Source plug-in for Apache CloudStack which
> helps to maintain generic key-value storages. There are three types are
> supported right now:
>
> - temporary KV with TTL (many per account);
> - persistent KV bound to VM (one per VM);
> - persistent KV bound to account (many per account).
>
> KV storages support a commit-log based approach where old values are not
> erased with new ones and it provides the access to the last key/values and
> to the history which helps to maintain things like quorums and other
> operations which require serialized access.
>
> The plugin uses Elasticsearch. I
>
> If you are interested to know more, contact our website for the full
> announcement and download details:
> https://bitworks.software/en/2018-09-19-acs-vm-kv-storage-plugin.html
>
> All questions, improvements and considerations are welcome!
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks LLC
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>



-- 
Rafael Weingärtner

Re: Apache CloudStack Generic Key-Value and Communication Plug-in

[Daan Hoogland]

sounds nice Ivan, Do you have a whitepaper on it? I strugle to see a clear
usecase for this plugin. maybe you have a casus or two!?!

On Wed, Sep 19, 2018 at 9:02 AM Ivan Kudryavtsev 
wrote:

> Hello, community.
>
> We glad to announce a new Open Source plug-in for Apache CloudStack which
> helps to maintain generic key-value storages. There are three types are
> supported right now:
>
> - temporary KV with TTL (many per account);
> - persistent KV bound to VM (one per VM);
> - persistent KV bound to account (many per account).
>
> KV storages support a commit-log based approach where old values are not
> erased with new ones and it provides the access to the last key/values and
> to the history which helps to maintain things like quorums and other
> operations which require serialized access.
>
> The plugin uses Elasticsearch. I
>
> If you are interested to know more, contact our website for the full
> announcement and download details:
> https://bitworks.software/en/2018-09-19-acs-vm-kv-storage-plugin.html
>
> All questions, improvements and considerations are welcome!
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks LLC
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>


-- 
Daan

[GitHub] dhlaluku commented on a change in pull request #4: Add run diagnostics to admin guide and new apis

[GitBox]

dhlaluku commented on a change in pull request #4: Add run diagnostics to admin 
guide and new apis
URL: 
https://github.com/apache/cloudstack-documentation/pull/4#discussion_r219231258
 
 

 ##
 File path: source/adminguide/systemvm.rst
 ##
 @@ -552,3 +552,25 @@ The administrator can log in to the secondary storage VM 
if needed.
:alt: Updating Console Proxy SSL Certificate
 .. |vr-upgrade.png| image:: /_static/images/vr-upgrade.png
:alt: Button to upgrade VR to use the new template.
+
+Troubleshoot networks from System VMs
+-
+
+
+For troubleshooting of network issues in CloudStack hosted networks, 
CloudStack allows
+the administrator to execute network-utility commands (ping, traceroute or 
arping)
+remotely on system VMs.
+
+To run either a ping, traceroute or arping through the CloudStack UI:
+
+#. As an administrator, log in to the CloudStack UI.
+
+#. Navigate to Infrastructure > System VMs or Virtual Routers.
+
+#. Click on the Run Diagnostics button. |run-diagnostics-icon.png|
+
+#. A form will pop up similar to this;
+  |diagnostics-form.png|
+
+#. Fill in the details and click OK.
 
 Review comment:
   
![image](https://user-images.githubusercontent.com/19572501/45832550-7aa4b900-bd02-11e8-9fb5-be351756297a.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dhlaluku commented on a change in pull request #4: Add run diagnostics to admin guide and new apis

[GitBox]

dhlaluku commented on a change in pull request #4: Add run diagnostics to admin 
guide and new apis
URL: 
https://github.com/apache/cloudstack-documentation/pull/4#discussion_r219198260
 
 

 ##
 File path: source/adminguide/systemvm.rst
 ##
 @@ -552,3 +552,25 @@ The administrator can log in to the secondary storage VM 
if needed.
:alt: Updating Console Proxy SSL Certificate
 .. |vr-upgrade.png| image:: /_static/images/vr-upgrade.png
:alt: Button to upgrade VR to use the new template.
+
+Troubleshoot networks from System VMs
+-
+
+
+For troubleshooting of network issues in CloudStack hosted networks, 
CloudStack allows
+the administrator to execute network-utility commands (ping, traceroute or 
arping)
+remotely on system VMs.
+
+To run either a ping, traceroute or arping through the CloudStack UI:
+
+#. As an administrator, log in to the CloudStack UI.
+
+#. Navigate to Infrastructure > System VMs or Virtual Routers.
+
+#. Click on the Run Diagnostics button. |run-diagnostics-icon.png|
+
+#. A form will pop up similar to this;
+  |diagnostics-form.png|
 
 Review comment:
   Added links to the images and check by building, works fine now.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: VRs swapping with 256 MB RAM

[Rakesh Venkatesh]

Hello Rene

Even for VR's running on KVM, 256MB is really less. Thats why we offer
extra VR with 2 cores and 1GB RAM as another option so that customers can
use it instead of the default 256MB and 1 core.

On Tue, Sep 18, 2018 at 5:56 PM Rene Moser  wrote:

> Hi
>
> While running test for a 4.11.1 (VMware) upgrade in our lab, we run into
> low memory / swapping of VRs having 256 MB RAM. After 2-3 days it became
> critical because the management server connections to VRs took very
> long, minutes, this resulted in many more problems all over.
>
> Make sure your VRs have enough RAM.
>
> Regards
> René
>


-- 
Thanks and regards
Rakesh venkatesh

Re: [VOTE] 4.11.2.0 RC1

[Rohit Yadav]

-0


A bug was found in the systemvmtemplate patching code and has been fixed in 
this PR:

https://github.com/apache/cloudstack/pull/2855 (tests OK)


The above fix does not require new packages per se, but only a new 
systemvmtemplate. Please continue testing RC1 this week so if we find any 
additional bugs we can attempt to fix that before RC2 is cut.


For convenience, I've updated the 4.11.2 systemvmtemplate here:

http://packages.shapeblue.com/testing/systemvm/


- Rohit






From: Paul Angus 
Sent: Wednesday, September 19, 2018 1:15:22 AM
To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
Subject: [VOTE] 4.11.2.0 RC1

Hi All,

I've created a 4.11.2.0 release (RC1), with the following artefacts up for 
testing and a vote:

Git Branch and Commit SH:
https://gitbox.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.11.2.0-RC20180918T1628
Commit: 115fb482409da71b5e82bbc2190c50291c833c15

Source release (checksums and signatures are available at the same location):
https://dist.apache.org/repos/dist/dev/cloudstack/4.11.2.0/

PGP release keys (signed using 8B309F7251EE0BC8):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

The vote will be open until the beginning of next week, 24th September 2018.

For sanity in tallying the vote, can PMC members please be sure to indicate 
"(binding)" with their vote

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Additional information:

For users' convenience, I've built packages from 
115fb482409da71b5e82bbc2190c50291c833c15 and published RC1 repository here:
http://packages.shapeblue.com/testing/41120rc1/

4.11.2 systemvm templates are available from here:
http://packages.shapeblue.com/testing/systemvm/




paul.an...@shapeblue.com
www.shapeblue.com
Amadeus House, Floral Street, London  WC2E 9DPUK
@shapeblue




rohit.ya...@shapeblue.com 
www.shapeblue.com
Amadeus House, Floral Street, London  WC2E 9DPUK
@shapeblue