Re: [ACS41] System VMs not syncing time - does this block the release?

[Chiradeep Vittal]

I am not sure why it is missing, but I will refer to
Citrix XenServer 6.0 Virtual Machine Installation Guide

http://s.apache.org/YGn

And quote
"Time Handling in Linux VMs
By default, the clocks in a Linux VM are synchronized to the clock running
on the control domain, and cannot be
independently changed. This mode is a convenient default, since only the
control domain needs to be running the
NTP service to keep accurate time across all VMs. Upon installation of a
new Linux VM, make sure you change the
time-zone from the default UTC to your local value (see the section called
³Release Notes² for specific distribution
instructions).
To set individual Linux VMs to maintain independent times
1. From a root prompt on the VM, run the command: echo 1 >
/proc/sys/xen/independent_wallclock
2. This can be persisted across reboots by changing the /etc/sysctl.conf
configuration file and adding:
# Set independent wall clock time
xen.independent_wallclock=1
3. As a third alternative, independent_wallclock=1 may also be passed as a
boot parameter to the VM.
"


On 5/15/13 12:09 PM, "Chip Childers"  wrote:

>On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
>> /proc/sys is not a regular filesystem and cannot be added to from the
>> shell.
>> Drivers need to add nodes into this filesystem.
>
>Backing up a bit...
>
>This is the current system VM image that the 4.1 software should be
>using on Xen:
>
>http://download.cloud.com/templates/acton/acton-
>systemvm-02062012.vhd.bz2
>
>Does this system VM have the correct drives installed to set
>/proc/sys/xen to use the Dom0 time for sync?
>
>If so, then is this a devcloud only issue for Xen?  If that's the case,
>then we shouldn't block a release to simply improve things.
>
>We do know that a patch for kvm was needed.
>
>We still don't have any
>answer or comments about the VMware system VM (specifically this
>template:  http://download.cloud.com/templates/burbank/burbank-
>systemvm-08012012.ova
>

Re: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 vs 4.2

[Chip Childers]

Adding relevant folks from previous discussions of this feature to the
CC list.

One other note...  From what I can tell, the work intended for 4.2 to
re-enable security groups within an advanced zone is limited to Xen and
KVM.  I believe that Nicolas (the issue reporter) is using VMware.

We do have a note from Wei (below) highlighting his desire to see this
feature as well (although, Wei, what HV are you using?).

Thoughts on what to do?

-chip

On Wed, May 15, 2013 at 09:20:06AM -0700, Alex Huang wrote:
> I'm a very strong believer that CloudStack releases should always be 
> upgradable from previous releases.  We can't strand our user base on a 
> previous release.
> 
> --Alex
> 
> > -Original Message-
> > From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
> > Sent: Wednesday, May 15, 2013 8:28 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 vs 4.2
> > 
> > Half of our platforms are on 2.2.14 (advanced zone with security groups).
> > These platform work well. We are looking for a way to upgrade to 4.* for
> > more functionalities, so that we do not need to take the difference of
> > cloudstack version into account in development.
> > 
> > As I know, the citrix guys are working on this. Jessica Wang said the 
> > feature
> > will be merged into master branch soon.It looks the coding is almost done.
> > 
> > I hope this feature could be included in 4.1, of course. However, we also
> > need some days for testing and bug fix. It means cloudstack 4.1 will delay 
> > for
> > uncertain days (it is very bad, right?). It is a difficult choice.
> > 
> > I do not know how many companies are using 2.2.14  (advanced zone with
> > security groups) and eager to upgrade. I will join the dev and testing if
> > needed.
> > 
> > 
> > 2013/5/15 Chip Childers 
> > 
> > > Sebastian re-opened CLOUDSTACK-2463 due to users wanting to upgrade
> > > from 2.x to 4.1.  This relates to the security groups feature being
> > > available when using VLANs in an advanced networking zone.  This
> > > feature was apparently broken in the 3.x series, and is not slated to
> > > be reintroduced until 4.2.
> > >
> > > This is a horrible situation, and one that we've now encountered for a
> > > third time.
> > >
> > > IMO, we have 2 very specific options:
> > >
> > > 1) We pull that new feature into 4.1, and do the relevant testing.
> > >
> > > 2) We do not pull that feature into 4.1, and release as is with a
> > > strong message in the release notes highlighting that we know that 2.x
> > > to 4.1 will not support it (and state that those users requiring the
> > > feature should wait for 4.2).
> > >
> > > At this point, I don't have a preference.  We probably need to
> > > understand the effort for (1), as well as understand who would do that
> > > work (dev AND testing).
> > >
> > > Thoughts?
> > >
> > > -chip
> > >
> 

[ACS41] Current blocker status

[Chip Childers]

The following are currently blocking a new RC for 4.1.0:

CLOUDSTACK-2039
Improve console access security with 128-bit AES
encryption and securely-randomized key generation

This needs to be addressed.  Kelven, do you mind reviewing Wido's error?


CLOUDSTACK-2463
CS Upgrade 2.Upgrade2.14 to 4.1.0 failed due to no public
network found (configuration : advanced network with security groups)

This is being discussed in another thread.


CLOUDSTACK-2492
System VM Clock Drift

This is also being discussed in another thread.


CLOUDSTACK-2516
Create User API compability broken now

Nobody has taken this on yet.

RE: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 vs 4.2

[Musayev, Ilya]

I'd ask for this be to enabled in vmware/vsphere is possible. 
There are by far more vmware users in corporate/enteprise world than kvm and 
xen.

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Wednesday, May 15, 2013 3:26 PM
> To: dev@cloudstack.apache.org
> Cc: Anthony Xu; Manan Shah; Angeline Shen; Alena Prokharchyk
> Subject: Re: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 vs 4.2
> 
> Adding relevant folks from previous discussions of this feature to the CC 
> list.
> 
> One other note...  From what I can tell, the work intended for 4.2 to re-
> enable security groups within an advanced zone is limited to Xen and KVM.  I
> believe that Nicolas (the issue reporter) is using VMware.
> 
> We do have a note from Wei (below) highlighting his desire to see this
> feature as well (although, Wei, what HV are you using?).
> 
> Thoughts on what to do?
> 
> -chip
> 
> On Wed, May 15, 2013 at 09:20:06AM -0700, Alex Huang wrote:
> > I'm a very strong believer that CloudStack releases should always be
> upgradable from previous releases.  We can't strand our user base on a
> previous release.
> >
> > --Alex
> >
> > > -Original Message-
> > > From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
> > > Sent: Wednesday, May 15, 2013 8:28 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1
> > > vs 4.2
> > >
> > > Half of our platforms are on 2.2.14 (advanced zone with security groups).
> > > These platform work well. We are looking for a way to upgrade to 4.*
> > > for more functionalities, so that we do not need to take the
> > > difference of cloudstack version into account in development.
> > >
> > > As I know, the citrix guys are working on this. Jessica Wang said
> > > the feature will be merged into master branch soon.It looks the coding is
> almost done.
> > >
> > > I hope this feature could be included in 4.1, of course. However, we
> > > also need some days for testing and bug fix. It means cloudstack 4.1
> > > will delay for uncertain days (it is very bad, right?). It is a difficult 
> > > choice.
> > >
> > > I do not know how many companies are using 2.2.14  (advanced zone
> > > with security groups) and eager to upgrade. I will join the dev and
> > > testing if needed.
> > >
> > >
> > > 2013/5/15 Chip Childers 
> > >
> > > > Sebastian re-opened CLOUDSTACK-2463 due to users wanting to
> > > > upgrade from 2.x to 4.1.  This relates to the security groups
> > > > feature being available when using VLANs in an advanced networking
> > > > zone.  This feature was apparently broken in the 3.x series, and
> > > > is not slated to be reintroduced until 4.2.
> > > >
> > > > This is a horrible situation, and one that we've now encountered
> > > > for a third time.
> > > >
> > > > IMO, we have 2 very specific options:
> > > >
> > > > 1) We pull that new feature into 4.1, and do the relevant testing.
> > > >
> > > > 2) We do not pull that feature into 4.1, and release as is with a
> > > > strong message in the release notes highlighting that we know that
> > > > 2.x to 4.1 will not support it (and state that those users
> > > > requiring the feature should wait for 4.2).
> > > >
> > > > At this point, I don't have a preference.  We probably need to
> > > > understand the effort for (1), as well as understand who would do
> > > > that work (dev AND testing).
> > > >
> > > > Thoughts?
> > > >
> > > > -chip
> > > >
> >

Re: [ACS41] System VMs not syncing time - does this block the release?

[Chiradeep Vittal]

For VMWare, the command
vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
patch for /etc/init.d/cloud-early-config to enable it


On 5/15/13 12:23 PM, "Chiradeep Vittal" 
wrote:

>I am not sure why it is missing, but I will refer to
>Citrix XenServer 6.0 Virtual Machine Installation Guide
>
>http://s.apache.org/YGn
>
>And quote
>"Time Handling in Linux VMs
>By default, the clocks in a Linux VM are synchronized to the clock running
>on the control domain, and cannot be
>independently changed. This mode is a convenient default, since only the
>control domain needs to be running the
>NTP service to keep accurate time across all VMs. Upon installation of a
>new Linux VM, make sure you change the
>time-zone from the default UTC to your local value (see the section called
>³Release Notes² for specific distribution
>instructions).
>To set individual Linux VMs to maintain independent times
>1. From a root prompt on the VM, run the command: echo 1 >
>/proc/sys/xen/independent_wallclock
>2. This can be persisted across reboots by changing the /etc/sysctl.conf
>configuration file and adding:
># Set independent wall clock time
>xen.independent_wallclock=1
>3. As a third alternative, independent_wallclock=1 may also be passed as a
>boot parameter to the VM.
>"
>
>
>On 5/15/13 12:09 PM, "Chip Childers"  wrote:
>
>>On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
>>> /proc/sys is not a regular filesystem and cannot be added to from the
>>> shell.
>>> Drivers need to add nodes into this filesystem.
>>
>>Backing up a bit...
>>
>>This is the current system VM image that the 4.1 software should be
>>using on Xen:
>>
>>http://download.cloud.com/templates/acton/acton-
>>systemvm-02062012.vhd.bz2
>>
>>Does this system VM have the correct drives installed to set
>>/proc/sys/xen to use the Dom0 time for sync?
>>
>>If so, then is this a devcloud only issue for Xen?  If that's the case,
>>then we shouldn't block a release to simply improve things.
>>
>>We do know that a patch for kvm was needed.
>>
>>We still don't have any
>>answer or comments about the VMware system VM (specifically this
>>template:  http://download.cloud.com/templates/burbank/burbank-
>>systemvm-08012012.ova
>>
>

Storage as a Service

[Mike Tutkowski]

Hi everyone,

I'm at Build a Cloud Day today in San Francisco and had a question from a
potential CloudStack user about Storage as a Service.

His business would like to sell Storage as a Service (like Amazon S3) and
would like to know if CloudStack was an applicable solution for this use
case.

Thoughts?

Thanks!

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Storage as a Service

[Chip Childers]

CloudStack isn't an object storage system really...  the best bet, if
that's the focus of what he wants to do, is to look at things like Riak
CS, Swift, Ceph, etc...


On Wed, May 15, 2013 at 02:13:29PM -0600, Mike Tutkowski wrote:
> Hi everyone,
> 
> I'm at Build a Cloud Day today in San Francisco and had a question from a
> potential CloudStack user about Storage as a Service.
> 
> His business would like to sell Storage as a Service (like Amazon S3) and
> would like to know if CloudStack was an applicable solution for this use
> case.
> 
> Thoughts?
> 
> Thanks!
> 
> -- 
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*

Re: Storage as a Service

[John Burwell]

Chip,

Disclaimer: I am an employee of Basho, makers of Riak CS.

+1.  

The S3 compatibility layer provided by CloudStack lacks may of S3's 
availability guarantees since it stored the index in a MySQL table, and files 
in a single folder on disk.  There are certainly ways to increase the 
reliability of this implementation (e.g. placing the file storage on a SAN, 
clustering MySQL).  However, the operational complexity would be far greater 
than Riak CS, and would still lack horizontal scalability on commodity hardware 
(i.e. much lower cost per GB).  If you interested in drilling in more deeply, I 
would be happy to have an offline conversation regarding Riak CS.

Thanks,
-John

On May 15, 2013, at 4:47 PM, Chip Childers  wrote:

> CloudStack isn't an object storage system really...  the best bet, if
> that's the focus of what he wants to do, is to look at things like Riak
> CS, Swift, Ceph, etc...
> 
> 
> On Wed, May 15, 2013 at 02:13:29PM -0600, Mike Tutkowski wrote:
>> Hi everyone,
>> 
>> I'm at Build a Cloud Day today in San Francisco and had a question from a
>> potential CloudStack user about Storage as a Service.
>> 
>> His business would like to sell Storage as a Service (like Amazon S3) and
>> would like to know if CloudStack was an applicable solution for this use
>> case.
>> 
>> Thoughts?
>> 
>> Thanks!
>> 
>> -- 
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud
>> *™*

Re: Storage as a Service

[Wido den Hollander]

On 05/15/2013 10:47 PM, Chip Childers wrote:

CloudStack isn't an object storage system really...  the best bet, if
that's the focus of what he wants to do, is to look at things like Riak
CS, Swift, Ceph, etc...



Indeed, I'd recommend looking at the RADOS Gateway [0] of Ceph. It gives 
you a Amazon S3 compatible gateway.


[0]: http://eu.ceph.com/docs/master/radosgw/

Wido



On Wed, May 15, 2013 at 02:13:29PM -0600, Mike Tutkowski wrote:

Hi everyone,

I'm at Build a Cloud Day today in San Francisco and had a question from a
potential CloudStack user about Storage as a Service.

His business would like to sell Storage as a Service (like Amazon S3) and
would like to know if CloudStack was an applicable solution for this use
case.

Thoughts?

Thanks!

--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Review Request: Implicit dedication of resources to an account

[Prachi Damle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11045/#review20619
---

Ship it!


The changes in the patch look good. Please also add an integration test to 
deploy VM using a service offering that uses the ImplicitPlanner.

- Prachi Damle


On May 15, 2013, 2:08 p.m., Devdeep Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11045/
> ---
> 
> (Updated May 15, 2013, 2:08 p.m.)
> 
> 
> Review request for cloudstack and Prachi Damle.
> 
> 
> Description
> ---
> 
> Changes for implicitly dedicating a resource. This patch implements the 
> implicit dedication portion of the feature described in the following FS
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dedicated+Resources+-+Private+zone%2C+pod%2C+cluster%2C+host+Functional+Spec
> 
> Patch includes the following changes:
> 1. A new implicit planner which extends the functionality provided by 
> FirstFitPlanner.
> 2. Implicit planner can be used in either strict or preferred mode. In strict 
> mode it tries to deploy a vm of a given account on a host on which vms of the 
> account are already running. If no such host is found it'll search for an 
> empty host to service the request. Otherwise the deploy vm request fails.
> 3. In preferred mode, if a host which is running vms of the account or an 
> empty host isn't found, the planner then tries to deploy on any other host 
> provided it isn't running implicitly dedicated strict vms of any other 
> account.
> 4. Updated the createServiceOffering api to configure the details for the 
> planner that the service offering is using.
> 5. Made db changes to store the service offering details for the planner.
> 6. Unit tests for testing the implicit planner functionality.
> 
> 
> This addresses bug https://issues.apache.org/jira/browse/CLOUDSTACK-681.
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/deploy/DeploymentPlanner.java eb56a59 
>   api/src/org/apache/cloudstack/api/ApiConstants.java f179aaa 
>   
> api/src/org/apache/cloudstack/api/command/admin/offering/CreateServiceOfferingCmd.java
>  c155b70 
>   client/pom.xml 197ba27 
>   client/tomcatconf/applicationContext.xml.in d5c61bb 
>   client/tomcatconf/componentContext.xml.in 8a45e5f 
>   engine/schema/src/com/cloud/service/ServiceOfferingDetailsVO.java 
> PRE-CREATION 
>   engine/schema/src/com/cloud/service/ServiceOfferingVO.java fd31d30 
>   engine/schema/src/com/cloud/service/dao/ServiceOfferingDao.java 589de7c 
>   engine/schema/src/com/cloud/service/dao/ServiceOfferingDaoImpl.java 062103e 
>   engine/schema/src/com/cloud/service/dao/ServiceOfferingDetailsDao.java 
> PRE-CREATION 
>   engine/schema/src/com/cloud/service/dao/ServiceOfferingDetailsDaoImpl.java 
> PRE-CREATION 
>   plugins/deployment-planners/implicit-dedication/pom.xml PRE-CREATION 
>   
> plugins/deployment-planners/implicit-dedication/src/com/cloud/deploy/ImplicitDedicationPlanner.java
>  PRE-CREATION 
>   
> plugins/deployment-planners/implicit-dedication/test/org/apache/cloudstack/implicitplanner/ImplicitPlannerTest.java
>  PRE-CREATION 
>   plugins/pom.xml e49fac9 
>   server/src/com/cloud/configuration/ConfigurationManager.java 2ce3fa0 
>   server/src/com/cloud/configuration/ConfigurationManagerImpl.java d17b4d1 
>   server/test/com/cloud/vpc/MockConfigurationManagerImpl.java f02895c 
>   
> server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java 
> 7ffbe32 
>   setup/db/db/schema-410to420.sql 86124ea 
> 
> Diff: https://reviews.apache.org/r/11045/diff/
> 
> 
> Testing
> ---
> 
> Following tests were done:
> 1. Deployed an instance for an account created using a service offering using 
> implicit planner. Verified the host on which the first vm was deployed gets 
> picked up for subsequent deployment requests for that account (using service 
> offering with implicit planner).
> 2. If a deploy vm request is placed from another account, the host running 
> implicit vm of a different account isn't picked up.
> 3. Verified for a deployment request with implicit planner in strict mode, 
> first the planner looks for hosts running implicit vm of the account. If none 
> are available then empty hosts are tried. Otherwise deploy vm fails.
> 4. Verified for a deployment request with implicit planner in preferred mode, 
> first the planner looks for hosts running implicit vm of the account. If none 
> are available then empty hosts are tried. Then in tries on all hosts besides 
> the one running implicit strict vms of other account.
> 5. Unit tests to validate different scenarios.
> 6. Made sure there are no rat failures and patch applies cleanly.
> 
> 
> Thanks,
> 
> Devdeep Singh
> 
>

Re: [ACS41] System VMs not syncing time - does this block the release?

[Chiradeep Vittal]

Did some further digging around as to why
/proc/sys/xen/independent_wallclock is not there on the Debian system vm.

TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
PV kernel, not a specialized paravirt kernel). To eliminate
special-casing, the independent_wallclock feature was dropped. However,
this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
required on ANY domU. So the clock on the domU is only sync'ed at domU
creation time. I suspect Citrix XenServer might have some workaround


http://www.gossamer-threads.com/lists/xen/users/234750


What this means is that we *may* need to run ntp on system vms on Xen.
This is of course a non-trivial change involving updates to systemvms.

I suspect that it does not matter much for virtual routers or console
proxy vms.

We could have an advisory that for those users that care (e.g., those
using S3 sync) that they need to run ntp after the SSVM has been created.
I.e, login to the SSVM and run
apt-get update
apt-get install ntp

--
Chiradeep



On 5/15/13 1:11 PM, "Chiradeep Vittal"  wrote:

>For VMWare, the command
>vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
>patch for /etc/init.d/cloud-early-config to enable it
>
>
>On 5/15/13 12:23 PM, "Chiradeep Vittal" 
>wrote:
>
>>I am not sure why it is missing, but I will refer to
>>Citrix XenServer 6.0 Virtual Machine Installation Guide
>>
>>http://s.apache.org/YGn
>>
>>And quote
>>"Time Handling in Linux VMs
>>By default, the clocks in a Linux VM are synchronized to the clock
>>running
>>on the control domain, and cannot be
>>independently changed. This mode is a convenient default, since only the
>>control domain needs to be running the
>>NTP service to keep accurate time across all VMs. Upon installation of a
>>new Linux VM, make sure you change the
>>time-zone from the default UTC to your local value (see the section
>>called
>>³Release Notes² for specific distribution
>>instructions).
>>To set individual Linux VMs to maintain independent times
>>1. From a root prompt on the VM, run the command: echo 1 >
>>/proc/sys/xen/independent_wallclock
>>2. This can be persisted across reboots by changing the /etc/sysctl.conf
>>configuration file and adding:
>># Set independent wall clock time
>>xen.independent_wallclock=1
>>3. As a third alternative, independent_wallclock=1 may also be passed as
>>a
>>boot parameter to the VM.
>>"
>>
>>
>>On 5/15/13 12:09 PM, "Chip Childers"  wrote:
>>
>>>On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
 /proc/sys is not a regular filesystem and cannot be added to from the
 shell.
 Drivers need to add nodes into this filesystem.
>>>
>>>Backing up a bit...
>>>
>>>This is the current system VM image that the 4.1 software should be
>>>using on Xen:
>>>
>>>http://download.cloud.com/templates/acton/acton-
>>>systemvm-02062012.vhd.bz2
>>>
>>>Does this system VM have the correct drives installed to set
>>>/proc/sys/xen to use the Dom0 time for sync?
>>>
>>>If so, then is this a devcloud only issue for Xen?  If that's the case,
>>>then we shouldn't block a release to simply improve things.
>>>
>>>We do know that a patch for kvm was needed.
>>>
>>>We still don't have any
>>>answer or comments about the VMware system VM (specifically this
>>>template:  http://download.cloud.com/templates/burbank/burbank-
>>>systemvm-08012012.ova
>>>
>>
>

Re: [ACS41] System VMs not syncing time - does this block the release?

[Chip Childers]

On Wed, May 15, 2013 at 01:59:13PM -0700, Chiradeep Vittal wrote:
> Did some further digging around as to why
> /proc/sys/xen/independent_wallclock is not there on the Debian system vm.
> 
> TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
> PV kernel, not a specialized paravirt kernel). To eliminate
> special-casing, the independent_wallclock feature was dropped. However,
> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
> required on ANY domU. So the clock on the domU is only sync'ed at domU
> creation time. I suspect Citrix XenServer might have some workaround
> 
> 
> http://www.gossamer-threads.com/lists/xen/users/234750
> 
> 
> What this means is that we *may* need to run ntp on system vms on Xen.
> This is of course a non-trivial change involving updates to systemvms.
> 
> I suspect that it does not matter much for virtual routers or console
> proxy vms.
> 
> We could have an advisory that for those users that care (e.g., those
> using S3 sync) that they need to run ntp after the SSVM has been created.
> I.e, login to the SSVM and run
> apt-get update
> apt-get install ntp

Fair idea, but what about when the MS decides that it needs more SSVMs?

Is another possibility to take the existing system VM templates and
simply update them for this issue?

The potential benefit of either approach above is that we would be able
to ensure that we did it on all three templates if required.

We also obviously need to consider what to do for 4.2, where we were
planning on officially releasing new system VMs.

> 
> --
> Chiradeep
> 
> 
> 
> On 5/15/13 1:11 PM, "Chiradeep Vittal"  wrote:
> 
> >For VMWare, the command
> >vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
> >patch for /etc/init.d/cloud-early-config to enable it
> >
> >
> >On 5/15/13 12:23 PM, "Chiradeep Vittal" 
> >wrote:
> >
> >>I am not sure why it is missing, but I will refer to
> >>Citrix XenServer 6.0 Virtual Machine Installation Guide
> >>
> >>http://s.apache.org/YGn
> >>
> >>And quote
> >>"Time Handling in Linux VMs
> >>By default, the clocks in a Linux VM are synchronized to the clock
> >>running
> >>on the control domain, and cannot be
> >>independently changed. This mode is a convenient default, since only the
> >>control domain needs to be running the
> >>NTP service to keep accurate time across all VMs. Upon installation of a
> >>new Linux VM, make sure you change the
> >>time-zone from the default UTC to your local value (see the section
> >>called
> >>³Release Notes² for specific distribution
> >>instructions).
> >>To set individual Linux VMs to maintain independent times
> >>1. From a root prompt on the VM, run the command: echo 1 >
> >>/proc/sys/xen/independent_wallclock
> >>2. This can be persisted across reboots by changing the /etc/sysctl.conf
> >>configuration file and adding:
> >># Set independent wall clock time
> >>xen.independent_wallclock=1
> >>3. As a third alternative, independent_wallclock=1 may also be passed as
> >>a
> >>boot parameter to the VM.
> >>"
> >>
> >>
> >>On 5/15/13 12:09 PM, "Chip Childers"  wrote:
> >>
> >>>On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
>  /proc/sys is not a regular filesystem and cannot be added to from the
>  shell.
>  Drivers need to add nodes into this filesystem.
> >>>
> >>>Backing up a bit...
> >>>
> >>>This is the current system VM image that the 4.1 software should be
> >>>using on Xen:
> >>>
> >>>http://download.cloud.com/templates/acton/acton-
> >>>systemvm-02062012.vhd.bz2
> >>>
> >>>Does this system VM have the correct drives installed to set
> >>>/proc/sys/xen to use the Dom0 time for sync?
> >>>
> >>>If so, then is this a devcloud only issue for Xen?  If that's the case,
> >>>then we shouldn't block a release to simply improve things.
> >>>
> >>>We do know that a patch for kvm was needed.
> >>>
> >>>We still don't have any
> >>>answer or comments about the VMware system VM (specifically this
> >>>template:  http://download.cloud.com/templates/burbank/burbank-
> >>>systemvm-08012012.ova
> >>>
> >>
> >
> 

Re: [ACS41] System VMs not syncing time - does this block the release?

[John Burwell]

Chiradeep,

As I think thought it, I don't think a documentation note will sufficient 
because the SSVM can be destroyed and respawned by CloudStack without 
intervention by a human.  Therefore, we can get into a situation where an 
operator installs and configures NTP, and then at some point in the future, the 
SSVM gets respawned and clocks drift.  The worst part about this scenario is 
that the failures for S3 sync become silent since we have no mechanism to 
surface the failure to a dashboard or monitoring system.

How much effort (i.e. hours/days) would it be to build a new image?

Thanks,
-John

On May 15, 2013, at 4:59 PM, Chiradeep Vittal  
wrote:

> Did some further digging around as to why
> /proc/sys/xen/independent_wallclock is not there on the Debian system vm.
> 
> TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
> PV kernel, not a specialized paravirt kernel). To eliminate
> special-casing, the independent_wallclock feature was dropped. However,
> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
> required on ANY domU. So the clock on the domU is only sync'ed at domU
> creation time. I suspect Citrix XenServer might have some workaround
> 
> 
> http://www.gossamer-threads.com/lists/xen/users/234750
> 
> 
> What this means is that we *may* need to run ntp on system vms on Xen.
> This is of course a non-trivial change involving updates to systemvms.
> 
> I suspect that it does not matter much for virtual routers or console
> proxy vms.
> 
> We could have an advisory that for those users that care (e.g., those
> using S3 sync) that they need to run ntp after the SSVM has been created.
> I.e, login to the SSVM and run
> apt-get update
> apt-get install ntp
> 
> --
> Chiradeep
> 
> 
> 
> On 5/15/13 1:11 PM, "Chiradeep Vittal"  wrote:
> 
>> For VMWare, the command
>> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
>> patch for /etc/init.d/cloud-early-config to enable it
>> 
>> 
>> On 5/15/13 12:23 PM, "Chiradeep Vittal" 
>> wrote:
>> 
>>> I am not sure why it is missing, but I will refer to
>>> Citrix XenServer 6.0 Virtual Machine Installation Guide
>>> 
>>> http://s.apache.org/YGn
>>> 
>>> And quote
>>> "Time Handling in Linux VMs
>>> By default, the clocks in a Linux VM are synchronized to the clock
>>> running
>>> on the control domain, and cannot be
>>> independently changed. This mode is a convenient default, since only the
>>> control domain needs to be running the
>>> NTP service to keep accurate time across all VMs. Upon installation of a
>>> new Linux VM, make sure you change the
>>> time-zone from the default UTC to your local value (see the section
>>> called
>>> ³Release Notes² for specific distribution
>>> instructions).
>>> To set individual Linux VMs to maintain independent times
>>> 1. From a root prompt on the VM, run the command: echo 1 >
>>> /proc/sys/xen/independent_wallclock
>>> 2. This can be persisted across reboots by changing the /etc/sysctl.conf
>>> configuration file and adding:
>>> # Set independent wall clock time
>>> xen.independent_wallclock=1
>>> 3. As a third alternative, independent_wallclock=1 may also be passed as
>>> a
>>> boot parameter to the VM.
>>> "
>>> 
>>> 
>>> On 5/15/13 12:09 PM, "Chip Childers"  wrote:
>>> 
 On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
> /proc/sys is not a regular filesystem and cannot be added to from the
> shell.
> Drivers need to add nodes into this filesystem.
 
 Backing up a bit...
 
 This is the current system VM image that the 4.1 software should be
 using on Xen:
 
 http://download.cloud.com/templates/acton/acton-
 systemvm-02062012.vhd.bz2
 
 Does this system VM have the correct drives installed to set
 /proc/sys/xen to use the Dom0 time for sync?
 
 If so, then is this a devcloud only issue for Xen?  If that's the case,
 then we shouldn't block a release to simply improve things.
 
 We do know that a patch for kvm was needed.
 
 We still don't have any
 answer or comments about the VMware system VM (specifically this
 template:  http://download.cloud.com/templates/burbank/burbank-
 systemvm-08012012.ova
 
>>> 
>> 
> 

Re: [ACS41] System VMs not syncing time - does this block the release?

[John Burwell]

All,

One other point to consider is that we only want NTP running for Xen system 
VMs.  Running NTP and VMWare Tools together causes big problems because the two 
fight each other to sync the clock.  I don;t know about KVM, but I wouldn't be 
surprised if it doesn't have same types of problems.

Thanks,
-John

On May 15, 2013, at 5:07 PM, John Burwell  wrote:

> Chiradeep,
> 
> As I think thought it, I don't think a documentation note will sufficient 
> because the SSVM can be destroyed and respawned by CloudStack without 
> intervention by a human.  Therefore, we can get into a situation where an 
> operator installs and configures NTP, and then at some point in the future, 
> the SSVM gets respawned and clocks drift.  The worst part about this scenario 
> is that the failures for S3 sync become silent since we have no mechanism to 
> surface the failure to a dashboard or monitoring system.
> 
> How much effort (i.e. hours/days) would it be to build a new image?
> 
> Thanks,
> -John
> 
> On May 15, 2013, at 4:59 PM, Chiradeep Vittal  
> wrote:
> 
>> Did some further digging around as to why
>> /proc/sys/xen/independent_wallclock is not there on the Debian system vm.
>> 
>> TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
>> PV kernel, not a specialized paravirt kernel). To eliminate
>> special-casing, the independent_wallclock feature was dropped. However,
>> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
>> required on ANY domU. So the clock on the domU is only sync'ed at domU
>> creation time. I suspect Citrix XenServer might have some workaround
>> 
>> 
>> http://www.gossamer-threads.com/lists/xen/users/234750
>> 
>> 
>> What this means is that we *may* need to run ntp on system vms on Xen.
>> This is of course a non-trivial change involving updates to systemvms.
>> 
>> I suspect that it does not matter much for virtual routers or console
>> proxy vms.
>> 
>> We could have an advisory that for those users that care (e.g., those
>> using S3 sync) that they need to run ntp after the SSVM has been created.
>> I.e, login to the SSVM and run
>> apt-get update
>> apt-get install ntp
>> 
>> --
>> Chiradeep
>> 
>> 
>> 
>> On 5/15/13 1:11 PM, "Chiradeep Vittal"  wrote:
>> 
>>> For VMWare, the command
>>> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
>>> patch for /etc/init.d/cloud-early-config to enable it
>>> 
>>> 
>>> On 5/15/13 12:23 PM, "Chiradeep Vittal" 
>>> wrote:
>>> 
 I am not sure why it is missing, but I will refer to
 Citrix XenServer 6.0 Virtual Machine Installation Guide
 
 http://s.apache.org/YGn
 
 And quote
 "Time Handling in Linux VMs
 By default, the clocks in a Linux VM are synchronized to the clock
 running
 on the control domain, and cannot be
 independently changed. This mode is a convenient default, since only the
 control domain needs to be running the
 NTP service to keep accurate time across all VMs. Upon installation of a
 new Linux VM, make sure you change the
 time-zone from the default UTC to your local value (see the section
 called
 ³Release Notes² for specific distribution
 instructions).
 To set individual Linux VMs to maintain independent times
 1. From a root prompt on the VM, run the command: echo 1 >
 /proc/sys/xen/independent_wallclock
 2. This can be persisted across reboots by changing the /etc/sysctl.conf
 configuration file and adding:
 # Set independent wall clock time
 xen.independent_wallclock=1
 3. As a third alternative, independent_wallclock=1 may also be passed as
 a
 boot parameter to the VM.
 "
 
 
 On 5/15/13 12:09 PM, "Chip Childers"  wrote:
 
> On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
>> /proc/sys is not a regular filesystem and cannot be added to from the
>> shell.
>> Drivers need to add nodes into this filesystem.
> 
> Backing up a bit...
> 
> This is the current system VM image that the 4.1 software should be
> using on Xen:
> 
> http://download.cloud.com/templates/acton/acton-
> systemvm-02062012.vhd.bz2
> 
> Does this system VM have the correct drives installed to set
> /proc/sys/xen to use the Dom0 time for sync?
> 
> If so, then is this a devcloud only issue for Xen?  If that's the case,
> then we shouldn't block a release to simply improve things.
> 
> We do know that a patch for kvm was needed.
> 
> We still don't have any
> answer or comments about the VMware system VM (specifically this
> template:  http://download.cloud.com/templates/burbank/burbank-
> systemvm-08012012.ova
> 
 
>>> 
>> 
> 

RE: System vm template build failed in Jenkins.cloudstack.org

[Rayees Namathponnan]

Created blocker ticket to track this 
https://issues.apache.org/jira/browse/CLOUDSTACK-2521


Anyone can please take a look on this ? we don't have latest template after 
below fix

https://issues.apache.org/jira/browse/CLOUDSTACK-2324


Regards,
Rayees

From: Rayees Namathponnan
Sent: Tuesday, May 14, 2013 7:29 AM
To: dev@cloudstack.apache.org
Subject: System vm template build failed in Jenkins.cloudstack.org

Hi,

System vm template creation build failed in Jenkins.cloudstack.org,  not sure 
who is responsible for this to fix ?

http://jenkins.cloudstack.org/view/master/job/build-systemvm-master/

Regards,
Rayees

Re: Firewall rule question

[Ahmad Emneina]

I'm with Prasanna here, the behavioral inconsistency is baffling. How would
we document the differentiation at the api level?


On Wed, May 15, 2013 at 12:34 AM, Koushik Das wrote:

>
>
> > -Original Message-
> > From: Prasanna Santhanam [mailto:t...@apache.org]
> > Sent: Wednesday, May 15, 2013 12:25 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: Firewall rule question
> >
> > On Wed, May 15, 2013 at 06:43:44AM +, Koushik Das wrote:
> > > Prasanna,
> > >
> > > Interesting point. On one hand there is consistency and on the other
> > > hand flexibility. Not sure if the framework should be restrictive or
> > > as flexible as possible but I personally like the latter option.
> >
> > Sorry, don't mean to hijack this thread:
> >
> > But I'm not sure of the flexibility you speak of, is it given to the
> tenant? If I
> > was a tenant using a network offering using the VR and had programmed my
> > FW rules accordingly. On upgrading my network offering to say, a PaloAlto
> > FW, if all my instances suddenly become unreachable, I don't see that as
> > favourable behaviour.
> >
>
> The tenant will have flexibility to select network offering but the admin
> will decide what all offerings to provide based on
> Capabilities/limitations etc. Let's take the e.g. of VR and PaloAlto as
> firewall service provider. Currently the framework
> allows certain type of firewall rules which works with VR provider. Now
> say PaloAlto is more restrictive and in order to
> accommodate it more validations are added in the framework. The use case
> you have mentioned above is still broken for
> existing networks.
>
> When there are varied external devices, it's a difficult problem to arrive
> at a common validation layer. It's possible that
> every time a new device is integrated you may find the validations too
> restrictive or too flexible.
>
> > --
> > Prasanna.,
> >
> > 
> > Powered by BigRock.com
>
>

Re: [ACS41] System VMs not syncing time - does this block the release?

[Chiradeep Vittal]

Sure, I agree. But I'd also point out that for the vast majority of
CloudStack users (4.1 at least), this is not going to be an issue. I
suggest deferring this to 4.1.1
A new template (easy or not) does require a full regression QA round.

On 5/15/13 2:07 PM, "John Burwell"  wrote:

>Chiradeep,
>
>As I think thought it, I don't think a documentation note will sufficient
>because the SSVM can be destroyed and respawned by CloudStack without
>intervention by a human.  Therefore, we can get into a situation where an
>operator installs and configures NTP, and then at some point in the
>future, the SSVM gets respawned and clocks drift.  The worst part about
>this scenario is that the failures for S3 sync become silent since we
>have no mechanism to surface the failure to a dashboard or monitoring
>system.
>
>How much effort (i.e. hours/days) would it be to build a new image?
>
>Thanks,
>-John
>
>On May 15, 2013, at 4:59 PM, Chiradeep Vittal
> wrote:
>
>> Did some further digging around as to why
>> /proc/sys/xen/independent_wallclock is not there on the Debian system
>>vm.
>> 
>> TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
>> PV kernel, not a specialized paravirt kernel). To eliminate
>> special-casing, the independent_wallclock feature was dropped. However,
>> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
>> required on ANY domU. So the clock on the domU is only sync'ed at domU
>> creation time. I suspect Citrix XenServer might have some workaround
>> 
>> 
>> http://www.gossamer-threads.com/lists/xen/users/234750
>> 
>> 
>> What this means is that we *may* need to run ntp on system vms on Xen.
>> This is of course a non-trivial change involving updates to systemvms.
>> 
>> I suspect that it does not matter much for virtual routers or console
>> proxy vms.
>> 
>> We could have an advisory that for those users that care (e.g., those
>> using S3 sync) that they need to run ntp after the SSVM has been
>>created.
>> I.e, login to the SSVM and run
>> apt-get update
>> apt-get install ntp
>> 
>> --
>> Chiradeep
>> 
>> 
>> 
>> On 5/15/13 1:11 PM, "Chiradeep Vittal" 
>>wrote:
>> 
>>> For VMWare, the command
>>> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
>>> patch for /etc/init.d/cloud-early-config to enable it
>>> 
>>> 
>>> On 5/15/13 12:23 PM, "Chiradeep Vittal" 
>>> wrote:
>>> 
 I am not sure why it is missing, but I will refer to
 Citrix XenServer 6.0 Virtual Machine Installation Guide
 
 http://s.apache.org/YGn
 
 And quote
 "Time Handling in Linux VMs
 By default, the clocks in a Linux VM are synchronized to the clock
 running
 on the control domain, and cannot be
 independently changed. This mode is a convenient default, since only
the
 control domain needs to be running the
 NTP service to keep accurate time across all VMs. Upon installation
of a
 new Linux VM, make sure you change the
 time-zone from the default UTC to your local value (see the section
 called
 ³Release Notes² for specific distribution
 instructions).
 To set individual Linux VMs to maintain independent times
 1. From a root prompt on the VM, run the command: echo 1 >
 /proc/sys/xen/independent_wallclock
 2. This can be persisted across reboots by changing the
/etc/sysctl.conf
 configuration file and adding:
 # Set independent wall clock time
 xen.independent_wallclock=1
 3. As a third alternative, independent_wallclock=1 may also be passed
as
 a
 boot parameter to the VM.
 "
 
 
 On 5/15/13 12:09 PM, "Chip Childers" 
wrote:
 
> On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
>> /proc/sys is not a regular filesystem and cannot be added to from
>>the
>> shell.
>> Drivers need to add nodes into this filesystem.
> 
> Backing up a bit...
> 
> This is the current system VM image that the 4.1 software should be
> using on Xen:
> 
> http://download.cloud.com/templates/acton/acton-
> systemvm-02062012.vhd.bz2
> 
> Does this system VM have the correct drives installed to set
> /proc/sys/xen to use the Dom0 time for sync?
> 
> If so, then is this a devcloud only issue for Xen?  If that's the
>case,
> then we shouldn't block a release to simply improve things.
> 
> We do know that a patch for kvm was needed.
> 
> We still don't have any
> answer or comments about the VMware system VM (specifically this
> template:  http://download.cloud.com/templates/burbank/burbank-
> systemvm-08012012.ova
> 
 
>>> 
>> 
>

Re: Storage as a Service

[Mike Tutkowski]

Thanks, everyone!

A person is actually presenting on Riak CS right now. :)


On Wed, May 15, 2013 at 2:53 PM, John Burwell  wrote:

> Chip,
>
> Disclaimer: I am an employee of Basho, makers of Riak CS.
>
> +1.
>
> The S3 compatibility layer provided by CloudStack lacks may of S3's
> availability guarantees since it stored the index in a MySQL table, and
> files in a single folder on disk.  There are certainly ways to increase the
> reliability of this implementation (e.g. placing the file storage on a SAN,
> clustering MySQL).  However, the operational complexity would be far
> greater than Riak CS, and would still lack horizontal scalability on
> commodity hardware (i.e. much lower cost per GB).  If you interested in
> drilling in more deeply, I would be happy to have an offline conversation
> regarding Riak CS.
>
> Thanks,
> -John
>
> On May 15, 2013, at 4:47 PM, Chip Childers 
> wrote:
>
> > CloudStack isn't an object storage system really...  the best bet, if
> > that's the focus of what he wants to do, is to look at things like Riak
> > CS, Swift, Ceph, etc...
> >
> >
> > On Wed, May 15, 2013 at 02:13:29PM -0600, Mike Tutkowski wrote:
> >> Hi everyone,
> >>
> >> I'm at Build a Cloud Day today in San Francisco and had a question from
> a
> >> potential CloudStack user about Storage as a Service.
> >>
> >> His business would like to sell Storage as a Service (like Amazon S3)
> and
> >> would like to know if CloudStack was an applicable solution for this use
> >> case.
> >>
> >> Thoughts?
> >>
> >> Thanks!
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkow...@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud
> >> *™*
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: [ACS41] System VMs not syncing time - does this block the release?

[John Burwell]

Chiradeep,

I disagree regarding the impact of this issue.  Anyone running an SSVM will be 
affected by this issue because clocks will eventually drift (sooner rather than 
later) and when they do, any timestamps rendered by a system VM will unreliable 
(e.g. files creation and modified times, log entries, etc).  When I put my sys 
admin hat on, that type behavior would 4.1 an unacceptable release to me.

Thanks,
-John

On May 15, 2013, at 5:24 PM, Chiradeep Vittal  
wrote:

> Sure, I agree. But I'd also point out that for the vast majority of
> CloudStack users (4.1 at least), this is not going to be an issue. I
> suggest deferring this to 4.1.1
> A new template (easy or not) does require a full regression QA round.
> 
> On 5/15/13 2:07 PM, "John Burwell"  wrote:
> 
>> Chiradeep,
>> 
>> As I think thought it, I don't think a documentation note will sufficient
>> because the SSVM can be destroyed and respawned by CloudStack without
>> intervention by a human.  Therefore, we can get into a situation where an
>> operator installs and configures NTP, and then at some point in the
>> future, the SSVM gets respawned and clocks drift.  The worst part about
>> this scenario is that the failures for S3 sync become silent since we
>> have no mechanism to surface the failure to a dashboard or monitoring
>> system.
>> 
>> How much effort (i.e. hours/days) would it be to build a new image?
>> 
>> Thanks,
>> -John
>> 
>> On May 15, 2013, at 4:59 PM, Chiradeep Vittal
>>  wrote:
>> 
>>> Did some further digging around as to why
>>> /proc/sys/xen/independent_wallclock is not there on the Debian system
>>> vm.
>>> 
>>> TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
>>> PV kernel, not a specialized paravirt kernel). To eliminate
>>> special-casing, the independent_wallclock feature was dropped. However,
>>> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
>>> required on ANY domU. So the clock on the domU is only sync'ed at domU
>>> creation time. I suspect Citrix XenServer might have some workaround
>>> 
>>> 
>>> http://www.gossamer-threads.com/lists/xen/users/234750
>>> 
>>> 
>>> What this means is that we *may* need to run ntp on system vms on Xen.
>>> This is of course a non-trivial change involving updates to systemvms.
>>> 
>>> I suspect that it does not matter much for virtual routers or console
>>> proxy vms.
>>> 
>>> We could have an advisory that for those users that care (e.g., those
>>> using S3 sync) that they need to run ntp after the SSVM has been
>>> created.
>>> I.e, login to the SSVM and run
>>> apt-get update
>>> apt-get install ntp
>>> 
>>> --
>>> Chiradeep
>>> 
>>> 
>>> 
>>> On 5/15/13 1:11 PM, "Chiradeep Vittal" 
>>> wrote:
>>> 
 For VMWare, the command
 vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
 patch for /etc/init.d/cloud-early-config to enable it
 
 
 On 5/15/13 12:23 PM, "Chiradeep Vittal" 
 wrote:
 
> I am not sure why it is missing, but I will refer to
> Citrix XenServer 6.0 Virtual Machine Installation Guide
> 
> http://s.apache.org/YGn
> 
> And quote
> "Time Handling in Linux VMs
> By default, the clocks in a Linux VM are synchronized to the clock
> running
> on the control domain, and cannot be
> independently changed. This mode is a convenient default, since only
> the
> control domain needs to be running the
> NTP service to keep accurate time across all VMs. Upon installation
> of a
> new Linux VM, make sure you change the
> time-zone from the default UTC to your local value (see the section
> called
> ³Release Notes² for specific distribution
> instructions).
> To set individual Linux VMs to maintain independent times
> 1. From a root prompt on the VM, run the command: echo 1 >
> /proc/sys/xen/independent_wallclock
> 2. This can be persisted across reboots by changing the
> /etc/sysctl.conf
> configuration file and adding:
> # Set independent wall clock time
> xen.independent_wallclock=1
> 3. As a third alternative, independent_wallclock=1 may also be passed
> as
> a
> boot parameter to the VM.
> "
> 
> 
> On 5/15/13 12:09 PM, "Chip Childers" 
> wrote:
> 
>> On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
>>> /proc/sys is not a regular filesystem and cannot be added to from
>>> the
>>> shell.
>>> Drivers need to add nodes into this filesystem.
>> 
>> Backing up a bit...
>> 
>> This is the current system VM image that the 4.1 software should be
>> using on Xen:
>> 
>> http://download.cloud.com/templates/acton/acton-
>> systemvm-02062012.vhd.bz2
>> 
>> Does this system VM have the correct drives installed to set
>> /proc/sys/xen to use the Dom0 time for sync?
>> 
>> If so, then is this a devcloud only issue for Xen?  If that's the
>> case,
>>>

Re: [ACS41] System VMs not syncing time - does this block the release?

[Wido den Hollander]

On 05/15/2013 11:33 PM, John Burwell wrote:
> Chiradeep,
> 
> I disagree regarding the impact of this issue.  Anyone running an SSVM will 
> be affected by this issue because clocks will eventually drift (sooner rather 
> than later) and when they do, any timestamps rendered by a system VM will 
> unreliable (e.g. files creation and modified times, log entries, etc).  When 
> I put my sys admin hat on, that type behavior would 4.1 an unacceptable 
> release to me.
> 

Ack. Clocks should always be on time. No matter what a machine is doing.
Have the correct time is crucial.

Think about log lines, debugging, etc, etc.

Nothing wrong with running NTP in KVM System VM btw.

Wido

> Thanks,
> -John
> 
> On May 15, 2013, at 5:24 PM, Chiradeep Vittal  
> wrote:
> 
>> Sure, I agree. But I'd also point out that for the vast majority of
>> CloudStack users (4.1 at least), this is not going to be an issue. I
>> suggest deferring this to 4.1.1
>> A new template (easy or not) does require a full regression QA round.
>>
>> On 5/15/13 2:07 PM, "John Burwell"  wrote:
>>
>>> Chiradeep,
>>>
>>> As I think thought it, I don't think a documentation note will sufficient
>>> because the SSVM can be destroyed and respawned by CloudStack without
>>> intervention by a human.  Therefore, we can get into a situation where an
>>> operator installs and configures NTP, and then at some point in the
>>> future, the SSVM gets respawned and clocks drift.  The worst part about
>>> this scenario is that the failures for S3 sync become silent since we
>>> have no mechanism to surface the failure to a dashboard or monitoring
>>> system.
>>>
>>> How much effort (i.e. hours/days) would it be to build a new image?
>>>
>>> Thanks,
>>> -John
>>>
>>> On May 15, 2013, at 4:59 PM, Chiradeep Vittal
>>>  wrote:
>>>
 Did some further digging around as to why
 /proc/sys/xen/independent_wallclock is not there on the Debian system
 vm.

 TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
 PV kernel, not a specialized paravirt kernel). To eliminate
 special-casing, the independent_wallclock feature was dropped. However,
 this means that the domU clock is NO LONGER synch'ed to dom0 and NTP is
 required on ANY domU. So the clock on the domU is only sync'ed at domU
 creation time. I suspect Citrix XenServer might have some workaround


 http://www.gossamer-threads.com/lists/xen/users/234750


 What this means is that we *may* need to run ntp on system vms on Xen.
 This is of course a non-trivial change involving updates to systemvms.

 I suspect that it does not matter much for virtual routers or console
 proxy vms.

 We could have an advisory that for those users that care (e.g., those
 using S3 sync) that they need to run ntp after the SSVM has been
 created.
 I.e, login to the SSVM and run
 apt-get update
 apt-get install ntp

 --
 Chiradeep



 On 5/15/13 1:11 PM, "Chiradeep Vittal" 
 wrote:

> For VMWare, the command
> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
> patch for /etc/init.d/cloud-early-config to enable it
>
>
> On 5/15/13 12:23 PM, "Chiradeep Vittal" 
> wrote:
>
>> I am not sure why it is missing, but I will refer to
>> Citrix XenServer 6.0 Virtual Machine Installation Guide
>>
>> http://s.apache.org/YGn
>>
>> And quote
>> "Time Handling in Linux VMs
>> By default, the clocks in a Linux VM are synchronized to the clock
>> running
>> on the control domain, and cannot be
>> independently changed. This mode is a convenient default, since only
>> the
>> control domain needs to be running the
>> NTP service to keep accurate time across all VMs. Upon installation
>> of a
>> new Linux VM, make sure you change the
>> time-zone from the default UTC to your local value (see the section
>> called
>> ³Release Notes² for specific distribution
>> instructions).
>> To set individual Linux VMs to maintain independent times
>> 1. From a root prompt on the VM, run the command: echo 1 >
>> /proc/sys/xen/independent_wallclock
>> 2. This can be persisted across reboots by changing the
>> /etc/sysctl.conf
>> configuration file and adding:
>> # Set independent wall clock time
>> xen.independent_wallclock=1
>> 3. As a third alternative, independent_wallclock=1 may also be passed
>> as
>> a
>> boot parameter to the VM.
>> "
>>
>>
>> On 5/15/13 12:09 PM, "Chip Childers" 
>> wrote:
>>
>>> On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
 /proc/sys is not a regular filesystem and cannot be added to from
 the
 shell.
 Drivers need to add nodes into this filesystem.
>>>
>>> Backing up a bit...
>>>
>>> This is the current system VM im

Re: [ACS41] System VMs not syncing time - does this block the release?

[Chiradeep Vittal]

Well, I disagree, from the perspective of hundreds of production clouds.
No harm has been perceived in those clouds due to this defect. If they can
live with it for several years, then they can live with it for a few more
months.

On 5/15/13 2:35 PM, "Wido den Hollander"  wrote:

>
>
>On 05/15/2013 11:33 PM, John Burwell wrote:
>> Chiradeep,
>> 
>> I disagree regarding the impact of this issue.  Anyone running an SSVM
>>will be affected by this issue because clocks will eventually drift
>>(sooner rather than later) and when they do, any timestamps rendered by
>>a system VM will unreliable (e.g. files creation and modified times, log
>>entries, etc).  When I put my sys admin hat on, that type behavior would
>>4.1 an unacceptable release to me.
>> 
>
>Ack. Clocks should always be on time. No matter what a machine is doing.
>Have the correct time is crucial.
>
>Think about log lines, debugging, etc, etc.
>
>Nothing wrong with running NTP in KVM System VM btw.
>
>Wido
>
>> Thanks,
>> -John
>> 
>> On May 15, 2013, at 5:24 PM, Chiradeep Vittal
>> wrote:
>> 
>>> Sure, I agree. But I'd also point out that for the vast majority of
>>> CloudStack users (4.1 at least), this is not going to be an issue. I
>>> suggest deferring this to 4.1.1
>>> A new template (easy or not) does require a full regression QA round.
>>>
>>> On 5/15/13 2:07 PM, "John Burwell"  wrote:
>>>
 Chiradeep,

 As I think thought it, I don't think a documentation note will
sufficient
 because the SSVM can be destroyed and respawned by CloudStack without
 intervention by a human.  Therefore, we can get into a situation
where an
 operator installs and configures NTP, and then at some point in the
 future, the SSVM gets respawned and clocks drift.  The worst part
about
 this scenario is that the failures for S3 sync become silent since we
 have no mechanism to surface the failure to a dashboard or monitoring
 system.

 How much effort (i.e. hours/days) would it be to build a new image?

 Thanks,
 -John

 On May 15, 2013, at 4:59 PM, Chiradeep Vittal
  wrote:

> Did some further digging around as to why
> /proc/sys/xen/independent_wallclock is not there on the Debian system
> vm.
>
> TLDR; the kernel is PvOps (I.e., just a regular kernel that works
>like a
> PV kernel, not a specialized paravirt kernel). To eliminate
> special-casing, the independent_wallclock feature was dropped.
>However,
> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP
>is
> required on ANY domU. So the clock on the domU is only sync'ed at
>domU
> creation time. I suspect Citrix XenServer might have some workaround
>
>
> http://www.gossamer-threads.com/lists/xen/users/234750
>
>
> What this means is that we *may* need to run ntp on system vms on
>Xen.
> This is of course a non-trivial change involving updates to
>systemvms.
>
> I suspect that it does not matter much for virtual routers or console
> proxy vms.
>
> We could have an advisory that for those users that care (e.g., those
> using S3 sync) that they need to run ntp after the SSVM has been
> created.
> I.e, login to the SSVM and run
> apt-get update
> apt-get install ntp
>
> --
> Chiradeep
>
>
>
> On 5/15/13 1:11 PM, "Chiradeep Vittal" 
> wrote:
>
>> For VMWare, the command
>> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit
>>a
>> patch for /etc/init.d/cloud-early-config to enable it
>>
>>
>> On 5/15/13 12:23 PM, "Chiradeep Vittal"
>>
>> wrote:
>>
>>> I am not sure why it is missing, but I will refer to
>>> Citrix XenServer 6.0 Virtual Machine Installation Guide
>>>
>>> http://s.apache.org/YGn
>>>
>>> And quote
>>> "Time Handling in Linux VMs
>>> By default, the clocks in a Linux VM are synchronized to the clock
>>> running
>>> on the control domain, and cannot be
>>> independently changed. This mode is a convenient default, since
>>>only
>>> the
>>> control domain needs to be running the
>>> NTP service to keep accurate time across all VMs. Upon installation
>>> of a
>>> new Linux VM, make sure you change the
>>> time-zone from the default UTC to your local value (see the section
>>> called
>>> ³Release Notes² for specific distribution
>>> instructions).
>>> To set individual Linux VMs to maintain independent times
>>> 1. From a root prompt on the VM, run the command: echo 1 >
>>> /proc/sys/xen/independent_wallclock
>>> 2. This can be persisted across reboots by changing the
>>> /etc/sysctl.conf
>>> configuration file and adding:
>>> # Set independent wall clock time
>>> xen.independent_wallclock=1
>>> 3. As a third alternative, independent_wallclock=1 may also be
>>>pas

DC CloudStack Meetup

[John Burwell]

All,

Basho Technologies [1] will be hosting Chip Childers for a meet up entitled 
"Cloudstack – What Is It and What’s Next"  on 29 May 2013 at our Herndon, VA 
office.  For more information and/or to RSVP, please see the meetup.com 
announcement [2].

Thanks,
-John

[1]: http://www.basho.com
[2]: http://www.meetup.com/Riak-DC/events/118224772/

Re: [ACS41] System VMs not syncing time - does this block the release?

[Marcus Sorensen]

>
> Nothing wrong with running NTP in KVM System VM btw.
>
> Wido
>

I think they are right that it will be fighting with the hypervisor
synced clock. It might not really pose much of an issue technically,
though.

RE: [ACS41] Current blocker status

[Animesh Chaturvedi]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Wednesday, May 15, 2013 12:43 PM
> To: dev@cloudstack.apache.org
> Subject: [ACS41] Current blocker status
> 
> The following are currently blocking a new RC for 4.1.0:
> 
> CLOUDSTACK-2039
> Improve console access security with 128-bit AES encryption and securely-
> randomized key generation
> 
> This needs to be addressed.  Kelven, do you mind reviewing Wido's error?
[Animesh>] Kelven is offline for next two weeks, we will need someone else to 
look into this issue
> 
> 
> CLOUDSTACK-2463
> CS Upgrade 2.Upgrade2.14 to 4.1.0 failed due to no public network found
> (configuration : advanced network with security groups)
> 
> This is being discussed in another thread.
> 
[Animesh>] I will respond in the other thread
> 
> CLOUDSTACK-2492
> System VM Clock Drift
> 
> This is also being discussed in another thread.
> 
> 
> CLOUDSTACK-2516
> Create User API compability broken now
> 
> Nobody has taken this on yet.
[Animesh>] I will update the JIRA ticket, looks like this was changed by Kishan 
to use clear text password. I think functionality wise it will still work but 
will cause double encryption. We will have to wait for Kishan  to respond on 
this.

RE: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 vs 4.2

[Anthony Xu]

I'll work on 4.2, after that, people can merge it to 4.1.

Anthony

-Original Message-
From: Chip Childers [mailto:chip.child...@sungard.com] 
Sent: Wednesday, May 15, 2013 9:25 AM
To: dev@cloudstack.apache.org
Subject: Re: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 vs 4.2

On Wed, May 15, 2013 at 09:20:06AM -0700, Alex Huang wrote:
> I'm a very strong believer that CloudStack releases should always be 
> upgradable from previous releases.  We can't strand our user base on a 
> previous release.

Agreed conceptually.  Let's be clear though...  these users have been stranded 
for many versions now.

I'm OK with focusing on fixing this for 4.1.0...  but we need someone(s) to 
commit to doing the work.

> 
> --Alex
> 
> > -Original Message-
> > From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
> > Sent: Wednesday, May 15, 2013 8:28 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [ACS41] Discuss CLOUDSTACK-2463 being resolved in 4.1 
> > vs 4.2
> > 
> > Half of our platforms are on 2.2.14 (advanced zone with security groups).
> > These platform work well. We are looking for a way to upgrade to 4.* 
> > for more functionalities, so that we do not need to take the 
> > difference of cloudstack version into account in development.
> > 
> > As I know, the citrix guys are working on this. Jessica Wang said 
> > the feature will be merged into master branch soon.It looks the coding is 
> > almost done.
> > 
> > I hope this feature could be included in 4.1, of course. However, we 
> > also need some days for testing and bug fix. It means cloudstack 4.1 
> > will delay for uncertain days (it is very bad, right?). It is a difficult 
> > choice.
> > 
> > I do not know how many companies are using 2.2.14  (advanced zone 
> > with security groups) and eager to upgrade. I will join the dev and 
> > testing if needed.
> > 
> > 
> > 2013/5/15 Chip Childers 
> > 
> > > Sebastian re-opened CLOUDSTACK-2463 due to users wanting to 
> > > upgrade from 2.x to 4.1.  This relates to the security groups 
> > > feature being available when using VLANs in an advanced networking 
> > > zone.  This feature was apparently broken in the 3.x series, and 
> > > is not slated to be reintroduced until 4.2.
> > >
> > > This is a horrible situation, and one that we've now encountered 
> > > for a third time.
> > >
> > > IMO, we have 2 very specific options:
> > >
> > > 1) We pull that new feature into 4.1, and do the relevant testing.
> > >
> > > 2) We do not pull that feature into 4.1, and release as is with a 
> > > strong message in the release notes highlighting that we know that 
> > > 2.x to 4.1 will not support it (and state that those users 
> > > requiring the feature should wait for 4.2).
> > >
> > > At this point, I don't have a preference.  We probably need to 
> > > understand the effort for (1), as well as understand who would do 
> > > that work (dev AND testing).
> > >
> > > Thoughts?
> > >
> > > -chip
> > >
> 

Re: System vm template build failed in Jenkins.cloudstack.org

[Chiradeep Vittal]

Fixed

On 5/15/13 2:14 PM, "Rayees Namathponnan" 
wrote:

>Created blocker ticket to track this
>https://issues.apache.org/jira/browse/CLOUDSTACK-2521
>
>
>Anyone can please take a look on this ? we don't have latest template
>after below fix
>
>https://issues.apache.org/jira/browse/CLOUDSTACK-2324
>
>
>Regards,
>Rayees
>
>From: Rayees Namathponnan
>Sent: Tuesday, May 14, 2013 7:29 AM
>To: dev@cloudstack.apache.org
>Subject: System vm template build failed in Jenkins.cloudstack.org
>
>Hi,
>
>System vm template creation build failed in Jenkins.cloudstack.org,  not
>sure who is responsible for this to fix ?
>
>http://jenkins.cloudstack.org/view/master/job/build-systemvm-master/
>
>Regards,
>Rayees

Re: [MERGE]PVLAN branch to MASTER(v2)

[Sheng Yang]

Merged.

--Sheng


On Tue, May 14, 2013 at 12:26 PM, Chip Childers
wrote:

> On Tue, May 14, 2013 at 12:05:21PM -0700, Sheng Yang wrote:
> > On Mon, May 13, 2013 at 6:58 PM, Chip Childers  >wrote:
> >
> > > On Mon, May 13, 2013 at 03:16:28PM -0700, Sheng Yang wrote:
> > > > FS at:
> > > >
> > >
> https://cwiki.apache.org/CLOUDSTACK/pvlan-for-isolation-within-a-vlan.html
> > > >
> > > > PVLAN branch has been under development for some time, and now the
> > > > functionality works on KVM and Xen, would be followed by VMware soon
> > > >
> > > > VM live migration is not supported so far. Code is ready basically,
> but I
> > > > am waiting for the fix of
> > > > https://issues.apache.org/jira/browse/CLOUDSTACK-1638 .
> > > >
> > > > We're using ovs/open flow to manipulate ingress/egress traffic to
> emulate
> > > > the isolation PVLAN function on KVM and Xen. The details are in the
> FS.
> > > >
> > > > The core code change is minimal and there is no DB change, because we
> > > took
> > > > advantage of "broadcast domain" to introduce "pvlan://" broadcast
> URI to
> > > > describe the primary and isolated PVLAN for the network.
> > > >
> > > > The code has passed the RAT, and no new dependency is added.
> > >
> > > How is the OpenFlow functionality controlled?  Via libvirt?
> > >
> >
> > No, CS would execute the ovs-ofctl command. Libvirt(in kvm) would in
> charge
> > of allocating vlan etc, but won't involve in openflow part.
> >
> > --Sheng
>
> Got it...  looked at the diff.  Thanks Sheng.
>

RE: [DISCUSS] include vhd-util in system vm template?

[Edison Su]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Wednesday, May 15, 2013 11:47 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] include vhd-util in system vm template?
> 
> On Wed, May 15, 2013 at 11:00:57AM -0700, Edison Su wrote:
> > Regarding to bug: https://issues.apache.org/jira/browse/CLOUDSTACK-
> 2481, there is no vhd-util installed in system vm template, thus certain
> storage related operations failed.
> > As discussed before, vhd-util binary is licensed under BSD and
> GPL(https://issues.apache.org/jira/browse/CLOUDSTACK-30), it should be
> ok to included in system vm template, right?
> 
> Edison - The issue isn't if we can include it in a built system VM really.  
> We are
> building those from a number of GPL projects (obviously including the OS
> itself).
> 
> The issue with the legal stuff is how we tie to that software and if we intend
> to distribute it from ASF infra.  As long as we don't store the utility in our
> version control, and do not actually ship it from ASF hardware, we can
> consider it OK IMO.  The inclusion of that utility will need to be part of the
> system VM build config, which we do store in the repo obviously.
I agree. We will not store that utility in ASF cloudstack git repository. And 
vhd-utility is same as any other GPL binary we already installed in system vm 
template. 

For example, in the system vm build 
script(tools/appliance/definitions/systemvmtemplate/postinstall.sh):

There is one line code like:
wget 
http://ftp.us.debian.org/debian/pool/main/h/haproxy/haproxy_1.4.8-1_i386.deb

then why we can't do the following:

wget http://some-place/vhd-util -O /bin/



> 
> We are shipping system VMs from
> the Citrix S3 account today, and will probably switch to Wido's hosting server
> (or similar) for the future.  This means that it's not an ASF issue...  and 
> frankly
> the OS being distributed itself would be a larger problem than what packages
> are installed in that OS.
> 
> Make sense?
> 
> -chip

RE: [DISCUSS] Should we be releasing -beta releases?

[Musayev, Ilya]

+1, perhaps I'm late to this thread,  but this makes lot of sense.


 Original message 
From: Pranav Saxena 
Date:
To: dev@cloudstack.apache.org,aemne...@gmail.com
Subject: RE: [DISCUSS] Should we be releasing -beta releases?


+1 to what Ahmad says here . Perfect reasoning .

There have been many users on the list asking for some capability /feature 
present in CloudStack when it's actually under development in the current 
release. Beta release would allow them to get a feel of it . Definitely , it 
would help to further refine any new feature further when actually tested in a 
production environment .

-Original Message-
From: Ahmad Emneina [mailto:aemne...@gmail.com]
Sent: Wednesday, May 15, 2013 12:07 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Should we be releasing -beta releases?

+1
I feel this allows for users who are chomping at the bit to get a hold of 
feature X. Tinker with feature X, expose bugs or use case issues well before an 
official release. Saves on the disappointment as well. ;)


On Tue, May 14, 2013 at 9:34 AM, Chip Childers wrote:

> On Tue, May 14, 2013 at 11:59:14AM -0400, David Nalley wrote:
> > Are you going to support upgrades from your Betas to release (and
> > betaN to betaN+1)?
> > If the answer is no, then there is no interest on my part. It's not
> > better than us producing nightly builds, or highlighting jenkins
> > builds.
>
> Perhaps doing a better job of highlighting nightly builds at key
> moments is the right answer to the problem I was trying to solve (more
> user testing of upgrades)?
>
> The beta idea comes with some overhead, and perhaps that overhead
> isn't worth the benefit (if there are other ways to achieve that
> goal).  And that's why I floated the idea...  to get reactions.
>
> >
> > On Tue, May 14, 2013 at 11:03 AM, Chip Childers
> >  wrote:
> > > On Tue, May 14, 2013 at 03:56:36PM +0100, Daan Hoogland wrote:
> > >> As a relative outsider;
> > >>
> > >> any branch that is not released yet is a beta release. Why make
> > >> it
> more
> > >> explicit. Wouldn't this add support burdon? Make a branch 'in beta'
> and
> > >> appoint a guard to make sure no new feartures but only fixes go
> > >> in
> (kind of
> > >> how you are working right now)
> > >
> > > So we do that today.  However, a "release" as a -beta will get
> > > more
> user
> > > attention eariler in our release cycle (at least that's my
> > > theory).  We need that user attention to help us ensure that upgrades 
> > > work.
> > >
> > >>
> > >> Daan
> > >>
> > >>
> > >> On Tue, May 14, 2013 at 3:49 PM, Joe Brockmeier 
> wrote:
> > >>
> > >> > On Tue, May 14, 2013, at 09:41 AM, Chip Childers wrote:
> > >> > > As a way to get more user feedback on our major feature
> > >> > > releases,
> what
> > >> > > does everyone think about releasing one or two -beta releases
> > >> > > for
> each
> > >> > > major feature release?
> > >> >
> > >> > Yes to beta releases. I know that users could test at any time,
> > >> > but
> we
> > >> > need explicit targets for users that say "now is a good time to
> > >> > test this and give feedback."
> > >> >
> > >> > +1
> > >> >
> > >> >
> > >> > Best,
> > >> >
> > >> > jzb
> > >> > --
> > >> > Joe Brockmeier
> > >> > j...@zonker.net
> > >> > Twitter: @jzb
> > >> > http://www.dissociatedpress.net/
> > >> >
> >
>

Re: [DISCUSS] include vhd-util in system vm template?

[David Nalley]

On Wed, May 15, 2013 at 9:19 PM, Edison Su  wrote:
>
>
>> -Original Message-
>> From: Chip Childers [mailto:chip.child...@sungard.com]
>> Sent: Wednesday, May 15, 2013 11:47 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DISCUSS] include vhd-util in system vm template?
>>
>> On Wed, May 15, 2013 at 11:00:57AM -0700, Edison Su wrote:
>> > Regarding to bug: https://issues.apache.org/jira/browse/CLOUDSTACK-
>> 2481, there is no vhd-util installed in system vm template, thus certain
>> storage related operations failed.
>> > As discussed before, vhd-util binary is licensed under BSD and
>> GPL(https://issues.apache.org/jira/browse/CLOUDSTACK-30), it should be
>> ok to included in system vm template, right?
>>
>> Edison - The issue isn't if we can include it in a built system VM really.  
>> We are
>> building those from a number of GPL projects (obviously including the OS
>> itself).
>>
>> The issue with the legal stuff is how we tie to that software and if we 
>> intend
>> to distribute it from ASF infra.  As long as we don't store the utility in 
>> our
>> version control, and do not actually ship it from ASF hardware, we can
>> consider it OK IMO.  The inclusion of that utility will need to be part of 
>> the
>> system VM build config, which we do store in the repo obviously.
> I agree. We will not store that utility in ASF cloudstack git repository. And 
> vhd-utility is same as any other GPL binary we already installed in system vm 
> template.
>
> For example, in the system vm build 
> script(tools/appliance/definitions/systemvmtemplate/postinstall.sh):
>
> There is one line code like:
> wget 
> http://ftp.us.debian.org/debian/pool/main/h/haproxy/haproxy_1.4.8-1_i386.deb
>
> then why we can't do the following:
>
> wget http://some-place/vhd-util -O /bin/
>
>

This doesn't seem to pollute our releases, so I don't see a difference
between haproxy and vhd-util, or a problem in general.

--David

Re: Review Request: (CLOUDSTACK-962) CloudStack usage server didn't store data to cloud_usage in correct time

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8971/#review20634
---


Commit 3fa8fda37c8b9fac1e38b57d28d23917106fe02b in branch 
refs/heads/object_store from Edison Su 
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=3fa8fda ]

CLOUDSTACK-962: Because of the same reason to this (CLOUDSTACK-685)
https://reviews.apache.org/r/11157/

Signed-off-by: Chip Childers 


- ASF Subversion and Git Services


On March 22, 2013, 9 a.m., Wei Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8971/
> ---
> 
> (Updated March 22, 2013, 9 a.m.)
> 
> 
> Review request for cloudstack, Kishan Kavala, Joe Brockmeier, and Rohit Yadav.
> 
> 
> Description
> ---
> 
> The usage sever does not handle data in the correct time. For example, it 
> regards the data traffic between 9:15-9:30 as the data traffic between 
> 9:30-9:45. 
> 
> If the interval is set to 1440(this means usge server calculate the data from 
> 0:00-23:59:59 at predefined time). The usage server regards the data traffic 
> between the server starting time(such as 13:00) to the next day(13:00) as the 
> data traffic between 0:00 an 23:59:59.
> 
> Another problem is, when the usage.stats.job.aggregation.range is set to N( 
> for example, 15), the usage job handle the data every N+1 minutes( for 
> example 16).
> 
> This is a patch for this issue (all details are described here: 
> https://issues.apache.org/jira/browse/CLOUDSTACK-962 ). 
> This patch depends on the patch for CLOUDSTACK-685 which is fixed in 4.0.1.  
> You can get from: 
> https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=commit;h=2f1d83037bd4bb1165b378d6c1dfc2815f696681
> 
> 
> This addresses bug CLOUDSTACK-962.
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java 
> 008c7c2 
>   server/src/com/cloud/user/dao/UserStatisticsDaoImpl.java fd7795a 
>   usage/src/com/cloud/usage/UsageManagerImpl.java 53ebb14 
> 
> Diff: https://reviews.apache.org/r/8971/diff/
> 
> 
> Testing
> ---
> 
> The patch is tested in CloudStack Virtual Router environment (Advanced 
> Networking, not Basic Networking).
> 
> usage.stats.job.aggregation.range = 5, the usage server collects data every 
> 10 minutes (such as collect data traffic between 13:00:00-13:10:00 at 
> 13:10:00).
> usage.stats.job.aggregation.range = 15, the usage server collects data every 
> 15 minutes (such as collect data traffic between 13:00:00-13:15:00 at 
> 13:15:00).
> usage.stats.job.aggregation.range = 60, the usage server collects data every 
> hour (such as collect data traffic between 13:00:00-14:00:00 at 14:10).
> usage.stats.job.aggregation.range = 1440, the usage server collects data 
> every day (such as collect data traffic between 0:00:00-23:59:59 at 0:30).
> 
> 
> Thanks,
> 
> Wei Zhou
> 
>

Re: Review Request: additional patch for CLOUDSTACK-962

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11181/#review20635
---


Commit 3fa8fda37c8b9fac1e38b57d28d23917106fe02b in branch 
refs/heads/object_store from Edison Su 
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=3fa8fda ]

CLOUDSTACK-962: Because of the same reason to this (CLOUDSTACK-685)
https://reviews.apache.org/r/11157/

Signed-off-by: Chip Childers 


- ASF Subversion and Git Services


On May 15, 2013, 2:47 p.m., Wei Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11181/
> ---
> 
> (Updated May 15, 2013, 2:47 p.m.)
> 
> 
> Review request for cloudstack and Chip Childers.
> 
> 
> Description
> ---
> 
> Because of the same reason to this (CLOUDSTACK-685)
> https://reviews.apache.org/r/11157/
> 
> 
> This addresses bug CLOUDSTACK-962.
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java 
> 522b90e 
> 
> Diff: https://reviews.apache.org/r/11181/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Wei Zhou
> 
>

Re: wrong validation in Netscaler Element while applyLBRules

[Prasanna Santhanam]

When you do merges (and not rebases) your bisect and blame also conk out.

http://mettadore.com/analysis/a-simple-git-rebase-workflow-explained/

On Wed, May 15, 2013 at 10:39:11AM -0700, Alena Prokharchyk wrote:
> Fixed this particular problem to unblock the QA and dev. It should have
> been if (!canHandleLbRules). The problem was introduced by my merge from
> internalLb branch done with the single squashed commit
> (2660a6b7a7f226ab757d2175222db62571813120) on May 9th. Not sure why
> Nitin's merge from May 11th overrode the master history for
> NetscalerElement.java file. Nitin, how did you make your merge?
> 
> Thanks,
> -Alena.
> 
> On 5/15/13 6:11 AM, "murali reddy"  wrote:
> 
> >Git blame shows un-intended change due
> >to c11dbad9c9ba7a876243ec02e90215906cfd9115. Nitin, can you see why your
> >merge brought these changes? Please figure root cause, its possible other
> >files got affected as well.
> >
> >On Wed, May 15, 2013 at 6:17 PM, Rajesh Battala
> >wrote:
> >
> >> Hi,
> >>
> >> I was not able to create LB rule on Netscaler device when Netscaler
> >>device
> >> is my LB provider in my network offering.
> >> I debugged and figured out that, in applyLBRules method
> >>
> >>
> >> if (canHandleLbRules(rules)) {
> >> return false;
> >> }
> >>
> >> Even the method canHandleLbRules is returning true[ means Netscaler
> >> element can handle the LB rule]
> >> its sending the wrong return value causing the failure of creating rule
> >>on
> >> NS device But showing the LB rule creation is success and LB rule is
> >> persisted in db.
> >>
> >> Fix is to add "!" in the if logic, but is there any other reason why
> >>it's
> >> not added. This method is introduced in recent merges.
> >>
> >> Thanks
> >> Rajesh Battala
> >>
> >
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: [MERGE] [CLOUDSTACK-2056] DeploymentPlanner choice via ServiceOffering

[Prasanna Santhanam]

If possible can you change the test to not use the API commands
directly? I changed the affinity_groups test when that merge happened
to make it use Resources as primary entities and not call API commands
directly. 

I'm trying to switch marvin to use the Resource based approach [1]
over API command based approach and would like to follow that as the
norm. The command approach will be deprecated gradually. I rewrote the
wiki [2] to include such an example which you can make use of and
extend.

[1] http://github.com/vogxn/docs/blob/master/MarvinRefactor.md 
[2] https://s.apache.org/


On Wed, May 15, 2013 at 10:36:33AM -0700, Prachi Damle wrote:
> If there are no concerns, I will start merging these changes to master today.
> 
> Thanks,
> Prachi
> 
> From: Prachi Damle [mailto:prachi.da...@citrix.com]
> Sent: Friday, May 10, 2013 4:04 PM
> To: dev@cloudstack.apache.org
> Subject: [MERGE] [CLOUDSTACK-2056] DeploymentPlanner choice via 
> ServiceOffering
> 
> Hi all,
> 
> I would like to merge the feature [CLOUDSTACK-2056] developed in 
> planner_reserve branch to master.
> 
> - Functional Specs: 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/DeploymentPlanner+choice+via+ServiceOffering
> - Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2056
> - Unit tests:  Have added unit test under 
> cloud-server/test/com/cloud/vm/DeploymentPlanningManagerImplTest.java
> - Integration tests: Added python test using marvin here:  
> cloud-testclient/integration/smoke/test_deploy_vms_with_varied_deploymentplanners.py
> 
> I have rebased the branch with master Commit hash: 
> 19f014585e1d2a85b66ecd5ad57ee0c6338beb4c
> Attached is the RAT report.
> 
> Thanks,
> -Prachi
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: Review Request: CLOUDSTACK-2130: UpdateDefaultNicForVirtualMachine api should also create usage events for updating new default network

[mice xia]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11156/#review20636
---

Ship it!


Ship It!

- mice xia


On May 14, 2013, 3:53 p.m., Saksham Srivastava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11156/
> ---
> 
> (Updated May 14, 2013, 3:53 p.m.)
> 
> 
> Review request for cloudstack and mice xia.
> 
> 
> Description
> ---
> 
> When we call UpdateDefaultNicForVirtualMachine we should call appropriate 
> usage events as well for updating information related to default nic for 
> proper usage calculation.
> Added 4 usage events : 2 for network.offerings.remove and 2 for 
> network.offerings.assign
> Events are :  network.offerings.assign for new nic to be made default, 
> network.offerings.remove for removal of non-default
> network.offerings.assign for old default nic to be made non default and 
> network.offerings.remove for removal of default.
> 
> 
> This addresses bug 2130.
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/vm/UserVmManagerImpl.java 0f6adc0 
> 
> Diff: https://reviews.apache.org/r/11156/diff/
> 
> 
> Testing
> ---
> 
> Tested manually.
> Rat build passed.
> Rebased to latest master.
> 
> 
> Thanks,
> 
> Saksham Srivastava
> 
>

[ACS42] Release Status Update - 15 days left to freeze

[Animesh Chaturvedi]

Folks

We are 15 days from ACS 42 feature freeze

Out of 98 proposed features /improvements, the status is

Closed: 5
Resolved: 32
In Progress: 15
Reopened: 1:
Ready to  review:  2
Open: 43

For the issues that are Open  I will ask to take a moment to update the jira 
items assigned to you with "In Progress" if you are already working on it and 
if your item is 'In Progress' please update the status.  


As for bugs here is a summary for this week


Blocker (b)  | Critical (c) | Major (m) | Total 
(t)

Incoming Defects (last 7 days)  :7 b |  19 c | 38 m   | 74 t
-
Outgoing Defects (last 7 days)  :   11 b|  18 c | 23 m   | 55 t
-
Open Unassigned : 2b | 19 c | 72 m| 110 t
-
Open Total  :   13 b | 53 c | 142 m | 237 t
-


And this is from  the previous week for reference, the net defect count went up 
by 30 issues even though there was a big jump is resolution in last 7 days

Blocker (b)  | Critical (c) | Major (m) | Total 
(t)

Incoming Defects (prev 7 days) : 7 b |  15 c | 29 m   | 54 t
-
Outgoing Defects (prev 7 days)  :   5 b   |  7 c   | 13 m   | 25 t
-
Open Unassigned : 6 b | 19 c | 53 m| 89 t
-
Open Total  :   13 b | 52 c | 121 m | 207 t
-

We have a large number of unassigned and open defects, If you are interested in 
helping out on defects please check the release dashboard widget on issues by 
components  http://s.apache.org/M5k 



Animesh

Re: [ACS41] Current blocker status

[Prasanna Santhanam]

On Wed, May 15, 2013 at 03:07:41PM -0700, Animesh Chaturvedi wrote:
> > CLOUDSTACK-2516
> > Create User API compability broken now
> > 
> > Nobody has taken this on yet.
> [Animesh>] I will update the JIRA ticket, looks like this was
> changed by Kishan to use clear text password. I think functionality
> wise it will still work but will cause double encryption. We will
> have to wait for Kishan  to respond on this.

I think Kishan only added regions support. That piece of code has
existed for a long time. I'm guessing something in the order of
authenticators is going wrong although componentContext.xml.in is
using MD5 auth first. Investigating ..

-- 
Prasanna.,


Powered by BigRock.com

RE: [MERGE] [CLOUDSTACK-2056] DeploymentPlanner choice via ServiceOffering

[Prachi Damle]

I had written the test as per the old wiki page,  now will refer to the new 
wiki and make the changes to the test. Thanks for the wiki example!
If there is no objection, I will do these changes post-merge.

Thanks,
Prachi

-Original Message-
From: Prasanna Santhanam [mailto:t...@apache.org] 
Sent: Wednesday, May 15, 2013 10:24 PM
To: dev@cloudstack.apache.org
Subject: Re: [MERGE] [CLOUDSTACK-2056] DeploymentPlanner choice via 
ServiceOffering

If possible can you change the test to not use the API commands directly? I 
changed the affinity_groups test when that merge happened to make it use 
Resources as primary entities and not call API commands directly. 

I'm trying to switch marvin to use the Resource based approach [1] over API 
command based approach and would like to follow that as the norm. The command 
approach will be deprecated gradually. I rewrote the wiki [2] to include such 
an example which you can make use of and extend.

[1] http://github.com/vogxn/docs/blob/master/MarvinRefactor.md
[2] https://s.apache.org/


On Wed, May 15, 2013 at 10:36:33AM -0700, Prachi Damle wrote:
> If there are no concerns, I will start merging these changes to master today.
> 
> Thanks,
> Prachi
> 
> From: Prachi Damle [mailto:prachi.da...@citrix.com]
> Sent: Friday, May 10, 2013 4:04 PM
> To: dev@cloudstack.apache.org
> Subject: [MERGE] [CLOUDSTACK-2056] DeploymentPlanner choice via 
> ServiceOffering
> 
> Hi all,
> 
> I would like to merge the feature [CLOUDSTACK-2056] developed in 
> planner_reserve branch to master.
> 
> - Functional Specs: 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/DeploymentPlanner+choice+via+ServiceOffering
> - Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2056
> - Unit tests:  Have added unit test under 
> cloud-server/test/com/cloud/vm/DeploymentPlanningManagerImplTest.java
> - Integration tests: Added python test using marvin here:  
> cloud-testclient/integration/smoke/test_deploy_vms_with_varied_deploymentplanners.py
> 
> I have rebased the branch with master Commit hash: 
> 19f014585e1d2a85b66ecd5ad57ee0c6338beb4c
> Attached is the RAT report.
> 
> Thanks,
> -Prachi
> 

-- 
Prasanna.,


Powered by BigRock.com

Re: Review Request: CLOUDSTACK-2130: UpdateDefaultNicForVirtualMachine api should also create usage events for updating new default network

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11156/#review20637
---


Commit e26081731898e2b3de53fdfb9f58d6fdb017f09f in branch refs/heads/master 
from Mice Xia 
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=e260817 ]

CLOUDSTACK-2130: UpdateDefaultNicForVirtualMachine api should also create usage 
events for updating new default network

Signed-off-by: Mice Xia 


- ASF Subversion and Git Services


On May 14, 2013, 3:53 p.m., Saksham Srivastava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11156/
> ---
> 
> (Updated May 14, 2013, 3:53 p.m.)
> 
> 
> Review request for cloudstack and mice xia.
> 
> 
> Description
> ---
> 
> When we call UpdateDefaultNicForVirtualMachine we should call appropriate 
> usage events as well for updating information related to default nic for 
> proper usage calculation.
> Added 4 usage events : 2 for network.offerings.remove and 2 for 
> network.offerings.assign
> Events are :  network.offerings.assign for new nic to be made default, 
> network.offerings.remove for removal of non-default
> network.offerings.assign for old default nic to be made non default and 
> network.offerings.remove for removal of default.
> 
> 
> This addresses bug 2130.
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/vm/UserVmManagerImpl.java 0f6adc0 
> 
> Diff: https://reviews.apache.org/r/11156/diff/
> 
> 
> Testing
> ---
> 
> Tested manually.
> Rat build passed.
> Rebased to latest master.
> 
> 
> Thanks,
> 
> Saksham Srivastava
> 
>

Re: [MERGE] [CLOUDSTACK-2056] DeploymentPlanner choice via ServiceOffering

[Prasanna Santhanam]

On Wed, May 15, 2013 at 11:22:13PM -0700, Prachi Damle wrote:
> I had written the test as per the old wiki page,  now will refer to
> the new wiki and make the changes to the test. Thanks for the wiki
> example!
> If there is no objection, I will do these changes post-merge.

Sure - not a problem. +1 to merge.
-- 
Prasanna.,


Powered by BigRock.com

RE: [ACS41] Current blocker status

[Kishan Kavala]

> -Original Message-
> From: Prasanna Santhanam [mailto:t...@apache.org]
> Sent: Thursday, 16 May 2013 11:51 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [ACS41] Current blocker status
> 
> On Wed, May 15, 2013 at 03:07:41PM -0700, Animesh Chaturvedi wrote:
> > > CLOUDSTACK-2516
> > > Create User API compability broken now
> > >
> > > Nobody has taken this on yet.
> > [Animesh>] I will update the JIRA ticket, looks like this was changed
> > by Kishan to use clear text password. I think functionality wise it
> > will still work but will cause double encryption. We will have to wait
> > for Kishan  to respond on this.
> 
> I think Kishan only added regions support. That piece of code has existed for
> a long time. I'm guessing something in the order of authenticators is going
> wrong although componentContext.xml.in is using MD5 auth first.
> Investigating ..

[KK]  I'm looking into this issue and update with my findings.

> 
> --
> Prasanna.,
> 
> 
> Powered by BigRock.com