[GitHub] cloudstack pull request: CLOUDSTACK-8651: [Browser Based Upload Te...

2015-07-27 Thread DaanHoogland
Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/607#issuecomment-125183196
  
never mind that last bot-comment, i found a working config its running. 
will get results in a minute. if impatient and a hero, go with your own result 
;)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: Edit rights to Wiki

2015-07-27 Thread Daan Hoogland
On Mon, Jul 27, 2015 at 2:26 PM, Sarath Kasi sarath.k...@citrix.com wrote:
 sarathkouk


is in

-- 
Daan


Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack

2015-07-27 Thread Wido den Hollander


On 27-07-15 11:52, Daan Hoogland wrote:
 gogogo
 

The PR is out there: https://github.com/apache/cloudstack/pull/625

 On Mon, Jul 27, 2015 at 10:28 AM, Wido den Hollander w...@widodh.nl wrote:
 Hi,

 It has been quite some time and this feature hasn't merged yet.

 I created a issue for this:
 https://issues.apache.org/jira/browse/CLOUDSTACK-8677

 The code is ready to merge and is in the reporter [0] branch on the
 Git repo.

 Even if the VM isn't ready, I would like to merge this feature into 4.6
 and get a VERY CLEAR doc into the Release Notes.

 Without Infra to send stats to, it seems like this feature is worthless,
 but it isn't. Since we got the code running, so as soon as we get
 call-home.cloudstack.org online we are good.

 Merging would just be sending a PR, before I do so, any objections?

 Wido

 [0]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter

 On 01-12-14 14:08, Wido den Hollander wrote:
 Hello,

 As a project we currently don't have a lot of insight information on
 about how CloudStack is being used. Surveys tell us a lot, but not
 everybody fills in the survey, so we still miss a lot of information.

 That's why I've written the Usage Reporting functionality for the
 management server which automatically sends back anonymous information
 about a CloudStack deployment.

 It's currently in the 'reporter' branch. [0]

 By default, every 7 days it generates a JSON document with:
 - Hosts (Number, version, type, hypervisor)
 - Clusters (Hypervisor en Management type)
 - Primary storage (Type and provider)
 - Zones (Network type and providers)
 - Instances (Number and types)

 This report is not complete yet, I'd like to add more information, but
 that will be Management Server information.

 The code on how this report is generated is obviously 100% Open Source,
 so end-users can always exactly see how the information was compiled.

 I want to discuss this new feature for CloudStack and the possible
 implications it might have.

 I'm opting for a opt-out. So every new or upgraded install to 4.6.0
 (master) will have this enabled. Yes, we have to be very explicit in the
 Release Notes that this has been added.

 Why? It's the small price we as a project ask for using CloudStack. We
 want a little bit of information on how CloudStack is being used so that
 we can use this to make CloudStack even better.

 Turning it off is also just one global setting and it will never turn on
 again.

 On the server-side there is a Python flask application [1] (found in the
 reporter directory) which stores all the incoming information in a
 ElasticSearch database. From there analytics can be gathered on
 CloudStack deployments.

 It currently points to http://cs-report.widodh.nl/report which will NOT
 be the endpoint when this is merged into master.

 For 'production' I want to have
 https://report.cloudstack.apache.org/report where all reports are submitted.

 For every setup a unique ID is determined by hashing the first row in
 the 'version' table. This is the version + timestamp and that is hashed
 using SHA256. Using this unique ID we can track changes in deployments
 and see how they grow or shrink.

 Technically this wasn't that hard to implement, but the politics
 surrounding it might be the hardest part.

 What do other have to say about this? Should there be a VOTE for this
 feature to come into CloudStack? Opt-in, opt-out?

 Wido

 [0]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
 [1]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter

 
 
 


RE: Edit rights to Wiki

2015-07-27 Thread Sarath Kasi
HI,
 My uid is  sarathkouk
Thanks,
Sarath

-Original Message-
From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] 
Sent: Monday, July 27, 2015 5:53 PM
To: dev dev@cloudstack.apache.org
Subject: Re: Edit rights to Wiki

please give your cwiki uid?

On Mon, Jul 27, 2015 at 2:13 PM, Sarath Kasi sarath.k...@citrix.com wrote:
 Hi,
 I'd like to update Testing section of LDAP: Trust AD and Auto 
 Importhttps://cwiki.apache.org/confluence/display/CLOUDSTACK/LDAP%3A+Trust+AD+and+Auto+Import
on wiki . Could someone please grant me the permissions needed?

 Thanks,

 Sarath






-- 
Daan


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread wido
GitHub user wido opened a pull request:

https://github.com/apache/cloudstack/pull/625

CLOUDSTACK-8677: Call-home functionality for CloudStack

With this commit the Management Server will be default generate a anonymous 
Usage
report every 7 (seven) days and submit this information back to the Apache 
CloudStack project.

These anonymous reports do NOT contain any information about Instance 
names, subnets, etc. It only
contains numbers about how CloudStack is being used.

This information is vital for the project to gain more insight in how 
CloudStack is being used.

Users can turn the reporting off by setting usage.report.interval to 0 
(zero)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wido/cloudstack CLOUDSTACK-8677

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/625.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #625


commit 1c61afb7b93313743e9df1d10242415f838e660b
Author: Wido den Hollander w...@widodh.nl
Date:   2015-07-08T19:17:30Z

CLOUDSTACK-8677: Call-home functionality for CloudStack

With this commit the Management Server will be default generate a anonymous 
Usage
report every 7 (seven) days and submit this information back to the Apache 
CloudStack project.

These anonymous reports do NOT contain any information about Instance 
names, subnets, etc. It only
contains numbers about how CloudStack is being used.

This information is vital for the project to gain more insight in how 
CloudStack is being used.

Users can turn the reporting off by setting usage.report.interval to 0 
(zero)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: [BLOCKER] unable to start VR in basic zone on master

2015-07-27 Thread Wilder Rodrigues
Hi there Rajani,

My apologies, I missed that one.

I haven’t tested basic zone in a while, to be honest. I will have a look to see 
if I can be of any help.

Cheers,
Wilder


 On 27 Jul 2015, at 11:11, Rajani Karuturi raj...@apache.org wrote:
 
 I faced the same issue with basic zone using xenserver on master.
 
 Wilder,
 Can you please comment on this?
 
 ~Rajani
 
 On Fri, Jul 24, 2015 at 4:00 PM, Kishan Kavala kishan.kav...@citrix.com
 wrote:
 
 @Wilder, can you please look into this issue.
 Looks like VR config file is incomplete and it is over-writing the IP
 information set by cloud-early-config.
 
 VR Config file:
 #Apache CloudStack Virtual Router Config File
 version
 1.0
 /version
 file
 /var/cache/cloud/monitor_service.json
 
 {config:[dhcp]:processname=dnsmasq:servicename=dnsmasq:pidfile=/var/run/dnsmasq/dnsmasq.pid:,[ssh]:processname=sshd:servicename=ssh:pidfile=/var/run/sshd.pid:,[webserver]:processname=apache2:servicename=apache2:pidfile=/var/run/apache2.pid:,,type:monitorservice}
 /file
 script
 /opt/cloud/bin/update_config.py monitor_service.json
 /script
 
 /var/log/cloud.log:
 2015-07-24 09:29:28,997 Loading data bag type monitorservice
 2015-07-24 09:29:28,998 Command of type monitorservice received
 2015-07-24 09:29:28,999 Writing data bag type monitorservice
 2015-07-24 09:29:28,999 Creating data bag type ips
 2015-07-24 09:29:28,999 Loading data bag type cmdline
 2015-07-24 09:29:29,000 Executing ip addr show dev eth1
 2015-07-24 09:29:29,019 Will remove all configured addresses on device eth1
 2015-07-24 09:29:29,019 Removing addresses from device eth1
 2015-07-24 09:29:29,036 Removed address 169.254.2.130/16 from device eth1
 2015-07-24 09:29:29,056 Removed address 169.254.2.130/16 from device eth1
 2015-07-24 09:29:29,057 Executing ip addr show dev eth0
 2015-07-24 09:29:29,075 Will remove all configured addresses on device eth0
 2015-07-24 09:29:29,076 Removing addresses from device eth0
 2015-07-24 09:29:29,093 Removed address 10.220.39.131/19 from device eth0
 2015-07-24 09:29:29,112 Removed address 10.220.39.131/19 from device eth
 
 
 -Original Message-
 From: Sanjeev N [mailto:sanj...@apache.org]
 Sent: 24 July 2015 12:06 PM
 To: dev@cloudstack.apache.org
 Subject: [BlockerBug] VR creation fails in basic zone
 
 Hi,
 
 In latest build with ACS master, VR creation fails in basic zone.
 Created a blocker bug for this. More details @
 https://issues.apache.org/jira/browse/CLOUDSTACK-8668
 
 Thanks,
 Sanjeev
 



[GitHub] cloudstack pull request: Unit tests for HA manager investigate met...

2015-07-27 Thread koushik-das
GitHub user koushik-das opened a pull request:

https://github.com/apache/cloudstack/pull/624

Unit tests for HA manager investigate method. Refer to CLOUDSTACK-866…

…6 for the code chenges

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/koushik-das/cloudstack unit_tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/624.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #624


commit f6a3c3e42ab9afdafa8d4692a6bba1c289c2461f
Author: Koushik Das kous...@apache.org
Date:   2015-07-27T10:44:19Z

Unit tests for HA manager investigate method. Refer to CLOUDSTACK-8666 for 
the code chenges




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread kansal
Github user kansal commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/618#discussion_r35524999
  
--- Diff: 
plugins/user-authenticators/saml2/src/org/apache/cloudstack/saml/SAML2AuthManagerImpl.java
 ---
@@ -358,7 +363,8 @@ private boolean setup() {
 }
 _timer = new Timer();
 final HttpClient client = new HttpClient();
-final String idpMetaDataUrl = 
SAMLIdentityProviderMetadataURL.value();
+//final String idpMetaDataUrl = 
SAMLIdentityProviderMetadataURL.value();
--- End diff --

@bhaisaab  Sure. Removing the comment. The function call is needed for 
mocking at the time of unit testing. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: boring resourse leaks again

2015-07-27 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/602


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request:

2015-07-27 Thread pdion891
Github user pdion891 commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12371406
  
Hi @wilderrodrigues,
I saw the job success on 22th, I'm wondering if dependencies for keepalived 
package would have change recently on the repo. 

I've changed the build keep to 2.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request:

2015-07-27 Thread pdion891
Github user pdion891 commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12372033
  
So we should just change the install  of  libnl to use  the backports ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request:

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12372819
  
I don't think that would be the solution because I still don't understand 
what happened. The build from 22nd, which was triggered by github, installed 
the deps automatically. However, the one you triggered manually did not.

Perhaps try to trigger another one or wait for a day and see if it gets 
built automatically once something gets merged?

How is the job configuration setup in terms of how often it builds?

Cheers,
Wilder



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: [GitHub] cloudstack pull request: CLOUDSTACK-8651: [Browser Based Upload Te...

2015-07-27 Thread Daan Hoogland
can't find a config that runs. I can do with some help. last state was:

\\sbpltc2kag3qr\dhoogland:~/cloudstack/cloudstack/test/integration
(CLOUDSTACK-8651) nosetests --with-marvin
--marvin-config=../../tools/devcloud/devcloud.cfg -s -a
tags=advanced,required_hardware=false
component/test_browse_templates.py

 Marvin Init Started 

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

 Log Folder Path: /tmp//MarvinLogs//Jul_27_2015_13_59_07_QHRK18.
All logs will be available here 

=== Marvin Init Logging Successful===

 Marvin Init Successful 
=== TestName: None | Status : EXCEPTION ===

===final results are now copied to: /tmp//MarvinLogs/test_suite_IGW70H===
\\sbpltc2kag3qr\dhoogland:~/cloudstack/cloudstack/test/integration
(CLOUDSTACK-8651) cat /tmp//MarvinLogs/test_suite_IGW70H/results.txt
=== TestName: None | Status : EXCEPTION ===
ERROR

==
ERROR: test suite for class
'integration.component.test_browse_templates.TestBrowseUploadVolume'
--
Traceback (most recent call last):
  File /usr/local/lib/python2.7/site-packages/nose/suite.py, line 209, in run
self.setUp()
  File /usr/local/lib/python2.7/site-packages/nose/suite.py, line
292, in setUp
self.setupContext(ancestor)
  File /usr/local/lib/python2.7/site-packages/nose/suite.py, line
315, in setupContext
try_run(context, names)
  File /usr/local/lib/python2.7/site-packages/nose/util.py, line
471, in try_run
return func()
  File 
/Users/dhoogland/cloudstack/cloudstack/test/integration/component/test_browse_templates.py,
line 84, in setUpClass

cls.uploadurl=cls.testdata[configurableData][browser_upload_template][cls.uploadtemplateformat][url]
KeyError: 'browser_upload_template'
  begin captured logging  
CSLog: DEBUG: Payload: {'apiKey':
u'_isVbffFSr5b7cheL713XiijhtId9I_2HLhlzHaH5psSyWNjlm2SBZB_ubyR4lhS_U8yioUdL58gav_-5mP5yA',
'command': 'listHosts', 'signature': 'Tm5IjuItrAMIg9ElRcLFQIb885Q=',
'type': 'Routing', 'response': 'json'}
CSLog: DEBUG: Sending GET Cmd : listHosts===
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP
connection (1): 127.0.0.1
requests.packages.urllib3.connectionpool: DEBUG: GET
/client/api?type=RoutingapiKey=_isVbffFSr5b7cheL713XiijhtId9I_2HLhlzHaH5psSyWNjlm2SBZB_ubyR4lhS_U8yioUdL58gav_-5mP5yAcommand=listHostsresponse=jsonsignature=Tm5IjuItrAMIg9ElRcLFQIb885Q%3D
HTTP/1.1 200 3083
CSLog: DEBUG: Response : [{cpuwithoverprovisioning : u'32000.0',
version : u'4.6.0-SNAPSHOT', memorytotal : 8589934592, zoneid :
u'7171e8fc-ca86-47d3-9ba4-58fdd9001196', cpunumber : 4,
managementserverid : 4278190080, cpuallocated : u'0%', memoryused : 0,
id : u'204677ca-dc48-4d38-abc5-b2be753758cf', cpuused : u'0%',
clusterid : u'8d2759a2-cbc2-4288-b6cb-aeafe3b3dc64', capabilities :
u'hvm', state : u'Up', memoryallocated : 0, networkkbswrite : 16384,
cpuspeed : 8000, type : u'Routing', events : u'ManagementServerDown;
AgentDisconnected; HostDown; Remove; Ping; AgentConnected;
StartAgentRebalance; PingTimeout; ShutdownRequested', zonename :
u'Sandbox-simulator', podid : u'1576694a-d70b-494a-b6fa-76b919892a39',
clustertype : u'CloudManaged', hahost : False, lastpinged :
u'1970-01-17T07:04:53+0100', ipaddress : u'172.16.15.15', name :
u'SimulatedAgent.3228f38e-541b-4a9a-87d6-1e0c46f9385c', networkkbsread
: 32768, created : u'2015-07-27T13:28:18+0200', clustername : u'C1',
hypervisor : u'Simulator', islocalstorageactive : False, resourcestate
: u'Enabled', podname : u'POD0'}, {cpuwithoverprovisioning :
u'32000.0', version : u'4.6.0-SNAPSHOT', memorytotal : 8589934592,
zoneid : u'7171e8fc-ca86-47d3-9ba4-58fdd9001196', cpunumber : 4,
managementserverid : 4278190080, cpuallocated : u'3.12%', memoryused :
0, id : u'78e50eaa-8ff8-4fd8-b050-18357c16711a', cpuused : u'0%',
clusterid : u'0689d8ed-7d46-4f0b-a4fc-c5931aab969a', capabilities :
u'hvm', state : u'Up', memoryallocated : 1610612736, networkkbswrite :
16384, cpuspeed : 8000, type : u'Routing', events :
u'ManagementServerDown; AgentDisconnected; HostDown; Remove; Ping;
AgentConnected; StartAgentRebalance; PingTimeout; ShutdownRequested',
zonename : u'Sandbox-simulator', podid :
u'1576694a-d70b-494a-b6fa-76b919892a39', clustertype :
u'CloudManaged', hahost : False, lastpinged :
u'1970-01-17T07:04:53+0100', ipaddress : u'172.16.15.14', name :
u'SimulatedAgent.8d99aff4-f3e5-4d15-9c5e-2321d0d9b0eb', networkkbsread
: 32768, created : u'2015-07-27T13:27:18+0200', clustername : u'C0',
hypervisor : u'Simulator', islocalstorageactive : False, resourcestate
: u'Enabled', podname : u'POD0'}, {cpuwithoverprovisioning :
u'32000.0', version : u'4.6.0-SNAPSHOT', memorytotal : 8589934592,
zoneid : u'7171e8fc-ca86-47d3-9ba4-58fdd9001196', cpunumber : 4,
managementserverid : 4278190080, cpuallocated : u'0%', memoryused : 0,
id : 

Re: [BLOCKER] unable to start VR in basic zone on master

2015-07-27 Thread Wilder Rodrigues
Just replied to the previous email to the list.

Sorry, I missed that one.

I haven’t tested basic zone in a while… will give it a try.

Cheers,
Wilder


 On 27 Jul 2015, at 14:51, Wilder Rodrigues wrodrig...@schubergphilis.com 
 wrote:
 
 Hi there Rajani,
 
 My apologies, I missed that one.
 
 I haven’t tested basic zone in a while, to be honest. I will have a look to 
 see if I can be of any help.
 
 Cheers,
 Wilder
 
 
 On 27 Jul 2015, at 11:11, Rajani Karuturi raj...@apache.org wrote:
 
 I faced the same issue with basic zone using xenserver on master.
 
 Wilder,
 Can you please comment on this?
 
 ~Rajani
 
 On Fri, Jul 24, 2015 at 4:00 PM, Kishan Kavala kishan.kav...@citrix.com
 wrote:
 
 @Wilder, can you please look into this issue.
 Looks like VR config file is incomplete and it is over-writing the IP
 information set by cloud-early-config.
 
 VR Config file:
 #Apache CloudStack Virtual Router Config File
 version
 1.0
 /version
 file
 /var/cache/cloud/monitor_service.json
 
 {config:[dhcp]:processname=dnsmasq:servicename=dnsmasq:pidfile=/var/run/dnsmasq/dnsmasq.pid:,[ssh]:processname=sshd:servicename=ssh:pidfile=/var/run/sshd.pid:,[webserver]:processname=apache2:servicename=apache2:pidfile=/var/run/apache2.pid:,,type:monitorservice}
 /file
 script
 /opt/cloud/bin/update_config.py monitor_service.json
 /script
 
 /var/log/cloud.log:
 2015-07-24 09:29:28,997 Loading data bag type monitorservice
 2015-07-24 09:29:28,998 Command of type monitorservice received
 2015-07-24 09:29:28,999 Writing data bag type monitorservice
 2015-07-24 09:29:28,999 Creating data bag type ips
 2015-07-24 09:29:28,999 Loading data bag type cmdline
 2015-07-24 09:29:29,000 Executing ip addr show dev eth1
 2015-07-24 09:29:29,019 Will remove all configured addresses on device eth1
 2015-07-24 09:29:29,019 Removing addresses from device eth1
 2015-07-24 09:29:29,036 Removed address 169.254.2.130/16 from device eth1
 2015-07-24 09:29:29,056 Removed address 169.254.2.130/16 from device eth1
 2015-07-24 09:29:29,057 Executing ip addr show dev eth0
 2015-07-24 09:29:29,075 Will remove all configured addresses on device eth0
 2015-07-24 09:29:29,076 Removing addresses from device eth0
 2015-07-24 09:29:29,093 Removed address 10.220.39.131/19 from device eth0
 2015-07-24 09:29:29,112 Removed address 10.220.39.131/19 from device eth
 
 
 -Original Message-
 From: Sanjeev N [mailto:sanj...@apache.org]
 Sent: 24 July 2015 12:06 PM
 To: dev@cloudstack.apache.org
 Subject: [BlockerBug] VR creation fails in basic zone
 
 Hi,
 
 In latest build with ACS master, VR creation fails in basic zone.
 Created a blocker bug for this. More details @
 https://issues.apache.org/jira/browse/CLOUDSTACK-8668
 
 Thanks,
 Sanjeev
 
 



[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread DaanHoogland
Github user DaanHoogland commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35532842
  
--- Diff: setup/db/db/schema-452to460.sql ---
@@ -398,3 +398,5 @@ CREATE TABLE `cloud`.`external_bigswitch_bcf_devices` (
   CONSTRAINT `fk_external_bigswitch_bcf_devices__host_id` FOREIGN KEY 
(`host_id`) REFERENCES `host`(`id`) ON DELETE CASCADE,
   CONSTRAINT `fk_external_bigswitch_bcf_devices__physical_network_id` 
FOREIGN KEY (`physical_network_id`) REFERENCES `physical_network`(`id`) ON 
DELETE CASCADE
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+INSERT IGNORE INTO `cloud`.`configuration` VALUES (Advanced, 'DEFAULT', 
'management-server', usage.report.interval, 7, Interval (days) between 
sending anonymous Usage Reports back to the CloudStack project, , NULL, 
NULL, 0);
--- End diff --

with the new style config (ConfigKeyT), this isn't needed is it? It 
should insert the default if it doesn't find the value. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread wido
Github user wido commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35533953
  
--- Diff: setup/db/db/schema-452to460.sql ---
@@ -398,3 +398,5 @@ CREATE TABLE `cloud`.`external_bigswitch_bcf_devices` (
   CONSTRAINT `fk_external_bigswitch_bcf_devices__host_id` FOREIGN KEY 
(`host_id`) REFERENCES `host`(`id`) ON DELETE CASCADE,
   CONSTRAINT `fk_external_bigswitch_bcf_devices__physical_network_id` 
FOREIGN KEY (`physical_network_id`) REFERENCES `physical_network`(`id`) ON 
DELETE CASCADE
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+INSERT IGNORE INTO `cloud`.`configuration` VALUES (Advanced, 'DEFAULT', 
'management-server', usage.report.interval, 7, Interval (days) between 
sending anonymous Usage Reports back to the CloudStack project, , NULL, 
NULL, 0);
--- End diff --

Hmm, I don't know. I wrote this code in 2014 and was that in there by now? 
I'm just doing this because I assumed it was still required.

Maybe @bhaisaab knows?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


aws in pom

2015-07-27 Thread Daan Hoogland
H,

aws is still metioned in our pom.xml. It can be removed, afaics.

Now that aws compatibility is taken care of in a proxy, we should not
be depending on the sdk anymore or is there some other stuff that
still needs it?

-- 
Daan


Re: aws in pom

2015-07-27 Thread Daan Hoogland
yes :( found it in my latest experiment :)

On Mon, Jul 27, 2015 at 3:43 PM, Wido den Hollander w...@widodh.nl wrote:


 On 27-07-15 15:36, Daan Hoogland wrote:
 H,

 aws is still metioned in our pom.xml. It can be removed, afaics.

 Now that aws compatibility is taken care of in a proxy, we should not
 be depending on the sdk anymore or is there some other stuff that
 still needs it?


 Yes, the S3 Secondary Storage uses the AWS SDK.

 Wido




-- 
Daan


Edit rights to Wiki

2015-07-27 Thread Sarath Kasi
Hi,
I'd like to update Testing section of LDAP: Trust AD and Auto 
Importhttps://cwiki.apache.org/confluence/display/CLOUDSTACK/LDAP%3A+Trust+AD+and+Auto+Import
   on wiki . Could someone please grant me the permissions needed?

Thanks,

Sarath





[GitHub] cloudstack pull request:

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12371179
  
Hi again @pdion891 

That's my SSVM built upon the template 17th July

* SSVM Linux distro:
  * root@s-2-VM:~# uname -a
  * Linux s-2-VM 3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u2 x86_64 
GNU/Linux

* QCOW downloaded on 20th July:
  * -rw-r--r--. 1 root root 325160960 Jul 20 13:26 
f2f43a13-77d0-432e-82c2-49f585a8b5a7.qcow2

According to Debian Packages page, the lbnl1 is equivalent to lbnl-3-200 
package. You cans ee it here:

* https://packages.debian.org/wheezy-backports/libnl-3-200

When I tried to install the lbnl1 from wheezy-backports, I got the 
following:

root@s-2-VM:~# apt-get install -t wheezy-backports libnl1
Reading package lists... Done
Building dependency tree   
Reading state information... Done
libnl1 is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 33 not upgraded.

I then tried to install the package lbnl-3-200 and got the following:

root@s-2-VM:~# apt-get install -t wheezy-backports libnl-3-200
Reading package lists... Done
Building dependency tree   
Reading state information... Done
libnl-3-200 is already the newest version.
libnl-3-200 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 33 not upgraded.

So, did the job configuration changed?

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request:

2015-07-27 Thread pdion891
Github user pdion891 commented on the pull request:


https://github.com/apache/cloudstack/commit/4cdfbf337d7e8a221e6e227836a72ee26a49641f#commitcomment-12371467
  
In debian/rules:
In debian/rules on line 3:
look like this line is not working on when building .deb  the ${VERSION} 
remain empty.
why not change it for something like this:

```
cat pom.xml |sed '22!d'| cut -d'' -f2 |cut -d'' -f1
```



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request:

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12371572
  
In the build that failed it mentioned the libnl-genl-3-200, which is 
already installed in my SSVM, based on the systemvm from 17th July.

But it also mentions the libnl-genl-3-200, which I did not try to install 
before. I just gave it a try and also go a successful response:

root@s-2-VM:~# apt-get install -t wheezy-backports  libnl-genl-3-200
Reading package lists... Done
Building dependency tree   
Reading state information... Done
libnl-genl-3-200 is already the newest version.
libnl-genl-3-200 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 33 not upgraded.

Checking the logs of the build from the 22nd I got the following:

+ apt-get --no-install-recommends -q -y --force-yes -t wheezy-backports 
install keepalived irqbalance open-vm-tools
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  libcap-ng0 libdumbnet1 libicu48 libnl-3-200 libnl-genl-3-200 libnuma1
  libperl5.14 libsnmp-base libsnmp15
Suggested packages:
  snmp-mibs-downloader open-vm-tools-desktop
Recommended packages:
  zerofree
The following NEW packages will be installed:
  irqbalance keepalived libcap-ng0 libdumbnet1 libicu48 libnl-3-200
  libnl-genl-3-200 libnuma1 libperl5.14 libsnmp-base libsnmp15 open-vm-tools
0 upgraded, 12 newly installed, 0 to remove and 33 not upgraded.

So, it installed all the new packages.

Perhaps starting the build manually messed up something. It's very weird. :(

cheers,
Wilder



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread sedukull
Github user sedukull commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35534192
  
--- Diff: server/src/org/apache/cloudstack/report/UsageReporter.java ---
@@ -0,0 +1,473 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// License); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.report;
+
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.net.URL;
+import java.net.SocketTimeoutException;
+import java.net.MalformedURLException;
+import java.net.ProtocolException;
+import java.net.UnknownHostException;
+import java.io.OutputStreamWriter;
+import java.io.IOException;
+
+import javax.inject.Inject;
+import javax.net.ssl.HttpsURLConnection;
+
+import org.apache.log4j.Logger;
+import org.springframework.stereotype.Component;
+
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.managed.context.ManagedContextRunnable;
+
+import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.StoragePoolVO;
+
+import org.apache.commons.codec.digest.DigestUtils;
+
+import com.cloud.host.HostVO;
+import com.cloud.host.dao.HostDao;
+import com.cloud.dc.ClusterVO;
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.vm.UserVmVO;
+import com.cloud.vm.dao.UserVmDao;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.component.ManagerBase;
+import com.cloud.utils.component.ComponentMethodInterceptable;
+import com.cloud.utils.concurrency.NamedThreadFactory;
+import com.cloud.utils.db.DB;
+import com.cloud.utils.db.TransactionLegacy;
+import com.cloud.upgrade.dao.VersionDao;
+import com.cloud.upgrade.dao.VersionVO;
+import com.cloud.storage.dao.DiskOfferingDao;
+import com.cloud.storage.DiskOfferingVO;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+import com.google.common.util.concurrent.AtomicLongMap;
+
+@Component
+public class UsageReporter extends ManagerBase implements 
ComponentMethodInterceptable {
+public static final Logger s_logger = 
Logger.getLogger(UsageReporter.class.getName());
+
+/* !FIX ME! This should point to a Apache Infra host with SSL! */
+private String reportHost = https://call-home.cloudstack.org/report;;
+
+private String uniqueID = null;
+
+private static UsageReporter s_instance = null;
+
+private ScheduledExecutorService _executor = null;
+
+@Inject
+private ConfigurationDao _configDao;
+@Inject
+private HostDao _hostDao;
+@Inject
+private ClusterDao _clusterDao;
+@Inject
+private PrimaryDataStoreDao _storagePoolDao;
+@Inject
+private DataCenterDao _dataCenterDao;
+@Inject
+private UserVmDao _userVmDao;
+@Inject
+private VMInstanceDao _vmInstance;
+@Inject
+private VersionDao _versionDao;
+@Inject
+private DiskOfferingDao _diskOfferingDao;
+
+int usageReportInterval = -1;
+
+public static UsageReporter getInstance() {
+return s_instance;
+}
+
+public static UsageReporter getInstance(MapString, String configs) {
+s_instance.init(configs);
+return s_instance;
+}
+
+public UsageReporter() {
  

Re: Loading a custom HostAllocator

2015-07-27 Thread Wido den Hollander


On 27-07-15 12:28, Amy Niklasson wrote:
 I have tried this and I still can't find it. I have tried to find it both
 in the regular installation and the source installation of CloudStack.
 However I have managed to find this link
 http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.0.html
 which implies that the file has changed name to componentContext.xml.
 Unfortunately I can't find this file either.
 
 I have found the file
 /home/ubuntu/cloudstack/server/test/async-job-component.xml which contains
 the lines:
 
 adapters key=com.cloud.agent.manager.allocator.HostAllocator
   adapter name=FirstFit
 class=com.cloud.agent.manager.allocator.impl.FirstFitAllocator/
 /adapters
 
 Which are the lines according to
 http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-
 http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-custom-hostallocatorcustom-hostallocator
 http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-custom-hostallocator
 should be changed. However this location is in nowhere close to the
 one mentioned.
 
 If anyone has any experience of implementing a custom hostallocator in
 CloudStack 4.5 or knows anything I would very much appreciate any
 information you got.
 

I haven't done this, but I don't think componentContext.xml is being
used anymore.

I think you need this one:
core/resources/META-INF/cloudstack/core/spring-core-registry-core-context.xml

Wido

 Kind Regards,
 Amy
 
 
 On 8 July 2015 at 09:47, Wido den Hollander w...@widodh.nl wrote:
 
 
 
 On 08-07-15 10:08, Amy Niklasson wrote:
 Hi, How do you load a custom HostAllocator in CloudStack 4.5? The
 only documentation I've managed to find is
 http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-
 custom-hostallocator


 However,I can't find the files mentioned, components.xml and
 components-premium.xml, in /client/ tomcatconf. Anyone who knows?

 
 Not exactly sure which file it is, but you could try a grep through
 the code.
 
 It's the Spring framework which loads the modules in boot.
 
 Wido
 
 Regards, Amy


 


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread DaanHoogland
Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/625#issuecomment-125199454
  
any way to verify this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread wido
Github user wido commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35535873
  
--- Diff: server/src/org/apache/cloudstack/report/UsageReporter.java ---
@@ -0,0 +1,473 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// License); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.report;
+
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.net.URL;
+import java.net.SocketTimeoutException;
+import java.net.MalformedURLException;
+import java.net.ProtocolException;
+import java.net.UnknownHostException;
+import java.io.OutputStreamWriter;
+import java.io.IOException;
+
+import javax.inject.Inject;
+import javax.net.ssl.HttpsURLConnection;
+
+import org.apache.log4j.Logger;
+import org.springframework.stereotype.Component;
+
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.managed.context.ManagedContextRunnable;
+
+import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.StoragePoolVO;
+
+import org.apache.commons.codec.digest.DigestUtils;
+
+import com.cloud.host.HostVO;
+import com.cloud.host.dao.HostDao;
+import com.cloud.dc.ClusterVO;
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.vm.UserVmVO;
+import com.cloud.vm.dao.UserVmDao;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.component.ManagerBase;
+import com.cloud.utils.component.ComponentMethodInterceptable;
+import com.cloud.utils.concurrency.NamedThreadFactory;
+import com.cloud.utils.db.DB;
+import com.cloud.utils.db.TransactionLegacy;
+import com.cloud.upgrade.dao.VersionDao;
+import com.cloud.upgrade.dao.VersionVO;
+import com.cloud.storage.dao.DiskOfferingDao;
+import com.cloud.storage.DiskOfferingVO;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+import com.google.common.util.concurrent.AtomicLongMap;
+
+@Component
+public class UsageReporter extends ManagerBase implements 
ComponentMethodInterceptable {
+public static final Logger s_logger = 
Logger.getLogger(UsageReporter.class.getName());
+
+/* !FIX ME! This should point to a Apache Infra host with SSL! */
+private String reportHost = https://call-home.cloudstack.org/report;;
+
+private String uniqueID = null;
+
+private static UsageReporter s_instance = null;
+
+private ScheduledExecutorService _executor = null;
+
+@Inject
+private ConfigurationDao _configDao;
+@Inject
+private HostDao _hostDao;
+@Inject
+private ClusterDao _clusterDao;
+@Inject
+private PrimaryDataStoreDao _storagePoolDao;
+@Inject
+private DataCenterDao _dataCenterDao;
+@Inject
+private UserVmDao _userVmDao;
+@Inject
+private VMInstanceDao _vmInstance;
+@Inject
+private VersionDao _versionDao;
+@Inject
+private DiskOfferingDao _diskOfferingDao;
+
+int usageReportInterval = -1;
+
+public static UsageReporter getInstance() {
+return s_instance;
+}
+
+public static UsageReporter getInstance(MapString, String configs) {
+s_instance.init(configs);
+return s_instance;
+}
+
+public UsageReporter() {
+ 

Re: Edit rights to Wiki

2015-07-27 Thread Daan Hoogland
please give your cwiki uid?

On Mon, Jul 27, 2015 at 2:13 PM, Sarath Kasi sarath.k...@citrix.com wrote:
 Hi,
 I'd like to update Testing section of LDAP: Trust AD and Auto 
 Importhttps://cwiki.apache.org/confluence/display/CLOUDSTACK/LDAP%3A+Trust+AD+and+Auto+Import
on wiki . Could someone please grant me the permissions needed?

 Thanks,

 Sarath






-- 
Daan


[GitHub] cloudstack pull request:

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12370885
  
@pdion891 ping...

I'm checking the dependencies on Debian packages website: 
https://packages.debian.org/wheezy-backports/keepalived and will add the libnl1 
to the wheezy-backports as well and see if there is any other package it might 
depend on.

Before I proceed, could you please  change the Job in a way that we keep 
the last 2 successful builds?

As a matter of fact, I'm still puzzled about how the builds from 17th and 
22nd July worked.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread sedukull
Github user sedukull commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35534267
  
--- Diff: server/src/org/apache/cloudstack/report/UsageReporter.java ---
@@ -0,0 +1,473 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// License); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.report;
+
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.net.URL;
+import java.net.SocketTimeoutException;
+import java.net.MalformedURLException;
+import java.net.ProtocolException;
+import java.net.UnknownHostException;
+import java.io.OutputStreamWriter;
+import java.io.IOException;
+
+import javax.inject.Inject;
+import javax.net.ssl.HttpsURLConnection;
+
+import org.apache.log4j.Logger;
+import org.springframework.stereotype.Component;
+
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.managed.context.ManagedContextRunnable;
+
+import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.StoragePoolVO;
+
+import org.apache.commons.codec.digest.DigestUtils;
+
+import com.cloud.host.HostVO;
+import com.cloud.host.dao.HostDao;
+import com.cloud.dc.ClusterVO;
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.vm.UserVmVO;
+import com.cloud.vm.dao.UserVmDao;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.component.ManagerBase;
+import com.cloud.utils.component.ComponentMethodInterceptable;
+import com.cloud.utils.concurrency.NamedThreadFactory;
+import com.cloud.utils.db.DB;
+import com.cloud.utils.db.TransactionLegacy;
+import com.cloud.upgrade.dao.VersionDao;
+import com.cloud.upgrade.dao.VersionVO;
+import com.cloud.storage.dao.DiskOfferingDao;
+import com.cloud.storage.DiskOfferingVO;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+import com.google.common.util.concurrent.AtomicLongMap;
+
+@Component
+public class UsageReporter extends ManagerBase implements 
ComponentMethodInterceptable {
+public static final Logger s_logger = 
Logger.getLogger(UsageReporter.class.getName());
+
+/* !FIX ME! This should point to a Apache Infra host with SSL! */
+private String reportHost = https://call-home.cloudstack.org/report;;
+
+private String uniqueID = null;
+
+private static UsageReporter s_instance = null;
+
+private ScheduledExecutorService _executor = null;
+
+@Inject
+private ConfigurationDao _configDao;
+@Inject
+private HostDao _hostDao;
+@Inject
+private ClusterDao _clusterDao;
+@Inject
+private PrimaryDataStoreDao _storagePoolDao;
+@Inject
+private DataCenterDao _dataCenterDao;
+@Inject
+private UserVmDao _userVmDao;
+@Inject
+private VMInstanceDao _vmInstance;
+@Inject
+private VersionDao _versionDao;
+@Inject
+private DiskOfferingDao _diskOfferingDao;
+
+int usageReportInterval = -1;
+
+public static UsageReporter getInstance() {
+return s_instance;
+}
+
+public static UsageReporter getInstance(MapString, String configs) {
+s_instance.init(configs);
+return s_instance;
+}
+
+public UsageReporter() {
  

Re: aws in pom

2015-07-27 Thread Wido den Hollander


On 27-07-15 15:36, Daan Hoogland wrote:
 H,
 
 aws is still metioned in our pom.xml. It can be removed, afaics.
 
 Now that aws compatibility is taken care of in a proxy, we should not
 be depending on the sdk anymore or is there some other stuff that
 still needs it?
 

Yes, the S3 Secondary Storage uses the AWS SDK.

Wido



[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread kansal
Github user kansal commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/618#discussion_r35524889
  
--- Diff: 
plugins/user-authenticators/saml2/src/org/apache/cloudstack/saml/SAML2AuthManagerImpl.java
 ---
@@ -119,12 +123,12 @@
 @Override
 public boolean start() {
 if (isSAMLPluginEnabled()) {
-setup();
 s_logger.info(SAML auth plugin loaded);
+return setup();
 } else {
--- End diff --

@bhaisaab I feel we can go with the if-else style because it anyhow is 
doing the same thing plus it give better code readability and logging. Your 
take?  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


RE: [PROPOSAL] Deploy user instance from VM snapshot

2015-07-27 Thread Somesh Naidu
+1

The FS LGTM.

Regards,
Somesh


-Original Message-
From: Sateesh Chodapuneedi [mailto:sateesh.chodapune...@citrix.com] 
Sent: Monday, July 27, 2015 2:11 AM
To: dev@cloudstack.apache.org
Subject: [PROPOSAL] Deploy user instance from VM snapshot

Hi all,

I am working on an enhancement to ACS - Deploy user instance from VM 
snapshot. Current work flow of deploying instance from volume snapshot is 
involves copying the snapshot image from primary storage to secondary storage 
and then to the primary storage where the instance is being deployed at. For 
some use cases like dev-test environments, user might want very quick 
deployment of instance from snapshot of existing VM (on primary storage).

JIRA ticket for this feature is [1].
FS for this feature is [2].
Please let me know your comments.

Added this feature to ACS road map wiki [3]

[1] https://issues.apache.org/jira/browse/CLOUDSTACK-8676
[2] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Deploy+instance+from+VM+snapshot
[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Roadmap

Regards,
Sateesh



[GitHub] cloudstack pull request:

2015-07-27 Thread pdion891
Github user pdion891 commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12377173
  
The jenkins job is triggered on new commits or daily. I've triggered it 
manually over the weekend because the last automated job failed. We can review 
the jenkins job but it shouldn't have changes recently.

On your side building systemvm64template work fine?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: Unit tests for HA manager investigate met...

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/624#issuecomment-125273928
  
LGTM :+1: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: [PROPOSAL] Deploy user instance from VM snapshot

2015-07-27 Thread Wido den Hollander


On 07/27/2015 07:32 PM, Somesh Naidu wrote:
 +1
 
 The FS LGTM.
 

Indeed, +1. Even in production this is very useful.

Wido

 Regards,
 Somesh
 
 
 -Original Message-
 From: Sateesh Chodapuneedi [mailto:sateesh.chodapune...@citrix.com] 
 Sent: Monday, July 27, 2015 2:11 AM
 To: dev@cloudstack.apache.org
 Subject: [PROPOSAL] Deploy user instance from VM snapshot
 
 Hi all,
 
 I am working on an enhancement to ACS - Deploy user instance from VM 
 snapshot. Current work flow of deploying instance from volume snapshot is 
 involves copying the snapshot image from primary storage to secondary storage 
 and then to the primary storage where the instance is being deployed at. For 
 some use cases like dev-test environments, user might want very quick 
 deployment of instance from snapshot of existing VM (on primary storage).
 
 JIRA ticket for this feature is [1].
 FS for this feature is [2].
 Please let me know your comments.
 
 Added this feature to ACS road map wiki [3]
 
 [1] https://issues.apache.org/jira/browse/CLOUDSTACK-8676
 [2] 
 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Deploy+instance+from+VM+snapshot
 [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Roadmap
 
 Regards,
 Sateesh
 


[GitHub] cloudstack pull request: Unit tests for HA manager investigate met...

2015-07-27 Thread DaanHoogland
Github user DaanHoogland commented on the pull request:

https://github.com/apache/cloudstack/pull/624#issuecomment-125350061
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Using cloudstack RabbitMQ events

2015-07-27 Thread Yiping Zhang
Hi, list:

First, please pardon me for some rant.

rant
I have been using cloudstack rabbitmq events to integrate with external apps 
since 4.3.x.  Recently we upgraded to CS 4.5.1 and noticed that CS 4.5 events 
are quite different from 4.3 events, at least for ones we are working with (for 
VM.CREATE and VM.DESTROY events, both the routing key and the event message are 
now different).

I have not found any documentation on such changes.   If  event notification 
framework is intended as an integration point with external apps,  I would have 
expected a stable and backward compatible interface to cloudstack events over 
upgrades, just like any other cloudstack API’s.   How can such changes be 
introduced and not documented in release notes?
/rant


While working with VM.CREATE and VM.DESTROY events (for CS 4.5.1), I noticed 
some weirdness of the messages when parsing them as JSON object.  Here is an 
example:


{

  cmdInfo: 
{\response\:\json\,\id\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\sessionkey\:\WY6E5WuM8SbqMw4bCumnVgGsgEQ\\u003d\,\ctxDetails\:\{\\\com.cloud.vm.VirtualMachine\\\:\\\b780c229-7064-47e5-97d0-a8b4590b36b8\\\}\,\cmdEventType\:\VM.DESTROY\,\ctxUserId\:\2\,\httpmethod\:\GET\,\_\:\1438027779033\,\uuid\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\ctxAccountId\:\2\,\ctxStartEventId\:\6282\},

  instanceType: VirtualMachine,

  instanceUuid: b780c229-7064-47e5-97d0-a8b4590b36b8,

  jobId: 61a62e5d-61ee-41eb-b947-0f8ef5d857c3,

  status: SUCCEEDED,

  processStatus: 0,

  commandEventType: VM.DESTROY,

  resultCode: 0,

  command: org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin,

  jobResult: 
org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{\id\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\name\:\yz-x1\,\displayname\:\yz-x1\,\account\:\admin\,\domainid\:\994ff03e-bb8f-11e4-b7d5-36d1d14da5e9\,\domain\:\ROOT\,\created\:\2015-07-27T12:01:12-0500\,\state\:\Destroyed\,\haenable\:false,\zoneid\:\1b0b4859-7b8a-41dd-8522-4dbf24345509\,\zonename\:\sjlab\,\templateid\:\e6fa410f-4bf0-4b3c-9982-9d60e7ffc07e\,\templatename\:\Base\,\templatedisplaytext\:\Base
 with 32 GB root and 
cloud-init\,\passwordenabled\:false,\serviceofferingid\:\11a5e901-bc78-45c6-8b81-a2a9e3530164\,\serviceofferingname\:\1CPU@1.0Ghz@1.5GB\,\cpunumber\:1,\cpuspeed\:1000,\memory\:1536,\cpuused\:\0.56%\,\networkkbsread\:0,\networkkbswrite\:2,\diskkbsread\:2670,\diskkbswrite\:163,\diskioread\:0,\diskiowrite\:0,\guestosid\:\a0c75a5b-bb8f-11e4-b7d5-36d1d14da5e9\,\rootdeviceid\:0,\rootdevicetype\:\ROOT\,\securitygroup\:[{\id\:\ad13aa78-bb8f-11e4-b7d5-36d1d14da5e9\,\name\:\default\,\description\:\Default
 Security 
Group\,\account\:\admin\,\ingressrule\:[],\egressrule\:[],\tags\:[]}],\nic\:[{\id\:\1c87d7e1-f8c9-425e-809d-7edd1a30c3a6\,\networkid\:\abe603fe-1d8b-4b23-9aa2-0234f18de686\,\networkname\:\vlan106\,\netmask\:\255.255.255.0\,\gateway\:\10.0.106.1\,\ipaddress\:\10.0.106.170\,\isolationuri\:\vlan://106\,\broadcasturi\:\vlan://106\,\traffictype\:\Guest\,\type\:\Shared\,\isdefault\:true,\macaddress\:\06:d9:2e:00:03:f6\}],\hypervisor\:\XenServer\,\instancename\:\i-2-346-VM\,\tags\:[],\details\:{\hypervisortoolsversion\:\xenserver56\},\affinitygroup\:[],\displayvm\:true,\isdynamicallyscalable\:true,\ostypeid\:206,\jobid\:\61a62e5d-61ee-41eb-b947-0f8ef5d857c3\,\jobstatus\:0},

  account: ad11bb05-bb8f-11e4-b7d5-36d1d14da5e9,

  user: ad129c91-bb8f-11e4-b7d5-36d1d14da5e9

}

There are two problems for above example:

 1.  The nested data structures are not parsed properly. As you can see, values 
for both “cmdInfo” and “jobResult” are strings instead of nested hash.  I have 
to make additional calls to JSON parser on those values to parse them.
 2.  The string 
org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/“ at the 
beginning of the value for “jobResult” makes this value invalid to be parsed as 
JSON object at all.

Thanks for listening.

Yiping





Re: Using cloudstack RabbitMQ events

2015-07-27 Thread Daan Hoogland
Yiping,


good rant; You are absolutely right. Did you create a ticket (or more
tickets) for this?

On Mon, Jul 27, 2015 at 11:47 PM, Yiping Zhang yzh...@marketo.com wrote:
 Hi, list:

 First, please pardon me for some rant.

 rant
 I have been using cloudstack rabbitmq events to integrate with external apps 
 since 4.3.x.  Recently we upgraded to CS 4.5.1 and noticed that CS 4.5 events 
 are quite different from 4.3 events, at least for ones we are working with 
 (for VM.CREATE and VM.DESTROY events, both the routing key and the event 
 message are now different).

 I have not found any documentation on such changes.   If  event notification 
 framework is intended as an integration point with external apps,  I would 
 have expected a stable and backward compatible interface to cloudstack events 
 over upgrades, just like any other cloudstack API’s.   How can such changes 
 be introduced and not documented in release notes?
 /rant


 While working with VM.CREATE and VM.DESTROY events (for CS 4.5.1), I noticed 
 some weirdness of the messages when parsing them as JSON object.  Here is an 
 example:


 {

   cmdInfo: 
 {\response\:\json\,\id\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\sessionkey\:\WY6E5WuM8SbqMw4bCumnVgGsgEQ\\u003d\,\ctxDetails\:\{\\\com.cloud.vm.VirtualMachine\\\:\\\b780c229-7064-47e5-97d0-a8b4590b36b8\\\}\,\cmdEventType\:\VM.DESTROY\,\ctxUserId\:\2\,\httpmethod\:\GET\,\_\:\1438027779033\,\uuid\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\ctxAccountId\:\2\,\ctxStartEventId\:\6282\},

   instanceType: VirtualMachine,

   instanceUuid: b780c229-7064-47e5-97d0-a8b4590b36b8,

   jobId: 61a62e5d-61ee-41eb-b947-0f8ef5d857c3,

   status: SUCCEEDED,

   processStatus: 0,

   commandEventType: VM.DESTROY,

   resultCode: 0,

   command: org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin,

   jobResult: 
 org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{\id\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\name\:\yz-x1\,\displayname\:\yz-x1\,\account\:\admin\,\domainid\:\994ff03e-bb8f-11e4-b7d5-36d1d14da5e9\,\domain\:\ROOT\,\created\:\2015-07-27T12:01:12-0500\,\state\:\Destroyed\,\haenable\:false,\zoneid\:\1b0b4859-7b8a-41dd-8522-4dbf24345509\,\zonename\:\sjlab\,\templateid\:\e6fa410f-4bf0-4b3c-9982-9d60e7ffc07e\,\templatename\:\Base\,\templatedisplaytext\:\Base
  with 32 GB root and 
 cloud-init\,\passwordenabled\:false,\serviceofferingid\:\11a5e901-bc78-45c6-8b81-a2a9e3530164\,\serviceofferingname\:\1CPU@1.0Ghz@1.5GB\,\cpunumber\:1,\cpuspeed\:1000,\memory\:1536,\cpuused\:\0.56%\,\networkkbsread\:0,\networkkbswrite\:2,\diskkbsread\:2670,\diskkbswrite\:163,\diskioread\:0,\diskiowrite\:0,\guestosid\:\a0c75a5b-bb8f-11e4-b7d5-36d1d14da5e9\,\rootdeviceid\:0,\rootdevicetype\:\ROOT\,\securitygroup\:[{\id\:\ad13aa78-bb8f-11e4-b7d5-36d1d14da5e9\,\name\:\default\,\description\:\Default
  Security 
 Group\,\account\:\admin\,\ingressrule\:[],\egressrule\:[],\tags\:[]}],\nic\:[{\id\:\1c87d7e1-f8c9-425e-809d-7edd1a30c3a6\,\networkid\:\abe603fe-1d8b-4b23-9aa2-0234f18de686\,\networkname\:\vlan106\,\netmask\:\255.255.255.0\,\gateway\:\10.0.106.1\,\ipaddress\:\10.0.106.170\,\isolationuri\:\vlan://106\,\broadcasturi\:\vlan://106\,\traffictype\:\Guest\,\type\:\Shared\,\isdefault\:true,\macaddress\:\06:d9:2e:00:03:f6\}],\hypervisor\:\XenServer\,\instancename\:\i-2-346-VM\,\tags\:[],\details\:{\hypervisortoolsversion\:\xenserver56\},\affinitygroup\:[],\displayvm\:true,\isdynamicallyscalable\:true,\ostypeid\:206,\jobid\:\61a62e5d-61ee-41eb-b947-0f8ef5d857c3\,\jobstatus\:0},

   account: ad11bb05-bb8f-11e4-b7d5-36d1d14da5e9,

   user: ad129c91-bb8f-11e4-b7d5-36d1d14da5e9

 }

 There are two problems for above example:

  1.  The nested data structures are not parsed properly. As you can see, 
 values for both “cmdInfo” and “jobResult” are strings instead of nested hash. 
  I have to make additional calls to JSON parser on those values to parse them.
  2.  The string 
 org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/“ at the 
 beginning of the value for “jobResult” makes this value invalid to be parsed 
 as JSON object at all.

 Thanks for listening.

 Yiping






-- 
Daan


Build failed in Jenkins: build-systemvm64-master #582

2015-07-27 Thread jenkins
See http://jenkins.buildacloud.org/job/build-systemvm64-master/582/changes

Changes:

[sanjeev] CLOUDSTACK-8664: Verify if queryAsyncJobResult api return 
jobinstanceid

[sanjeev] CLOUDSTACK-8659: Verify presentation of volume id in description of 
events table for 'SNAPSHOT.CREATE' type

[Daan Hoogland] coverity 1147049: buffered file reader resource guard

[Daan Hoogland] coverity 1133705: output writer close

[Daan Hoogland] coverity 1147051: trivial db resource leak in endpoint selector 
should have been nested try catches before try-with-resource beacame available

[Daan Hoogland] coverity 1125362, 1125363: assorted resource leaks

--
[...truncated 1440 lines...]
Executing command: chmod +x build_time.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/apt_upgrade.sh
 to apt_upgrade.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x apt_upgrade.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/configure_grub.sh
 to configure_grub.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x configure_grub.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/configure_locale.sh
 to configure_locale.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x configure_locale.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/configure_login.sh
 to configure_login.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x configure_login.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/configure_networking.sh
 to configure_networking.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x configure_networking.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/configure_acpid.sh
 to configure_acpid.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x configure_acpid.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/install_systemvm_packages.sh
 to install_systemvm_packages.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x install_systemvm_packages.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/configure_conntrack.sh
 to configure_conntrack.sh 
..
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Executing command: chmod +x configure_conntrack.sh
Waiting for ssh login on 127.0.0.1 with user root to sshd on port = 7222 to 
work, timeout=1200 sec
.
Transferring 
http://jenkins.buildacloud.org/job/build-systemvm64-master/ws/tools/appliance/definitions/systemvm64template-master-4.6.0/../../cloud_scripts_shar_archive.sh
 to cloud_scripts_shar_archive.sh 

Re: Using cloudstack RabbitMQ events

2015-07-27 Thread Yiping Zhang
Just files a Doc bug, CLOUDSTACK-8679 and a code bug CLOUDSTACK-8680.

Yiping

On 7/27/15, 3:29 PM, Daan Hoogland daan.hoogl...@gmail.com wrote:

Yiping,


good rant; You are absolutely right. Did you create a ticket (or more
tickets) for this?

On Mon, Jul 27, 2015 at 11:47 PM, Yiping Zhang yzh...@marketo.com wrote:
 Hi, list:

 First, please pardon me for some rant.

 rant
 I have been using cloudstack rabbitmq events to integrate with external
apps since 4.3.x.  Recently we upgraded to CS 4.5.1 and noticed that CS
4.5 events are quite different from 4.3 events, at least for ones we are
working with (for VM.CREATE and VM.DESTROY events, both the routing key
and the event message are now different).

 I have not found any documentation on such changes.   If  event
notification framework is intended as an integration point with external
apps,  I would have expected a stable and backward compatible interface
to cloudstack events over upgrades, just like any other cloudstack
API¹s.   How can such changes be introduced and not documented in
release notes?
 /rant


 While working with VM.CREATE and VM.DESTROY events (for CS 4.5.1), I
noticed some weirdness of the messages when parsing them as JSON object.
 Here is an example:


 {

   cmdInfo: 
{\response\:\json\,\id\:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\
sessionkey\:\WY6E5WuM8SbqMw4bCumnVgGsgEQ\\u003d\,\ctxDetails\:\{\\
\com.cloud.vm.VirtualMachine\\\:\\\b780c229-7064-47e5-97d0-a8b4590b36b
8\\\}\,\cmdEventType\:\VM.DESTROY\,\ctxUserId\:\2\,\httpmethod
\:\GET\,\_\:\1438027779033\,\uuid\:\b780c229-7064-47e5-97d0-a8b
4590b36b8\,\ctxAccountId\:\2\,\ctxStartEventId\:\6282\},

   instanceType: VirtualMachine,

   instanceUuid: b780c229-7064-47e5-97d0-a8b4590b36b8,

   jobId: 61a62e5d-61ee-41eb-b947-0f8ef5d857c3,

   status: SUCCEEDED,

   processStatus: 0,

   commandEventType: VM.DESTROY,

   resultCode: 0,

   command: 
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin,

   jobResult: 
org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{\id\
:\b780c229-7064-47e5-97d0-a8b4590b36b8\,\name\:\yz-x1\,\displaynam
e\:\yz-x1\,\account\:\admin\,\domainid\:\994ff03e-bb8f-11e4-b7d
5-36d1d14da5e9\,\domain\:\ROOT\,\created\:\2015-07-27T12:01:12-05
00\,\state\:\Destroyed\,\haenable\:false,\zoneid\:\1b0b4859-7b8
a-41dd-8522-4dbf24345509\,\zonename\:\sjlab\,\templateid\:\e6fa41
0f-4bf0-4b3c-9982-9d60e7ffc07e\,\templatename\:\Base\,\templatedisp
laytext\:\Base with 32 GB root and
cloud-init\,\passwordenabled\:false,\serviceofferingid\:\11a5e901-b
c78-45c6-8b81-a2a9e3530164\,\serviceofferingname\:\1CPU@1.0Ghz@1.5GB\
,\cpunumber\:1,\cpuspeed\:1000,\memory\:1536,\cpuused\:\0.56%\
,\networkkbsread\:0,\networkkbswrite\:2,\diskkbsread\:2670,\diskkb
swrite\:163,\diskioread\:0,\diskiowrite\:0,\guestosid\:\a0c75a5b-
bb8f-11e4-b7d5-36d1d14da5e9\,\rootdeviceid\:0,\rootdevicetype\:\ROO
T\,\securitygroup\:[{\id\:\ad13aa78-bb8f-11e4-b7d5-36d1d14da5e9\,\
name\:\default\,\description\:\Default Security
Group\,\account\:\admin\,\ingressrule\:[],\egressrule\:[],\tags
\:[]}],\nic\:[{\id\:\1c87d7e1-f8c9-425e-809d-7edd1a30c3a6\,\netwo
rkid\:\abe603fe-1d8b-4b23-9aa2-0234f18de686\,\networkname\:\vlan106
\,\netmask\:\255.255.255.0\,\gateway\:\10.0.106.1\,\ipaddress\
:\10.0.106.170\,\isolationuri\:\vlan://106\,\broadcasturi\:\vlan
://106\,\traffictype\:\Guest\,\type\:\Shared\,\isdefault\:true
,\macaddress\:\06:d9:2e:00:03:f6\}],\hypervisor\:\XenServer\,\in
stancename\:\i-2-346-VM\,\tags\:[],\details\:{\hypervisortoolsver
sion\:\xenserver56\},\affinitygroup\:[],\displayvm\:true,\isdynam
icallyscalable\:true,\ostypeid\:206,\jobid\:\61a62e5d-61ee-41eb-b94
7-0f8ef5d857c3\,\jobstatus\:0},

   account: ad11bb05-bb8f-11e4-b7d5-36d1d14da5e9,

   user: ad129c91-bb8f-11e4-b7d5-36d1d14da5e9

 }

 There are two problems for above example:

  1.  The nested data structures are not parsed properly. As you can
see, values for both ³cmdInfo² and ³jobResult² are strings instead of
nested hash.  I have to make additional calls to JSON parser on those
values to parse them.
  2.  The string 
org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/³ at
the beginning of the value for ³jobResult² makes this value invalid to
be parsed as JSON object at all.

 Thanks for listening.

 Yiping






-- 
Daan



[GitHub] cloudstack pull request: CLOUDSTACK-8652: remove old upgrade code

2015-07-27 Thread pdion891
Github user pdion891 commented on the pull request:

https://github.com/apache/cloudstack/pull/608#issuecomment-125378707
  
+1 for the concept, we need to get rid of old crappy stuff, none of those 
upgrade path are tested during RC and dev a release. Some of them have been 
removed from 4.5 release notes because they were not tested.

Although, some upgrade path should remain, maybe from 4.3.x. also, I 
wouldn't add this in 4.6 but maybe the next one 4.7 or 5.0? Out of curiosity, 
where the database structure get created/updated from removed code? 

In theory if someone want's to upgrade from something as old a 3.x to 4.6, 
he would probably need to do some intermediary upgrade, which is fairly 
standard in the firmware world and anyway, he wouldn't have just cloudstack to 
upgrade but far probably the hypervisor stack too.

My 2 cents ;)  I'm more in favor of a much more solid app confirmed by 
coverity then outdated upgrade path to a so-so version.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread sedukull
Github user sedukull commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35605790
  
--- Diff: reporter/usage-report-collector.py ---
@@ -0,0 +1,64 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# License); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+# 
+#   http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from flask import abort, Flask, request, Response
+from elasticsearch import Elasticsearch
+import json
+import time
+
+def json_response(response):
+return json.dumps(response, indent=2) + \n, 200, {'Content-Type': 
'application/json; charset=utf-8'}
+
+def generate_app(config=None):
+app = Flask(__name__)
+
+@app.route('/report/unique_id', methods=['POST'])
+def report(unique_id):
+# We expect JSON data, so if the Content-Type doesn't match JSON 
data we throw an error
+if 'Content-Type' in request.headers:
+if request.headers['Content-Type'] != 'application/json':
+abort(417, No or incorrect Content-Type header was 
supplied)
+
+index = cloudstack-%s % time.strftime(%Y.%m.%d, time.gmtime())
+timestamp = time.strftime(%Y-%m-%dT%H:%M:%SZ, time.gmtime())
+
+es = Elasticsearch()
+es.indices.create(index=index, ignore=400)
+
+report = json.loads(request.data)
+report[unique_id] = unique_id
+report[timestamp] = timestamp
+
+es.index(index=index, doc_type=usage-report, 
body=json.dumps(report), timestamp=timestamp, refresh=True)
+
+response = {}
+return json_response(response)
+
+return app
+
+
+app = generate_app()
+
+# Only run the App if this script is invoked from a Shell
+if __name__ == '__main__':
+app.debug = True
+app.run(host='0.0.0.0', port=8088)
+
+# Otherwise provide a variable called 'application' for mod_wsgi
+else:
--- End diff --

Where is this application var used? Its not global var as well?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread sedukull
Github user sedukull commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35605766
  
--- Diff: reporter/usage-report-collector.py ---
@@ -0,0 +1,64 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# License); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+# 
+#   http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from flask import abort, Flask, request, Response
+from elasticsearch import Elasticsearch
+import json
+import time
+
+def json_response(response):
+return json.dumps(response, indent=2) + \n, 200, {'Content-Type': 
'application/json; charset=utf-8'}
+
+def generate_app(config=None):
+app = Flask(__name__)
+
+@app.route('/report/unique_id', methods=['POST'])
+def report(unique_id):
+# We expect JSON data, so if the Content-Type doesn't match JSON 
data we throw an error
+if 'Content-Type' in request.headers:
+if request.headers['Content-Type'] != 'application/json':
+abort(417, No or incorrect Content-Type header was 
supplied)
+
+index = cloudstack-%s % time.strftime(%Y.%m.%d, time.gmtime())
+timestamp = time.strftime(%Y-%m-%dT%H:%M:%SZ, time.gmtime())
+
+es = Elasticsearch()
+es.indices.create(index=index, ignore=400)
+
+report = json.loads(request.data)
--- End diff --

May be we wanted to add a small check to see report is None and proceed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread sedukull
Github user sedukull commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35605842
  
--- Diff: reporter/usage-report-collector.py ---
@@ -0,0 +1,64 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# License); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+# 
+#   http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from flask import abort, Flask, request, Response
--- End diff --

This is good work i would say, but just add some comments on this service 
purpose, may be for people seeing it later can understand it easily. Any reason 
to have this service written in python and other usage code to run along with 
cs in java?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8677: Call-home functionality ...

2015-07-27 Thread sedukull
Github user sedukull commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/625#discussion_r35606107
  
--- Diff: server/src/org/apache/cloudstack/report/UsageReporter.java ---
@@ -0,0 +1,487 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// License); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.report;
+
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.net.URL;
+import java.net.SocketTimeoutException;
+import java.net.MalformedURLException;
+import java.net.ProtocolException;
+import java.net.UnknownHostException;
+import java.io.OutputStreamWriter;
+import java.io.IOException;
+
+import javax.inject.Inject;
+import javax.net.ssl.HttpsURLConnection;
+
+import org.apache.log4j.Logger;
+import org.springframework.stereotype.Component;
+
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.managed.context.ManagedContextRunnable;
+
+import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.StoragePoolVO;
+
+import org.apache.commons.codec.digest.DigestUtils;
+
+import com.cloud.host.HostVO;
+import com.cloud.host.dao.HostDao;
+import com.cloud.dc.ClusterVO;
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.vm.UserVmVO;
+import com.cloud.vm.dao.UserVmDao;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.component.ManagerBase;
+import com.cloud.utils.component.ComponentMethodInterceptable;
+import com.cloud.utils.concurrency.NamedThreadFactory;
+import com.cloud.utils.db.DB;
+import com.cloud.utils.db.TransactionLegacy;
+import com.cloud.upgrade.dao.VersionDao;
+import com.cloud.upgrade.dao.VersionVO;
+import com.cloud.storage.dao.DiskOfferingDao;
+import com.cloud.storage.DiskOfferingVO;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+import com.google.common.util.concurrent.AtomicLongMap;
+
+@Component
+public class UsageReporter extends ManagerBase implements 
ComponentMethodInterceptable {
+public static final Logger s_logger = 
Logger.getLogger(UsageReporter.class.getName());
+
+/* !FIX ME! This should point to a Apache Infra host with SSL! */
+private String reportHost = https://call-home.cloudstack.org/report;;
+
+private String uniqueID = null;
+
+private static UsageReporter s_instance = null;
+
+private ScheduledExecutorService _executor = null;
+
+@Inject
+private ConfigurationDao _configDao;
+@Inject
+private HostDao _hostDao;
+@Inject
+private ClusterDao _clusterDao;
+@Inject
+private PrimaryDataStoreDao _storagePoolDao;
+@Inject
+private DataCenterDao _dataCenterDao;
+@Inject
+private UserVmDao _userVmDao;
+@Inject
+private VMInstanceDao _vmInstance;
+@Inject
+private VersionDao _versionDao;
+@Inject
+private DiskOfferingDao _diskOfferingDao;
+
+int usageReportInterval = -1;
+
+public static UsageReporter getInstance() {
+return s_instance;
+}
+
+public static UsageReporter getInstance(MapString, String configs) {
+s_instance.init(configs);
+return s_instance;
+}
+
+public UsageReporter() {
  

[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread bhaisaab
Github user bhaisaab commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/618#discussion_r35521326
  
--- Diff: 
plugins/user-authenticators/saml2/src/org/apache/cloudstack/saml/SAML2AuthManagerImpl.java
 ---
@@ -119,12 +123,12 @@
 @Override
 public boolean start() {
 if (isSAMLPluginEnabled()) {
-setup();
 s_logger.info(SAML auth plugin loaded);
+return setup();
 } else {
--- End diff --

we can remove else {} here just keep;
if (condition){
return stuff();
}
return otherstuff();


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread bhaisaab
Github user bhaisaab commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/618#discussion_r35521408
  
--- Diff: 
plugins/user-authenticators/saml2/src/org/apache/cloudstack/saml/SAML2AuthManagerImpl.java
 ---
@@ -358,7 +363,8 @@ private boolean setup() {
 }
 _timer = new Timer();
 final HttpClient client = new HttpClient();
-final String idpMetaDataUrl = 
SAMLIdentityProviderMetadataURL.value();
+//final String idpMetaDataUrl = 
SAMLIdentityProviderMetadataURL.value();
--- End diff --

Remove commented deadcode, or simply keep final String idpMetaDataUrl = 
SAMLIdentityProviderMetadataURL.value(); as not many consumers of the new 
getSAMLIdentityProviderMetadataURL() method.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread bhaisaab
Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/618#issuecomment-125154356
  
@kansal Please see my comments. While it's okay, please squash the commits 
into two separate commits; one targeting the saml related stuff and second one 
targeting the other component, so as to allow us to port the commits to other 
branches.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request:

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:


https://github.com/apache/cloudstack/commit/08348593a2ec03767e92ff247f6279df76434194#commitcomment-12368701
  
Hi @pdion891 

Quite weird since it was tested with a SVM having the old version of 
keepalived (1.2.2) and installing the new one (1.2.13), from back-ports and 
also we have a SVM that was successfully built on the 22nd July, after the 
merge. I also used that new SVM (22nd July) to run tests and to check the 
keepalived, and it was working and with the version 1.2.13.

SVM here: http://jenkins.buildacloud.org/job/build-systemvm64-master/577/

The code was merged on Thu Jul 16. Prior to the SVM pointed above. I also 
tested with the one created on the 17th July. Checking the failed job, it was 
started manually. Has the job configuration been changed?

Unfortunately the SVM from 17th July is no longer available via Jenkins. 
Perhaps would be nice to keep the last 2 successful builds just for reference.

Do you have any clue why the build from 22nd worked and the SVM is doing 
just fine, including keepalived 1.2.13?

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8610. Unable to attach 7th Dis...

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/554#issuecomment-125155019
  
+1 for @DaanHoogland's comments: please add unit tests to cover the changes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8666: Put host in Alert state ...

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/621#issuecomment-12513
  
Hi @koushik-das 

Perhaps refactoring the code in a way that its design is improved could 
make easy to add tests for it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8666: Put host in Alert state ...

2015-07-27 Thread koushik-das
Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/621#issuecomment-125156829
  
@wilderrodrigues Agree that refactoring the code might help with the tests. 
But that will be a bigger effort.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8651: [Browser Based Upload Te...

2015-07-27 Thread koushik-das
Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/607#issuecomment-125157246
  
@DaanHoogland Are you running some private simulator test? If so is it done?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: Loading a custom HostAllocator

2015-07-27 Thread Amy Niklasson
I have tried this and I still can't find it. I have tried to find it both
in the regular installation and the source installation of CloudStack.
However I have managed to find this link
http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.0.html
which implies that the file has changed name to componentContext.xml.
Unfortunately I can't find this file either.

I have found the file
/home/ubuntu/cloudstack/server/test/async-job-component.xml which contains
the lines:

adapters key=com.cloud.agent.manager.allocator.HostAllocator
  adapter name=FirstFit
class=com.cloud.agent.manager.allocator.impl.FirstFitAllocator/
/adapters

Which are the lines according to
http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-
http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-custom-hostallocatorcustom-hostallocator
http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-custom-hostallocator
should be changed. However this location is in nowhere close to the
one mentioned.

If anyone has any experience of implementing a custom hostallocator in
CloudStack 4.5 or knows anything I would very much appreciate any
information you got.

Kind Regards,
Amy


On 8 July 2015 at 09:47, Wido den Hollander w...@widodh.nl wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1



 On 08-07-15 10:08, Amy Niklasson wrote:
  Hi, How do you load a custom HostAllocator in CloudStack 4.5? The
  only documentation I've managed to find is
  http://docs.cloudstack.apache.org/en/master/alloc.html#implementing-a-
 custom-hostallocator
 
 
 However,I can't find the files mentioned, components.xml and
  components-premium.xml, in /client/ tomcatconf. Anyone who knows?
 

 Not exactly sure which file it is, but you could try a grep through
 the code.

 It's the Spring framework which loads the modules in boot.

 Wido

  Regards, Amy
 
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1

 iQIcBAEBAgAGBQJVnOOoAAoJEAGbWC3bPspCOLUP/jSIaX0/3nXp3o33BoC3udOb
 YDxBcgQA6LdHQEWNJUWfKiUmc5nPbVmpcEnVB+wLp1CFmWbG04Mbkq06Ujhjx5W3
 EqH5sulWGKG5RbBvtBbhA52rs/SJw4WuKhVULItTk3VqhgZQn9Sz3nmVgzO9p8fU
 2GQCRNXkA5IGNCcq1/3d44Y0zWz0Vc7b2ghzhHFsVqDjorJF5I1m62F1uIXz12w3
 vxZtQtdkdWF8xzRuwQ46v1/R0oYHYWKXqkS3S9JyGjIjOiWKV8i0eg1KVxEGO8qU
 3UUZOw7eiReCP2LXEkX1r703oU835y177il7tWDOAibg4s22vqtWmuquCSCP1g5t
 Avvzoz+8IPD6r1JqnViFpXAPdzD4DSNol3yIJ+s6iDSMasqpfTrHpKREepCFE0F+
 EpqsWPXLIMzdTvGZpgKL5CV4BUp8/lf9CjJ8ZvE/34N6kphUe7LbYawLs5B8Ijp6
 QkWttmwuy+esUkxxTOeNuO3ghKKqp92CvT6vw8jQLz8vETgSYLGXFeUkOlHNZU3q
 bTwi2O75cpsKNieUwTY4WkKVleED/tY+8xMz4RWyDR9OopLimN5Pkp1JsNOBtux3
 w+L0ffr4E5HUoA5nw8x/DE5MxdbQA70/4QPe+k0jOuyVRGcxhY2h5ZdJBr74NqqR
 CHEAC0F969Szp+oUZuBR
 =lBgf
 -END PGP SIGNATURE-



[4.6] RC1 soon ?

2015-07-27 Thread Pierre-Luc Dion
Hi,

I've create this jira filter[1] for the Release Manager, based on it, there
would be only 4 maybe just 3 blockers on 4.6. Based on this, should we
consider placing a target date for RC1 somewhere like end of August ?

What's missing and to do in 4.6 as far as I know:

1. Basic documentation of new features,
2. Decide if we let it in master and freeze merge during RC ?
3. Build all job as 4.6 in jenkins ?
4. Organize a QA-party, like old time lan-party

[1] https://issues.apache.org/jira/issues/?filter=12332940


[GitHub] cloudstack pull request: fix systemvm64template build, fix .deb pa...

2015-07-27 Thread pdion891
GitHub user pdion891 opened a pull request:

https://github.com/apache/cloudstack/pull/626

fix systemvm64template build, fix .deb package creation.

- build .deb package job must be change for ``dpkg-buildpackage -uc -us`` 
ubuntu 14.04 enforce signature of package. CLOUDSTACK-8638

- @wilderrodrigues: this fix dependency for keepalived on building 
systemvm64template. Changing ``libnl1`` by ``libnl-3-200 libnl-genl-3-200`` 
seams resolve the build problem.

For unknown reason the default way to retrieve version ``mvn 
org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate 
-Dexpression=project.version | grep -v \[`` does not work in package 
creation, it return a null value.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pdion891/cloudstack 8675

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/626.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #626


commit 6a01f48e1bb55826790af354686d150b5d440e02
Author: Pierre-Luc Dion pdion...@apache.org
Date:   2015-07-28T00:49:40Z

fix dependency for keepalived from wheezy-backports

commit a3319b76ba04f64f3121470a976799c44a58d53b
Author: Pierre-Luc Dion pdion...@apache.org
Date:   2015-07-28T01:16:30Z

fix release version automatically updated using pom.xml

commit d4bfca6330a3fbe50156e9e249a2535e80be9cda
Author: Pierre-Luc Dion pdion...@apache.org
Date:   2015-07-28T01:20:48Z

change last owner of change for pgp signature




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[CLEANUP] j.bac.o: duplicate jobs and outdated

2015-07-27 Thread Pierre-Luc Dion
Hi,

I've spend some times understanding Jenkins job on j.bac.o and it look like
there is few job that should be removed and pause? Maybe persons aware of
them can answer?

1. There is 2 build-master-slowbuild
http://jenkins.buildacloud.org/job/build-master-slowbuild/ running every
3hours , this job take ~2h, does just 1 would be enough?
2. We start to have failed/offline slave, who have access to them to fix
them?
*  cloudstack-buildslave-static-001
   * fastbuildserver
   * linodebuildserver
   * rpmbuilder-2
   * systemvmbuilder
3. view: cloudstack-qa-master   half jobs are in RED
4. fastsimulatorbuild:  I think this job use a old branch, not even testing
master ?
5. do we drop jobs for 4.3 ?


Thanks

PL


[PROPOSAL] Deploy user instance from VM snapshot

2015-07-27 Thread Sateesh Chodapuneedi
Hi all,

I am working on an enhancement to ACS - Deploy user instance from VM 
snapshot. Current work flow of deploying instance from volume snapshot is 
involves copying the snapshot image from primary storage to secondary storage 
and then to the primary storage where the instance is being deployed at. For 
some use cases like dev-test environments, user might want very quick 
deployment of instance from snapshot of existing VM (on primary storage).

JIRA ticket for this feature is [1].
FS for this feature is [2].
Please let me know your comments.

Added this feature to ACS road map wiki [3]

[1] https://issues.apache.org/jira/browse/CLOUDSTACK-8676
[2] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Deploy+instance+from+VM+snapshot
[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Roadmap

Regards,
Sateesh



[GitHub] cloudstack pull request: CLOUDSTACK-8664: Verify if queryAsyncJobR...

2015-07-27 Thread pritisarap12
Github user pritisarap12 commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/619#discussion_r35508814
  
--- Diff: test/integration/testpaths/testpath_queryAsyncJobResult.py ---
@@ -0,0 +1,133 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# License); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+ Test case to check if queryAsyncJobResult returns jobinstanceid 
+
+from nose.plugins.attrib import attr
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.lib.utils import (cleanup_resources)
+from marvin.lib.base import (Account,
+ ServiceOffering,
+ VirtualMachine,
+ )
+from marvin.lib.common import (get_domain,
+   get_zone,
+   get_template,
+  )
+
+from marvin.cloudstackAPI import queryAsyncJobResult
+
+class TestJobinstanceid(cloudstackTestCase):
+
+@classmethod
+def setUpClass(cls):
+testClient = super(TestJobinstanceid, cls).getClsTestClient()
+cls.apiclient = testClient.getApiClient()
+cls.testdata = testClient.getParsedTestDataConfig()
+
+cls.hypervisor = cls.testClient.getHypervisorInfo()
+# Get Zone, Domain and templates
+cls.domain = get_domain(cls.apiclient)
+cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
+
+cls.template = get_template(
+cls.apiclient,
+cls.zone.id,
+cls.testdata[ostype])
+
+cls._cleanup = []
+
+try:
+
+# Create an account
+cls.account = Account.create(
+cls.apiclient,
+cls.testdata[account],
+domainid=cls.domain.id
+)
+# Create user api client of the account
+cls.userapiclient = testClient.getUserApiClient(
+UserName=cls.account.name,
+DomainName=cls.account.domain
+)
+
+   # Create Service offering
+cls.service_offering = ServiceOffering.create(
+cls.apiclient,
+cls.testdata[service_offering],
+)
+
+cls._cleanup = [
+cls.account,
+cls.service_offering,
+]
+except Exception as e:
+cls.tearDownClass()
+raise e
+return
+
+
+@classmethod
+def tearDownClass(cls):
+try:
+cleanup_resources(cls.apiclient, cls._cleanup)
+except Exception as e:
+raise Exception(Warning: Exception during cleanup : %s % e)
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.cleanup = []
+
+def tearDown(self):
+try:
+cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+raise Exception(Warning: Exception during cleanup : %s % e)
+return
+
+@attr(tags=[advanced, basic], required_hardware=false)
+def test_queryAsyncJobResult_jobinstanceid(self):
+ Test queryAsyncJobResult api return jobinstanceid
+
+# 1. Deploy a VM 
+# 2. Call queryAsyncJobResult API with jobid of previous step  
+# 3. Verify that queryAsyncJobResult returns jobinstanceid
+
+
+# Step 1
+
--- End diff --

Fixed pep8 issues.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: [GitHub] cloudstack pull request: CLOUDSTACK-8664: Verify if queryAsyncJobR...

2015-07-27 Thread Sanjeev N
LGTM!!

On Mon, Jul 27, 2015 at 12:11 PM, pritisarap12 g...@git.apache.org wrote:

 Github user pritisarap12 commented on a diff in the pull request:

 https://github.com/apache/cloudstack/pull/619#discussion_r35508814

 --- Diff: test/integration/testpaths/testpath_queryAsyncJobResult.py
 ---
 @@ -0,0 +1,133 @@
 +# Licensed to the Apache Software Foundation (ASF) under one
 +# or more contributor license agreements.  See the NOTICE file
 +# distributed with this work for additional information
 +# regarding copyright ownership.  The ASF licenses this file
 +# to you under the Apache License, Version 2.0 (the
 +# License); you may not use this file except in compliance
 +# with the License.  You may obtain a copy of the License at
 +#
 +#   http://www.apache.org/licenses/LICENSE-2.0
 +#
 +# Unless required by applicable law or agreed to in writing,
 +# software distributed under the License is distributed on an
 +# AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 +# KIND, either express or implied.  See the License for the
 +# specific language governing permissions and limitations
 +# under the License.
 + Test case to check if queryAsyncJobResult returns jobinstanceid
 +
 +from nose.plugins.attrib import attr
 +from marvin.cloudstackTestCase import cloudstackTestCase
 +from marvin.lib.utils import (cleanup_resources)
 +from marvin.lib.base import (Account,
 + ServiceOffering,
 + VirtualMachine,
 + )
 +from marvin.lib.common import (get_domain,
 +   get_zone,
 +   get_template,
 +  )
 +
 +from marvin.cloudstackAPI import queryAsyncJobResult
 +
 +class TestJobinstanceid(cloudstackTestCase):
 +
 +@classmethod
 +def setUpClass(cls):
 +testClient = super(TestJobinstanceid, cls).getClsTestClient()
 +cls.apiclient = testClient.getApiClient()
 +cls.testdata = testClient.getParsedTestDataConfig()
 +
 +cls.hypervisor = cls.testClient.getHypervisorInfo()
 +# Get Zone, Domain and templates
 +cls.domain = get_domain(cls.apiclient)
 +cls.zone = get_zone(cls.apiclient,
 testClient.getZoneForTests())
 +
 +cls.template = get_template(
 +cls.apiclient,
 +cls.zone.id,
 +cls.testdata[ostype])
 +
 +cls._cleanup = []
 +
 +try:
 +
 +# Create an account
 +cls.account = Account.create(
 +cls.apiclient,
 +cls.testdata[account],
 +domainid=cls.domain.id
 +)
 +# Create user api client of the account
 +cls.userapiclient = testClient.getUserApiClient(
 +UserName=cls.account.name,
 +DomainName=cls.account.domain
 +)
 +
 +   # Create Service offering
 +cls.service_offering = ServiceOffering.create(
 +cls.apiclient,
 +cls.testdata[service_offering],
 +)
 +
 +cls._cleanup = [
 +cls.account,
 +cls.service_offering,
 +]
 +except Exception as e:
 +cls.tearDownClass()
 +raise e
 +return
 +
 +
 +@classmethod
 +def tearDownClass(cls):
 +try:
 +cleanup_resources(cls.apiclient, cls._cleanup)
 +except Exception as e:
 +raise Exception(Warning: Exception during cleanup : %s
 % e)
 +
 +def setUp(self):
 +self.apiclient = self.testClient.getApiClient()
 +self.dbclient = self.testClient.getDbConnection()
 +self.cleanup = []
 +
 +def tearDown(self):
 +try:
 +cleanup_resources(self.apiclient, self.cleanup)
 +except Exception as e:
 +raise Exception(Warning: Exception during cleanup : %s
 % e)
 +return
 +
 +@attr(tags=[advanced, basic], required_hardware=false)
 +def test_queryAsyncJobResult_jobinstanceid(self):
 + Test queryAsyncJobResult api return jobinstanceid
 +
 +# 1. Deploy a VM
 +# 2. Call queryAsyncJobResult API with jobid of previous step
 +# 3. Verify that queryAsyncJobResult returns jobinstanceid
 +
 +
 +# Step 1
 +
 --- End diff --

 Fixed pep8 issues.


 ---
 If your project is set up for it, you can reply to this email and have your
 reply appear on GitHub as well. If your project does not have this 

[GitHub] cloudstack pull request: CLOUDSTACK-8664: Verify if queryAsyncJobR...

2015-07-27 Thread sanju1010
Github user sanju1010 commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/619#discussion_r35507858
  
--- Diff: test/integration/testpaths/testpath_queryAsyncJobResult.py ---
@@ -0,0 +1,133 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# License); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+ Test case to check if queryAsyncJobResult returns jobinstanceid 
+
+from nose.plugins.attrib import attr
+from marvin.cloudstackTestCase import cloudstackTestCase
+from marvin.lib.utils import (cleanup_resources)
+from marvin.lib.base import (Account,
+ ServiceOffering,
+ VirtualMachine,
+ )
+from marvin.lib.common import (get_domain,
+   get_zone,
+   get_template,
+  )
+
+from marvin.cloudstackAPI import queryAsyncJobResult
+
+class TestJobinstanceid(cloudstackTestCase):
+
+@classmethod
+def setUpClass(cls):
+testClient = super(TestJobinstanceid, cls).getClsTestClient()
+cls.apiclient = testClient.getApiClient()
+cls.testdata = testClient.getParsedTestDataConfig()
+
+cls.hypervisor = cls.testClient.getHypervisorInfo()
+# Get Zone, Domain and templates
+cls.domain = get_domain(cls.apiclient)
+cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
+
+cls.template = get_template(
+cls.apiclient,
+cls.zone.id,
+cls.testdata[ostype])
+
+cls._cleanup = []
+
+try:
+
+# Create an account
+cls.account = Account.create(
+cls.apiclient,
+cls.testdata[account],
+domainid=cls.domain.id
+)
+# Create user api client of the account
+cls.userapiclient = testClient.getUserApiClient(
+UserName=cls.account.name,
+DomainName=cls.account.domain
+)
+
+   # Create Service offering
+cls.service_offering = ServiceOffering.create(
+cls.apiclient,
+cls.testdata[service_offering],
+)
+
+cls._cleanup = [
+cls.account,
+cls.service_offering,
+]
+except Exception as e:
+cls.tearDownClass()
+raise e
+return
+
+
+@classmethod
+def tearDownClass(cls):
+try:
+cleanup_resources(cls.apiclient, cls._cleanup)
+except Exception as e:
+raise Exception(Warning: Exception during cleanup : %s % e)
+
+def setUp(self):
+self.apiclient = self.testClient.getApiClient()
+self.dbclient = self.testClient.getDbConnection()
+self.cleanup = []
+
+def tearDown(self):
+try:
+cleanup_resources(self.apiclient, self.cleanup)
+except Exception as e:
+raise Exception(Warning: Exception during cleanup : %s % e)
+return
+
+@attr(tags=[advanced, basic], required_hardware=false)
+def test_queryAsyncJobResult_jobinstanceid(self):
+ Test queryAsyncJobResult api return jobinstanceid
+
+# 1. Deploy a VM 
+# 2. Call queryAsyncJobResult API with jobid of previous step  
+# 3. Verify that queryAsyncJobResult returns jobinstanceid
+
+
+# Step 1
+
--- End diff --

Please fix pep8 issues


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


RE: [Proposal] Replacing Openswan ipsec with Strongswan ipsec

2015-07-27 Thread Suresh Sadhu
+1 

Kindly share any practical limitation or any know issues exists in strong 
swan[if anybody using this protocol  in real time ,please share  your 
experience ]...  that will really helpful us(both dev and qa) to design ,code 
and test this feature in a better way.


Regards
Sadhu



-Original Message-
From: Jayapal Reddy Uradi [mailto:jayapalreddy.ur...@citrix.com] 
Sent: 24 July 2015 17:40
To: dev@cloudstack.apache.org
Subject: [Proposal] Replacing Openswan ipsec with Strongswan ipsec

Hi All,

Openswan is not being actively maintained by community.
Any security updates to the packages then it is difficult to upgrade.
latest version of OS X clients are not working on openswan.

To address the above concerns I am proposing to move from openswan ipsec to 
strongswan ipsec.

I will be sharing the FS for this feature soon.

Thanks,
Jayapal




[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread kansal
Github user kansal commented on the pull request:

https://github.com/apache/cloudstack/pull/618#issuecomment-125106632
  
@DaanHoogland Added the test file. Checking only the false cases. 
@wilderrodrigues Got away with the IF statements and replaced with the api 
@karuturi suggested. Please review the changes. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8664: Verify if queryAsyncJobR...

2015-07-27 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/619


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack

2015-07-27 Thread Wido den Hollander
Hi,

It has been quite some time and this feature hasn't merged yet.

I created a issue for this:
https://issues.apache.org/jira/browse/CLOUDSTACK-8677

The code is ready to merge and is in the reporter [0] branch on the
Git repo.

Even if the VM isn't ready, I would like to merge this feature into 4.6
and get a VERY CLEAR doc into the Release Notes.

Without Infra to send stats to, it seems like this feature is worthless,
but it isn't. Since we got the code running, so as soon as we get
call-home.cloudstack.org online we are good.

Merging would just be sending a PR, before I do so, any objections?

Wido

[0]:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter

On 01-12-14 14:08, Wido den Hollander wrote:
 Hello,
 
 As a project we currently don't have a lot of insight information on
 about how CloudStack is being used. Surveys tell us a lot, but not
 everybody fills in the survey, so we still miss a lot of information.
 
 That's why I've written the Usage Reporting functionality for the
 management server which automatically sends back anonymous information
 about a CloudStack deployment.
 
 It's currently in the 'reporter' branch. [0]
 
 By default, every 7 days it generates a JSON document with:
 - Hosts (Number, version, type, hypervisor)
 - Clusters (Hypervisor en Management type)
 - Primary storage (Type and provider)
 - Zones (Network type and providers)
 - Instances (Number and types)
 
 This report is not complete yet, I'd like to add more information, but
 that will be Management Server information.
 
 The code on how this report is generated is obviously 100% Open Source,
 so end-users can always exactly see how the information was compiled.
 
 I want to discuss this new feature for CloudStack and the possible
 implications it might have.
 
 I'm opting for a opt-out. So every new or upgraded install to 4.6.0
 (master) will have this enabled. Yes, we have to be very explicit in the
 Release Notes that this has been added.
 
 Why? It's the small price we as a project ask for using CloudStack. We
 want a little bit of information on how CloudStack is being used so that
 we can use this to make CloudStack even better.
 
 Turning it off is also just one global setting and it will never turn on
 again.
 
 On the server-side there is a Python flask application [1] (found in the
 reporter directory) which stores all the incoming information in a
 ElasticSearch database. From there analytics can be gathered on
 CloudStack deployments.
 
 It currently points to http://cs-report.widodh.nl/report which will NOT
 be the endpoint when this is merged into master.
 
 For 'production' I want to have
 https://report.cloudstack.apache.org/report where all reports are submitted.
 
 For every setup a unique ID is determined by hashing the first row in
 the 'version' table. This is the version + timestamp and that is hashed
 using SHA256. Using this unique ID we can track changes in deployments
 and see how they grow or shrink.
 
 Technically this wasn't that hard to implement, but the politics
 surrounding it might be the hardest part.
 
 What do other have to say about this? Should there be a VOTE for this
 feature to come into CloudStack? Opt-in, opt-out?
 
 Wido
 
 [0]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
 [1]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter
 


[GitHub] cloudstack pull request: CLOUDSTACK-8651: [Browser Based Upload Te...

2015-07-27 Thread wido
Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/607#issuecomment-125133501
  
I think this one can be merged after simulator is done, right?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: [BLOCKER] unable to start VR in basic zone on master

2015-07-27 Thread Rajani Karuturi
I faced the same issue with basic zone using xenserver on master.

Wilder,
Can you please comment on this?

~Rajani

On Fri, Jul 24, 2015 at 4:00 PM, Kishan Kavala kishan.kav...@citrix.com
wrote:

 @Wilder, can you please look into this issue.
 Looks like VR config file is incomplete and it is over-writing the IP
 information set by cloud-early-config.

 VR Config file:
 #Apache CloudStack Virtual Router Config File
 version
 1.0
 /version
 file
 /var/cache/cloud/monitor_service.json

 {config:[dhcp]:processname=dnsmasq:servicename=dnsmasq:pidfile=/var/run/dnsmasq/dnsmasq.pid:,[ssh]:processname=sshd:servicename=ssh:pidfile=/var/run/sshd.pid:,[webserver]:processname=apache2:servicename=apache2:pidfile=/var/run/apache2.pid:,,type:monitorservice}
 /file
 script
 /opt/cloud/bin/update_config.py monitor_service.json
 /script

 /var/log/cloud.log:
 2015-07-24 09:29:28,997 Loading data bag type monitorservice
 2015-07-24 09:29:28,998 Command of type monitorservice received
 2015-07-24 09:29:28,999 Writing data bag type monitorservice
 2015-07-24 09:29:28,999 Creating data bag type ips
 2015-07-24 09:29:28,999 Loading data bag type cmdline
 2015-07-24 09:29:29,000 Executing ip addr show dev eth1
 2015-07-24 09:29:29,019 Will remove all configured addresses on device eth1
 2015-07-24 09:29:29,019 Removing addresses from device eth1
 2015-07-24 09:29:29,036 Removed address 169.254.2.130/16 from device eth1
 2015-07-24 09:29:29,056 Removed address 169.254.2.130/16 from device eth1
 2015-07-24 09:29:29,057 Executing ip addr show dev eth0
 2015-07-24 09:29:29,075 Will remove all configured addresses on device eth0
 2015-07-24 09:29:29,076 Removing addresses from device eth0
 2015-07-24 09:29:29,093 Removed address 10.220.39.131/19 from device eth0
 2015-07-24 09:29:29,112 Removed address 10.220.39.131/19 from device eth


 -Original Message-
 From: Sanjeev N [mailto:sanj...@apache.org]
 Sent: 24 July 2015 12:06 PM
 To: dev@cloudstack.apache.org
 Subject: [BlockerBug] VR creation fails in basic zone

 Hi,

 In latest build with ACS master, VR creation fails in basic zone.
 Created a blocker bug for this. More details @
 https://issues.apache.org/jira/browse/CLOUDSTACK-8668

 Thanks,
 Sanjeev



[GitHub] cloudstack pull request: CLOUDSTACK-8659: Verify presentation of v...

2015-07-27 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/613


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread wilderrodrigues
Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/618#issuecomment-125150991
  
LGTM :+1: 

Thanks for the improvements!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---



Re: [DISCUSS] Automatic usage reporting / call-home for CloudStack

2015-07-27 Thread Daan Hoogland
gogogo

On Mon, Jul 27, 2015 at 10:28 AM, Wido den Hollander w...@widodh.nl wrote:
 Hi,

 It has been quite some time and this feature hasn't merged yet.

 I created a issue for this:
 https://issues.apache.org/jira/browse/CLOUDSTACK-8677

 The code is ready to merge and is in the reporter [0] branch on the
 Git repo.

 Even if the VM isn't ready, I would like to merge this feature into 4.6
 and get a VERY CLEAR doc into the Release Notes.

 Without Infra to send stats to, it seems like this feature is worthless,
 but it isn't. Since we got the code running, so as soon as we get
 call-home.cloudstack.org online we are good.

 Merging would just be sending a PR, before I do so, any objections?

 Wido

 [0]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter

 On 01-12-14 14:08, Wido den Hollander wrote:
 Hello,

 As a project we currently don't have a lot of insight information on
 about how CloudStack is being used. Surveys tell us a lot, but not
 everybody fills in the survey, so we still miss a lot of information.

 That's why I've written the Usage Reporting functionality for the
 management server which automatically sends back anonymous information
 about a CloudStack deployment.

 It's currently in the 'reporter' branch. [0]

 By default, every 7 days it generates a JSON document with:
 - Hosts (Number, version, type, hypervisor)
 - Clusters (Hypervisor en Management type)
 - Primary storage (Type and provider)
 - Zones (Network type and providers)
 - Instances (Number and types)

 This report is not complete yet, I'd like to add more information, but
 that will be Management Server information.

 The code on how this report is generated is obviously 100% Open Source,
 so end-users can always exactly see how the information was compiled.

 I want to discuss this new feature for CloudStack and the possible
 implications it might have.

 I'm opting for a opt-out. So every new or upgraded install to 4.6.0
 (master) will have this enabled. Yes, we have to be very explicit in the
 Release Notes that this has been added.

 Why? It's the small price we as a project ask for using CloudStack. We
 want a little bit of information on how CloudStack is being used so that
 we can use this to make CloudStack even better.

 Turning it off is also just one global setting and it will never turn on
 again.

 On the server-side there is a Python flask application [1] (found in the
 reporter directory) which stores all the incoming information in a
 ElasticSearch database. From there analytics can be gathered on
 CloudStack deployments.

 It currently points to http://cs-report.widodh.nl/report which will NOT
 be the endpoint when this is merged into master.

 For 'production' I want to have
 https://report.cloudstack.apache.org/report where all reports are submitted.

 For every setup a unique ID is determined by hashing the first row in
 the 'version' table. This is the version + timestamp and that is hashed
 using SHA256. Using this unique ID we can track changes in deployments
 and see how they grow or shrink.

 Technically this wasn't that hard to implement, but the politics
 surrounding it might be the hardest part.

 What do other have to say about this? Should there be a VOTE for this
 feature to come into CloudStack? Opt-in, opt-out?

 Wido

 [0]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/reporter
 [1]:
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=reporter/usage-report-collector.py;h=500a4d284b5172fd93acea08f5460cfff5520855;hb=reporter




-- 
Daan


[GitHub] cloudstack pull request: CLOUDSTACK-8666: Put host in Alert state ...

2015-07-27 Thread koushik-das
Github user koushik-das commented on the pull request:

https://github.com/apache/cloudstack/pull/621#issuecomment-125152445
  
@DaanHoogland Writing an unit test for changes in AgentManagerImpl.java is 
hard due to the way the method gets invoked. Let me know if you have any 
suggestion.
Fot the HighAvailabilityManagerImpl.java changes I have written a unit 
test, will send out the PR for that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: CLOUDSTACK-8659: Verify presentation of v...

2015-07-27 Thread wido
Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/613#issuecomment-125133196
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: initial dockerization commit.

2015-07-27 Thread wido
Github user wido commented on the pull request:

https://github.com/apache/cloudstack/pull/623#issuecomment-125132961
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] cloudstack pull request: Dereference NULL return value

2015-07-27 Thread kishankavala
Github user kishankavala commented on the pull request:

https://github.com/apache/cloudstack/pull/618#issuecomment-125136388
  
LGTM. @bhaisaab can you also take a look at the changes?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---