Re: RBD snapshot format
Hi Wido, I didn't see the format information for snapshots stored in the database, so I'm assuming it references the associated volume format. I assume the cleanest way to get around that would be to have a separate format field for snapshots, but that wouldn't really make sense just for this one use case. Another way would be to just have the restore (create template from snapshot, etc.) function always assume QCOW2 for RBD snapshots, but that's dirty, and would cause problems for existing snapshots. It seems like the best method would be to have the restore function look at the actual file to get the format information instead of pulling it from the database, though I'm not sure if that's even possible. Does anyone else have any thoughts or suggestions? Thank You, Logan Barfield Tranquil Hosting On Thu, Jan 29, 2015 at 4:21 PM, Wido den Hollander w...@widodh.nl wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/29/2015 10:05 PM, Logan Barfield wrote: Hi Wido, The relevant code for this is in: I saw that the destination format is set to QCOW2 for createTemplateFromVolume, but in backupSnapshot it's set to use the format of the source image destFile.setFormat(snapshotDisk.getFormat()); I believe just changing the backupSnapshot code to the use QCOW2 destFile.setFormat(PhysicalDiskFormat.QCOW2); will work fine. I can submit a patch for that, but I wanted to make sure there wasn't some other reason for the RAW output first. No, that won't work. Because the management server will then still think it's in RAW format and store that in the database. It has been a long time since I looked at this. I know I did some work in this area and made some patches. Wido Thank You, Logan Barfield Tranquil Hosting On Thu, Jan 29, 2015 at 3:52 PM, Wido den Hollander w...@widodh.nl wrote: On 01/29/2015 05:16 PM, Logan Barfield wrote: I was doing some testing earlier this week on a KVM cluster, and noticed that when using S3 for secondary storage snapshots on RBD primary storage take a lot longer than they do with NFS primary storage. I think this is due to the NFS snapshots being output/uploaded in QCOW2 format, while RBD snapshots are output in RAW format. It appears that even when using sparse RAW files they are uploaded to S3 with the 'allocated' size instead of the sparse size. The problem here lies in the code in CloudStack. Since the source volume is in RAW format, the snapshot also becomes QCOW2. I remember that I wrote some patches for this, you might want to look in the Git history. My question is: Is there a good reason for RBD snapshots to be output as RAW instead of QCOW2? It appears that QCOW2 templates are automatically converted when deploying them as RBD, so that shouldn't be a problem. The only downside I can think of would be having to convert the RAW RBD data to QCOW2 as an extra step when creating snapshots, but even that seems like it would take less time than uploading the RAW images to S3. Thoughts? Like I said, I think I wrote patches for this. Qemu-img is used to convert the RAW RBD into the snapshot, so QCOW2 should be no problem as output format. But the code in CloudStack seems to assume that the snapshot volume is always the same format as the source. Wido P.S.: I'm on vacation in Norway at the moment, so I'm pretty laggy with replying. Thank You, Logan Barfield Tranquil Hosting -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJUyqRBAAoJEAGbWC3bPspCam8QALupIaRKc09pDT4Fk8bA0Bqs TS97Mw+VIE8GMszLOt8sKadGerC/8/qLJ5j1xQkfPQWijim64JaebGmTEShQHNWu SYvi/sHkqMLlgzrdsB7jW5rcFNCeQ6ZoncSLvZ9shepte5jaOJcjHQJs7cJmJa7F 6f2W4C9e7iBXLpddmM6A7jh1PCRo2sv+5hLR5Vxf/w93+3BZF9N8MTHyQfFIElpH PP2hgnem7uOsV/4PzuvJ8Sy2U1/0SFLD9JOWop9aS7BcuqxmB2iB7YJlS/NGTG9V pT+23edjDLAGAjUBdtol2VfqcXgdiXf3NTQuBS8XIaRDaTP5O1t96X24Ux7yo/1T ptpcqmZCMIj4NTfv7SMln6nkCBZ4iX0kw36Sv3X1vKMAqyPnA57r6h65Y7h9IYK2 ihaDAjEByoLruvVMzr0wWR3kEmE5EWzkV5Za7Jy76QcQkJh6XDgXFkkc5RmO7/pE ufyX7iQtukKbsRlYiFkOZzhVbTGTRzm8gNc1pge6cJmtGK0A6Jf4IZPHbAkE4Lyq wcNR/OH2feYeqoKe9pFh06GC99kkl08uLRTuucnXia7Hb0GOX3u4uEvUEv77lPbh xQiokSmxMtHqqBmMtSfcApfy4ZDz1ji+VAPixyvUI8bVpQBMieKEZRBgwXzaz+3P 8CoCyy5uX1HjwbRbMcMh =2UKe -END PGP SIGNATURE-
Re: [DISCUSS] we need a better SSVM solution
Paul, For your #1, there is this page [1] that might be a bit outdated but the system-vm section is fairly complete. but effectively things could be better address. I know there is currently an initiative on changing how configuration are sent to VR and system-vms. theres was some discussions at CCCEU14 on this. I think it would be interesting to have a design spec in the wiki about it. Also, the way the sysvm is currently provision kind of make sense, look like it's a basic debian templates, cloudstack push a system.iso into at when VR, SSVM, CPVM are created, the iso contain all latest scripts from the cloudstack-management server. the system iso can also be modified if required (ex to install xentool in VR). it's far from being perfect, but might worth digging into the current deployment method? I would add to the current discussion this : Our current method of build system-vm templates used by the community is becoming outdated, the way we generate generic system-vm template is due for an update, as example we use veewee which might worth replacing this by vagrant... also, we should be able to embed xentools or vmware tools inside the templates... I haven't found much documentation on how to customize system vm templates (ex: send cloud.log into a syslog server) but I think it's something we can do with the current system. Still, I'm +1 [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Compatibility+Matrices On Fri, Jan 30, 2015 at 1:58 AM, linux...@gmail.com linux...@gmail.com wrote: I thinks John's idea is very good. we should fource on the security ,not others. Another possible use case: * User installs new ACS system * User uploads SSVM template that has CM agent configured to talk to their CM server (I’ve been wanting to lab this for a while now) * As ACS creates system VMs, they phone home to CM server, it provides them with instructions to install various packages and config as needed to be domr/console proxy/whatever. We provide basic “recipes” for CM systems for people to use and grow from. * Security issue is announced. User updates recipe in CM system, a few minutes later the SSVMs are up-to-date. 2015-01-30 7:47 GMT+08:00 Adrian Lewis adr...@alsiconsulting.co.uk: From a non-dev user's perspective I think Paul's pretty much nailed the key issues I'd like to see improve with the system VMs. The big one for us is the ability to customise the VR template to add things like netflow export and other value-add services through additional software packages without having to do this individually on each VR deployed. -Original Message- From: Ahmad Emneina [mailto:aemne...@gmail.com] Sent: 29 January 2015 22:17 To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] we need a better SSVM solution Pauls suggestion reminds me of some awesome functionality I see in the aftermarket android ROM community. That is 'Kitchens'[1]. A utility/site that provides functionality that allows for admins to create customized system templates... Giving choices of: - OS - kernel - VPN server - various other services... Of course this is fantasy at the moment, I see the lowest barrier to entry would be a cloud-init style utility where we can pass in commands or scripts, like the steps to mitigate the GHOST vuln (which seems to be a few apt commands). That would easily resolve issues where a vulnerable service could easily be updated post boot, and propagated to all new/restarted system vm's. [1] http://forum.xda-developers.com/showthread.php?t=633246 On Thu, Jan 29, 2015 at 1:55 PM, John Kinsella j...@stratosec.co wrote: Decent points. You think the difference between the VR/CP is different enough to have a second image? On Jan 29, 2015, at 1:41 PM, Paul Angus paul.an...@shapeblue.com wrote: Hi All, I think that there are 3 things people would like to see: 1. clear versioning of system vm templates, with some kind of compatibility matrix so they know which one(s) they can use with different versions of CloudStack 2. an easy way to update the system vm template 3. an easy(ish) way to customise system vm templates It might be worth considering have two types of template a. the console proxy and secondary storage template b. the virtual router/ VPC template. Regards Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus paul.an...@shapeblue.com -Original Message- From: John Kinsella [mailto:j...@stratosec.co] Sent: 29 January 2015 18:06 To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] we need a better SSVM solution Interesting… Concur on having an open/standardized protocol. Something clustered like Serf/Consul could be attractive, but the overhead/requirements of those type of things usually scares me away. Having ACS act as a CA would be quite
RE: [DISCUSS] we need a better SSVM solution
...also. I'm not a fan of us different configuration processes for different hypervisors. I'm fairly sure that where the hypervisor still uses the systemvm.iso it overwrites the scripts which are already on the system vm, whereas with xenserver we only inject config in through pvgrub. I think that the systemvm.iso should only contain the same data as is injected via xenserver for consistency, or xenserver system vms should go back to using systemvm.iso There are a load of global settings which allow you to specify the system vm template for each hypervisor, I guess this effectively allows you to have different router templates to the ssvm/consoleproxy template. router.template.vmwareName of the default router template on Vmware router.template.xenName of the default router template on Xenserver I'm not sure why it exists, you can't change the ssvm or cpvm template per hypervisor. Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com -Original Message- From: Adrian Lewis [mailto:adr...@alsiconsulting.co.uk] Sent: 29 January 2015 23:47 To: dev@cloudstack.apache.org Subject: RE: [DISCUSS] we need a better SSVM solution From a non-dev user's perspective I think Paul's pretty much nailed the key issues I'd like to see improve with the system VMs. The big one for us is the ability to customise the VR template to add things like netflow export and other value-add services through additional software packages without having to do this individually on each VR deployed. -Original Message- From: Ahmad Emneina [mailto:aemne...@gmail.com] Sent: 29 January 2015 22:17 To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] we need a better SSVM solution Pauls suggestion reminds me of some awesome functionality I see in the aftermarket android ROM community. That is 'Kitchens'[1]. A utility/site that provides functionality that allows for admins to create customized system templates... Giving choices of: - OS - kernel - VPN server - various other services... Of course this is fantasy at the moment, I see the lowest barrier to entry would be a cloud-init style utility where we can pass in commands or scripts, like the steps to mitigate the GHOST vuln (which seems to be a few apt commands). That would easily resolve issues where a vulnerable service could easily be updated post boot, and propagated to all new/restarted system vm's. [1] http://forum.xda-developers.com/showthread.php?t=633246 On Thu, Jan 29, 2015 at 1:55 PM, John Kinsella j...@stratosec.co wrote: Decent points. You think the difference between the VR/CP is different enough to have a second image? On Jan 29, 2015, at 1:41 PM, Paul Angus paul.an...@shapeblue.com wrote: Hi All, I think that there are 3 things people would like to see: 1. clear versioning of system vm templates, with some kind of compatibility matrix so they know which one(s) they can use with different versions of CloudStack 2. an easy way to update the system vm template 3. an easy(ish) way to customise system vm templates It might be worth considering have two types of template a. the console proxy and secondary storage template b. the virtual router/ VPC template. Regards Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus paul.an...@shapeblue.com -Original Message- From: John Kinsella [mailto:j...@stratosec.co] Sent: 29 January 2015 18:06 To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] we need a better SSVM solution Interesting… Concur on having an open/standardized protocol. Something clustered like Serf/Consul could be attractive, but the overhead/requirements of those type of things usually scares me away. Having ACS act as a CA would be quite interesting for some things. It’s one of the reasons I’ve pondered a “hook” in the past to notify 3rd party upon VM creation/deletion/etc. Wonder if we could take advantage of dogtag or similar. All that said - setup/management of a CA is a PIA and probably outside scope of ACS, unless you did a “light” one similar to Puppet by default... An aside on that “hook” idea - something scriptable similar to (I said “similar to, no flames!) systemd for this could be interesting. A good portion of users would resist having an agent installed on the user VM, but I guess we’re in that position already, and they just wouldn’t get the added functionality. One user experience point: Almost every time Parallels comes out with a new version, I have to update their agent on my VMs, which on the Windows side means a reboot. That gets old, and I’ve only got a handful of win VMs there... Going to see if I can puppet-ize one of the SSVMs over the weekend to see what other thoughts come up. John On Jan 29, 2015, at 2:06 AM, Rohit Yadav rohit.ya...@shapeblue.com wrote: Good ideas John.
Re: eclipse build problems
Now that I think about it, I don't know if I ever bothered to get this working from Eclipse (starting the CloudStack Management Server from Eclipse, that is). I typically build and run it from a shell and then remotely connect to it from Eclipse for debugging purposes. On Fri, Jan 30, 2015 at 12:40 PM, Roger Crerie roger.cre...@hds.com wrote: Thanks for the reply Mike. I see though I am now running into the same problem you did when trying to start the client from eclipse. I have already edited my eclipse.ini file thusly. -startup plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar --launcher.library plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326 -product org.eclipse.epp.package.jee.product --launcher.defaultAction openFile --launcher.XXMaxPermSize 256M -showsplash org.eclipse.platform --launcher.XXMaxPermSize 2048m --launcher.defaultAction openFile --launcher.appendVmargs -vmargs -Dosgi.requiredJavaVersion=1.6 -Xms1024m -Xmx2048m And based on one of your emails I found on the web I also set the Environment variable MAVEN_OPTS to -XX:MaxPermSize=256m -Xmx1g I am still seeing OutOfMemoryErrors NFO [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name = UserDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap = 5000 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 300 timeToIdleSeconds = 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ] 2015-01-30 14:32:30.297:WARN:oejw.WebAppContext:main: Failed startup of context o.e.j.m.p.JettyWebAppContext@14efeb2b {/client,file:/C:/workspaces/Cloudstack/WS1/cloudstack/client/target/generated-webapp/,STARTING}{file:/C:/workspaces/Cloudstack/WS1/cloudstack/client/target/generated-webapp/} org.springframework.context.ApplicationContextException: Failed to start bean 'cloudStackLifeCycle'; nested exception is java.lang.OutOfMemoryError: PermGen space at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:170) at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51) at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339) at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143) at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108) at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:947) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70) at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:57) at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:61) at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:798) at
Re: [VOTE] Apache CloudStack 4.5.0 RC1
Hi everyone, Any news on this? I am still having trouble creating a VM on local storage on KVM (with or without kvmclock.disable=true in agent.properties). I'm on Ubuntu 12.04.1. Thanks! Mike On Sat, Jan 24, 2015 at 2:19 AM, Wilder Rodrigues wrodrig...@schubergphilis.com wrote: Okay, thanks for the clarification. I will test it over the weekend. Cheers, Wilder On 22 Jan 2015, at 15:20, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Wilder, If you’re testing please use latest 4.5 branch which should become the next RC. At the moment the latest is SHA d08369ad06b6d5ef801f79493c2aa4bdaeab1b83. Thanks. On 22-Jan-2015, at 6:29 pm, Wilder Rodrigues wrodrig...@schubergphilis.com wrote: Hi Rohit, Tests were based on the commit id gave put in the RC1: 8db3cbd4ff62b17a8b496026b68cf60ee0c76740 Please let me know if there is a new commit ID to be used and I will test it. Apologies for the misunderstanding, I have been a bit away from the list. :) Cheers, Wilder On 22 Jan 2015, at 12:29, Rohit Yadav rohit.ya...@shapeblue.com wrote: Hi Wilder, Thanks for sharing, but I’m confused if these tests were against latest 4.5 branch or the RC1? Looking forward to your tests (on latest 4.5). Thanks. Regards. On 22-Jan-2015, at 4:24 pm, Wilder Rodrigues wrodrig...@schubergphilis.com wrote: Hi all, Sorry for the delay on the tests and also coming to it only after the RC has been cancelled, but I was way too busy with the redundant VPC work. Remarks: * I will test Basic Zone later today * For those willing to read, I added the whole events log after the tests results. :) There go my results: :: Environment :: Host: XenServer 6.2 - running under a VMware zone within our ACS Beta-Cloud infra Management Server: MacBook Pro Database: MySQL running locally on my MacBook Pro :: Advanced Zone :: === Test Create Account and user for that account ... === TestName: test_01_create_account | Status : SUCCESS === ok Test Sub domain allowed to launch VM when a Domain level zone is created ... === TestName: test_01_add_vm_to_subdomain | Status : SUCCESS === ok Test delete domain without force option ... === TestName: test_DeleteDomain | Status : SUCCESS === ok Test delete domain with force option ... === TestName: test_forceDeleteDomain | Status : SUCCESS === ok Test update admin details ... === TestName: test_updateAdminDetails | Status : SUCCESS === ok Test update domain admin details ... === TestName: test_updateDomainAdminDetails | Status : SUCCESS === ok Test user update API ... === TestName: test_updateUserDetails | Status : SUCCESS === ok Test login API with domain ... === TestName: test_LoginApiDomain | Status : SUCCESS === ok Test if Login API does not return UUID's ... === TestName: test_LoginApiUuidResponse | Status : SUCCESS === ok -- Ran 9 tests in 1094.799s OK Test reset virtual machine on reboot ... === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS === ok -- Ran 1 test in 217.296s OK Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS === ok Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS === ok Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS === ok Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS === ok Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS === ok Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS === ok Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS === ok Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS === ok Test migrate VM ... SKIP: At least two hosts should be present in the zone for migration Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS === ok -- Ran 10 tests in 816.707s OK (SKIP=1) test_privategw_acl (integration.acs.tests.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_privategw_acl | Status : SUCCESS === ok -- Ran 1 test in 85.883s OK Test to create service offering ... === TestName: test_01_create_service_offering | Status : SUCCESS === ok Test to update existing service offering ... === TestName: test_02_edit_service_offering | Status : SUCCESS === ok Test to delete service offering ... === TestName:
Re: Any Plan of Distributed VR Within cloudstack?
Hi Yitao, We, at Schuberg Philis, are currently working on the implementation of Redundant VPCs. We are now on the test phase and the first open demo of it will take place in London, 12th February, during the Cloudstack European User Group meeting (http://shapeblue.com/events/cloudstack-european-user-group-3/) Once the work is pushed towards master we will put some documentation on the Apache CS wiki and ando get the community involved with the development. After the 12th Feb I will send more details about the status and what we are using. Cheers, Wilder On 30 Jan 2015, at 01:16, Yitao Jiang willier...@gmail.commailto:willier...@gmail.com wrote: Hi,guys I just want to know is there any plan to develop distributed vr feature, and What's the progress now? --- Thanks, Yitao(依涛 姜) jiangyt.github.io
Re: Review Request 28030: CLOUDSTACK-7911: Automation test cases for Usage test path
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28030/ --- (Updated Jan. 30, 2015, 4:29 p.m.) Review request for cloudstack and SrikanteswaraRao Talluri. Bugs: CLOUDSTACK-7911 https://issues.apache.org/jira/browse/CLOUDSTACK-7911 Repository: cloudstack-git Description (updated) --- Automation test cases for Usage test path. More test cases to follow. This is first patch. Diffs (updated) - test/integration/testpaths/testpath_usage.py PRE-CREATION tools/marvin/marvin/config/test_data.py d5ed353 tools/marvin/marvin/dbConnection.py 66c6cb1 tools/marvin/marvin/lib/base.py e38c394 tools/marvin/marvin/lib/utils.py 8788b3b Diff: https://reviews.apache.org/r/28030/diff/ Testing (updated) --- Yes. Thanks, Ashutosh Kelkar
RE: eclipse build problems
Thanks for the reply Mike. I see though I am now running into the same problem you did when trying to start the client from eclipse. I have already edited my eclipse.ini file thusly. -startup plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar --launcher.library plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326 -product org.eclipse.epp.package.jee.product --launcher.defaultAction openFile --launcher.XXMaxPermSize 256M -showsplash org.eclipse.platform --launcher.XXMaxPermSize 2048m --launcher.defaultAction openFile --launcher.appendVmargs -vmargs -Dosgi.requiredJavaVersion=1.6 -Xms1024m -Xmx2048m And based on one of your emails I found on the web I also set the Environment variable MAVEN_OPTS to -XX:MaxPermSize=256m -Xmx1g I am still seeing OutOfMemoryErrors NFO [c.c.u.d.GenericDaoBase] (main:null) Cache created: [ name = UserDaoImpl status = STATUS_ALIVE eternal = false overflowToDisk = false maxEntriesLocalHeap = 5000 maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 300 timeToIdleSeconds = 300 persistence = none diskExpiryThreadIntervalSeconds = 120 cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ] 2015-01-30 14:32:30.297:WARN:oejw.WebAppContext:main: Failed startup of context o.e.j.m.p.JettyWebAppContext@14efeb2b{/client,file:/C:/workspaces/Cloudstack/WS1/cloudstack/client/target/generated-webapp/,STARTING}{file:/C:/workspaces/Cloudstack/WS1/cloudstack/client/target/generated-webapp/} org.springframework.context.ApplicationContextException: Failed to start bean 'cloudStackLifeCycle'; nested exception is java.lang.OutOfMemoryError: PermGen space at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:170) at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51) at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339) at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143) at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108) at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:947) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70) at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:57) at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:61) at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:798) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444) at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:789) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1345) ... Has anyone
Re: Database locking code
Hi, That looks like it will work for my purposes. When I have the time to continue the task I was working on yesterday I will try this way. Jeff On Thu, Jan 29, 2015 at 2:34 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Like this: GlobalLock lock = GlobalLock.getInternLock( someStringYouMadeUpThatIsUniqueForThisPurpose); if (!lock.lock(timeoutInSeconds)) { s_logger.debug(Couldn't lock the db on the string + someStringYouMadeUpThatIsUniqueForThisPurpose); throw new CloudRuntimeException(Couldn't acquire DB lock); } try { // do your work } finally { lock.unlock(); lock.releaseRef(); } On Thu, Jan 29, 2015 at 7:32 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: I wrote that example code from memory, but now that I think about it, you wouldn't call the lock method in the try block. Call lock and if you're successful, entry a try block to do your work and have a finally to unlock and release the lock. On Thu, Jan 29, 2015 at 7:27 AM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: Does this work for you? GlobalLock lock = GlobalLock.getInternLock( someStringYouMadeUpThatIsUniqueForThisPurpose); try { if (!lock.lock(timeoutInSeconds)) { s_logger.debug(Couldn't lock the db on the string + someStringYouMadeUpThatIsUniqueForThisPurpose); throw new CloudRuntimeException(Couldn't acquire DB lock); } // do your work } finally { lock.unlock(); lock.releaseRef(); } On Thu, Jan 29, 2015 at 3:32 AM, Jeff Hair j...@greenqloud.com wrote: I have some code I've added to CloudStack that I am currently synchronizing on in the traditional manner (synchronized block, ExecutorService, etc). I'm currently running a single instance of CloudStack so I don't have to deal with clustering. I've read the Data Access Layer documentation and am wondering if the database locks are the actual appropriate solution rather than what I'm doing. Basically my issue is that I need to lock access to an IP address reserved to the account and perform some long operations on it. I also need to guarantee that no other IPs on the account are manipulated while performing this task. Synchronization accomplishes this because I can force the IP operations through an ExecutorService, but I'm really looking for a better solution that will also work with clustered management servers. Thanks, Jeff -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™* -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™* -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud http://solidfire.com/solution/overview/?video=play*™* -- Jeff Hair Core Systems Developer Tel: (+354) 415 0200 j...@greenqloud.com www.greenqloud.com
Build failed in Jenkins: build-master #2205
See http://jenkins.buildacloud.org/job/build-master/2205/changes Changes: [talluri] CLOUDSTACK-8161: added skips for the tests which are not supported on LXC: -- [...truncated 1415 lines...] Running com.cloud.network.ExternalLoadBalancerDeviceManagerImplTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.613 sec - in com.cloud.network.ExternalLoadBalancerDeviceManagerImplTest Running com.cloud.network.element.VirtualRouterElementTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.005 sec - in com.cloud.network.element.VirtualRouterElementTest Running com.cloud.network.UpdatePhysicalNetworkTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.207 sec - in com.cloud.network.UpdatePhysicalNetworkTest Running com.cloud.network.CreatePrivateNetworkTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec - in com.cloud.network.CreatePrivateNetworkTest Running com.cloud.network.NetworkModelTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec - in com.cloud.network.NetworkModelTest Running com.cloud.network.router.NetworkHelperImplTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 sec - in com.cloud.network.router.NetworkHelperImplTest Running com.cloud.network.router.RouterControlHelperTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec - in com.cloud.network.router.RouterControlHelperTest Running com.cloud.network.router.VirtualNetworkApplianceManagerImplTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.206 sec - in com.cloud.network.router.VirtualNetworkApplianceManagerImplTest Running com.cloud.network.security.SecurityGroupManagerImplTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.944 sec - in com.cloud.network.security.SecurityGroupManagerImplTest Running com.cloud.network.security.SecurityGroupQueueTest Total jobs dequeued = 10, num queued=1002 queue current size=992 Num Vms= 50 Queue size = 50 Num Vms= 2 Queue size = 2 time=713 ms Total jobs dequeued = 10, num queued=0 queue current size=4999 Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 1.696 sec FAILURE! - in com.cloud.network.security.SecurityGroupQueueTest testNumJobsEqToNumVms3(com.cloud.network.security.SecurityGroupQueueTest) Time elapsed: 1.428 sec ERROR! java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:693) at com.cloud.network.security.SecurityGroupQueueTest.testNumJobsEqToNumVms2(SecurityGroupQueueTest.java:124) at com.cloud.network.security.SecurityGroupQueueTest.testNumJobsEqToNumVms3(SecurityGroupQueueTest.java:140) testDequeueOneJob(com.cloud.network.security.SecurityGroupQueueTest) Time elapsed: 0.037 sec FAILURE! junit.framework.AssertionFailedError: expected:-10 but was:4999 at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:234) at junit.framework.Assert.assertEquals(Assert.java:241) at junit.framework.TestCase.assertEquals(TestCase.java:409) at com.cloud.network.security.SecurityGroupQueueTest._testDequeueOneJob(SecurityGroupQueueTest.java:195) at com.cloud.network.security.SecurityGroupQueueTest.testDequeueOneJob(SecurityGroupQueueTest.java:213) Running com.cloud.network.vpc.VpcManagerImplTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in com.cloud.network.vpc.VpcManagerImplTest Running com.cloud.network.DedicateGuestVlanRangesTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec - in com.cloud.network.DedicateGuestVlanRangesTest Running com.cloud.network.lb.AssignLoadBalancerTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec - in com.cloud.network.lb.AssignLoadBalancerTest Running com.cloud.network.lb.UpdateLoadBalancerTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.065 sec - in com.cloud.network.lb.UpdateLoadBalancerTest Running com.cloud.network.dao.NetworkDaoTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in com.cloud.network.dao.NetworkDaoTest Running com.cloud.vm.FirstFitPlannerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.481 sec - in com.cloud.vm.FirstFitPlannerTest Running com.cloud.vm.DeploymentPlanningManagerImplTest Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.609 sec - in com.cloud.vm.DeploymentPlanningManagerImplTest Running com.cloud.vm.snapshot.VMSnapshotManagerTest Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.134 sec - in com.cloud.vm.snapshot.VMSnapshotManagerTest Running
Jenkins build is still unstable: simulator-singlerun #948
See http://jenkins.buildacloud.org/job/simulator-singlerun/changes
Jenkins build is still unstable: simulator-singlerun #949
See http://jenkins.buildacloud.org/job/simulator-singlerun/changes
Build failed in Jenkins: build-master #2206
See http://jenkins.buildacloud.org/job/build-master/2206/changes Changes: [talluri] CLOUDSTACK-8161: added skips for the tests which are not supported on LXC: -- [...truncated 1408 lines...] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.163 sec - in com.cloud.servlet.StaticResourceServletTest Running com.cloud.servlet.ConsoleProxyServletTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.cloud.servlet.ConsoleProxyServletTest Running com.cloud.resourcelimit.ResourceLimitManagerImplTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.cloud.resourcelimit.ResourceLimitManagerImplTest Running com.cloud.network.firewall.FirewallManagerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.001 sec - in com.cloud.network.firewall.FirewallManagerTest Running com.cloud.network.ExternalLoadBalancerDeviceManagerImplTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.587 sec - in com.cloud.network.ExternalLoadBalancerDeviceManagerImplTest Running com.cloud.network.element.VirtualRouterElementTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.004 sec - in com.cloud.network.element.VirtualRouterElementTest Running com.cloud.network.UpdatePhysicalNetworkTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.174 sec - in com.cloud.network.UpdatePhysicalNetworkTest Running com.cloud.network.CreatePrivateNetworkTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec - in com.cloud.network.CreatePrivateNetworkTest Running com.cloud.network.NetworkModelTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.03 sec - in com.cloud.network.NetworkModelTest Running com.cloud.network.router.NetworkHelperImplTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec - in com.cloud.network.router.NetworkHelperImplTest Running com.cloud.network.router.RouterControlHelperTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec - in com.cloud.network.router.RouterControlHelperTest Running com.cloud.network.router.VirtualNetworkApplianceManagerImplTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.271 sec - in com.cloud.network.router.VirtualNetworkApplianceManagerImplTest Running com.cloud.network.security.SecurityGroupManagerImplTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.088 sec - in com.cloud.network.security.SecurityGroupManagerImplTest Running com.cloud.network.security.SecurityGroupQueueTest Total jobs dequeued = 10, num queued=1002 queue current size=992 Num Vms= 50 Queue size = 50 Num Vms= 2 Queue size = 2 time=1045 ms Num Vms= 5000 Queue size = 5000 time=611 ms Num Vms= 1 Queue size = 1 time=1 ms Num Vms= 100 Queue size = 100 time=1032 ms Total jobs dequeued = 10, num queued=1010 queue current size=1000 Total jobs dequeued = 1, num queued=1001 queue current size=1000 Total jobs dequeued = 10, num queued=1000 queue current size=990 Total jobs dequeued = 10, num queued=10 queue current size=0 Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.125 sec FAILURE! - in com.cloud.network.security.SecurityGroupQueueTest testNumJobsEqToNumVms3(com.cloud.network.security.SecurityGroupQueueTest) Time elapsed: 2.914 sec ERROR! java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:693) at com.cloud.network.security.SecurityGroupQueueTest.testNumJobsEqToNumVms2(SecurityGroupQueueTest.java:124) at com.cloud.network.security.SecurityGroupQueueTest.testNumJobsEqToNumVms3(SecurityGroupQueueTest.java:143) Running com.cloud.network.vpc.VpcManagerImplTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in com.cloud.network.vpc.VpcManagerImplTest Running com.cloud.network.DedicateGuestVlanRangesTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec - in com.cloud.network.DedicateGuestVlanRangesTest Running com.cloud.network.lb.AssignLoadBalancerTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.069 sec - in com.cloud.network.lb.AssignLoadBalancerTest Running com.cloud.network.lb.UpdateLoadBalancerTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.077 sec - in com.cloud.network.lb.UpdateLoadBalancerTest Running com.cloud.network.dao.NetworkDaoTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.cloud.network.dao.NetworkDaoTest Running com.cloud.vm.FirstFitPlannerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.407 sec - in com.cloud.vm.FirstFitPlannerTest Running com.cloud.vm.DeploymentPlanningManagerImplTest Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.449 sec - in
Re: Application programming on CS
Is there something like the following for cloudstack application programming.I am interested in creating applications like banking, insurance etc running on top of cloudstack vm's. https://netbeans.org/kb/docs/web/oracle-cloud.html http://www.javacodegeeks.com/2012/01/openshift-express-deploy-java-ee.html# From: Giri Prasad g_p...@yahoo.com.INVALID To: Pierre-Luc Dion pd...@cloudops.com; dev@cloudstack.apache.org dev@cloudstack.apache.org Cc: us...@cloudstack.apache.org us...@cloudstack.apache.org; Mike Tutkowski mike.tutkow...@solidfire.com Sent: Wednesday, 31 December 2014 10:40 AM Subject: Re: Application programming on CS Thanks everybody for all your valuable comments. I am currently surfing around looking for the information. Will keep posted, when I find something useful. Giri From: Pierre-Luc Dion pd...@cloudops.com To: dev@cloudstack.apache.org dev@cloudstack.apache.org Cc: us...@cloudstack.apache.org us...@cloudstack.apache.org; Giri Prasad g_p...@yahoo.com; Mike Tutkowski mike.tutkow...@solidfire.com Sent: Monday, 29 December 2014 11:22 PM Subject: Re: Application programming on CS sorry, sent too fast... This one [1] contain a lot of examples on leveraging CloudStack with various tools. O'Reilly: 60 Recipes for Apache CloudStack [1] http://shop.oreilly.com/product/0636920034377.do PL On Mon, Dec 29, 2014 at 12:48 PM, Pierre-Luc Dion pd...@cloudops.com wrote: This one [1] contain a lot of examples to liverage CloudStack On Mon, Dec 29, 2014 at 12:20 PM, Sebastien Goasguen run...@gmail.com wrote: On Dec 29, 2014, at 12:15 AM, Giri Prasad g_p...@yahoo.com.INVALID wrote: Can someone point for java and php application development in cloud, as in: http://www.microsoftvirtualacademy.com/ebooks#9780735695658 I downloaded this pdf to understand what you are looking for. As far as I can see, this pdf has nothing to do with programming, it's a collection of narrative around cloud concepts: -automation -monitoring -CI/CD -single sign-on etc All these apply to a CloudStack cloud and can be implemented with all sorts of tools. My suggestion for you is to read The Phoenix Project Building Cloud Apps with Microsoft Azure Scott Guthrie, Mark Simms, Tom Dykstra, Rick Anderson, and Mike Wasson July 2014 198 pages Introducing Windows Azure for IT Professionals Mitch Tulloch with the Windows Azure Team October 2013 142 pages - Original Message - From: Giri Prasad g_p...@yahoo.com.INVALID To: Mike Tutkowski mike.tutkow...@solidfire.com; dev@cloudstack.apache.org dev@cloudstack.apache.org Cc: us...@cloudstack.apache.org us...@cloudstack.apache.org Sent: Saturday, 27 December 2014 3:44 PM Subject: Re: Application programming on CS Thanks for the information. I want to write application programs in java and php for the cloud architechture. Applications like banking, insurance etc running on top of cloudstack vm's. The java/php should use the features of the cloud not available for a web app, like high availability etc etc. Any reference on how to do the application design and implementation, for the cloud (private/public), with some practical examples? From: Mike Tutkowski mike.tutkow...@solidfire.com To: dev@cloudstack.apache.org dev@cloudstack.apache.org; Giri Prasad g_p...@yahoo.com Cc: us...@cloudstack.apache.org us...@cloudstack.apache.org Sent: Saturday, 27 December 2014 12:47 PM Subject: Re: Application programming on CS What about using something like jclouds? https://github.com/jclouds/jclouds/blob/master/apis/cloudstack/src/test/java/org/jclouds/cloudstack/compute/CloudStackExperimentLiveTest.java On Sat, Dec 27, 2014 at 12:06 AM, Giri Prasad g_p...@yahoo.com.invalid wrote: Hi All, Could somebody inform about books/reading material on how to write application programs, using the cloudstack infra. I am planning to do java and php applications using cs infra, on linux vm's. Then later on, to do some .net programming on windows vm's. Any reference on how to do the application design and implementation, for the cloud (private/public), with some practical examples? Thanks Regards,D.Giriprasad -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc.e: mike.tutkow...@solidfire.como: 303.746.7302Advancing the way the world uses the cloud™