+1 I would love to see a screencast of how this works, once it's put in its own branch (and you back from paternity leave…congratulations :) )
On Sep 25, 2013, at 5:25 PM, Chiradeep Vittal <chiradeep.vit...@citrix.com> wrote: > +1 to put this into a branch off of master. Can merge into master after > unit tests for the agent API. > > > On 9/24/13 12:37 PM, "Donal Lafferty" <donal.laffe...@citrix.com> wrote: > >> On paternity leave, so I don't get to these emails right away... >> >>> -----Original Message----- >>> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] >>> Sent: 20 September 2013 06:40 >>> To: dev@cloudstack.apache.org >>> Subject: Re: [Merge] Minimal Hyper-V Plugin >>> >>> Thanks for the preliminary testing. >>> Questions: >>> 1. More for the community: should the C# code be in a separate repo? >>> According to the merge request, mono and maven can be used to build the >>> agent. >> >> Silence == acceptance? >> >>> 2. Packaging: how is the C# agent installed? >> >> The agent is implemented to as a self-contained Windows Service, which is >> the Microsoft Windows equivalent of a Linux daemon. >> >> To make the agent distributable, package the agent and an app.config >> consistent with your data center in an MSI. WiX is the preferred tool >> (http://en.wikipedia.org/wiki/WiX ). When executed, the MSI will add the >> agent to set of Windows Services. >> >> To distribute and run this MSI, use Active Directory's GPO (global policy >> object) service. In typical deployments machines running Hyper-V will be >> domain joined. Where machines are not domain joined, look at something >> like puppet. >> >>> 3. What does minimal mean? What works? What doesn't? Local storage? >>> Shared storage? Networking modes? Are the hypervisors supposed to be >>> clustered? >> >> Minimal = create / start / stop / destroy a local storage VM in a >> QuickCloud network offering and CIFS secondary storage. >> >> No clustering required. >> >>> 4. How does one extend the "minimal" plugin? >> >> Each CloudStack command has a corresponding an HTTP URI served by the >> agent. These are written in ASP.NET MVC4. Data received by the agent is >> kept in a JSON object graph. >> >> E.g. >> >> // POST api/HypervResource/ReadyCommand >> [HttpPost] >> [ActionName(CloudStackTypes.ReadyCommand)] >> public JContainer ReadyCommand([FromBody]dynamic cmd) >> { >> using (log4net.NDC.Push(Guid.NewGuid().ToString())) >> { >> logger.Info(CloudStackTypes.ReadyCommand + >> cmd.ToString()); >> object ansContent = new >> { >> result = true, >> details = (string)null >> }; >> return ReturnCloudStackTypedJArray(ansContent, >> CloudStackTypes.ReadyAnswer); >> } >> >> } >> >> Therefore, to extend the plugin, add new HTTP URIs corresponding to >> missing commands, or extend the capabilities of existing commands. >> >> I can follow up with an explanation in a blog entry. >> >>> 5. Can the unit tests (at least those that test the agent API) be run >>> in a non- >>> hyper-v environment? >> >> Unit tests start the agent in a local process. Provided Mono is >> installed on your system, the unit tests will run. However, they will >> complain of bad output. >> >>> 6. Is the RDP console you had earlier mentioned included in the merge? >> >> Yes, but it serves no purpose at the moment. >> >> If there is an IP clearance protocol to follow for this console, I would >> prefer to remove the console from the submission. >> >>> 7. Any known issues? >>> >> >> There seems to be a bug with local paths that include spaces. I've asked >> Rajesh to provide a bug report, but it's unclear where to put this. Can >> we use JIRA for code not merged, or should the bug appear in the comments. >> >> >>> On 9/11/13 8:00 AM, "Donal Lafferty" <donal.laffe...@citrix.com> wrote: >>> >>>> Hi Rajesh, >>>> >>>> Thanks for spotting this problem with the Hyper-V Agent. Sounds like >>>> it should first URL decode the field. >>>> >>>> Can you update the review with details of your testing? >>>> >>>> I would need to know the command and which incoming field is causing >>>> problems. Also, can you add a serialised example of the instruction >>>> that fails? There should be an example in the agent's log file. By >>>> default, the log file is in the same folder as the agent executable. I >>>> will use this to update the automated tests. >>>> >>>> If you want to go ahead and made the fixes from a git clone, send a >>>> Pull Request. As long as there is an appropriate automated test, I'll >>>> update the feature branch with your changes. >>>> >>>> >>>> DL >>>> >>>>> -----Original Message----- >>>>> From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >>>>> Sent: 11 September 2013 09:08 >>>>> To: dev@cloudstack.apache.org >>>>> Subject: RE: [Merge] Minimal Hyper-V Plugin >>>>> >>>>> Hi Donal, >>>>> I had figured out the issue why "+" is coming in the path value. >>>>> The root cause is while encoding the URI, we use >>>>> URLEncoder.encode(path) >>>>> >>>>> The encode method is converting/replace "space" with "+". >>>>> >>>>> API doc: >>>>> When encoding a String, the following rules apply: >>>>> >>>>> The alphanumeric characters "a" through "z", "A" through "Z" and "0" >>>>> through "9" remain the same. >>>>> The special characters ".", "-", "*", and "_" remain the same. >>>>> The space character " " is converted into a plus sign "+". >>>>> All other characters are unsafe and are first converted into one or >>>>> more bytes using some encoding scheme. Then each byte is represented >>>>> by the >>>>> 3- >>>>> character string "%xy", where xy is the two-digit hexadecimal >>>>> representation of the byte. The recommended encoding scheme to use is >>>>> UTF-8. However, for compatibility reasons, if an encoding is not >>>>> specified, then the default encoding of the platform is used. >>>>> >>>>> Thanks >>>>> Rajesh Battala >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Donal Lafferty [mailto:donal.laffe...@citrix.com] >>>>> Sent: Monday, September 2, 2013 3:03 PM >>>>> To: dev@cloudstack.apache.org >>>>> Subject: RE: [Merge] Minimal Hyper-V Plugin >>>>> >>>>> Hi Rajesh, >>>>> >>>>> I'll PM the scripts. >>>>> >>>>> WRT problem below, it looks like a CS persists paths with spaces as >>>>> URL encoded strings. Is it the Hyper-V code storing the path in the >>>>> wrong formant? I can fix that if it's an issue. >>>>> >>>>> I would be very reluctant to change the the format of columns in the >>>>> database. There will be other code reliant on URL encoding. >>>>> >>>>> DL >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >>>>>> Sent: 02 September 2013 08:37 >>>>>> To: dev@cloudstack.apache.org >>>>>> Subject: RE: [Merge] Minimal Hyper-V Plugin >>>>>> >>>>>> Hi Donal, >>>>>> One more issue is, currently local storage is discovered when host >>>>>> is >>>>> added. >>>>>> But when mgmt. server got restarted, there is an exception >>>>>> happening while add/discovering already existing local storage >>> pool. >>>>>> >>>>>> >>>>>> Root cause: >>>>>> ========== >>>>>> I had debugged and figured out the root cause, CS not able to find >>>>>> the storage pool and tries to created it but it fails to add to DB >>>>>> as the entry is already persisted. >>>>>> The reason why CS not able to get the existing localstorage is , >>>>>> when executing the query to get the storage pool, it's getting >>> empty set >>> . >>>>>> It's because of the storage path issue. >>>>>> In the db the local storage path is stored as >>>>>> "C:\Users\Public\Documents\Hyper-V\Virtual+Hard+Disks", ('+' >>>>>> symbol in place of space) In the db query its searching without "+" >>>>>> and with space which is causing the query to have empty set and >>>>>> hence the issue. >>>>>> >>>>>> I can fix the issue by removing the "+" when we are persisting the >>>>>> local storage pool from hyperv >>>>>> >>>>>> >>>>>> >>>>>> Exception: >>>>>> ========= >>>>>> WARN [c.c.r.ResourceManagerImpl] (AgentTaskPool-14:ctx-ddbbf089) >>>>>> Unable to connect due to >>>>>> com.cloud.exception.ConnectionException: Unable to setup the local >>>>>> storage pool for Host[-1-Routing] >>>>>> at >>>>>> >>>>> >>> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManage >>>>>> rImpl.java:598) >>>>>> at >>>>>> >>>>> >>> com.cloud.utils.component.ComponentInstantiationPostProcessor$Interce >>>>> p >>>>>> >>> torDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>>> at >>>>>> com.cloud.storage.LocalStoragePoolListener.processConnect(LocalStor >>>>>> age >>>>>> Po >>>>>> olListener.java:86) >>>>>> at >>>>>> >>>>> >>> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection( >>>>>> AgentManagerImpl.java:519) >>>>>> at >>>>>> >>>>> >>> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(A >>>>>> gentManagerImpl.java:1414) >>>>>> at >>>>>> >>> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource >>>>>> ManagerImpl.java:1760) >>>>>> at >>>>>> >>> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource >>>>>> ManagerImpl.java:1922) >>>>>> at >>>>>> >>>>> >>> com.cloud.agent.manager.AgentManagerImpl$SimulateStartTask.run(Agent >>>>>> ManagerImpl.java:1063) >>>>>> at >>>>>> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto >>>>>> r.j >>>>>> av >>>>>> a:1110) >>>>>> at >>>>>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. >>>>>> ja >>>>>> va:603) >>>>>> at java.lang.Thread.run(Thread.java:722) >>>>>> Caused by: com.cloud.utils.exception.CloudRuntimeException: >>>>>> duplicate >>>>>> uuid: 35aa91ba-c95b-3fc4-91df-9c95dc31b350-HypervResource >>>>>> at >>>>>> >>> org.apache.cloudstack.storage.volume.datastore.PrimaryDataStoreHelper. >>>>>> cr >>>>>> eatePrimaryDataStore(PrimaryDataStoreHelper.java:66) >>>>>> at >>>>>> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimary >>>>>> Dat >>>>>> aSto >>>>>> >>>>> >>>>> reLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl.jav >>>>> a:3 >>>>> 49) >>>>>> at >>>>>> >>>>> >>> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManage >>>>>> rImpl.java:589) >>>>>> ... 14 more >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Donal Lafferty [mailto:donal.laffe...@citrix.com] >>>>>> Sent: Saturday, August 31, 2013 5:19 AM >>>>>> To: dev@cloudstack.apache.org >>>>>> Subject: [Merge] Minimal Hyper-V Plugin >>>>>> >>>>>> A plugin for Hyper-V control is available for CloudStack. The >>>>>> plugin implements basic VM control; however, its architecture >>>>>> allows additional functionality to be easily added. Incorporating >>>>>> the plugin in CloudStack will allow the community to participate in >>>>>> improving the features available with Hyper-V. >>>>>> >>>>>> The plugin uses a Director Connect Agent architecture described >>> here: >>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress >>>>>> >>>>>> This links provides details of third party libraries used by the >>>>>> plugin including their licenses. No source for these libraries is >>>>>> used, and the binaries are downloaded from their distributors at >>>>> build time. >>>>>> >>>>>> No proprietary tools are required for the build. For instance, C# >>>>>> compiled with Mono has been tested. Therefore, the plugin has been >>>>>> added to the default build and the default deployment. >>>>>> >>>>>> The plugin includes unit and functional tests that can be triggered >>>>>> at build time. See the link above for details. >>>>>> >>>>>> The Apache header is applied to source and where feasible to build >>>>>> config files. >>>>>> >>>>>> The source is the >>>>>> https://github.com/lafferty/cloudstack/tree/hyperv_plugin >>>>>> >>>>>> Finally, the review request is at >>>>>> https://reviews.apache.org/r/13922/ >>>>>> >>>>>> Testing? >>>>>> >>>>>> Integration test: created zone with CIFS secondary storage, added >>>>>> Hyper-V host, registered templates, create/stop/start/destroy VM. >>>>>> Scripts for zone setup are available on request. >>>>>> >>>>>> Functional & unit tests: the Java versions are part of the build. >>>>>> The C# versions were run inside Visual Studio's Test Explorer. >> >
