+1 to put this into a branch off of master. Can merge into master after unit tests for the agent API.
On 9/24/13 12:37 PM, "Donal Lafferty" <donal.laffe...@citrix.com> wrote: >On paternity leave, so I don't get to these emails right away... > >> -----Original Message----- >> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] >> Sent: 20 September 2013 06:40 >> To: dev@cloudstack.apache.org >> Subject: Re: [Merge] Minimal Hyper-V Plugin >> >> Thanks for the preliminary testing. >> Questions: >> 1. More for the community: should the C# code be in a separate repo? >> According to the merge request, mono and maven can be used to build the >> agent. > >Silence == acceptance? > >> 2. Packaging: how is the C# agent installed? > >The agent is implemented to as a self-contained Windows Service, which is >the Microsoft Windows equivalent of a Linux daemon. > >To make the agent distributable, package the agent and an app.config >consistent with your data center in an MSI. WiX is the preferred tool >(http://en.wikipedia.org/wiki/WiX ). When executed, the MSI will add the >agent to set of Windows Services. > >To distribute and run this MSI, use Active Directory's GPO (global policy >object) service. In typical deployments machines running Hyper-V will be >domain joined. Where machines are not domain joined, look at something >like puppet. > >> 3. What does minimal mean? What works? What doesn't? Local storage? >> Shared storage? Networking modes? Are the hypervisors supposed to be >> clustered? > >Minimal = create / start / stop / destroy a local storage VM in a >QuickCloud network offering and CIFS secondary storage. > >No clustering required. > >> 4. How does one extend the "minimal" plugin? > >Each CloudStack command has a corresponding an HTTP URI served by the >agent. These are written in ASP.NET MVC4. Data received by the agent is >kept in a JSON object graph. > >E.g. > > // POST api/HypervResource/ReadyCommand > [HttpPost] > [ActionName(CloudStackTypes.ReadyCommand)] > public JContainer ReadyCommand([FromBody]dynamic cmd) > { > using (log4net.NDC.Push(Guid.NewGuid().ToString())) > { > logger.Info(CloudStackTypes.ReadyCommand + >cmd.ToString()); > object ansContent = new > { > result = true, > details = (string)null > }; > return ReturnCloudStackTypedJArray(ansContent, >CloudStackTypes.ReadyAnswer); > } > > } > >Therefore, to extend the plugin, add new HTTP URIs corresponding to >missing commands, or extend the capabilities of existing commands. > >I can follow up with an explanation in a blog entry. > >> 5. Can the unit tests (at least those that test the agent API) be run >>in a non- >> hyper-v environment? > >Unit tests start the agent in a local process. Provided Mono is >installed on your system, the unit tests will run. However, they will >complain of bad output. > >> 6. Is the RDP console you had earlier mentioned included in the merge? > >Yes, but it serves no purpose at the moment. > >If there is an IP clearance protocol to follow for this console, I would >prefer to remove the console from the submission. > >> 7. Any known issues? >> > >There seems to be a bug with local paths that include spaces. I've asked >Rajesh to provide a bug report, but it's unclear where to put this. Can >we use JIRA for code not merged, or should the bug appear in the comments. > > >> On 9/11/13 8:00 AM, "Donal Lafferty" <donal.laffe...@citrix.com> wrote: >> >> >Hi Rajesh, >> > >> >Thanks for spotting this problem with the Hyper-V Agent. Sounds like >> >it should first URL decode the field. >> > >> >Can you update the review with details of your testing? >> > >> >I would need to know the command and which incoming field is causing >> >problems. Also, can you add a serialised example of the instruction >> >that fails? There should be an example in the agent's log file. By >> >default, the log file is in the same folder as the agent executable. I >> >will use this to update the automated tests. >> > >> >If you want to go ahead and made the fixes from a git clone, send a >> >Pull Request. As long as there is an appropriate automated test, I'll >> >update the feature branch with your changes. >> > >> > >> >DL >> > >> >> -----Original Message----- >> >> From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >> >> Sent: 11 September 2013 09:08 >> >> To: dev@cloudstack.apache.org >> >> Subject: RE: [Merge] Minimal Hyper-V Plugin >> >> >> >> Hi Donal, >> >> I had figured out the issue why "+" is coming in the path value. >> >> The root cause is while encoding the URI, we use >> >> URLEncoder.encode(path) >> >> >> >> The encode method is converting/replace "space" with "+". >> >> >> >> API doc: >> >> When encoding a String, the following rules apply: >> >> >> >> The alphanumeric characters "a" through "z", "A" through "Z" and "0" >> >> through "9" remain the same. >> >> The special characters ".", "-", "*", and "_" remain the same. >> >> The space character " " is converted into a plus sign "+". >> >> All other characters are unsafe and are first converted into one or >> >>more bytes using some encoding scheme. Then each byte is represented >> >>by the >> >>3- >> >> character string "%xy", where xy is the two-digit hexadecimal >> >>representation of the byte. The recommended encoding scheme to use is >> >>UTF-8. However, for compatibility reasons, if an encoding is not >> >>specified, then the default encoding of the platform is used. >> >> >> >> Thanks >> >> Rajesh Battala >> >> >> >> >> >> >> >> >> >> -----Original Message----- >> >> From: Donal Lafferty [mailto:donal.laffe...@citrix.com] >> >> Sent: Monday, September 2, 2013 3:03 PM >> >> To: dev@cloudstack.apache.org >> >> Subject: RE: [Merge] Minimal Hyper-V Plugin >> >> >> >> Hi Rajesh, >> >> >> >> I'll PM the scripts. >> >> >> >> WRT problem below, it looks like a CS persists paths with spaces as >> >> URL encoded strings. Is it the Hyper-V code storing the path in the >> >> wrong formant? I can fix that if it's an issue. >> >> >> >> I would be very reluctant to change the the format of columns in the >> >> database. There will be other code reliant on URL encoding. >> >> >> >> DL >> >> >> >> >> >> > -----Original Message----- >> >> > From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >> >> > Sent: 02 September 2013 08:37 >> >> > To: dev@cloudstack.apache.org >> >> > Subject: RE: [Merge] Minimal Hyper-V Plugin >> >> > >> >> > Hi Donal, >> >> > One more issue is, currently local storage is discovered when host >> >> > is >> >>added. >> >> > But when mgmt. server got restarted, there is an exception >> >> > happening while add/discovering already existing local storage >>pool. >> >> > >> >> > >> >> > Root cause: >> >> > ========== >> >> > I had debugged and figured out the root cause, CS not able to find >> >> > the storage pool and tries to created it but it fails to add to DB >> >> > as the entry is already persisted. >> >> > The reason why CS not able to get the existing localstorage is , >> >> > when executing the query to get the storage pool, it's getting >>empty set >> . >> >> > It's because of the storage path issue. >> >> > In the db the local storage path is stored as >> >> > "C:\Users\Public\Documents\Hyper-V\Virtual+Hard+Disks", ('+' >> >> > symbol in place of space) In the db query its searching without "+" >> >> > and with space which is causing the query to have empty set and >> >> > hence the issue. >> >> > >> >> > I can fix the issue by removing the "+" when we are persisting the >> >> > local storage pool from hyperv >> >> > >> >> > >> >> > >> >> > Exception: >> >> > ========= >> >> > WARN [c.c.r.ResourceManagerImpl] (AgentTaskPool-14:ctx-ddbbf089) >> >> > Unable to connect due to >> >> > com.cloud.exception.ConnectionException: Unable to setup the local >> >> > storage pool for Host[-1-Routing] >> >> > at >> >> > >> >> >> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManage >> >> > rImpl.java:598) >> >> > at >> >> > >> >> >> com.cloud.utils.component.ComponentInstantiationPostProcessor$Interce >> >> p >> >> > >> torDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >> >> > at >> >> > com.cloud.storage.LocalStoragePoolListener.processConnect(LocalStor >> >> > age >> >> > Po >> >> > olListener.java:86) >> >> > at >> >> > >> >> >> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection( >> >> > AgentManagerImpl.java:519) >> >> > at >> >> > >> >> >> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(A >> >> > gentManagerImpl.java:1414) >> >> > at >> >> > >> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource >> >> > ManagerImpl.java:1760) >> >> > at >> >> > >> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource >> >> > ManagerImpl.java:1922) >> >> > at >> >> > >> >> >> com.cloud.agent.manager.AgentManagerImpl$SimulateStartTask.run(Agent >> >> > ManagerImpl.java:1063) >> >> > at >> >> > >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto >> >> > r.j >> >> > av >> >> > a:1110) >> >> > at >> >> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. >> >> > ja >> >> > va:603) >> >> > at java.lang.Thread.run(Thread.java:722) >> >> > Caused by: com.cloud.utils.exception.CloudRuntimeException: >> >> > duplicate >> >> > uuid: 35aa91ba-c95b-3fc4-91df-9c95dc31b350-HypervResource >> >> > at >> >> > >> org.apache.cloudstack.storage.volume.datastore.PrimaryDataStoreHelper. >> >> > cr >> >> > eatePrimaryDataStore(PrimaryDataStoreHelper.java:66) >> >> > at >> >> > org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimary >> >> > Dat >> >> > aSto >> >> > >> >> >> >>reLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl.jav >> >>a:3 >> >>49) >> >> > at >> >> > >> >> >> com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManage >> >> > rImpl.java:589) >> >> > ... 14 more >> >> > >> >> > >> >> > >> >> > -----Original Message----- >> >> > From: Donal Lafferty [mailto:donal.laffe...@citrix.com] >> >> > Sent: Saturday, August 31, 2013 5:19 AM >> >> > To: dev@cloudstack.apache.org >> >> > Subject: [Merge] Minimal Hyper-V Plugin >> >> > >> >> > A plugin for Hyper-V control is available for CloudStack. The >> >> > plugin implements basic VM control; however, its architecture >> >> > allows additional functionality to be easily added. Incorporating >> >> > the plugin in CloudStack will allow the community to participate in >> >> > improving the features available with Hyper-V. >> >> > >> >> > The plugin uses a Director Connect Agent architecture described >>here: >> >> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress >> >> > >> >> > This links provides details of third party libraries used by the >> >> > plugin including their licenses. No source for these libraries is >> >> > used, and the binaries are downloaded from their distributors at >> >>build time. >> >> > >> >> > No proprietary tools are required for the build. For instance, C# >> >> > compiled with Mono has been tested. Therefore, the plugin has been >> >> > added to the default build and the default deployment. >> >> > >> >> > The plugin includes unit and functional tests that can be triggered >> >> > at build time. See the link above for details. >> >> > >> >> > The Apache header is applied to source and where feasible to build >> >> > config files. >> >> > >> >> > The source is the >> >> > https://github.com/lafferty/cloudstack/tree/hyperv_plugin >> >> > >> >> > Finally, the review request is at >> >> > https://reviews.apache.org/r/13922/ >> >> > >> >> > Testing? >> >> > >> >> > Integration test: created zone with CIFS secondary storage, added >> >> > Hyper-V host, registered templates, create/stop/start/destroy VM. >> >> > Scripts for zone setup are available on request. >> >> > >> >> > Functional & unit tests: the Java versions are part of the build. >> >> > The C# versions were run inside Visual Studio's Test Explorer. >
