Donal, First, thank you for sending this out! It seems like really good progress. Who are you working with on this? Are you running solo?
The technical elements of this seem sound at first review. The licensing implications are going to take some time to parse through though. One of the things that could help us on that front, would be to understand which of the licenses are runtime vs. buildtime. It would also be good to highlight which dependencies would easily be considered "system dependencies" (as in: expected to be installed prior to our software being installed). Again, thanks! -chip On Fri, May 24, 2013 at 02:22:41AM +0000, Donal Lafferty wrote: > Reproduced from > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress > > Apologies for the length - skip to the heading that interests you. > > h1. Update on Hyper-V plugin work (2013-05-23): > > h2. New plugin Architecture: > > https://cwiki.apache.org/confluence/download/attachments/31822487/Phase2_HyperV_Remote_Agent.png > > h3. How does is the new architecture an improvement? > > The plugin's architecture uses JSON over HTTP. This allows the plugin agent > to be written in C# rather than Java. .NET has tools to create WMI wrappers > that allows the Hyper-V API calls to be quickly rewritten in C#. The result > is easier to maintain and does not suffer the IP problems of the Python used > in phase 1. An all C# agent can be contained in a single C#/.NET Windows > Service. The result is a more robust design, because there are far fewer > points of failure. > > Switching from CloudStack Message bus to JSON over HTTP communications allows > the plugin's remote agent to be implemented in C#. Previously, this agent > was written in Java to take advantage to the existing CloudStack message bus > agent. The CloudStack message bus is difficult to port due it's reliance on > the Java NIO API. The NIO API is peculiar to Java. As a consequence, porting > to even similar languages such as C# is difficult. Design work is required > that would consume significant time. In contrast, JSON over HTTP passes > kernel commands as JSON in the body of an HTTP POST. This solution is > RPC-like and not RESTful, because the destination URI that corresponds to the > command and not the resource being manipulated. The HTTP stack removes the > dependency on NIO. HTTP stacks themselves are easy to come by regardless of > the language or platform. Using an RPC approach avoids undertaking design > work to translate incoming kernel commands to a suitable RESTful equivalent. > Therefore, JSON over HTTP minimises the effort required to adopt the best > language for a platform. > > C# code to call the Hyper-V's API directly removes IP restrictions and > complexity of the existing Python code. Hyper-V is controlled through > Windows Management Interface (WMI). WMI provides a uniform framework through > which services and operating subsystems can provide information and register > control APIs. All Hyper-V control packages ultimately call this API. Phase 1 > of the plugin controlled Hyper-V using a port of cloud.com's OpenStack driver > for Hyper-V. The driver could be easily repurposed, because the OpenStack > information model is similar to that of CloudStack. However, this Python > code has IP, support, and coding drawbacks. The code is a derivative of an > existing work, which prevented immediate donation to ASF. The code used a > third party WMI wrapper package that limited the version of Python that could > be used. Finally, the python wrapper did not address discover of WMI > objects, properties and methods. In contrast, the C# code can be transferred > to ASF immediately. The strongly typed wrappers rely on C# and .NET, which > is well supported. These wrappers are simple to update as they are machine > generated code. More importantly, they enable property and method discovery > through autocomplete. Therefore, the C# version is easier to maintain and > addresses Apache's copyright requirements. > > Access to native error handling, APIs and services make the C#/.NET agent > more robust. Previously, the remote agent created a new process every time > Python was invoked. Communications with this new process were over stdin and > stdout. The serialisation requirements meant exceptions could not be used > for error handling. Also, the design was vulnerable to deadlock. Should the > new process not return but not die either, the call would be stuck. Now that > the agent is written entirely in C# agent, it can pass errors in a single > exception rather than a series of method returns. Furthermore, using Python > along side Java in phase 1 provides access to platform-specific APIs not > exposed by the Java Runtime Environment (JRE). In contrast, .NET allows > direct access to Windows APIs, which means the C# agent has no need for > external processes. Finally, additional scripts were required to > start/stop/restart the phase 1 agent. With phase 2, the agent is implemented > with hooks that allow it to be launched as a Windows Service. Using these > hooks, the Windows Service Control Manager (SCM) takes care of executing the > agent as if it were a daemon. Therefore, switching to a single language and > process means the new remote agent has fewer points of failure. > > An additional strength of this design is that pushes the hypervisor-specific > code out of the server component. Previously, a hypervisor's remote agent > would have to register itself with the management server. This meant that > the agent had to somehow know its zone, pod, and cluster location before it > joined itself to the cloud. For example, the KVM server component uses SSH > to launch the remote agent with the necessary configuration data. In our new > design, the remote agent can be told its cloud configuration after it has > been started. This removes the need for a hypervisor-specific control to > launch remote agent. Therefore, the resulting server component is much more > reusable. > > To summarise, the plugin has an updated communications stack called JSON over > HTTP. This has simplified the implementation of the remote agent and the > corresponding server component. The result is a more robust and reusable > design, and a new implementation that meets Apache CloudStack's IP > requirements. > > > h2. Implementation Status: > > A VisualStudio 2012 SP1 solution on GitHub > ([https://github.com/lafferty/cshv3/tree/master/plugins/hypervisors/hyperv/DotNet/ServerResource]) > holds the plugin's new remote agent. VM and volume creation / deletion > commands are mostly complete. Additional storage commands are required to > allow transfer of template and volume images. Attach / detach commands > should be added to allow System VM patching. > > The solution includes unit tests. The tests are a port of the functional > tests from phase 1. At the time of writing, the port was still being > completed. > > The implementation avoids using features that would require the hypervisor to > run Windows Server 2012 in the parent partition. The image store uses S3 for > template storage, because the Windows NFS client is an advanced feature that > is not available with the basic and free Hyper-V Server 2012. Likewise, the > remote agent hosts its own HTTP stack to avoid any dependency of Microsoft's > Internet Information Services (IIS) web server. > > A Server component is available that uses JSON over HTTP to communicate the > remote agent. The existing unit tests are being updated to call this server > component. Additional scripting is required to launch the remote agent so > that HTTP requests can be properly received and acted on. The server > component's Discoverer algorithm needs to be updated. > > The build and test system has dependencies on Microsoft tools and platforms. > To build the system with free tools, the remote agent should be compiled > against VisualStudio express. To build with Linux, the solution should be > compiled with Mono. However, neither of these options have been tried. > > > h2. IP Dependencies > > The C# source code and configuration files that are input for code generate > are in the process of being marked with the Apache header > ([http://www.apache.org/legal/src-headers.html#headers]) to indicate that > Apache CloudStack will hold the copyright. > > Third party binaries used by the solution are the Microsoft .NET Framework > 4.5, NuGet packages, and AWS .NET SDK. The AWS SDK is used to access S3 > storage. The NuGet packages provide logging, JSON serialisation, and a light > weight HTTP stack. The background to NuGet is that it is akin to a Maven > repo, but each packages has a corresponding web page that includes details of > the license for use of the package. > > Licenses for the NuGet packages are enumerated below. I do not have the .NET > Framework 4.5 EULA to hand. > > > Apache License, Version 2.0 > ([http://logging.apache.org/log4net/license.html)]: > > AWS .NET SDK ([http://aws.amazon.com/sdkfornet/faqs/#13]) > Log4net 2.0.0 ([http://nuget.org/packages/log4net/]) > > > The MIT License (MIT): > > Newtonsoft.Json 4.5.11 [http://json.codeplex.com/license] > > > MICROSOFT ASP.NET MODEL VIEW CONTROLLER 4 EULA > ([http://www.microsoft.com/web/webpi/eula/mvc_4_eula_enu.htm]) > > Microsoft ASP.NET Web API Core Libraries 4.0.20710.0 > [http://nuget.org/packages/Microsoft.AspNet.WebApi.Core/4.0.20710.0] NuGet > id="Microsoft.AspNet.WebApi.Core" version="4.0.20710.0" > Microsoft ASP.NET Web API Client Libraries 4.0.20710.0 > [http://nuget.org/packages/Microsoft.AspNet.WebApi.Client/4.0.20710.0] NuGet > id="Microsoft.AspNet.WebApi.Client" version="4.0.20710.0" > Microsoft .NET Framework 4 HTTP Client Libraries 2.0.20710.0 > [http://nuget.org/packages/Microsoft.Net.Http/2.0.20710.0] NuGet > id="Microsoft.Net.Http" version="2.0.20710.0" > Microsoft ASP.NET Web API Self Host 4.0.20918.0 > [http://nuget.org/packages/Microsoft.AspNet.WebApi.SelfHost/4.0.20918.0] > NuGet id="Microsoft.AspNet.WebApi.SelfHost" version="4.0.20918.0" > > > > h2. System VM services: > > The plugin depends on system VM services that are coded separately. System > VM services refer to CloudStack functionality that is offloaded from the > management server to VMs that sit in the cloud. The services in question are > template/snapshot storage, networking services, and console access. > Traditionally, CloudStack has implemented these services using system VMs: > secondary storage VM for image persistence, virtual router VM for networking > services, and console VM for access to guest VM consoles. For the Hyper-V > plugin to work properly, CloudStack has to be able to offer these same > services on Hyper-V hypervisors. > > The image that the CloudStack database points to for Hyper-V system VM needs > to be updated with a version that includes the console proxy. The latest > system VM includes Hyper-V kernel modules. This provides it with the high > performance network driver to act as a virtual router VM. The secondary > storage VM capabilities are not required. The secondary storage VM allows > CloudStack to upload images to an NFS store. However, a NFS client is not > available in the "Hyper-V Server" that we are targeting. Therefore, we will > not be using NFS for image storage. Finally, the console services must be > updated. Hyper-V's guest consoles are accessed through RDP, which means an > RDP proxy must be added to the system VM. Therefore, changes to the system > VM appear to be limited to the addition of a Hyper-V compatible console proxy.
