On Sep 23, 2013, at 2:54 PM, Chiradeep Vittal <chiradeep.vit...@citrix.com> wrote:
> I still think this is the wrong way to implement the Whirr integration. Probably, but it's too late now. We can just get it in a branch and look at it > > On 9/23/13 6:45 AM, "sebgoa" <run...@gmail.com> wrote: > >> >> On Sep 23, 2013, at 7:12 AM, Darren Shepherd >> <darren.s.sheph...@gmail.com> wrote: >> >>> The only way this will get committed is if we either move to latest >>> gson or >>> jackson. Regardless, the outcome will be that Meng can assume this gson >>> conflict won't happen. The problem is just how fast can we move all of >>> ACS >>> to a new json library. Is it possible for Meng to commit to a branch >>> that >>> just blindly updates gson to the latest (in /pom.xml). >> >> Yes I will talk to Meng about this. thanks >> >>> When we get ACS on >>> a newer json library, we can merge in that branch. Tomorrow (9/23), I >>> can >>> attempt to put together a patch to move to jackson. I think I ironed >>> out >>> most of the technical issues so I just have to globally apply the >>> changes. >>> >>> Darren >>> >>> >>> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen >>> <run...@gmail.com>wrote: >>> >>>> >>>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <h...@trippaers.nl> wrote: >>>> >>>>> >>>>> The Nicira NVP plugin is also depending on gson. If we make any >>>>> changes >>>> we need to validate against their api to ensure that the interface >>>> still >>>> works. >>>>> >>>>> If we put the changes in a separate branch i'd be happy to run the >>>> changes against the api. >>>>> >>>>> Cheers, >>>>> >>>>> Hugo >>>> >>>> So how can Meng get out of this bind, she is trying to complete her >>>> google >>>> summer of code project. >>>> >>>> thanks, >>>> >>>> -Sebastien >>>> >>>>> >>>>> >>>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd < >>>> darren.s.sheph...@gmail.com> wrote: >>>>> >>>>>> After much searching I found the two lines of code I needed to get it >>>> to serialize right. I'll keep looking and see if I find other snags. >>>> Maybe we can just move to jackson.... we'll see, haven't tried to >>>> deserialize yet. >>>>>> >>>>>> Darren >>>>>> >>>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> >>>>>>> wrote: >>>>>>> >>>>>>> Darren, >>>>>>> >>>>>>> When I looked into updating to the latest gson, the problem, IIRC, >>>>>>> is >>>> things that weren't considered cyclical dependencies are suddenly >>>> considered cyclical with the latest. I don't recall where though. >>>>>>> >>>>>>> --Alex >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >>>>>>>> Sent: Thursday, September 19, 2013 1:33 PM >>>>>>>> To: dev@cloudstack.apache.org >>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>>>> >>>>>>>> Alright, I looked into this and it will take a bit more work to >>>> switch to Jackson. >>>>>>>> The snag I hit was how we serialize commands for the agents. We >>>>>>>> use a >>>>>>>> customer typeadatper in gson to produce a format like { >>>>>>>> "org.MyClass" >>>>>>>> : {...} }. In jackson I don't see producing a similar format as >>>> being so straight >>>>>>>> forward. I'm sure there's an elegant solution, but I didn't >>>> immediately find it. >>>>>>>> The problem is if you use a custom serializer in jackson and do >>>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a >>>> loop >>>>>>>> because it will call your serializer again. So if somebody can >>>> figure out how to >>>>>>>> do this format easily in jackson, the rest looks simple enough. >>>>>>>> >>>>>>>> So, being that it is not an obvious switch to using jackson, what >>>>>>>> is >>>> the impact >>>>>>>> of moving to the latest gson? What were the issue encountered when >>>>>>>> people tried it before? >>>>>>>> >>>>>>>> Darren >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >>>>>>>> darren.s.sheph...@gmail.com> wrote: >>>>>>>> >>>>>>>>> I can do some analysis on this. I'm always up for a terribly >>>>>>>>> painful >>>>>>>>> refactor :) >>>>>>>>> >>>>>>>>> Darren >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang >>>>>>>>>> <alex.hu...@citrix.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I actually did a quick try to update cloudstack to use newest >>>>>>>>>> gson >>>>>>>>>> version about 3 months back. Had to roll it back but I didn't >>>>>>>>>> try >>>>>>>>>> very hard though. Part of the reason why I decided to rollback >>>>>>>>>> is >>>>>>>>>> due to gson is used differently by various components in >>>>>>>>>> CloudStack >>>>>>>>>> and I didn't have time to get into each component to see if I >>>>>>>>>> break >>>>>>>>>> anything. The other is also I thought using Jackson would be >>>>>>>>>> much >>>>>>>>>> better and thought our next attempt should be with Jackson. >>>>>>>>>> >>>>>>>>>> Not that any of these helps you. Just know updating CloudStack >>>>>>>>>> to >>>>>>>>>> latest gson is not simple. >>>>>>>>>> >>>>>>>>>> --Alex >>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>>>>>>>> To: d...@whirr.apache.org >>>>>>>>>>> Cc: dev@cloudstack.apache.org >>>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and >>>>>>>>>>> Whirr >>>>>>>>>>> >>>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - >>>>>>>>>>> just >>>>>>>>>>> add >>>>>>>>>> the >>>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface >>>>>>>>>>> (part >>>>>>>>>>> of >>>>>>>>>> jclouds cli). >>>>>>>>>>> >>>>>>>>>>> Another option is to upgrade the CloudStack API to use the new >>>> version. >>>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Dear all, >>>>>>>>>>>> >>>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency >>>>>>>>>> conflicting >>>>>>>>>>> issue. >>>>>>>>>>>> >>>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>>>>>>>> gson >>>>>>>>>> 1.7.1. >>>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>>>>>>>> >>>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>>>>>>>> java.lang.**NoClassDefFoundError: >>>>>>>>>>>> com/google/gson/TypeAdapter >>>>>>>>>>>> >>>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack >>>>>>>>>>>> will >>>>>>>>>>>> not build successfully due to the following test error. >>>>>>>>>>>> >>>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>>>>>>>> >>>>>>>>>>>> ------------------------------**------------------------------** >>>>>>>>>>>> ------------ >>>>>>>>>>>> [INFO] >>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>>>>>>>> cloud-core >>>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>>>>>>>> [INFO] >>>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process >>>>>>>>>>>> (default) >>>>>>>>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using >>>>>>>>>>>> 'UTF-8' >>>>>>>>>>>> encoding to copy filtered resources. >>>>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >>>>>>>> 3 >>>>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>>>> maven-compiler-plugin:2.5.1:**compile >>>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered >>>> resources. >>>>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >>>>>>>> 3 >>>>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 >>>>>>>>>>>> source >>>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes >>>>>>>>>>>> [INFO] >>>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>>>>>>>> cloud-core >>>>>>>>>>>> --- >>>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>>>>>>>> target/surefire-reports >>>>>>>>>>>> >>>>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>>>> T E S T S >>>>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>>>>>>>> log4j:WARN No appenders could be found for logger >>>>>>>>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>>>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>>>>>>> log4j:WARN See >>>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>>>>>>>> http://logging.apa >>>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>>>>>>>> 3.054 sec <<< FAILURE! >>>>>>>>>>>> >>>>>>>>>>>> Results : >>>>>>>>>>>> >>>>>>>>>>>> Failed tests: >>>> testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>>>>>>>> >>>>>>>>>>>> Tests in error: >>>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>>>>>>>> >>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>>>>>>>> CloudStack building process went through. But when I issued an >>>>>>>>>>>> API --launchCluster to CloudStack, the following error related >>>>>>>>>>>> to >>>>>>>>>>>> gson popped up on CloudStack Management Server: >>>>>>>>>>>> >>>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) >>>>>>>>>>>> Caught >>>>>>>>>>>> problem with >>>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >>>>>>>> >>>>>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg >>>>>>>> y" >>>>>>>>>>>> :"** >>>>>>>>>>>> >>>>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>>>> **LibvirtComputingResource","* >>>>>>>>>>>> * >>>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>>>>>>>> 10.244.18 >>>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>>>>>>>> >>>>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>>>>>>>> >>>>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>>>>> * >>>>>>>>>>>> * >>>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>>>>>>>> >>>>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc >>>>>>>>>>>> e >>>>>>>>>>>> ","** >>>>>>>>>>>> contextMap":{},"wait":0}}] >>>>>>>>>>>> java.lang.ClassCastException: >>>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>>>>>>>> cannot be cast to java.lang.Class >>>>>>>>>>>> at >>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>>>> ArrayTypeAdaptor.java:84) >>>>>>>>>>>> at >>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>>>> ArrayTypeAdaptor.java:37) >>>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>>>>>>>> TreeTypeAdapter.java:58) >>>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>>>>>>>> at >>>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>>>>>>>> java:235) >>>>>>>>>>>> at >>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>>>>>>>> at >>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >>>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>>>>>>>> ) >>>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>>>>>>>> ThreadPoolExecutor.java:1110) >>>>>>>>>>>> at >>>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>>>>>>>> ThreadPoolExecutor.java:603) >>>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>>>>>>>> following exception but pushing on >>>>>>>>>>>> >>>>>>>>>>>> On the CS Managment console I can see the cluster was started >>>>>>>>>>>> then quickly died. >>>>>>>>>>>> Running on provider cloudstack using identity >>>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>>>>>>>> ZRGXVO1VApG >>>>>>>>>>>> 6q0a >>>>>>>>>>>> **K-A-tQRQsZFwnOXQ >>>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>>>>>>>> [whirr.compute.**BootstrapTemplate] >>>>>>>>>>>> (729061754@qtp-385354117-0:) >>>>>>>>>>>> Configuring template for >>>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>>>>>>>> (729061754@qtp-385354117- >>>>>>>>>>> 0:) >>>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>>>>>>>> jobtracker >>>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 >>>>>>>>>>>> node(s) >>>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>>>>>>>> >>>>>>>>>>>> I attached the debuging log to this email. >>>>>>>>>>>> >>>>>>>>>>>> The launchCluster API is simple, it calls the >>>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>>>>>>>> >>>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>>>>>>>> response.setObjectName("**launchCluster"); >>>>>>>>>>>> LaunchClusterCommand command = null; >>>>>>>>>>>> try { >>>>>>>>>>>> command = new LaunchClusterCommand(); >>>>>>>>>>>> } catch (Exception ex) { >>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>>>> VER >>>>>>>>>>>> E, >>>>>>>>>>>> null, ex); >>>>>>>>>>>> } >>>>>>>>>>>> String[] args = new String[2]; >>>>>>>>>>>> args[0] = "--config"; >>>>>>>>>>>> args[1] = config; >>>>>>>>>>>> >>>>>>>>>>>> try { >>>>>>>>>>>> command.run(System.in, System.out, System.err, >>>>>>>>>>>> Arrays.asList(args)); >>>>>>>>>>>> } catch (Exception ex) { >>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>>>> VER >>>>>>>>>>>> E, >>>>>>>>>>>> null, ex); >>>>>>>>>>>> } >>>>>>>>>>>> response.setResponseName(**getCommandName()); >>>>>>>>>>>> output = "successfully launched the cluster."; >>>>>>>>>>>> response.setOutPut(output); >>>>>>>>>>>> response.setAsync(Boolean.**FALSE); >>>>>>>>>>>> this.setResponseObject(**response); >>>>>>>>>>>> >>>>>>>>>>>> Could someone help me out here? What would be a proper gson >>>>>>>>>>>> version for CloudStack and Whirr to run at the same time? >>>>>>>>>>>> >>>>>>>>>>>> Thanks loads. >>>>>>>>>>>> >>>>>>>>>>>> Best Regards, >>>>>>>>>>>> Meng >>>>>>>>> >>>>>>>>> >>>>> >>>> >>>> >> >
