On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <h...@trippaers.nl> wrote:
> > The Nicira NVP plugin is also depending on gson. If we make any changes we > need to validate against their api to ensure that the interface still works. > > If we put the changes in a separate branch i'd be happy to run the changes > against the api. > > Cheers, > > Hugo So how can Meng get out of this bind, she is trying to complete her google summer of code project. thanks, -Sebastien > > > On Sep 20, 2013, at 12:08 PM, Darren Shepherd <darren.s.sheph...@gmail.com> > wrote: > >> After much searching I found the two lines of code I needed to get it to >> serialize right. I'll keep looking and see if I find other snags. Maybe we >> can just move to jackson.... we'll see, haven't tried to deserialize yet. >> >> Darren >> >>> On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> wrote: >>> >>> Darren, >>> >>> When I looked into updating to the latest gson, the problem, IIRC, is >>> things that weren't considered cyclical dependencies are suddenly >>> considered cyclical with the latest. I don't recall where though. >>> >>> --Alex >>> >>>> -----Original Message----- >>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >>>> Sent: Thursday, September 19, 2013 1:33 PM >>>> To: dev@cloudstack.apache.org >>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>> >>>> Alright, I looked into this and it will take a bit more work to switch to >>>> Jackson. >>>> The snag I hit was how we serialize commands for the agents. We use a >>>> customer typeadatper in gson to produce a format like { "org.MyClass" >>>> : {...} }. In jackson I don't see producing a similar format as being so >>>> straight >>>> forward. I'm sure there's an elegant solution, but I didn't immediately >>>> find it. >>>> The problem is if you use a custom serializer in jackson and do >>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a loop >>>> because it will call your serializer again. So if somebody can figure out >>>> how to >>>> do this format easily in jackson, the rest looks simple enough. >>>> >>>> So, being that it is not an obvious switch to using jackson, what is the >>>> impact >>>> of moving to the latest gson? What were the issue encountered when >>>> people tried it before? >>>> >>>> Darren >>>> >>>> >>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >>>> darren.s.sheph...@gmail.com> wrote: >>>> >>>>> I can do some analysis on this. I'm always up for a terribly painful >>>>> refactor :) >>>>> >>>>> Darren >>>>> >>>>> >>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <alex.hu...@citrix.com> >>>>> wrote: >>>>> >>>>>> I actually did a quick try to update cloudstack to use newest gson >>>>>> version about 3 months back. Had to roll it back but I didn't try >>>>>> very hard though. Part of the reason why I decided to rollback is >>>>>> due to gson is used differently by various components in CloudStack >>>>>> and I didn't have time to get into each component to see if I break >>>>>> anything. The other is also I thought using Jackson would be much >>>>>> better and thought our next attempt should be with Jackson. >>>>>> >>>>>> Not that any of these helps you. Just know updating CloudStack to >>>>>> latest gson is not simple. >>>>>> >>>>>> --Alex >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>>>> To: d...@whirr.apache.org >>>>>>> Cc: dev@cloudstack.apache.org >>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>>> >>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just >>>>>>> add >>>>>> the >>>>>>> feature url. Bonus: you can also use jclouds shell interface (part >>>>>>> of >>>>>> jclouds cli). >>>>>>> >>>>>>> Another option is to upgrade the CloudStack API to use the new version. >>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>>>> >>>>>>>> Dear all, >>>>>>>> >>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>>>> various clusters on CloudStack. Now I am facing a dependency >>>>>> conflicting >>>>>>> issue. >>>>>>>> >>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>>>> gson >>>>>> 1.7.1. >>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>>>> >>>>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>>>> java.lang.**NoClassDefFoundError: >>>>>>>> com/google/gson/TypeAdapter >>>>>>>> >>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will >>>>>>>> not build successfully due to the following test error. >>>>>>>> >>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>>>> ------------------------------**------------------------------** >>>>>>>> ------------ >>>>>>>> [INFO] >>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>>>> cloud-core >>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>>>> [INFO] >>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) >>>>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8' >>>>>>>> encoding to copy filtered resources. >>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >>>> 3 >>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile >>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources. >>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >>>> 3 >>>>>>>> resources [INFO] [INFO] --- >>>>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source >>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO] >>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>>>> cloud-core >>>>>>>> --- >>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>>>> target/surefire-reports >>>>>>>> >>>>>>>> ------------------------------**------------------------- >>>>>>>> T E S T S >>>>>>>> ------------------------------**------------------------- >>>>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>>>> log4j:WARN No appenders could be found for logger >>>>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>>> log4j:WARN See >>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>>>> http://logging.apa >>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>>>> 3.054 sec <<< FAILURE! >>>>>>>> >>>>>>>> Results : >>>>>>>> >>>>>>>> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>>>> >>>>>>>> Tests in error: >>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>>>> >>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>>>> >>>>>>>> >>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>>>> CloudStack building process went through. But when I issued an >>>>>>>> API --launchCluster to CloudStack, the following error related to >>>>>>>> gson popped up on CloudStack Management Server: >>>>>>>> >>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught >>>>>>>> problem with >>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy" >>>>>>>> :"** >>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>> **LibvirtComputingResource","* >>>>>>>> * >>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>>>> 10.244.18 >>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-* >>>>>>>> * >>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource >>>>>>>> ","** >>>>>>>> contextMap":{},"wait":0}}] >>>>>>>> java.lang.ClassCastException: >>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>>>> cannot be cast to java.lang.Class >>>>>>>> at >>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>> ArrayTypeAdaptor.java:84) >>>>>>>> at >>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>> ArrayTypeAdaptor.java:37) >>>>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>>>> TreeTypeAdapter.java:58) >>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>>>> at >>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>>>> java:235) >>>>>>>> at >>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>>>> at >>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>>>> ) >>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>>>> ThreadPoolExecutor.java:1110) >>>>>>>> at >>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>>>> ThreadPoolExecutor.java:603) >>>>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>>>> following exception but pushing on >>>>>>>> >>>>>>>> On the CS Managment console I can see the cluster was started >>>>>>>> then quickly died. >>>>>>>> Running on provider cloudstack using identity >>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>>>> ZRGXVO1VApG >>>>>>>> 6q0a >>>>>>>> **K-A-tQRQsZFwnOXQ >>>>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:) >>>>>>>> Configuring template for >>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>>>> (729061754@qtp-385354117- >>>>>>> 0:) >>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>>>> jobtracker >>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) >>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>>>> >>>>>>>> I attached the debuging log to this email. >>>>>>>> >>>>>>>> The launchCluster API is simple, it calls the >>>>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>>>> >>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>>>> response.setObjectName("**launchCluster"); >>>>>>>> LaunchClusterCommand command = null; >>>>>>>> try { >>>>>>>> command = new LaunchClusterCommand(); >>>>>>>> } catch (Exception ex) { >>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>> VER >>>>>>>> E, >>>>>>>> null, ex); >>>>>>>> } >>>>>>>> String[] args = new String[2]; >>>>>>>> args[0] = "--config"; >>>>>>>> args[1] = config; >>>>>>>> >>>>>>>> try { >>>>>>>> command.run(System.in, System.out, System.err, >>>>>>>> Arrays.asList(args)); >>>>>>>> } catch (Exception ex) { >>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>> VER >>>>>>>> E, >>>>>>>> null, ex); >>>>>>>> } >>>>>>>> response.setResponseName(**getCommandName()); >>>>>>>> output = "successfully launched the cluster."; >>>>>>>> response.setOutPut(output); >>>>>>>> response.setAsync(Boolean.**FALSE); >>>>>>>> this.setResponseObject(**response); >>>>>>>> >>>>>>>> Could someone help me out here? What would be a proper gson >>>>>>>> version for CloudStack and Whirr to run at the same time? >>>>>>>> >>>>>>>> Thanks loads. >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Meng >>>>> >>>>> >
