After much searching I found the two lines of code I needed to get it to serialize right. I'll keep looking and see if I find other snags. Maybe we can just move to jackson.... we'll see, haven't tried to deserialize yet.
Darren > On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> wrote: > > Darren, > > When I looked into updating to the latest gson, the problem, IIRC, is things > that weren't considered cyclical dependencies are suddenly considered > cyclical with the latest. I don't recall where though. > > --Alex > >> -----Original Message----- >> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >> Sent: Thursday, September 19, 2013 1:33 PM >> To: dev@cloudstack.apache.org >> Subject: Re: conflicting dependencies between CloudStack and Whirr >> >> Alright, I looked into this and it will take a bit more work to switch to >> Jackson. >> The snag I hit was how we serialize commands for the agents. We use a >> customer typeadatper in gson to produce a format like { "org.MyClass" >> : {...} }. In jackson I don't see producing a similar format as being so >> straight >> forward. I'm sure there's an elegant solution, but I didn't immediately >> find it. >> The problem is if you use a custom serializer in jackson and do >> writeObjectField(x.getClass().getName(), x), you'll get stuck in a loop >> because it will call your serializer again. So if somebody can figure out >> how to >> do this format easily in jackson, the rest looks simple enough. >> >> So, being that it is not an obvious switch to using jackson, what is the >> impact >> of moving to the latest gson? What were the issue encountered when >> people tried it before? >> >> Darren >> >> >> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >> darren.s.sheph...@gmail.com> wrote: >> >>> I can do some analysis on this. I'm always up for a terribly painful >>> refactor :) >>> >>> Darren >>> >>> >>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <alex.hu...@citrix.com> >>> wrote: >>> >>>> I actually did a quick try to update cloudstack to use newest gson >>>> version about 3 months back. Had to roll it back but I didn't try >>>> very hard though. Part of the reason why I decided to rollback is >>>> due to gson is used differently by various components in CloudStack >>>> and I didn't have time to get into each component to see if I break >>>> anything. The other is also I thought using Jackson would be much >>>> better and thought our next attempt should be with Jackson. >>>> >>>> Not that any of these helps you. Just know updating CloudStack to >>>> latest gson is not simple. >>>> >>>> --Alex >>>> >>>>> -----Original Message----- >>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>> To: d...@whirr.apache.org >>>>> Cc: dev@cloudstack.apache.org >>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>> >>>>> It's easy to usr jclouds and whirr inside an OSGi container - just >>>>> add >>>> the >>>>> feature url. Bonus: you can also use jclouds shell interface (part >>>>> of >>>> jclouds cli). >>>>> >>>>> Another option is to upgrade the CloudStack API to use the new version. >>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>> >>>>>> Dear all, >>>>>> >>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>> various clusters on CloudStack. Now I am facing a dependency >>>> conflicting >>>>> issue. >>>>>> >>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>> gson >>>> 1.7.1. >>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>> >>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>> java.lang.**NoClassDefFoundError: >>>>>> com/google/gson/TypeAdapter >>>>>> >>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will >>>>>> not build successfully due to the following test error. >>>>>> >>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>> ------------------------------**------------------------------** >>>>>> ------------ >>>>>> [INFO] >>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>> cloud-core >>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>> [INFO] >>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) >>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8' >>>>>> encoding to copy filtered resources. >>>>>> [INFO] skip non existing resourceDirectory >>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >> 3 >>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile >>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources. >>>>>> [INFO] skip non existing resourceDirectory >>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >> 3 >>>>>> resources [INFO] [INFO] --- >>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source >>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO] >>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>> cloud-core >>>>>> --- >>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>> target/surefire-reports >>>>>> >>>>>> ------------------------------**------------------------- >>>>>> T E S T S >>>>>> ------------------------------**------------------------- >>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>> log4j:WARN No appenders could be found for logger >>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>> log4j:WARN See >>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>> http://logging.apa >>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>> 3.054 sec <<< FAILURE! >>>>>> >>>>>> Results : >>>>>> >>>>>> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>> >>>>>> Tests in error: >>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>> >>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>> >>>>>> >>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>> CloudStack building process went through. But when I issued an >>>>>> API --launchCluster to CloudStack, the following error related to >>>>>> gson popped up on CloudStack Management Server: >>>>>> >>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught >>>>>> problem with >>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy" >>>>>> :"** >>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>> **LibvirtComputingResource","* >>>>>> * >>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>> 10.244.18 >>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-* >>>>>> * >>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource >>>>>> ","** >>>>>> contextMap":{},"wait":0}}] >>>>>> java.lang.ClassCastException: >>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>> cannot be cast to java.lang.Class >>>>>> at >>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>> ArrayTypeAdaptor.java:84) >>>>>> at >>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>> ArrayTypeAdaptor.java:37) >>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>> TreeTypeAdapter.java:58) >>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>> at >>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>> java:235) >>>>>> at >>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>> at >>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>> ) >>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>> ThreadPoolExecutor.java:1110) >>>>>> at >>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>> ThreadPoolExecutor.java:603) >>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>> following exception but pushing on >>>>>> >>>>>> On the CS Managment console I can see the cluster was started >>>>>> then quickly died. >>>>>> Running on provider cloudstack using identity >> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>> ZRGXVO1VApG >>>>>> 6q0a >>>>>> **K-A-tQRQsZFwnOXQ >>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:) >>>>>> Configuring template for >>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>> (729061754@qtp-385354117- >>>>> 0:) >>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>> jobtracker >>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) >>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>> >>>>>> I attached the debuging log to this email. >>>>>> >>>>>> The launchCluster API is simple, it calls the >>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>> >>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>> response.setObjectName("**launchCluster"); >>>>>> LaunchClusterCommand command = null; >>>>>> try { >>>>>> command = new LaunchClusterCommand(); >>>>>> } catch (Exception ex) { >> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>> VER >>>>>> E, >>>>>> null, ex); >>>>>> } >>>>>> String[] args = new String[2]; >>>>>> args[0] = "--config"; >>>>>> args[1] = config; >>>>>> >>>>>> try { >>>>>> command.run(System.in, System.out, System.err, >>>>>> Arrays.asList(args)); >>>>>> } catch (Exception ex) { >> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>> VER >>>>>> E, >>>>>> null, ex); >>>>>> } >>>>>> response.setResponseName(**getCommandName()); >>>>>> output = "successfully launched the cluster."; >>>>>> response.setOutPut(output); >>>>>> response.setAsync(Boolean.**FALSE); >>>>>> this.setResponseObject(**response); >>>>>> >>>>>> Could someone help me out here? What would be a proper gson >>>>>> version for CloudStack and Whirr to run at the same time? >>>>>> >>>>>> Thanks loads. >>>>>> >>>>>> Best Regards, >>>>>> Meng >>> >>>
