I still think this is the wrong way to implement the Whirr integration. On 9/23/13 6:45 AM, "sebgoa" <run...@gmail.com> wrote:
> >On Sep 23, 2013, at 7:12 AM, Darren Shepherd ><darren.s.sheph...@gmail.com> wrote: > >> The only way this will get committed is if we either move to latest >>gson or >> jackson. Regardless, the outcome will be that Meng can assume this gson >> conflict won't happen. The problem is just how fast can we move all of >>ACS >> to a new json library. Is it possible for Meng to commit to a branch >>that >> just blindly updates gson to the latest (in /pom.xml). > >Yes I will talk to Meng about this. thanks > >> When we get ACS on >> a newer json library, we can merge in that branch. Tomorrow (9/23), I >>can >> attempt to put together a patch to move to jackson. I think I ironed >>out >> most of the technical issues so I just have to globally apply the >>changes. >> >> Darren >> >> >> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen >><run...@gmail.com>wrote: >> >>> >>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <h...@trippaers.nl> wrote: >>> >>>> >>>> The Nicira NVP plugin is also depending on gson. If we make any >>>>changes >>> we need to validate against their api to ensure that the interface >>>still >>> works. >>>> >>>> If we put the changes in a separate branch i'd be happy to run the >>> changes against the api. >>>> >>>> Cheers, >>>> >>>> Hugo >>> >>> So how can Meng get out of this bind, she is trying to complete her >>>google >>> summer of code project. >>> >>> thanks, >>> >>> -Sebastien >>> >>>> >>>> >>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd < >>> darren.s.sheph...@gmail.com> wrote: >>>> >>>>> After much searching I found the two lines of code I needed to get it >>> to serialize right. I'll keep looking and see if I find other snags. >>> Maybe we can just move to jackson.... we'll see, haven't tried to >>> deserialize yet. >>>>> >>>>> Darren >>>>> >>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> >>>>>>wrote: >>>>>> >>>>>> Darren, >>>>>> >>>>>> When I looked into updating to the latest gson, the problem, IIRC, >>>>>>is >>> things that weren't considered cyclical dependencies are suddenly >>> considered cyclical with the latest. I don't recall where though. >>>>>> >>>>>> --Alex >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >>>>>>> Sent: Thursday, September 19, 2013 1:33 PM >>>>>>> To: dev@cloudstack.apache.org >>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>>> >>>>>>> Alright, I looked into this and it will take a bit more work to >>> switch to Jackson. >>>>>>> The snag I hit was how we serialize commands for the agents. We >>>>>>>use a >>>>>>> customer typeadatper in gson to produce a format like { >>>>>>>"org.MyClass" >>>>>>> : {...} }. In jackson I don't see producing a similar format as >>> being so straight >>>>>>> forward. I'm sure there's an elegant solution, but I didn't >>> immediately find it. >>>>>>> The problem is if you use a custom serializer in jackson and do >>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a >>> loop >>>>>>> because it will call your serializer again. So if somebody can >>> figure out how to >>>>>>> do this format easily in jackson, the rest looks simple enough. >>>>>>> >>>>>>> So, being that it is not an obvious switch to using jackson, what >>>>>>>is >>> the impact >>>>>>> of moving to the latest gson? What were the issue encountered when >>>>>>> people tried it before? >>>>>>> >>>>>>> Darren >>>>>>> >>>>>>> >>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >>>>>>> darren.s.sheph...@gmail.com> wrote: >>>>>>> >>>>>>>> I can do some analysis on this. I'm always up for a terribly >>>>>>>>painful >>>>>>>> refactor :) >>>>>>>> >>>>>>>> Darren >>>>>>>> >>>>>>>> >>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang >>>>>>>>><alex.hu...@citrix.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I actually did a quick try to update cloudstack to use newest >>>>>>>>>gson >>>>>>>>> version about 3 months back. Had to roll it back but I didn't >>>>>>>>>try >>>>>>>>> very hard though. Part of the reason why I decided to rollback >>>>>>>>>is >>>>>>>>> due to gson is used differently by various components in >>>>>>>>>CloudStack >>>>>>>>> and I didn't have time to get into each component to see if I >>>>>>>>>break >>>>>>>>> anything. The other is also I thought using Jackson would be >>>>>>>>>much >>>>>>>>> better and thought our next attempt should be with Jackson. >>>>>>>>> >>>>>>>>> Not that any of these helps you. Just know updating CloudStack >>>>>>>>>to >>>>>>>>> latest gson is not simple. >>>>>>>>> >>>>>>>>> --Alex >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>>>>>>> To: d...@whirr.apache.org >>>>>>>>>> Cc: dev@cloudstack.apache.org >>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and >>>>>>>>>>Whirr >>>>>>>>>> >>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - >>>>>>>>>>just >>>>>>>>>> add >>>>>>>>> the >>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface >>>>>>>>>>(part >>>>>>>>>> of >>>>>>>>> jclouds cli). >>>>>>>>>> >>>>>>>>>> Another option is to upgrade the CloudStack API to use the new >>> version. >>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>>>>>>> >>>>>>>>>>> Dear all, >>>>>>>>>>> >>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency >>>>>>>>> conflicting >>>>>>>>>> issue. >>>>>>>>>>> >>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>>>>>>> gson >>>>>>>>> 1.7.1. >>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>>>>>>> >>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>>>>>>> java.lang.**NoClassDefFoundError: >>>>>>>>>>> com/google/gson/TypeAdapter >>>>>>>>>>> >>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack >>>>>>>>>>>will >>>>>>>>>>> not build successfully due to the following test error. >>>>>>>>>>> >>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>>>>>>> >>>>>>>>>>>------------------------------**------------------------------** >>>>>>>>>>> ------------ >>>>>>>>>>> [INFO] >>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>>>>>>> cloud-core >>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>>>>>>> [INFO] >>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process >>>>>>>>>>>(default) >>>>>>>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using >>>>>>>>>>>'UTF-8' >>>>>>>>>>> encoding to copy filtered resources. >>>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >>>>>>> 3 >>>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>>>maven-compiler-plugin:2.5.1:**compile >>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered >>> resources. >>>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >>>>>>> 3 >>>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 >>>>>>>>>>>source >>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes >>>>>>>>>>>[INFO] >>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>>>>>>> cloud-core >>>>>>>>>>> --- >>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>>>>>>> target/surefire-reports >>>>>>>>>>> >>>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>>> T E S T S >>>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>>>>>>> log4j:WARN No appenders could be found for logger >>>>>>>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>>>>>> log4j:WARN See >>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>>>>>>> http://logging.apa >>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>>>>>>> 3.054 sec <<< FAILURE! >>>>>>>>>>> >>>>>>>>>>> Results : >>>>>>>>>>> >>>>>>>>>>> Failed tests: >>> testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>>>>>>> >>>>>>>>>>> Tests in error: >>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>>>>>>> >>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>>>>>>> CloudStack building process went through. But when I issued an >>>>>>>>>>> API --launchCluster to CloudStack, the following error related >>>>>>>>>>>to >>>>>>>>>>> gson popped up on CloudStack Management Server: >>>>>>>>>>> >>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) >>>>>>>>>>>Caught >>>>>>>>>>> problem with >>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >>>>>>> >>>>>>>hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg >>>>>>>y" >>>>>>>>>>> :"** >>>>>>>>>>> >>>>>>>>>>>HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>>> **LibvirtComputingResource","* >>>>>>>>>>> * >>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>>>>>>> 10.244.18 >>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>>>>>>> >>>>>>>>>>>STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>>>>>>> >>>>>>>>>>>:"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>>>>* >>>>>>>>>>> * >>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>>>>>>> >>>>>>>>>>>:2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc >>>>>>>>>>>e >>>>>>>>>>> ","** >>>>>>>>>>> contextMap":{},"wait":0}}] >>>>>>>>>>> java.lang.ClassCastException: >>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>>>>>>> cannot be cast to java.lang.Class >>>>>>>>>>> at >>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>>> ArrayTypeAdaptor.java:84) >>>>>>>>>>> at >>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>>> ArrayTypeAdaptor.java:37) >>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>>>>>>> TreeTypeAdapter.java:58) >>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>>>>>>> at >>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>>>>>>> java:235) >>>>>>>>>>> at >>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>>>>>>> at >>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>>>>>>> ) >>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>>>>>>> ThreadPoolExecutor.java:1110) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>>>>>>> ThreadPoolExecutor.java:603) >>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>>>>>>> following exception but pushing on >>>>>>>>>>> >>>>>>>>>>> On the CS Managment console I can see the cluster was started >>>>>>>>>>> then quickly died. >>>>>>>>>>> Running on provider cloudstack using identity >>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>>>>>>> ZRGXVO1VApG >>>>>>>>>>> 6q0a >>>>>>>>>>> **K-A-tQRQsZFwnOXQ >>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>>>>>>> [whirr.compute.**BootstrapTemplate] >>>>>>>>>>>(729061754@qtp-385354117-0:) >>>>>>>>>>> Configuring template for >>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>>>>>>> (729061754@qtp-385354117- >>>>>>>>>> 0:) >>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>>>>>>> jobtracker >>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 >>>>>>>>>>>node(s) >>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>>>>>>> >>>>>>>>>>> I attached the debuging log to this email. >>>>>>>>>>> >>>>>>>>>>> The launchCluster API is simple, it calls the >>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>>>>>>> >>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>>>>>>> response.setObjectName("**launchCluster"); >>>>>>>>>>> LaunchClusterCommand command = null; >>>>>>>>>>> try { >>>>>>>>>>> command = new LaunchClusterCommand(); >>>>>>>>>>> } catch (Exception ex) { >>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>>> VER >>>>>>>>>>> E, >>>>>>>>>>> null, ex); >>>>>>>>>>> } >>>>>>>>>>> String[] args = new String[2]; >>>>>>>>>>> args[0] = "--config"; >>>>>>>>>>> args[1] = config; >>>>>>>>>>> >>>>>>>>>>> try { >>>>>>>>>>> command.run(System.in, System.out, System.err, >>>>>>>>>>> Arrays.asList(args)); >>>>>>>>>>> } catch (Exception ex) { >>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>>> VER >>>>>>>>>>> E, >>>>>>>>>>> null, ex); >>>>>>>>>>> } >>>>>>>>>>> response.setResponseName(**getCommandName()); >>>>>>>>>>> output = "successfully launched the cluster."; >>>>>>>>>>> response.setOutPut(output); >>>>>>>>>>> response.setAsync(Boolean.**FALSE); >>>>>>>>>>> this.setResponseObject(**response); >>>>>>>>>>> >>>>>>>>>>> Could someone help me out here? What would be a proper gson >>>>>>>>>>> version for CloudStack and Whirr to run at the same time? >>>>>>>>>>> >>>>>>>>>>> Thanks loads. >>>>>>>>>>> >>>>>>>>>>> Best Regards, >>>>>>>>>>> Meng >>>>>>>> >>>>>>>> >>>> >>> >>> >
