Okay, I officially throw in the towel on this one. I made really good progress on getting jackson into place, but after I was about 75% done I just felt it was WAY too big of a change to do at once. If we want to move to jackson, I would say the best approach would be more incremental and not big bang style. So gson is here to stay for awhile. Anybody else want to try upgrading to latest gson?
Darren On Mon, Sep 23, 2013 at 2:00 PM, Sebastien Goasguen <run...@gmail.com> wrote: > > On Sep 23, 2013, at 2:54 PM, Chiradeep Vittal <chiradeep.vit...@citrix.com> > wrote: > >> I still think this is the wrong way to implement the Whirr integration. > > Probably, but it's too late now. > > We can just get it in a branch and look at it > >> >> On 9/23/13 6:45 AM, "sebgoa" <run...@gmail.com> wrote: >> >>> >>> On Sep 23, 2013, at 7:12 AM, Darren Shepherd >>> <darren.s.sheph...@gmail.com> wrote: >>> >>>> The only way this will get committed is if we either move to latest >>>> gson or >>>> jackson. Regardless, the outcome will be that Meng can assume this gson >>>> conflict won't happen. The problem is just how fast can we move all of >>>> ACS >>>> to a new json library. Is it possible for Meng to commit to a branch >>>> that >>>> just blindly updates gson to the latest (in /pom.xml). >>> >>> Yes I will talk to Meng about this. thanks >>> >>>> When we get ACS on >>>> a newer json library, we can merge in that branch. Tomorrow (9/23), I >>>> can >>>> attempt to put together a patch to move to jackson. I think I ironed >>>> out >>>> most of the technical issues so I just have to globally apply the >>>> changes. >>>> >>>> Darren >>>> >>>> >>>> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen >>>> <run...@gmail.com>wrote: >>>> >>>>> >>>>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <h...@trippaers.nl> wrote: >>>>> >>>>>> >>>>>> The Nicira NVP plugin is also depending on gson. If we make any >>>>>> changes >>>>> we need to validate against their api to ensure that the interface >>>>> still >>>>> works. >>>>>> >>>>>> If we put the changes in a separate branch i'd be happy to run the >>>>> changes against the api. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Hugo >>>>> >>>>> So how can Meng get out of this bind, she is trying to complete her >>>>> google >>>>> summer of code project. >>>>> >>>>> thanks, >>>>> >>>>> -Sebastien >>>>> >>>>>> >>>>>> >>>>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd < >>>>> darren.s.sheph...@gmail.com> wrote: >>>>>> >>>>>>> After much searching I found the two lines of code I needed to get it >>>>> to serialize right. I'll keep looking and see if I find other snags. >>>>> Maybe we can just move to jackson.... we'll see, haven't tried to >>>>> deserialize yet. >>>>>>> >>>>>>> Darren >>>>>>> >>>>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <alex.hu...@citrix.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Darren, >>>>>>>> >>>>>>>> When I looked into updating to the latest gson, the problem, IIRC, >>>>>>>> is >>>>> things that weren't considered cyclical dependencies are suddenly >>>>> considered cyclical with the latest. I don't recall where though. >>>>>>>> >>>>>>>> --Alex >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >>>>>>>>> Sent: Thursday, September 19, 2013 1:33 PM >>>>>>>>> To: dev@cloudstack.apache.org >>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr >>>>>>>>> >>>>>>>>> Alright, I looked into this and it will take a bit more work to >>>>> switch to Jackson. >>>>>>>>> The snag I hit was how we serialize commands for the agents. We >>>>>>>>> use a >>>>>>>>> customer typeadatper in gson to produce a format like { >>>>>>>>> "org.MyClass" >>>>>>>>> : {...} }. In jackson I don't see producing a similar format as >>>>> being so straight >>>>>>>>> forward. I'm sure there's an elegant solution, but I didn't >>>>> immediately find it. >>>>>>>>> The problem is if you use a custom serializer in jackson and do >>>>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a >>>>> loop >>>>>>>>> because it will call your serializer again. So if somebody can >>>>> figure out how to >>>>>>>>> do this format easily in jackson, the rest looks simple enough. >>>>>>>>> >>>>>>>>> So, being that it is not an obvious switch to using jackson, what >>>>>>>>> is >>>>> the impact >>>>>>>>> of moving to the latest gson? What were the issue encountered when >>>>>>>>> people tried it before? >>>>>>>>> >>>>>>>>> Darren >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd < >>>>>>>>> darren.s.sheph...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> I can do some analysis on this. I'm always up for a terribly >>>>>>>>>> painful >>>>>>>>>> refactor :) >>>>>>>>>> >>>>>>>>>> Darren >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang >>>>>>>>>>> <alex.hu...@citrix.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I actually did a quick try to update cloudstack to use newest >>>>>>>>>>> gson >>>>>>>>>>> version about 3 months back. Had to roll it back but I didn't >>>>>>>>>>> try >>>>>>>>>>> very hard though. Part of the reason why I decided to rollback >>>>>>>>>>> is >>>>>>>>>>> due to gson is used differently by various components in >>>>>>>>>>> CloudStack >>>>>>>>>>> and I didn't have time to get into each component to see if I >>>>>>>>>>> break >>>>>>>>>>> anything. The other is also I thought using Jackson would be >>>>>>>>>>> much >>>>>>>>>>> better and thought our next attempt should be with Jackson. >>>>>>>>>>> >>>>>>>>>>> Not that any of these helps you. Just know updating CloudStack >>>>>>>>>>> to >>>>>>>>>>> latest gson is not simple. >>>>>>>>>>> >>>>>>>>>>> --Alex >>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Andrei Savu [mailto:savu.and...@gmail.com] >>>>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM >>>>>>>>>>>> To: d...@whirr.apache.org >>>>>>>>>>>> Cc: dev@cloudstack.apache.org >>>>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and >>>>>>>>>>>> Whirr >>>>>>>>>>>> >>>>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - >>>>>>>>>>>> just >>>>>>>>>>>> add >>>>>>>>>>> the >>>>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface >>>>>>>>>>>> (part >>>>>>>>>>>> of >>>>>>>>>>> jclouds cli). >>>>>>>>>>>> >>>>>>>>>>>> Another option is to upgrade the CloudStack API to use the new >>>>> version. >>>>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <meng...@ufl.edu> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Dear all, >>>>>>>>>>>>> >>>>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch >>>>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency >>>>>>>>>>> conflicting >>>>>>>>>>>> issue. >>>>>>>>>>>>> >>>>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires >>>>>>>>>>>>> gson >>>>>>>>>>> 1.7.1. >>>>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen: >>>>>>>>>>>>> >>>>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError: >>>>>>>>>>>> java.lang.**NoClassDefFoundError: >>>>>>>>>>>>> com/google/gson/TypeAdapter >>>>>>>>>>>>> >>>>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar. >>>>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack >>>>>>>>>>>>> will >>>>>>>>>>>>> not build successfully due to the following test error. >>>>>>>>>>>>> >>>>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO] >>>>>>>>>>>>> >>>>>>>>>>>>> ------------------------------**------------------------------** >>>>>>>>>>>>> ------------ >>>>>>>>>>>>> [INFO] >>>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ >>>>>>>>>>>>> cloud-core >>>>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target >>>>>>>>>>>>> [INFO] >>>>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process >>>>>>>>>>>>> (default) >>>>>>>>>>>>> @ cloud-core --- [INFO] [INFO] --- >>>>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @ >>>>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using >>>>>>>>>>>>> 'UTF-8' >>>>>>>>>>>>> encoding to copy filtered resources. >>>>>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying >>>>>>>>> 3 >>>>>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**compile >>>>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source >>>>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO] >>>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources >>>>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute >>>>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered >>>>> resources. >>>>>>>>>>>>> [INFO] skip non existing resourceDirectory >>>>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying >>>>>>>>> 3 >>>>>>>>>>>>> resources [INFO] [INFO] --- >>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile >>>>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 >>>>>>>>>>>>> source >>>>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes >>>>>>>>>>>>> [INFO] >>>>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ >>>>>>>>>>>>> cloud-core >>>>>>>>>>>>> --- >>>>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/** >>>>>>>>>>>>> target/surefire-reports >>>>>>>>>>>>> >>>>>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>>>>> T E S T S >>>>>>>>>>>>> ------------------------------**------------------------- >>>>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest >>>>>>>>>>>>> log4j:WARN No appenders could be found for logger >>>>>>>>>>>>> (com.cloud.agent.transport.**RequestTest). >>>>>>>>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>>>>>>>> log4j:WARN See >>>>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig< >>>>>>>>>>> http://logging.apa >>>>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info. >>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: >>>>>>>>>>>>> 3.054 sec <<< FAILURE! >>>>>>>>>>>>> >>>>>>>>>>>>> Results : >>>>>>>>>>>>> >>>>>>>>>>>>> Failed tests: >>>>> testSerDeser(com.cloud.agent.**transport.RequestTest) >>>>>>>>>>>>> >>>>>>>>>>>>> Tests in error: >>>>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest) >>>>>>>>>>>>> >>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the >>>>>>>>>>>>> CloudStack building process went through. But when I issued an >>>>>>>>>>>>> API --launchCluster to CloudStack, the following error related >>>>>>>>>>>>> to >>>>>>>>>>>>> gson popped up on CloudStack Management Server: >>>>>>>>>>>>> >>>>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) >>>>>>>>>>>>> Caught >>>>>>>>>>>>> problem with >>>>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":** >>>>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"** >>>>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"** >>>>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,** >>>>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","** >>>>>>>>> >>>>>>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg >>>>>>>>> y" >>>>>>>>>>>>> :"** >>>>>>>>>>>>> >>>>>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.** >>>>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"** >>>>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":** >>>>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>>>>> **LibvirtComputingResource","* >>>>>>>>>>>>> * >>>>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","** >>>>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.** >>>>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","** >>>>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:** >>>>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","** >>>>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.** >>>>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"** >>>>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"! >>>>>>>>>>>>> 10.244.18 >>>>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"** >>>>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-** >>>>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/** >>>>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/** >>>>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":** >>>>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"** >>>>>>>>>>>>> >>>>>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"** >>>>>>>>>>>>> >>>>>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986- >>>>>>>>>>>>> * >>>>>>>>>>>>> * >>>>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"** >>>>>>>>>>>>> >>>>>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc >>>>>>>>>>>>> e >>>>>>>>>>>>> ","** >>>>>>>>>>>>> contextMap":{},"wait":0}}] >>>>>>>>>>>>> java.lang.ClassCastException: >>>>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl >>>>>>>>>>>>> cannot be cast to java.lang.Class >>>>>>>>>>>>> at >>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>>>>> ArrayTypeAdaptor.java:84) >>>>>>>>>>>>> at >>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(** >>>>>>>>>>>>> ArrayTypeAdaptor.java:37) >>>>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(** >>>>>>>>>>>>> TreeTypeAdapter.java:58) >>>>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795) >>>>>>>>>>>>> at >>>>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.** >>>>>>>>>>>>> java:235) >>>>>>>>>>>>> at >>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221) >>>>>>>>>>>>> at >>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.** >>>>>>>>>>>>> doTask(AgentManagerImpl.java:**1374) >>>>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$** >>>>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659 >>>>>>>>>>>> ) >>>>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83) >>>>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(** >>>>>>>>>>>>> ThreadPoolExecutor.java:1110) >>>>>>>>>>>>> at >>>>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(** >>>>>>>>>>>>> ThreadPoolExecutor.java:603) >>>>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722) >>>>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the >>>>>>>>>>>>> following exception but pushing on >>>>>>>>>>>>> >>>>>>>>>>>>> On the CS Managment console I can see the cluster was started >>>>>>>>>>>>> then quickly died. >>>>>>>>>>>>> Running on provider cloudstack using identity >>>>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a >>>>>>>>>>>> ZRGXVO1VApG >>>>>>>>>>>>> 6q0a >>>>>>>>>>>>> **K-A-tQRQsZFwnOXQ >>>>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction] >>>>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO >>>>>>>>>>>>> [whirr.compute.**BootstrapTemplate] >>>>>>>>>>>>> (729061754@qtp-385354117-0:) >>>>>>>>>>>>> Configuring template for >>>>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker >>>>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate] >>>>>>>>>>>>> (729061754@qtp-385354117- >>>>>>>>>>>> 0:) >>>>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop- >>>>>>>>>>>> jobtracker >>>>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 >>>>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO >>>>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 >>>>>>>>>>>>> node(s) >>>>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ... >>>>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction] >>>>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed >>>>>>>>>>>>> >>>>>>>>>>>>> I attached the debuging log to this email. >>>>>>>>>>>>> >>>>>>>>>>>>> The launchCluster API is simple, it calls the >>>>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster. >>>>>>>>>>>>> >>>>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse(); >>>>>>>>>>>>> response.setObjectName("**launchCluster"); >>>>>>>>>>>>> LaunchClusterCommand command = null; >>>>>>>>>>>>> try { >>>>>>>>>>>>> command = new LaunchClusterCommand(); >>>>>>>>>>>>> } catch (Exception ex) { >>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>>>>> VER >>>>>>>>>>>>> E, >>>>>>>>>>>>> null, ex); >>>>>>>>>>>>> } >>>>>>>>>>>>> String[] args = new String[2]; >>>>>>>>>>>>> args[0] = "--config"; >>>>>>>>>>>>> args[1] = config; >>>>>>>>>>>>> >>>>>>>>>>>>> try { >>>>>>>>>>>>> command.run(System.in, System.out, System.err, >>>>>>>>>>>>> Arrays.asList(args)); >>>>>>>>>>>>> } catch (Exception ex) { >>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE >>>>>>>>>>>> VER >>>>>>>>>>>>> E, >>>>>>>>>>>>> null, ex); >>>>>>>>>>>>> } >>>>>>>>>>>>> response.setResponseName(**getCommandName()); >>>>>>>>>>>>> output = "successfully launched the cluster."; >>>>>>>>>>>>> response.setOutPut(output); >>>>>>>>>>>>> response.setAsync(Boolean.**FALSE); >>>>>>>>>>>>> this.setResponseObject(**response); >>>>>>>>>>>>> >>>>>>>>>>>>> Could someone help me out here? What would be a proper gson >>>>>>>>>>>>> version for CloudStack and Whirr to run at the same time? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks loads. >>>>>>>>>>>>> >>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>> Meng >>>>>>>>>> >>>>>>>>>> >>>>>> >>>>> >>>>> >>> >> >
