Yep I’d like to see the non changing core structs version, in a patch to evaluate it. You said you know how, so let’s see it and then go from there. We can reserve 0.9 to consider changing structs if the code becomes too unwieldily to maintain.
Good? Cheers, Chris ------------------------ Chris Mattmann [email protected] -----Original Message----- From: Michael Starch <[email protected]> Reply-To: <[email protected]> Date: Friday, November 7, 2014 at 9:21 AM To: <[email protected]> Subject: Re: Extra Compiler Tools >Chris, > >Did you ever come to a conclusion on this? > >-Michael > >On Wed, Nov 5, 2014 at 7:14 PM, Chris Mattmann <[email protected]> >wrote: > >> OK, let me think about this tonight. >> Maybe we can figure this out tomorrow, >> I won’t hold this up longer than that. >> >> ------------------------ >> Chris Mattmann >> [email protected] >> >> >> >> >> -----Original Message----- >> From: Michael Starch <[email protected]> >> Reply-To: <[email protected]> >> Date: Wednesday, November 5, 2014 at 8:23 PM >> To: <[email protected]> >> Subject: Re: Extra Compiler Tools >> >> >According to the specs, any subclass that implements Serializable must >> >manually implement the serialization of the parents' members. I tested >> >this and it fails exactly as expected. The parent's members aren't >> >serialized. >> > >> >Also, JobInput is an interface.... so I would have no way of catching >>all >> >of the possible implementations that could come at me. >> > >> >Michael >> >On Nov 5, 2014 7:06 PM, "Mattmann, Chris A (3980)" < >> >[email protected]> wrote: >> > >> >> Got it, Mike. >> >> >> >> Hmm, how about simply creating SerializableJobSpec and >> >> SerializableJob and SerializableJobInput and then making >> >> them sub-class their parents and implement Serializable. >> >> Then, use these classes in your Mesos implementation. >> >> That seems self-contained, doesn’t change core classes, >> >> and pretty easy, right? >> >> >> >> Cheers, >> >> Chris >> >> >> >> >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> Chris Mattmann, Ph.D. >> >> Chief Architect >> >> Instrument Software and Science Data Systems Section (398) >> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> >> Office: 168-519, Mailstop: 168-527 >> >> Email: [email protected] >> >> WWW: http://sunset.usc.edu/~mattmann/ >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> Adjunct Associate Professor, Computer Science Department >> >> University of Southern California, Los Angeles, CA 90089 USA >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> >> >> >> >> >> >> >> >> >> >> -----Original Message----- >> >> From: Michael Starch <[email protected]> >> >> Reply-To: "[email protected]" <[email protected]> >> >> Date: Wednesday, November 5, 2014 at 8:00 PM >> >> To: "[email protected]" <[email protected]> >> >> Subject: Re: Extra Compiler Tools >> >> >> >> >I need to serialize a JobSpec and children (Job and JobInput) to a >> >>byte[]. >> >> >Java can do this automatically by marking all three as Serializable. >> >> > >> >> >The work around is to manually serialize to a private inner struct >>and >> >> >back >> >> >out again. The inner class will have members for each member in the >> >> >JobSpec and children. Java can the auto-serialize that without >> >>changing >> >> >the other three. >> >> > >> >> >It is ugly, and essentially a reimplementation of those three >> >> >classes....but it is entirely self-contained. >> >> > >> >> >Michael >> >> >On Nov 5, 2014 6:45 PM, "Chris Mattmann" <[email protected]> >> >> wrote: >> >> > >> >> >> Hey Mike, >> >> >> >> >> >> Hmm, what’s the work around just so I know >> >> >> what we’re trading against? >> >> >> >> >> >> Cheers, >> >> >> Chris >> >> >> >> >> >> ------------------------ >> >> >> Chris Mattmann >> >> >> [email protected] >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -----Original Message----- >> >> >> From: Michael Starch <[email protected]> >> >> >> Reply-To: <[email protected]> >> >> >> Date: Wednesday, November 5, 2014 at 6:31 PM >> >> >> To: <[email protected]> >> >> >> Subject: Re: Extra Compiler Tools >> >> >> >> >> >> >That is basically what I did. Regardless, protobuff proves to be >> >> >>overkill. >> >> >> > >> >> >> >If I mark those classes as serializable, the correct solution is >>2 >> >> >>lines >> >> >> >of >> >> >> >code. (protobuff was like 20). Wrote a test case, and it works >> >> >> >perfectly. >> >> >> > >> >> >> >If I cannot make JobSpec Job and JonInput implement Serializable >> >>then >> >> >>the >> >> >> >work around is simple too. >> >> >> > >> >> >> >What do you think? Should I mark them as Serializable, or use a >> >> >> >work-around. Either is a better solution than protobuff. >> >> >> > >> >> >> >Michael >> >> >> >On Nov 5, 2014 4:44 PM, "Chris Mattmann" >><[email protected]> >> >> >> wrote: >> >> >> > >> >> >> >> Mike, have you looked at this yet? >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>http://techtraits.com/build%20management/maven/2011/09/09/compiling-proto >> >> >> >>co >> >> >> >> l-buffers-from-maven/ >> >> >> >> >> >> >> >> >> >> >> >> I’m going to play with it tonight and see if >> >> >> >> I can help here. Do you have some files I can test >> >> >> >> with? Can you attach them to JIRA or dropbox them to me >> >> >> >> so I can scope? >> >> >> >> >> >> >> >> Cheers, >> >> >> >> Chris >> >> >> >> >> >> >> >> ------------------------ >> >> >> >> Chris Mattmann >> >> >> >> [email protected] >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -----Original Message----- >> >> >> >> From: Michael Starch <[email protected]> >> >> >> >> Reply-To: <[email protected]> >> >> >> >> Date: Wednesday, November 5, 2014 at 5:37 PM >> >> >> >> To: <[email protected]> >> >> >> >> Subject: Re: Extra Compiler Tools >> >> >> >> >> >> >> >> >Ok....time for an audible. Protoc needs to be built from >> >>source, no >> >> >> >> >binary >> >> >> >> >distributions available. Thus I am going to purge >>proto-buffers >> >> >>from >> >> >> >>the >> >> >> >> >new code and be done with it. >> >> >> >> > >> >> >> >> >Any problem making the following classes/interfaces implement >> >> >> >> >java.io.Serializable: >> >> >> >> > >> >> >> >> >JobSpec >> >> >> >> >Job >> >> >> >> >JobInput >> >> >> >> > >> >> >> >> >Doing so would allow apache and native java serialization and >> >>thus >> >> >>we >> >> >> >> >wouldn't need something like proto-buffers. >> >> >> >> > >> >> >> >> >-Michael >> >> >> >> >Thanks Mike +1 >> >> >> >> > >> >> >> >> >------------------------ >> >> >> >> >Chris Mattmann >> >> >> >> >[email protected] >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> >-----Original Message----- >> >> >> >> >From: Michael Starch <[email protected]> >> >> >> >> >Reply-To: <[email protected]> >> >> >> >> >Date: Wednesday, November 5, 2014 at 12:31 PM >> >> >> >> >To: <[email protected]> >> >> >> >> >Subject: Re: Extra Compiler Tools >> >> >> >> > >> >> >> >> >>Looks like you followed the same reasoning chain that I did. >> >>Yes, >> >> >>I >> >> >> >>came >> >> >> >> >>to the same conclusion that ant-build was best. >> >> >> >> >> >> >> >> >> >>I wasn't sure how to download protoc, but you just answered >> >> >> >>that....so I >> >> >> >> >>think this is a great solution! >> >> >> >> >> >> >> >> >> >>Thanks, >> >> >> >> >> >> >> >> >> >>Michael >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>On Wed, Nov 5, 2014 at 10:23 AM, Chris Mattmann >> >> >> >> >><[email protected]> >> >> >> >> >>wrote: >> >> >> >> >> >> >> >> >> >>> Hi Mike, >> >> >> >> >>> >> >> >> >> >>> Thanks for flushing this out. >> >> >> >> >>> >> >> >> >> >>> My thoughts on the below: >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> -----Original Message----- >> >> >> >> >>> From: Michael Starch <[email protected]> >> >> >> >> >>> Reply-To: <[email protected]> >> >> >> >> >>> Date: Wednesday, November 5, 2014 at 12:12 PM >> >> >> >> >>> To: <[email protected]> >> >> >> >> >>> Subject: Re: Extra Compiler Tools >> >> >> >> >>> >> >> >> >> >>> >I tried this approach. The plugin requires a path to the >> >> >>"protoc" >> >> >> >>tool >> >> >> >> >>>and >> >> >> >> >>> >thus a working installation. This is what prompted the >> >> >>discussion. >> >> >> >> >>> >> >> >> >> >>> Ah - no worries, what you could do is: >> >> >> >> >>> >> >> >> >> >>> 1. only enable to plugin if -Pwith-mesos is enabled; and >> >> >> >> >>> >> >> >> >> >>> > >> >> >> >> >>> >Running the plugin under a profile works. >> >> >> >> >>> >> >> >> >> >>> Yep. >> >> >> >> >>> >> >> >> >> >>> > However, not running the plugin >> >> >> >> >>> >causes compile errors in dependant code. Excluding this >>code >> >> >> >>except >> >> >> >> >>> >within >> >> >> >> >>> >the profile doesn't seem to work, and is considered by >>some >> >>to >> >> >>be >> >> >> >>bad >> >> >> >> >>>form >> >> >> >> >>> >because there is nothing inside the jar file that notes >>which >> >> >> >>profiles >> >> >> >> >>> >were >> >> >> >> >>> >used to compile. >> >> >> >> >>> >> >> >> >> >>> Got it. Suggestion here would be: >> >> >> >> >>> >> >> >> >> >>> 2. create a new module, cas-resource-mesos, and inside of >>that >> >> >> >>module, >> >> >> >> >>> take one of the following approaches, assuming the module >>is >> >> >> >>activated >> >> >> >> >>> when -Pwith-mesos is enabled: >> >> >> >> >>> >> >> >> >> >>> 2a. Maven Antrun like so (in this old example): >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >> >> >> >> >> >> http://stackoverflow.com/questions/1578456/integrate-protocol-buffers-in >> >> >> >> >>>t >> >> >> >> >>>o- >> >> >> >> >>> maven2-build >> >> >> >> >>> >> >> >> >> >>> (pro: more flexibility in case protoc isn¹t there; to fail >>on >> >> >> >>error; to >> >> >> >> >>> only compile if >> >> >> >> >>> protoc is available >> >> >> >> >>> >> >> >> >> >>> 2b. Maven protobuf plugin >> >> >> >> >>> >>http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html >> >> >> >> >>> >> >> >> >> >>> Here¹s how to enable a module with a profile: >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >> >> >> >> >> >> http://blog.soebes.de/blog/2013/11/09/why-is-it-bad-to-activate-slash-de >> >> >> >> >>>a >> >> >> >> >>>ct >> >> >> >> >>> ive-modules-by-profiles-in-maven/ >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> It seems like that is a bad idea though, based on that >> >> >>discussion. >> >> >> >> >>> >> >> >> >> >>> So, here¹s another option: >> >> >> >> >>> >> >> >> >> >>> 1. Inside of cas-resource (no special new module or >>anything >> >> >>else) >> >> >> >> >>> 2. include some custom Ant magic via a build.xml file and >>the >> >> >>Maven >> >> >> >> >>> AntRun plugin: >> >> >> >> >>> 2a. test if protoc is on the system path, and if not, >> >>download >> >> >>it, >> >> >> >> >>>e.g., >> >> >> >> >>> into the target directory (gets deleted on clean) >> >> >> >> >>> 2b. call protoc and compile after 2a >> >> >> >> >>> >> >> >> >> >>> I would suggest this solution as I think it¹s the most >>robust >> >>and >> >> >> >> >>>ensures >> >> >> >> >>> we always have a cas-resource that includes mesos and >>compiled >> >> >> >> >>>correctly. >> >> >> >> >>> >> >> >> >> >>> Cheers, >> >> >> >> >>> Chris >> >> >> >> >>> >> >> >> >> >>> > >> >> >> >> >>> >Any ideas on how to continue? >> >> >> >> >>> > >> >> >> >> >>> >Michael >> >> >> >> >>> > On Nov 5, 2014 11:04 AM, "Chris Mattmann" >> >> >> >><[email protected]> >> >> >> >> >>> >wrote: >> >> >> >> >>> > >> >> >> >> >>> >> Hi Mike, >> >> >> >> >>> >> >> >> >> >> >>> >> Great discussion. It would be nice if there was >> >> >> >> >>> >> a protoc Maven plugin: >> >> >> >> >>> >> >> >> >> >> >>> >> >> >>http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> Looks like there is. My suggestion: >> >> >> >> >>> >> >> >> >> >> >>> >> 1. use a Profile, something like -Pwith-mesos and >> >> >> >> >>> >> then when activated; >> >> >> >> >>> >> 2. call the above plugin if -Pwith-mesos is activated >> >> >> >> >>> >> in the resource manager >> >> >> >> >>> >> >> >> >> >> >>> >> Sound good? >> >> >> >> >>> >> >> >> >> >> >>> >> Cheers, >> >> >> >> >>> >> Chris >> >> >> >> >>> >> >> >> >> >> >>> >> ------------------------ >> >> >> >> >>> >> Chris Mattmann >> >> >> >> >>> >> [email protected] >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> -----Original Message----- >> >> >> >> >>> >> From: Michael Starch <[email protected]> >> >> >> >> >>> >> Reply-To: <[email protected]> >> >> >> >> >>> >> Date: Wednesday, November 5, 2014 at 11:46 AM >> >> >> >> >>> >> To: <[email protected]> >> >> >> >> >>> >> Subject: Extra Compiler Tools >> >> >> >> >>> >> >> >> >> >> >>> >> >All, >> >> >> >> >>> >> > >> >> >> >> >>> >> >I am trying to integrate apache-mesos with our resource >> >> >>manager. >> >> >> >> >>> >>However, >> >> >> >> >>> >> >mesos uses a technology called "protobuff" from Google >>for >> >> >> >> >>> >> >marshaling/unmarshaling data. >> >> >> >> >>> >> > >> >> >> >> >>> >> >This requires running a tool called "protoc" to >>generate a >> >> >> >>source >> >> >> >> >>>file >> >> >> >> >>> >>in >> >> >> >> >>> >> >java. What is the best way to integrate this step into >> >>our >> >> >> >>build >> >> >> >> >>> >>process? >> >> >> >> >>> >> > >> >> >> >> >>> >> >Options I can conceive of: >> >> >> >> >>> >> > -Check in generated java file >> >> >> >> >>> >> > -Require "protoc" installation to build resource >> >>manager >> >> >> >> >>> >> > -Separate extra resource package into new module >> >> >> >> >>> >> > >> >> >> >> >>> >> >None of these ideas are very clean. >> >> >> >> >>> >> > >> >> >> >> >>> >> >Any other ideas? I tried setting up a profile to only >> >> >>compile >> >> >> >> >>>these >> >> >> >> >>> >> >sources when selected, but that turned out not to work. >> >> >> >> >>> >> > >> >> >> >> >>> >> >-Michael Starch >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>
