Sound fine, expect to see this on review board when I get it done and passing my test.
Michael On Nov 7, 2014 9:44 AM, "Chris Mattmann" <[email protected]> wrote: > Yep I’d like to see the non changing > core structs version, in a patch to > evaluate it. You said you know how, so let’s see it > and then go from there. We can reserve 0.9 to > consider changing structs if the code becomes > too unwieldily to maintain. > > Good? > > Cheers, > Chris > > ------------------------ > Chris Mattmann > [email protected] > > > > > -----Original Message----- > From: Michael Starch <[email protected]> > Reply-To: <[email protected]> > Date: Friday, November 7, 2014 at 9:21 AM > To: <[email protected]> > Subject: Re: Extra Compiler Tools > > >Chris, > > > >Did you ever come to a conclusion on this? > > > >-Michael > > > >On Wed, Nov 5, 2014 at 7:14 PM, Chris Mattmann <[email protected]> > >wrote: > > > >> OK, let me think about this tonight. > >> Maybe we can figure this out tomorrow, > >> I won’t hold this up longer than that. > >> > >> ------------------------ > >> Chris Mattmann > >> [email protected] > >> > >> > >> > >> > >> -----Original Message----- > >> From: Michael Starch <[email protected]> > >> Reply-To: <[email protected]> > >> Date: Wednesday, November 5, 2014 at 8:23 PM > >> To: <[email protected]> > >> Subject: Re: Extra Compiler Tools > >> > >> >According to the specs, any subclass that implements Serializable must > >> >manually implement the serialization of the parents' members. I tested > >> >this and it fails exactly as expected. The parent's members aren't > >> >serialized. > >> > > >> >Also, JobInput is an interface.... so I would have no way of catching > >>all > >> >of the possible implementations that could come at me. > >> > > >> >Michael > >> >On Nov 5, 2014 7:06 PM, "Mattmann, Chris A (3980)" < > >> >[email protected]> wrote: > >> > > >> >> Got it, Mike. > >> >> > >> >> Hmm, how about simply creating SerializableJobSpec and > >> >> SerializableJob and SerializableJobInput and then making > >> >> them sub-class their parents and implement Serializable. > >> >> Then, use these classes in your Mesos implementation. > >> >> That seems self-contained, doesn’t change core classes, > >> >> and pretty easy, right? > >> >> > >> >> Cheers, > >> >> Chris > >> >> > >> >> > >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> >> Chris Mattmann, Ph.D. > >> >> Chief Architect > >> >> Instrument Software and Science Data Systems Section (398) > >> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >> >> Office: 168-519, Mailstop: 168-527 > >> >> Email: [email protected] > >> >> WWW: http://sunset.usc.edu/~mattmann/ > >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> >> Adjunct Associate Professor, Computer Science Department > >> >> University of Southern California, Los Angeles, CA 90089 USA > >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> -----Original Message----- > >> >> From: Michael Starch <[email protected]> > >> >> Reply-To: "[email protected]" <[email protected]> > >> >> Date: Wednesday, November 5, 2014 at 8:00 PM > >> >> To: "[email protected]" <[email protected]> > >> >> Subject: Re: Extra Compiler Tools > >> >> > >> >> >I need to serialize a JobSpec and children (Job and JobInput) to a > >> >>byte[]. > >> >> >Java can do this automatically by marking all three as Serializable. > >> >> > > >> >> >The work around is to manually serialize to a private inner struct > >>and > >> >> >back > >> >> >out again. The inner class will have members for each member in the > >> >> >JobSpec and children. Java can the auto-serialize that without > >> >>changing > >> >> >the other three. > >> >> > > >> >> >It is ugly, and essentially a reimplementation of those three > >> >> >classes....but it is entirely self-contained. > >> >> > > >> >> >Michael > >> >> >On Nov 5, 2014 6:45 PM, "Chris Mattmann" <[email protected]> > >> >> wrote: > >> >> > > >> >> >> Hey Mike, > >> >> >> > >> >> >> Hmm, what’s the work around just so I know > >> >> >> what we’re trading against? > >> >> >> > >> >> >> Cheers, > >> >> >> Chris > >> >> >> > >> >> >> ------------------------ > >> >> >> Chris Mattmann > >> >> >> [email protected] > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> -----Original Message----- > >> >> >> From: Michael Starch <[email protected]> > >> >> >> Reply-To: <[email protected]> > >> >> >> Date: Wednesday, November 5, 2014 at 6:31 PM > >> >> >> To: <[email protected]> > >> >> >> Subject: Re: Extra Compiler Tools > >> >> >> > >> >> >> >That is basically what I did. Regardless, protobuff proves to be > >> >> >>overkill. > >> >> >> > > >> >> >> >If I mark those classes as serializable, the correct solution is > >>2 > >> >> >>lines > >> >> >> >of > >> >> >> >code. (protobuff was like 20). Wrote a test case, and it works > >> >> >> >perfectly. > >> >> >> > > >> >> >> >If I cannot make JobSpec Job and JonInput implement Serializable > >> >>then > >> >> >>the > >> >> >> >work around is simple too. > >> >> >> > > >> >> >> >What do you think? Should I mark them as Serializable, or use a > >> >> >> >work-around. Either is a better solution than protobuff. > >> >> >> > > >> >> >> >Michael > >> >> >> >On Nov 5, 2014 4:44 PM, "Chris Mattmann" > >><[email protected]> > >> >> >> wrote: > >> >> >> > > >> >> >> >> Mike, have you looked at this yet? > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> >> > >> >> > >> >> > >> > >> > http://techtraits.com/build%20management/maven/2011/09/09/compiling-proto > >> >> >> >>co > >> >> >> >> l-buffers-from-maven/ > >> >> >> >> > >> >> >> >> > >> >> >> >> I’m going to play with it tonight and see if > >> >> >> >> I can help here. Do you have some files I can test > >> >> >> >> with? Can you attach them to JIRA or dropbox them to me > >> >> >> >> so I can scope? > >> >> >> >> > >> >> >> >> Cheers, > >> >> >> >> Chris > >> >> >> >> > >> >> >> >> ------------------------ > >> >> >> >> Chris Mattmann > >> >> >> >> [email protected] > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> -----Original Message----- > >> >> >> >> From: Michael Starch <[email protected]> > >> >> >> >> Reply-To: <[email protected]> > >> >> >> >> Date: Wednesday, November 5, 2014 at 5:37 PM > >> >> >> >> To: <[email protected]> > >> >> >> >> Subject: Re: Extra Compiler Tools > >> >> >> >> > >> >> >> >> >Ok....time for an audible. Protoc needs to be built from > >> >>source, no > >> >> >> >> >binary > >> >> >> >> >distributions available. Thus I am going to purge > >>proto-buffers > >> >> >>from > >> >> >> >>the > >> >> >> >> >new code and be done with it. > >> >> >> >> > > >> >> >> >> >Any problem making the following classes/interfaces implement > >> >> >> >> >java.io.Serializable: > >> >> >> >> > > >> >> >> >> >JobSpec > >> >> >> >> >Job > >> >> >> >> >JobInput > >> >> >> >> > > >> >> >> >> >Doing so would allow apache and native java serialization and > >> >>thus > >> >> >>we > >> >> >> >> >wouldn't need something like proto-buffers. > >> >> >> >> > > >> >> >> >> >-Michael > >> >> >> >> >Thanks Mike +1 > >> >> >> >> > > >> >> >> >> >------------------------ > >> >> >> >> >Chris Mattmann > >> >> >> >> >[email protected] > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> >-----Original Message----- > >> >> >> >> >From: Michael Starch <[email protected]> > >> >> >> >> >Reply-To: <[email protected]> > >> >> >> >> >Date: Wednesday, November 5, 2014 at 12:31 PM > >> >> >> >> >To: <[email protected]> > >> >> >> >> >Subject: Re: Extra Compiler Tools > >> >> >> >> > > >> >> >> >> >>Looks like you followed the same reasoning chain that I did. > >> >>Yes, > >> >> >>I > >> >> >> >>came > >> >> >> >> >>to the same conclusion that ant-build was best. > >> >> >> >> >> > >> >> >> >> >>I wasn't sure how to download protoc, but you just answered > >> >> >> >>that....so I > >> >> >> >> >>think this is a great solution! > >> >> >> >> >> > >> >> >> >> >>Thanks, > >> >> >> >> >> > >> >> >> >> >>Michael > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >>On Wed, Nov 5, 2014 at 10:23 AM, Chris Mattmann > >> >> >> >> >><[email protected]> > >> >> >> >> >>wrote: > >> >> >> >> >> > >> >> >> >> >>> Hi Mike, > >> >> >> >> >>> > >> >> >> >> >>> Thanks for flushing this out. > >> >> >> >> >>> > >> >> >> >> >>> My thoughts on the below: > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> >>> -----Original Message----- > >> >> >> >> >>> From: Michael Starch <[email protected]> > >> >> >> >> >>> Reply-To: <[email protected]> > >> >> >> >> >>> Date: Wednesday, November 5, 2014 at 12:12 PM > >> >> >> >> >>> To: <[email protected]> > >> >> >> >> >>> Subject: Re: Extra Compiler Tools > >> >> >> >> >>> > >> >> >> >> >>> >I tried this approach. The plugin requires a path to the > >> >> >>"protoc" > >> >> >> >>tool > >> >> >> >> >>>and > >> >> >> >> >>> >thus a working installation. This is what prompted the > >> >> >>discussion. > >> >> >> >> >>> > >> >> >> >> >>> Ah - no worries, what you could do is: > >> >> >> >> >>> > >> >> >> >> >>> 1. only enable to plugin if -Pwith-mesos is enabled; and > >> >> >> >> >>> > >> >> >> >> >>> > > >> >> >> >> >>> >Running the plugin under a profile works. > >> >> >> >> >>> > >> >> >> >> >>> Yep. > >> >> >> >> >>> > >> >> >> >> >>> > However, not running the plugin > >> >> >> >> >>> >causes compile errors in dependant code. Excluding this > >>code > >> >> >> >>except > >> >> >> >> >>> >within > >> >> >> >> >>> >the profile doesn't seem to work, and is considered by > >>some > >> >>to > >> >> >>be > >> >> >> >>bad > >> >> >> >> >>>form > >> >> >> >> >>> >because there is nothing inside the jar file that notes > >>which > >> >> >> >>profiles > >> >> >> >> >>> >were > >> >> >> >> >>> >used to compile. > >> >> >> >> >>> > >> >> >> >> >>> Got it. Suggestion here would be: > >> >> >> >> >>> > >> >> >> >> >>> 2. create a new module, cas-resource-mesos, and inside of > >>that > >> >> >> >>module, > >> >> >> >> >>> take one of the following approaches, assuming the module > >>is > >> >> >> >>activated > >> >> >> >> >>> when -Pwith-mesos is enabled: > >> >> >> >> >>> > >> >> >> >> >>> 2a. Maven Antrun like so (in this old example): > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> > >> >> >> > >> >> > >> > http://stackoverflow.com/questions/1578456/integrate-protocol-buffers-in > >> >> >> >> >>>t > >> >> >> >> >>>o- > >> >> >> >> >>> maven2-build > >> >> >> >> >>> > >> >> >> >> >>> (pro: more flexibility in case protoc isn¹t there; to fail > >>on > >> >> >> >>error; to > >> >> >> >> >>> only compile if > >> >> >> >> >>> protoc is available > >> >> >> >> >>> > >> >> >> >> >>> 2b. Maven protobuf plugin > >> >> >> >> >>> > >>http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html > >> >> >> >> >>> > >> >> >> >> >>> Here¹s how to enable a module with a profile: > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> > >> >> >> > >> >> > >> > http://blog.soebes.de/blog/2013/11/09/why-is-it-bad-to-activate-slash-de > >> >> >> >> >>>a > >> >> >> >> >>>ct > >> >> >> >> >>> ive-modules-by-profiles-in-maven/ > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> >>> It seems like that is a bad idea though, based on that > >> >> >>discussion. > >> >> >> >> >>> > >> >> >> >> >>> So, here¹s another option: > >> >> >> >> >>> > >> >> >> >> >>> 1. Inside of cas-resource (no special new module or > >>anything > >> >> >>else) > >> >> >> >> >>> 2. include some custom Ant magic via a build.xml file and > >>the > >> >> >>Maven > >> >> >> >> >>> AntRun plugin: > >> >> >> >> >>> 2a. test if protoc is on the system path, and if not, > >> >>download > >> >> >>it, > >> >> >> >> >>>e.g., > >> >> >> >> >>> into the target directory (gets deleted on clean) > >> >> >> >> >>> 2b. call protoc and compile after 2a > >> >> >> >> >>> > >> >> >> >> >>> I would suggest this solution as I think it¹s the most > >>robust > >> >>and > >> >> >> >> >>>ensures > >> >> >> >> >>> we always have a cas-resource that includes mesos and > >>compiled > >> >> >> >> >>>correctly. > >> >> >> >> >>> > >> >> >> >> >>> Cheers, > >> >> >> >> >>> Chris > >> >> >> >> >>> > >> >> >> >> >>> > > >> >> >> >> >>> >Any ideas on how to continue? > >> >> >> >> >>> > > >> >> >> >> >>> >Michael > >> >> >> >> >>> > On Nov 5, 2014 11:04 AM, "Chris Mattmann" > >> >> >> >><[email protected]> > >> >> >> >> >>> >wrote: > >> >> >> >> >>> > > >> >> >> >> >>> >> Hi Mike, > >> >> >> >> >>> >> > >> >> >> >> >>> >> Great discussion. It would be nice if there was > >> >> >> >> >>> >> a protoc Maven plugin: > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >>http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >> >> >> >>> >> Looks like there is. My suggestion: > >> >> >> >> >>> >> > >> >> >> >> >>> >> 1. use a Profile, something like -Pwith-mesos and > >> >> >> >> >>> >> then when activated; > >> >> >> >> >>> >> 2. call the above plugin if -Pwith-mesos is activated > >> >> >> >> >>> >> in the resource manager > >> >> >> >> >>> >> > >> >> >> >> >>> >> Sound good? > >> >> >> >> >>> >> > >> >> >> >> >>> >> Cheers, > >> >> >> >> >>> >> Chris > >> >> >> >> >>> >> > >> >> >> >> >>> >> ------------------------ > >> >> >> >> >>> >> Chris Mattmann > >> >> >> >> >>> >> [email protected] > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >> >> >> >>> >> -----Original Message----- > >> >> >> >> >>> >> From: Michael Starch <[email protected]> > >> >> >> >> >>> >> Reply-To: <[email protected]> > >> >> >> >> >>> >> Date: Wednesday, November 5, 2014 at 11:46 AM > >> >> >> >> >>> >> To: <[email protected]> > >> >> >> >> >>> >> Subject: Extra Compiler Tools > >> >> >> >> >>> >> > >> >> >> >> >>> >> >All, > >> >> >> >> >>> >> > > >> >> >> >> >>> >> >I am trying to integrate apache-mesos with our resource > >> >> >>manager. > >> >> >> >> >>> >>However, > >> >> >> >> >>> >> >mesos uses a technology called "protobuff" from Google > >>for > >> >> >> >> >>> >> >marshaling/unmarshaling data. > >> >> >> >> >>> >> > > >> >> >> >> >>> >> >This requires running a tool called "protoc" to > >>generate a > >> >> >> >>source > >> >> >> >> >>>file > >> >> >> >> >>> >>in > >> >> >> >> >>> >> >java. What is the best way to integrate this step into > >> >>our > >> >> >> >>build > >> >> >> >> >>> >>process? > >> >> >> >> >>> >> > > >> >> >> >> >>> >> >Options I can conceive of: > >> >> >> >> >>> >> > -Check in generated java file > >> >> >> >> >>> >> > -Require "protoc" installation to build resource > >> >>manager > >> >> >> >> >>> >> > -Separate extra resource package into new module > >> >> >> >> >>> >> > > >> >> >> >> >>> >> >None of these ideas are very clean. > >> >> >> >> >>> >> > > >> >> >> >> >>> >> >Any other ideas? I tried setting up a profile to only > >> >> >>compile > >> >> >> >> >>>these > >> >> >> >> >>> >> >sources when selected, but that turned out not to work. > >> >> >> >> >>> >> > > >> >> >> >> >>> >> >-Michael Starch > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >> >> >> >>> >> > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > >> >> > >> > >> > >> > > >
