Maybe I am not give a clear description. I am runing Spark on yarn. instead of Mesos. I just want build Spark with protobuf2.5. I am not care about Mesos.
I've changed Spark pom.xml to probobuf2.5 manually. On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia <[email protected]>wrote: > Mesos will almost certainly compile fine with protobuf 2.5. The protobuf > compiler and binary format is forward-compatible across releases, it’s just > the Java artifacts that aren’t. You’ll need to ask Mesos to provide a > version with protobuf 2.5, and use that with these versions of Hadoop. > > Matei > > On Dec 15, 2013, at 7:00 PM, Liu, Raymond <[email protected]> wrote: > > > That issue is for 0.9's solution. > > > > And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, > protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine > with hadoop 2.2 > > And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon > mesos... strange combination, I am not sure, might have problem. If have > problem, you might need to build mesos against 2.5.0, I don't test that, if > you got time, mind take a test? > > > > Best Regards, > > Raymond Liu > > > > > > -----Original Message----- > > From: Liu, Raymond [mailto:[email protected]] > > Sent: Monday, December 16, 2013 10:48 AM > > To: [email protected] > > Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > > > > Hi Azuryy > > > > Please Check https://spark-project.atlassian.net/browse/SPARK-995 for > this protobuf version issue > > > > Best Regards, > > Raymond Liu > > > > -----Original Message----- > > From: Azuryy Yu [mailto:[email protected]] > > Sent: Monday, December 16, 2013 10:30 AM > > To: [email protected] > > Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > > > > Hi here, > > Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some > uncompatable API between these two versions. > > Hadoop-2.x using protobuf-2.5.0 > > > > > > but if some guys want to run Spark on mesos, then mesos using > > protobuf-2.4.1 currently. so we may discuss here for a better solution. > > > > > > > > On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <[email protected]> wrote: > > > >> Thanks Patrick. > >> On 16 Dec 2013 02:43, "Patrick Wendell" <[email protected]> wrote: > >> > >>> You can checkout the docs mentioned in the vote thread. There is also > >>> a pre-build binary for hadoop2 that is compiled for YARN 2.2 > >>> > >>> - Patrick > >>> > >>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <[email protected]> wrote: > >>>> yarn 2.2, not yarn 0.22, I am so sorry. > >>>> > >>>> > >>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <[email protected]> > wrote: > >>>> > >>>>> Hi, > >>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? > >>>>> Thanks. > >>>>> > >>>>> > >>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < > >>> [email protected]>wrote: > >>>>> > >>>>>> Yeah seems like it. He was ok with our prev release. > >>>>>> Let's wait for his reply > >>>>>> > >>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote: > >>>>>> > >>>>>>> Henry - from that thread it looks like sebb's concern was > >>>>>>> something different than this. > >>>>>>> > >>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < > >>>>>> [email protected]> > >>>>>>> wrote: > >>>>>>>> Hi Patrick, > >>>>>>>> > >>>>>>>> Yeap I agree, but technically ASF VOTE release on source > >>>>>>>> only, > >>> there > >>>>>>>> even debate about it =), so putting it in the vote staging > >>> artifact > >>>>>>>> could confuse people because in our case we do package 3rd > >>>>>>>> party libraries in the binary jars. > >>>>>>>> > >>>>>>>> I have sent email to sebb asking clarification about his > >>>>>>>> concern > >>> in > >>>>>>>> general@ list. > >>>>>>>> > >>>>>>>> - Henry > >>>>>>>> > >>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell < > >>> [email protected] > >>>>>>> > >>>>>>> wrote: > >>>>>>>>> Hey Henry, > >>>>>>>>> > >>>>>>>>> One thing a lot of people do during the vote is test the > >>> binaries and > >>>>>>>>> make sure they work. This is really valuable. If you'd like > >>>>>>>>> I > >>> could > >>>>>>>>> add a caveat to the vote thread explaining that we are only > >>> voting on > >>>>>>>>> the source. > >>>>>>>>> > >>>>>>>>> - Patrick > >>>>>>>>> > >>>>>>>>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < > >>>>>>> [email protected]> wrote: > >>>>>>>>>> Actually we should be fine putting the binaries there as > >>>>>>>>>> long > >>> as the > >>>>>>>>>> VOTE is for the source. > >>>>>>>>>> > >>>>>>>>>> Let's verify with sebb in the general@ list about his concern. > >>>>>>>>>> > >>>>>>>>>> - Henry > >>>>>>>>>> > >>>>>>>>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < > >>>>>>> [email protected]> wrote: > >>>>>>>>>>> Hi Patrick, as sebb has mentioned let's move the binaries > >>>>>>>>>>> from > >>> the > >>>>>>>>>>> voting directory in your people.apache.org directory. > >>>>>>>>>>> ASF release voting is for source code and not binaries, > >>>>>>>>>>> and technically we provide binaries for convenience. > >>>>>>>>>>> > >>>>>>>>>>> And add link to the KEYS location in the dist[1] to let > >>>>>>>>>>> verify > >>>>>>> signatures. > >>>>>>>>>>> > >>>>>>>>>>> Sorry for the late response to the VOTE thread, guys. > >>>>>>>>>>> > >>>>>>>>>>> - Henry > >>>>>>>>>>> > >>>>>>>>>>> [1] > >>>>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS > >>>>>>>>>>> > >>>>>>>>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell < > >>>>>> [email protected]> > >>>>>>> wrote: > >>>>>>>>>>>> The vote is now closed. This vote passes with 5 PPMC +1's > >>>>>>>>>>>> and > >>> no 0 > >>>>>>> or -1 > >>>>>>>>>>>> votes. > >>>>>>>>>>>> > >>>>>>>>>>>> +1 (5 Total) > >>>>>>>>>>>> Matei Zaharia* > >>>>>>>>>>>> Nick Pentreath* > >>>>>>>>>>>> Patrick Wendell* > >>>>>>>>>>>> Prashant Sharma* > >>>>>>>>>>>> Tom Graves* > >>>>>>>>>>>> > >>>>>>>>>>>> 0 (0 Total) > >>>>>>>>>>>> > >>>>>>>>>>>> -1 (0 Total) > >>>>>>>>>>>> > >>>>>>>>>>>> * = Binding Vote > >>>>>>>>>>>> > >>>>>>>>>>>> As per the incubator release guide [1] I'll be sending > >>>>>>>>>>>> this > >>> to the > >>>>>>>>>>>> general incubator list for a final vote from IPMC members. > >>>>>>>>>>>> > >>>>>>>>>>>> [1] > >>>>>>>>>>>> > >>>>>>> > >>>>>> > >>> http://incubator.apache.org/guides/releasemanagement.html#best-practi > >>> ce-incubator-release- > >>>>>>>>>>>> vote > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan > >>>>>>>>>>>> <[email protected]> > >>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> I'd be personally fine with a standard workflow of > >>> assemble-deps > >>>>>> + > >>>>>>>>>>>>> packaging just the Spark files as separate packages, if > >>>>>>>>>>>>> it > >>>>>> speeds up > >>>>>>>>>>>>> everyone's development time. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < > >>>>>>> [email protected] > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> I don't know how to make sense of the numbers, but > >>>>>>>>>>>>>> here's > >>> what > >>>>>>> I've got > >>>>>>>>>>>>>> from a very small sample size. > >>>>>> > >>>>> > >>>>> > >>> > >> > >
