I actually find this to be a bit confusing scheme. I have not seen any project or product doing something similar.
Olga -----Original Message----- From: Dmitriy Ryaboy [mailto:[email protected]] Sent: Monday, October 24, 2011 5:38 PM To: [email protected] Subject: Re: Next Pig release proposal To be a little more concrete about what I am saying here -- I don't think we should put a "1.0" label on any *.0 release. 0.8.1 is pretty solid; 0.9.0 has some holes, 0.9.1 is better. If we put 1.0 on what is currently being thought of as 0.10, it will have some stability / usability issues (things tend to show up after we make a release and people in the wild start trying it), and those issues will make a poor impression on those who expect 1.0 to be shiny and polished after so much time. I'm in favor of waiting a couple of dot releases, promoting a stabilized release into 1.0, and going from there. So, pictorially: -- trunk --- 0.11-dev ----------0.12-dev------------------| 1.2-dev! \ \ \ \ ---------------- 0.11.0 --------------------| 1.1.0! \ \------- 0.10.0 ------- 0.10.1 ------- 0.10.2 --------| 1.0.0 !! On Mon, Oct 24, 2011 at 12:43 PM, Dmitriy Ryaboy <[email protected]> wrote: > I am good with Scheme 2. > > We are finding a fair number of issues trying to move from Pig 0.8.1 to > 0.9, and I don't think those issues are fixed in 10, either.. not sure that > this "stabilization" process has happened yet. > > D > > > On Mon, Oct 24, 2011 at 11:59 AM, Daniel Dai <[email protected]>wrote: > >> Yes, we need a versioning scheme. There are two versioning scheme I can >> think of: >> >> Scheme 1: >> <major>.<patch> >> <major> will be the feature rich release every 3 month >> <patch> will be the bug fix release when necessary >> >> Nov release will be 1.0, Feb release will be 2.0. There will be 1.1, 2.1 >> etc >> for bug fixes. >> >> Scheme 2: >> <major>.<minor>.<patch> >> Most of our 3 month release will be counted as <minor> release unless >> there >> are major user facing/disruptive changes. >> >> Nov release will be 1.0.0, Feb release will be 1.1.0. There will be 1.0.1, >> 1.1.1 etc for bug fixes. >> >> I personally prefer scheme 2, increasing major version too frequently >> might >> be confusing to users. How's other folks feel? >> >> Daniel >> >> >> On Sat, Oct 22, 2011 at 2:31 AM, Gianmarco De Francisci Morales < >> [email protected]> wrote: >> >> > Hi, >> > >> > just my 2 cents. >> > >> > I think the issue here is not 1.0 vs 0.10, but what's the versioning >> scheme >> > we want to use for Pig. >> > Up to now it has been just an increasing number after a '0.' prefix, >> > changed >> > when the community felt it was time. I think this works well for a small >> > project, but it is somewhat fuzzy. >> > >> > I like the idea of having <major>.<minor>.<patch> versions like many >> other >> > projects. It's a very clear and almost standard way of versioning a >> piece >> > of >> > software. It has clear rules on when to change each of the numbers, and >> > lets >> > the user get an idea of backward compatibility at a glance. >> > >> > So, to conclude, I am in favor of going 1.0 (or 1.0.0) as long as we >> decide >> > a clear versioning policy (whichever it is). >> > So that the 1.0 milestone would mark the beginning of our new policy. >> > >> > Cheers, >> > -- >> > Gianmarco >> > >> > >> > >> > On Fri, Oct 21, 2011 at 23:10, <[email protected]> wrote: >> > >> > > If one were to rewrite input and output formats to use the webhdfs:// >> > > APIs, this would not be an issue, right ? >> > > >> > > - milind >> > > >> > > >> > > On 10/21/11 1:50 PM, "Santhosh Srinivasan" <[email protected]> wrote: >> > > >> > > >If I was not clear in my earlier email, I apologize for the lack of >> > > >clarity. I am no longer in favour of waiting for Hadoop API stability >> > > >across Hadoop versions. It's a pipe dream. >> > > > >> > > >When we had PigInputFormat and PigOutputFormat, your reasoning would >> be >> > > >spot on. I am concerned about the following. Our tight integration >> with >> > > >Hadoop due to the use of Input and Output format might lead to a >> break >> > in >> > > >backward compatibility. I am not sure if the comparison with that of >> > Java >> > > >is valid. Probably a majority of the users don't use JNI. Its very >> hard >> > > >to use Pig without writing custom load and store functions. The >> default >> > > >load and store don't suffice for a majority of use cases that I have >> > > >observed. >> > > > >> > > >I am trying to get all factors that might influence this decision. >> From >> > > >the few emails that have been exchanged since yesterday, we have the >> > > >following factors: >> > > > >> > > >1. Hadoop 0.20.205 (support for Append) >> > > >2. Hadoop 0.22 >> > > >3. Hadoop 0.23 >> > > >4. Maturity of the new parser >> > > >5. Stability of the new logical plan >> > > >6. Other components in the eco system. >> > > > - Avro (1.5.4, 1.4.1, ...) >> > > > - Cassandra (1.0.0, 0.8.7, ...) >> > > > - Chukwa (0.4.0, 0.3.0, ...) >> > > > - Hama (0.3.0, 0.2.0, ...) >> > > > - Hbase (0.90.4, 0.90.3, 0.90.2, 0.90.1, ...) >> > > > - Hive (Releases - 0.7.1, 0.7.0, 0.6.0, ...) >> > > > - Zookeeper (3.3.3, 3.3.2, 3.2.2, 3.1.2, ...) >> > > > >> > > >Santhosh >> > > > >> > > > >> > > >-----Original Message----- >> > > >From: Thejas Nair [mailto:[email protected]] >> > > >Sent: Friday, October 21, 2011 11:22 AM >> > > >To: [email protected] >> > > >Subject: Re: Next Pig release proposal >> > > > >> > > > >> > > >Santosh, >> > > >I thought you meant API stability for hadoop across major versions, >> but >> > I >> > > >guess you are referring to stability within 0.23 versions. But >> argument >> > > >applies to that as well, if 0.23.1 is not compatible with 0.23.0, we >> > need >> > > >to call the release for 0.23.1 as 'pig 1.x for 0.23.1 api' . >> > > > >> > > >We just need to communicate to the users that the >> > > >InputFormat/OutputFormat api's (and any anything else we expose from >> > > >hadoop) depends on the hadoop version they are using. >> > > > >> > > >I think it is just like different JNI libraries that you would write >> for >> > > >different OS. But the java version remains the same across OSs. >> > > > >> > > >-Thejas >> > > > >> > > > >> > > >On 10/21/11 10:59 AM, Santhosh Srinivasan wrote: >> > > >> Thejas, >> > > >> >> > > >> I guess you did not read my email completely. You are referring to >> the >> > > >>premise without examining the conclusion. I am repasting my entire >> > email >> > > >>to avoid confusion (I hate truncated references). If you could >> respond >> > > >>again, it will bring us onto the same page. >> > > >> >> > > >> <email> >> > > >> >> > > >> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0) >> > > >> >> > > >> How far have we progressed from our last discussion in March. There >> > was >> > > >>no consensus on the 1.0 release. Opinions ranged from having more >> > > >>releases to bake in the maturity of the new parser and logical plan >> > > >>changes to compatibility with Hadoop API (was compared to Social >> > > >>Security - a very hot topic these days). >> > > >> >> > > >> My concerns were around Hadoop API stability. I have heard that the >> > > >>APIs will not be stable for at least 1 year. This is taking me away >> > from >> > > >>the Hadoop API stability factor (They passed healthcare in that >> > > >>duration. Really!) Do we want compatibility with 0.23 as a gating >> > factor >> > > >>- not sure if this is anywhere close to getting done in the near >> > future. >> > > >>Will we support append (0.20.205)? >> > > >> >> > > >> Btw, Hbase has been doing 0.90.1, 0.90.2, etc. So we can take a >> look >> > at >> > > >>this option too. >> > > >> >> > > >> Santhosh >> > > >> >> > > >> >> > > >> >> > > >> -----Original Message----- >> > > >> From: Olga Natkovich [mailto:[email protected]] >> > > >> Sent: Thursday, October 20, 2011 4:40 PM >> > > >> To: [email protected] >> > > >> Subject: Next Pig release proposal >> > > >> >> > > >> Hi, >> > > >> >> > > >> Here is what I propose we do for the next Pig release: >> > > >> >> > > >> >> > > >> (1) Branch early next week - we have major features and many >> bug >> > > >>fixes in and will be fixing remaining bugs on the branch >> > > >> >> > > >> (2) Publish the release by 11/15 - that will give us a couple of >> > > >>weeks to stabilize the branch and get last minute bug fixes in >> > > >> >> > > >> (3) Make this release a 1.0 release. Reasons to go for 1.0 and >> not >> > > >>0.10 >> > > >> >> > > >> a. This release has minimal number of features and was >> focused >> > on >> > > >>code stabilization and bug fixes. We believe it will be a stable >> > release >> > > >> >> > > >> <email/> >> > > >> >> > > >> Thanks, >> > > >> Santhosh >> > > >> >> > > >> -----Original Message----- >> > > >> From: Thejas Nair [mailto:[email protected]] >> > > >> Sent: Friday, October 21, 2011 10:45 AM >> > > >> To: [email protected] >> > > >> Subject: Re: Next Pig release proposal >> > > >> >> > > >> On 10/20/11 4:58 PM, Santhosh Srinivasan wrote: >> > > >>> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0) >> > > >>> >> > > >>> How far have we progressed from our last discussion in March. >> There >> > > >>>was no consensus on the 1.0 release. Opinions ranged from having >> more >> > > >>>releases to bake in the maturity of the new parser and logical plan >> > > >>>changes to compatibility with Hadoop API (was compared to Social >> > > >>>Security - a very hot topic these days). >> > > >>> >> > > >>> My concerns were around Hadoop API stability. >> > > >> >> > > >> Over the next year or so, there are going to be two API versions of >> > > >>hadoop to be supported - 0.20.x api's and 0.23 apis, as we will have >> > > >>userbase on both. >> > > >> >> > > >> I think it is just a matter of releasing pig 1.0 for 0.20.x api's >> and >> > > >>1.0 for 0.23.x api's. We will have to come up with a numbering >> scheme >> > > >>that reflects 'for hadoop version X' in our pig releases, regardless >> of >> > > >>it being 0.10 or 1.0. >> > > >> >> > > >> As there will be support for different api's of hadoop in pig >> > releases, >> > > >>I don't see a reason why the hadoop api stability should stop pig >> from >> > > >>going 1.0 . >> > > >> >> > > >> -Thejas >> > > > >> > > > >> > > >> > > >> > >> > >
