Typos (hit send too soon): I meant, comment on api compatibility of pig-tez and pig-trunk (including Utils etc.)
I would like to make sure 0.13 is compatible with the next version to avoid ramping up tools that depend on pig (Ambrose etc). ~Aniket On Fri, Feb 14, 2014 at 1:23 PM, Aniket Mokashi <aniket...@gmail.com> wrote: > Another thing: Pig-trunk has loads of incompatible interface changes > already. It would be great if someone from pig-tez dev (Daniel et al) can > comment on compatibility of these with pig-trunk? If they are not > compatible or if there is any api refactor on wish list, can we please open > jiras for it? We should try to ensure that there is NO api incompatibility > between 0.13 and 0.14 (when it is out future). > > Thanks, > Aniket > > > On Thu, Feb 13, 2014 at 8:09 PM, Koji Noguchi <knogu...@yahoo-inc.com>wrote: > >> For 0.13, can we get >> >> PIG-3508 (covered by Daniel's PIG-3545) >> Otherwise, "explain" would show pre-optimized logical plan in 0.13. >> >> > I can wrap a 0.12.1 release in a couple of weeks. >> > >> Great! >> >> Koji >> >> >> >> On Feb 13, 2014, at 7:11 PM, Daniel Dai <da...@hortonworks.com> wrote: >> >> > Ok, it seems to leave no choice for a sooner 0.14.0 release then. >> > >> > On Thu, Feb 13, 2014 at 3:50 PM, Julien Le Dem <jul...@ledem.net> >> wrote: >> >> We can't use SNAPSHOT releases as dependencies of our official >> releases. >> >> They can change and disappear. >> >> Let's use the official release of hive when it's available. >> >> >> >> On Feb 13, 2014, at 3:33 PM, Daniel Dai wrote: >> >> >> >>> I can also use a hive 0.13.0 SNAPSHOT in PIG-3558, and then switch to >> >>> a released version later. Seems that's much easier. >> >>> >> >>> On Thu, Feb 13, 2014 at 2:35 PM, Daniel Dai <da...@hortonworks.com> >> wrote: >> >>>> That works. I will roll a 0.14.0 release as soon as Hive 0.13.0 >> released. >> >>>> >> >>>> On Thu, Feb 13, 2014 at 2:25 PM, Julien Le Dem <jul...@ledem.net> >> wrote: >> >>>>> I think it would be OK to release Pig 0.14 soon after if you want >> new features in. >> >>>>> It's not more work to release 0.14 than 0.13.1 and that keeps the >> semantic versioning. >> >>>>> >> >>>>> On Feb 13, 2014, at 1:29 PM, Daniel Dai wrote: >> >>>>> >> >>>>>> Hive-0.13.0 will be release sometime March AFAIK. If we cannot make >> >>>>>> PIG-3558 to Pig 0.13.0, I still want to put in Pig 0.13.1, though >> it >> >>>>>> is a new feature. >> >>>>>> >> >>>>>> Other than those listed Jiras, there are also several unit >> >>>>>> tests/system tests failures (eg, TestHBaseStorage) we need to fix. >> >>>>>> >> >>>>>> On Thu, Feb 13, 2014 at 11:45 AM, Aniket Mokashi < >> aniket...@gmail.com> wrote: >> >>>>>>> Sounds good. Do we know a timeline for Hive-0.13 release? If its >> not too >> >>>>>>> far, we can try to do Pig-0.13 it a few weeks after that. >> >>>>>>> >> >>>>>>> So far we have following jiras to wrap up before Pig-0.13 release. >> >>>>>>> PIG-3591 >> >>>>>>> PIG-3740 >> >>>>>>> PIG-3745 >> >>>>>>> PIG-3347 >> >>>>>>> PIG-3731 >> >>>>>>> PIG-3558 >> >>>>>>> >> >>>>>>> Any other? We can start marking jiras with corresponding version >> numbers. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> On Thu, Feb 13, 2014 at 9:58 AM, Daniel Dai < >> da...@hortonworks.com> wrote: >> >>>>>>> >> >>>>>>>> I'd like to get PIG-3558 in, which depends on HIVE-0.13.0 >> release in >> >>>>>>>> weeks. Otherwise I am fine with a Pig 0.13.0 release. >> >>>>>>>> >> >>>>>>>> I can wrap a 0.12.1 release in a couple of weeks. >> >>>>>>>> >> >>>>>>>> Thanks, >> >>>>>>>> Daniel >> >>>>>>>> >> >>>>>>>> On Thu, Feb 13, 2014 at 3:38 AM, Gianmarco De Francisci Morales >> >>>>>>>> <g...@apache.org> wrote: >> >>>>>>>>> +1 on releasing a 0.13 and if somebody feels strongly about >> releasing a >> >>>>>>>>> 0.12.1 that fixes PIG-3492 I am +1 on that too. >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> Gianmarco >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On 13 February 2014 03:08, Dmitriy Ryaboy <dvrya...@gmail.com> >> wrote: >> >>>>>>>>> >> >>>>>>>>>> So I think we agree that we should branch 0.13 at this point, >> right? >> >>>>>>>>>> >> >>>>>>>>>> (and possibly look at releasing an incremental bump to 12 or >> 10? I'm not >> >>>>>>>>>> sure what should be included there but I support the general >> idea). >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> On Thu, Feb 6, 2014 at 2:22 PM, Koji Noguchi < >> knogu...@yahoo-inc.com> >> >>>>>>>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>>>> Releasing 0.13 and 0.10.1 is totally independent in my >> opinion >> >>>>>>>>>>>> >> >>>>>>>>>>> I should have referenced my previous request on including >> 0.10.1 on >> >>>>>>>> the >> >>>>>>>>>>> top release page. >> >>>>>>>>>>> http://www.mail-archive.com/dev@pig.apache.org/msg20629.html >> >>>>>>>>>>> >> >>>>>>>>>>>> By minor I meant 0.13 >> >>>>>>>>>>>> 0.10.1 is a bug fix release. >> >>>>>>>>>>>> as in Major.Minor.BugFix >> >>>>>>>>>>>> >> >>>>>>>>>>> I see. Then I should have said, >> >>>>>>>>>>> "but I'd like to request we make BugFix releases more often." >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks for correcting my mistake. >> >>>>>>>>>>> >> >>>>>>>>>>> Koji >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> On Feb 6, 2014, at 5:05 PM, Julien Le Dem <jul...@ledem.net> >> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Releasing 0.13 and 0.10.1 is totally independent in my >> opinion. >> >>>>>>>>>>>> It just takes the time of a committer that needs the release >> to >> >>>>>>>> happen >> >>>>>>>>>>> to do it. >> >>>>>>>>>>>> >> >>>>>>>>>>>> By minor I meant 0.13 >> >>>>>>>>>>>> 0.10.1 is a bug fix release. >> >>>>>>>>>>>> as in Major.Minor.BugFix >> >>>>>>>>>>>> >> >>>>>>>>>>>> Our Major version is still 0 >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Feb 6, 2014, at 1:43 PM, Koji Noguchi wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>>> To add to the discussion, I think we should release more >> often, >> >>>>>>>> based >> >>>>>>>>>>> on time elapsed rather than volume of change. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>> I don't have preference on the frequency, but I'd like to >> request >> >>>>>>>> we >> >>>>>>>>>>> make minor releases more often. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> At this moment, stable pig release (to me) is still 0.10.1. >> >>>>>>>>>>>>> 0.11.1 and 0.12.0 both have regression bug PIG-3492 that >> caused >> >>>>>>>>>>> multiple production pig scripts in our clusters to fail >> randomly. >> >>>>>>>>>>>>> (unless user is disabling ColumnMapKeyPrune) >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> If releasing 0.13 means 0.10.1 gets kicked out from the >> front >> >>>>>>>> release >> >>>>>>>>>>> list, I'd like to see minor release on 0.11 or 0.12 first. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Koji >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Feb 6, 2014, at 4:25 PM, Cheolsoo Park < >> piaozhe...@gmail.com> >> >>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> +1 to 0.13 release. Why not if someone is volunteering? >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Thu, Feb 6, 2014 at 4:06 PM, Julien Le Dem < >> jul...@ledem.net> >> >>>>>>>>>>> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> To add to the discussion, I think we should release more >> often, >> >>>>>>>>>> based >> >>>>>>>>>>> on >> >>>>>>>>>>>>>>> time elapsed rather than volume of change. >> >>>>>>>>>>>>>>> The more often we release, the easier it is to release. >> >>>>>>>>>>>>>>> Also that makes it easier for contributors to use their >> own >> >>>>>>>>>>> contributions >> >>>>>>>>>>>>>>> in official releases. >> >>>>>>>>>>>>>>> It is also probably a good idea to have a clean starting >> point >> >>>>>>>>>> before >> >>>>>>>>>>>>>>> merging the Tez branch >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> That said, I think those changes by themselves are enough >> to >> >>>>>>>>>> warrant a >> >>>>>>>>>>>>>>> minor release. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Julien >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On Feb 6, 2014, at 12:24 PM, Dmitriy Ryaboy wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Major updates since we release 12 that are currently in >> trunk: >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> - lazy output (don't generate empty part files) >> >>>>>>>>>>>>>>>> - jar caching optimization >> >>>>>>>>>>>>>>>> - automatic local mode for small job (big wall-clock >> wins for >> >>>>>>>>>>> long-tail >> >>>>>>>>>>>>>>>> jobs) >> >>>>>>>>>>>>>>>> - improved support for BigInteger, BigDecimal >> >>>>>>>>>>>>>>>> - hbase loader improvements >> >>>>>>>>>>>>>>>> - debug mode that leaves temp files around for >> examination (!) >> >>>>>>>>>>>>>>>> - fixes to a few nasty bugs (PIG-3641) >> >>>>>>>>>>>>>>>> - pluggable execution engine allowing work like Tez and >> Spork >> >>>>>>>>>>>>>>>> - .. and more >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> I'd say this justifies a release. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> D >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> On Wed, Feb 5, 2014 at 3:55 PM, Aniket Mokashi < >> >>>>>>>>>> aniket...@gmail.com> >> >>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> List I mentioned is pending tasks before we can make a >> release. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> A complete list of contributions can be seen at - >> >>>>>>>>>>>>>>>>> >> http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?view=markup >> >>>>>>>> . >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Some of the things that make it a good candidate for a >> release- >> >>>>>>>>>>>>>>>>> - PIG-3419 (has several backwards incompatible api >> changes) >> >>>>>>>>>>>>>>>>> - PIG-2672 >> >>>>>>>>>>>>>>>>> - PIG-3642 >> >>>>>>>>>>>>>>>>> - PIG-3463 >> >>>>>>>>>>>>>>>>> - PIG-3511 >> >>>>>>>>>>>>>>>>> - PIG-3657 >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>>>>> Aniket >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> On Wed, Feb 5, 2014 at 3:23 PM, Olga Natkovich < >> >>>>>>>>>>> onatkov...@yahoo.com> >> >>>>>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Just going by the list that Aniket provided, I don't >> really >> >>>>>>>> see >> >>>>>>>>>>> enough >> >>>>>>>>>>>>>>>>> for >> >>>>>>>>>>>>>>>>>> a full release. Two mentioned JIRAs are doc updates >> and one >> >>>>>>>> is a >> >>>>>>>>>>> bug >> >>>>>>>>>>>>>>> fix >> >>>>>>>>>>>>>>>>>> that was ported into Pig 12. >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> On Wednesday, February 5, 2014 3:13 PM, Aniket Mokashi >> < >> >>>>>>>>>>>>>>>>>> aniket...@gmail.com> wrote: >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Hi All, >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> A good number of improvements and bug fixes have gone >> into >> >>>>>>>> trunk >> >>>>>>>>>>>>>>>>> recently. >> >>>>>>>>>>>>>>>>>> I'd like to know if we can roll out a Pig 0.13 release >> around >> >>>>>>>>>>>>>>> mid-March? >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> I am aware that we are planning to merge tez branch >> into trunk >> >>>>>>>>>>> soon. >> >>>>>>>>>>>>>>>>>> However, making a release before tez branch is merged >> will be >> >>>>>>>>>>> good. Any >> >>>>>>>>>>>>>>>>>> objections? >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Following are few jiras we need to wrap up before 0.13 >> >>>>>>>> release- >> >>>>>>>>>>>>>>>>>> PIG-3591 >> >>>>>>>>>>>>>>>>>> PIG-3740 >> >>>>>>>>>>>>>>>>>> PIG-3745 >> >>>>>>>>>>>>>>>>>> PIG-3347 >> >>>>>>>>>>>>>>>>>> PIG-3731 >> >>>>>>>>>>>>>>>>>> Any other? >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>>>>>> Aniket >> >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> -- >> >>>>>>>>>>>>>>>>> "...:::Aniket:::... Quetzalco@tl" >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> CONFIDENTIALITY NOTICE >> >>>>>>>> NOTICE: This message is intended for the use of the individual >> or entity to >> >>>>>>>> which it is addressed and may contain information that is >> confidential, >> >>>>>>>> privileged and exempt from disclosure under applicable law. If >> the reader >> >>>>>>>> of this message is not the intended recipient, you are hereby >> notified that >> >>>>>>>> any printing, copying, dissemination, distribution, disclosure or >> >>>>>>>> forwarding of this communication is strictly prohibited. If you >> have >> >>>>>>>> received this communication in error, please contact the sender >> immediately >> >>>>>>>> and delete it from your system. Thank You. >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> -- >> >>>>>>> "...:::Aniket:::... Quetzalco@tl" >> >>>>>> >> >>>>>> -- >> >>>>>> CONFIDENTIALITY NOTICE >> >>>>>> NOTICE: This message is intended for the use of the individual or >> entity to >> >>>>>> which it is addressed and may contain information that is >> confidential, >> >>>>>> privileged and exempt from disclosure under applicable law. If the >> reader >> >>>>>> of this message is not the intended recipient, you are hereby >> notified that >> >>>>>> any printing, copying, dissemination, distribution, disclosure or >> >>>>>> forwarding of this communication is strictly prohibited. If you >> have >> >>>>>> received this communication in error, please contact the sender >> immediately >> >>>>>> and delete it from your system. Thank You. >> >>>>> >> >>> >> >>> -- >> >>> CONFIDENTIALITY NOTICE >> >>> NOTICE: This message is intended for the use of the individual or >> entity to >> >>> which it is addressed and may contain information that is >> confidential, >> >>> privileged and exempt from disclosure under applicable law. If the >> reader >> >>> of this message is not the intended recipient, you are hereby >> notified that >> >>> any printing, copying, dissemination, distribution, disclosure or >> >>> forwarding of this communication is strictly prohibited. If you have >> >>> received this communication in error, please contact the sender >> immediately >> >>> and delete it from your system. Thank You. >> >> >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or >> entity to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the >> reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> >> > > > -- > "...:::Aniket:::... Quetzalco@tl" > -- "...:::Aniket:::... Quetzalco@tl"
