Do you think we could wait one more week? I've been really hammered with some other work and there's more that I'd like to review here that I haven't been able to get to.
thanks - Robert On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <bafna.i...@gmail.com> wrote: > Sure. I guess we can one extra week and complete some more work. > Thanks everyone for responding. > > ~Abhishek > > On Aug 12, 2016, at 2:46 PM, Purshotam Shah <purus...@yahoo-inc.com.INVALID> > wrote: > > > > Yes. I also have some pending patches. > > > > On Friday, August 12, 2016 2:11 AM, Robert Kanter < > rkan...@cloudera.com> wrote: > > > > > > Ya, I could use some more time as well. I have some deadlines early next > > week, which has kept me from doing more reviews. > > > > thanks > > - Robert > > > > On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma < > > jaydeepma...@gmail.com> wrote: > > > >> Let see if we can wait for another week. So we can merge few more > important > >> jiras. Few of them I personally want to get reviewed and merged. > >> > >> Regards, > >> Jaydeep > >> > >> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <bafna.i...@gmail.com> > >> wrote: > >> > >>> As previously discussed, we will be moving with whatever patches can be > >>> reviewed/merged till end of this week. > >>> > >>> But there are some which needs to be reviewed and merged, mainly > >> regarding > >>> the upgrades. Some of the patches require Java 1.7, so I think we > should > >>> review OOZIE-2036 and merged it. > >>> > >>> ~Abhishek > >>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy < > >> rohini.adi...@gmail.com> > >>> wrote: > >>>> > >>>> Cleaned up that list and now no jira refers to "trunk". Patch > >> Available > >>>> jiras targeted for trunk now point to 4.4.0. > >>>> > >>>> https://issues.apache.org/jira/issues/?jql=project%20% > >>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status% > >>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC > >>>> > >>>> There are 42 jiras to go. We can take a look mid next week to see if > >> any > >>>> pending ones needs to be pushed out. There are quite a few which are > >>>> unassigned. > >>>> > >>>> If there is any jira, someone would like to include in 4.3.0 please > >>> update > >>>> the jira. > >>>> > >>>> Regards, > >>>> Rohini > >>>> > >>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy < > >>> rohini.adi...@gmail.com > >>>>> wrote: > >>>> > >>>>> Unset the open ones without sending emails. Will update the resolved, > >>>>> closed and Patch Available ones sending emails. > >>>>> > >>>>> Regards, > >>>>> Rohini > >>>>> > >>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkan...@cloudera.com > > > >>>>> wrote: > >>>>> > >>>>>> Ya, let's unset those remaining open trunk ones. I agree, we need > to > >>> put > >>>>>> in some effort to clean out old JIRAs and to review Patch Available > >>> JIRAs. > >>>>>> > >>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator. We can > >>> discuss > >>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about > >>> why. > >>>>>> > >>>>>> - Robert > >>>>>> > >>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy < > >>>>>> rohini.adi...@gmail.com> > >>>>>> wrote: > >>>>>> > >>>>>>> > >>>>>>> > >>>>>> https://issues.apache.org/jira/issues/?jql=project%20% > >>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and% > >>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C% > >>> 20status%20DESC&startIndex=50 > >>>>>>> > >>>>>>> These are jiras open and marked trunk. Should we unset fix version > >> on > >>>>>> these > >>>>>>> or mark them for next release? Don't see majority of them being > >> worked > >>>>>> on > >>>>>>> even for next release. We need to do bug scrub and close many of > >> these > >>>>>> some > >>>>>>> time. > >>>>>>> > >>>>>>>> Instead > >>>>>>> of deleting "trunk", I think we're better off just renaming it to > >> the > >>>>>>> release number. > >>>>>>> There is some amount of relabeling either way. If we go with > >> unsetting > >>>>>> fix > >>>>>>> version, then we can as well delete trunk. Also, I did create a > >> 4.4.0 > >>>>>>> yesterday to move some of the patch available jiras from 4.3.0. > >>>>>>> > >>>>>>> > >>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy < > >>>>>>> rohini.adi...@gmail.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> I think it would be better to send out email for the bulk change. > >> If > >>>>>>> there > >>>>>>>> are any issues or jiras wrongly marked someone can respond. We do > >>> that > >>>>>>> with > >>>>>>>> Pig and there are always couple of jiras that folks come back with > >>>>>>> feedback > >>>>>>>> on pulling back to release. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter < > >> rkan...@cloudera.com> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell > >> it > >>>>>> not > >>>>>>>>> to > >>>>>>>>> send out an email for every change :) > >>>>>>>>> > >>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter < > >> rkan...@cloudera.com > >>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Rohini, that sounds good to me. Especially because we're on > >>>>>> "master" > >>>>>>>>>> branch but "trunk" version in JIRA and it gets a little > >> confusing. > >>>>>>>>> Instead > >>>>>>>>>> of deleting "trunk", I think we're better off just renaming it > to > >>>>>> the > >>>>>>>>>> release number. Otherwise, we'll have to re-label any current > >>>>>> "trunk" > >>>>>>>>>> fields. We can use 4.4.0 for now, but I imagine that the next > >>>>>> release > >>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always > >>>>>> rename it > >>>>>>>>>> later. > >>>>>>>>>> > >>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we > >>>>>> should > >>>>>>>>>> probably have a vote in the private list to make things official > >> as > >>>>>>>>> this is > >>>>>>>>>> a release-management-related change. Can you send an email? > >>>>>>>>>> > >>>>>>>>>> Abhishek, thanks for putting this all together. I think > Rohini's > >>>>>>>>>> suggestion of updating the JIRAs is the right next step. You're > >>>>>>>>> probably > >>>>>>>>>> best off using the bulk change feature instead of doing each > JIRA > >>>>>>>>>> individually. I should have time to do a bunch of reviews this > >>>>>> week, > >>>>>>>>> I've > >>>>>>>>>> actually already started. I should have some time to continue > >>>>>> doing > >>>>>>>>> more > >>>>>>>>>> reviews. As a general thing, we've been pretty slow about > >> getting > >>>>>>> these > >>>>>>>>>> done :( > >>>>>>>>>> > >>>>>>>>>> - Robert > >>>>>>>>>> > >>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy < > >>>>>>>>>> rohini.adi...@gmail.com> wrote: > >>>>>>>>>> > >>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was > >>>>>> working > >>>>>>>>> on > >>>>>>>>>>> and which might take time to review to 4.4.0 > >>>>>>>>>>> > >>>>>>>>>>> Regards, > >>>>>>>>>>> Rohini > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy < > >>>>>>>>>>> rohini.adi...@gmail.com> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Abhishek, > >>>>>>>>>>>> Can you mark all jiras already committed to trunk as 4.3.0 > >>>>>> and > >>>>>>>>> those > >>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter > >> on > >>>>>>>>> those > >>>>>>>>>>>> and review instead of referring to a google doc. I can pick up > >>>>>>> 10-15 > >>>>>>>>>>>> jiras for review end of this week. We should try to get > >>>>>> majority of > >>>>>>>>>>> patches > >>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use > >> the > >>>>>>> 3rd > >>>>>>>>>>> week > >>>>>>>>>>>> for any pending critical issues and rolling the release > >>>>>> candidate. > >>>>>>>>>>>> > >>>>>>>>>>>> Robert, > >>>>>>>>>>>> I am thinking, that we should get rid of the trunk version > and > >>>>>>> use > >>>>>>>>>>>> release numbers when committing from the beginning like all > >>>>>> other > >>>>>>>>>>> projects > >>>>>>>>>>>> do. Unnecessary step to change from trunk to release version > >> for > >>>>>>>>> every > >>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk > >>>>>>>>> version > >>>>>>>>>>>> altogether. Does that require a mail to private? > >>>>>>>>>>>> > >>>>>>>>>>>> Regards, > >>>>>>>>>>>> Rohini > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna < > >>>>>>>>> bafna.i...@gmail.com> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>> > >>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document < > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G > >>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu > >>>>>>>>>>>> > >>>>>>>>>>>>> and listed all the things which we wanted to include in the > >>>>>>> release. > >>>>>>>>>>> In the > >>>>>>>>>>>>> last few days, I spent some time and reviewed some of the > >>>>>> patches > >>>>>>>>> and > >>>>>>>>>>>>> provided my feedback. I will continue my efforts to review > >> more > >>>>>>>>>>> patches. > >>>>>>>>>>>>> These patches needs to be reviewed by committers and get > >>>>>>> committed. > >>>>>>>>> I > >>>>>>>>>>>>> request other contributors also, review some of the patches > >> and > >>>>>>>>> provide > >>>>>>>>>>>>> their feedback. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed > >>>>>> (approximate) > >>>>>>>>>>>>> release date and start working accordingly. Because, If we > >> wait > >>>>>>> for > >>>>>>>>>>> all the > >>>>>>>>>>>>> listed items to be reviewed and committed, then it might take > >> a > >>>>>>> lot > >>>>>>>>>>> more > >>>>>>>>>>>>> time. I am thinking we should target third week of August to > >>>>>>>>> complete > >>>>>>>>>>> and > >>>>>>>>>>>>> release. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Feel free to provide your feedback and suggestions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna < > >>>>>>> bafna.i...@gmail.com > >>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I looked into the available patches and prepared the list. I > >>>>>>> have > >>>>>>>>>>> put, > >>>>>>>>>>>>> scope content which we have discussed till now, into a word > >>>>>>>>> document. > >>>>>>>>>>> (For > >>>>>>>>>>>>> the better formatting and easy to read and comment). > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two > >>>>>> more > >>>>>>>>> Jira > >>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property). > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Please take a look at the document (link below) and provide > >>>>>> your > >>>>>>>>>>>>> comments. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> If this looks good, we can start the release process. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Scope Document: > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G > >>> mFsuHQZmrT63Ow/edit# > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter < > >>>>>>> rkan...@cloudera.com > >>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> That list looks good to me. The only one I'm not sure of > is > >>>>>>>>>>> OOZIE-2306 > >>>>>>>>>>>>>>> (Oozie Health Check). It's a new feature and it might make > >>>>>>>>> sense to > >>>>>>>>>>>>> let it > >>>>>>>>>>>>>>> "bake" a while first? I'm also not sure it's quite ready > >>>>>> yet. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I'd also like to add a few others to the list: > >>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie) > >>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6) > >>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2) > >>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie) > >>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and > >>>>>> other > >>>>>>>>>>> fixes) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your > >>>>>>> list > >>>>>>>>>>>>> (including > >>>>>>>>>>>>>>> the ones I mentioned above). While we don't need to > include > >>>>>>>>>>> everything > >>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review > >>>>>>>>>>> bandwidth), > >>>>>>>>>>>>> did > >>>>>>>>>>>>>>> you consider each of them when creating the list? I > >>>>>> currently > >>>>>>>>> see > >>>>>>>>>>> 97 > >>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available > >>>>>>>>>>>>>>> I think we should include all Spark Action > >>>>>> fixes/improvements, > >>>>>>>>> given > >>>>>>>>>>>>> how > >>>>>>>>>>>>>>> popular it seems to be. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> thanks > >>>>>>>>>>>>>>> - Robert > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna < > >>>>>>>>>>> bafna.i...@gmail.com> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Below is the list of issues, picked from currently > >>>>>> available > >>>>>>>>>>> patches. > >>>>>>>>>>>>> I > >>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is > anything, > >>>>>>>>> please > >>>>>>>>>>>>> reply > >>>>>>>>>>>>>>>> here. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback. > >>>>>>>>> Important > >>>>>>>>>>>>> is, > >>>>>>>>>>>>>>>> how many of these we will be able to review and commit. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I will starting looking into these issues and provide > >>>>>>> feedback. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> ------------------------------------------------------------ > >>> --------------- > >>>>>>>>>>>>>>>> OOZIE-1978 Forkjoin validation code is ridiculously > >>>>>> slow > >>>>>>> in > >>>>>>>>>>> some > >>>>>>>>>>>>> cases > >>>>>>>>>>>>>>>> OOZIE-2508 Documentation change for Coord action > rerun > >>>>>>>>>>>>> [OOZIE-1735] > >>>>>>>>>>>>>>>> OOZIE-2583 oozie throws EL Exception when reference > >>>>>>>>> variable > >>>>>>>>>>> name > >>>>>>>>>>>>>>>> containing dot > >>>>>>>>>>>>>>>> OOZIE-2582 Populating external child Ids for action > >>>>>>>>> failures > >>>>>>>>>>>>>>>> OOZIE-2517 Add support for startCreatedTime and > >>>>>>>>> endCreatedTime > >>>>>>>>>>>>>>>> filters for coord and bundles > >>>>>>>>>>>>>>>> OOZIE-2512 ShareLibservice returns incorrect path for > >>>>>> jar > >>>>>>>>>>>>>>>> OOZIE-2572 SLA DURATION miss not shown when job is > >>>>>>> running > >>>>>>>>> for > >>>>>>>>>>>>> longer > >>>>>>>>>>>>>>>> than expected time > >>>>>>>>>>>>>>>> OOZIE-2457 Oozie log parsing regex consume more than > >>>>>> 90% > >>>>>>>>> cpu > >>>>>>>>>>>>>>>> OOZIE-2574 Oozie to support replication-enabled mysql > >>>>>>> urls > >>>>>>>>>>>>>>>> OOZIE-2506 Add logs into RecoverService for logging > >>>>>>>>>>> information > >>>>>>>>>>>>> about > >>>>>>>>>>>>>>>> queued commnads > >>>>>>>>>>>>>>>> OOZIE-2573 dataset url which contains spaces can not > >>>>>> be > >>>>>>>>>>> handled > >>>>>>>>>>>>>>>> rightly > >>>>>>>>>>>>>>>> OOZIE-2569 Adding yarn-site, core-site, hdfs-site and > >>>>>>>>>>> mapred-site > >>>>>>>>>>>>>>>> into spark launcher > >>>>>>>>>>>>>>>> OOZIE-2491 oozie acl cannot specify group,it does`t > >>>>>> work > >>>>>>>>>>>>>>>> OOZIE-2539 Incorrect property key is used for 'hive > >>>>>> log4j > >>>>>>>>>>>>>>>> configuration file for execution mode' > >>>>>>>>>>>>>>>> OOZIE-2555 Oozie SSL enable setup does not return > port > >>>>>>> for > >>>>>>>>>>> admin > >>>>>>>>>>>>>>>> -servers > >>>>>>>>>>>>>>>> OOZIE-2515 Duplicate information for "Changing > >>>>>>>>>>> endtime/pausetime > >>>>>>>>>>>>> of a > >>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki > >>>>>>>>>>>>>>>> OOZIE-2498 Oozie CallerId configuration for > downstream > >>>>>>>>>>> components > >>>>>>>>>>>>>>>> OOZIE-2516 Update web service documentation for jobs > >>>>>> API > >>>>>>>>>>>>>>>> OOZIE-2500 -DtestJarSimple option mentioned in > >>>>>> minioozie > >>>>>>>>> doc > >>>>>>>>>>>>> does not > >>>>>>>>>>>>>>>> work > >>>>>>>>>>>>>>>> OOZIE-2513 log.scan.duration should not be used for > >>>>>> error > >>>>>>>>> and > >>>>>>>>>>>>> audit > >>>>>>>>>>>>>>>> logs > >>>>>>>>>>>>>>>> OOZIE-2461 Workflow, Coordinator and Bundle job > >>>>>> querying > >>>>>>>>>>> should > >>>>>>>>>>>>> have > >>>>>>>>>>>>>>>> last modified filter > >>>>>>>>>>>>>>>> OOZIE-2306 Oozie health check > >>>>>>>>>>>>>>>> OOZIE-2518 Oozie is creating many files and > >>>>>> directories > >>>>>>> in > >>>>>>>>>>> /tmp > >>>>>>>>>>>>> per > >>>>>>>>>>>>>>>> day > >>>>>>>>>>>>>>>> OOZIE-2409 hive2 action with hive 1.2.1 failed > >>>>>>>>>>>>>>>> OOZIE-2473 Connection pool for SMTP connection > >>>>>>>>>>>>>>>> OOZIE-2440 Exponential re-try policy for workflow > >>>>>> action > >>>>>>>>>>>>>>>> OOZIE-2458 'oozie-setup.sh sharelib create' should > >>>>>> ensure > >>>>>>>>>>>>> uploaded > >>>>>>>>>>>>>>>> jars are world readable > >>>>>>>>>>>>>>>> OOZIE-2390 Rerun with failed option removing > completed > >>>>>>>>> output > >>>>>>>>>>>>> data > >>>>>>>>>>>>>>>> OOZIE-1810 Workflow cannot get into Failed state when > >>>>>>> kill > >>>>>>>>>>>>> control > >>>>>>>>>>>>>>>> node cannot resolve variable in message > >>>>>>>>>>>>>>>> OOZIE-2244 Oozie should mask passwords in the logs > >>>>>> when > >>>>>>>>>>> logging > >>>>>>>>>>>>>>>> command arguments > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> ------------------------------------------------------------ > >>> --------------- > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna < > >>>>>>>>> bafna.i...@gmail.com > >>>>>>>>>>>> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good > >>>>>> to > >>>>>>> me > >>>>>>>>> and > >>>>>>>>>>>>>>>> should be included into the release. If there are things, > >>>>>>> which > >>>>>>>>>>> needs > >>>>>>>>>>>>> to be > >>>>>>>>>>>>>>>> excluded, provide your comments. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> We still need to decide on the patch which are in > progress > >>>>>>>>> status. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On the upgrade side: > >>>>>>>>>>>>>>>>> Java version 1.6 to 1.7 > >>>>>>>>>>>>>>>>> OpenJPA version 2.2.2 to 2.4.1 > >>>>>>>>>>>>>>>>> Curator version 2.5.0 to 2.11.0 > >>>>>>>>>>>>>>>>> Pig version 0.12.1 to 0.16.0 > >>>>>>>>>>>>>>>>> Hive version 0.13.1 to 1.2.0 > >>>>>>>>>>>>>>>>> HttpClient version 4.3 to 4.3.6 > >>>>>>>>>>>>>>>>> commons-httpclient version 3.1 to 4.3.6 (discontinued) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start > >>>>>>>>> working > >>>>>>>>>>> on > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>> completing/testing the release. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna < > >>>>>>>>>>> bafna.i...@gmail.com > >>>>>>>>>>>>>>>> <mailto:bafna.i...@gmail.com>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator, > >>>>>>>>> HttpClient, > >>>>>>>>>>> Pig > >>>>>>>>>>>>> etc. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done > >>>>>>>>> internally as > >>>>>>>>>>>>> well. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 < > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, > curator > >>>>>>>>> version > >>>>>>>>>>>>> also > >>>>>>>>>>>>>>>> can be upgraded to 2.11. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter < > >>>>>>>>>>> rkan...@cloudera.com > >>>>>>>>>>>>>>>> <mailto:rkan...@cloudera.com>> wrote: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0 > >>>>>> for > >>>>>>>>> over > >>>>>>>>>>> a > >>>>>>>>>>>>> year > >>>>>>>>>>>>>>>> now > >>>>>>>>>>>>>>>>>>> with no problems. We did have to comment out a few > >>>>>> lines > >>>>>>> in > >>>>>>>>>>>>>>>> MiniHCatServer > >>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> diff --git > >>>>>>>>>>>>>>>> > >>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>>>>>>>>>>>>>>>>>> > >>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644 > >>>>>>>>>>>>>>>>>>> --- > >>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>>>>>>>>>>>>>>>>>> +++ > >>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer { > >>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws > >>>>>>>>> Exception { > >>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not be > >>>>>>> created > >>>>>>>>>>> for > >>>>>>>>>>>>>>>> local > >>>>>>>>>>>>>>>>>>> metastores. > >>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with > >>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database > >>>>>> named > >>>>>>>>>>> default) > >>>>>>>>>>>>>>>>>>> - Field declaredField = > >>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >>>>>>>>>>>>>>>>>>> - declaredField.setAccessible(true); > >>>>>>>>>>>>>>>>>>> - declaredField.set(null, false); > >>>>>>>>>>>>>>>>>>> + // CLOUDERA-BUILD: Don't need this for Hive > >>>>>> 0.14 > >>>>>>> and > >>>>>>>>>>> later > >>>>>>>>>>>>>>>>>>> + //Field declaredField = > >>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >>>>>>>>>>>>>>>>>>> + //declaredField.setAccessible(true); > >>>>>>>>>>>>>>>>>>> + //declaredField.set(null, false); > >>>>>>>>>>>>>>>>>>> } > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables() > >>>>>> throws > >>>>>>>>>>>>> Exception > >>>>>>>>>>>>>>>> { > >>>>>>>>>>>>>>>>>>> diff --git > >>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>>>>>>>>>>>>>>>>>> > >>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644 > >>>>>>>>>>>>>>>>>>> --- > >>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>>>>>>>>>>>>>>>>>> +++ > >>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends > >>>>>>>>>>> AbstractHiveService > >>>>>>>>>>>>> { > >>>>>>>>>>>>>>>>>>> > >>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, > >>>>>>>>>>>>> getHost()); > >>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars. > HIVE_SERVER2_THRIFT_PORT, > >>>>>>>>>>>>>>>> getBinaryPort()); > >>>>>>>>>>>>>>>>>>> > >>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, > >>>>>>>>>>>>>>>>>>> getHttpPort()); > >>>>>>>>>>>>>>>>>>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >>>>>>>>>>>>>>>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 > >>>>>> and > >>>>>>>>> later > >>>>>>>>>>>>>>>>>>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch"); > >>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir); > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < > >>>>>>>>>>>>>>>>>>> vranganat...@hortonworks.com <mailto: > >>>>>>>>>>> vranganat...@hortonworks.com > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also > >>>>>> to > >>>>>>>>> 1.x. > >>>>>>>>>>>>> 0.13 > >>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>> old > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Venkat > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" < > >>>>>>> rkan...@cloudera.com > >>>>>>>>>>>>> <mailto: > >>>>>>>>>>>>>>>> rkan...@cloudera.com>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere > >>>>>>> about > >>>>>>>>>>> when > >>>>>>>>>>>>>>>> it's > >>>>>>>>>>>>>>>>>>>> okay > >>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we > >>>>>> need > >>>>>>> to > >>>>>>>>>>> wait > >>>>>>>>>>>>>>>> until > >>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>> next major release (5.0)? That said, Hadoop did this > >>>>>> in > >>>>>>> a > >>>>>>>>>>> minor > >>>>>>>>>>>>>>>>>>>> release > >>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013. So I'm fine with doing > >>>>>> it in > >>>>>>>>> 4.3 > >>>>>>>>>>> if > >>>>>>>>>>>>>>>>>>>> nobody has > >>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that > >>>>>> out > >>>>>>> in > >>>>>>>>>>> our > >>>>>>>>>>>>>>>> release > >>>>>>>>>>>>>>>>>>>> notes. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we > >>>>>> should > >>>>>>>>> leave > >>>>>>>>>>> out > >>>>>>>>>>>>>>>> other > >>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently > >>>>>> in > >>>>>>>>> trunk > >>>>>>>>>>>>>>>> anyway. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we > >>>>>> should > >>>>>>>>> make > >>>>>>>>>>>>> sure > >>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>> include all of these and any pending ones. > >>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more > >>>>>>>>> time to > >>>>>>>>>>>>> bake > >>>>>>>>>>>>>>>>>>>> that we > >>>>>>>>>>>>>>>>>>>> should hold off on including in the release? > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> thanks > >>>>>>>>>>>>>>>>>>>> - Robert > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna < > >>>>>>>>>>>>>>>> bafna.i...@gmail.com <mailto:bafna.i...@gmail.com>> > >>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the > >>>>>> release? > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna < > >>>>>>>>>>>>> bafna.i...@gmail.com > >>>>>>>>>>>>>>>> <mailto:bafna.i...@gmail.com>> > >>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie > >>>>>> 4.3.0. > >>>>>>>>> This > >>>>>>>>>>>>> thread > >>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please > >>>>>>>>> provide > >>>>>>>>>>>>> your > >>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any > >>>>>>>>> exclusions) > >>>>>>>>>>> in > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>> release. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to > >>>>>>> trunk. > >>>>>>>>>>> Based > >>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required. > >>>>>>>>>>>>>>>>>>>>>> Version Upgrade > >>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 < > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 < > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>) > >>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 < > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 < > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>), > >>>>>>>>>>>>>>>>>>>>>> Curator (OOZIE-2231 < > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 < > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231> > >>>>>>>>>>>>>>>>>>>>>> ), > >>>>>>>>>>>>>>>>>>>>>> HttpClient (OOZIE-2538 < > >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 < > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>), > >>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 < > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 < > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602> > >>>>>>>>>>>>>>>>>>>>> ). > >>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN, > >>>>>>> Database > >>>>>>>>>>>>> schema > >>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade > >>>>>> etc. > >>>>>>>>>>>>>>>>>>>>>> Bug Fixes. > >>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the > >>>>>> above > >>>>>>>>>>> points > >>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some > >>>>>> work > >>>>>>>>>>> which > >>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>> already > >>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the > >>>>>>>>>>>>> availability > >>>>>>>>>>>>>>>>>>>> of the > >>>>>>>>>>>>>>>>>>>>> reviewers). > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>>> Abhishek > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>> > >>> > >> > > > > > >
