Sure Robert. We can use some more time.
~Abhishek
> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rkan...@cloudera.com> wrote:
>
> Do you think we could wait one more week? I've been really hammered with
> some other work and there's more that I'd like to review here that I
> haven't been able to get to.
>
> thanks
> - Robert
>
> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <bafna.i...@gmail.com>
> wrote:
>
>> Sure. I guess we can one extra week and complete some more work.
>> Thanks everyone for responding.
>>
>> ~Abhishek
>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <purus...@yahoo-inc.com.INVALID>
>> wrote:
>>>
>>> Yes. I also have some pending patches.
>>>
>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>> rkan...@cloudera.com> wrote:
>>>
>>>
>>> Ya, I could use some more time as well. I have some deadlines early next
>>> week, which has kept me from doing more reviews.
>>>
>>> thanks
>>> - Robert
>>>
>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>> jaydeepma...@gmail.com> wrote:
>>>
>>>> Let see if we can wait for another week. So we can merge few more
>> important
>>>> jiras. Few of them I personally want to get reviewed and merged.
>>>>
>>>> Regards,
>>>> Jaydeep
>>>>
>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <bafna.i...@gmail.com>
>>>> wrote:
>>>>
>>>>> As previously discussed, we will be moving with whatever patches can be
>>>>> reviewed/merged till end of this week.
>>>>>
>>>>> But there are some which needs to be reviewed and merged, mainly
>>>> regarding
>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>> should
>>>>> review OOZIE-2036 and merged it.
>>>>>
>>>>> ~Abhishek
>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>> rohini.adi...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Cleaned up that list and now no jira refers to "trunk". Patch
>>>> Available
>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>
>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>
>>>>>> There are 42 jiras to go. We can take a look mid next week to see if
>>>> any
>>>>>> pending ones needs to be pushed out. There are quite a few which are
>>>>>> unassigned.
>>>>>>
>>>>>> If there is any jira, someone would like to include in 4.3.0 please
>>>>> update
>>>>>> the jira.
>>>>>>
>>>>>> Regards,
>>>>>> Rohini
>>>>>>
>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>> rohini.adi...@gmail.com
>>>>>>> wrote:
>>>>>>
>>>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Rohini
>>>>>>>
>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkan...@cloudera.com
>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Ya, let's unset those remaining open trunk ones. I agree, we need
>> to
>>>>> put
>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>>>> JIRAs.
>>>>>>>>
>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator. We can
>>>>> discuss
>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>>>> why.
>>>>>>>>
>>>>>>>> - Robert
>>>>>>>>
>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>> rohini.adi...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>
>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>>>> on
>>>>>>>> these
>>>>>>>>> or mark them for next release? Don't see majority of them being
>>>> worked
>>>>>>>> on
>>>>>>>>> even for next release. We need to do bug scrub and close many of
>>>> these
>>>>>>>> some
>>>>>>>>> time.
>>>>>>>>>
>>>>>>>>>> Instead
>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>> the
>>>>>>>>> release number.
>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>> unsetting
>>>>>>>> fix
>>>>>>>>> version, then we can as well delete trunk. Also, I did create a
>>>> 4.4.0
>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>> rohini.adi...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I think it would be better to send out email for the bulk change.
>>>> If
>>>>>>>>> there
>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>>>> that
>>>>>>>>> with
>>>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>>>> feedback
>>>>>>>>>> on pulling back to release.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>> rkan...@cloudera.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>>>> it
>>>>>>>> not
>>>>>>>>>>> to
>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>> rkan...@cloudera.com
>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Rohini, that sounds good to me. Especially because we're on
>>>>>>>> "master"
>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>> confusing.
>>>>>>>>>>> Instead
>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
>> to
>>>>>>>> the
>>>>>>>>>>>> release number. Otherwise, we'll have to re-label any current
>>>>>>>> "trunk"
>>>>>>>>>>>> fields. We can use 4.4.0 for now, but I imagine that the next
>>>>>>>> release
>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>>>> rename it
>>>>>>>>>>>> later.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>>>> should
>>>>>>>>>>>> probably have a vote in the private list to make things official
>>>> as
>>>>>>>>>>> this is
>>>>>>>>>>>> a release-management-related change. Can you send an email?
>>>>>>>>>>>>
>>>>>>>>>>>> Abhishek, thanks for putting this all together. I think
>> Rohini's
>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step. You're
>>>>>>>>>>> probably
>>>>>>>>>>>> best off using the bulk change feature instead of doing each
>> JIRA
>>>>>>>>>>>> individually. I should have time to do a bunch of reviews this
>>>>>>>> week,
>>>>>>>>>>> I've
>>>>>>>>>>>> actually already started. I should have some time to continue
>>>>>>>> doing
>>>>>>>>>>> more
>>>>>>>>>>>> reviews. As a general thing, we've been pretty slow about
>>>> getting
>>>>>>>>> these
>>>>>>>>>>>> done :(
>>>>>>>>>>>>
>>>>>>>>>>>> - Robert
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>> rohini.adi...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>>>> working
>>>>>>>>>>> on
>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>> rohini.adi...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as 4.3.0
>>>>>>>> and
>>>>>>>>>>> those
>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>>>> on
>>>>>>>>>>> those
>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>>>> 10-15
>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>> majority of
>>>>>>>>>>>>> patches
>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>>>> the
>>>>>>>>> 3rd
>>>>>>>>>>>>> week
>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>> candidate.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk version
>> and
>>>>>>>>> use
>>>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>>>> other
>>>>>>>>>>>>> projects
>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>>>> for
>>>>>>>>>>> every
>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>>>> version
>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>> bafna.i...@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>>>> release.
>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>>>> patches
>>>>>>>>>>> and
>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>>>> more
>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>>>> committed.
>>>>>>>>>>> I
>>>>>>>>>>>>>>> request other contributors also, review some of the patches
>>>> and
>>>>>>>>>>> provide
>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>> (approximate)
>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>>>> wait
>>>>>>>>> for
>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>>>> a
>>>>>>>>> lot
>>>>>>>>>>>>> more
>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>>>> complete
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>> bafna.i...@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>>>> have
>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>>>> document.
>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>>>> more
>>>>>>>>>>> Jira
>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>>>> your
>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>> rkan...@cloudera.com
>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> That list looks good to me. The only one I'm not sure of
>> is
>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>> (Oozie Health Check). It's a new feature and it might make
>>>>>>>>>>> sense to
>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>> "bake" a while first? I'm also not sure it's quite ready
>>>>>>>> yet.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>>>> other
>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>>>> list
>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>> the ones I mentioned above). While we don't need to
>> include
>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>> you consider each of them when creating the list? I
>>>>>>>> currently
>>>>>>>>>>> see
>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>> fixes/improvements,
>>>>>>>>>>> given
>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.i...@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>> available
>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>> anything,
>>>>>>>>>>> please
>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>>>> Important
>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>> ------------------------------------------------------------
>>>>> ---------------
>>>>>>>>>>>>>>>>>> OOZIE-1978 Forkjoin validation code is ridiculously
>>>>>>>> slow
>>>>>>>>> in
>>>>>>>>>>>>> some
>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>> OOZIE-2508 Documentation change for Coord action
>> rerun
>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>> OOZIE-2583 oozie throws EL Exception when reference
>>>>>>>>>>> variable
>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>> OOZIE-2582 Populating external child Ids for action
>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>> OOZIE-2517 Add support for startCreatedTime and
>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>> OOZIE-2512 ShareLibservice returns incorrect path for
>>>>>>>> jar
>>>>>>>>>>>>>>>>>> OOZIE-2572 SLA DURATION miss not shown when job is
>>>>>>>>> running
>>>>>>>>>>> for
>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>> OOZIE-2457 Oozie log parsing regex consume more than
>>>>>>>> 90%
>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>> OOZIE-2574 Oozie to support replication-enabled mysql
>>>>>>>>> urls
>>>>>>>>>>>>>>>>>> OOZIE-2506 Add logs into RecoverService for logging
>>>>>>>>>>>>> information
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>> OOZIE-2573 dataset url which contains spaces can not
>>>>>>>> be
>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>> OOZIE-2569 Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>> OOZIE-2491 oozie acl cannot specify group,it does`t
>>>>>>>> work
>>>>>>>>>>>>>>>>>> OOZIE-2539 Incorrect property key is used for 'hive
>>>>>>>> log4j
>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>> OOZIE-2555 Oozie SSL enable setup does not return
>> port
>>>>>>>>> for
>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>> OOZIE-2515 Duplicate information for "Changing
>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>> OOZIE-2498 Oozie CallerId configuration for
>> downstream
>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>> OOZIE-2516 Update web service documentation for jobs
>>>>>>>> API
>>>>>>>>>>>>>>>>>> OOZIE-2500 -DtestJarSimple option mentioned in
>>>>>>>> minioozie
>>>>>>>>>>> doc
>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>> OOZIE-2513 log.scan.duration should not be used for
>>>>>>>> error
>>>>>>>>>>> and
>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>> OOZIE-2461 Workflow, Coordinator and Bundle job
>>>>>>>> querying
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>> OOZIE-2306 Oozie health check
>>>>>>>>>>>>>>>>>> OOZIE-2518 Oozie is creating many files and
>>>>>>>> directories
>>>>>>>>> in
>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>> OOZIE-2409 hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>> OOZIE-2473 Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>> OOZIE-2440 Exponential re-try policy for workflow
>>>>>>>> action
>>>>>>>>>>>>>>>>>> OOZIE-2458 'oozie-setup.sh sharelib create' should
>>>>>>>> ensure
>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>> OOZIE-2390 Rerun with failed option removing
>> completed
>>>>>>>>>>> output
>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>> OOZIE-1810 Workflow cannot get into Failed state when
>>>>>>>>> kill
>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>> OOZIE-2244 Oozie should mask passwords in the logs
>>>>>>>> when
>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>> ------------------------------------------------------------
>>>>> ---------------
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>> bafna.i...@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>>>> to
>>>>>>>>> me
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>>>> which
>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>> progress
>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>> Java version 1.6 to 1.7
>>>>>>>>>>>>>>>>>>> OpenJPA version 2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>> Curator version 2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>> Pig version 0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>> Hive version 0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>> HttpClient version 4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>> commons-httpclient version 3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>>>> working
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.i...@gmail.com
>>>>>>>>>>>>>>>>>> <mailto:bafna.i...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>>>> internally as
>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>> curator
>>>>>>>>>>> version
>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>> rkan...@cloudera.com
>>>>>>>>>>>>>>>>>> <mailto:rkan...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>>>> for
>>>>>>>>>>> over
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>> with no problems. We did have to comment out a few
>>>>>>>> lines
>>>>>>>>> in
>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>
>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws
>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not be
>>>>>>>>> created
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>>>> named
>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>> - Field declaredField =
>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>> - declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>> - declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>> + // CLOUDERA-BUILD: Don't need this for Hive
>>>>>>>> 0.14
>>>>>>>>> and
>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>> + //Field declaredField =
>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>> + //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>> + //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
>>>>>>>> throws
>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>>>> and
>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>>>> vranganat...@hortonworks.com <mailto:
>>>>>>>>>>>>> vranganat...@hortonworks.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>>>> to
>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>> rkan...@cloudera.com
>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>> rkan...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>>>> about
>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>>>> need
>>>>>>>>> to
>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)? That said, Hadoop did this
>>>>>>>> in
>>>>>>>>> a
>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013. So I'm fine with doing
>>>>>>>> it in
>>>>>>>>>>> 4.3
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>>>> out
>>>>>>>>> in
>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>> should
>>>>>>>>>>> leave
>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>>>> in
>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>>>> should
>>>>>>>>>>> make
>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>>>> time to
>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>> bafna.i...@gmail.com <mailto:bafna.i...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.i...@gmail.com
>>>>>>>>>>>>>>>>>> <mailto:bafna.i...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>>>> 4.3.0.
>>>>>>>>>>> This
>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>>>> provide
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>>>> exclusions)
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>>>> trunk.
>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>>>> Curator (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>> HttpClient (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>>>> Database
>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>>>> above
>>>>>>>>>>>>> points
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>>>> work
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
