After speaking with a couple of large users of Flume, because log4j2 is ABI compatible with log4j1 It sounds like the config change will be acceptable to them at upgrade time and so I am +1 on this proposal.
Regards, Mike > On Nov 28, 2018, at 5:37 AM, Ralph Goers <ralph.go...@dslextreme.com> wrote: > > Again, please update the release notes to indicate the logging configuration > change will be coming with the next release. > > Ralph > >> On Nov 28, 2018, at 1:17 AM, Denes Arvay <de...@cloudera.com.INVALID> wrote: >> >> Hi All, >> >> I agree that the current content doesn't justify the 2.0 release, so I'd >> vote for moving forward with the 1.9 as Ferenc recommended, i.e. without >> the 2 previously mentioned breaking changes. >> I also agree with the proposed new features, especially with the plugin >> system/modularization, which was discussed on the other thread: >> https://s.apache.org/2nm9 >> >> So, Ferenc, I think you are good to go with the proposed plan, unless >> anybody vetoes. >> >> Regards, >> Denes >> >> On Tue, Nov 27, 2018 at 4:37 PM Ralph Goers <ralph.go...@dslextreme.com> >> wrote: >> >>> While Log4j 2 supports using properties files for configuration it is not >>> syntactically compatible with Log4j 1. However, migrating a Log4j 1 >>> configuration to Log4j 2 isn’t a difficult task. >>> >>> Ralph >>> >>>> On Nov 27, 2018, at 4:38 AM, Tristan Stevens >>> <tris...@cloudera.com.INVALID> wrote: >>>> >>>> I did start thinking a while back about a REST API that could be used for >>>> configuring Flume, after which you could potentially bolt on a UI. I >>>> stopped work on it because it got tricky around configuring sources (and >>>> the way in which they relate to channels etc). If someone had more time >>> I’d >>>> be happy to push to a repo the work that I did. This could be a really >>>> useful 2.0 feature. >>>> >>>> Regarding move from Log4j1.x to 2, it’d be interesting to see what other >>>> Apache projects, such as Hadoop, Hive etc did around this (in fact, are >>>> they even at 2.x yet?). For me, if you need to change config, it’s not a >>>> minor release, it become major, unless there’s a way you can >>> automatically >>>> migrate config from one to the other. I’ve not checked, but are we sure >>>> that .properties file won’t work with log4j2? >>>> >>>> Tristan >>>> >>>> >>>> >>>> On 26 November 2018 at 19:54:01, Mike Percy (mpe...@apache.org) wrote: >>>> >>>> Yeah, I agree with this, especially classloader isolation would be great >>> to >>>> have as well on the plugin side. >>>> >>>> Mike >>>> >>>> On Mon, Nov 26, 2018 at 11:50 AM Ralph Goers <ralph.go...@dslextreme.com >>>> >>>> wrote: >>>> >>>>> I think people want to do more than just upgrade Log4j for a Flume 2.0 >>>>> release. I would like to make configuration much more pluggable for >>>>> example. There was also talk about splitting all the sources, sinks, >>>>> interceptors, etc out of the core module and making them some sort of >>>>> plugin. At this point those are just at the idea stage. >>>>> >>>>>> On Nov 26, 2018, at 12:19 PM, Bessenyei Balázs Donát < >>> bes...@apache.org> >>>> >>>>> wrote: >>>>>> >>>>>> I wonder, is there anything blocking us from releasing 2.0 next instead >>>>> of 1.x? >>>>>> >>>>>> >>>>>> Donat >>>>>> >>>>>>> On Mon, Nov 26, 2018 at 6:27 PM Mike Percy <mpe...@apache.org> wrote: >>>>>>> >>>>>>> If only the PMC would be affected by this decision, we could have had >>>>> this >>>>>>> discussion on the PMC private list. But this decision impacts >>>> everybody >>>>>>> that uses Flume. So let's hear from anybody who cares about this, >>>>> including >>>>>>> committers, contributors, and users on whether they are okay with >>>>> switching >>>>>>> to log4j2 in a minor release version, knowing that they will need to >>>>> change >>>>>>> their config files when they upgrade Flume. >>>>>>> >>>>>>> Ferenc, it seems like we will have to ship one or the other with the >>>>> binary >>>>>>> artifacts at release time. It seems to me that we still have to make a >>>>>>> choice about the built and shipped default, even if both could work at >>>>>>> runtime, right? >>>>>>> >>>>>>> Thanks, >>>>>>> Mike >>>>>>> >>>>>>> On Mon, Nov 26, 2018 at 5:58 AM Ferenc Szabo >>>>> <fsz...@cloudera.com.invalid> >>>>>>> wrote: >>>>>>> >>>>>>>> I have directed it to the community, the PMC members. >>>>>>>> I am looking for a decision from the PMC members (including You) on >>>>> whether >>>>>>>> we should continue as planned or not removing log4j2 form the minor >>>>> release >>>>>>>> branch. >>>>>>>> >>>>>>>> On Mon, Nov 26, 2018 at 2:43 PM Ralph Goers < >>>>> ralph.go...@dslextreme.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Was this directed at me? I am not sure what decision you are >>>> referring >>>>>>>> to. >>>>>>>>> I stated my position in the email you replied to. >>>>>>>>> >>>>>>>>> Ralph >>>>>>>>> >>>>>>>>>> On Nov 26, 2018, at 5:29 AM, Ferenc Szabo >>>>> <fsz...@cloudera.com.INVALID >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> With the new release, you can just replace the jars on the >>>> classpath >>>>> as >>>>>>>>> we >>>>>>>>>> removed every code dependency and using slf4j. So you do not need >>>> to >>>>>>>> fork >>>>>>>>>> and change anything. >>>>>>>>>> >>>>>>>>>> Let me know your decision and we will continue with that. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Ferenc >>>>>>>>>> >>>>>>>>>> On Mon, Nov 26, 2018 at 12:10 PM Apache < >>> ralph.go...@dslextreme.com> >>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> As I said, I am fine with adding a notice to the release notes >>>>> stating >>>>>>>>> the >>>>>>>>>>> logging will be changed in the next release regardless of its >>>>> version >>>>>>>>>>> number. This way the revert only needs to happen on the release >>>>> branch >>>>>>>>> for >>>>>>>>>>> this release. >>>>>>>>>>> >>>>>>>>>>> Ralph >>>>>>>>>>> >>>>>>>>>>>> On Nov 25, 2018, at 9:59 PM, Mike Percy <mpe...@apache.org> >>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> While the committer veto rules are well documented here >>>>>>>>>>>> <https://www.apache.org/foundation/voting.html#Veto>, and there >>>> is >>>>>>>> no >>>>>>>>>>>> mention of a time limit, I propose we shelve that discussion and >>>>> work >>>>>>>>> on >>>>>>>>>>>> getting to consensus on what we as a community want to include in >>>>> the >>>>>>>>>>> Flume >>>>>>>>>>>> 1.9 release. >>>>>>>>>>>> >>>>>>>>>>>> As far as I can tell, the decision we have to make is whether to >>>>>>>>> include >>>>>>>>>>>> the FLUME-2050 logging changes in the Flume 1.9 release. We can >>>>>>>> either: >>>>>>>>>>>> >>>>>>>>>>>> 1) Require users upgrading from Flume 1.8 to Flume 1.9 to have to >>>>>>>>> modify >>>>>>>>>>>> their log4j configuration files to use the different log4j2 XML >>>>>>>> format, >>>>>>>>>>>> with the procedure for doing so documented in the release notes. >>>>>>>>>>>> -or- >>>>>>>>>>>> 2) Defer that change to a future release where incompatible >>>> changes >>>>>>>> are >>>>>>>>>>>> expected, such as Flume 2.0. >>>>>>>>>>>> >>>>>>>>>>>> Also, maybe there are other options I haven't thought of...? >>>>>>>>>>>> >>>>>>>>>>>> I would like to get some input from more people on this matter. >>>> How >>>>>>>> do >>>>>>>>>>>> others feel about this? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Mike >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Nov 24, 2018 at 9:36 PM Ralph Goers < >>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I should also point out that the time to raise this was a year >>>> ago >>>>>>>>> when >>>>>>>>>>>>> the PR for FLUME-2050 was reviewed and committed. As it stands >>>>> now I >>>>>>>>>>> could >>>>>>>>>>>>> be a jerk and vote -1 on the patch for FLUME-3296 with valid >>>>>>>> technical >>>>>>>>>>>>> grounds. If this was causing a true binary incompatibility I >>>> would >>>>>>>>>>> approve >>>>>>>>>>>>> reverting it in a heartbeat, but I just don’t see how having >>>> users >>>>>>>>> have >>>>>>>>>>> to >>>>>>>>>>>>> change logging configuration is “intolerable”, especially with >>>> the >>>>>>>>> known >>>>>>>>>>>>> security issues in Log4j 1, however unlikely user’s might be to >>>>>>>>>>> encounter >>>>>>>>>>>>> them. >>>>>>>>>>>>> >>>>>>>>>>>>> That said, I wouldn’t veto the revert on the release branch, but >>>> I >>>>>>>>> would >>>>>>>>>>>>> suggest that the release notes provide fair warning that the >>>> next >>>>>>>>>>> release >>>>>>>>>>>>> will upgrade the logging dependency. It would also be nice if >>>>>>>>> releases >>>>>>>>>>>>> could be more frequent than once a year. >>>>>>>>>>>>> >>>>>>>>>>>>> I would also like to say that I’m not doing this for the fun of >>>>> it. >>>>>>>> My >>>>>>>>>>>>> company uses Flume for some of its most critical processing. We >>>>> run >>>>>>>>>>>>> Veracode scans on all of our software and I expect Flume would >>>> be >>>>>>>>>>> flagged >>>>>>>>>>>>> if I hadn’t repacked it with Log4j 2. It also may not show up >>>>> since >>>>>>>>> the >>>>>>>>>>> CVE >>>>>>>>>>>>> is marked against Log4j 2 since Log4j 1 is EOL, but the security >>>>>>>>>>> scanning >>>>>>>>>>>>> tools should be flagging that as well. >>>>>>>>>>>>> >>>>>>>>>>>>> FWIW I’ve had to hack or enhance a few Flume components to make >>>> it >>>>>>>>> work >>>>>>>>>>>>> for my needs but overall it works really, really well. I’d like >>>> to >>>>>>>>>>> commit >>>>>>>>>>>>> back some of the changes but the main one - Flume configuration >>>> - >>>>> I >>>>>>>>>>> really >>>>>>>>>>>>> don’t like and need to redo the whole thing. >>>>>>>>>>>>> >>>>>>>>>>>>> Ralph >>>>>>>>>>>>> >>>>>>>>>>>>>> On Nov 24, 2018, at 5:11 PM, Mike Percy <mpe...@apache.org> >>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I wasn't aware of this security issue. Do you have a link to >>>> the >>>>>>>>>>> details? >>>>>>>>>>>>>> >>>>>>>>>>>>>> There's no ASF requirement for package names; it's really just >>>> a >>>>>>>> Java >>>>>>>>>>>>>> language convention, and especially not enforced for >>>>> compatibility >>>>>>>>>>> shims. >>>>>>>>>>>>>> Even Flume has some of those in the legacy sources. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I understand that it's very difficult to provide a >>>> compatibility >>>>>>>>> layer. >>>>>>>>>>>>>> Maybe also a boring task. I'm just saying that without log4j1 >>>>>>>>> backwards >>>>>>>>>>>>>> compatibility provided by log4j2, there will have to be a >>>> really >>>>>>>>>>> critical >>>>>>>>>>>>>> reason to inflict this migration pain on Flume users -- >>>> something >>>>>>>>> that >>>>>>>>>>>>>> simply can't be tolerated, even in a minor release, like a new >>>>> and >>>>>>>>>>> highly >>>>>>>>>>>>>> dangerous security bug. Without such a motivation, I don't see >>>>> how >>>>>>>>> this >>>>>>>>>>>>>> incompatible dependency change can be justified. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Mike >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Nov 24, 2018 at 3:20 PM Ralph Goers < >>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Nov 24, 2018, at 2:35 PM, Mike Percy <mpe...@apache.org> >>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Flume has long had a policy of backwards compatibility with >>>> its >>>>>>>> own >>>>>>>>>>>>>>>> configuration files and people expect things to "just work" >>>>> when >>>>>>>>>>>>>>> upgrading >>>>>>>>>>>>>>>> Flume. If log4j2 can't parse the log4j1 config file format >>>> then >>>>>>>>> it's >>>>>>>>>>> an >>>>>>>>>>>>>>>> incompatible upgrade and should not be done in a minor Flume >>>>>>>>> release. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If log4j2 wants to be a drop-in replacement for log4j1 then >>>> by >>>>>>>>>>> default >>>>>>>>>>>>> it >>>>>>>>>>>>>>>> should find and parse the traditional log4j.properties config >>>>>>>>> files, >>>>>>>>>>> at >>>>>>>>>>>>>>>> least as a fallback, rather than force users to convert to >>>> the >>>>>>>> new >>>>>>>>>>> XML >>>>>>>>>>>>>>>> format before upgrading. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> That is simply not possible: >>>>>>>>>>>>>>> 1. Log4j 1 requires the use of fully qualified class names. >>>> The >>>>>>>>>>> package >>>>>>>>>>>>>>> names log4j 1 used didn’t conform to ASF naming guidelines and >>>>>>>> don’t >>>>>>>>>>>>> line >>>>>>>>>>>>>>> up with the package names used in Log4j2. >>>>>>>>>>>>>>> 2. Log4j 1 did not delineate between what was public and what >>>>> was >>>>>>>>>>>>> private >>>>>>>>>>>>>>> so there is code all over the place mucking with the internals >>>>> of >>>>>>>>>>> Log4j. >>>>>>>>>>>>>>> Log4j 2 implemented a compatibility layer by implementing the >>>>>>>>> classes >>>>>>>>>>>>> that >>>>>>>>>>>>>>> are being used but by and large they don’t actually do >>>> anything. >>>>>>>>>>>>>>> 3. The Appenders and Filters in Log4j 2 implement different >>>>>>>>> interfaces >>>>>>>>>>>>> and >>>>>>>>>>>>>>> cannot use components from Log4j 1. >>>>>>>>>>>>>>> 4. The Appenders in Log4j 2 are not identical with Log4j 1. In >>>>>>>> fact, >>>>>>>>>>>>> many >>>>>>>>>>>>>>> people didn’t use the Rolling File Appender that shipped with >>>>>>>> Log4j >>>>>>>>>>> but >>>>>>>>>>>>>>> used the one from Log4j extras. So it is hard to.know what >>>>> Log4j 2 >>>>>>>>>>> would >>>>>>>>>>>>>>> have needed to be compatible with. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Many organizations (including mine) have security requirements >>>>>>>> that >>>>>>>>>>> say >>>>>>>>>>>>>>> they must use software that is supported with security fixes. >>>>>>>> Log4j >>>>>>>>> 1 >>>>>>>>>>>>> has a >>>>>>>>>>>>>>> known security bug that will never be fixed as it reached >>>>>>>>> end-of-life >>>>>>>>>>> in >>>>>>>>>>>>>>> August of 2015. This means the use of Log4j 1 is not >>>> acceptable >>>>>>>> for >>>>>>>>>>> any >>>>>>>>>>>>>>> security conscious organization. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> While folks have brought up this issue from time to time most >>>>> seem >>>>>>>>> to >>>>>>>>>>>>>>> adapt quite quickly to the change. Most of the issues are >>>>>>>> developers >>>>>>>>>>>>>>> wanting to know how to make something they were doing in Log4j >>>> 1 >>>>>>>>> work >>>>>>>>>>> in >>>>>>>>>>>>>>> Log4j 2. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As you can imagine, since Log4j 2 has been GA now for 4 1/2 >>>>> years >>>>>>>>> and >>>>>>>>>>>>>>> Log4j 1 has been EOL for over 3, making the configuration >>>>>>>> compatible >>>>>>>>>>>>> with >>>>>>>>>>>>>>> Log4j 1 is not a high priority. FWIW, there was an effort to >>>> do >>>>>>>>> that a >>>>>>>>>>>>> few >>>>>>>>>>>>>>> years ago, and while we were able to get some basic stuff to >>>>> work >>>>>>>>>>>>> making it >>>>>>>>>>>>>>> work in a general way wasn’t possible. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Nov 23, 2018 at 9:39 AM Ralph Goers < >>>>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> No, you are correct. However, requiring a change to logging >>>>>>>>>>>>>>> configuration >>>>>>>>>>>>>>>>> has never been considered a binary compatibility break in >>>> any >>>>>>>>>>> project >>>>>>>>>>>>> I >>>>>>>>>>>>>>>>> have ever worked on. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 10:05 AM, Ferenc Szabo >>>>>>>>>>>>> <fsz...@cloudera.com.INVALID >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> As you mentioned, you have the freedom to use Log4j 2 and >>>> the >>>>>>>>> same >>>>>>>>>>>>> time >>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>> have to keep the out of the box experience the same in a >>>>> minor >>>>>>>>>>>>> version. >>>>>>>>>>>>>>>>>> Users should be able to upgrade flume without changing any >>>> of >>>>>>>>> their >>>>>>>>>>>>>>>>>> configurations. >>>>>>>>>>>>>>>>>> If they have a log4j.properties (Log4j 1) then they would >>>> not >>>>>>>> be >>>>>>>>>>> able >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> use it after the upgrade without changing it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Or am I missing a feature that would solve this case? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Nov 23, 2018 at 5:49 PM Ralph Goers < >>>>>>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also, please put details in the Jira issues. It is much >>>>> easier >>>>>>>>> to >>>>>>>>>>>>> find >>>>>>>>>>>>>>>>> out >>>>>>>>>>>>>>>>>>> why something was done by searching Jira later on then >>>>>>>> searching >>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> mailing list. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 9:47 AM, Ralph Goers < >>>>>>>>>>>>> ralph.go...@dslextreme.com >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I do not understand this at all. Log4j 2 provides runtime >>>>>>>>>>>>>>> compatibility >>>>>>>>>>>>>>>>>>> with Log4j 1. What is the problem that requires a revert? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I have been running Flume with Log4j 2 since 1.6 so I >>>> don’t >>>>>>>>>>>>>>> understand >>>>>>>>>>>>>>>>>>> what the problem could possibly be. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 8:50 AM, Ferenc Szabo < >>>>>>>>>>> szabofe...@apache.org> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi everyone >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I am about to branch the 1.9 release from trunk. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On the 1.9 branch we will revert the following breaking >>>>>>>>> changes: >>>>>>>>>>>>>>>>>>>>> - FLUME-2957. Remove Guava from our public API: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>> >>>> >>> https://github.com/apache/flume/commit/7f85df9e473ee675d461d5b76650694c5a6c0088 >>>>>>>>>>>>>>>>>>>>> - part of FLUME-2050. Upgrade to Log4j 2.10.0: >>>>>>>>>>>>>>>>>>>>> as the new release should work with the previous >>>>>>>>> configurations >>>>>>>>>>> we >>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>> to release it with log4j 1.x >>>>>>>>>>>>>>>>>>>>> For the log4j2 upgrade, we will provide a guide, how to >>>>>>>>> replace >>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> jars >>>>>>>>>>>>>>>>>>>>> if users would like to start using it in the 1.9 release >>>>> on >>>>>>>>> the >>>>>>>>>>>>> wiki >>>>>>>>>>>>>>>>>>> page. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Because of these changes the first release candidate >>>> might >>>>>>>> be >>>>>>>>>>>>>>>>> postponed >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>> Monday. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>> Ferenc Szabo >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Wed, Nov 7, 2018 at 5:04 PM ema...@cloudera.com < >>>>>>>>>>>>>>>>> ema...@cloudera.com >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi Ferenc, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> +1 >>>>>>>>>>>>>>>>>>>>>> I am working on FLUME-3281 Update to Kafka 2.0 client, >>>>>>>> should >>>>>>>>>>> be >>>>>>>>>>>>>>> able >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>> finish it >>>>>>>>>>>>>>>>>>>>>> till the suggested deadline. >>>>>>>>>>>>>>>>>>>>>> I am also happy to do some reviews. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Regards >>>>>>>>>>>>>>>>>>>>>> Endre >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2018/11/06 21:23:17, Ferenc Szabo < >>>>>>>> szabofe...@apache.org> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> Hello Flume Community, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1.8 was released about a year ago and since that quite >>>> a >>>>>>>> few >>>>>>>>>>> bug >>>>>>>>>>>>>>>>>>> fixes, >>>>>>>>>>>>>>>>>>>>>>> improvements, features and documentation were >>>>> introduced. >>>>>>>>>>>>>>>>>>>>>>> I would like to propose to publish the next minor >>>>> release >>>>>>>> of >>>>>>>>>>>>> Flume >>>>>>>>>>>>>>>>>>>>>>> to make these changes available to the users. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I would be more than happy to be the Release Manager >>>>> with >>>>>>>>> the >>>>>>>>>>>>> help >>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>> Denes Arvay for anything that requires PMC access - if >>>>>>>> both >>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> community >>>>>>>>>>>>>>>>>>>>>>> and he are >>>>>>>>>>>>>>>>>>>>>>> OK with it. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Among others the following changes will be included in >>>>> the >>>>>>>>>>> next >>>>>>>>>>>>>>>>>>> release: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Fixed bugs: >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3117 Application can be dead loop when call >>>>>>>>>>>>> System.exit() >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>> methodconfigure >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3237 Handling RuntimeExceptions coming from >>>> the >>>>>>>> JMS >>>>>>>>>>>>>>> provider >>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>> JMSSource >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3201 Fix SyslogUtil to handle RFC3164 format >>>> in >>>>>>>>>>> december >>>>>>>>>>>>>>>>>>>>>> correctly >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3056 TestApplication hangs indefinitely >>>>>>>>>>>>>>>>>>>>>>> - FLUME-2976 Exception when JMS source tries to >>>> connect >>>>>>>> to a >>>>>>>>>>>>>>>>> Weblogic >>>>>>>>>>>>>>>>>>>>>>> server without authentication >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3270 Close JMS resources in JMSMessageConsumer >>>>>>>>>>>>> constructor >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>> case >>>>>>>>>>>>>>>>>>>>>>> of failure >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3222 java.nio.file.NoSuchFileException thrown >>>>> when >>>>>>>>>>> files >>>>>>>>>>>>>>> are >>>>>>>>>>>>>>>>>>>>>> being >>>>>>>>>>>>>>>>>>>>>>> deleted from the TAILDIR source >>>>>>>>>>>>>>>>>>>>>>> - FLUME-2894 Flume components should stop in the >>>> correct >>>>>>>>> order >>>>>>>>>>>>>>>>>>> (graceful >>>>>>>>>>>>>>>>>>>>>>> shutdown) >>>>>>>>>>>>>>>>>>>>>>> - FLUME-2973 Deadlock in hdfs sink >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3278 Handling -D keystore parameters in Kafka >>>>>>>>>>> components >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3265 Cannot set batch-size for >>>>>>>>> LoadBalancingRpcClient >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Improvements: >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3186 Make asyncHbaseClient configuration >>>>>>>> parameters >>>>>>>>>>>>>>>>> available >>>>>>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>>>>>> flume config >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3239 Do not rename files in >>>> SpoolDirectorySource >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3227 Add Rate Limiter to StressSource >>>>>>>>>>>>>>>>>>>>>>> - FLUME-2050 Upgrade to log4j2 (when GA) >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3246 Validate flume configuration to prevent >>>>>>>> larger >>>>>>>>>>>>> source >>>>>>>>>>>>>>>>>>> batch >>>>>>>>>>>>>>>>>>>>>>> size than the channel transaction capacity >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3050 add counters for error conditions and >>>>> expose >>>>>>>> to >>>>>>>>>>>>>>> monitor >>>>>>>>>>>>>>>>>>> URL >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3269 Support JSSE keystore/trustore -D system >>>>>>>>>>> properties >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3223 Flume HDFS Sink should retry close prior >>>> to >>>>>>>>>>>>>>> performing >>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>> recoverLease >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3276 Components supporting SSL/TLS should be >>>>> able >>>>>>>> to >>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>>>> cipher >>>>>>>>>>>>>>>>>>>>>>> suite list >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3275 Components supporting SSL/TLS should be >>>>> able >>>>>>>> to >>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>>>>> protocol list >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> New Features: >>>>>>>>>>>>>>>>>>>>>>> - FLUME-3142 Adding HBase2 sink >>>>>>>>>>>>>>>>>>>>>>> - FLUME-2442 Need an alternative to providing clear >>>> text >>>>>>>>>>>>> passwords >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>> flume >>>>>>>>>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> There are 26 open tickets targeted for 1.9 in patch >>>>>>>>> available >>>>>>>>>>>>>>> state: >>>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/flume-1.9-target-tickets >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> We also have a number of open pull requests on Github: >>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/flume/pulls >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> There are a few tickets in progress that would be good >>>>> to >>>>>>>>>>>>> include >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> release so >>>>>>>>>>>>>>>>>>>>>>> I would say that we focus on reviewing and testing >>>> them >>>>> in >>>>>>>>> the >>>>>>>>>>>>>>> next >>>>>>>>>>>>>>>>> 2 >>>>>>>>>>>>>>>>>>>>>>> weeks. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I would like to propose to target the 23rd of November >>>>> for >>>>>>>>> the >>>>>>>>>>>>>>> first >>>>>>>>>>>>>>>>>>>>>>> release candidate. >>>>>>>>>>>>>>>>>>>>>>> That would mean that the branch date would be a few >>>> days >>>>>>>>>>> before >>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>> Any significant code change should get in by that >>>> date. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> If nobody has any concerns then I am going to create >>>> an >>>>>>>>>>> umbrella >>>>>>>>>>>>>>>>>>> ticket >>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>> track the release process. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Some movement can be expected in the related JIRA >>>>> tickets. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Kind regards, >>>>>>>>>>>>>>>>>>>>>>> Ferenc >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>>> >>> >>> >>> > >
