Ok, I’m fine with that. Ralph
> On Nov 30, 2018, at 3:31 PM, Mike Percy <mpe...@apache.org> wrote: > > Hi Ralph, > Let’s update the release notes to indicate that it is coming and do it in the > next release so people are forewarned of the change. After speaking with > others it sounds like Flume users are willing to accept the change (some have > already had to do it because of Solr) so I have changed my view regarding > whether we should wait for Flume 2.0 to do it. I’m now on board doing the > log4j2 switch in a 1.10 release if that is what we want to release next time. > > Mike > >> On Nov 30, 2018, at 2:20 PM, Ralph Goers <ralph.go...@dslextreme.com> wrote: >> >> Mike, >> >> This response confuses me. Are you saying that because the customers are >> good with the config change Flume 1.9 can go out with Log4j 2 or are you >> saying you are good with updating the release notes to say it will happen in >> the next release? >> >> Ralph >> >>> On Nov 29, 2018, at 10:53 PM, Mike Percy <mpe...@apache.org> wrote: >>> >>> After speaking with a couple of large users of Flume, because log4j2 is ABI >>> compatible with log4j1 It sounds like the config change will be acceptable >>> to them at upgrade time and so I am +1 on this proposal. >>> >>> Regards, >>> Mike >>> >>>> On Nov 28, 2018, at 5:37 AM, Ralph Goers <ralph.go...@dslextreme.com> >>>> wrote: >>>> >>>> Again, please update the release notes to indicate the logging >>>> configuration change will be coming with the next release. >>>> >>>> Ralph >>>> >>>>> On Nov 28, 2018, at 1:17 AM, Denes Arvay <de...@cloudera.com.INVALID> >>>>> wrote: >>>>> >>>>> Hi All, >>>>> >>>>> I agree that the current content doesn't justify the 2.0 release, so I'd >>>>> vote for moving forward with the 1.9 as Ferenc recommended, i.e. without >>>>> the 2 previously mentioned breaking changes. >>>>> I also agree with the proposed new features, especially with the plugin >>>>> system/modularization, which was discussed on the other thread: >>>>> https://s.apache.org/2nm9 >>>>> >>>>> So, Ferenc, I think you are good to go with the proposed plan, unless >>>>> anybody vetoes. >>>>> >>>>> Regards, >>>>> Denes >>>>> >>>>> On Tue, Nov 27, 2018 at 4:37 PM Ralph Goers <ralph.go...@dslextreme.com> >>>>> wrote: >>>>> >>>>>> While Log4j 2 supports using properties files for configuration it is not >>>>>> syntactically compatible with Log4j 1. However, migrating a Log4j 1 >>>>>> configuration to Log4j 2 isn’t a difficult task. >>>>>> >>>>>> Ralph >>>>>> >>>>>>> On Nov 27, 2018, at 4:38 AM, Tristan Stevens >>>>>> <tris...@cloudera.com.INVALID> wrote: >>>>>>> >>>>>>> I did start thinking a while back about a REST API that could be used >>>>>>> for >>>>>>> configuring Flume, after which you could potentially bolt on a UI. I >>>>>>> stopped work on it because it got tricky around configuring sources (and >>>>>>> the way in which they relate to channels etc). If someone had more time >>>>>> I’d >>>>>>> be happy to push to a repo the work that I did. This could be a really >>>>>>> useful 2.0 feature. >>>>>>> >>>>>>> Regarding move from Log4j1.x to 2, it’d be interesting to see what other >>>>>>> Apache projects, such as Hadoop, Hive etc did around this (in fact, are >>>>>>> they even at 2.x yet?). For me, if you need to change config, it’s not a >>>>>>> minor release, it become major, unless there’s a way you can >>>>>> automatically >>>>>>> migrate config from one to the other. I’ve not checked, but are we sure >>>>>>> that .properties file won’t work with log4j2? >>>>>>> >>>>>>> Tristan >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 26 November 2018 at 19:54:01, Mike Percy (mpe...@apache.org) wrote: >>>>>>> >>>>>>> Yeah, I agree with this, especially classloader isolation would be great >>>>>> to >>>>>>> have as well on the plugin side. >>>>>>> >>>>>>> Mike >>>>>>> >>>>>>> On Mon, Nov 26, 2018 at 11:50 AM Ralph Goers <ralph.go...@dslextreme.com >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>>> I think people want to do more than just upgrade Log4j for a Flume 2.0 >>>>>>>> release. I would like to make configuration much more pluggable for >>>>>>>> example. There was also talk about splitting all the sources, sinks, >>>>>>>> interceptors, etc out of the core module and making them some sort of >>>>>>>> plugin. At this point those are just at the idea stage. >>>>>>>> >>>>>>>>> On Nov 26, 2018, at 12:19 PM, Bessenyei Balázs Donát < >>>>>> bes...@apache.org> >>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I wonder, is there anything blocking us from releasing 2.0 next >>>>>>>>> instead >>>>>>>> of 1.x? >>>>>>>>> >>>>>>>>> >>>>>>>>> Donat >>>>>>>>> >>>>>>>>>> On Mon, Nov 26, 2018 at 6:27 PM Mike Percy <mpe...@apache.org> wrote: >>>>>>>>>> >>>>>>>>>> If only the PMC would be affected by this decision, we could have had >>>>>>>> this >>>>>>>>>> discussion on the PMC private list. But this decision impacts >>>>>>> everybody >>>>>>>>>> that uses Flume. So let's hear from anybody who cares about this, >>>>>>>> including >>>>>>>>>> committers, contributors, and users on whether they are okay with >>>>>>>> switching >>>>>>>>>> to log4j2 in a minor release version, knowing that they will need to >>>>>>>> change >>>>>>>>>> their config files when they upgrade Flume. >>>>>>>>>> >>>>>>>>>> Ferenc, it seems like we will have to ship one or the other with the >>>>>>>> binary >>>>>>>>>> artifacts at release time. It seems to me that we still have to make >>>>>>>>>> a >>>>>>>>>> choice about the built and shipped default, even if both could work >>>>>>>>>> at >>>>>>>>>> runtime, right? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Mike >>>>>>>>>> >>>>>>>>>> On Mon, Nov 26, 2018 at 5:58 AM Ferenc Szabo >>>>>>>> <fsz...@cloudera.com.invalid> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I have directed it to the community, the PMC members. >>>>>>>>>>> I am looking for a decision from the PMC members (including You) on >>>>>>>> whether >>>>>>>>>>> we should continue as planned or not removing log4j2 form the minor >>>>>>>> release >>>>>>>>>>> branch. >>>>>>>>>>> >>>>>>>>>>> On Mon, Nov 26, 2018 at 2:43 PM Ralph Goers < >>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Was this directed at me? I am not sure what decision you are >>>>>>> referring >>>>>>>>>>> to. >>>>>>>>>>>> I stated my position in the email you replied to. >>>>>>>>>>>> >>>>>>>>>>>> Ralph >>>>>>>>>>>> >>>>>>>>>>>>> On Nov 26, 2018, at 5:29 AM, Ferenc Szabo >>>>>>>> <fsz...@cloudera.com.INVALID >>>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> With the new release, you can just replace the jars on the >>>>>>> classpath >>>>>>>> as >>>>>>>>>>>> we >>>>>>>>>>>>> removed every code dependency and using slf4j. So you do not need >>>>>>> to >>>>>>>>>>> fork >>>>>>>>>>>>> and change anything. >>>>>>>>>>>>> >>>>>>>>>>>>> Let me know your decision and we will continue with that. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Ferenc >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Nov 26, 2018 at 12:10 PM Apache < >>>>>> ralph.go...@dslextreme.com> >>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> As I said, I am fine with adding a notice to the release notes >>>>>>>> stating >>>>>>>>>>>> the >>>>>>>>>>>>>> logging will be changed in the next release regardless of its >>>>>>>> version >>>>>>>>>>>>>> number. This way the revert only needs to happen on the release >>>>>>>> branch >>>>>>>>>>>> for >>>>>>>>>>>>>> this release. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Nov 25, 2018, at 9:59 PM, Mike Percy <mpe...@apache.org> >>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> While the committer veto rules are well documented here >>>>>>>>>>>>>>> <https://www.apache.org/foundation/voting.html#Veto>, and there >>>>>>> is >>>>>>>>>>> no >>>>>>>>>>>>>>> mention of a time limit, I propose we shelve that discussion and >>>>>>>> work >>>>>>>>>>>> on >>>>>>>>>>>>>>> getting to consensus on what we as a community want to include >>>>>>>>>>>>>>> in >>>>>>>> the >>>>>>>>>>>>>> Flume >>>>>>>>>>>>>>> 1.9 release. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As far as I can tell, the decision we have to make is whether to >>>>>>>>>>>> include >>>>>>>>>>>>>>> the FLUME-2050 logging changes in the Flume 1.9 release. We can >>>>>>>>>>> either: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1) Require users upgrading from Flume 1.8 to Flume 1.9 to have >>>>>>>>>>>>>>> to >>>>>>>>>>>> modify >>>>>>>>>>>>>>> their log4j configuration files to use the different log4j2 XML >>>>>>>>>>> format, >>>>>>>>>>>>>>> with the procedure for doing so documented in the release notes. >>>>>>>>>>>>>>> -or- >>>>>>>>>>>>>>> 2) Defer that change to a future release where incompatible >>>>>>> changes >>>>>>>>>>> are >>>>>>>>>>>>>>> expected, such as Flume 2.0. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also, maybe there are other options I haven't thought of...? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I would like to get some input from more people on this matter. >>>>>>> How >>>>>>>>>>> do >>>>>>>>>>>>>>> others feel about this? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Nov 24, 2018 at 9:36 PM Ralph Goers < >>>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I should also point out that the time to raise this was a year >>>>>>> ago >>>>>>>>>>>> when >>>>>>>>>>>>>>>> the PR for FLUME-2050 was reviewed and committed. As it stands >>>>>>>> now I >>>>>>>>>>>>>> could >>>>>>>>>>>>>>>> be a jerk and vote -1 on the patch for FLUME-3296 with valid >>>>>>>>>>> technical >>>>>>>>>>>>>>>> grounds. If this was causing a true binary incompatibility I >>>>>>> would >>>>>>>>>>>>>> approve >>>>>>>>>>>>>>>> reverting it in a heartbeat, but I just don’t see how having >>>>>>> users >>>>>>>>>>>> have >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>> change logging configuration is “intolerable”, especially with >>>>>>> the >>>>>>>>>>>> known >>>>>>>>>>>>>>>> security issues in Log4j 1, however unlikely user’s might be to >>>>>>>>>>>>>> encounter >>>>>>>>>>>>>>>> them. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> That said, I wouldn’t veto the revert on the release branch, >>>>>>>>>>>>>>>> but >>>>>>> I >>>>>>>>>>>> would >>>>>>>>>>>>>>>> suggest that the release notes provide fair warning that the >>>>>>> next >>>>>>>>>>>>>> release >>>>>>>>>>>>>>>> will upgrade the logging dependency. It would also be nice if >>>>>>>>>>>> releases >>>>>>>>>>>>>>>> could be more frequent than once a year. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I would also like to say that I’m not doing this for the fun of >>>>>>>> it. >>>>>>>>>>> My >>>>>>>>>>>>>>>> company uses Flume for some of its most critical processing. We >>>>>>>> run >>>>>>>>>>>>>>>> Veracode scans on all of our software and I expect Flume would >>>>>>> be >>>>>>>>>>>>>> flagged >>>>>>>>>>>>>>>> if I hadn’t repacked it with Log4j 2. It also may not show up >>>>>>>> since >>>>>>>>>>>> the >>>>>>>>>>>>>> CVE >>>>>>>>>>>>>>>> is marked against Log4j 2 since Log4j 1 is EOL, but the >>>>>>>>>>>>>>>> security >>>>>>>>>>>>>> scanning >>>>>>>>>>>>>>>> tools should be flagging that as well. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> FWIW I’ve had to hack or enhance a few Flume components to make >>>>>>> it >>>>>>>>>>>> work >>>>>>>>>>>>>>>> for my needs but overall it works really, really well. I’d like >>>>>>> to >>>>>>>>>>>>>> commit >>>>>>>>>>>>>>>> back some of the changes but the main one - Flume configuration >>>>>>> - >>>>>>>> I >>>>>>>>>>>>>> really >>>>>>>>>>>>>>>> don’t like and need to redo the whole thing. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Nov 24, 2018, at 5:11 PM, Mike Percy <mpe...@apache.org> >>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I wasn't aware of this security issue. Do you have a link to >>>>>>> the >>>>>>>>>>>>>> details? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> There's no ASF requirement for package names; it's really just >>>>>>> a >>>>>>>>>>> Java >>>>>>>>>>>>>>>>> language convention, and especially not enforced for >>>>>>>> compatibility >>>>>>>>>>>>>> shims. >>>>>>>>>>>>>>>>> Even Flume has some of those in the legacy sources. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I understand that it's very difficult to provide a >>>>>>> compatibility >>>>>>>>>>>> layer. >>>>>>>>>>>>>>>>> Maybe also a boring task. I'm just saying that without log4j1 >>>>>>>>>>>> backwards >>>>>>>>>>>>>>>>> compatibility provided by log4j2, there will have to be a >>>>>>> really >>>>>>>>>>>>>> critical >>>>>>>>>>>>>>>>> reason to inflict this migration pain on Flume users -- >>>>>>> something >>>>>>>>>>>> that >>>>>>>>>>>>>>>>> simply can't be tolerated, even in a minor release, like a new >>>>>>>> and >>>>>>>>>>>>>> highly >>>>>>>>>>>>>>>>> dangerous security bug. Without such a motivation, I don't see >>>>>>>> how >>>>>>>>>>>> this >>>>>>>>>>>>>>>>> incompatible dependency change can be justified. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Sat, Nov 24, 2018 at 3:20 PM Ralph Goers < >>>>>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Nov 24, 2018, at 2:35 PM, Mike Percy <mpe...@apache.org> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Flume has long had a policy of backwards compatibility with >>>>>>> its >>>>>>>>>>> own >>>>>>>>>>>>>>>>>>> configuration files and people expect things to "just work" >>>>>>>> when >>>>>>>>>>>>>>>>>> upgrading >>>>>>>>>>>>>>>>>>> Flume. If log4j2 can't parse the log4j1 config file format >>>>>>> then >>>>>>>>>>>> it's >>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>> incompatible upgrade and should not be done in a minor Flume >>>>>>>>>>>> release. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If log4j2 wants to be a drop-in replacement for log4j1 then >>>>>>> by >>>>>>>>>>>>>> default >>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>> should find and parse the traditional log4j.properties >>>>>>>>>>>>>>>>>>> config >>>>>>>>>>>> files, >>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> least as a fallback, rather than force users to convert to >>>>>>> the >>>>>>>>>>> new >>>>>>>>>>>>>> XML >>>>>>>>>>>>>>>>>>> format before upgrading. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> That is simply not possible: >>>>>>>>>>>>>>>>>> 1. Log4j 1 requires the use of fully qualified class names. >>>>>>> The >>>>>>>>>>>>>> package >>>>>>>>>>>>>>>>>> names log4j 1 used didn’t conform to ASF naming guidelines >>>>>>>>>>>>>>>>>> and >>>>>>>>>>> don’t >>>>>>>>>>>>>>>> line >>>>>>>>>>>>>>>>>> up with the package names used in Log4j2. >>>>>>>>>>>>>>>>>> 2. Log4j 1 did not delineate between what was public and what >>>>>>>> was >>>>>>>>>>>>>>>> private >>>>>>>>>>>>>>>>>> so there is code all over the place mucking with the >>>>>>>>>>>>>>>>>> internals >>>>>>>> of >>>>>>>>>>>>>> Log4j. >>>>>>>>>>>>>>>>>> Log4j 2 implemented a compatibility layer by implementing the >>>>>>>>>>>> classes >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>> are being used but by and large they don’t actually do >>>>>>> anything. >>>>>>>>>>>>>>>>>> 3. The Appenders and Filters in Log4j 2 implement different >>>>>>>>>>>> interfaces >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>> cannot use components from Log4j 1. >>>>>>>>>>>>>>>>>> 4. The Appenders in Log4j 2 are not identical with Log4j 1. >>>>>>>>>>>>>>>>>> In >>>>>>>>>>> fact, >>>>>>>>>>>>>>>> many >>>>>>>>>>>>>>>>>> people didn’t use the Rolling File Appender that shipped with >>>>>>>>>>> Log4j >>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>> used the one from Log4j extras. So it is hard to.know what >>>>>>>> Log4j 2 >>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>> have needed to be compatible with. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Many organizations (including mine) have security >>>>>>>>>>>>>>>>>> requirements >>>>>>>>>>> that >>>>>>>>>>>>>> say >>>>>>>>>>>>>>>>>> they must use software that is supported with security fixes. >>>>>>>>>>> Log4j >>>>>>>>>>>> 1 >>>>>>>>>>>>>>>> has a >>>>>>>>>>>>>>>>>> known security bug that will never be fixed as it reached >>>>>>>>>>>> end-of-life >>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>> August of 2015. This means the use of Log4j 1 is not >>>>>>> acceptable >>>>>>>>>>> for >>>>>>>>>>>>>> any >>>>>>>>>>>>>>>>>> security conscious organization. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> While folks have brought up this issue from time to time most >>>>>>>> seem >>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> adapt quite quickly to the change. Most of the issues are >>>>>>>>>>> developers >>>>>>>>>>>>>>>>>> wanting to know how to make something they were doing in >>>>>>>>>>>>>>>>>> Log4j >>>>>>> 1 >>>>>>>>>>>> work >>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>> Log4j 2. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> As you can imagine, since Log4j 2 has been GA now for 4 1/2 >>>>>>>> years >>>>>>>>>>>> and >>>>>>>>>>>>>>>>>> Log4j 1 has been EOL for over 3, making the configuration >>>>>>>>>>> compatible >>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>> Log4j 1 is not a high priority. FWIW, there was an effort to >>>>>>> do >>>>>>>>>>>> that a >>>>>>>>>>>>>>>> few >>>>>>>>>>>>>>>>>> years ago, and while we were able to get some basic stuff to >>>>>>>> work >>>>>>>>>>>>>>>> making it >>>>>>>>>>>>>>>>>> work in a general way wasn’t possible. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Fri, Nov 23, 2018 at 9:39 AM Ralph Goers < >>>>>>>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> No, you are correct. However, requiring a change to logging >>>>>>>>>>>>>>>>>> configuration >>>>>>>>>>>>>>>>>>>> has never been considered a binary compatibility break in >>>>>>> any >>>>>>>>>>>>>> project >>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>> have ever worked on. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 10:05 AM, Ferenc Szabo >>>>>>>>>>>>>>>> <fsz...@cloudera.com.INVALID >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> As you mentioned, you have the freedom to use Log4j 2 and >>>>>>> the >>>>>>>>>>>> same >>>>>>>>>>>>>>>> time >>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>> have to keep the out of the box experience the same in a >>>>>>>> minor >>>>>>>>>>>>>>>> version. >>>>>>>>>>>>>>>>>>>>> Users should be able to upgrade flume without changing any >>>>>>> of >>>>>>>>>>>> their >>>>>>>>>>>>>>>>>>>>> configurations. >>>>>>>>>>>>>>>>>>>>> If they have a log4j.properties (Log4j 1) then they would >>>>>>> not >>>>>>>>>>> be >>>>>>>>>>>>>> able >>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>> use it after the upgrade without changing it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Or am I missing a feature that would solve this case? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Fri, Nov 23, 2018 at 5:49 PM Ralph Goers < >>>>>>>>>>>>>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Also, please put details in the Jira issues. It is much >>>>>>>> easier >>>>>>>>>>>> to >>>>>>>>>>>>>>>> find >>>>>>>>>>>>>>>>>>>> out >>>>>>>>>>>>>>>>>>>>>> why something was done by searching Jira later on then >>>>>>>>>>> searching >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> mailing list. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 9:47 AM, Ralph Goers < >>>>>>>>>>>>>>>> ralph.go...@dslextreme.com >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I do not understand this at all. Log4j 2 provides >>>>>>>>>>>>>>>>>>>>>>> runtime >>>>>>>>>>>>>>>>>> compatibility >>>>>>>>>>>>>>>>>>>>>> with Log4j 1. What is the problem that requires a revert? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I have been running Flume with Log4j 2 since 1.6 so I >>>>>>> don’t >>>>>>>>>>>>>>>>>> understand >>>>>>>>>>>>>>>>>>>>>> what the problem could possibly be. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Nov 23, 2018, at 8:50 AM, Ferenc Szabo < >>>>>>>>>>>>>> szabofe...@apache.org> >>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi everyone >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I am about to branch the 1.9 release from trunk. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On the 1.9 branch we will revert the following breaking >>>>>>>>>>>> changes: >>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2957. Remove Guava from our public API: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> https://github.com/apache/flume/commit/7f85df9e473ee675d461d5b76650694c5a6c0088 >>>>>>>>>>>>>>>>>>>>>>>> - part of FLUME-2050. Upgrade to Log4j 2.10.0: >>>>>>>>>>>>>>>>>>>>>>>> as the new release should work with the previous >>>>>>>>>>>> configurations >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>>>>> to release it with log4j 1.x >>>>>>>>>>>>>>>>>>>>>>>> For the log4j2 upgrade, we will provide a guide, how to >>>>>>>>>>>> replace >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> jars >>>>>>>>>>>>>>>>>>>>>>>> if users would like to start using it in the 1.9 >>>>>>>>>>>>>>>>>>>>>>>> release >>>>>>>> on >>>>>>>>>>>> the >>>>>>>>>>>>>>>> wiki >>>>>>>>>>>>>>>>>>>>>> page. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Because of these changes the first release candidate >>>>>>> might >>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>> postponed >>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>> Monday. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>>>>> Ferenc Szabo >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Nov 7, 2018 at 5:04 PM ema...@cloudera.com < >>>>>>>>>>>>>>>>>>>> ema...@cloudera.com >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Ferenc, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> +1 >>>>>>>>>>>>>>>>>>>>>>>>> I am working on FLUME-3281 Update to Kafka 2.0 client, >>>>>>>>>>> should >>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>> able >>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>> finish it >>>>>>>>>>>>>>>>>>>>>>>>> till the suggested deadline. >>>>>>>>>>>>>>>>>>>>>>>>> I am also happy to do some reviews. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Regards >>>>>>>>>>>>>>>>>>>>>>>>> Endre >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2018/11/06 21:23:17, Ferenc Szabo < >>>>>>>>>>> szabofe...@apache.org> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hello Flume Community, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1.8 was released about a year ago and since that >>>>>>>>>>>>>>>>>>>>>>>>>> quite >>>>>>> a >>>>>>>>>>> few >>>>>>>>>>>>>> bug >>>>>>>>>>>>>>>>>>>>>> fixes, >>>>>>>>>>>>>>>>>>>>>>>>>> improvements, features and documentation were >>>>>>>> introduced. >>>>>>>>>>>>>>>>>>>>>>>>>> I would like to propose to publish the next minor >>>>>>>> release >>>>>>>>>>> of >>>>>>>>>>>>>>>> Flume >>>>>>>>>>>>>>>>>>>>>>>>>> to make these changes available to the users. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I would be more than happy to be the Release Manager >>>>>>>> with >>>>>>>>>>>> the >>>>>>>>>>>>>>>> help >>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>> Denes Arvay for anything that requires PMC access - >>>>>>>>>>>>>>>>>>>>>>>>>> if >>>>>>>>>>> both >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> community >>>>>>>>>>>>>>>>>>>>>>>>>> and he are >>>>>>>>>>>>>>>>>>>>>>>>>> OK with it. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Among others the following changes will be included >>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>> the >>>>>>>>>>>>>> next >>>>>>>>>>>>>>>>>>>>>> release: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Fixed bugs: >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3117 Application can be dead loop when call >>>>>>>>>>>>>>>> System.exit() >>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>> methodconfigure >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3237 Handling RuntimeExceptions coming from >>>>>>> the >>>>>>>>>>> JMS >>>>>>>>>>>>>>>>>> provider >>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>> JMSSource >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3201 Fix SyslogUtil to handle RFC3164 format >>>>>>> in >>>>>>>>>>>>>> december >>>>>>>>>>>>>>>>>>>>>>>>> correctly >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3056 TestApplication hangs indefinitely >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2976 Exception when JMS source tries to >>>>>>> connect >>>>>>>>>>> to a >>>>>>>>>>>>>>>>>>>> Weblogic >>>>>>>>>>>>>>>>>>>>>>>>>> server without authentication >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3270 Close JMS resources in >>>>>>>>>>>>>>>>>>>>>>>>>> JMSMessageConsumer >>>>>>>>>>>>>>>> constructor >>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>> case >>>>>>>>>>>>>>>>>>>>>>>>>> of failure >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3222 java.nio.file.NoSuchFileException thrown >>>>>>>> when >>>>>>>>>>>>>> files >>>>>>>>>>>>>>>>>> are >>>>>>>>>>>>>>>>>>>>>>>>> being >>>>>>>>>>>>>>>>>>>>>>>>>> deleted from the TAILDIR source >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2894 Flume components should stop in the >>>>>>> correct >>>>>>>>>>>> order >>>>>>>>>>>>>>>>>>>>>> (graceful >>>>>>>>>>>>>>>>>>>>>>>>>> shutdown) >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2973 Deadlock in hdfs sink >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3278 Handling -D keystore parameters in Kafka >>>>>>>>>>>>>> components >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3265 Cannot set batch-size for >>>>>>>>>>>> LoadBalancingRpcClient >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Improvements: >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3186 Make asyncHbaseClient configuration >>>>>>>>>>> parameters >>>>>>>>>>>>>>>>>>>> available >>>>>>>>>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>>>>>>>>> flume config >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3239 Do not rename files in >>>>>>> SpoolDirectorySource >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3227 Add Rate Limiter to StressSource >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2050 Upgrade to log4j2 (when GA) >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3246 Validate flume configuration to prevent >>>>>>>>>>> larger >>>>>>>>>>>>>>>> source >>>>>>>>>>>>>>>>>>>>>> batch >>>>>>>>>>>>>>>>>>>>>>>>>> size than the channel transaction capacity >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3050 add counters for error conditions and >>>>>>>> expose >>>>>>>>>>> to >>>>>>>>>>>>>>>>>> monitor >>>>>>>>>>>>>>>>>>>>>> URL >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3269 Support JSSE keystore/trustore -D system >>>>>>>>>>>>>> properties >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3223 Flume HDFS Sink should retry close prior >>>>>>> to >>>>>>>>>>>>>>>>>> performing >>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>> recoverLease >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3276 Components supporting SSL/TLS should be >>>>>>>> able >>>>>>>>>>> to >>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>>>>>>> cipher >>>>>>>>>>>>>>>>>>>>>>>>>> suite list >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3275 Components supporting SSL/TLS should be >>>>>>>> able >>>>>>>>>>> to >>>>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>>>>>>>> protocol list >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> New Features: >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-3142 Adding HBase2 sink >>>>>>>>>>>>>>>>>>>>>>>>>> - FLUME-2442 Need an alternative to providing clear >>>>>>> text >>>>>>>>>>>>>>>> passwords >>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>> flume >>>>>>>>>>>>>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> There are 26 open tickets targeted for 1.9 in patch >>>>>>>>>>>> available >>>>>>>>>>>>>>>>>> state: >>>>>>>>>>>>>>>>>>>>>>>>>> https://s.apache.org/flume-1.9-target-tickets >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> We also have a number of open pull requests on >>>>>>>>>>>>>>>>>>>>>>>>>> Github: >>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/flume/pulls >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> There are a few tickets in progress that would be >>>>>>>>>>>>>>>>>>>>>>>>>> good >>>>>>>> to >>>>>>>>>>>>>>>> include >>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>> release so >>>>>>>>>>>>>>>>>>>>>>>>>> I would say that we focus on reviewing and testing >>>>>>> them >>>>>>>> in >>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> next >>>>>>>>>>>>>>>>>>>> 2 >>>>>>>>>>>>>>>>>>>>>>>>>> weeks. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I would like to propose to target the 23rd of >>>>>>>>>>>>>>>>>>>>>>>>>> November >>>>>>>> for >>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> first >>>>>>>>>>>>>>>>>>>>>>>>>> release candidate. >>>>>>>>>>>>>>>>>>>>>>>>>> That would mean that the branch date would be a few >>>>>>> days >>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>>>>> Any significant code change should get in by that >>>>>>> date. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> If nobody has any concerns then I am going to create >>>>>>> an >>>>>>>>>>>>>> umbrella >>>>>>>>>>>>>>>>>>>>>> ticket >>>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>> track the release process. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Some movement can be expected in the related JIRA >>>>>>>> tickets. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Kind regards, >>>>>>>>>>>>>>>>>>>>>>>>>> Ferenc >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >>> >>> >> >> > >
