Was this directed at me? I am not sure what decision you are referring to. I stated my position in the email you replied to.
Ralph > On Nov 26, 2018, at 5:29 AM, Ferenc Szabo <fsz...@cloudera.com.INVALID> wrote: > > With the new release, you can just replace the jars on the classpath as we > removed every code dependency and using slf4j. So you do not need to fork > and change anything. > > Let me know your decision and we will continue with that. > > Regards, > Ferenc > > On Mon, Nov 26, 2018 at 12:10 PM Apache <ralph.go...@dslextreme.com> wrote: > >> As I said, I am fine with adding a notice to the release notes stating the >> logging will be changed in the next release regardless of its version >> number. This way the revert only needs to happen on the release branch for >> this release. >> >> Ralph >> >>> On Nov 25, 2018, at 9:59 PM, Mike Percy <mpe...@apache.org> wrote: >>> >>> While the committer veto rules are well documented here >>> <https://www.apache.org/foundation/voting.html#Veto>, and there is no >>> mention of a time limit, I propose we shelve that discussion and work on >>> getting to consensus on what we as a community want to include in the >> Flume >>> 1.9 release. >>> >>> As far as I can tell, the decision we have to make is whether to include >>> the FLUME-2050 logging changes in the Flume 1.9 release. We can either: >>> >>> 1) Require users upgrading from Flume 1.8 to Flume 1.9 to have to modify >>> their log4j configuration files to use the different log4j2 XML format, >>> with the procedure for doing so documented in the release notes. >>> -or- >>> 2) Defer that change to a future release where incompatible changes are >>> expected, such as Flume 2.0. >>> >>> Also, maybe there are other options I haven't thought of...? >>> >>> I would like to get some input from more people on this matter. How do >>> others feel about this? >>> >>> Thanks, >>> Mike >>> >>> >>> On Sat, Nov 24, 2018 at 9:36 PM Ralph Goers <ralph.go...@dslextreme.com> >>> wrote: >>> >>>> I should also point out that the time to raise this was a year ago when >>>> the PR for FLUME-2050 was reviewed and committed. As it stands now I >> could >>>> be a jerk and vote -1 on the patch for FLUME-3296 with valid technical >>>> grounds. If this was causing a true binary incompatibility I would >> approve >>>> reverting it in a heartbeat, but I just don’t see how having users have >> to >>>> change logging configuration is “intolerable”, especially with the known >>>> security issues in Log4j 1, however unlikely user’s might be to >> encounter >>>> them. >>>> >>>> That said, I wouldn’t veto the revert on the release branch, but I would >>>> suggest that the release notes provide fair warning that the next >> release >>>> will upgrade the logging dependency. It would also be nice if releases >>>> could be more frequent than once a year. >>>> >>>> I would also like to say that I’m not doing this for the fun of it. My >>>> company uses Flume for some of its most critical processing. We run >>>> Veracode scans on all of our software and I expect Flume would be >> flagged >>>> if I hadn’t repacked it with Log4j 2. It also may not show up since the >> CVE >>>> is marked against Log4j 2 since Log4j 1 is EOL, but the security >> scanning >>>> tools should be flagging that as well. >>>> >>>> FWIW I’ve had to hack or enhance a few Flume components to make it work >>>> for my needs but overall it works really, really well. I’d like to >> commit >>>> back some of the changes but the main one - Flume configuration - I >> really >>>> don’t like and need to redo the whole thing. >>>> >>>> Ralph >>>> >>>>> On Nov 24, 2018, at 5:11 PM, Mike Percy <mpe...@apache.org> wrote: >>>>> >>>>> I wasn't aware of this security issue. Do you have a link to the >> details? >>>>> >>>>> There's no ASF requirement for package names; it's really just a Java >>>>> language convention, and especially not enforced for compatibility >> shims. >>>>> Even Flume has some of those in the legacy sources. >>>>> >>>>> I understand that it's very difficult to provide a compatibility layer. >>>>> Maybe also a boring task. I'm just saying that without log4j1 backwards >>>>> compatibility provided by log4j2, there will have to be a really >> critical >>>>> reason to inflict this migration pain on Flume users -- something that >>>>> simply can't be tolerated, even in a minor release, like a new and >> highly >>>>> dangerous security bug. Without such a motivation, I don't see how this >>>>> incompatible dependency change can be justified. >>>>> >>>>> Mike >>>>> >>>>> On Sat, Nov 24, 2018 at 3:20 PM Ralph Goers < >> ralph.go...@dslextreme.com> >>>>> wrote: >>>>> >>>>>> >>>>>>> On Nov 24, 2018, at 2:35 PM, Mike Percy <mpe...@apache.org> wrote: >>>>>>> >>>>>>> Flume has long had a policy of backwards compatibility with its own >>>>>>> configuration files and people expect things to "just work" when >>>>>> upgrading >>>>>>> Flume. If log4j2 can't parse the log4j1 config file format then it's >> an >>>>>>> incompatible upgrade and should not be done in a minor Flume release. >>>>>>> >>>>>>> If log4j2 wants to be a drop-in replacement for log4j1 then by >> default >>>> it >>>>>>> should find and parse the traditional log4j.properties config files, >> at >>>>>>> least as a fallback, rather than force users to convert to the new >> XML >>>>>>> format before upgrading. >>>>>> >>>>>> That is simply not possible: >>>>>> 1. Log4j 1 requires the use of fully qualified class names. The >> package >>>>>> names log4j 1 used didn’t conform to ASF naming guidelines and don’t >>>> line >>>>>> up with the package names used in Log4j2. >>>>>> 2. Log4j 1 did not delineate between what was public and what was >>>> private >>>>>> so there is code all over the place mucking with the internals of >> Log4j. >>>>>> Log4j 2 implemented a compatibility layer by implementing the classes >>>> that >>>>>> are being used but by and large they don’t actually do anything. >>>>>> 3. The Appenders and Filters in Log4j 2 implement different interfaces >>>> and >>>>>> cannot use components from Log4j 1. >>>>>> 4. The Appenders in Log4j 2 are not identical with Log4j 1. In fact, >>>> many >>>>>> people didn’t use the Rolling File Appender that shipped with Log4j >> but >>>>>> used the one from Log4j extras. So it is hard to.know what Log4j 2 >> would >>>>>> have needed to be compatible with. >>>>>> >>>>>> Many organizations (including mine) have security requirements that >> say >>>>>> they must use software that is supported with security fixes. Log4j 1 >>>> has a >>>>>> known security bug that will never be fixed as it reached end-of-life >> in >>>>>> August of 2015. This means the use of Log4j 1 is not acceptable for >> any >>>>>> security conscious organization. >>>>>> >>>>>> While folks have brought up this issue from time to time most seem to >>>>>> adapt quite quickly to the change. Most of the issues are developers >>>>>> wanting to know how to make something they were doing in Log4j 1 work >> in >>>>>> Log4j 2. >>>>>> >>>>>> As you can imagine, since Log4j 2 has been GA now for 4 1/2 years and >>>>>> Log4j 1 has been EOL for over 3, making the configuration compatible >>>> with >>>>>> Log4j 1 is not a high priority. FWIW, there was an effort to do that a >>>> few >>>>>> years ago, and while we were able to get some basic stuff to work >>>> making it >>>>>> work in a general way wasn’t possible. >>>>>> >>>>>> Ralph >>>>>> >>>>>>> >>>>>>> Mike >>>>>>> >>>>>>> On Fri, Nov 23, 2018 at 9:39 AM Ralph Goers < >>>> ralph.go...@dslextreme.com> >>>>>>> wrote: >>>>>>> >>>>>>>> No, you are correct. However, requiring a change to logging >>>>>> configuration >>>>>>>> has never been considered a binary compatibility break in any >> project >>>> I >>>>>>>> have ever worked on. >>>>>>>> >>>>>>>> Ralph >>>>>>>> >>>>>>>>> On Nov 23, 2018, at 10:05 AM, Ferenc Szabo >>>> <fsz...@cloudera.com.INVALID >>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> As you mentioned, you have the freedom to use Log4j 2 and the same >>>> time >>>>>>>> we >>>>>>>>> have to keep the out of the box experience the same in a minor >>>> version. >>>>>>>>> Users should be able to upgrade flume without changing any of their >>>>>>>>> configurations. >>>>>>>>> If they have a log4j.properties (Log4j 1) then they would not be >> able >>>>>> to >>>>>>>>> use it after the upgrade without changing it. >>>>>>>>> >>>>>>>>> Or am I missing a feature that would solve this case? >>>>>>>>> >>>>>>>>> On Fri, Nov 23, 2018 at 5:49 PM Ralph Goers < >>>>>> ralph.go...@dslextreme.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Also, please put details in the Jira issues. It is much easier to >>>> find >>>>>>>> out >>>>>>>>>> why something was done by searching Jira later on then searching >> the >>>>>>>>>> mailing list. >>>>>>>>>> >>>>>>>>>> Ralph >>>>>>>>>> >>>>>>>>>>> On Nov 23, 2018, at 9:47 AM, Ralph Goers < >>>> ralph.go...@dslextreme.com >>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I do not understand this at all. Log4j 2 provides runtime >>>>>> compatibility >>>>>>>>>> with Log4j 1. What is the problem that requires a revert? >>>>>>>>>>> >>>>>>>>>>> I have been running Flume with Log4j 2 since 1.6 so I don’t >>>>>> understand >>>>>>>>>> what the problem could possibly be. >>>>>>>>>>> >>>>>>>>>>> Ralph >>>>>>>>>>> >>>>>>>>>>>> On Nov 23, 2018, at 8:50 AM, Ferenc Szabo < >> szabofe...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi everyone >>>>>>>>>>>> >>>>>>>>>>>> I am about to branch the 1.9 release from trunk. >>>>>>>>>>>> >>>>>>>>>>>> On the 1.9 branch we will revert the following breaking changes: >>>>>>>>>>>> - FLUME-2957. Remove Guava from our public API: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> https://github.com/apache/flume/commit/7f85df9e473ee675d461d5b76650694c5a6c0088 >>>>>>>>>>>> - part of FLUME-2050. Upgrade to Log4j 2.10.0: >>>>>>>>>>>> as the new release should work with the previous configurations >> we >>>>>>>> have >>>>>>>>>>>> to release it with log4j 1.x >>>>>>>>>>>> For the log4j2 upgrade, we will provide a guide, how to replace >>>> the >>>>>>>> jars >>>>>>>>>>>> if users would like to start using it in the 1.9 release on the >>>> wiki >>>>>>>>>> page. >>>>>>>>>>>> >>>>>>>>>>>> Because of these changes the first release candidate might be >>>>>>>> postponed >>>>>>>>>> to >>>>>>>>>>>> Monday. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Ferenc Szabo >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Nov 7, 2018 at 5:04 PM ema...@cloudera.com < >>>>>>>> ema...@cloudera.com >>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ferenc, >>>>>>>>>>>>> >>>>>>>>>>>>> +1 >>>>>>>>>>>>> I am working on FLUME-3281 Update to Kafka 2.0 client, should >> be >>>>>> able >>>>>>>>>> to >>>>>>>>>>>>> finish it >>>>>>>>>>>>> till the suggested deadline. >>>>>>>>>>>>> I am also happy to do some reviews. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards >>>>>>>>>>>>> Endre >>>>>>>>>>>>> >>>>>>>>>>>>> On 2018/11/06 21:23:17, Ferenc Szabo <szabofe...@apache.org> >>>>>> wrote: >>>>>>>>>>>>>> Hello Flume Community, >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1.8 was released about a year ago and since that quite a few >> bug >>>>>>>>>> fixes, >>>>>>>>>>>>>> improvements, features and documentation were introduced. >>>>>>>>>>>>>> I would like to propose to publish the next minor release of >>>> Flume >>>>>>>>>>>>>> to make these changes available to the users. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would be more than happy to be the Release Manager with the >>>> help >>>>>>>> of >>>>>>>>>>>>>> Denes Arvay for anything that requires PMC access - if both >> the >>>>>>>>>> community >>>>>>>>>>>>>> and he are >>>>>>>>>>>>>> OK with it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Among others the following changes will be included in the >> next >>>>>>>>>> release: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Fixed bugs: >>>>>>>>>>>>>> - FLUME-3117 Application can be dead loop when call >>>> System.exit() >>>>>> in >>>>>>>>>>>>>> methodconfigure >>>>>>>>>>>>>> - FLUME-3237 Handling RuntimeExceptions coming from the JMS >>>>>> provider >>>>>>>>>> in >>>>>>>>>>>>>> JMSSource >>>>>>>>>>>>>> - FLUME-3201 Fix SyslogUtil to handle RFC3164 format in >> december >>>>>>>>>>>>> correctly >>>>>>>>>>>>>> - FLUME-3056 TestApplication hangs indefinitely >>>>>>>>>>>>>> - FLUME-2976 Exception when JMS source tries to connect to a >>>>>>>> Weblogic >>>>>>>>>>>>>> server without authentication >>>>>>>>>>>>>> - FLUME-3270 Close JMS resources in JMSMessageConsumer >>>> constructor >>>>>>>> in >>>>>>>>>>>>> case >>>>>>>>>>>>>> of failure >>>>>>>>>>>>>> - FLUME-3222 java.nio.file.NoSuchFileException thrown when >> files >>>>>> are >>>>>>>>>>>>> being >>>>>>>>>>>>>> deleted from the TAILDIR source >>>>>>>>>>>>>> - FLUME-2894 Flume components should stop in the correct order >>>>>>>>>> (graceful >>>>>>>>>>>>>> shutdown) >>>>>>>>>>>>>> - FLUME-2973 Deadlock in hdfs sink >>>>>>>>>>>>>> - FLUME-3278 Handling -D keystore parameters in Kafka >> components >>>>>>>>>>>>>> - FLUME-3265 Cannot set batch-size for LoadBalancingRpcClient >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Improvements: >>>>>>>>>>>>>> - FLUME-3186 Make asyncHbaseClient configuration parameters >>>>>>>> available >>>>>>>>>>>>> from >>>>>>>>>>>>>> flume config >>>>>>>>>>>>>> - FLUME-3239 Do not rename files in SpoolDirectorySource >>>>>>>>>>>>>> - FLUME-3227 Add Rate Limiter to StressSource >>>>>>>>>>>>>> - FLUME-2050 Upgrade to log4j2 (when GA) >>>>>>>>>>>>>> - FLUME-3246 Validate flume configuration to prevent larger >>>> source >>>>>>>>>> batch >>>>>>>>>>>>>> size than the channel transaction capacity >>>>>>>>>>>>>> - FLUME-3050 add counters for error conditions and expose to >>>>>> monitor >>>>>>>>>> URL >>>>>>>>>>>>>> - FLUME-3269 Support JSSE keystore/trustore -D system >> properties >>>>>>>>>>>>>> - FLUME-3223 Flume HDFS Sink should retry close prior to >>>>>> performing >>>>>>>> a >>>>>>>>>>>>>> recoverLease >>>>>>>>>>>>>> - FLUME-3276 Components supporting SSL/TLS should be able to >>>>>> specify >>>>>>>>>>>>> cipher >>>>>>>>>>>>>> suite list >>>>>>>>>>>>>> - FLUME-3275 Components supporting SSL/TLS should be able to >>>>>> specify >>>>>>>>>>>>>> protocol list >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> New Features: >>>>>>>>>>>>>> - FLUME-3142 Adding HBase2 sink >>>>>>>>>>>>>> - FLUME-2442 Need an alternative to providing clear text >>>> passwords >>>>>>>> in >>>>>>>>>>>>> flume >>>>>>>>>>>>>> config >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> There are 26 open tickets targeted for 1.9 in patch available >>>>>> state: >>>>>>>>>>>>>> https://s.apache.org/flume-1.9-target-tickets >>>>>>>>>>>>>> >>>>>>>>>>>>>> We also have a number of open pull requests on Github: >>>>>>>>>>>>>> https://github.com/apache/flume/pulls >>>>>>>>>>>>>> >>>>>>>>>>>>>> There are a few tickets in progress that would be good to >>>> include >>>>>> in >>>>>>>>>> the >>>>>>>>>>>>>> release so >>>>>>>>>>>>>> I would say that we focus on reviewing and testing them in the >>>>>> next >>>>>>>> 2 >>>>>>>>>>>>>> weeks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would like to propose to target the 23rd of November for the >>>>>> first >>>>>>>>>>>>>> release candidate. >>>>>>>>>>>>>> That would mean that the branch date would be a few days >> before >>>>>>>> that. >>>>>>>>>>>>>> Any significant code change should get in by that date. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If nobody has any concerns then I am going to create an >> umbrella >>>>>>>>>> ticket >>>>>>>>>>>>> to >>>>>>>>>>>>>> track the release process. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Some movement can be expected in the related JIRA tickets. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Kind regards, >>>>>>>>>>>>>> Ferenc >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >> >> >>
