Flume has long had a policy of backwards compatibility with its own configuration files and people expect things to "just work" when upgrading Flume. If log4j2 can't parse the log4j1 config file format then it's an incompatible upgrade and should not be done in a minor Flume release.
If log4j2 wants to be a drop-in replacement for log4j1 then by default it should find and parse the traditional log4j.properties config files, at least as a fallback, rather than force users to convert to the new XML format before upgrading. Mike On Fri, Nov 23, 2018 at 9:39 AM Ralph Goers <ralph.go...@dslextreme.com> wrote: > No, you are correct. However, requiring a change to logging configuration > has never been considered a binary compatibility break in any project I > have ever worked on. > > Ralph > > > On Nov 23, 2018, at 10:05 AM, Ferenc Szabo <fsz...@cloudera.com.INVALID> > wrote: > > > > As you mentioned, you have the freedom to use Log4j 2 and the same time > we > > have to keep the out of the box experience the same in a minor version. > > Users should be able to upgrade flume without changing any of their > > configurations. > > If they have a log4j.properties (Log4j 1) then they would not be able to > > use it after the upgrade without changing it. > > > > Or am I missing a feature that would solve this case? > > > > On Fri, Nov 23, 2018 at 5:49 PM Ralph Goers <ralph.go...@dslextreme.com> > > wrote: > > > >> Also, please put details in the Jira issues. It is much easier to find > out > >> why something was done by searching Jira later on then searching the > >> mailing list. > >> > >> Ralph > >> > >>> On Nov 23, 2018, at 9:47 AM, Ralph Goers <ralph.go...@dslextreme.com> > >> wrote: > >>> > >>> I do not understand this at all. Log4j 2 provides runtime compatibility > >> with Log4j 1. What is the problem that requires a revert? > >>> > >>> I have been running Flume with Log4j 2 since 1.6 so I don’t understand > >> what the problem could possibly be. > >>> > >>> Ralph > >>> > >>>> On Nov 23, 2018, at 8:50 AM, Ferenc Szabo <szabofe...@apache.org> > >> wrote: > >>>> > >>>> Hi everyone > >>>> > >>>> I am about to branch the 1.9 release from trunk. > >>>> > >>>> On the 1.9 branch we will revert the following breaking changes: > >>>> - FLUME-2957. Remove Guava from our public API: > >>>> > >>>> > >> > https://github.com/apache/flume/commit/7f85df9e473ee675d461d5b76650694c5a6c0088 > >>>> - part of FLUME-2050. Upgrade to Log4j 2.10.0: > >>>> as the new release should work with the previous configurations we > have > >>>> to release it with log4j 1.x > >>>> For the log4j2 upgrade, we will provide a guide, how to replace the > jars > >>>> if users would like to start using it in the 1.9 release on the wiki > >> page. > >>>> > >>>> Because of these changes the first release candidate might be > postponed > >> to > >>>> Monday. > >>>> > >>>> Regards, > >>>> Ferenc Szabo > >>>> > >>>> On Wed, Nov 7, 2018 at 5:04 PM ema...@cloudera.com < > ema...@cloudera.com > >>> > >>>> wrote: > >>>> > >>>>> Hi Ferenc, > >>>>> > >>>>> +1 > >>>>> I am working on FLUME-3281 Update to Kafka 2.0 client, should be able > >> to > >>>>> finish it > >>>>> till the suggested deadline. > >>>>> I am also happy to do some reviews. > >>>>> > >>>>> Regards > >>>>> Endre > >>>>> > >>>>> On 2018/11/06 21:23:17, Ferenc Szabo <szabofe...@apache.org> wrote: > >>>>>> Hello Flume Community, > >>>>>> > >>>>>> 1.8 was released about a year ago and since that quite a few bug > >> fixes, > >>>>>> improvements, features and documentation were introduced. > >>>>>> I would like to propose to publish the next minor release of Flume > >>>>>> to make these changes available to the users. > >>>>>> > >>>>>> I would be more than happy to be the Release Manager with the help > of > >>>>>> Denes Arvay for anything that requires PMC access - if both the > >> community > >>>>>> and he are > >>>>>> OK with it. > >>>>>> > >>>>>> Among others the following changes will be included in the next > >> release: > >>>>>> > >>>>>> Fixed bugs: > >>>>>> - FLUME-3117 Application can be dead loop when call System.exit() in > >>>>>> methodconfigure > >>>>>> - FLUME-3237 Handling RuntimeExceptions coming from the JMS provider > >> in > >>>>>> JMSSource > >>>>>> - FLUME-3201 Fix SyslogUtil to handle RFC3164 format in december > >>>>> correctly > >>>>>> - FLUME-3056 TestApplication hangs indefinitely > >>>>>> - FLUME-2976 Exception when JMS source tries to connect to a > Weblogic > >>>>>> server without authentication > >>>>>> - FLUME-3270 Close JMS resources in JMSMessageConsumer constructor > in > >>>>> case > >>>>>> of failure > >>>>>> - FLUME-3222 java.nio.file.NoSuchFileException thrown when files are > >>>>> being > >>>>>> deleted from the TAILDIR source > >>>>>> - FLUME-2894 Flume components should stop in the correct order > >> (graceful > >>>>>> shutdown) > >>>>>> - FLUME-2973 Deadlock in hdfs sink > >>>>>> - FLUME-3278 Handling -D keystore parameters in Kafka components > >>>>>> - FLUME-3265 Cannot set batch-size for LoadBalancingRpcClient > >>>>>> > >>>>>> > >>>>>> Improvements: > >>>>>> - FLUME-3186 Make asyncHbaseClient configuration parameters > available > >>>>> from > >>>>>> flume config > >>>>>> - FLUME-3239 Do not rename files in SpoolDirectorySource > >>>>>> - FLUME-3227 Add Rate Limiter to StressSource > >>>>>> - FLUME-2050 Upgrade to log4j2 (when GA) > >>>>>> - FLUME-3246 Validate flume configuration to prevent larger source > >> batch > >>>>>> size than the channel transaction capacity > >>>>>> - FLUME-3050 add counters for error conditions and expose to monitor > >> URL > >>>>>> - FLUME-3269 Support JSSE keystore/trustore -D system properties > >>>>>> - FLUME-3223 Flume HDFS Sink should retry close prior to performing > a > >>>>>> recoverLease > >>>>>> - FLUME-3276 Components supporting SSL/TLS should be able to specify > >>>>> cipher > >>>>>> suite list > >>>>>> - FLUME-3275 Components supporting SSL/TLS should be able to specify > >>>>>> protocol list > >>>>>> > >>>>>> > >>>>>> New Features: > >>>>>> - FLUME-3142 Adding HBase2 sink > >>>>>> - FLUME-2442 Need an alternative to providing clear text passwords > in > >>>>> flume > >>>>>> config > >>>>>> > >>>>>> > >>>>>> There are 26 open tickets targeted for 1.9 in patch available state: > >>>>>> https://s.apache.org/flume-1.9-target-tickets > >>>>>> > >>>>>> We also have a number of open pull requests on Github: > >>>>>> https://github.com/apache/flume/pulls > >>>>>> > >>>>>> There are a few tickets in progress that would be good to include in > >> the > >>>>>> release so > >>>>>> I would say that we focus on reviewing and testing them in the next > 2 > >>>>>> weeks. > >>>>>> > >>>>>> I would like to propose to target the 23rd of November for the first > >>>>>> release candidate. > >>>>>> That would mean that the branch date would be a few days before > that. > >>>>>> Any significant code change should get in by that date. > >>>>>> > >>>>>> If nobody has any concerns then I am going to create an umbrella > >> ticket > >>>>> to > >>>>>> track the release process. > >>>>>> > >>>>>> Some movement can be expected in the related JIRA tickets. > >>>>>> > >>>>>> Kind regards, > >>>>>> Ferenc > >>>>>> > >>>>> > >>> > >> > >> > >> > > >
