Both approaches look quite "hacky" to me.
Thank you,
Vlad
On 11/23/16 00:01, Mohit Jotwani wrote:
+1 - Approach 2
Regards,
Mohit
On Wed, Nov 23, 2016 at 12:35 PM, AJAY GUPTA <ajaygit...@gmail.com> wrote:
+1 for approach 2.
Regards,
Ajay
On Wed, Nov 23, 2016 at 12:16 PM, David Yan <da...@datatorrent.com> wrote:
The goal of this log4j appender is to provide a log offset and the fixed
name of the container log file (instead of apex.log becoming apex.log.1
and
then apex.log.2, etc due to rotation) as part of an error STRAM event so
users can easily locate the log entries around the error.
The user can override the appender, but in that case, the engine detects
that and will not include the log location as part of the STRAM event.
David
On Tue, Nov 22, 2016 at 7:10 PM, Priyanka Gugale <
priya...@datatorrent.com
wrote:
Hi,
Thomas,
Yes log4j is ultimately owned by user, and they should be able to
override
it. What I am trying to do is provide a default behavior for Apex. In
case
user isn't using any logger of their own we should use this new
appender
of
Apex rather than using standard log4j appender as per hadoop config.
Sanjay,
Archetype is the good place to put this and I will add it there, but
many
time people won't use it. So I wanted to keep it at ~/.dt as well. Is
there
any other default config folder for Apex?
Also I am not relying on anything. If we fail to find config in app jar
or
~/.dt we are going to skip usage of this new appender.
-Priyanka
On Wed, Nov 23, 2016 at 5:58 AM, Sanjay Pujare <san...@datatorrent.com
wrote:
The only way to “enforce” this new appender is to update the
archetypes
(apex-app-archetype and apex-conf-archetype under apex-core/ ) to
use
the
new ones as default. But there does not seem to be a way to enforce
this
for anyone not using the archetypes.
I agree with not relying on ~/.dt in apex-core.
On 11/22/16, 1:08 PM, "Thomas Weise" <t...@apache.org> wrote:
The log4j configuration is ultimately owned by the user, so how
do
you
want
to enforce a custom appender?
I don't think that this should rely on anything in ~/.dt either
Thomas
On Tue, Nov 22, 2016 at 10:00 AM, Priyanka Gugale <
priya...@datatorrent.com>
wrote:
> Hi,
>
> I am working on APEXCORE-563
> <https://issues.apache.org/jira/browse/APEXCORE-563>
> As per this Jira we should put log file name in
container/operator
events.
> The problem is current RollingFileAppender keeps renaming files
from
1 to 2
> to ... n as files reach maximum allowed file size. Because of
constant
> renaming of files we can't put a fixed file name in stram
event.
>
> To overcome this I would like to add a new log4j appender to
ApexCore.
> There are two ways I can implement this:
> 1. Have Daily rolling file appender. The current file will be
recognized
> based on timestamp in file name. Also to control max file size,
we
need to
> keep rolling files based on size as well.
> 2. Have Rolling File Appender but do not rename files. When max
file
size
> is reached create new file with next number e.g. crate log file
dt.log.2
> after dt.log.1 is full. Also to recognize the latest file keep
the
softlink
> named dt.log pointing to current log file.
>
> I would prefer to implement approach 2. Please provide your
> comments/feedback if you feel otherwise.
>
> Also to use this new appender we need to use our custom
log4j.properties
> file instead of one present in hadoop conf. For that we need to
set
jvm
> option -Dlog4j.configuration. I am planning to update file
dt-site.xml in
> folder ~/.dt and default properties file available in apex
archetype to
> set jvm options as follows:
> <property>
> <name>dt.attr.CONTAINER_JVM_OPTIONS</name>
> <value>-Dlog4j.configuration=log4j.props</value>
> </property>
>
> And I will copy log4j.props file in ~/.dt folder as well as
> apex-archetypes.
>
> Lastly if someone still miss this new log4j properties file or
jvm
option
> to set -Dlog4j.configuration we will not put log file name in
event
raised
> by container or operator.
>
> Please provide your feedback on this approach.
>
> -Priyanka
>
