Ankit Sharma created FLINK-11324:
------------------------------------
Summary: Custom Log4j properties for each flink job
Key: FLINK-11324
URL: https://issues.apache.org/jira/browse/FLINK-11324
Project: Flink
Issue Type: Improvement
Components: Flink on Tez
Affects Versions: 1.6.2
Reporter: Ankit Sharma
It would be a good idea if we can have a custom log4j files/location for each
job we run via flink.
Currently there is no option for that in flink command.
I tried providing log4j properties by using below command but it didn't worked
for me:
flink run /usr/lib/flink/examples/batch/WordCount.jar
-Dlog4j.configuration=~/log4j_ABC.properties
So, i checked the source code of flink and found that the path of log4j
proerpties are hard-coded in flink command script, so did few twik to make it
work. It worked for me but i am really not sure about what other components it
might affect.
Below are the steps/changes done:
1. Add below code in $FLINK_HOME/bin/flink file just after the config.sh being
called, this code will add a new environment variable FLINK_LOG_SETTING in
which we will be setting the log4j-cli.properties absolute path:
if [ -z "$FLINK_LOG_SETTING" ]; then FLINK_LOG_SETTING=$FLINK_CONF_DIR; fi
2. In log_setting variable mentioned in flink, change the -Dlog4j.configuration
as below:
log_setting=(-Dlog.file="$log"
-Dlog4j.configuration=file:"$FLINK_LOG_SETTING"/log4j-cli.properties
-Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml)
3. save the changes.
4. Now before executing any flink job in cli, export the FLINK_LOG_SETTING to
any custom path where you have log4j-cli.properties present for the respective
job.
$ export FLINK_LOG_SETTING=/home/hadoop/test
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
