Dear devs,
Currently, for log output, Flink does not explicitly distinguish between
framework logs and user logs. In Task Manager, logs from the framework are
intermixed with the user's business logs. In some deployment models, such
as Standalone or YARN session, there are different task instances
Is that something that can just be done by the right logging framework and
configuration?
Like having a log framework with two targets, one filtered on
"org.apache.flink" and the other one filtered on "my.company.project" or so?
On Fri, Mar 1, 2019 at 3:44 AM vino yang wrote:
> Hi Jamie Grier,
From what I understand this isn't about logging Flink/user messages to
different files, but log everything relevant to a specific job to a
separate file (including what is being logged in runtime classes, i.e.
Tasks, Operators etc.)
On 04/07/2019 12:37, Stephan Ewen wrote:
Is that something t
Hi Stephan,
Thanks for your reply.
In some cases, your solution can take effects.
However, in some scenarios, it does not meet the requirement:
- One program has multiple job instances;
- If we make Flink as a platform, we can not know the package of the
users' program to config the l
I think maybe if I understood this correctly this design is going in the
wrong direction. The problem with Flink logging, when you are running
multiple jobs in the same TMs, is not just about separating out the
business level logging into separate files. The Flink framework itself
logs many thing
Hi Jamie Grier,
Thank you for your reply, let me add some explanations to this design.
First of all, as stated in "Goal", it is mainly for the "Standalone"
cluster model, although we have implemented it for Flink on YARN, this does
not mean that we can't turn off this feature by means of options.