[
https://issues.apache.org/jira/browse/TEZ-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171519#comment-14171519
]
Siddharth Seth commented on TEZ-1566:
-------------------------------------
- {code}- mapOutput.getType());
+ decompressedLength + " len: " + compressedLength);{code}
Isn't type useful, and doesn't add too much overhead.
- DataMovementEvent logging. I think this is fairly useful, for figuring out
why a specific task has hung. Instead of moving it completely to debug, I think
we should add a less verbose version at INFO level. This will need to change in
the other InputHandler as well.
One more thing we can look at is to change the default log4j configuration to
not log the entire package hierarchy, and instead log either the classname or
one level of the package hierarchy - that should save quite a bit.
I think there'll be multiple iterations of this spanning mulitple jiras - but
the IFile and DME changes will help a lot to start with.
> Reduce log verbosity
> --------------------
>
> Key: TEZ-1566
> URL: https://issues.apache.org/jira/browse/TEZ-1566
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Siddharth Seth
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1566.1.patch
>
>
> We generate 40+GB of logs for a simple 20K X 1K sleep job, at INFO level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
