Thanks! I'll check these out. On Tue, Oct 27, 2020 at 12:58 AM Robert Metzger <rmetz...@apache.org> wrote:
> Hi Rex, > > 1. You can also use the Flink UI for retrieving logs. That usually works > quite fast (unless your logs are huge). > > 2. These are the correct configuration files for setting the log level. > Are you running on a vanilla EMR cluster, or are there modifications? The > "problem" is that Flink on YARN adds jar files (and other files) provided > by the environment (YARN) to its classpath. The vanilla EMR configuration > should be fine to not interfere with Flink's logging. But maybe there are > some changes in your environment that cause problems? > > Since you are SSHing into the machines already: At the top of each Flink > log file, we are logging the location of the log4j configuration file > (search for "-Dlog4j.configuration="). Try to open that file to verify > what's in there. > > Hope this helps! > > Robert > > > On Tue, Oct 27, 2020 at 12:03 AM Rex Fenley <r...@remind101.com> wrote: > >> Hello, >> >> After lots of testing in local environments we're now trying to get our >> cluster running on AWS EMR. We followed much of the documentation from both >> AWS and Flink and have gotten to the point of creating a yarn session and >> submitting jobs. We successfully get back a Job ID and in the Yarn Timeline >> Server UI it says our application is running. However, we are having a hard >> time with logging. >> >> 2 main issues: >> 1. Logs for the jobmanager and taskmanager seem to take a long time to >> show up or in some cases just seem to never show up in the Yarn / Hadoop >> UI, even though we can see them just fine when ssh'ing into the cluster's >> nodes. Anything we can do to speed this up? >> >> 2. We can't seem to see anything except for WARN and ERROR logs for the >> jobmanager and taskmanager, we need at least INFO right now to confirm >> things are working as expected. We have been jumping through hoops going >> through a multitude of configuration files including >> log4j-session.properties and log4j.properties setting level to DEBUG but >> it has not helped. Are these the correct configuration files? >> >> Thanks! >> >> -- >> >> Rex Fenley | Software Engineer - Mobile and Backend >> >> >> Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> >> | FOLLOW US <https://twitter.com/remindhq> | LIKE US >> <https://www.facebook.com/remindhq> >> > -- Rex Fenley | Software Engineer - Mobile and Backend Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | FOLLOW US <https://twitter.com/remindhq> | LIKE US <https://www.facebook.com/remindhq>
