When you create a EMR cluster you can specify a S3 path where logs will be saved after cluster, something like this:
s3://bucket/j-18ASDKLJLAKSD/containers/application_1494074597524_0001/container_1494074597524_0001_01_000001/stderr.gz http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-manage-view-web-log-files.html On Mon, Aug 28, 2017 at 4:43 PM, Mikhailau, Alex <alex.mikhai...@mlb.com> wrote: > Does anyone have a working solution for logging YARN application id, YARN > container hostname, Executor ID and YARN attempt for jobs running on Spark > EMR 5.7.0 in log statements? Are there specific ENV variables available or > other workflow for doing that? > > > > Thank you > > > > Alex >
