Harish Jaiprakash created TEZ-4028: -------------------------------------- Summary: Events not visible from proto history logging for s3a filesystem until dag completes. Key: TEZ-4028 URL: https://issues.apache.org/jira/browse/TEZ-4028 Project: Apache Tez Issue Type: Bug Reporter: Harish Jaiprakash Assignee: Harish Jaiprakash
The events are not visible in the files because s3 filesystem * flush writes to local disk and only upload/commit to s3 on close. * does not support append As an initial fix we log the dag submitted, initialized and started events into a file and these can be read to get the dag plan, config from the AM. The counters are anyways not available until the dag completes. The in-progress information cannot be read, this can be obtained from the AM once we have the above events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)