[
https://issues.apache.org/jira/browse/YARN-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159102#comment-15159102
]
Jason Lowe commented on YARN-4705:
----------------------------------
Another workaround could be a scheme to get a raw local filesystem instead of
one with a CRC and use that for the unit tests, e.g.: something like rawfile://
instead of file://. Then flushes would work as ATS1.5 expects.
Agree that we could move the errors encountered during "live" file reads to
debug to avoid scaring users and document that ATS1.5 relies on flush actually
flushing which means it can't be used with the crc'd local filesystem.
> ATS 1.5 parse pipeline to consider handling open() events recoverably
> ---------------------------------------------------------------------
>
> Key: YARN-4705
> URL: https://issues.apache.org/jira/browse/YARN-4705
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Priority: Minor
>
> During one of my own timeline test runs, I've been seeing a stack trace
> warning that the CRC check failed in Filesystem.open() file; something the FS
> was ignoring.
> Even though its swallowed (and probably not the cause of my test failure),
> looking at the code in {{LogInfo.parsePath()}} that it considers a failure to
> open a file as unrecoverable.
> on some filesystems, this may not be the case, i.e. if its open for writing
> it may not be available for reading; checksums maybe a similar issue.
> Perhaps a failure at open() should be viewed as recoverable while the app is
> still running?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
