Aaron Gresch created STORM-3724: ----------------------------------- Summary: Use blobstore dir modtime to avoid update lookups by HDFSBlobstore Key: STORM-3724 URL: https://issues.apache.org/jira/browse/STORM-3724 Project: Apache Storm Issue Type: Improvement Reporter: Aaron Gresch Assignee: Aaron Gresch
We have multiple storm clusters with 100's of supervisors polling for blob updates. This causes high load on our Hadoop namenodes that are also used by multiple other clusters. An improvement would be for the AsyncLocalizer to check the remote blobstore last mod time once and then skip checking each individual blob if it was already checked for the same mod time. -- This message was sent by Atlassian Jira (v8.3.4#803005)