Ethanlm commented on a change in pull request #3363:
URL: https://github.com/apache/storm/pull/3363#discussion_r549128188
##########
File path:
external/storm-hdfs-blobstore/src/main/java/org/apache/storm/hdfs/blobstore/HdfsBlobStoreImpl.java
##########
@@ -312,4 +313,54 @@ public void shutdown() {
timer.cancel();
}
}
+
+ /**
+ * Get the last modification time of any blob.
+ *
+ * @return the last modification time of blobs within the blobstore.
+ * @throws IOException on any error
+ */
+ public long getLastModTime() throws IOException {
+ long modtime =
fileSystem.getFileStatus(fullPath).getModificationTime();
+ return modtime;
+ }
+
+ /**
+ * Updates the modification time of the blobstore to the current time.
+ *
+ * @throws IOException on any error
+ */
+ public void updateLastModTime() throws IOException {
+ long timestamp = Time.currentTimeMillis();
+ fileSystem.setTimes(fullPath, timestamp, timestamp);
+ LOG.debug("Updated blobstore modtime of {} to {}", fullPath,
timestamp);
+ }
+
+ /**
+ * Validates that the modification time of the blobstore is up to date
with the current existing blobs.
+ *
+ * @throws IOException on any error
+ */
+ public void validateModTime() throws IOException {
+ int currentBucket = 0;
+ long baseModTime = 0;
+ while (currentBucket < BUCKETS) {
+ String name = String.valueOf(currentBucket);
+ Path bucketDir = new Path(fullPath, name);
+
+ // only consider bucket dirs that exist with files in them
+ if (fileSystem.exists(bucketDir) &&
fileSystem.listStatus(bucketDir).length > 0) {
+ long modtime =
fileSystem.getFileStatus(bucketDir).getModificationTime();
Review comment:
This looks at the mod time of the bucketDir. But the blob files can be
updated.
https://git.vzbuilders.com/storm/storm/blob/master/docs/distcache-blobstore.md#updating-a-cached-file
Will this work properly in this case? Or does the code change at Nimbus.java
serve the purpose?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
