reuvenlax commented on code in PR #40320:
URL: https://github.com/apache/beam/pull/40320#discussion_r4127648429
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/Environments.java:
##########
@@ -538,6 +549,58 @@ public static String createStagingFileName(File path,
HashCode hash) {
return String.format("%s-%s%s", fileName, encodedHash, suffix);
}
+ /**
+ * Returns the SHA-256 {@link HashCode} for {@code file}, caching the result
in memory keyed by
+ * the file's absolute path, length, and last-modified timestamp.
+ */
+ public static HashCode getFileHash(File file) throws IOException {
+ FileHashCacheKey key = new FileHashCacheKey(file);
+ try {
+ return FILE_HASH_CACHE.computeIfAbsent(
+ key,
+ k -> {
+ try {
+ return Files.asByteSource(file).hash(Hashing.sha256());
+ } catch (IOException e) {
+ throw new UncheckedIOException(e);
+ }
+ });
+ } catch (UncheckedIOException e) {
+ throw e.getCause();
+ }
+ }
+
+ private static final class FileHashCacheKey {
+ private final String absolutePath;
+ private final long length;
+ private final long lastModified;
+
+ FileHashCacheKey(File file) {
+ this.absolutePath = file.getAbsolutePath();
+ this.length = file.length();
+ this.lastModified = file.lastModified();
+ }
+
+ @Override
+ public boolean equals(@Nullable Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof FileHashCacheKey)) {
+ return false;
+ }
+ FileHashCacheKey that = (FileHashCacheKey) o;
+ return length == that.length
Review Comment:
unclear how this would be exploited. any exploit means that an attacker has
full write access to the staging bucket, in which case they could anyway
overwrite any binary with malicious code.
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/Environments.java:
##########
@@ -98,6 +104,11 @@ public class Environments {
.put(ENVIRONMENT_PROCESS, ImmutableSet.of(processCommandOption,
processVariablesOption))
.build();
+ private static final ConcurrentHashMap<FileHashCacheKey, HashCode>
FILE_HASH_CACHE =
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
