[ https://issues.apache.org/jira/browse/MAPREDUCE-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ray Chiang reopened MAPREDUCE-6685: ----------------------------------- > LocalDistributedCacheManager can have overlapping filenames > ----------------------------------------------------------- > > Key: MAPREDUCE-6685 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6685 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: 3.0.0-alpha1 > Reporter: Ray Chiang > Assignee: Ray Chiang > Attachments: MAPREDUCE-6685.001.patch, MAPREDUCE-6685.002.patch > > > LocalDistributedCacheManager has this setup: > bq. AtomicLong uniqueNumberGenerator = new > AtomicLong(System.currentTimeMillis()); > to create this temporary filename: > bq. new FSDownload(localFSFileContext, ugi, conf, new Path(destPath, > Long.toString(uniqueNumberGenerator.incrementAndGet())), resource); > when using LocalJobRunner. When two or more start on the same machine, then > it's possible to end up having the same timestamp or a large enough overlap > that two successive timestamps may not be sufficiently far apart. > Given the assumptions: > 1) Assume timestamp is the same. Then the most common starting random seed > will be the same. > 2) Process ID will very likely be unique, but will likely be close in value. > 3) Thread ID is not guaranteed to be unique. > A unique ID based on PID as a seed (in addition to the timestamp) should be a > better unique identifier for temporary filenames. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org