[ https://issues.apache.org/jira/browse/HADOOP-11029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gopal V updated HADOOP-11029: ----------------------------- Affects Version/s: 2.6.0 2.5.0 > LocalFS Statistics performs thread local call per byte written > -------------------------------------------------------------- > > Key: HADOOP-11029 > URL: https://issues.apache.org/jira/browse/HADOOP-11029 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 2.5.0, 2.6.0 > Reporter: Gopal V > Attachments: local-fs-locking.png > > > This code is there in the hot-path of IFile writer via RawLocalFileSystem. > !local-fs-locking.png! > From a preliminary glance, the lock prefix calls are coming from a > threadlocal.get() within FileSystem.Statistics > {code} > /** > * Get or create the thread-local data associated with the current thread. > */ > private StatisticsData getThreadData() { > StatisticsData data = threadData.get(); > if (data == null) { > data = new StatisticsData( > new WeakReference<Thread>(Thread.currentThread())); > threadData.set(data); > synchronized(this) { > if (allData == null) { > allData = new LinkedList<StatisticsData>(); > } > allData.add(data); > } > } > return data; > } > /** > * Increment the bytes read in the statistics > * @param newBytes the additional bytes read > */ > public void incrementBytesRead(long newBytes) { > getThreadData().bytesRead += newBytes; > } > {code} > This is incredibly inefficient when used from FSDataOutputStream > {code} > public void write(int b) throws IOException { > out.write(b); > position++; > if (statistics != null) { > statistics.incrementBytesWritten(1); > } > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)