Gopal V created HADOOP-11029: -------------------------------- Summary: LocalFS Statistics performs thread local call per byte written Key: HADOOP-11029 URL: https://issues.apache.org/jira/browse/HADOOP-11029 Project: Hadoop Common Issue Type: Bug Reporter: Gopal V Attachments: local-fs-locking.png
This code is there in the hot-path of IFile writer via RawLocalFileSystem. !local-fs-locking.png! >From a preliminary glance, the lock prefix calls are coming from a >threadlocal.get() within FileSystem.Statistics {code} /** * Get or create the thread-local data associated with the current thread. */ private StatisticsData getThreadData() { StatisticsData data = threadData.get(); if (data == null) { data = new StatisticsData( new WeakReference<Thread>(Thread.currentThread())); threadData.set(data); synchronized(this) { if (allData == null) { allData = new LinkedList<StatisticsData>(); } allData.add(data); } } return data; } /** * Increment the bytes read in the statistics * @param newBytes the additional bytes read */ public void incrementBytesRead(long newBytes) { getThreadData().bytesRead += newBytes; } {code} This is incredibly inefficient when used from FSDataOutputStream {code} public void write(int b) throws IOException { out.write(b); position++; if (statistics != null) { statistics.incrementBytesWritten(1); } } {code} -- This message was sent by Atlassian JIRA (v6.2#6252)