RE: How to get the HDFS I/O information

Hi Qu, You can access the HDFS read/write bytes for each task or job level using the below counters. FileSystemCounters : HDFS_BYTES_READ FILE_BYTES_WRITTEN These can be accessed by using UI or API. Thanks Devaraj

Re: How to get the HDFS I/O information

I am not sure if you want to use APIs, but to access through APIs, you can use the following API in Counters.java: /** * Find a counter given the group and the name. * @param group the name of the group * @param name the internal name of the counter * @return the counter for that

dump of Hadoop Mapreduce bugs data

Hi all, I am a PhD student at DERI, NUI Galway, Ireland. My main area of research is in investigating social and socio-technical relationship among developers in open source projects. For my research, i am extracting information from various software repositories (source code, mailing list