[ 
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539756
 ] 

Konstantin Shvachko commented on HADOOP-2000:
---------------------------------------------

I think we should have the following two measures:
# average operation execution time =   Number_of_operations / Sum(t_i)
where t_i is the time of each map task.
# Cluster throughput = Number_of_operations / Max(t_i).
where Max(t_i) is the longest map task.

(1) is what is currently called TPS in the patch.
(2) is a new measure, which measures how many operations per second the cluster 
can perform as a whole.
I think 2 might be a good candidate for the real throughput, because completion 
of the longest map means that all other maps are done by that time too. 
So this should indicate the actual end of the map stage of the job.
The only problem I see here is that this works only under the assumption that 
all maps 
run in parallel, which is not always true, especially when failed maps have to 
be reptried.
So this is in a sense an ideal cluster throughput.

> Re-write NNBench to use MapReduce
> ---------------------------------
>
>                 Key: HADOOP-2000
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2000
>             Project: Hadoop
>          Issue Type: Test
>          Components: test
>    Affects Versions: 0.15.0
>            Reporter: Mukund Madhugiri
>            Assignee: Mukund Madhugiri
>             Fix For: 0.16.0
>
>         Attachments: HADOOP-2000.patch, HADOOP-2000.patch, HADOOP-2000.patch, 
> HADOOP-2000.patch, HADOOP-2000.patch
>
>
> The proposal is to re-write the NNBench benchmark/test to measure Namenode 
> operations using MapReduce. Two buckets of measurements will be done:
> 1. Transactions per second 
> 2. Average latency
> for these operations
> - Create and Close file
> - Open file
> - Rename file
> - Delete file

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to