Teragen against the Openstack Swift

2015-11-19 Thread 林家銘
Hi I am trying to generate 100GB data in Swift by Teragen. My environments 1) All in one Devstack 2) 2T space for Swift with LVM installed 3) 6+1 node for hadoop cluster 4) 64G RAM, 16 Core CPU, 1G NIC for devstack 5) 12G RAM, 4 Core CPU, 1G NIC, 1 local disk for Hadoop cluster nodes. 6) three

Re: Yarn application reading from Data node using short-circuit.

2015-11-19 Thread Chris Nauroth
Hello Sandeep, As long as you have enabled short-circuit read as per the documentation [1], I expect any Hadoop process will take advantage of it while reading a local replica. However, short-circuit read will not completely eliminate TCP connection activity to the DataNode. There will still