Re: Hadoop 2.4 test jar files.
"..You can expand the one(s) you're interested in and run tests contained in them..." How is that done? How do I know what these classes do and what arguments they take? On Tue, Jul 22, 2014 at 1:42 PM, Ted Yu wrote: > These jar files contain source code for the respective hadoop modules. > You can expand the one(s) you're interested in and run tests contained in > them. > > Cheers > > > On Tue, Jul 22, 2014 at 9:47 AM, Charley Newtonne > wrote: > >> I have spent hours trying to find out how to run these jar files. The >> older version are documented on the web and some of the books. These, >> however, are not. >> >> How do I know ... >> - The purpose of each one of these jar files. >> - The class to call and what it does. >> - The arguments to pass. >> >> >> >> >> /a01/hadoop/2.4.0/share/hadoop/hdfs/hadoop-hdfs-2.4.0-tests.jar >> >> /a01/hadoop/2.4.0/share/hadoop/hdfs/sources/hadoop-hdfs-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-sls-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-datajoin-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-archives-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-gridmix-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-extras-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-streaming-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-distcp-2.4.0-test-sources.jar >> >> /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-rumen-2.4.0-test-sources.jar >> > >
Hadoop 2.4 test jar files.
I have spent hours trying to find out how to run these jar files. The older version are documented on the web and some of the books. These, however, are not. How do I know ... - The purpose of each one of these jar files. - The class to call and what it does. - The arguments to pass. /a01/hadoop/2.4.0/share/hadoop/hdfs/hadoop-hdfs-2.4.0-tests.jar /a01/hadoop/2.4.0/share/hadoop/hdfs/sources/hadoop-hdfs-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-sls-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-datajoin-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-archives-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-gridmix-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-extras-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-streaming-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-distcp-2.4.0-test-sources.jar /a01/hadoop/2.4.0/share/hadoop/tools/sources/hadoop-rumen-2.4.0-test-sources.jar
Bench-marking Hadoop Performance
This is a new cluster I'm putting up and I need to get an idea on what to expect from a performance standpoint. Older docs point to gridmix and TestDFSIO . However, most of this doc is obsolete and no longer applies on 2.4. Where can I find benchmarking docs for 2.4? What are my options? Also, I have searched safari books online including rough cuts, but not seeing books for the 2.4 release. If you know of a book for this release, please share. Thank you.
Configuring Hadoop Client: Where is fail-over configured.
I have hadoop 2.4 installed in HA mode using QJM. I have verified the cluster failover works as expected. The java clients are configured to connect to the active NN by specifying the hdf://nn1.company.com:8020. If this nn1 is down, how's the client know the location of the standby nn. Where is the client spillover configured? I have seen some references that this is configured in client's core-site.xml, but this file only specifies the defaultFS (which has the value of the logical cluster name) and ZK quorum nodes. None of these appear to be related to client side failover. Thanks in advance
