Unit tests on real hadoop cluster
I would like to execute unittest on real hadoop cluster. I try to execute the following command against giraph trunk version. mvn test -Dprop.mapred.job.tracker=xxx.korea.ac.kr:8021 -Dprop.zookeeper.list=xxx.korea.ac.kr:2181 However, the unit tests are failed as follows: https://gist.github.com/1252309 I think It may be my fault because the source code is trunk version. Any suggestion to this will be helpful. -- Hyunsik Choi Database Lab, Korea University
Re: Unit tests on real hadoop cluster
Actually, to be fair, I've only executed the distributed unittests on my own local Hadoop instance. I just ran the Hadoop unittests against trunk on my local machine to check mvn test -Dprop.mapred.job.tracker=localhost:50300 Results : Tests run: 27, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 12:19.143s [INFO] Finished at: Thu Sep 29 21:55:55 PDT 2011 [INFO] Final Memory: 6M/81M [INFO] Everything should be fine. Avery On 9/29/11 5:18 PM, Hyunsik Choi wrote: I would like to execute unittest on real hadoop cluster. I try to execute the following command against giraph trunk version. mvn test -Dprop.mapred.job.tracker=xxx.korea.ac.kr:8021 -Dprop.zookeeper.list=xxx.korea.ac.kr:2181 However, the unit tests are failed as follows: https://gist.github.com/1252309 I think It may be my fault because the source code is trunk version. Any suggestion to this will be helpful. -- Hyunsik Choi Database Lab, Korea University
Re: Unit tests on real hadoop cluster
I've run them on our 20x cluster with no problems, but with a local ZK, not a specified instance. On Thu, Sep 29, 2011 at 10:05 PM, Avery Ching wrote: > Actually, to be fair, I've only executed the distributed unittests on my own > local Hadoop instance. > > I just ran the Hadoop unittests against trunk on my local machine to check > > mvn test -Dprop.mapred.job.tracker=localhost:50300 > > > Results : > > Tests run: 27, Failures: 0, Errors: 0, Skipped: 0 > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 12:19.143s > [INFO] Finished at: Thu Sep 29 21:55:55 PDT 2011 > [INFO] Final Memory: 6M/81M > [INFO] > > > Everything should be fine. > > Avery > > On 9/29/11 5:18 PM, Hyunsik Choi wrote: >> >> I would like to execute unittest on real hadoop cluster. >> >> I try to execute the following command against giraph trunk version. >> >> mvn test -Dprop.mapred.job.tracker=xxx.korea.ac.kr:8021 >> -Dprop.zookeeper.list=xxx.korea.ac.kr:2181 >> >> However, the unit tests are failed as follows: >> https://gist.github.com/1252309 >> >> I think It may be my fault because the source code is trunk version. >> >> Any suggestion to this will be helpful. >> >> -- >> Hyunsik Choi >> Database Lab, Korea University > > >
Re: Unit tests on real hadoop cluster
Avery and Jake, Thank you for your comments =) I executed the unit test on the hadoop cluster that installed Hadoop cdh 0.20.2+923.97. On this cluster, many of the unit tests are failed because MR jobs could not be submitted to that cluster. I didn't investigate this problem. Is here my mistake? If this is unknown problem, I'll create a jira issue about that. Anyway, I installed hadoop 0.20.203 to my local machine. All unit tests work fine =) -- Hyunsik Choi Database Lab, Korea University On Fri, Sep 30, 2011 at 2:16 PM, Jakob Homan wrote: > I've run them on our 20x cluster with no problems, but with a local > ZK, not a specified instance. > > On Thu, Sep 29, 2011 at 10:05 PM, Avery Ching wrote: >> Actually, to be fair, I've only executed the distributed unittests on my own >> local Hadoop instance. >> >> I just ran the Hadoop unittests against trunk on my local machine to check >> >> mvn test -Dprop.mapred.job.tracker=localhost:50300 >> >> >> Results : >> >> Tests run: 27, Failures: 0, Errors: 0, Skipped: 0 >> >> [INFO] >> >> [INFO] BUILD SUCCESS >> [INFO] >> >> [INFO] Total time: 12:19.143s >> [INFO] Finished at: Thu Sep 29 21:55:55 PDT 2011 >> [INFO] Final Memory: 6M/81M >> [INFO] >> >> >> Everything should be fine. >> >> Avery >> >> On 9/29/11 5:18 PM, Hyunsik Choi wrote: >>> >>> I would like to execute unittest on real hadoop cluster. >>> >>> I try to execute the following command against giraph trunk version. >>> >>> mvn test -Dprop.mapred.job.tracker=xxx.korea.ac.kr:8021 >>> -Dprop.zookeeper.list=xxx.korea.ac.kr:2181 >>> >>> However, the unit tests are failed as follows: >>> https://gist.github.com/1252309 >>> >>> I think It may be my fault because the source code is trunk version. >>> >>> Any suggestion to this will be helpful. >>> >>> -- >>> Hyunsik Choi >>> Database Lab, Korea University >> >> >> >
