Sent from my HTC
----- Reply message ----- From: "Alex Huang" <alex.hu...@citrix.com> To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org> Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest Date: Wed, May 29, 2013 21:50 Daan, I'm new to mockito as well and I haven't had time to really look at what the right patterns are. In general, in my dealings with other mocking frameworks, I have never liked having the mocking framework returning a default value. I've always like the mocking framework to throw an exception if the method being access was not explicitly set. Not sure if that can be a setting in mockito. Thanks. --Alex > -----Original Message----- > From: Daan Hoogland [mailto:dhoogl...@schubergphilis.com] > Sent: Wednesday, May 29, 2013 8:58 AM > To: 'dev@cloudstack.apache.org' > Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest > > Alex, > > Thanks for the mockito lesson. I think a lot of us are now going to improve > their tests. > Can one make mockito forget the when-conditions? I would intuitively say > MockitoAnnotations.initMocks(this); would do this. > > Alternative to the answer to that last question would be to follow the > javadoc for when: > //you can set different behavior for consecutive method calls. > //Last stubbing (e.g: thenReturn("foo")) determines the behavior of further > consecutive calls. > when(mock.someMethod("some arg")) > .thenThrow(new RuntimeException()) > .thenReturn("foo"); > > //Alternative, shorter version for consecutive stubbing: > when(mock.someMethod("some arg")) > .thenReturn("one", "two"); > //is the same as: > when(mock.someMethod("some arg")) > .thenReturn("one") > .thenReturn("two"); > > Regards, > Daan (not new to unit-tests sec but to mockito though) Hoogland > > -----Original Message----- > From: Alex Huang [mailto:alex.hu...@citrix.com] > Sent: woensdag 29 mei 2013 16:34 > To: Kishan Kavala; dev@cloudstack.apache.org > Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest > > Kishan, > > The problem is caused by these two methods. I hope everyone reads this > because I know some of us are new to writing unit tests. The problem is > basically as I described in a previous email. In here, both tests makes use > of > and shares of the _networkACLItemDao and the mock is shared because the > DAO is a singleton and is injected. Now, note that testCreateACLItem() > invokes the method findByAclAndNumber() via the _networkAclMgr but it > didn't specifically mock the return value of the method in the test. If this > test > is ran before testCreateACLItemDuplicateNumber(), then it will be successful > because by default mocks return null. However, > testCreateACLItemDuplicateNumber() mocks findByAclAndNumber() to > return a value, so if testCreateACLItem() is ran after > testCreateACLItemDuplicateNumber(), the test will fail because a value is > returned and the condition inside _networkAclMgr will fire just as Daan > found it. > > I was hoping maven prints out the order of the tests, but from Daan's log, it > only printed out the order of the bigger test suite so it's not helpful. > However, it's not too hard to find. Just go to the code with the error, check > the method that fails and go back to the test and see if your test mocked that > method and if it didn't, was it mocked by any other tests. > > @Test > public void testCreateACLItem() throws Exception { > Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new > VpcVO()); > > Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenR > eturn(acl); > > Mockito.when(_networkAclMgr.createNetworkACLItem(Mockito.anyInt(), > Mockito.anyInt(), Mockito.anyString(), Mockito.anyList(), Mockito.anyInt(), > Mockito.anyInt(), > Mockito.any(NetworkACLItem.TrafficType.class), > Mockito.anyLong(), > Mockito.anyString(), Mockito.anyInt())).thenReturn(new > NetworkACLItemVO()); > assertNotNull(_aclService.createNetworkACLItem(createACLItemCmd)); > } > > @Test(expected = InvalidParameterValueException.class) > public void testCreateACLItemDuplicateNumber() throws Exception { > Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new > VpcVO()); > > Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenR > eturn(acl); > > Mockito.when(_networkACLItemDao.findByAclAndNumber(Mockito.anyLon > g(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO()); > _aclService.createNetworkACLItem(createACLItemCmd); > } > > > -----Original Message----- > > From: Kishan Kavala > > Sent: Wednesday, May 29, 2013 4:48 AM > > To: dev@cloudstack.apache.org > > Cc: Alex Huang > > Subject: RE: master build breaks at > > com.cloud.vpc.NetworkACLServiceTest > > > > Daan, > > I mocked the method causing the failure (I couldn't replicate the > > error you are seeing though). Please check with latest master and let > > us know if you still see issues. > > > > ~kishan > > ________________________________________ > > From: Daan Hoogland [dhoogl...@schubergphilis.com] > > Sent: Wednesday, May 29, 2013 1:49 PM > > To: dev@cloudstack.apache.org > > Cc: Alex Huang > > Subject: RE: master build breaks at > > com.cloud.vpc.NetworkACLServiceTest > > > > H, > > > > I hope this is what Alex means the test programmer needs. > > > > The order that is reported is as follows (I included the server test run > > only): > > ------------------------------------------------------- > > T E S T S > > ------------------------------------------------------- > > Running com.cloud.alert.AlertControlsUnitTest > > log4j:WARN No appenders could be found for logger > > (com.cloud.alert.AlertControlsUnitTest). > > log4j:WARN Please initialize the log4j system properly. > > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig > > for more info. > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 > > sec Running com.cloud.capacity.CapacityManagerTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 > > sec Running com.cloud.configuration.ConfigurationManagerTest > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499 > > sec Running com.cloud.configuration.ValidateIpRangeTest > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 > > sec Running com.cloud.event.EventControlsUnitTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec > > Running com.cloud.keystore.KeystoreTest > > org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","secu > > rit ygroup":[],"nic":[],"tags":[],"affinitygroup":[]} > > org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","des > > crip > > tion":"Hello"} > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 > > sec Running com.cloud.metadata.ResourceMetaDataManagerTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 > > sec Running com.cloud.network.CreatePrivateNetworkTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec > > Running com.cloud.network.dao.NetworkDaoTest > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec > > Running com.cloud.network.DedicateGuestVlanRangesTest > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 > > sec Running com.cloud.network.firewall.FirewallManagerTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec > > Running com.cloud.network.NetworkManagerTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec > > Running com.cloud.network.NetworkModelTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec > > Running com.cloud.network.security.SecurityGroupQueueTest > > Num Vms= 50 Queue size = 50 > > Num Vms= 20000 Queue size = 20000 time=827 ms Num Vms= 5000 Queue > size > > = 5000 time=2544 ms Num Vms= 1 Queue size = 1 time=0 ms Num Vms= > > 1000000 Queue size = 1000000 time=1171 ms Num Vms= 1 Queue size = 1 > > time=1140 ms Total jobs dequeued = 10, num queued=1010 queue current > > size=1000 Total jobs dequeued = 10, num queued=1008 queue current > > size=998 Total jobs dequeued = 1, num queued=1001 queue current > > size=1000 Total jobs dequeued = 10, num queued=1000 queue current > > size=990 Total jobs dequeued = 10, num queued=10 queue current size=0 > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745 > > sec Running com.cloud.network.UpdatePhysicalNetworkTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312 > > sec Running com.cloud.resourcelimit.ResourceLimitManagerImplTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec > > Running com.cloud.vm.DeploymentPlanningManagerImplTest > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713 > > sec Running com.cloud.vm.snapshot.VMSnapshotManagerTest > > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203 > > sec Running com.cloud.vm.UserVmManagerTest Tests run: 10, Failures: 0, > > Errors: 0, > > Skipped: 0, Time elapsed: 0.64 sec Running > > com.cloud.vm.VirtualMachineManagerImplTest > > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297 > > sec Running com.cloud.vpc.NetworkACLManagerTest > > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 > > sec Running com.cloud.vpc.NetworkACLServiceTest > > Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203 > > sec <<< FAILURE! > > Running com.cloud.vpc.Site2SiteVpnTest Tests run: 1, Failures: 0, > > Errors: 0, Skipped: 0, Time elapsed: 5.491 sec Running > > com.cloud.vpc.VpcTest Tests run: 1, Failures: 0, Errors: 0, Skipped: > > 0, Time > > elapsed: 0.235 sec Running > > org.apache.cloudstack.affinity.AffinityApiUnitTest > > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 > > sec Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest > > Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249 > > sec Running > > org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest > > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817 > > sec Running org.apache.cloudstack.privategw.AclOnPrivateGwTest > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 > > sec Running > > org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplT > > est Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.032 sec Running org.apache.cloudstack.region.RegionManagerTest > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec > > > > Results : > > > > Tests in error: > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item > > with number 1 already exists in ACL: null > > > > -----Original Message----- > > From: Pranav Saxena [mailto:pranav.sax...@citrix.com] > > Sent: woensdag 29 mei 2013 5:58 > > To: dev@cloudstack.apache.org > > Cc: Alex Huang > > Subject: RE: master build breaks at > > com.cloud.vpc.NetworkACLServiceTest > > > > Thanks Alex for throwing some light on this . I have myself seen this > > when one of the intenalLb tests were failing on my machine and few > > others but compiling successfully on Alena's machine but that time I > > also didn't know how this could have happened and I ended up reporting > > this to her and then after probably some amendments , unit tests compiled > on all the machines. > > > > -----Original Message----- > > From: Alex Huang [mailto:alex.hu...@citrix.com] > > Sent: Wednesday, May 29, 2013 12:05 AM > > To: dev@cloudstack.apache.org > > Subject: RE: master build breaks at > > com.cloud.vpc.NetworkACLServiceTest > > > > Unit tests are ran in random order because each unit test should be > > self- sufficient. I'm saying I've seen cases where unit tests aren't > > written to be self-sufficient and that's why an unit test fails > > sometimes and passes sometimes. > > > > In those cases, we should fix the unit test and not the order. A > > common problem is sharing the mocked classes. For example, unit test > > A tests the set of a mock object and unit test B tests if the field is > > null but they shared the mock instance. In this case, unit test B > > passes if it is ran before unit test A but fails if it was ran after. > > > > But often times the randomness of how it's ran is actually > > pseudo-random so it works consistently on someone's laptop but as soon > > as it gets ran on another machine, the order changes and it fails. So > > when you see the problem such as this, you should always let the unit > > test writer know the order in which the unit tests were executed so > > that the unit test writer can try it. > > > > --Alex > > > > > -----Original Message----- > > > From: Daan Hoogland [mailto:dhoogl...@schubergphilis.com] > > > Sent: Monday, May 27, 2013 9:23 AM > > > To: dev@cloudstack.apache.org > > > Subject: RE: master build breaks at > > > com.cloud.vpc.NetworkACLServiceTest > > > > > > And if not how do I manipulate the order of tests? > > > > > > -----Original Message----- > > > From: Daan Hoogland [mailto:dhoogl...@schubergphilis.com] > > > Sent: maandag 27 mei 2013 17:02 > > > To: dev@cloudstack.apache.org > > > Subject: RE: master build breaks at > > > com.cloud.vpc.NetworkACLServiceTest > > > > > > Alex, > > > > > > Can this change out of the box? I had the problem on a clean > > > checkout of master. > > > > > > -----Original Message----- > > > From: Alex Huang [mailto:alex.hu...@citrix.com] > > > Sent: maandag 27 mei 2013 15:55 > > > To: dev@cloudstack.apache.org > > > Subject: RE: master build breaks at > > > com.cloud.vpc.NetworkACLServiceTest > > > > > > Daan, > > > > > > The other possibility is that the test is affected by another test > > > so that it breaks when the tests are in a certain order. Have you > > > noticed what's the order of the tests ran when it breaks? > > > > > > --Alex > > > > > > > -----Original Message----- > > > > From: Daan Hoogland [mailto:dhoogl...@schubergphilis.com] > > > > Sent: Monday, May 27, 2013 2:37 AM > > > > To: dev@cloudstack.apache.org > > > > Subject: RE: master build breaks at > > > > com.cloud.vpc.NetworkACLServiceTest > > > > > > > > Being in the habit of replying on my own mails as I seems to be on > > > > this list; > > > > > > > > This seems to be related to eclipse refresh/build code. Does > > > > anybody have a solution to it? The problem is not consistently > > > > present but all the more irritating. > > > > > > > > Regards, > > > > > > > > -----Original Message----- > > > > From: Daan Hoogland [mailto:dhoogl...@schubergphilis.com] > > > > Sent: maandag 27 mei 2013 7:49 > > > > To: dev@cloudstack.apache.org > > > > Subject: master build breaks at > > > > com.cloud.vpc.NetworkACLServiceTest > > > > > > > > LS, > > > > > > > > Lately I've been getting, > > > > "com.cloud.exception.InvalidParameterValueException: ACL item with > > > > number 1 already exists in ACL: null", while building cloudstack. > > > > I first thought it was my own hack so I don't know when it was > > > > introduced (it is in a clean master that it happens). The full > > > > surfire report > > > follows: > > > > > > > > ------------------------------------------------------------------ > > > > -- > > > > -- > > > > --------- Test set: com.cloud.vpc.NetworkACLServiceTest > > > > ------------------------------------------------------------------ > > > > -- > > > > -- > > > > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time > > > > elapsed: 0.219 sec <<< FAILURE! > > > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time > > elapsed: > > > > 0.016 sec <<< ERROR! > > > > com.cloud.exception.InvalidParameterValueException: ACL item with > > > > number 1 already exists in ACL: null > > > > at > > > > > > > > > > com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne > > > > tworkACLServiceImpl.java:270) > > > > at > > > > > > > > > > com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ > > > > iceTest.java:141) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) > > > > at > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > > > > Source) > > > > at java.lang.reflect.Method.invoke(Unknown Source) > > > > at > > > > > > > > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework > > > > Method.java:45) > > > > at > > > > org.junit.internal.runners.model.ReflectiveCallable.run(Reflective > > > > Ca > > > > ll > > > > able.jav > > > > a:15) > > > > at > > > > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework > > > > Method.java:42) > > > > at > > > > > > > > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth > > > > od.java:20) > > > > at > > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefor > > > > es > > > > .j > > > > ava > > > > :28) > > > > at > > > > > > > > > > org.springframework.test.context.junit4.statements.RunBeforeTestMethod > > > > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74) > > > > at > > > > > > > > > > org.springframework.test.context.junit4.statements.RunAfterTestMethodC > > > > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83) > > > > at > > > > org.springframework.test.context.junit4.statements.SpringRepeat.ev > > > > al > > > > ua > > > > te( > > > > SpringRepeat.java:72) > > > > at > > > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru > > > > nC > > > > hi > > > > ld(S > > > > pringJUnit4ClassRunner.java:231) > > > > at > > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4Class > > > > Ru > > > > nn > > > > er.j > > > > ava:47) > > > > at > > > > org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > > > > at > > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > > > > at > > > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > > > > at > > org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > > > > at > > > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > > > > at > > > > org.springframework.test.context.junit4.statements.RunBeforeTestCl > > > > as > > > > sC > > > > all > > > > backs.evaluate(RunBeforeTestClassCallbacks.java:61) > > > > at > > > > org.springframework.test.context.junit4.statements.RunAfterTestCla > > > > ss > > > > Ca > > > > llb > > > > acks.evaluate(RunAfterTestClassCallbacks.java:71) > > > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > > > > at > > > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru > > > > n( > > > > Sp > > > > ring > > > > JUnit4ClassRunner.java:174) > > > > at > > > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Prov > > > > id > > > > er > > > > .jav > > > > a:236) > > > > at > > > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUn > > > > it > > > > 4P > > > > rov > > > > ider.java:134) > > > > at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider. > > > > java > > > > :113) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown > Source) > > > > at > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > > > > Source) > > > > at java.lang.reflect.Method.invoke(Unknown Source) > > > > at > > > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArr > > > > ay > > > > (R > > > > efl > > > > ectionUtils.java:189) > > > > at > > > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.inv > > > > ok > > > > e( > > > > Pr > > > > oviderFactory.java:165) > > > > at > > > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Pr > > > > ov > > > > id > > > > er > > > > Factory.java:85) > > > > at > > > > > > > > > > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke > > > > d > > > > Booter.java:103) > > > > at > > > > > > > > > > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7 > > > > 4)
