For diligence and curiosity leading up to the Pharo3 release, I
downloaded build image 30733 (with PharoLauncher) and immediately ran
all tests, which reported 7 failures and 1 error (see attached
snapshot). Now presumably these did not occur in the CI build since
otherwise it would not have been available to PharoLauncher, so this
raises a few questions/ideas:
1. Why the discrepancy / how did these slip through ? Perhaps some
difference in local environment / difference in VM? Can anyone else
reproduce this for 30733? I've logged an new case [1] for this where I
plan to either link to existing cases or add a subcase for each
failure. Please add your results there, particularly if you see a known
issue.
Here are my failures...
BlockClosureTest>>#testTrace
JobTest>>#testCurrent
MetacelloRepositorySqueakCommonTestCase>>#testAsRepositorySpecFor
MetacelloRepositorySqueakCommonTestCase>>#testDirectoryRepository
MetacelloRepositorySqueakCommonTestCase>>#testFileTreeRepository
ReleaseTest>>#testObsoleteClasses
SimmulateMouseSpecification>>#testSimulateRightClick
and errors...
OpenToolTest>>#testOpenBrowseOnInstalledTraitMethod
2. How is the PharoLauncher VM synced with that used to test the Pharo
CI builds ? (My VM comes from the PharoLauncher Windows Installer)
3. Assuming Pharo3 will go through a Release Candidate phase, apart from
functionality testing which anyone may-or-not have time for, as many
spectators as possible should download those Release Candidate images an
just run All Tests.
4. It might be good for a small number of releases between now and
Release Candidate to be designated for the community to similarly run
All Tests - to reduce surprises at Release Candidate time?
5. I've seen times before when some failures as transitory/random (which
had to do with unordered Sets being used by a test that presumed
order). So at times when there are a lot of Pharoers together, such as
at a Sprint, maybe the convention could be for everyone to run All Tests
at the start of the day to provide broad comparison (and know you're
starting with a clean slate)
cheers -ben
[1] https://pharo.fogbugz.com/f/cases/12770/
