For the cross compilation, the "deployment" section in the .pro file is supposed to take care of including the test data in the package to upload to the device under test. WinCE and Symbian ports in Qt4 have ad hoc solutions for this (cetester and runonphone tools respectively) Running out of storage space can be a problem, and so is the speed of testing.
The main task for a platform is to have some tool that uploads the test + test data in deployment to the target, runs the test, and downloads the test results. Running tests in parallel over multiple test devices could ease the speed problems. Running tests on an emulator is an option, but is an unrealistic environment for network and mobility testing. Another problem is if the test device has only wireless connectivity, then tests can be more flaky than usual. > -----Original Message----- > From: development-bounces+shane.kearns=accenture....@qt-project.org > [mailto:development-bounces+shane.kearns=accenture....@qt-project.org] > On Behalf Of Holger Hans Peter Freyther > Sent: Sunday, January 29, 2012 20:13 > To: development > Subject: [Development] Tests, Shadow-Build and Cross-Compilation > > Hi all, > > I finally got around to play with my personal jenkins[1] setup again, > specially to learn (and be kind of an example on how a non Tier1 > platform > could ever become one). > > There are some issues and I would like to discuss how these can be > solved in a > way that doesn't look like an ad-hoc solution and that might scale. > > I will mumble about the Problems, the Ad-Hoc solutions and muse about a > fix... > > > > Problems: > Shadow builds: > > I always liked shadow builds as I could have several Qt configurations > but a > command/shared source directory (e.g. X11, QWS, for i386 and ARM). It is > quite > common for tests to break in shadow builds (the QA page in the wiki > discourages this as well). > > Cross compilation: > When cross compiling the machine that compiles the tests will not be the > machine that ends up running them. > > > Ad-hoc solutions: > Shadow builds: > > The ad-hoc solution appears to change tests that refer to files in a way > like > this: > VERIFY(file.open("testfile")); > > to: > VERIFY(file.open(SRCDIR "testfile")); > > and put DEFINES += SRCDIR=... into the pro file. > > > Cross compilation: > > The attribute of cross-compilation and running the test on a different > architecture is somehow attributed to both Windows-CE (and Symbian). > This > manifests in the xmlpatterns test by adding deployment targes, deploy > files, > change the sourcecode to access different directories. > > > Musing: > So somehow cross-compilation and shadow-builds suffer from the same > problem > that accessing files from the source directory with a relative path. One > proposal would be to use something like 'QDir::addSearchPath("testdata", > SRCDIR)'" and have it set to the path of the .pro file, and access > everything > using the prefix. Danimo mentioned that this will fail for multimedia > that > might not use the Qt resource system. So we might want to have a method > in the > testlib to resolve a path like "testdata:foo" to a proper path? For > cross > compiling one could change the testdata prefix with a command line > option or > environment variable? > > The other part/wish would be to always have deployment targets for the > testcases and generate a run script or such as part of the installation. > > comments > holger > > > > [1] http://qt-jenkins.moiji-mobile.com/jenkins/ > _______________________________________________ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development ________________________________ Subject to local law, communications with Accenture and its affiliates including telephone calls and emails (including content), may be monitored by our systems for the purposes of security and the assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com _______________________________________________ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
