On Mon, Jul 01, 2019 at 05:34:35PM +0200, Philippe Mathieu-Daudé wrote: > Add a test of the NeXTcube framebuffer using the Tesseract OCR > engine on a screenshot of the framebuffer device. > > The test is very quick: > > $ avocado --show=app,ocr run tests/acceptance/machine_m68k_nextcube.py
Shouldn't we stick to "console" here? I understand we're resorting to ocr but the content, for what it's worth, should be the same as in the console for other tests. This allows a common expectation across tests too. > JOB ID : f7d3c27976047036dc568183baf64c04863d9985 > JOB LOG : ~/avocado/job-results/job-2019-06-29T16.18-f7d3c27/job.log > (1/1) > tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer: > |ocr: > ue r pun Honl'flx ; 5‘ 55‘ > avg ncaaaaa 25 MHZ, memary jag m > Backplane slat «a > Ethernet address a a r a r3 2 > Memgry sackets aea canflqured far 16MB Darlly page made stMs but have 16MB > page made stMs )nstalled > Memgry sackets a and 1 canflqured far 16MB Darlly page made stMs but have > 16MB page made stMs )nstalled > [...] > Yestlnq the rpu, 5:: > system test raneg Errar egge 51 > Egg: cammand > Default pggc devlce nut fauna > NEXY>I > PASS (3.59 s) > RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | > CANCEL 0 > JOB TIME : 3.97 s > > Documentation on how to install tesseract: > https://github.com/tesseract-ocr/tesseract/wiki#installation > > Signed-off-by: Philippe Mathieu-Daudé <f4...@amsat.org> > --- > v2: > - test fb sizes > - handle 2 versions of teseract > --- > tests/acceptance/machine_m68k_nextcube.py | 102 ++++++++++++++++++++++ > 1 file changed, 102 insertions(+) > create mode 100644 tests/acceptance/machine_m68k_nextcube.py > > diff --git a/tests/acceptance/machine_m68k_nextcube.py > b/tests/acceptance/machine_m68k_nextcube.py > new file mode 100644 > index 0000000000..f8e514a058 > --- /dev/null > +++ b/tests/acceptance/machine_m68k_nextcube.py > @@ -0,0 +1,102 @@ > +# Functional test that boots a VM and run OCR on the framebuffer > +# > +# Copyright (c) Philippe Mathieu-Daudé <f4...@amsat.org> > +# > +# This work is licensed under the terms of the GNU GPL, version 2 or > +# later. See the COPYING file in the top-level directory. > + > +import logging > +import time > +import distutils.spawn > + > +from avocado import skipUnless > +from avocado_qemu import Test > +from avocado.utils import process Style nitpick: from avocado_qemu import Test from avocado import skipUnless from avocado.utils import process > + > +try: > + from PIL import Image > + pil_available = True Another style nitpick, but very minor issue is the use of ALL_CAPS variables if they are at the module level. So that would become PIL_AVAILABLE = True > +except ImportError: > + pil_available = False > + > + > +def tesseract_available(expected_version): > + if not distutils.spawn.find_executable('tesseract'): Just though of pointing out that there's a similar function in avocado.utils.path, called find_command: https://avocado-framework.readthedocs.io/en/68.0/api/utils/avocado.utils.html#avocado.utils.path.find_command Feel free to pick your poison! :) > + return False > + res = process.run('tesseract --version') > + try: > + version = res.stdout_text.split()[1] > + except IndexError: > + version = res.stderr_text.split()[1] Do some versions write this to stdout and others to stderr? > + return int(version.split('.')[0]) == expected_version This can raise an exception if some other sort of output is produced. How about: import re match = re.match(r'tesseract\s(\d)', res) if match is not None: # now this is guaranteed to be a digit if int(match.groups()[0]) == expected_version: return True return False > + > + > +class NextCubeMachine(Test): > + > + timeout = 15 > + > + def check_bootrom_framebuffer(self, screenshot_path): > + rom_url = > ('http://www.nextcomputers.org/NeXTfiles/Software/ROM_Files/' > + '68040_Non-Turbo_Chipset/Rev_2.5_v66.BIN') > + rom_hash = 'b3534796abae238a0111299fc406a9349f7fee24' > + rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash) > + > + self.vm.set_machine('next-cube') > + self.vm.add_args('-bios', rom_path) > + self.vm.launch() > + > + self.log.info('VM launched, waiting for display') > + # FIXME how to catch the 'displaysurface_create 1120x832' > trace-event? > + time.sleep(2) There's avocado.utils.wait.wait_for() to *help* with waiting, but I'm not sure about the trace-events. > + > + self.vm.command('human-monitor-command', > + command_line='screendump %s' % screenshot_path) > + > + @skipUnless(pil_available, 'Python PIL not installed') > + def test_bootrom_framebuffer_size(self): > + """ > + :avocado: tags=arch:m68k > + :avocado: tags=machine:next-cube Here we hit the syntax limitation of the Avocado tags regex again: https://avocado-framework.readthedocs.io/en/68.0/api/core/avocado.core.html#avocado.core.safeloader.DOCSTRING_DIRECTIVE_RE_RAW I'll look into raising that limitation on the next release, but, for the time being, this will need to be: :avocado: tags=machine:next_cube The same applies to the other tests, of course. > + :avocado: tags=device:framebuffer > + """ > + screenshot_path = self.workdir + "dump" Best practice is to use os.path.join() instead. > + self.check_bootrom_framebuffer(screenshot_path) > + > + width, height = Image.open(screenshot_path).size > + self.assertEqual(width, 1120) > + self.assertEqual(height, 832) > + > + @skipUnless(tesseract_available(3), 'tesseract v3 OCR tool not > available') > + def test_bootrom_framebuffer_ocr_with_tesseract_v3(self): > + """ > + :avocado: tags=arch:m68k > + :avocado: tags=machine:next-cube > + :avocado: tags=device:framebuffer > + """ > + screenshot_path = self.workdir + "dump" > + self.check_bootrom_framebuffer(screenshot_path) > + > + console_logger = logging.getLogger('ocr') > + text = process.run("tesseract %s stdout" % > screenshot_path).stdout_text > + console_logger.debug(text) > + self.assertIn('Backplane', text) > + self.assertIn('Ethernet address', text) I haven't tried v3, but I'm curious... is this about the change in command line syntax only? Or do v3 and v4 are able to recognize different characters? - Cleber. > + > + @skipUnless(tesseract_available(4), 'tesseract v4 OCR tool not > available') > + def test_bootrom_framebuffer_ocr_with_tesseract_v4(self): > + """ > + :avocado: tags=arch:m68k > + :avocado: tags=machine:next-cube > + :avocado: tags=device:framebuffer > + """ > + screenshot_path = self.workdir + "dump" > + self.check_bootrom_framebuffer(screenshot_path) > + > + console_logger = logging.getLogger('ocr') > + proc = process.run("tesseract --oem 1 %s stdout" % screenshot_path) > + text = proc.stdout_text > + console_logger.debug(text) > + self.assertIn('Testing the FPU, SCC', text) > + self.assertIn('System test failed. Error code 51', text) > + self.assertIn('Boot command', text) > + self.assertIn('Next>', text) > -- > 2.19.1 >