Re: What are you doing with Perl in the QA department?

[Steffen Schwigon]

Gabor Szabo  writes:
> Hi,
>
> In this question I am not so much interested in how you do unit
> testing of your Perl code but how you use Perl in a QA or QC department?
>
>
> Some of you might know that I have a training course on how to
> use Perl in test automation. That course assumes you already know
> Perl. (First I teach TAP and how to test Perl code and then things
> like WWW::Mechanize, Selenium for web, Expect and Net::Telnet for CLI
> and Win32::GUITest for, well GUI related things to test anything else.)
>
>
> Now I am thinking on building a course or writing a book or just a
> series of blog
> entries that will not assume any Perl knowledge and will help people in the
> QA/QC department and in test automation.
> So I wonder, if you are working in QA, what are you doing with Perl?
>
> Is that processing log files?
> Is that running batch jobs or wrapping other utilities?
> Is that for preparing input files for the test?
> Do you also use Perl to test various applications (web, CLI, GUI, etc)?

We test operating systems (kernels, kernel subsystems) against cpu
features with and without virtualization (Xen, KVM) and also running
on cpu simulator. The whole infrastructure is written in Perl.

It allows any incoming reports based on TAP (Test Anything Protocol),
so it's also a TAP database. We also transport benchmark values inside
TAP using TAPv13's embedded YAML feature.

To access this history of TAP reports it additionally provides a query
API based on TAP::DOM and Data::DPath.

Using this query interface we generate data for several kinds of data
evaluation, using either Perl and Data::DPath directly or just provide
data as YAML/JSON to other scripts, written in Python or Perl.

Having TAP as the only common interface allows us to incorporate
other, non-Perl test suites by only enabling them to output TAP. We
have a collection of test and benchmark suites (dedicated or wrappers
around existing ones to produce TAP) written in Shell, Python and
Perl. They do everything from simple log file analysis to stressing
cpu features.

ah, and yes, we also have a Web app to present data. It's written in
Perl using Catalyst, DBIx::Class and TAP::Formatter::HTML.

We presented several parts on German Perlworkshop and YAPC::EU 2009:

 http://www.perl-workshop.de/talks/157/view
 http://yapceurope2009.org/ye2009/talk/1950
 http://conferences.yapceurope.org/gpw2010/talk/2621 (REDUX at GPW2010)


All in all, your mission to advocate TAP without Perl is very
useful! TAP is a killer API for test infrastructures.

Kind regards,
Steffen 
-- 
Steffen Schwigon 
Dresden Perl Mongers 

Re: What are you doing with Perl in the QA department?

[Thomas Maier]

On Sun, May 9, 2010 at 8:58 AM, Gabor Szabo  wrote:
>
> Hi,
>
> In this question I am not so much interested in how you do unit
> testing of your Perl code but how you use Perl in a QA or QC department?
>
>
> Some of you might know that I have a training course on how to
> use Perl in test automation. That course assumes you already know
> Perl. (First I teach TAP and how to test Perl code and then things
> like WWW::Mechanize, Selenium for web, Expect and Net::Telnet for CLI
> and Win32::GUITest for, well GUI related things to test anything else.)
I use Net::Telnet for Telnet and Expect for SSH to control remote
devices (DUT CLI,switch,etc`).

Lately I also run into XML-RPC. It's very comfortable especially when
the very same test run on remote DUT
as well as on local developer station. But that's my case.
Obviously installing application like XML-RPC server on DUT maybe
challenging or even not possible at all.

> Now I am thinking on building a course or writing a book or just a
> series of blog
> entries that will not assume any Perl knowledge and will help people in the
> QA/QC department and in test automation.
> So I wonder, if you are working in QA, what are you doing with Perl?
>
> Is that processing log files?
A lot. I even parsed logs files on DUT, but apparently it wasn't smart solution
at all..

>
> Is that running batch jobs or wrapping other utilities?
Stand alone server to collect data from different sources for later
use in testing
or monitoring.

> Is that for preparing input files for the test?
In my case - it's more reading configuration files before/during the test
session

>
> Do you also use Perl to test various applications (web, CLI, GUI, etc)?
I test application installed on chassis with couple of management switch blades
and dozen computing blades (storage,video processing).

Other interfaces I'm using to control this beast are: SNMP (a lot),
DB, sockets. I've tried even Message Bus once.
Sometimes I have to combine CLI, SNMP and DB calls to get/set
appropriate test case results/conditions.
On other occasions I run third party applications, locally or remotely.

In addition - a lot of Log::Log4Perl, mostly to find problems in the
testing framework but it
helps with DUT debugging as well.

To deliver if only possible, I would use local CPAN server. Due to the
environment constraints I use PAR combined with given packaging
system (rpm, in my case). It works but it's terrible.

And of course TAP to report the test results, Console and HTML over
email. The output is dynamic and depends on
who's running the test . The same test scenario can produce "this user
TAP" or "the other user TAP" output upon request.
When your framework is used by people with different skills, knowledge
and background it maybe worth
the effort. In my case it finally ends the never-ending "You have to
show *this* and not *that*l !!" argue.

Regards,
Thomas

>
> regards
>   Gabor

What are you doing with Perl in the QA department?

[Gabor Szabo]

Hi,

In this question I am not so much interested in how you do unit
testing of your Perl code but how you use Perl in a QA or QC department?


Some of you might know that I have a training course on how to
use Perl in test automation. That course assumes you already know
Perl. (First I teach TAP and how to test Perl code and then things
like WWW::Mechanize, Selenium for web, Expect and Net::Telnet for CLI
and Win32::GUITest for, well GUI related things to test anything else.)


Now I am thinking on building a course or writing a book or just a
series of blog
entries that will not assume any Perl knowledge and will help people in the
QA/QC department and in test automation.
So I wonder, if you are working in QA, what are you doing with Perl?

Is that processing log files?
Is that running batch jobs or wrapping other utilities?
Is that for preparing input files for the test?
Do you also use Perl to test various applications (web, CLI, GUI, etc)?

regards
   Gabor