Awesome. The branch is now ready for merge.
Essentially, this is the same code, but renamed to be less horrible to
type/say + cleaned up / updated. Everything is now a python unittest
module (except for the dtr stuff).
This enables us to write tests in Python, so we are less likely to bang
our heads against the wall in certain testing scenarios.
This means no more --mode stuff. We just write helper methods to do
what we need and we can mix and match fun stuff:
execute_randgen(cmd, server)
execute_query(query, server)
compare_servers_by_checksum(server_list)
server.start()
server.dump_errlog()...stuff like that.
: )
Once our Jenkins jobs are updated / passing, we can clean out the dbqp
code \o/. AFAIK, the only Jenkins job that needs changing is
staging-crashme. A 'make test-qp' job(s) might also be good, depending
on time / machines, but the world will continue to spin without it <g>.
The updates to the Makefile:
test-qp:runs the DTR test suite (includes a couple of tests not run by
drizzle-test-run - backwards_compatibility + slave stuff)
test-randgen-all: runs all randgen suites
test-randgen: same as current staging run, but with kewpie
test-randgen-basic: runs the basic stress suite
test-randgen-slave: randgen slave plugin tests
test-randgen-innodb-log: tests the innodb-based trx log
test-randgen-trx-log: tests the file-based trx log
test-crashme: runs the crashme tests, vs. the current calls to dbqp with
a --mode argument.
test-sqlbench: runs sql-bench, but doesn't do all the magic
drizzle-automation currently does vis-a-vis parsing + storing results in
db + emailing comparisons (yet) >: )
Please let me know if I can help with anything,
Patrick
On 02/09/2012 07:00 PM, Mark Atwood wrote:
Excellent. Let me know when you push it to LP, and we will work on merging it.
Also, a compare and contrast with dbqp would be nice, so I can quickly
update the dbqp-using Jenkins targets.
..m
On Wed, Feb 8, 2012 at 5:30 PM, pcrews<[email protected]> wrote:
Actually, I did a serious strip-down and got the kewpie dir down to 21MB.
Still finishing some tests before pushing, but it would enable us to keep
everything needed to run tests in-tree and things will be even lighter once
we've removed dbqp.
Thanks,
--
Patrick
On 02/06/2012 10:34 PM, Mark Atwood wrote:
I like it.
Tho I would like "make test" / "make check" / "make kewpie" (whatever
is most appropriate to run a kewpie test.
If we do it in make test or make check, have it emit a sane warning
message if kewpie is not installed, including a URL and a few lines of
instructions on installing kewpie.
..m
On Mon, Feb 6, 2012 at 4:04 PM, pcrews<[email protected]> wrote:
Tell me what you guys think about this:
For xtrabackup + other percona projects we are going to keep a config
file +
the test files in-tree. One should only need one installation of kewpie
to
run everything. Each project will use an in-tree workdir, etc. The
tests
living in-tree will help to ensure they are up-to-date and in sync and
have
a shared history with the code.
./kewpie.py --sys-config=../qp/qp-config.py
--wsrep-provider-path=/home/pcrews/bzr/galera2/libgalera_smm.so --force
This will *significantly* reduce the bulk of the Drizzle tree. The other
thing we can do is to pare down some further test data files that *might*
have been missed.
An example branch: lp:~patrick-crews/percona-xtradb-cluster/pxc-5520-qp
Thoughts?
On 01/30/2012 12:30 PM, pcrews wrote:
On 01/30/2012 03:25 AM, Henrik Ingo wrote:
On Mon, Jan 30, 2012 at 6:16 AM, pcrews<[email protected]> wrote:
Hi. It is pretty darned huge as it has the randgen + sql-bench in
there as
well as tests, etc and I was wondering about this myself.
Here are two suggestions:
1) I can remove the Percona stuff. Apologies for not having done it
sooner, but I was thinking some tests would prove useful over time,
like the
cluster_* and xtrabackup_* suites. However, a fair bit of this won't
be
needed.
We already include xtrabackup, so those should be usable immediately.
I'm also a little bit positive for the idea of having around one test
suite that is capable of testing all kinds of stuff. Like you say,
randgen can be used in a way that it tests 3 different databases side
by side. However, an issue here is the maintenance and churn: If you
add cluster_* tests to drizzle tree now, how do you intend to keep
this updated? Will it be broken before anyone tries to add Galera to
Drizzle, or will you actually keep it working? If you have some low
effort way to make sure the Percona tests, Drizzle tests, etc will
stay in sync, then it sounds ok. If they are expected to diverge
anyway, then I don't see a point in adding stuff that wouldn't be used
with Drizzle specifically.
++ to your thoughts. One of my goals is to minimize moving parts / the
amount of work needed to keep things up to date (and I have to do this
for several projects). For the cluster_* tests, they aren't tied to
Galera. They're written in a generic fashion so one can test replication
against Galera, mysql-built-in, etc...provided you've written
appropriate methods (set_master, start_slave, etc) for the server type.
Once I have time to do this for Drizzle, we'll have a nifty suite for
the slave plugin.
Also, I took a look at the branch and I had removed everything
percona_test related. The only additional deletion was removing some
mysql data files. I will be cleaning up / consolidating the server code
soon, so that should cut out a bit as well.
2) We've been discussing this for Percona projects: Have the test code
live outside the tree and simply keep a config file in the drizzle
tree that
says things like "I am a drizzle server. This is my basedir. Run these
suites, etc"
I like the tests being in the main tree. For same reasons that I have
now included rpm and deb stuff into the tree. Since we have become
reasonably good at requiring developers to also create tests as part
of their merge proposals, I wouldn't want to have arrangements where
you hear sentences like: "This is the new feature I've developed and I
promise that some day I will also add some tests to this other
branch."
Yes, that introduces gaps that rely as much on prayer and miracles as
process when you put it that way. Ah, how much easier life was when it
was *just* Drizzle that I was beating on ; )
I'll have an updated branch with #1 done tomorrow. Let me know what
you
guys think about #2.
henrik
Thanks for the input,
Patrick
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
