Re: [Drizzle-discuss] On merging lp:~patrick-crews/drizzle/qp-merge2
Awesome. The branch is now ready for merge. Essentially, this is the same code, but renamed to be less horrible to type/say + cleaned up / updated. Everything is now a python unittest module (except for the dtr stuff). This enables us to write tests in Python, so we are less likely to bang our heads against the wall in certain testing scenarios. This means no more --mode stuff. We just write helper methods to do what we need and we can mix and match fun stuff: execute_randgen(cmd, server) execute_query(query, server) compare_servers_by_checksum(server_list) server.start() server.dump_errlog()...stuff like that. : ) Once our Jenkins jobs are updated / passing, we can clean out the dbqp code \o/. AFAIK, the only Jenkins job that needs changing is staging-crashme. A 'make test-qp' job(s) might also be good, depending on time / machines, but the world will continue to spin without it g. The updates to the Makefile: test-qp:runs the DTR test suite (includes a couple of tests not run by drizzle-test-run - backwards_compatibility + slave stuff) test-randgen-all: runs all randgen suites test-randgen: same as current staging run, but with kewpie test-randgen-basic: runs the basic stress suite test-randgen-slave: randgen slave plugin tests test-randgen-innodb-log: tests the innodb-based trx log test-randgen-trx-log: tests the file-based trx log test-crashme: runs the crashme tests, vs. the current calls to dbqp with a --mode argument. test-sqlbench: runs sql-bench, but doesn't do all the magic drizzle-automation currently does vis-a-vis parsing + storing results in db + emailing comparisons (yet) : ) Please let me know if I can help with anything, Patrick On 02/09/2012 07:00 PM, Mark Atwood wrote: Excellent. Let me know when you push it to LP, and we will work on merging it. Also, a compare and contrast with dbqp would be nice, so I can quickly update the dbqp-using Jenkins targets. ..m On Wed, Feb 8, 2012 at 5:30 PM, pcrewsglee...@gmail.com wrote: Actually, I did a serious strip-down and got the kewpie dir down to 21MB. Still finishing some tests before pushing, but it would enable us to keep everything needed to run tests in-tree and things will be even lighter once we've removed dbqp. Thanks, -- Patrick On 02/06/2012 10:34 PM, Mark Atwood wrote: I like it. Tho I would like make test / make check / make kewpie (whatever is most appropriate to run a kewpie test. If we do it in make test or make check, have it emit a sane warning message if kewpie is not installed, including a URL and a few lines of instructions on installing kewpie. ..m On Mon, Feb 6, 2012 at 4:04 PM, pcrewsglee...@gmail.comwrote: Tell me what you guys think about this: For xtrabackup + other percona projects we are going to keep a config file + the test files in-tree. One should only need one installation of kewpie to run everything. Each project will use an in-tree workdir, etc. The tests living in-tree will help to ensure they are up-to-date and in sync and have a shared history with the code. ./kewpie.py --sys-config=../qp/qp-config.py --wsrep-provider-path=/home/pcrews/bzr/galera2/libgalera_smm.so --force This will *significantly* reduce the bulk of the Drizzle tree. The other thing we can do is to pare down some further test data files that *might* have been missed. An example branch: lp:~patrick-crews/percona-xtradb-cluster/pxc-5520-qp Thoughts? On 01/30/2012 12:30 PM, pcrews wrote: On 01/30/2012 03:25 AM, Henrik Ingo wrote: On Mon, Jan 30, 2012 at 6:16 AM, pcrewsglee...@gmail.comwrote: Hi. It is pretty darned huge as it has the randgen + sql-bench in there as well as tests, etc and I was wondering about this myself. Here are two suggestions: 1) I can remove the Percona stuff. Apologies for not having done it sooner, but I was thinking some tests would prove useful over time, like the cluster_* and xtrabackup_* suites. However, a fair bit of this won't be needed. We already include xtrabackup, so those should be usable immediately. I'm also a little bit positive for the idea of having around one test suite that is capable of testing all kinds of stuff. Like you say, randgen can be used in a way that it tests 3 different databases side by side. However, an issue here is the maintenance and churn: If you add cluster_* tests to drizzle tree now, how do you intend to keep this updated? Will it be broken before anyone tries to add Galera to Drizzle, or will you actually keep it working? If you have some low effort way to make sure the Percona tests, Drizzle tests, etc will stay in sync, then it sounds ok. If they are expected to diverge anyway, then I don't see a point in adding stuff that wouldn't be used with Drizzle specifically. ++ to your thoughts. One of my goals is to minimize moving parts / the amount of work needed to keep things up to date (and I have to do this for several projects). For the cluster_* tests, they aren't tied to Galera.
Re: [Drizzle-discuss] On merging lp:~patrick-crews/drizzle/qp-merge2
Actually, I did a serious strip-down and got the kewpie dir down to 21MB. Still finishing some tests before pushing, but it would enable us to keep everything needed to run tests in-tree and things will be even lighter once we've removed dbqp. Thanks, -- Patrick On 02/06/2012 10:34 PM, Mark Atwood wrote: I like it. Tho I would like make test / make check / make kewpie (whatever is most appropriate to run a kewpie test. If we do it in make test or make check, have it emit a sane warning message if kewpie is not installed, including a URL and a few lines of instructions on installing kewpie. ..m On Mon, Feb 6, 2012 at 4:04 PM, pcrewsglee...@gmail.com wrote: Tell me what you guys think about this: For xtrabackup + other percona projects we are going to keep a config file + the test files in-tree. One should only need one installation of kewpie to run everything. Each project will use an in-tree workdir, etc. The tests living in-tree will help to ensure they are up-to-date and in sync and have a shared history with the code. ./kewpie.py --sys-config=../qp/qp-config.py --wsrep-provider-path=/home/pcrews/bzr/galera2/libgalera_smm.so --force This will *significantly* reduce the bulk of the Drizzle tree. The other thing we can do is to pare down some further test data files that *might* have been missed. An example branch: lp:~patrick-crews/percona-xtradb-cluster/pxc-5520-qp Thoughts? On 01/30/2012 12:30 PM, pcrews wrote: On 01/30/2012 03:25 AM, Henrik Ingo wrote: On Mon, Jan 30, 2012 at 6:16 AM, pcrewsglee...@gmail.com wrote: Hi. It is pretty darned huge as it has the randgen + sql-bench in there as well as tests, etc and I was wondering about this myself. Here are two suggestions: 1) I can remove the Percona stuff. Apologies for not having done it sooner, but I was thinking some tests would prove useful over time, like the cluster_* and xtrabackup_* suites. However, a fair bit of this won't be needed. We already include xtrabackup, so those should be usable immediately. I'm also a little bit positive for the idea of having around one test suite that is capable of testing all kinds of stuff. Like you say, randgen can be used in a way that it tests 3 different databases side by side. However, an issue here is the maintenance and churn: If you add cluster_* tests to drizzle tree now, how do you intend to keep this updated? Will it be broken before anyone tries to add Galera to Drizzle, or will you actually keep it working? If you have some low effort way to make sure the Percona tests, Drizzle tests, etc will stay in sync, then it sounds ok. If they are expected to diverge anyway, then I don't see a point in adding stuff that wouldn't be used with Drizzle specifically. ++ to your thoughts. One of my goals is to minimize moving parts / the amount of work needed to keep things up to date (and I have to do this for several projects). For the cluster_* tests, they aren't tied to Galera. They're written in a generic fashion so one can test replication against Galera, mysql-built-in, etc...provided you've written appropriate methods (set_master, start_slave, etc) for the server type. Once I have time to do this for Drizzle, we'll have a nifty suite for the slave plugin. Also, I took a look at the branch and I had removed everything percona_test related. The only additional deletion was removing some mysql data files. I will be cleaning up / consolidating the server code soon, so that should cut out a bit as well. 2) We've been discussing this for Percona projects: Have the test code live outside the tree and simply keep a config file in the drizzle tree that says things like I am a drizzle server. This is my basedir. Run these suites, etc I like the tests being in the main tree. For same reasons that I have now included rpm and deb stuff into the tree. Since we have become reasonably good at requiring developers to also create tests as part of their merge proposals, I wouldn't want to have arrangements where you hear sentences like: This is the new feature I've developed and I promise that some day I will also add some tests to this other branch. Yes, that introduces gaps that rely as much on prayer and miracles as process when you put it that way. Ah, how much easier life was when it was *just* Drizzle that I was beating on ; ) I'll have an updated branch with #1 done tomorrow. Let me know what you guys think about #2. henrik Thanks for the input, Patrick ___ Mailing list: https://launchpad.net/~drizzle-discuss Post to : drizzle-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
Re: [Drizzle-discuss] On merging lp:~patrick-crews/drizzle/qp-merge2
Tell me what you guys think about this: For xtrabackup + other percona projects we are going to keep a config file + the test files in-tree. One should only need one installation of kewpie to run everything. Each project will use an in-tree workdir, etc. The tests living in-tree will help to ensure they are up-to-date and in sync and have a shared history with the code. ./kewpie.py --sys-config=../qp/qp-config.py --wsrep-provider-path=/home/pcrews/bzr/galera2/libgalera_smm.so --force This will *significantly* reduce the bulk of the Drizzle tree. The other thing we can do is to pare down some further test data files that *might* have been missed. An example branch: lp:~patrick-crews/percona-xtradb-cluster/pxc-5520-qp Thoughts? On 01/30/2012 12:30 PM, pcrews wrote: On 01/30/2012 03:25 AM, Henrik Ingo wrote: On Mon, Jan 30, 2012 at 6:16 AM, pcrewsglee...@gmail.com wrote: Hi. It is pretty darned huge as it has the randgen + sql-bench in there as well as tests, etc and I was wondering about this myself. Here are two suggestions: 1) I can remove the Percona stuff. Apologies for not having done it sooner, but I was thinking some tests would prove useful over time, like the cluster_* and xtrabackup_* suites. However, a fair bit of this won't be needed. We already include xtrabackup, so those should be usable immediately. I'm also a little bit positive for the idea of having around one test suite that is capable of testing all kinds of stuff. Like you say, randgen can be used in a way that it tests 3 different databases side by side. However, an issue here is the maintenance and churn: If you add cluster_* tests to drizzle tree now, how do you intend to keep this updated? Will it be broken before anyone tries to add Galera to Drizzle, or will you actually keep it working? If you have some low effort way to make sure the Percona tests, Drizzle tests, etc will stay in sync, then it sounds ok. If they are expected to diverge anyway, then I don't see a point in adding stuff that wouldn't be used with Drizzle specifically. ++ to your thoughts. One of my goals is to minimize moving parts / the amount of work needed to keep things up to date (and I have to do this for several projects). For the cluster_* tests, they aren't tied to Galera. They're written in a generic fashion so one can test replication against Galera, mysql-built-in, etc...provided you've written appropriate methods (set_master, start_slave, etc) for the server type. Once I have time to do this for Drizzle, we'll have a nifty suite for the slave plugin. Also, I took a look at the branch and I had removed everything percona_test related. The only additional deletion was removing some mysql data files. I will be cleaning up / consolidating the server code soon, so that should cut out a bit as well. 2) We've been discussing this for Percona projects: Have the test code live outside the tree and simply keep a config file in the drizzle tree that says things like I am a drizzle server. This is my basedir. Run these suites, etc I like the tests being in the main tree. For same reasons that I have now included rpm and deb stuff into the tree. Since we have become reasonably good at requiring developers to also create tests as part of their merge proposals, I wouldn't want to have arrangements where you hear sentences like: This is the new feature I've developed and I promise that some day I will also add some tests to this other branch. Yes, that introduces gaps that rely as much on prayer and miracles as process when you put it that way. Ah, how much easier life was when it was *just* Drizzle that I was beating on ; ) I'll have an updated branch with #1 done tomorrow. Let me know what you guys think about #2. henrik Thanks for the input, Patrick ___ Mailing list: https://launchpad.net/~drizzle-discuss Post to : drizzle-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
Re: [Drizzle-discuss] On merging lp:~patrick-crews/drizzle/qp-merge2
I like it. Tho I would like make test / make check / make kewpie (whatever is most appropriate to run a kewpie test. If we do it in make test or make check, have it emit a sane warning message if kewpie is not installed, including a URL and a few lines of instructions on installing kewpie. ..m On Mon, Feb 6, 2012 at 4:04 PM, pcrews glee...@gmail.com wrote: Tell me what you guys think about this: For xtrabackup + other percona projects we are going to keep a config file + the test files in-tree. One should only need one installation of kewpie to run everything. Each project will use an in-tree workdir, etc. The tests living in-tree will help to ensure they are up-to-date and in sync and have a shared history with the code. ./kewpie.py --sys-config=../qp/qp-config.py --wsrep-provider-path=/home/pcrews/bzr/galera2/libgalera_smm.so --force This will *significantly* reduce the bulk of the Drizzle tree. The other thing we can do is to pare down some further test data files that *might* have been missed. An example branch: lp:~patrick-crews/percona-xtradb-cluster/pxc-5520-qp Thoughts? On 01/30/2012 12:30 PM, pcrews wrote: On 01/30/2012 03:25 AM, Henrik Ingo wrote: On Mon, Jan 30, 2012 at 6:16 AM, pcrewsglee...@gmail.com wrote: Hi. It is pretty darned huge as it has the randgen + sql-bench in there as well as tests, etc and I was wondering about this myself. Here are two suggestions: 1) I can remove the Percona stuff. Apologies for not having done it sooner, but I was thinking some tests would prove useful over time, like the cluster_* and xtrabackup_* suites. However, a fair bit of this won't be needed. We already include xtrabackup, so those should be usable immediately. I'm also a little bit positive for the idea of having around one test suite that is capable of testing all kinds of stuff. Like you say, randgen can be used in a way that it tests 3 different databases side by side. However, an issue here is the maintenance and churn: If you add cluster_* tests to drizzle tree now, how do you intend to keep this updated? Will it be broken before anyone tries to add Galera to Drizzle, or will you actually keep it working? If you have some low effort way to make sure the Percona tests, Drizzle tests, etc will stay in sync, then it sounds ok. If they are expected to diverge anyway, then I don't see a point in adding stuff that wouldn't be used with Drizzle specifically. ++ to your thoughts. One of my goals is to minimize moving parts / the amount of work needed to keep things up to date (and I have to do this for several projects). For the cluster_* tests, they aren't tied to Galera. They're written in a generic fashion so one can test replication against Galera, mysql-built-in, etc...provided you've written appropriate methods (set_master, start_slave, etc) for the server type. Once I have time to do this for Drizzle, we'll have a nifty suite for the slave plugin. Also, I took a look at the branch and I had removed everything percona_test related. The only additional deletion was removing some mysql data files. I will be cleaning up / consolidating the server code soon, so that should cut out a bit as well. 2) We've been discussing this for Percona projects: Have the test code live outside the tree and simply keep a config file in the drizzle tree that says things like I am a drizzle server. This is my basedir. Run these suites, etc I like the tests being in the main tree. For same reasons that I have now included rpm and deb stuff into the tree. Since we have become reasonably good at requiring developers to also create tests as part of their merge proposals, I wouldn't want to have arrangements where you hear sentences like: This is the new feature I've developed and I promise that some day I will also add some tests to this other branch. Yes, that introduces gaps that rely as much on prayer and miracles as process when you put it that way. Ah, how much easier life was when it was *just* Drizzle that I was beating on ; ) I'll have an updated branch with #1 done tomorrow. Let me know what you guys think about #2. henrik Thanks for the input, Patrick ___ Mailing list: https://launchpad.net/~drizzle-discuss Post to : drizzle-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
[Drizzle-discuss] On merging lp:~patrick-crews/drizzle/qp-merge2
Hi! We have lp:~patrick-crews/drizzle/qp-merge2 sitting in the Branch Merge Proposal list on Launchpad. It is 166905 lines, which is just over 10% of the existing 1431202 lines. I only really noticed it's huge size when I was trying to run the merge last week while I was in Australia, and BZR had taken over 10 minutes before I aborted it. On the other hand, it appears that the existing dbqp is itself nearly as big. It appears to me that kewpie contains code to also test stock MySQL and to test Percona server. While I am a fan of putting the test code in-tree (it certainly makes parts of my own Drizzle work much easier), I have hesitations about carrying around the weight of test code for other projects as well. Any thoughts about my concerns? Or should I just suck it up and merge it? ..m ___ Mailing list: https://launchpad.net/~drizzle-discuss Post to : drizzle-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
