Re: [galaxy-dev] API v/s twill based testing
On Fri, Jul 18, 2014 at 5:14 PM, Dave Bouvier wrote: > John, Peter, > > The buildbot builders are already using the api interactor for both > functional tests and the install and test framework. > > --Dave B. Great news. When did that happen? Did it cause any regressions (and can/did you flag those to the repository authors to alert them)? Assuming that change over went smoothly, is the plan for changing the default test back-end in the master branch of galaxy-central (and thus eventually galaxy-stable) for those running tool tests locally? Thanks, Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] API v/s twill based testing
John, Peter, The buildbot builders are already using the api interactor for both functional tests and the install and test framework. --Dave B. On Fri 18 Jul 2014 11:34:01 AM EDT, John Chilton wrote: I think the confusion over two different test methods has reached a high level and Dave and I agreed previously that we should try to make the API driven version the default during this release cycle - but it is not our call :). Greg - is there anything else you feel that needs to happen to at least switch the test tool shed over and see how things go? Once the main tool shed is ready to target API testing we can swap over the default and issues like this and Aaron's earlier this week with collections will hopefully not cause confusion. -John On Wed, Jul 16, 2014 at 2:13 PM, Peter Cock wrote: On Wed, Jul 16, 2014 at 7:44 PM, Saket Choudhary wrote: Thanks Peter, I guess I should then rely on API based tests. If it is just the order, make sure the order of the output files in the is consistent with that in the and it make be OK with Twill... I wonder if I filed a Trello card on this, or just an email? Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] API v/s twill based testing
I think the confusion over two different test methods has reached a high level and Dave and I agreed previously that we should try to make the API driven version the default during this release cycle - but it is not our call :). Greg - is there anything else you feel that needs to happen to at least switch the test tool shed over and see how things go? Once the main tool shed is ready to target API testing we can swap over the default and issues like this and Aaron's earlier this week with collections will hopefully not cause confusion. -John On Wed, Jul 16, 2014 at 2:13 PM, Peter Cock wrote: > On Wed, Jul 16, 2014 at 7:44 PM, Saket Choudhary wrote: >> Thanks Peter, I guess I should then rely on API based tests. >> > > If it is just the order, make sure the order of the output files in the > is consistent with that in the and it make be OK with Twill... > I wonder if I filed a Trello card on this, or just an email? > > Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] API v/s twill based testing
On Wed, Jul 16, 2014 at 7:44 PM, Saket Choudhary wrote: > Thanks Peter, I guess I should then rely on API based tests. > If it is just the order, make sure the order of the output files in the is consistent with that in the and it make be OK with Twill... I wonder if I filed a Trello card on this, or just an email? Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] API v/s twill based testing
Thanks Peter, I guess I should then rely on API based tests. On 15 July 2014 14:18, Peter Cock wrote: > Hi Saket, > > From memory the Twill tests are fragile with the output file order in the > XML. > > John was discussing switching the default from the Twill to API backend, > not sure when that is happening though... > > Peter > > On Tue, Jul 15, 2014 at 9:31 AM, Saket Choudhary > wrote: > > I recently updated tests for one of my wrappers and came across this > strange > > behaviour: > > > > The twill based testing reports a failure: > > https://travis-ci.org/saketkc/galaxy_tools/jobs/29956682#L1463 > > > > whereas, the API based testing shows success: > > https://travis-ci.org/saketkc/galaxy_tools/jobs/29956683 > > > > Unfortunately I cannot run these tests locally since I am behind a system > > proxy [Refer: > > > http://dev.list.galaxyproject.org/Functional-Tests-and-ftype-td4664233.html > ] > > and have to rely on travis.. > > > > The place where twii tests fails shows that it is trying to compare the > > diff between 'chasm_output_genes.tabular' and > > 'chasm_output_variants.tabular' instead of 'chasm_output_genes.tabular'. > > [See : https://travis-ci.org/saketkc/galaxy_tools/jobs/29956682#L1469] > > > > I tried running my tools locally and I did not come across any case where > > the 'variants' output gets replaced by the 'genes' output, thus possibly > > ruling out unexpected behavior from the tool's server end. > > > > Is this a possible bug or am I missing something? > > > > Saket > > > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] API v/s twill based testing
Hi Saket, >From memory the Twill tests are fragile with the output file order in the XML. John was discussing switching the default from the Twill to API backend, not sure when that is happening though... Peter On Tue, Jul 15, 2014 at 9:31 AM, Saket Choudhary wrote: > I recently updated tests for one of my wrappers and came across this strange > behaviour: > > The twill based testing reports a failure: > https://travis-ci.org/saketkc/galaxy_tools/jobs/29956682#L1463 > > whereas, the API based testing shows success: > https://travis-ci.org/saketkc/galaxy_tools/jobs/29956683 > > Unfortunately I cannot run these tests locally since I am behind a system > proxy [Refer: > http://dev.list.galaxyproject.org/Functional-Tests-and-ftype-td4664233.html] > and have to rely on travis.. > > The place where twii tests fails shows that it is trying to compare the > diff between 'chasm_output_genes.tabular' and > 'chasm_output_variants.tabular' instead of 'chasm_output_genes.tabular'. > [See : https://travis-ci.org/saketkc/galaxy_tools/jobs/29956682#L1469] > > I tried running my tools locally and I did not come across any case where > the 'variants' output gets replaced by the 'genes' output, thus possibly > ruling out unexpected behavior from the tool's server end. > > Is this a possible bug or am I missing something? > > Saket > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
