Hello, > #!/bin/sh > rm db/foreman_test.sqlite3 db/development.sqlite3 > bundle exec rake db:drop db:create db:migrate > bundle exec rake db:migrate > bundle exec rake db:migrate:status > bundle exec rake jenkins:unit jenkins:integration TESTOPTS=-v
What happens here is that db:migrate and db:migrate:status operate on the development database and the test one remains untouched. However, running the test for the first time automagically (and silently) migrates the test db and the tests fail. Running the tests once more makes them pass. $ bundle exec rake db:drop db:create db:migrate | tail == 20160914125418 UpdateParameterPriorities: migrated (0.0001s) =============== == 20160922144222 ChangeFactNameToPuppetFactName: migrating =================== == 20160922144222 ChangeFactNameToPuppetFactName: migrated (0.0028s) ========== == 20160924213018 ChangeWidgetNames: migrating ================================ == 20160924213018 ChangeWidgetNames: migrated (0.0069s) ======================= Recreating cache 2016-10-05T11:55:50 [app] [W] Recreating the whole DB cache $ bundle exec rake db:migrate:status RAILS_ENV=test The Apipie cache is turned off. Enable it and run apipie:cache rake task to speed up API calls. database: foreman_test Status Migration ID Migration Name -------------------------------------------------- down 20090714132448 Create hosts down 20090714132449 Add audits table down 20090715143858 Create architectures down 20090717025820 Create media $ ruby -I test -I lib test/unit/foreman/access_permissions_test.rb # Same results with bundle exec rake jenkins:unit # Output omitted, permission tests failing here 868 runs, 867 assertions, 75 failures, 0 errors, 1 skips $ bundle exec rake db:migrate:status RAILS_ENV=test 2>/dev/null | head The Apipie cache is turned off. Enable it and run apipie:cache rake task to speed up API calls. database: foreman_test Status Migration ID Migration Name -------------------------------------------------- up 20090714132448 Create hosts up 20090714132449 Add audits table up 20090715143858 Create architectures up 20090717025820 Create media $ ruby -I test -I lib test/unit/foreman/access_permissions_test.rb # Same results with bundle exec rake jenkins:unit # Output omitted, tests passing here 868 runs, 867 assertions, 0 failures, 0 errors, 1 skips What I don't get how this situation could arise on Jenkins, where both test and development databases point to the same db. Adam On Wed, Oct 5, 2016 at 11:27 AM, Daniel Lobato <elobat...@gmail.com> wrote: > Fairly certain now it happens because of the removal of the lib task on > that PR you link. It used to run migrations twice, once on units, and > yet again on 'lib', but now 'units' runs the lib task twice. > > You can reproduce by running this: > > #!/bin/sh > rm db/foreman_test.sqlite3 db/development.sqlite3 > bundle exec rake db:drop db:create db:migrate > bundle exec rake db:migrate > bundle exec rake db:migrate:status > bundle exec rake jenkins:unit jenkins:integration TESTOPTS=-v > > with this patch (https://github.com/theforeman/foreman/pull/3916) and > you'll see the problem is that plugins believe there are pending > migrations. This is false though, as bundle exec rake db:migrate:status > confirms it in the script. > > I'm now checking if the pending_migrations method is wrong and how can I > fix it to detect that better. > > > On 10/04, Daniel Lobato wrote: > > Remove your test db, then try RAILS_ENV=test ruby -I'test' > > test/unit/foreman/access_permissions_test.rb and you'll see it > > happening. I think it's just because tests are loaded earlier than > > permissions from plugins now. > > > > On Tue, Oct 4, 2016 at 12:39 PM, Ivan Necas <ine...@redhat.com> wrote: > > > So far, I can tell this PR > > > https://github.com/theforeman/foreman/pull/3843 introduced the > > > behaviour > > > > > > as it's green on > > > http://ci.theforeman.org/job/test_plugin_matrix/2059/ > database=mysql,ruby=2.3,slave=fast/ > > > and red on following commit > > > http://ci.theforeman.org/job/test_plugin_matrix/2060/ > database=mysql,ruby=2.3,slave=fast/ > > > > > > Perhaps the change is the access permissions test was in lib, and not > > > it's part of the unit > > > > > > Still no idea why it happens in jenkins only > > > > > > -- Ivan > > > > > > On Mon, Oct 3, 2016 at 10:35 PM, Ivan Necas <ine...@redhat.com> wrote: > > >> The only think I can tell so far is that I can't reproduce this > > >> locally and I have no > > >> idea on what might be different from my vs. jenkins environment that > causes this > > >> to happen. > > >> > > >> -- Ivan > > >> > > >> On Mon, Oct 3, 2016 at 10:28 PM, Stephen Benjamin <step...@redhat.com> > wrote: > > >>> Tasks, salt and a bunch of plugins are failing tests, has anyone has > a chance to look into it? e.g.: > > >>> > > >>> http://ci.theforeman.org/job/test_plugin_matrix/2015/# > showFailuresLink > > >>> > > >>> This most recent failure has me thinking again about the > difficulties of being a plugin maintainer > > >>> for Foreman. > > >>> > > >>> Would it be possible to get a standard plugin in Foreman's test > matrix for PR? Perhaps tasks is a good > > >>> candidate. Not on all databases/rubies, but something might help > gain awareness of the impact of > > >>> certain changes on plugins. Katello is there, but it doesn't run all > (any?) of Foreman's tests. > > >>> > > >>> Along side that, we should be tracking what plugin maintainers need > to do to update from one release > > >>> to the next. > > >>> > > >>> It really is a recurring problem for me, that Salt is constantly > broken due to incompatibilities > > >>> introduced in Foreman. We do a poor job of tracking them. Maybe > if I'm lucky someone is nice > > >>> enough to send a mail to foreman-dev. Or if the planets align, > someone opens a PR to foreman_salt > > >>> and fixes it. But more often than not, my project just goes red and > I have to figure out why. > > >>> But this is just a side thing, and I rarely get to immediately look > at failures. They end up piling > > >>> up. > > >>> > > >>> Very frustrating, as when I do find time to work on it, it's not to > do something cool, but just > > >>> to figure out what changed in core and how to fix it in the plugin > :-\ > > >>> > > >>> > > >>> - Stephen > > >>> > > >>> -- > > >>> You received this message because you are subscribed to the Google > Groups "foreman-dev" group. > > >>> To unsubscribe from this group and stop receiving emails from it, > send an email to foreman-dev+unsubscr...@googlegroups.com. > > >>> For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "foreman-dev" group. > > > To unsubscribe from this group and stop receiving emails from it, send > an email to foreman-dev+unsubscr...@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > -- > > Daniel Lobato > > > > @dlobatog > > daniellobato.me > > > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > > -- > Daniel Lobato Garcia > > @dLobatog > blog.daniellobato.me > daniellobato.me > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > Keybase: https://keybase.io/elobato > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
