Can a before hook ever access the metadata of an example (before(:each)?) A before(:each)/before(:example) hook can access the metadata of the example, but a before(:all)/before(:context) hook cannot, because in the latter case, there is no specific example the hook is running as a part of. It’s another reason to generally avoid :all/:context hooks.
To access the metadata from a hook, you just receive an example argument, which has a metadata method: before(:example) do |example| pp example.metadataend HTH, Myron On Wed, Dec 20, 2017 at 11:15 AM, Jan P. <[email protected]> wrote: > Thanks for the explanation, now it makes sense what was happening. > > I applied that to our project and it is working as expected. > Thank you so much for your help. > > > Can a `before` hook ever access the metadata of an example > (`before(:each)`?), or does this only get set after `before` is already > finished executing? > I am lucky to not have a test file right now where I have to differentiate > between tests with a `before` that should run and tests that shold not - > but I am sure this will happen sooner or later :/ > > Jan > > > > Am Mittwoch, 20. Dezember 2017 17:30:11 UTC+1 schrieb Myron Marston: >> >> The problem is that you have only set :requires_xcodebuild on the >> individual examples, and not on the example group as a whole. So when the >> example group is defined, there is no :requires_xcodebuild metadata, and >> the define_derived_metadata block does not get invoked to set :skip. >> Then when you call before, metadata[:skip] returns false, so it proceeds >> to define the hook normally. >> >> RSpec examples inherit metadata from their parent example group (and >> enclosing example groups inherit from their parent group), so my suggestion >> is to remove :requires_xcodebuild from the individual examples in that >> file, and set it on the example group instead. That’ll both fix the issue >> and ensure that every example automatically has that metadata. >> >> Myron >> >> >> On Wed, Dec 20, 2017 at 6:34 AM, Jan P. <[email protected]> wrote: >> >>> That sounds perfect for my use case. As I introduced `:skip` into the >>> codebase, I would be pretty comfortable with adding this as well (and >>> confine it to non-macOS platforms to limit exposure). >>> >>> But I can't get it to work. The `before` just runs anyway: >>> >>> https://circleci.com/gh/fastlane/fastlane/12612 >>> https://github.com/fastlane/fastlane/blob/a103c655c4162723b9 >>> 6cae8f4a8b1258a1419321/spec_helper.rb#L90-L98 >>> (Tried integrating it into my existing RSpec.configure block as well: >>> https://github.com/fastlane/fastlane/pull/11284/commits/1815 >>> b4bca04f58ec84fd365097f61b8dd4e68b89#diff-82c4f19cbedb2b12a4 >>> 1b04cd4444aa3cR87) >>> >>> What am I missing? >>> >>> Jan >>> >>> >>> Am Mittwoch, 20. Dezember 2017 09:04:32 UTC+1 schrieb Myron Marston: >>>> >>>> There is a way you can simplify this further: >>>> >>>> module HookOverrides >>>> def before(*args) >>>> super unless metadata[:skip] >>>> endend >>>> RSpec.configure do |c| >>>> c.extend HookOverridesend >>>> >>>> This overrides before so that it’s a no-op if :skip metadata is set on >>>> the example group. With that in place, you don’t need to call skip >>>> from your before(:all) hooks. But bear in mind that if you ever set >>>> :skip on an example group (to set a default for the group) and then >>>> set skip: false for a specific example in the group…this override will >>>> cause the before hook to be skipped even though you’d probably expect >>>> it in that case. (Which is why we can’t apply this as a generic patch to >>>> RSpec itself). >>>> >>>> HTH, >>>> Myron >>>> >>>> >>>> On Tue, Dec 19, 2017 at 5:45 PM, Jan P. <[email protected]> wrote: >>>> >>>>> Thanks Myron, that worked! >>>>> >>>>> > 4704 examples, 0 failures, 175 pending >>>>> >>>>> With code: >>>>> >>>>> before(:all) do >>>>> skip "Requires `xcodebuild` to be installed which is not possible >>>>> on this platform" unless FastlaneCore::Helper.is_mac? >>>>> >>>>> As you can see right now I duplicate the reason text, and use the >>>>> condition that also triggers `skip`/metadata block: >>>>> https://github.com/fastlane/fastlane/blob/c6b1ac4621941b2efe >>>>> f702b923a572357c64eea9/spec_helper.rb#L68-L84 >>>>> >>>>> Is there somehow a better way to "connect" the skip in `before` to the >>>>> filters/metadata defined here? >>>>> It would be nicer if I could e.g. call a function with the "metadata" >>>>> ("xcodebuild") as parameter to trigger the skip or something. Any idea? >>>>> Best, >>>>> Jan >>>>> >>>>> >>>>> >>>>> Am Mittwoch, 20. Dezember 2017 02:11:56 UTC+1 schrieb Myron Marston: >>>>>> >>>>>> Unfortunately, there’s no way to use :skip metadata at the group >>>>>> level to skip before(:all) hooks. That’s because of how metadata in >>>>>> RSpec is modeled, and how RSpec implements :skip. :skip metadata is >>>>>> handled at the level of individual examples, and metadata is inherited >>>>>> from >>>>>> a group to its enclosing examples. So, for example, this works: >>>>>> >>>>>> RSpec.describe MyClass, :skip do >>>>>> it("is skipped"){ } >>>>>> it("is also skipped") { }end >>>>>> >>>>>> Both examples will be skipped here. >>>>>> >>>>>> But consider that you can also do this: >>>>>> >>>>>> RSpec.describe MyClass, :skip do >>>>>> it("is skipped"){ } >>>>>> it("is also skipped") { } >>>>>> it("is not skipped", skip: false) { }end >>>>>> >>>>>> Here we have on example that is overriding the skip: true metadata >>>>>> inherited from the group. As a result, a before(:all) hook can’t >>>>>> simply look at the group metadata to decide whether or not to skip or >>>>>> not. >>>>>> In general, before(:all) hooks have lots of gotchas like this >>>>>> because they don’t really fit well into the per-example semantics of so >>>>>> many parts of RSpec. If you can refactor your tests to no longer need >>>>>> such >>>>>> a hook (possibly using a aggregate_failures >>>>>> <https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures>), >>>>>> that is worth considering. Besides that, the other option you can do is >>>>>> to >>>>>> manually call the skip method from your before(:all) hook: >>>>>> >>>>>> before(:all) do >>>>>> skip "reason to skip" if should_skip? >>>>>> # rest of your hook logicend >>>>>> >>>>>> HTH, >>>>>> Myron >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 19, 2017 at 3:47 PM, Jan P. <[email protected]> wrote: >>>>>> >>>>>>> Jon, I tried >>>>>>> >>>>>>> describe "xcpretty reporter options generation", requires_xcodebuild: >>>>>>> true do >>>>>>> >>>>>>> >>>>>>> and >>>>>>> >>>>>>> before(:all), requires_xcodebuild: true do >>>>>>> >>>>>>> >>>>>>> and >>>>>>> >>>>>>> >>>>>>> describe Scan::XCPrettyReporterOptionsGenerator, requires_xcodebuild: >>>>>>> true do >>>>>>> >>>>>>> >>>>>>> but all didn't work and the tests were logged as failure because of >>>>>>> the `before` being executed. >>>>>>> >>>>>>> Am I doing this wrong somehow? >>>>>>> >>>>>>> >>>>>>> Code (last iteration) is here: >>>>>>> https://github.com/fastlane/fastlane/blob/janpio-mark_skippe >>>>>>> d_tests_as_pending_with_reason/scan/spec/xcpretty_reporter_ >>>>>>> options_generator_spec.rb >>>>>>> https://github.com/fastlane/fastlane/blob/janpio-mark_skippe >>>>>>> d_tests_as_pending_with_reason/spec_helper.rb#L68-L83 >>>>>>> Matching Circle CI run: >>>>>>> https://circleci.com/gh/fastlane/fastlane/12586 >>>>>>> >>>>>>> -Jan >>>>>>> >>>>>>> >>>>>>> >>>>>>> Am Dienstag, 19. Dezember 2017 23:05:53 UTC+1 schrieb Jon Rowe: >>>>>>>> >>>>>>>> You should be able to specify `requires_xcodebuild: true` on the >>>>>>>> describe as we’ll to prevent the before all being triggered >>>>>>>> >>>>>>>> Jon Rowe >>>>>>>> --------------------------- >>>>>>>> [email protected] >>>>>>>> jonrowe.co.uk >>>>>>>> >>>>>>>> On Wednesday, 20 December 2017 at 05:56, Jan P. wrote: >>>>>>>> >>>>>>>> Excuse the terrible code formatting, seems Google Groups wanted to >>>>>>>> be helpful. >>>>>>>> >>>>>>>> Another try: >>>>>>>> >>>>>>>> describe Scan do >>>>>>>> describe Scan::XCPrettyReporterOptionsGenerator do >>>>>>>> before(:all) do >>>>>>>> // code that fails when executed on non-macOS >>>>>>>> end >>>>>>>> >>>>>>>> describe "xcpretty reporter options generation" do >>>>>>>> it "generates options for the junit tempfile report required by >>>>>>>> scan", requires_xcodebuild: true do >>>>>>>> ... >>>>>>>> >>>>>>>> -J >>>>>>>> >>>>>>>> >>>>>>>> Am Dienstag, 19. Dezember 2017 19:55:45 UTC+1 schrieb Jan P.: >>>>>>>> >>>>>>>> Yes, that works mostly like expected: >>>>>>>> >>>>>>>> >>>>>>>> 4704 examples, 21 failures, 154 pending >>>>>>>> >>>>>>>> >>>>>>>> and >>>>>>>> >>>>>>>> ... >>>>>>>> [18:39:20]: ▸ Pending: (Failures listed here are expected and do >>>>>>>> not affect your suite's status) >>>>>>>> [18:39:20]: ▸ 1) Fastlane Fastlane::EnvironmentPrinter contains >>>>>>>> main information about the stack >>>>>>>> [18:39:20]: ▸ # Requires Xcode to be installed which is not >>>>>>>> possible on this platform >>>>>>>> [18:39:20]: ▸ # ./fastlane/spec/env_spec.rb:28 >>>>>>>> [18:39:20]: ▸ 2) Fastlane Fastlane::EnvironmentPrinter >>>>>>>> FastlaneCore::Helper.xcode_version cannot be obtained contains >>>>>>>> stack information other than Xcode Version >>>>>>>> [18:39:20]: ▸ # Requires Xcode to be installed which is not >>>>>>>> possible on this platform >>>>>>>> [18:39:20]: ▸ # ./fastlane/spec/env_spec.rb:47 >>>>>>>> ... >>>>>>>> >>>>>>>> >>>>>>>> Awesome! >>>>>>>> >>>>>>>> >>>>>>>> The 21 failures are new though. >>>>>>>> >>>>>>>> I have a _spec.rb file with 21 examples that has a *before(:all)* >>>>>>>> that seems to have been filtered with my old solution, but with *skip >>>>>>>> *is now executed and fails: >>>>>>>> >>>>>>>> describe Scan do >>>>>>>> describe Scan::XCPrettyReporterOptionsGenerator do >>>>>>>> before(:all) do >>>>>>>> .. code that fails when executed on non-macOS ... >>>>>>>> end >>>>>>>> >>>>>>>> describe "xcpretty reporter options generation" do >>>>>>>> it "generates options for the junit tempfile report required by >>>>>>>> scan", requires_xcodebuild: true do >>>>>>>> ... >>>>>>>> >>>>>>>> Any idea what I can do about this? >>>>>>>> -J >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Am Dienstag, 19. Dezember 2017 19:26:17 UTC+1 schrieb Jan P.: >>>>>>>> >>>>>>>> Thanks for the quick answer. >>>>>>>> >>>>>>>> I missed "skipping examples" because I was so happy to have found >>>>>>>> exclusion filters. Sounds like pretty much what I am looking for - even >>>>>>>> better with the explicit reason I can set for skipping. Will try and >>>>>>>> report >>>>>>>> back. >>>>>>>> >>>>>>>> Best, >>>>>>>> Jan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Am Dienstag, 19. Dezember 2017 17:37:56 UTC+1 schrieb Myron Marston: >>>>>>>> >>>>>>>> RSpec does not provide a way to get the number of examples that >>>>>>>> were excluded by its inclusion or exclusion filters, but there’s a >>>>>>>> different mechanism that will do what you want. Instead of filtering >>>>>>>> the >>>>>>>> examples (which excludes them from consideration entirely), you can >>>>>>>> skip >>>>>>>> them, which prevents the body of the example from running, sets the >>>>>>>> example’s status to :pending, will print the example in yellow in >>>>>>>> the formatter output, and will count the example in the summary total >>>>>>>> printed at the end (e.g. “500 examples, 0 failures, 20 pending”). >>>>>>>> Normally, >>>>>>>> :skip metadata will cause an example to be skipped >>>>>>>> <https://www.google.com/url?q=https%3A%2F%2Frelishapp.com%2Frspec%2Frspec-core%2Fv%2F3-7%2Fdocs%2Fpending-and-skipped-examples%2Fskip-examples%23skipping-using-metadata&sa=D&sntz=1&usg=AFQjCNH2bA7au32CPocvg1H0M1Vzmcf5IQ>, >>>>>>>> but you’ve overwritten it to cause :skip to cause examples to be >>>>>>>> filtered out. >>>>>>>> >>>>>>>> Here’s my suggestion for how to wire this up. >>>>>>>> >>>>>>>> First, tag any examples that depend upon xcode with :uses_xcode >>>>>>>> (rather than :skip), e.g.: >>>>>>>> >>>>>>>> it "uses a feature of xcode", :xcode do >>>>>>>> # ...end >>>>>>>> >>>>>>>> it "does not use xcode at all" do >>>>>>>> # ...end >>>>>>>> >>>>>>>> Then use define_derived_metadata to automatically tag these >>>>>>>> examples with :skip if you are not running on OS X: >>>>>>>> >>>>>>>> # spec_helper.rbrequire 'rbconfig' >>>>>>>> RSpec.configure do |config| >>>>>>>> unless RbConfig::CONFIG['host_os'] =~ /darwin/ >>>>>>>> config.define_derived_metadata(:xcode) do |meta| >>>>>>>> meta[:skip] = "Can only be run on OS X" >>>>>>>> end >>>>>>>> endend >>>>>>>> >>>>>>>> The “Can only be run on OS X” bit will be printed in the output as >>>>>>>> the reason the examples are pending. >>>>>>>> >>>>>>>> HTH, >>>>>>>> Myron >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Dec 19, 2017 at 3:06 AM, Jan P. <[email protected]> wrote: >>>>>>>> >>>>>>>> RSpec has this nice method to exclude individual tests/examples or >>>>>>>> whole groups by using filter_run_excluding in the config, then >>>>>>>> tagging the examples: >>>>>>>> >>>>>>>> https://relishapp.com/rspec/rspec-core/v/3-7/docs/filtering/ >>>>>>>> exclusion-filters >>>>>>>> >>>>>>>> RSpec.configure do |c| >>>>>>>> c.filter_run_excluding :skip => trueend >>>>>>>> RSpec.describe "something" do >>>>>>>> it "does one thing" do >>>>>>>> end >>>>>>>> >>>>>>>> it "does another thing", :skip => true do >>>>>>>> endend >>>>>>>> >>>>>>>> "does one thing" will be checked, >>>>>>>> "does another thing" will not. >>>>>>>> >>>>>>>> >>>>>>>> We are using this, for example, to skip some tests depending on the >>>>>>>> platform the test is run on by wrapping the c.filter_run_excluding >>>>>>>> :skip => true in an if block: >>>>>>>> >>>>>>>> If Mac, >>>>>>>> no exclusions, if Ubuntu, >>>>>>>> exclude tests that do something with Xcode. >>>>>>>> >>>>>>>> >>>>>>>> Right now the numbers of passing examples/test is just lower if the >>>>>>>> exclusion filter is used, but it would be nice to see the actual >>>>>>>> number of >>>>>>>> tests that are skipped. >>>>>>>> >>>>>>>> Is there a way to get the number of tests skipped by this method >>>>>>>> during a test run? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Jan >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "rspec" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/rspec/3108ef8e-303d-425b- >>>>>>>> 9b00-ab83dfec7633%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/rspec/3108ef8e-303d-425b-9b00-ab83dfec7633%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "rspec" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/rspec/bcfd5676-92bd-4051- >>>>>>>> a3dd-5cb942784698%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/rspec/bcfd5676-92bd-4051-a3dd-5cb942784698%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "rspec" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/rspec/715c3e8b-c8e0-4c98- >>>>>>> a0ee-9ebf1d9c4ae4%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/rspec/715c3e8b-c8e0-4c98-a0ee-9ebf1d9c4ae4%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "rspec" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rspec/bb687ad6-5489-4342- >>>>> b1c8-c1656b9bf2f3%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/rspec/bb687ad6-5489-4342-b1c8-c1656b9bf2f3%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "rspec" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/rspec/5896f17f-84f3-48da-86f1-b99924e2a5ba%40googlegroups.com >>> <https://groups.google.com/d/msgid/rspec/5896f17f-84f3-48da-86f1-b99924e2a5ba%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "rspec" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/rspec/f001d009-28cb-4be2-9c41-2f8fa29893a1%40googlegroups.com > <https://groups.google.com/d/msgid/rspec/f001d009-28cb-4be2-9c41-2f8fa29893a1%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "rspec" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/CADUxQmsPUoUcweLU_SabpkaZTNFzdt8gts8prq9Q8e5n%2Bt9HrQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
