Thanks Myron, that worked!
> 4704 examples, 0 failures, 175 pending
With code:
before(:all) do
skip "Requires `xcodebuild` to be installed which is not possible on this
platform" unless FastlaneCore::Helper.is_mac?
As you can see right now I duplicate the reason text, and use the condition
that also triggers `skip`/metadata block:
https://github.com/fastlane/fastlane/blob/c6b1ac4621941b2efef702b923a572357c64eea9/spec_helper.rb#L68-L84
Is there somehow a better way to "connect" the skip in `before` to the
filters/metadata defined here?
It would be nicer if I could e.g. call a function with the "metadata"
("xcodebuild") as parameter to trigger the skip or something. Any idea?
Best,
Jan
Am Mittwoch, 20. Dezember 2017 02:11:56 UTC+1 schrieb Myron Marston:
>
> Unfortunately, there’s no way to use :skip metadata at the group level to
> skip before(:all) hooks. That’s because of how metadata in RSpec is
> modeled, and how RSpec implements :skip. :skip metadata is handled at the
> level of individual examples, and metadata is inherited from a group to its
> enclosing examples. So, for example, this works:
>
> RSpec.describe MyClass, :skip do
> it("is skipped"){ }
> it("is also skipped") { }end
>
> Both examples will be skipped here.
>
> But consider that you can also do this:
>
> RSpec.describe MyClass, :skip do
> it("is skipped"){ }
> it("is also skipped") { }
> it("is not skipped", skip: false) { }end
>
> Here we have on example that is overriding the skip: true metadata
> inherited from the group. As a result, a before(:all) hook can’t simply
> look at the group metadata to decide whether or not to skip or not. In
> general, before(:all) hooks have lots of gotchas like this because they
> don’t really fit well into the per-example semantics of so many parts of
> RSpec. If you can refactor your tests to no longer need such a hook
> (possibly using a aggregate_failures
> <https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures>),
>
> that is worth considering. Besides that, the other option you can do is to
> manually call the skip method from your before(:all) hook:
>
> before(:all) do
> skip "reason to skip" if should_skip?
> # rest of your hook logicend
>
> HTH,
> Myron
>
>
>
>
> On Tue, Dec 19, 2017 at 3:47 PM, Jan P. <[email protected] <javascript:>>
> wrote:
>
>> Jon, I tried
>>
>> describe "xcpretty reporter options generation", requires_xcodebuild: true do
>>
>>
>> and
>>
>> before(:all), requires_xcodebuild: true do
>>
>>
>> and
>>
>>
>> describe Scan::XCPrettyReporterOptionsGenerator, requires_xcodebuild: true
>> do
>>
>>
>> but all didn't work and the tests were logged as failure because of the
>> `before` being executed.
>>
>> Am I doing this wrong somehow?
>>
>>
>> Code (last iteration) is here:
>>
>> https://github.com/fastlane/fastlane/blob/janpio-mark_skipped_tests_as_pending_with_reason/scan/spec/xcpretty_reporter_options_generator_spec.rb
>>
>> https://github.com/fastlane/fastlane/blob/janpio-mark_skipped_tests_as_pending_with_reason/spec_helper.rb#L68-L83
>> Matching Circle CI run:
>> https://circleci.com/gh/fastlane/fastlane/12586
>>
>> -Jan
>>
>>
>>
>> Am Dienstag, 19. Dezember 2017 23:05:53 UTC+1 schrieb Jon Rowe:
>>>
>>> You should be able to specify `requires_xcodebuild: true` on the
>>> describe as we’ll to prevent the before all being triggered
>>>
>>> Jon Rowe
>>> ---------------------------
>>> [email protected]
>>> jonrowe.co.uk
>>>
>>> On Wednesday, 20 December 2017 at 05:56, Jan P. wrote:
>>>
>>> Excuse the terrible code formatting, seems Google Groups wanted to be
>>> helpful.
>>>
>>> Another try:
>>>
>>> describe Scan do
>>> describe Scan::XCPrettyReporterOptionsGenerator do
>>> before(:all) do
>>> // code that fails when executed on non-macOS
>>> end
>>>
>>> describe "xcpretty reporter options generation" do
>>> it "generates options for the junit tempfile report required by
>>> scan", requires_xcodebuild: true do
>>> ...
>>>
>>> -J
>>>
>>>
>>> Am Dienstag, 19. Dezember 2017 19:55:45 UTC+1 schrieb Jan P.:
>>>
>>> Yes, that works mostly like expected:
>>>
>>>
>>> 4704 examples, 21 failures, 154 pending
>>>
>>>
>>> and
>>>
>>> ...
>>> [18:39:20]: ▸ Pending: (Failures listed here are expected and do not
>>> affect your suite's status)
>>> [18:39:20]: ▸ 1) Fastlane Fastlane::EnvironmentPrinter contains main
>>> information about the stack
>>> [18:39:20]: ▸ # Requires Xcode to be installed which is not possible on
>>> this platform
>>> [18:39:20]: ▸ # ./fastlane/spec/env_spec.rb:28
>>> [18:39:20]: ▸ 2) Fastlane Fastlane::EnvironmentPrinter
>>> FastlaneCore::Helper.xcode_version cannot be obtained contains stack
>>> information other than Xcode Version
>>> [18:39:20]: ▸ # Requires Xcode to be installed which is not possible on
>>> this platform
>>> [18:39:20]: ▸ # ./fastlane/spec/env_spec.rb:47
>>> ...
>>>
>>>
>>> Awesome!
>>>
>>>
>>> The 21 failures are new though.
>>>
>>> I have a _spec.rb file with 21 examples that has a *before(:all)* that
>>> seems to have been filtered with my old solution, but with *skip *is
>>> now executed and fails:
>>>
>>> describe Scan do
>>> describe Scan::XCPrettyReporterOptionsGenerator do
>>> before(:all) do
>>> .. code that fails when executed on non-macOS ...
>>> end
>>>
>>> describe "xcpretty reporter options generation" do
>>> it "generates options for the junit tempfile report required by scan",
>>> requires_xcodebuild: true do
>>> ...
>>>
>>> Any idea what I can do about this?
>>> -J
>>>
>>>
>>>
>>> Am Dienstag, 19. Dezember 2017 19:26:17 UTC+1 schrieb Jan P.:
>>>
>>> Thanks for the quick answer.
>>>
>>> I missed "skipping examples" because I was so happy to have found
>>> exclusion filters. Sounds like pretty much what I am looking for - even
>>> better with the explicit reason I can set for skipping. Will try and report
>>> back.
>>>
>>> Best,
>>> Jan
>>>
>>>
>>>
>>> Am Dienstag, 19. Dezember 2017 17:37:56 UTC+1 schrieb Myron Marston:
>>>
>>> RSpec does not provide a way to get the number of examples that were
>>> excluded by its inclusion or exclusion filters, but there’s a different
>>> mechanism that will do what you want. Instead of filtering the examples
>>> (which excludes them from consideration entirely), you can skip them, which
>>> prevents the body of the example from running, sets the example’s status to
>>> :pending, will print the example in yellow in the formatter output, and
>>> will count the example in the summary total printed at the end (e.g. “500
>>> examples, 0 failures, 20 pending”). Normally, :skip metadata will cause
>>> an example to be skipped
>>> <https://www.google.com/url?q=https%3A%2F%2Frelishapp.com%2Frspec%2Frspec-core%2Fv%2F3-7%2Fdocs%2Fpending-and-skipped-examples%2Fskip-examples%23skipping-using-metadata&sa=D&sntz=1&usg=AFQjCNH2bA7au32CPocvg1H0M1Vzmcf5IQ>,
>>>
>>> but you’ve overwritten it to cause :skip to cause examples to be
>>> filtered out.
>>>
>>> Here’s my suggestion for how to wire this up.
>>>
>>> First, tag any examples that depend upon xcode with :uses_xcode (rather
>>> than :skip), e.g.:
>>>
>>> it "uses a feature of xcode", :xcode do
>>> # ...end
>>>
>>> it "does not use xcode at all" do
>>> # ...end
>>>
>>> Then use define_derived_metadata to automatically tag these examples
>>> with :skip if you are not running on OS X:
>>>
>>> # spec_helper.rbrequire 'rbconfig'
>>> RSpec.configure do |config|
>>> unless RbConfig::CONFIG['host_os'] =~ /darwin/
>>> config.define_derived_metadata(:xcode) do |meta|
>>> meta[:skip] = "Can only be run on OS X"
>>> end
>>> endend
>>>
>>> The “Can only be run on OS X” bit will be printed in the output as the
>>> reason the examples are pending.
>>>
>>> HTH,
>>> Myron
>>>
>>>
>>>
>>> On Tue, Dec 19, 2017 at 3:06 AM, Jan P. <[email protected]> wrote:
>>>
>>> RSpec has this nice method to exclude individual tests/examples or whole
>>> groups by using filter_run_excluding in the config, then tagging the
>>> examples:
>>>
>>>
>>> https://relishapp.com/rspec/rspec-core/v/3-7/docs/filtering/exclusion-filters
>>>
>>> RSpec.configure do |c|
>>> c.filter_run_excluding :skip => trueend
>>> RSpec.describe "something" do
>>> it "does one thing" do
>>> end
>>>
>>> it "does another thing", :skip => true do
>>> endend
>>>
>>> "does one thing" will be checked,
>>> "does another thing" will not.
>>>
>>>
>>> We are using this, for example, to skip some tests depending on the
>>> platform the test is run on by wrapping the c.filter_run_excluding
>>> :skip => true in an if block:
>>>
>>> If Mac,
>>> no exclusions, if Ubuntu,
>>> exclude tests that do something with Xcode.
>>>
>>>
>>> Right now the numbers of passing examples/test is just lower if the
>>> exclusion filter is used, but it would be nice to see the actual number of
>>> tests that are skipped.
>>>
>>> Is there a way to get the number of tests skipped by this method during
>>> a test run?
>>>
>>> Thanks,
>>>
>>> Jan
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "rspec" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/rspec/3108ef8e-303d-425b-9b00-ab83dfec7633%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/rspec/3108ef8e-303d-425b-9b00-ab83dfec7633%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "rspec" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/rspec/bcfd5676-92bd-4051-a3dd-5cb942784698%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/rspec/bcfd5676-92bd-4051-a3dd-5cb942784698%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "rspec" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected] <javascript:>
>> .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/rspec/715c3e8b-c8e0-4c98-a0ee-9ebf1d9c4ae4%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/rspec/715c3e8b-c8e0-4c98-a0ee-9ebf1d9c4ae4%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/rspec/bb687ad6-5489-4342-b1c8-c1656b9bf2f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
