> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution > <swift-evolution@swift.org> wrote: > > Hello Swift Community, > > Harlan Haskins and I have been working on libraries > <https://github.com/trill-lang> to make interacting with LLVM and Clang’s > APIs more elegant with native Swift interfaces. While writing up the > packages we realized the package manager wouldn’t allow us to specify testing > targets and test-only dependencies. To rectify that, I have attached a draft > proposal for adding test-only targets and dependency fields to the Swift > Package manager. This proposal can also be read in gist form > <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38>. > > Cheers, > > ~Robert Widmann >
Thanks for driving this! It is a very desirable feature which needs proposal work. Comments inline. > Test-Only Package Dependencies and Targets > > Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md> > Authors: Harlan Haskins <https://github.com/harlanhaskins>, Robert Widmann > <https://github.com/codafi> > Review Manager: TBD > Status: Awaiting review > > <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#introduction>Introduction > > This proposal reinstates Swift package manager’s ability to fetch > dependencies and build targets scoped exclusively to the testing module(s) of > a given package. > > Swift-evolution thread: Discussion thread topic for that proposal > <https://lists.swift.org/pipermail/swift-evolution/> > > <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#motivation>Motivation > > Soon after SE-0019 > <https://github.com/apple/swift-evolution/blob/master/proposals/0019-package-manager-testing.md#test-only-dependencies> > identified the need for richer test-only dependencies and targets, a > decision was made to remove the package manager’s fledgling ability to treat > certain dependencies as test-only. This has led to a myriad of > clever-but-needlessly-complex workarounds ([1] > <https://github.com/ReactiveCocoa/ReactiveSwift/blob/master/.travis.yml#L85>, > [2] <https://github.com/ReactiveX/RxSwift/blob/master/Package.swift#L3>, [3] > <https://github.com/Quick/Quick/blob/master/.Package.test.swift>) on the part > of 3rd parties to recover the feature themselves. In addition, the Swift > community has come up with a number of their own frameworks to augment > functionality in XCTest but depending on these external testing frameworks is > brittle and difficult to get right. > > > <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#proposed-solution>Proposed > solution > > We propose the re-introduction of the testDependencies parameter in Package > Manifests to support external test-only dependencies. To support local > test-only targets we also propose the introduction of the testTargets > parameter and an extension of the existing swift test command to support > individual invocation of these targets. > > > <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#detailed-design>Detailed > design > > The behavior of the new testDependencies parameter mirrors that of the > existing dependencies parameter with one important difference: fetched > dependencies are only built to support package-defined test targets as part > of an invocation of swift test. > > import PackageDescription > > let package = Package( > name: "Foo", > targets: [ > Target(name: "Foo") > ], > dependencies: [ > .Package(url: "https://github.com/org/ana.git > <https://github.com/org/ana.git>", versions: Version(1,0,0)...Version(1,9,9)), > ], > testDependencies: [ > .Package(url: "https://github.com/org/anism.git > <https://github.com/org/anism.git>", versions: > Version(1,0,0)...Version(1,9,9)), > ] > ) I think this feature should be called local dependencies (or maybe dev dependencies) because it can be used for tests as well as regular targets. As an example say you have a networking library package and you want to create an example CLI target which uses a JSON mapper package. You wouldn't want to vend the CLI tool when you act as a dependency to other packages, or include the JSON mapper in your dependencies. Test dependency doesn't sound right in that context. After the product proposal <https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md> is implemented, you will be able to control what you vend to your clients, and we thought about adding ability to define dependencies in-line with target dependencies but left it out of the proposal to keep it simpler. Maybe this proposal can add that instead of a separate `testDependencies` property. Consider this manifest: let package = Package( name: "FooNetworking", targets: [ Target( name: "FooNetworking"), Target( name: "FooNetworkingExample", dependencies: [ "FooNetworking", .package(url: "/path/to/BarJSON.git", version: "1.0.0", product: "BarJSON"), // Note: This doesn't actually exists right now. ]), Target( name: "FooNetworkingTests", dependencies: [ "FooNetworking", .package(url: "/path/to/Quick.git", version: "1.0.0", product: "Quick"), // Note: This doesn't actually exists right now. ]), ], products: [ .Library(name: "FooNetworking", targets: ["FooNetworking"]), ] ) This manifest defines three targets and one product. FooNetworking: The base target and the actual library. FooNetworkingExample: The example cli tool. It depends on FooNetworking target and BarJSON product from an external package. FooNetworkingTests: The test target depends on an external package Quick. Both BarJSON and Quick are local dependencies to this package. If FooNetworkingExample was also vended as a product, BarJSON would automatically become a regular external dependency. > Similarly, the behavior of the testTargets field mirrors that of the existing > targets field but defines a set of targets that are only built during an > invocation of swift test. Importantly, a target defined in testTargets may > reference a target defined in targets but not vice-versa. Should that > behavior be needed, the test target should be promoted to a “full” target. > > import PackageDescription > > let package = Package( > name: "SwiftPM", > targets: [ > Target( > name: "PackageDescription", > dependencies: []), > > // MARK: Support libraries > > Target( > /** Cross-platform access to bare `libc` functionality. */ > name: "libc", > dependencies: []), > Target( > /** “Swifty” POSIX functions from libc */ > name: "POSIX", > dependencies: ["libc"]), > Target( > /** Basic support library */ > name: "Basic", > dependencies: ["libc", "POSIX"]), > > /* Omitted for Brevity */ > ], > testTargets: [ > Target( > name: "BasicPerformanceTests", > dependencies: ["Basic"]), > /* Omitted for Brevity */ > ] > ) > Finally, with well-defined test targets in hand, we propose swift test be > amended to support individual test execution. > > > We propose the following syntax to execute all tests of all known test > targets. > > $ swift test > To run a set of specific test cases, reference the module-defining test > target and the specific name of a subclass of XCTestCase: > > $ swift test TestModule.TestCase > To run an individual test case, reference the module-defining test target, > the name of the test case subclass, and the name of the test: > > $ swift test TestModule.TestCase.exampleTest We already have all these features. A target is a test target if it has a suffix Tests and is placed under Tests/ directory — we might loosen these restrictions with custom target conventions feature. This proposal <https://github.com/apple/swift-evolution/blob/master/proposals/0129-package-manager-test-naming-conventions.md> could clarify further. Here is a summary of what `swift test` currently supports: * Test targets are only built (and run) when you run `swift test`. * `swift test --skip-build` skips building of test targets and runs whatever was last built. * `swift test -s <test-module>.<test-case>` will run a test case class. * `swift test -s <test-module>.<test-case>/<test>` will run an individual test. * `swift test --list-tests` will list all tests in above format. * `swift test --parallel` will run tests in parallel. You can view these options and their help text using `swift test --help`.
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution
