This new proposal is great, I'm all in.
On Wed, Apr 26, 2017, at 17:25, Rick Ballard via swift-evolution wrote:> Hi all, > > We have a proposal we'd like feedback on to revise how Swift Package > Manager dependency resolution, updating, and pinning works. These > changes weren't planned in the roadmap we published a few months ago, > but it's become clear since then that we have some holes in our > dependency resolution behavior that need fixing. We've also come to > the conclusion that our original design for pinning was > overcomplicated and should be revised.> > Please give us your feedback; we're hoping to submit this proposal for > official review next week.> > The current draft of the proposal can be found at > https://github.com/rballard/swift-evolution/commit/e5e7ce76f29c855aa7162ed3733b09e701d662d6. > I'm also including it below.> > Thanks, > > - Rick > > Package Manager Revised Dependency Resolution > * Proposal: SE-NNNN[1] > * Author: Rick Ballard[2] > * Review Manager: TBD > * Status: Draft in progress > * Bug: TBD> Introduction > This proposal makes the package manager's dependency resolution > behavior clearer and more intuitive. It removes the pinning commands > (swift package pin & swift package unpin), replaces the swift package > fetch command with a new swift package resolve command with improved > behavior, and replaces the optional Package.pins file with a > Package.resolved file which is always created during dependency > resolution.> Motivation > When SE-0145 Package Manager Version Pinning[3] was proposed, it was > observed that the proposal was overly complex. In particular, it > introduced a configuration option allowing some packages to have > autopinning on (the default), while others turned it off; this option > affected the behavior of other commands (like swift package update, > which has a --repinflag that does nothing for packages that use > autopinning). This configuration option has proved to be unnecessarily > confusing.> In the existing design, when autopinning is on (which is true by > default) the swift package pin command can't be used to pin packages > at specific revisions while allowing other packages to be updated. In > particular, if you edit your package's version requirements in the > Package.swift manifest, there is no way to resolve your package graph > to conform to those new requirements without automatically repinning > all packages to the latest allowable versions. Thus, specific, > intentional pins can not be preserved without turning off autopinning.> The > problems here stem from trying to use one mechanism (pinning) to > solve two different use cases: wanting to record and share resolved > dependency versions, vs wanting to keep a badly-behaved package at a > specific version. We think the package manager could be simplified by > splitting these two use cases out into different mechanisms ("resolved > versions" vs "pinning"), instead of using an "autopinning" option > which makes these two features mutually-exclusive and confusing.> > Additionally, some dependency resolution behaviors were not well- > specified and do not behave well. The package manager is lax about > detecting changes to the versions specified in the Package.swift > manifest or Package.pinspinfile, and fails to automatically update > packages when needed, or to issue errors if the version requirements > are unsatisfiable, until the user explicitly runs swift package > update, or until a new user without an existing checkout attempts to > build. We'd like to clarify and revise the rules around when and how > the package manager performs dependency resolution.> Proposed solution > The pinning feature will be removed. This removes the swift package > pin and swift package unpin commands, the --repin flag to swift > package update, and use of the Package.pins file.> In a future version of the > package manager we may re-introduce > pinning. If we do, pins will only be recorded in the Package.pins file > when explicitly set with swift package pin, and any pinned > dependencies will *not* be updated by the swift package update > command; instead, they would need to be unpinned to be updated. This > would be a purely additive feature which packages could use in > addition to the resolved versions feature when desired.> A new "resolved > versions" feature will be added, which behaves very > similarly to how pinning previously behaved when autopinning was on. > The version of every resolved dependency will be recorded in a > Package.resolved file in the top-level package, and when this file is > present in the top-level package it will be used when performing > dependency resolution, rather than the package manager finding the > latest eligible version of each package. swift package updatewill > update all dependencies to the latest eligible versions and update the > Package.resolved file accordingly.> Resolved versions will always be recorded > by the package manager. Some > users may chose to add the Package.resolved file to their package's > .gitignore file. When this file is checked in, it allows a team to > coordinate on what versions of the dependencies they should use. If > this file is gitignored, each user will seperately choose when to get > new versions based on when they run the swift package update command, > and new users will start with the latest eligible version of each > dependency. Either way, for a package which is a dependency of other > packages (e.g. a library package), that package's Package.resolved > file will not have any effect on its client packages.> The existing swift > package fetch command will be deprecated, removed > from the help message, and removed completely in a future release of > the Package Manager. In its place, a new swift package resolve command > will be added. The behavior of resolve will be to resolve > dependencies, taking into account the current version restrictions in > the Package.swift manifest and Package.resolved resolved versions > file, and issuing an error if the graph cannot be resolved. For > packages which have previously resolved versions recorded in the > Package.resolved file, the resolvecommand will resolve to those > versions as long as they are still eligible. If the resolved versions > file changes (e.g. because a teammate pushed a new version of the > file) the next resolve command will update packages to match that > file. After a successful resolve command, the checked out versions of > all dependencies and the versions recorded in the resolved versions > file will match. In most cases the resolve command will perform no > changes unless the Package.swiftmanifest or Package.resolved file have > changed.> The following commands will implicitly invoke the swift package > resolve functionality before running, and will cancel with an error if > dependencies cannot be resolved: > * swift build > * swift test > * swift package generate-xcodeproj> The swift package show-dependencies > command will also implicitly > invoke swift package resolve, but it will show whatever information > about the dependency graph is available even if the resolve fails.> The swift > package edit command will implicitly invoke swift package > resolve, but if the resolve fails yet did identify and fetch a package > with the package name the command supplied, the command will allow > that package to be edited anyway. This is useful if you wish to use > the edit command to edit version requirements and fix an unresolvable > dependency graph. swift package unedit will unedit the package and > *then* perform a resolve.> Detailed design > The resolve command is allowed to automatically add new dependencies > to the resolved versions file, and to remove dependencies which are no > longer in the dependency graph. It can also automatically update the > recorded versions of any package whose previously-resolved version is > no longer allowed by the version requirements from the > Package.swiftmanifests. When changed version requirements force a > dependency to be automatically re-resolved, the latest eligible > version will be chosen; any other dependencies affected by that change > will prefer to remain at their previously-resolved versions as long as > those versions are eligible, and will otherwise update likewise.> The > Package.resolved resolved versions file will record the git > revision used for each resolved dependency in addition to its version. > In future versions of the package manager we may use this information > to detect when a previously-resolved version of a package resolves to > a new revision, and warn the user if this happens.> The swift package resolve > command will not actually perform a git > fetch on any dependencies unless it needs to in order to correctly > resolve dependencies. As such, if all dependencies are already > resolved correctly and allowed by the version constraints in the > Package.swift manifest and Package.resolved resolved versions file, > the resolvecommand will not need to do anything (e.g. a normal swift > build won't hit the network or make unnecessary changes during its > implicit resolve).> If a dependency is in edit mode, it is allowed to have a > different > version checked out than that recorded in the resolved versions file. > The version recorded for an edited package will not change > automatically. If a swift package updateoperation is performed while > any packages are in edit mode, the versions of those edited packages > will be removed from the resolved versions file, so that when those > packages leave edit mode the next resolution will record a new version > for them. Any packages in the dependency tree underneath an edited > package will also have their resolved version removed by swift package > update, as otherwise the resolved versions file might record versions > that wouldn't have been chosen without whatever edited package > modifications have been made.> Alternatives considered > We considered repurposing the existing fetch command for this new > behavior, instead of renaming the command to resolve. However, the > name fetch is defined by git to mean getting the latest content for a > repository over the network. Since this package manager command does > not always actually fetch new content from the network, it is > confusing to use the name fetch. In the future, we may offer > additional control over when dependency resolution is allowed to > perform network access, and we will likely use the word fetch in flag > names that control that behavior.> We considered continuing to write out the > Package.pins file for > packages whose Swift tools version[4] was less than 4.0, for maximal > compatibility with the Swift 3.1 tools. However, as the old pinning > behavior was a workflow feature and not a fundamental piece of > package compatibility, we do not consider it necessary to support in > the 4.0 tools.> We considered keeping the pin and unpin commands, with the new > behavior as discussed briefly in this proposal. While we think we may > wish to bring this feature back in the future, we do not consider it > critical for this release; the workflow it supports (updating all > packages except a handful which have been pinned) is not something > most users will need, and there are workarounds (e.g. specify an > explicit dependency in the Package.swift manifest).> Why we didn't use > "Package.lock" > We considered using the .lock file extension for the new resolved > versions file, to be consistent with many other package managers. We > expect that the decision not to use this extension will be > controversial, as following established precedent is valuable. > However, we think that a "lockfile" is a very poor name for this > concept, and that using that name would cause confusion when we re- > introduce pins. Specifically: > * Calling this a "lock" implies a stronger lockdown of dependencies > than is supported by the actual behavior. As a simple update > command will reset the locks, and a change to the specified > versions in Package.swift will override them, they're not really > "locked" at all. This is misleading. > * When we re-introduce pinning, it would be very confusing to have > both "locks" and "pins". Having "resolved versions" and "pins" is > not so confusing. > * The term "lock" is already overloaded between POSIX file locks and > locks in concurrent programming.> For comparison, here is a list of other > package managers which > implement similar behavior and their name for this file: > Package Manager Language Resolved versions file name Yarn JS yarn.lock > Composer PHP composer.lock Cargo Rust Cargo.lock Bundler Ruby > Gemfile.lock CocoaPods ObjC/Swift Podfile.lock Glide Go glide.lock Pub > Dart pubspec.lock Mix Elixir mix.lock rebar3 Erlang rebar.lock Carton > Perl carton.lock Carthage ObjC/Swift Cartfile.resolved Pip Python > requirements.txt NPM JS npm-shrinkwrap.json Meteor JS versions> Some > arguments for using ".lock" instead of ".resolved" are: > * Users of other package managers will already be familiar with the > terminology and behavior. > * For packages which support multiple package managers, it will be > possible to put "*.lock" into the gitignore file instead of needing > a seperate entry for "*.resolved".> However, we do not feel that these > arguments outweigh the problems > with the term "lock". If providing feedback asking that we reconsider > this decision, please be clear about why the above decision is > incorrect, with new information not already considered.> > _________________________________________________ > swift-evolution mailing list > swift-evolution@swift.org > https://lists.swift.org/mailman/listinfo/swift-evolution Links: 1. https://github.com/rballard/swift-evolution/blob/e5e7ce76f29c855aa7162ed3733b09e701d662d6/proposals/NNNN-package-manager-revised-dependency-resolution.md 2. https://github.com/rballard 3. https://github.com/apple/swift-evolution/blob/master/proposals/0145-package-manager-version-pinning.md 4. https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution