Re: [swift-users] SwiftPM manual dependency management

[Geordie Jay via swift-users]

Hi Ankit, thanks for your explanations.

Ankit Aggarwal  schrieb am Sa. 22. Juli 2017 um
13:44:

> On 22-Jul-2017, at 3:37 PM, Geordie Jay  wrote:
>
>
> Geordie J  schrieb am Fr. 21. Juli 2017 um 14:39:
>
>> Hi Ankit, thanks for your reply.
>>
>> Am 21.07.2017 um 07:33 schrieb Ankit Aggarwal via swift-users <
>> swift-users@swift.org>:
>>
>>
>>
>> On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users <
>> swift-users@swift.org> wrote:
>>
>>> Hi all,
>>>
>>> My team and I are trying to use SwiftPM to develop a relatively complex
>>> app with multiple dependencies, all of which are being developed locally
>>> and in parallel. The reason for this is compatibility with an existing
>>> module/import structure used by our iOS app. Maybe I’m doing something very
>>> wrong but my experience so far (2 months in) is that this is extremely
>>> difficult with SwiftPM.
>>>
>>> What I’d love to be able to do is to just run `git add submodule
>>> http://blah.com/mysubmodule.git` in the Packages subdirectory and
>>> SwiftPM would just let me manage dependencies from there myself.
>>>
>>> I was excited to see that SwiftPM 4 has a "Top of Tree" development
>>> option for this purpose. So far my experience with this has not been good.
>>> Firstly because SwiftPM *still* unnecessarily tries to clone my repos
>>> itself (some of which are huge), and secondly because this creates an
>>> absolute path dependency in `.build/dependencies-state.json`, meaning this
>>> setup isn’t sharable within our dev team.
>>>
>>> Attempting this with "local" git urls adds an almost absurd level of
>>> complexity, having to tag each commit for SwiftPM to build. The fact that
>>> we'd need to make a commit to test whether the project even builds is
>>> insane enough as is, let alone the tagging and trying to tell the base
>>> project to use a newer minor version etc etc.
>>>
>>> Adding multiple subtargets is also not an option because the
>>> dependencies (as dynamic libraries) really are shared between multiple
>>> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
>>>
>>> *tldr;* *Please* let us manage dependencies ourselves. It’d be so easy
>>> if Package.swift had an option along the lines of *.Package.local(named:
>>> "XYZ")* that it then looked for in ./Packages/XYZ. Again, maybe I’m
>>> overlooking something but this seems like an obvious and vital option to
>>> have. It’d also simplify the introductory SwiftPM docs significantly.
>>>
>>> Is anyone else having this issue? Would this change really be as simple
>>> and painless as it sounds? I would be prepared to make a pull request along
>>> these lines.
>>>
>>
>> I think you're not really using the Top of Tree feature. You need to add
>> each dependency using its canonical URL, hosted at some server like github.
>> After adding the dependencies, you can use edit feature to put a dependency
>> in Top of Tree mode. To do so, run:
>>
>> $ swift package edit  --path
>> ../path/to/self/managed/checkout/of/the/package
>>
>>
>> Yes, this is what I tried this week. I’m pretty sure this is not a case
>> of misunderstanding the feature or the docs.
>>
>>
>> The package manager will then stop using the cloned repository and use
>> the checkout present at that path (regardless of the state it is in).
>>
>>
>> Yes, but then I have – per dependency – two checkouts of a potentially
>> huge repository. Why force everyone on the dev team to clone a huge repo
>> twice, only to *never* use one of the clones. Also, SwiftPM breaks when
>> —path points at Packages/PackageName, which is exactly where I’d expect the
>> package to be, not in some arbitrary external path (+ some kind of internal
>> checkout cache that will never be used) as well.
>>
>> I haven’t tried to test this recently because it’s a slow process but I
>> have the impression the deps could be even be cloned more than twice,
>> depending on how cleverly SwiftPM realises that multiple Packages have the
>> same dependency.
>>
>> Also, this makes managing interdependent state of development amongst
>> dependencies more difficult than needed. How do we guarantee that devs are
>> on the same commit when using top of tree development? Tagging and managing
>> version numbers etc for day-to-day development is emphatically not an
>> option for us. Since SwiftPM packages only work from a git context anyway,
>> why not allow use of git’s established pattern of dealing with this, namely
>> submodules?
>>
>> Sharing this setup is not automatic, but simple. Each user just needs to
>> run the above command once per dependency.
>>
>>
>> We have about 10 dependencies, *all *of which will* always* be in this
>> state. This seems like a lot of overhead and room for user error, plus it’s
>> a huge workaround for something that could be very simple.
>>
>> Also, you only need to do this if you're actively working on a dependency.
>>
>>
>> The point is that we will *always* be 

Re: [swift-users] SwiftPM manual dependency management

[Ankit Aggarwal via swift-users]

> On 22-Jul-2017, at 3:37 PM, Geordie Jay  wrote:
> 
> 
> Geordie J > schrieb am Fr. 21. 
> Juli 2017 um 14:39:
> Hi Ankit, thanks for your reply.
> 
>> Am 21.07.2017 um 07:33 schrieb Ankit Aggarwal via swift-users 
>> >:
>> 
>> 
>> 
>> On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users 
>> > wrote:
>> Hi all,
>> 
>> My team and I are trying to use SwiftPM to develop a relatively complex app 
>> with multiple dependencies, all of which are being developed locally and in 
>> parallel. The reason for this is compatibility with an existing 
>> module/import structure used by our iOS app. Maybe I’m doing something very 
>> wrong but my experience so far (2 months in) is that this is extremely 
>> difficult with SwiftPM.
>> 
>> What I’d love to be able to do is to just run `git add submodule 
>> http://blah.com/mysubmodule.git`  in the 
>> Packages subdirectory and SwiftPM would just let me manage dependencies from 
>> there myself.
>> 
>> I was excited to see that SwiftPM 4 has a "Top of Tree" development option 
>> for this purpose. So far my experience with this has not been good. Firstly 
>> because SwiftPM still unnecessarily tries to clone my repos itself (some of 
>> which are huge), and secondly because this creates an absolute path 
>> dependency in `.build/dependencies-state.json`, meaning this setup isn’t 
>> sharable within our dev team.
>> 
>> Attempting this with "local" git urls adds an almost absurd level of 
>> complexity, having to tag each commit for SwiftPM to build. The fact that 
>> we'd need to make a commit to test whether the project even builds is insane 
>> enough as is, let alone the tagging and trying to tell the base project to 
>> use a newer minor version etc etc.
>> 
>> Adding multiple subtargets is also not an option because the dependencies 
>> (as dynamic libraries) really are shared between multiple 
>> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
>> 
>> tldr; *Please* let us manage dependencies ourselves. It’d be so easy if 
>> Package.swift had an option along the lines of .Package.local(named: "XYZ") 
>> that it then looked for in ./Packages/XYZ. Again, maybe I’m overlooking 
>> something but this seems like an obvious and vital option to have. It’d also 
>> simplify the introductory SwiftPM docs significantly.
>> 
>> Is anyone else having this issue? Would this change really be as simple and 
>> painless as it sounds? I would be prepared to make a pull request along 
>> these lines.
>> 
>> I think you're not really using the Top of Tree feature. You need to add 
>> each dependency using its canonical URL, hosted at some server like github. 
>> After adding the dependencies, you can use edit feature to put a dependency 
>> in Top of Tree mode. To do so, run:
>> 
>> $ swift package edit  --path 
>> ../path/to/self/managed/checkout/of/the/package
> 
> Yes, this is what I tried this week. I’m pretty sure this is not a case of 
> misunderstanding the feature or the docs.
> 
>> 
>> The package manager will then stop using the cloned repository and use the 
>> checkout present at that path (regardless of the state it is in).
> 
> Yes, but then I have – per dependency – two checkouts of a potentially huge 
> repository. Why force everyone on the dev team to clone a huge repo twice, 
> only to *never* use one of the clones. Also, SwiftPM breaks when —path points 
> at Packages/PackageName, which is exactly where I’d expect the package to be, 
> not in some arbitrary external path (+ some kind of internal checkout cache 
> that will never be used) as well.
> 
> I haven’t tried to test this recently because it’s a slow process but I have 
> the impression the deps could be even be cloned more than twice, depending on 
> how cleverly SwiftPM realises that multiple Packages have the same dependency.
> 
> Also, this makes managing interdependent state of development amongst 
> dependencies more difficult than needed. How do we guarantee that devs are on 
> the same commit when using top of tree development? Tagging and managing 
> version numbers etc for day-to-day development is emphatically not an option 
> for us. Since SwiftPM packages only work from a git context anyway, why not 
> allow use of git’s established pattern of dealing with this, namely 
> submodules?
> 
>> Sharing this setup is not automatic, but simple. Each user just needs to run 
>> the above command once per dependency.
> 
> We have about 10 dependencies, all of which will always be in this state. 
> This seems like a lot of overhead and room for user error, plus it’s a huge 
> workaround for something that could be very simple.
> 
>> Also, you only need to do this if you're actively working on a dependency.
> 
> The point is that we will always be working on the dependencies. 

Re: [swift-users] SwiftPM manual dependency management

[Geordie Jay via swift-users]

Geordie J  schrieb am Fr. 21. Juli 2017 um 14:39:

> Hi Ankit, thanks for your reply.
>
> Am 21.07.2017 um 07:33 schrieb Ankit Aggarwal via swift-users <
> swift-users@swift.org>:
>
>
>
> On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users <
> swift-users@swift.org> wrote:
>
>> Hi all,
>>
>> My team and I are trying to use SwiftPM to develop a relatively complex
>> app with multiple dependencies, all of which are being developed locally
>> and in parallel. The reason for this is compatibility with an existing
>> module/import structure used by our iOS app. Maybe I’m doing something very
>> wrong but my experience so far (2 months in) is that this is extremely
>> difficult with SwiftPM.
>>
>> What I’d love to be able to do is to just run `git add submodule
>> http://blah.com/mysubmodule.git` in the Packages subdirectory and
>> SwiftPM would just let me manage dependencies from there myself.
>>
>> I was excited to see that SwiftPM 4 has a "Top of Tree" development
>> option for this purpose. So far my experience with this has not been good.
>> Firstly because SwiftPM *still* unnecessarily tries to clone my repos
>> itself (some of which are huge), and secondly because this creates an
>> absolute path dependency in `.build/dependencies-state.json`, meaning this
>> setup isn’t sharable within our dev team.
>>
>> Attempting this with "local" git urls adds an almost absurd level of
>> complexity, having to tag each commit for SwiftPM to build. The fact that
>> we'd need to make a commit to test whether the project even builds is
>> insane enough as is, let alone the tagging and trying to tell the base
>> project to use a newer minor version etc etc.
>>
>> Adding multiple subtargets is also not an option because the dependencies
>> (as dynamic libraries) really are shared between multiple
>> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
>>
>> *tldr;* *Please* let us manage dependencies ourselves. It’d be so easy
>> if Package.swift had an option along the lines of *.Package.local(named:
>> "XYZ")* that it then looked for in ./Packages/XYZ. Again, maybe I’m
>> overlooking something but this seems like an obvious and vital option to
>> have. It’d also simplify the introductory SwiftPM docs significantly.
>>
>> Is anyone else having this issue? Would this change really be as simple
>> and painless as it sounds? I would be prepared to make a pull request along
>> these lines.
>>
>
> I think you're not really using the Top of Tree feature. You need to add
> each dependency using its canonical URL, hosted at some server like github.
> After adding the dependencies, you can use edit feature to put a dependency
> in Top of Tree mode. To do so, run:
>
> $ swift package edit  --path
> ../path/to/self/managed/checkout/of/the/package
>
>
> Yes, this is what I tried this week. I’m pretty sure this is not a case of
> misunderstanding the feature or the docs.
>
>
> The package manager will then stop using the cloned repository and use the
> checkout present at that path (regardless of the state it is in).
>
>
> Yes, but then I have – per dependency – two checkouts of a potentially
> huge repository. Why force everyone on the dev team to clone a huge repo
> twice, only to *never* use one of the clones. Also, SwiftPM breaks when
> —path points at Packages/PackageName, which is exactly where I’d expect the
> package to be, not in some arbitrary external path (+ some kind of internal
> checkout cache that will never be used) as well.
>
> I haven’t tried to test this recently because it’s a slow process but I
> have the impression the deps could be even be cloned more than twice,
> depending on how cleverly SwiftPM realises that multiple Packages have the
> same dependency.
>
> Also, this makes managing interdependent state of development amongst
> dependencies more difficult than needed. How do we guarantee that devs are
> on the same commit when using top of tree development? Tagging and managing
> version numbers etc for day-to-day development is emphatically not an
> option for us. Since SwiftPM packages only work from a git context anyway,
> why not allow use of git’s established pattern of dealing with this, namely
> submodules?
>
> Sharing this setup is not automatic, but simple. Each user just needs to
> run the above command once per dependency.
>
>
> We have about 10 dependencies, *all *of which will* always* be in this
> state. This seems like a lot of overhead and room for user error, plus it’s
> a huge workaround for something that could be very simple.
>
> Also, you only need to do this if you're actively working on a dependency.
>
>
> The point is that we will *always* be working on the dependencies. This
> is the core of what we’re doing, not a short aside. This is what makes me
> think we are either doing something wrong, or there is a big feature gap
> (as it appears from here).
>
> The new manifest also supports using branch instead of version range,
> which is 

Re: [swift-users] SwiftPM manual dependency management

[Taylor Swift via swift-users]

I just want to add that I have been immensely frustrated and inconvenienced
by the lack of this, what I believe is, basic and essential feature in the
SwiftPM.

On Fri, Jul 21, 2017 at 9:32 AM, Ankit Aggarwal via swift-users <
swift-users@swift.org> wrote:

> I’m just surprised the idea of a "local dependency" is not seen as a first
> class citizen in SwiftPM, still trying to understand the logic behind that.
> Maybe you can give me an idea of the reasoning behind this?
>
>
> I understand your usecase will benefit a lot from the local dependency
> feature. We did consider adding this feature, I think the main concern was
> package authors ending up publishing package which only works for them
> incase they forget updating their manifest before a release. Also, note
> that SwiftPM is still under heavy development and we can definitely
> re-consider adding this!
>
> Also, I think this is a good usecase of a multipackage repository feature.
> We don't have that yet but you can read some of the discussion here
> 
> .
>
> PS: We also have a slack channel if you want to hang out -
> https://lists.swift.org/pipermail/swift-build-dev/
> Week-of-Mon-20160530/000497.html
>
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users
>
>
___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] SwiftPM manual dependency management

[Geordie J via swift-users]

Hi Ankit,

> Am 21.07.2017 um 15:32 schrieb Ankit Aggarwal via swift-users 
> :
> 
> +swift-build-dev
> 
>> On 21-Jul-2017, at 6:09 PM, Geordie J > > wrote:
>> 
>> Hi Ankit, thanks for your reply.
>> 
>>> Am 21.07.2017 um 07:33 schrieb Ankit Aggarwal via swift-users 
>>> >:
>>> 
>>> 
>>> 
>>> On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users 
>>> > wrote:
>>> Hi all,
>>> 
>>> My team and I are trying to use SwiftPM to develop a relatively complex app 
>>> with multiple dependencies, all of which are being developed locally and in 
>>> parallel. The reason for this is compatibility with an existing 
>>> module/import structure used by our iOS app. Maybe I’m doing something very 
>>> wrong but my experience so far (2 months in) is that this is extremely 
>>> difficult with SwiftPM.
>>> 
>>> What I’d love to be able to do is to just run `git add submodule 
>>> http://blah.com/mysubmodule.git`  in 
>>> the Packages subdirectory and SwiftPM would just let me manage dependencies 
>>> from there myself.
>>> 
>>> I was excited to see that SwiftPM 4 has a "Top of Tree" development option 
>>> for this purpose. So far my experience with this has not been good. Firstly 
>>> because SwiftPM still unnecessarily tries to clone my repos itself (some of 
>>> which are huge), and secondly because this creates an absolute path 
>>> dependency in `.build/dependencies-state.json`, meaning this setup isn’t 
>>> sharable within our dev team.
>>> 
>>> Attempting this with "local" git urls adds an almost absurd level of 
>>> complexity, having to tag each commit for SwiftPM to build. The fact that 
>>> we'd need to make a commit to test whether the project even builds is 
>>> insane enough as is, let alone the tagging and trying to tell the base 
>>> project to use a newer minor version etc etc.
>>> 
>>> Adding multiple subtargets is also not an option because the dependencies 
>>> (as dynamic libraries) really are shared between multiple 
>>> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
>>> 
>>> tldr; *Please* let us manage dependencies ourselves. It’d be so easy if 
>>> Package.swift had an option along the lines of .Package.local(named: "XYZ") 
>>> that it then looked for in ./Packages/XYZ. Again, maybe I’m overlooking 
>>> something but this seems like an obvious and vital option to have. It’d 
>>> also simplify the introductory SwiftPM docs significantly.
>>> 
>>> Is anyone else having this issue? Would this change really be as simple and 
>>> painless as it sounds? I would be prepared to make a pull request along 
>>> these lines.
>>> 
>>> I think you're not really using the Top of Tree feature. You need to add 
>>> each dependency using its canonical URL, hosted at some server like github. 
>>> After adding the dependencies, you can use edit feature to put a dependency 
>>> in Top of Tree mode. To do so, run:
>>> 
>>> $ swift package edit  --path 
>>> ../path/to/self/managed/checkout/of/the/package
>> 
>> Yes, this is what I tried this week. I’m pretty sure this is not a case of 
>> misunderstanding the feature or the docs.
>> 
> 
> Ah, cool!
> 
>>> 
>>> The package manager will then stop using the cloned repository and use the 
>>> checkout present at that path (regardless of the state it is in).
>> 
>> Yes, but then I have – per dependency – two checkouts of a potentially huge 
>> repository. Why force everyone on the dev team to clone a huge repo twice, 
>> only to *never* use one of the clones.
> 
> Sure, you could end up with clones but that is only an implementation detail. 
> SwiftPM can become smart enough and remove a clone that is not being used.
> 
>> Also, SwiftPM breaks when —path points at Packages/PackageName, which is 
>> exactly where I’d expect the package to be, not in some arbitrary external 
>> path (+ some kind of internal checkout cache that will never be used) as 
>> well.
> 
> If --path Packages/PackageName doesn't work, that is a bug! Do you mind 
> filing a JIRA at bugs.swift.org ? 

SwiftPM in this case tries to make a symlink from Packages/PackageName -> 
Packages/PackageName and doesn’t allow continuing from there.

> 
> I understand internal caches could be an inconvenience if repositories are 
> large but we can definitely improve how caching works.

It just seems unnecessary. Imagine (as an extreme example) you pull in the 
apple/swift repo as a dependency, with all its tens of thousands of commits, 
then you have a git submodule also in the repo with the same dependency. I’d 
just expect SwiftPM not to clone the package at all if I tell it I want it 
locally.

> 
>> 
>> I haven’t tried to test this recently because it’s a slow process but I have 
>> the impression the deps could be even be cloned more than twice, depending 

Re: [swift-users] SwiftPM manual dependency management

[Ankit Aggarwal via swift-users]

+swift-build-dev

> On 21-Jul-2017, at 6:09 PM, Geordie J  wrote:
> 
> Hi Ankit, thanks for your reply.
> 
>> Am 21.07.2017 um 07:33 schrieb Ankit Aggarwal via swift-users 
>> >:
>> 
>> 
>> 
>> On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users 
>> > wrote:
>> Hi all,
>> 
>> My team and I are trying to use SwiftPM to develop a relatively complex app 
>> with multiple dependencies, all of which are being developed locally and in 
>> parallel. The reason for this is compatibility with an existing 
>> module/import structure used by our iOS app. Maybe I’m doing something very 
>> wrong but my experience so far (2 months in) is that this is extremely 
>> difficult with SwiftPM.
>> 
>> What I’d love to be able to do is to just run `git add submodule 
>> http://blah.com/mysubmodule.git`  in the 
>> Packages subdirectory and SwiftPM would just let me manage dependencies from 
>> there myself.
>> 
>> I was excited to see that SwiftPM 4 has a "Top of Tree" development option 
>> for this purpose. So far my experience with this has not been good. Firstly 
>> because SwiftPM still unnecessarily tries to clone my repos itself (some of 
>> which are huge), and secondly because this creates an absolute path 
>> dependency in `.build/dependencies-state.json`, meaning this setup isn’t 
>> sharable within our dev team.
>> 
>> Attempting this with "local" git urls adds an almost absurd level of 
>> complexity, having to tag each commit for SwiftPM to build. The fact that 
>> we'd need to make a commit to test whether the project even builds is insane 
>> enough as is, let alone the tagging and trying to tell the base project to 
>> use a newer minor version etc etc.
>> 
>> Adding multiple subtargets is also not an option because the dependencies 
>> (as dynamic libraries) really are shared between multiple 
>> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
>> 
>> tldr; *Please* let us manage dependencies ourselves. It’d be so easy if 
>> Package.swift had an option along the lines of .Package.local(named: "XYZ") 
>> that it then looked for in ./Packages/XYZ. Again, maybe I’m overlooking 
>> something but this seems like an obvious and vital option to have. It’d also 
>> simplify the introductory SwiftPM docs significantly.
>> 
>> Is anyone else having this issue? Would this change really be as simple and 
>> painless as it sounds? I would be prepared to make a pull request along 
>> these lines.
>> 
>> I think you're not really using the Top of Tree feature. You need to add 
>> each dependency using its canonical URL, hosted at some server like github. 
>> After adding the dependencies, you can use edit feature to put a dependency 
>> in Top of Tree mode. To do so, run:
>> 
>> $ swift package edit  --path 
>> ../path/to/self/managed/checkout/of/the/package
> 
> Yes, this is what I tried this week. I’m pretty sure this is not a case of 
> misunderstanding the feature or the docs.
> 

Ah, cool!

>> 
>> The package manager will then stop using the cloned repository and use the 
>> checkout present at that path (regardless of the state it is in).
> 
> Yes, but then I have – per dependency – two checkouts of a potentially huge 
> repository. Why force everyone on the dev team to clone a huge repo twice, 
> only to *never* use one of the clones.

Sure, you could end up with clones but that is only an implementation detail. 
SwiftPM can become smart enough and remove a clone that is not being used.

> Also, SwiftPM breaks when —path points at Packages/PackageName, which is 
> exactly where I’d expect the package to be, not in some arbitrary external 
> path (+ some kind of internal checkout cache that will never be used) as well.

If --path Packages/PackageName doesn't work, that is a bug! Do you mind filing 
a JIRA at bugs.swift.org ? 

I understand internal caches could be an inconvenience if repositories are 
large but we can definitely improve how caching works.

> 
> I haven’t tried to test this recently because it’s a slow process but I have 
> the impression the deps could be even be cloned more than twice, depending on 
> how cleverly SwiftPM realises that multiple Packages have the same dependency.

Right now, SwiftPM will never clone a dependency twice.

> Also, this makes managing interdependent state of development amongst 
> dependencies more difficult than needed. How do we guarantee that devs are on 
> the same commit when using top of tree development? Tagging and managing 
> version numbers etc for day-to-day development is emphatically not an option 
> for us.

When you use top of the tree mode, the checkout is expected to be managed by 
the user. That is exactly what top of the tree mode means. If we get local 
package support in the manifest, you will still need to manage the checkout 
state of 

Re: [swift-users] SwiftPM manual dependency management

[Geordie J via swift-users]

Hi Ankit, thanks for your reply.

> Am 21.07.2017 um 07:33 schrieb Ankit Aggarwal via swift-users 
> :
> 
> 
> 
> On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users 
> > wrote:
> Hi all,
> 
> My team and I are trying to use SwiftPM to develop a relatively complex app 
> with multiple dependencies, all of which are being developed locally and in 
> parallel. The reason for this is compatibility with an existing module/import 
> structure used by our iOS app. Maybe I’m doing something very wrong but my 
> experience so far (2 months in) is that this is extremely difficult with 
> SwiftPM.
> 
> What I’d love to be able to do is to just run `git add submodule 
> http://blah.com/mysubmodule.git`  in the 
> Packages subdirectory and SwiftPM would just let me manage dependencies from 
> there myself.
> 
> I was excited to see that SwiftPM 4 has a "Top of Tree" development option 
> for this purpose. So far my experience with this has not been good. Firstly 
> because SwiftPM still unnecessarily tries to clone my repos itself (some of 
> which are huge), and secondly because this creates an absolute path 
> dependency in `.build/dependencies-state.json`, meaning this setup isn’t 
> sharable within our dev team.
> 
> Attempting this with "local" git urls adds an almost absurd level of 
> complexity, having to tag each commit for SwiftPM to build. The fact that 
> we'd need to make a commit to test whether the project even builds is insane 
> enough as is, let alone the tagging and trying to tell the base project to 
> use a newer minor version etc etc.
> 
> Adding multiple subtargets is also not an option because the dependencies (as 
> dynamic libraries) really are shared between multiple 
> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
> 
> tldr; *Please* let us manage dependencies ourselves. It’d be so easy if 
> Package.swift had an option along the lines of .Package.local(named: "XYZ") 
> that it then looked for in ./Packages/XYZ. Again, maybe I’m overlooking 
> something but this seems like an obvious and vital option to have. It’d also 
> simplify the introductory SwiftPM docs significantly.
> 
> Is anyone else having this issue? Would this change really be as simple and 
> painless as it sounds? I would be prepared to make a pull request along these 
> lines.
> 
> I think you're not really using the Top of Tree feature. You need to add each 
> dependency using its canonical URL, hosted at some server like github. After 
> adding the dependencies, you can use edit feature to put a dependency in Top 
> of Tree mode. To do so, run:
> 
> $ swift package edit  --path 
> ../path/to/self/managed/checkout/of/the/package

Yes, this is what I tried this week. I’m pretty sure this is not a case of 
misunderstanding the feature or the docs.

> 
> The package manager will then stop using the cloned repository and use the 
> checkout present at that path (regardless of the state it is in).

Yes, but then I have – per dependency – two checkouts of a potentially huge 
repository. Why force everyone on the dev team to clone a huge repo twice, only 
to *never* use one of the clones. Also, SwiftPM breaks when —path points at 
Packages/PackageName, which is exactly where I’d expect the package to be, not 
in some arbitrary external path (+ some kind of internal checkout cache that 
will never be used) as well.

I haven’t tried to test this recently because it’s a slow process but I have 
the impression the deps could be even be cloned more than twice, depending on 
how cleverly SwiftPM realises that multiple Packages have the same dependency.

Also, this makes managing interdependent state of development amongst 
dependencies more difficult than needed. How do we guarantee that devs are on 
the same commit when using top of tree development? Tagging and managing 
version numbers etc for day-to-day development is emphatically not an option 
for us. Since SwiftPM packages only work from a git context anyway, why not 
allow use of git’s established pattern of dealing with this, namely submodules?

> Sharing this setup is not automatic, but simple. Each user just needs to run 
> the above command once per dependency.

We have about 10 dependencies, all of which will always be in this state. This 
seems like a lot of overhead and room for user error, plus it’s a huge 
workaround for something that could be very simple.

> Also, you only need to do this if you're actively working on a dependency.

The point is that we will always be working on the dependencies. This is the 
core of what we’re doing, not a short aside. This is what makes me think we are 
either doing something wrong, or there is a big feature gap (as it appears from 
here).

> The new manifest also supports using branch instead of version range, which 
> is very helpful during the development period.

This has much the 

Re: [swift-users] SwiftPM manual dependency management

[Ankit Aggarwal via swift-users]

On Thu, Jul 20, 2017 at 10:34 PM, Geordie J via swift-users <
swift-users@swift.org> wrote:

> Hi all,
>
> My team and I are trying to use SwiftPM to develop a relatively complex
> app with multiple dependencies, all of which are being developed locally
> and in parallel. The reason for this is compatibility with an existing
> module/import structure used by our iOS app. Maybe I’m doing something very
> wrong but my experience so far (2 months in) is that this is extremely
> difficult with SwiftPM.
>
> What I’d love to be able to do is to just run `git add submodule
> http://blah.com/mysubmodule.git` in the Packages subdirectory and SwiftPM
> would just let me manage dependencies from there myself.
>
> I was excited to see that SwiftPM 4 has a "Top of Tree" development option
> for this purpose. So far my experience with this has not been good. Firstly
> because SwiftPM *still* unnecessarily tries to clone my repos itself
> (some of which are huge), and secondly because this creates an absolute
> path dependency in `.build/dependencies-state.json`, meaning this setup
> isn’t sharable within our dev team.
>
> Attempting this with "local" git urls adds an almost absurd level of
> complexity, having to tag each commit for SwiftPM to build. The fact that
> we'd need to make a commit to test whether the project even builds is
> insane enough as is, let alone the tagging and trying to tell the base
> project to use a newer minor version etc etc.
>
> Adding multiple subtargets is also not an option because the dependencies
> (as dynamic libraries) really are shared between multiple
> targets/sub-dependencies, which SwiftPM seems to deal with quite well.
>
> *tldr;* *Please* let us manage dependencies ourselves. It’d be so easy if
> Package.swift had an option along the lines of *.Package.local(named:
> "XYZ")* that it then looked for in ./Packages/XYZ. Again, maybe I’m
> overlooking something but this seems like an obvious and vital option to
> have. It’d also simplify the introductory SwiftPM docs significantly.
>
> Is anyone else having this issue? Would this change really be as simple
> and painless as it sounds? I would be prepared to make a pull request along
> these lines.
>

I think you're not really using the Top of Tree feature. You need to add
each dependency using its canonical URL, hosted at some server like github.
After adding the dependencies, you can use edit feature to put a dependency
in Top of Tree mode. To do so, run:

$ swift package edit  --path
../path/to/self/managed/checkout/of/the/package

The package manager will then stop using the cloned repository and use the
checkout present at that path (regardless of the state it is in). Sharing
this setup is not automatic, but simple. Each user just needs to run the
above command once per dependency. Also, you only need to do this if you're
actively working on a dependency. The new manifest also supports using
branch instead of version range, which is very helpful during the
development period. Let me know if something is unclear or if you have more
questions!


> Best Regards,
> Geordie
>
>
> PS. In SwiftPM 3 we had been using a hack that worked great: by filling in
> the dependencies' "basedOn" key in `workspace-state.json`, SwiftPM just
> left us alone.. We were able to commit `workspace-state.json` into our base
> project’s git repo and the rest Just Worked™. Now with the absolute paths
> being checked for this doesn’t seem to be an option.
>
>
Please do not rely on internals of the package manager as they're not
stable and will change without notice.



> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users
>
>
___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

[swift-users] SwiftPM manual dependency management

[Geordie J via swift-users]

Hi all,

My team and I are trying to use SwiftPM to develop a relatively complex app 
with multiple dependencies, all of which are being developed locally and in 
parallel. The reason for this is compatibility with an existing module/import 
structure used by our iOS app. Maybe I’m doing something very wrong but my 
experience so far (2 months in) is that this is extremely difficult with 
SwiftPM.

What I’d love to be able to do is to just run `git add submodule 
http://blah.com/mysubmodule.git`  in the 
Packages subdirectory and SwiftPM would just let me manage dependencies from 
there myself.

I was excited to see that SwiftPM 4 has a "Top of Tree" development option for 
this purpose. So far my experience with this has not been good. Firstly because 
SwiftPM still unnecessarily tries to clone my repos itself (some of which are 
huge), and secondly because this creates an absolute path dependency in 
`.build/dependencies-state.json`, meaning this setup isn’t sharable within our 
dev team.

Attempting this with "local" git urls adds an almost absurd level of 
complexity, having to tag each commit for SwiftPM to build. The fact that we'd 
need to make a commit to test whether the project even builds is insane enough 
as is, let alone the tagging and trying to tell the base project to use a newer 
minor version etc etc.

Adding multiple subtargets is also not an option because the dependencies (as 
dynamic libraries) really are shared between multiple targets/sub-dependencies, 
which SwiftPM seems to deal with quite well.

tldr; *Please* let us manage dependencies ourselves. It’d be so easy if 
Package.swift had an option along the lines of .Package.local(named: "XYZ") 
that it then looked for in ./Packages/XYZ. Again, maybe I’m overlooking 
something but this seems like an obvious and vital option to have. It’d also 
simplify the introductory SwiftPM docs significantly.

Is anyone else having this issue? Would this change really be as simple and 
painless as it sounds? I would be prepared to make a pull request along these 
lines.

Best Regards,
Geordie


PS. In SwiftPM 3 we had been using a hack that worked great: by filling in the 
dependencies' "basedOn" key in `workspace-state.json`, SwiftPM just left us 
alone.. We were able to commit `workspace-state.json` into our base project’s 
git repo and the rest Just Worked™. Now with the absolute paths being checked 
for this doesn’t seem to be an option.___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users