Thank you, José! I put together a repro here: https://github.com/josevalim/umbrella_sample/pull/5
<https://github.com/josevalim/umbrella_sample/pull/5> Turns out, there was something I was doing that I didn’t realize caused the issue: defp deps do [ {:util, path: Path.expand("../../delorean/apps/util", __DIR__)}, {:shard, path: Path.expand("../../delorean/apps/shard", __DIR__)}, ]end The fact that I was declaring a dependency on both util and shard (and not just on shard, as I had said before) caused the error. Myron On Tue, May 17, 2016 at 11:34 AM, José Valim < [email protected]> wrote: > Given your report, I cannot understand why it does not work. We consider > path deps equal if they point to the same destination and in all cases > (umbrella sibling, umbrella children and regular path deps), the > destination is expanded, which means they should point to the same place: > > > https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/scm/path.ex#L22 > > If you can provide a way to reproduce the error using my umbrella repo ( > https://github.com/josevalim/umbrella_sample) or another repo, I can take > a look. :) > > > > *José Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Tue, May 17, 2016 at 7:40 PM, Myron Marston <[email protected]> > wrote: > >> That's probably solvable though. Do you remember which error in >>> particular did you get? >> >> >> Here’s what I get: >> >> ➜ shaardwolf git:(myron/use-delorean) ✗ mix deps.get >> Dependencies have diverged: >> * util (/Users/myron/moz/shaardwolf/delorean/apps/util) >> the dependency util in apps/crawl_processor/mix.exs is overriding a child >> dependency: >> >> > In apps/crawl_processor/mix.exs: >> {:util, nil, [path: "/Users/myron/moz/shaardwolf/delorean/apps/util", >> manager: :mix]} >> >> > In delorean/apps/shard/mix.exs: >> {:util, nil, [path: "../util", in_umbrella: true]} >> >> Ensure they match or specify one of the above in your deps and set >> "override: true" >> ** (Mix) Can't continue due to errors on dependencies >> >> To explain what the various named things in the error are: >> >> - delorean is my team’s umbrella app >> - shaardwolf is the other’s team’s new umbrella app >> - shard is a library application in delorean at apps/shard which has >> an in-umbrella dependency on util (which is defined at apps/util in >> delorean) >> - crawl_processing is a sub-app in shaardwolf which is declaring a >> dependency like so: >> >> defp deps do >> [ >> {:shard, path: Path.expand("../../delorean/apps/shard", __DIR__)} >> ]end >> >> Delorean is cloned in delorean (under the shaardwolf root) via the >> snippet I pasted before. >> >> One work around I had in mind was to change our dependency declarations >> in delorean to use path: Path.expand(...) instead of in_umbrella: true. >> I tried this on the delorean side, but it triggers errors there: >> >> ➜ delorean git:(myron/use-path-instead-of-in-umbrella) ✗ mix deps.get >> Dependencies have diverged: >> * util (apps/util) >> the dependency util in mix.exs is overriding a child dependency: >> >> > In mix.exs: >> {:util, nil, [path: "apps/util", from_umbrella: true, manager: :mix]} >> >> > In apps/admin/mix.exs: >> {:util, nil, [path: "/Users/myron/moz/delorean/apps/util"]} >> >> Ensure they match or specify one of the above in your deps and set >> "override: true" >> # ... followed by a similar message for every single in-umbrella dependency >> >> I’ve come up with a work-around for this that actually seems to work, >> though — in delorean, I’m conditionally defining the in-umbrella >> dependencies using either {:in_umbrella, true} if System.cwd indicates >> we are in delorean or a subdirectory (meaning we are developing delorean) >> or {:path, Path.expand("../apps/#{dep}", __DIR__)} otherwise (meaning >> delorean has been cloned into shaardwolf and that is being developed). It’s >> quite the hack but I’ve confirmed it works OK on both sides! >> >> Myron >> >> >> >> On Tue, May 17, 2016 at 9:48 AM, José Valim < >> [email protected]> wrote: >> >>> > But once we tried to depend upon one of our sub-apps that had an >>> :in_umbrella dependency, it failed, because mix did not understand that >>> it was an umbrella dependency from *our* umbrella instead of theirs. >>> >>> That's probably solvable though. Do you remember which error in >>> particular did you get? >>> >>> > It sounds like running our own private hex server might be the right >>> approach, unless you want to mix dependencies to handle the :in_umbrella >>> issue >>> when pulling in from another umbrella. Any idea when private hex servers >>> might become viable? >>> >>> Eric should have an idea but no guarantees. >>> >>> >>> >>> *José Valim* >>> www.plataformatec.com.br >>> Skype: jv.ptec >>> Founder and Director of R&D >>> >>> On Tue, May 17, 2016 at 6:14 PM, Myron Marston <[email protected]> >>> wrote: >>> >>>> Is there a way to checkout only a subdirectory in Git? It seems to be >>>>> possible, so someone could contribute this feature to the Git SCM. >>>>> Although >>>>> I would call the option something other than :path. >>>> >>>> >>>> It appears that this kind of checkout is possible in Git 1.7+ using a >>>> feature called sparse checkout >>>> <http://stackoverflow.com/questions/600079/is-there-any-way-to-clone-a-git-repositorys-sub-directory-only/13738951#13738951>. >>>> I'm not sure that that alone would completely solve the issue, though; I >>>> had hacked together an attempted work around in the other team's app like >>>> this: >>>> >>>> # in root mix.exs filedefmodule TheirUmbrella.Mixfile do >>>> use Mix.Project >>>> >>>> def project do >>>> [apps_path: "apps", >>>> build_embedded: Mix.env == :prod, >>>> start_permanent: Mix.env == :prod, >>>> aliases: aliases, >>>> deps: deps] >>>> end >>>> >>>> defp deps do >>>> [] >>>> end >>>> >>>> defp aliases do >>>> ["deps.get": [&fetch_our_umbrella/1, "deps.get"]] >>>> end >>>> >>>> @sha "3d237160a52aa38a33a8413e54e51ad3bf3b041a" >>>> @repo_url "[email protected]:our_org/our_umbrella.git" >>>> >>>> defp fetch_our_umbrella(_args) do >>>> if File.dir?("our_umbrella") do >>>> File.cd!("our_umbrella", fn -> System.cmd("git", ["fetch"]) end) >>>> else >>>> System.cmd("git", ["clone", @repo_url, "our_umbrella"]) >>>> end >>>> >>>> File.cd!("our_umbrella", fn -> System.cmd("git", ["checkout", @sha]) >>>> end) >>>> endend >>>> >>>> >>>> >>>> And then in one of the sub-apps in their project we added {:foo, path: >>>> Path.expand("../../our_umbrella/foo", __DIR__)}. This worked since foo >>>> was a simple dependency that had no :in_umbrella dependencies of its >>>> own. But once we tried to depend upon one of our sub-apps that had an >>>> :in_umbrella dependency, it failed, because mix did not understand >>>> that it was an umbrella dependency from *our* umbrella instead of >>>> theirs. >>>> >>>> So I think that cloning just a subdirectory out of a git repository >>>> isn’t quite sufficient, and in fact it might not be the best approach since >>>> you may need other directories for :in_umbrella dependencies. >>>> >>>> That said, I recognize that this is pretty complex and is probably >>>> going far beyond what umbrella apps were designed to do :). >>>> >>>> The only option for now is to break those into git dependencies. We are >>>>> working on private Hex servers and registries, so that should be an option >>>>> soon. One of these two options are arguably the best approach: since those >>>>> applications are now used by different projects, it probably makes sense >>>>> to >>>>> move them elsewhere. >>>> >>>> From a purist’s point of view, it absolutely makes sense to move these >>>> into separate git repositories. But from a pragmatic point of view, that is >>>> undesirable for us. Since we are the primary creators, users and >>>> maintainers of these libraries, it has been really helpful to have them in >>>> the same git repository. It means that our CI builds uncover integration >>>> issues between a dependency and its consumer within our umbrella, and that >>>> it is easy for us to change a library and the consumer in the same >>>> commit—kinda like how for elixir-lang development, you’ve found it useful >>>> to keep mix, ex_unit, iex, etc in the same repository with elixir itself. >>>> It facilitates simpler development workflows. >>>> >>>> Even if we wanted to split these into separate git repositories, we’d >>>> probably have to recon with the :in_umbrella issue I mentioned above: >>>> if the other team wanted to depend upon just one of our apps foo, if >>>> foo depended upon bar and baz in our umbrella, it would not be >>>> sufficient to just move foo into a separate git repository (since it >>>> would run into the very issue the other team has run into). We’d also have >>>> to move bar and baz out into their own repositories so that foo could >>>> depend on them. Suddenly my team has to maintain code across 4 git >>>> repositories instead of 1. That’s definitely undesirable. >>>> >>>> It sounds like running our own private hex server might be the right >>>> approach, unless you want to mix dependencies to handle the >>>> :in_umbrella issue when pulling in from another umbrella. Any idea >>>> when private hex servers might become viable? >>>> >>>> Thanks, >>>> >>>> Myron >>>> >>>> >>>> >>>> >>>> On Tuesday, May 17, 2016 at 8:41:29 AM UTC-7, José Valim wrote: >>>>> >>>>> Ideally we’d like to be able to get this to work: >>>>>> >>>>>> defp deps do >>>>>> {:foo, git: "[email protected]:my_org/our_umbrella.git", path: >>>>>> "apps/foo"}end >>>>>> >>>>>> >>>>> Is there a way to checkout only a subdirectory in Git? It seems to be >>>>> possible, so someone could contribute this feature to the Git SCM. >>>>> Although >>>>> I would call the option something other than :path. >>>>> >>>>>> Is there a currently supported way to get this to work, or are we >>>>>> going to have to extract the bits they want to use into separate, >>>>>> stand-alone git repositories (something we’d rather not have to do)? Or >>>>>> is >>>>>> it something the Elixir team would consider supporting in a future >>>>>> version >>>>>> of Elixir? Is there a suggested work around for now? >>>>>> >>>>> The only option for now is to break those into git dependencies. We >>>>> are working on private Hex servers and registries, so that should be an >>>>> option soon. One of these two options are arguably the best approach: >>>>> since >>>>> those applications are now used by different projects, it probably makes >>>>> sense to move them elsewhere. >>>>> >>>>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/CADUxQmtx7PzMoYwjZxHZcN%3DJRyqQACL%2BOXJr%3Dg%2Bhs_irB20fvA%40mail.gmail.com >> <https://groups.google.com/d/msgid/elixir-lang-core/CADUxQmtx7PzMoYwjZxHZcN%3DJRyqQACL%2BOXJr%3Dg%2Bhs_irB20fvA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elixir-lang-core" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elixir-lang-core/aynMv_jMNFw/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JiakRM5FcPxoaDJ%3D01vEO6_XHKFFHfAoENy%3DeoTdTppA%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JiakRM5FcPxoaDJ%3D01vEO6_XHKFFHfAoENy%3DeoTdTppA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CADUxQmvMDQ%3DL3TSJeDBe5PofDo6bz_RJPvX72eG302zfsV5yuQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
