The only other technique I know of is library "shading", where the build system aliases the packages of the classes to something unique and "relinks" the build to use that custom namespace for the library, allowing multiple versions to coexist on the same classpath. Daffodil could then shade the most common libs, to avoid upstream mixing, or there could be pointers to instructions for integrators of Daffodil so they can more easily shade their own dependencies.
I don't recall the mechanics of shading in Scala, but there's plenty of links via Google, like https://medium.com/the-code-shelf/are-you-stuck-in-the-dependency-hell-2faa9943798a . Apologies if this possibility has already been considered. .. Adam On Tue, Jun 29, 2021 at 9:42 AM Beckerle, Mike < mbecke...@owlcyberdefense.com> wrote: > We have many *many* dependencies from Daffodil, and this has the potential > to cause conflicts when creating systems that use Daffodil as part of a > larger system. > > If an application requires libraries A and B (suppose B is daffodil), and > those each in turn require library C (suppose C is the ICU libraries), but > A and B each depend on different incompatible versions of C, then you have > an unresolvable conflict. > > This is called the "dependency isolation" problem. One would like to be > able to link libraries A and B with their own respective isolated versions > of library C, with no interactions. > > In the past I had examined the OGSI modular components system (used by > Eclipse) and rejected it for excess complexity. > OGSI solves the dependency isolation problem, but it also introduces a lot > of additional long-running software system lifecycle mechanism that is > quite complex, and not well motivated for a library like Daffodil. > > I had hoped that the Java 9 modules stuff would be a rethink on all of > this, and that it would be focused on solving only *selective linking* > (leaving out what you don't use), and dependency isolation. > > What I have learned is that Java 9 modules does half of what I wanted. > > It successfully allows selective linking and has been used to modularize > the gigantic Java runtime to enable smaller footprint in memory. That much > is good. > > But the Java 9 modules system does *not* solve the dependency isolation > problem. There can still only be one copy of each library version for all > dependencies of all modules. > > I would include references, but a web search for Java 9 Modules OGSI gets > you many hits and you can find the discussions easily. > > Tools like scala-steward don't eliminate the need for dependency > isolation, but they do minimize it, especially for open-source software. > > People who are building systems combining large libraries like Daffodil > with commercial slow-changing software libraries having dependencies on > older versions of many other things, those are the people who really run > into the need for a dependency isolation solution. > > For the time being they will have to continue to rely on running > incompatible things in separate processes or containers or using OGSI if > they want everything in the same process/JVM. > > Mike Beckerle | Principal Engineer > > mbecke...@owlcyberdefense.com <bhum...@owlcyberdefense.com> > P +1-781-330-0412 > >
