Juju's dependencies, juju/utils and a road forward
Hi folks, Juju has a problem with its dependencies. We have been trying to upgrade a number of our dependencies recently have have been hitting road blocks. Mostly because of changes to common libraries that are incompatible with other libraries that we use. I'm pretty sure that the root of all these problems is the juju/utils package. It has become a dumping ground for any common functions that packages may want to share. However these common functions are changed arbitrarily with only thoughts around how they impact a subset of the downstreams of juju/utils. I think that the only sane way forward is to move all the functionality we care about out of juju/utils into more cohesive, smaller, packages that have meaningful names and you should be able to determine what they do. Where the current clashes came from were a few places. 1) we wanted to change our mgo dependency 2) a new file path utility function was wanted 3) the debugstatus package inside utils was updated in a backward incompatible way The root problem here is that the utils package tries to do too many things. Any package called "utils" should be a warning, and we should have listened to Dave Cheney when he brought this up originally. I think the way forward is to break up the utils package. I'd like to propose that we don't add anything new to the utils package, and only deletions are allowed. We create new packages that contain the functionality we require, and have them be much more controlled on their own dependencies. Thoughts? Tim -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju's dependencies, juju/utils and a road forward
As a follow on... I'd like to move juju to only use "stable" dependency branches. No more '-unstable'. Tim On 04/04/18 12:52, Tim Penhey wrote: > Hi folks, > > Juju has a problem with its dependencies. We have been trying to upgrade > a number of our dependencies recently have have been hitting road > blocks. Mostly because of changes to common libraries that are > incompatible with other libraries that we use. > > I'm pretty sure that the root of all these problems is the juju/utils > package. It has become a dumping ground for any common functions that > packages may want to share. However these common functions are changed > arbitrarily with only thoughts around how they impact a subset of the > downstreams of juju/utils. > > I think that the only sane way forward is to move all the functionality > we care about out of juju/utils into more cohesive, smaller, packages > that have meaningful names and you should be able to determine what they do. > > Where the current clashes came from were a few places. > 1) we wanted to change our mgo dependency > 2) a new file path utility function was wanted > 3) the debugstatus package inside utils was updated in a backward > incompatible way > > The root problem here is that the utils package tries to do too many > things. Any package called "utils" should be a warning, and we should > have listened to Dave Cheney when he brought this up originally. > > I think the way forward is to break up the utils package. I'd like to > propose that we don't add anything new to the utils package, and only > deletions are allowed. We create new packages that contain the > functionality we require, and have them be much more controlled on their > own dependencies. > > Thoughts? > > Tim > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju's dependencies, juju/utils and a road forward
Which then begs the question... Why do we have stable gopkg.in branches depending on unstable branches? i.e. gopkg.in/juju/charmstore.v5 depending on gopkg.in/macaroon-bakery.v2-unstable? This seems to smell bad. Tim On 04/04/18 14:17, Tim Penhey wrote: > As a follow on... > > I'd like to move juju to only use "stable" dependency branches. No more > '-unstable'. > > Tim > > On 04/04/18 12:52, Tim Penhey wrote: >> Hi folks, >> >> Juju has a problem with its dependencies. We have been trying to upgrade >> a number of our dependencies recently have have been hitting road >> blocks. Mostly because of changes to common libraries that are >> incompatible with other libraries that we use. >> >> I'm pretty sure that the root of all these problems is the juju/utils >> package. It has become a dumping ground for any common functions that >> packages may want to share. However these common functions are changed >> arbitrarily with only thoughts around how they impact a subset of the >> downstreams of juju/utils. >> >> I think that the only sane way forward is to move all the functionality >> we care about out of juju/utils into more cohesive, smaller, packages >> that have meaningful names and you should be able to determine what they do. >> >> Where the current clashes came from were a few places. >> 1) we wanted to change our mgo dependency >> 2) a new file path utility function was wanted >> 3) the debugstatus package inside utils was updated in a backward >> incompatible way >> >> The root problem here is that the utils package tries to do too many >> things. Any package called "utils" should be a warning, and we should >> have listened to Dave Cheney when he brought this up originally. >> >> I think the way forward is to break up the utils package. I'd like to >> propose that we don't add anything new to the utils package, and only >> deletions are allowed. We create new packages that contain the >> functionality we require, and have them be much more controlled on their >> own dependencies. >> >> Thoughts? >> >> Tim >> > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju's dependencies, juju/utils and a road forward
I agree with breaking up utils. I've certainly read similar comments elsewhere that calling something 'utils' inherently causes issues like this, and it seems to fit with our experience. I also agree that X stable depending on Y unstable doesn't seem right. Though we're also very bad at ever making a genuine release of dependencies. (it's a lot of overhead if nobody else is actually consuming them) There are probably other small things like it is "mgo/bson" where bson should really be it's own package. Also some questions about how well packages avoid leaking. (charmstore uses mgo as a backend. if it doesn't expose those types then we probably don't care if some of our tree depends on it while the other part doesn't. At least as a transition phase.) John =:-> On Apr 4, 2018 06:19, "Tim Penhey" wrote: Which then begs the question... Why do we have stable gopkg.in branches depending on unstable branches? i.e. gopkg.in/juju/charmstore.v5 depending on gopkg.in/macaroon-bakery.v2-unstable? This seems to smell bad. Tim On 04/04/18 14:17, Tim Penhey wrote: > As a follow on... > > I'd like to move juju to only use "stable" dependency branches. No more > '-unstable'. > > Tim > > On 04/04/18 12:52, Tim Penhey wrote: >> Hi folks, >> >> Juju has a problem with its dependencies. We have been trying to upgrade >> a number of our dependencies recently have have been hitting road >> blocks. Mostly because of changes to common libraries that are >> incompatible with other libraries that we use. >> >> I'm pretty sure that the root of all these problems is the juju/utils >> package. It has become a dumping ground for any common functions that >> packages may want to share. However these common functions are changed >> arbitrarily with only thoughts around how they impact a subset of the >> downstreams of juju/utils. >> >> I think that the only sane way forward is to move all the functionality >> we care about out of juju/utils into more cohesive, smaller, packages >> that have meaningful names and you should be able to determine what they do. >> >> Where the current clashes came from were a few places. >> 1) we wanted to change our mgo dependency >> 2) a new file path utility function was wanted >> 3) the debugstatus package inside utils was updated in a backward >> incompatible way >> >> The root problem here is that the utils package tries to do too many >> things. Any package called "utils" should be a warning, and we should >> have listened to Dave Cheney when he brought this up originally. >> >> I think the way forward is to break up the utils package. I'd like to >> propose that we don't add anything new to the utils package, and only >> deletions are allowed. We create new packages that contain the >> functionality we require, and have them be much more controlled on their >> own dependencies. >> >> Thoughts? >> >> Tim >> > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev