Splitting out state/api into its own repo
(I've put a more structured proposal below, but here's some context.) Over the last couple weeks I've been spinning up on the juju code base, which is large enough to dissipate any hope of understanding it all quickly. Most of what I've focused on is relative to the juju tools and the remote API, both for the sake of backup/restore. One thing that threw me off is that it has not been obvious that the code under `state/api` is the public-facing API for juju's state (as someone recently explained to me). For a while I thought `state/api` held the state API client code, but from what I understand now it actually contains all the public facing code for the juju state API (and [consequently?] for juju as a whole?). In some regard I would expect a public API to be sit in a top-level package (rather than nested down like it is). A couple of other things got in the way. For one, we basically don't have any documentation for the API. I expect that it will mirror the documentation for the juju subcommands/tools pretty closely, but regardless the doc doesn't exist. Setting up godoc for the api package would be great and so would a new page in the juju documentation. I realize there has been some discussion on this point of late, from which I expect a doc will take shape in the short term. In the meantime, we've actually been telling people to wrap calls to the CLI tools rather than using the API. The documentation side of things is a somewhat orthogonal, though topically related, issue. For another, while there has been a pretty good effort to keep the `api` package relatively un-entwined from the rest of juju, there were a few times when I found it hard to follow what was going on. This was particularly true of the underlying state RPC implementation, though at this point things make a lot more sense. Having a separate repo would help delineate the boundary between the API and juju itself, which should in turn help make the API code easier to follow. So... In the interest of understanding juju better and of making the API more accessible, I took a little time to investigate possible improvements. One of the first ones to come to mind was to split `state/api` into its own repo. That smelled like a heavy lift especially considering the many interdependencies between `state/api` and juju proper (though apparently `go` mitigates that somewhat). Undaunted, I gave it an initial stab (see `Implementation` below). The bulk of this effort was fixing all the imports, which I ended up writing a script to solve. All the tests pass on both sides. I wouldn't be comfortable with the split as-is (see `Left to do` below) but it demonstrated to me that it is possible. That said, *possible* should not imply *advisable*, and given my inexperience with the juju project I don't presume to do much more about splitting out an api repo without feedback. At the very least this will be a chance for you to educate me (and presumably others) about the juju code base. :) If the code split is a bad idea then this is your change to officially put the idea to rest. (It would not surprise me if I've misunderstood something important here!) If illumination and a public rejection of the idea are all that come of this I'll still be satisfied. However, I'm hopeful that it's a good enough idea to warrant the effort and that I understand enough about juju at this point to have at least a vague sense of that. :) Thoughts? -eric ++ Proposal == * split `github.com/juju/juju/state/api` into its own repo * place the new api repo at `github.com/juju/api` * reduce dependencies in the new repo on the code in the main juju repo * introduce a juju API client interface type * godocs for the `api` package (should happen regardless) * more? (see `Open questions below`) Pros === * helps bless the API as a supported first-class feature of juju * makes the API more discoverable * encourages a stronger separation between the public API (and implementation?) and juju proper, including on the server side * reduces the chance that we inadvertently change/break the public API Cons * touches a lot of code + churn + requires that a lot of (most?) outstanding patches be updated * risk of introducing new bugs * effort to make the split * effort to reduce dependencies in the new repo + the API *implementation* is tightly entwined with juju proper? Open questions === Here are some questions that have bearing on this proposal. Some of them would directly impact the scope of the proposal. Others could be addressed separately (to spread the scope out in manageable chunks). * is `state/api` just for the state client/RPC, juju state in general, or juju as a whole? From what I understand, it is the middle one (though I originally thought it was the first). * is there other public juju API other than the state-related API? * should the `api` repo contains *all* public-facing juju API (sta
Re: Splitting out state/api into its own repo
Hey Eric, Some comments below, offering a slightly different perspective to be used as a data point in your quest. On Thu, Jun 26, 2014 at 1:41 PM, Eric Snow wrote: > In the interest of understanding juju better and of making the API > more accessible, I took a little time to investigate possible > improvements. One of the first ones to come to mind was to split > `state/api` into its own repo. That smelled like a heavy lift > especially considering the many interdependencies between `state/api` > and juju proper (though apparently `go` mitigates that somewhat). There are a few different arguments bundled together: 1. state/api is a bad location for the state API endpoints 2. documentation is not great 3. moving code into a separate repository makes its orthogonality more clear The answer for 2 is easy: let's have better documentation. That's unrelated to where the code sits. For 1, in my simplistic view state/api feels like a pretty reasonable place for an http interface that offers access to the functionality in state. For 3, splitting it off not only seems to needlessly increase the maintenance burden, but also feels incorrect from the orthogonality standpoint: state/api maps state into a public API, and is a critical dependency for juju to work at all. gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Splitting out state/api into its own repo
I have a slightly different proposal, inspired by the recent Go proposal for internal imports (http://golang.org/s/go14internal), which currently looks like it will actually be implemented. We move all public facing APIs into top level packages within the juju repo, and move everything else under github.com/juju/juju/internal Packages I would suggest should be exported: github.com/juju/juju/api// moved from state/api github.com/juju/juju/cmd/juju github.com/juju/juju/cmd/jujud ... (all the other commands, but not the cmd package itself) github.com/juju/ github.com/juju/juju/cmd/internal/jujucmd// containing supercommand.go from current cmd github.com/juju/juju/juju // but only the API-related pieces Some packages would want moving, because they're user-facing but currently inside packages that we would probably not want to export: github.com/juju/environs/configstore is the only one I can think of right now. Others would want splitting. For example, the environs package is a mix between user-facing and internal stuff right now. It would be great to take out all the user-facing config file stuff (that might sit well inside the juju package). Still others would benefit from being made available externally. I think the rpc package is probably one of those that would sit well inside its own repo. Some packages could benefit from having their own internal directory - the uniter is one of those, for example. The apiserver too has many sub-packages that should not really be visible to the rest of juju. In the end, we should end up with an API that it might actually be feasible to stabilise. I'd like juju to move under gopkg.in at some point, providing useful stability guarantees for external users that might want to build Go programs based on our code. cheers, rog. On 26 June 2014 17:41, Eric Snow wrote: > (I've put a more structured proposal below, but here's some context.) > > Over the last couple weeks I've been spinning up on the juju code > base, which is large enough to dissipate any hope of understanding it > all quickly. Most of what I've focused on is relative to the > juju tools and the remote API, both for the sake of backup/restore. > > One thing that threw me off is that it has not been obvious that the > code under `state/api` is the public-facing API for juju's state (as > someone recently explained to me). For a while I thought `state/api` > held the state API client code, but from what I understand now it > actually contains all the public facing code for the juju state API > (and [consequently?] for juju as a whole?). In some regard I would > expect a public API to be sit in a top-level package (rather than > nested down like it is). > > A couple of other things got in the way. For one, we basically don't > have any documentation for the API. I expect that it will mirror the > documentation for the juju subcommands/tools pretty closely, but > regardless the doc doesn't exist. Setting up godoc for the api > package would be great and so would a new page in the juju > documentation. I realize there has been some discussion on this point > of late, from which I expect a doc will take shape in the short term. > In the meantime, we've actually been telling people to wrap calls to > the CLI tools rather than using the API. The documentation side of > things is a somewhat orthogonal, though topically related, issue. > > For another, while there has been a pretty good effort to keep the > `api` package relatively un-entwined from the rest of juju, there were > a few times when I found it hard to follow what was going on. This > was particularly true of the underlying state RPC implementation, > though at this point things make a lot more sense. Having a separate > repo would help delineate the boundary between the API and juju > itself, which should in turn help make the API code easier to follow. > > So... > > In the interest of understanding juju better and of making the API > more accessible, I took a little time to investigate possible > improvements. One of the first ones to come to mind was to split > `state/api` into its own repo. That smelled like a heavy lift > especially considering the many interdependencies between `state/api` > and juju proper (though apparently `go` mitigates that somewhat). > > Undaunted, I gave it an initial stab (see `Implementation` below). > The bulk of this effort was fixing all the imports, which I ended up > writing a script to solve. All the tests pass on both sides. I > wouldn't be comfortable with the split as-is (see `Left to do` below) > but it demonstrated to me that it is possible. > > That said, *possible* should not imply *advisable*, and given my > inexperience with the juju project I don't presume to do much more > about splitting out an api repo without feedback. At the very least > this will be a chance for you to educate me (and presumably others) > about the juju code base
juju devel 1.19.4 is released
juju-core 1.19.4 A new development release of Juju, juju-core 1.19.4, is now available. Getting Juju juju-core 1.19.4 is available for trusty and backported to earlier series in the following PPA: https://launchpad.net/~juju/+archive/devel Upgrading from stable releases to development releases is not supported. You can upgrade test environments to development releases to test new features and fixes, but it is not advised to upgrade production environments to 1.19.4. If you are using a development release of juju-core, and find you need to go back to a stable release, you can find it in the juju stable PPA: https://launchpad.net/~juju/+archive/stable If you have multiple sources of juju-core, you can select the version you want using apt: sudo apt-get install juju-core=1.18.4* New and Notable * Availability zone placement * Network constraints and deploy argument for MasS * Server side API Versioning * New juju-local-kvm package Resolved issues * Images are not found if endpoint and region are inherited from the top level in simplestreams metadata Lp 1329805 * Replicaset initiation fails when mongo is on a big, slow disk Lp 1327940 * Missing @ syntax for reading config setting from file content Lp 1216967 * Upgrading 1.18 to 1.19 breaks agent.conf Lp 1333682 * Juju upgrade-juju needs a dry run mode Lp 1272544 * Juju debug-hooks should display a start message Lp 1270856 * Can't determine which relation is in error from status Lp 1194481 * local charm deployment fails with symlinks Lp 1330919 * The root-disk constraint is broken on ec2 Lp 1324729 * Creating a local environment stops the syslog (1.19.3) Lp 1332358 * Can't destroy MAAS environment with LXCs Lp 1325830 * Default bootstrap timeout is too low for MAAS environments Lp 1314665 * Azure destroy-environment does not complete Lp 1324910 * Azure bootstrap dies with xml schema validation error Lp 1259947 * Azure provider stat output does not show machine hardware info Lp 1215177 * Bootstrapping azure causes memory to fill Lp 1250007 * Floating IPs are not recycled in OpenStack Havana Lp 1247500 Availability zone placement Juju supports explicit placement of machines to availability zones (AZs), and implicit spread units across the available zones. When bootstrapping or adding a machine, you can specify the availability zone explicitly as a placement directive. e.g. juju bootstrap --to zone=us-east-1b juju add-machine zone=us-east-1c If you don't specify a zone explicitly, Juju will automatically and uniformly distribute units across the available zones within the region. Assuming the charm and the charm's service are well written, you can rest assured that IaaS downtime will not affect your application. Commands you already use will ensure your services are always available. e.g. juju deploy -n 10 When adding machines without an AZ explicitly specified, or when adding units to a service, the ec2 and openstack providers will now automatically spread instances across all available AZs in the region. The spread is based on density of instance "distribution groups". State servers compose a distribution group: when running "juju ensure-availability", state servers will be spread across AZs. Each deployed service (e.g. mysql, redis, whatever) composes a separate distribution group; the AZ spread of one service does not affect the AZ spread of another service. Amazon's EC2 and OpenStack Havana-based clouds and newer are supported. This includes HP Cloud. Older versions of OpenStack are not supported. Azure uses an inverted concept of availability sets, and Juju announced support for this in 1.19.0 Network constraints and deploy argument for MasS You can specify which networks to include or exclude as a constraint to the deploy command. The constraint is used to select a machine to deploy the service's units too. The value of "networks=" is a comma-delimited list of juju network names (provided by MaaS). Excluded networks are prefixed with a "^". For example, this command specify the service requires the "logging" and "storage" networks and conflicts with the "db" and "dmz" networks. juju deploy mongodb --constraints networks=logging,storage,^db,^dmz The network constraint does not enable the network for the service. It only defines what machine to pick. Use the "deploy" command's "--networks" argument to specify service-specific network requirements. The "--networks" argument takes a comma-delimited list of juju-specific network names. Juju will enable the networks on the machines that host service units. Juju networking support is still experimental and under development, currently only supported with the MaaS provider. The "--exclude-network" argument was removed from the deploy command as it is superseded by the constraint option. There are plans to add support for network constraint and argument with Amazon EC2, Azure, and OpenStack Havana-base
Critical bugs for 1.20.
We are tracking a list of critical bugs that must be fixed to release 1.20. https://launchpad.net/juju-core/+milestone/1.20.0 I intend to create a 1.20 branch in https://github.com/juju/juju that we can merge fixes into. Actually I have done that, but github doesn't want me to merge by version branch...I need to tinker with the issue to resolve it. I think the 1.20 branch needs to be version 1.20.0 I propose we set the master branch to 1.21-alpha1 now so that development can continue to add new features. -- Curtis Hovey Canonical Cloud Development and Operations http://launchpad.net/~sinzui -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Port ranges - restricting opening and closing ranges
Hi, me and Matthew Williams are working on support for port ranges in juju. There is one question that the networking model document does not answer explicitly and the simplicity (or complexity) of the implementation depends greatly on that. Should we only allow units to close exactly the same port ranges that they have opened? That is, if a unit opens the port range [8000-9000], can it later close ports [8500-8600], effectively splitting the previously opened port range in half? Domas -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Port ranges - restricting opening and closing ranges
My belief is that as long as the error messages are clear, and it is easy to close 8000-9000 and then open 8000-8499 and 8600-9000, we are fine. Of course it is "nicer" if we can do that automatically for you, but I don't see why we can't add that later, and I think there is a value in keeping a port-range as an atomic data-object either way. --Mark Ramm On Thu, Jun 26, 2014 at 2:11 PM, Domas Monkus wrote: > Hi, > me and Matthew Williams are working on support for port ranges in juju. > There is one question that the networking model document does not answer > explicitly and the simplicity (or complexity) of the implementation depends > greatly on that. > > Should we only allow units to close exactly the same port ranges that they > have opened? That is, if a unit opens the port range [8000-9000], can it > later close ports [8500-8600], effectively splitting the previously opened > port range in half? > > Domas > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Port ranges - restricting opening and closing ranges
+1 to Mark's point. Handling exact matches is much easier, and does not prevent a fancier feature later, if there's ever the need. On Thu, Jun 26, 2014 at 3:38 PM, Mark Ramm-Christensen (Canonical.com) wrote: > My belief is that as long as the error messages are clear, and it is easy to > close 8000-9000 and then open 8000-8499 and 8600-9000, we are fine.Of > course it is "nicer" if we can do that automatically for you, but I don't > see why we can't add that later, and I think there is a value in keeping a > port-range as an atomic data-object either way. > > --Mark Ramm > > > On Thu, Jun 26, 2014 at 2:11 PM, Domas Monkus > wrote: >> >> Hi, >> me and Matthew Williams are working on support for port ranges in juju. >> There is one question that the networking model document does not answer >> explicitly and the simplicity (or complexity) of the implementation depends >> greatly on that. >> >> Should we only allow units to close exactly the same port ranges that they >> have opened? That is, if a unit opens the port range [8000-9000], can it >> later close ports [8500-8600], effectively splitting the previously opened >> port range in half? >> >> Domas >> >> -- >> Juju-dev mailing list >> Juju-dev@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju-dev >> > > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
gerritt & dependent reviews
BTW, Gabirel, the Cloudbase guy I am working with, mentioned that gerritt allows for dependent reviews I thought some on this list might find that worth any other objections they might have with gerritt. I don't know any more about the feature than that, but I figured it was worth looking into. -Nate -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Juju 1.20 release branch
Hi folks The Juju 1.19.4 release will serve as a release candidate for Juju 1.20, the next stable release. There are still a couple of known bugs to fix before 1.20 goes out the door. To allow trunk development to continue unabated, a 1.20 branch has been forked off the rev used to cut 1.19.4: https://github.com/juju/juju/tree/1.20 Any commits to the 1.20 branch will of course need to be ported forward to trunk. CI has been set up to monitor changes to the 1.20 branch, and we'll also have a landing job set up to gate commits to the branch (real soon now). As with previous releases, we won't release until CI is happy. The 1.20 milestone with bugs needing to be fixed before release is here: https://launchpad.net/juju-core/+milestone/1.20.0 Thanks, Ian -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Critical bugs for 1.20.
I'm currently taking a look at #1334273. It could be related to the "restricted API logins during upgrades" change I made. On 27 June 2014 06:00, Curtis Hovey-Canonical wrote: > We are tracking a list of critical bugs that must be fixed to release 1.20. > https://launchpad.net/juju-core/+milestone/1.20.0 > > I intend to create a 1.20 branch in > https://github.com/juju/juju > that we can merge fixes into. Actually I have done that, but github > doesn't want me to merge by version branch...I need to tinker with the > issue to resolve it. > > I think the 1.20 branch needs to be version 1.20.0 > I propose we set the master branch to 1.21-alpha1 now so that > development can continue to add new features. > > -- > Curtis Hovey > Canonical Cloud Development and Operations > http://launchpad.net/~sinzui > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Splitting out state/api into its own repo
Just my quick thought, I think moving it out from "state/api" into just a top level "api" would be reasonable and a lot less clumsy than trying to pull it out into an entirely separate repository. I'm not sure if Gustavo realized that "state/apiserver" is the HTTP(-ish, its really just JSON rpc over a websocket) interface to state. "state/api" is all go client code (agents or user client code). And it would at least be really nice for all those things to have an import test that says "does not depend on anything under "state".) It certainly would make it clearer that nothing that is using the api should import something from, say, apiserver, or state directly. John =:-> On Thu, Jun 26, 2014 at 9:23 PM, roger peppe wrote: > I have a slightly different proposal, inspired by the recent > Go proposal for internal imports (http://golang.org/s/go14internal), > which currently looks like it will actually be implemented. > > We move all public facing APIs into top level packages within > the juju repo, and move everything else under > github.com/juju/juju/internal > > Packages I would suggest should be exported: > > github.com/juju/juju/api// moved from state/api > github.com/juju/juju/cmd/juju > github.com/juju/juju/cmd/jujud > ... (all the other commands, but not the cmd package itself) > github.com/juju/ > github.com/juju/juju/cmd/internal/jujucmd// containing > supercommand.go from current cmd > github.com/juju/juju/juju // but only the API-related pieces > > Some packages would want moving, because they're user-facing > but currently inside packages that we would probably not want to > export: > > github.com/juju/environs/configstore > > is the only one I can think of right now. > > Others would want splitting. For example, the environs package > is a mix between user-facing and internal stuff right now. > It would be great to take out all the user-facing config file stuff > (that might sit > well inside the juju package). > > Still others would benefit from being made available externally. > I think the rpc package is probably one of those that would > sit well inside its own repo. > > Some packages could benefit from having their own internal > directory - the uniter is one of those, for example. The > apiserver too has many sub-packages that should not really > be visible to the rest of juju. > > In the end, we should end up with an API that it might actually be > feasible to stabilise. I'd like juju to move under gopkg.in at some > point, providing useful stability guarantees for external users > that might want to build Go programs based on our code. > > cheers, > rog. > > On 26 June 2014 17:41, Eric Snow wrote: > > (I've put a more structured proposal below, but here's some context.) > > > > Over the last couple weeks I've been spinning up on the juju code > > base, which is large enough to dissipate any hope of understanding it > > all quickly. Most of what I've focused on is relative to the > > juju tools and the remote API, both for the sake of backup/restore. > > > > One thing that threw me off is that it has not been obvious that the > > code under `state/api` is the public-facing API for juju's state (as > > someone recently explained to me). For a while I thought `state/api` > > held the state API client code, but from what I understand now it > > actually contains all the public facing code for the juju state API > > (and [consequently?] for juju as a whole?). In some regard I would > > expect a public API to be sit in a top-level package (rather than > > nested down like it is). > > > > A couple of other things got in the way. For one, we basically don't > > have any documentation for the API. I expect that it will mirror the > > documentation for the juju subcommands/tools pretty closely, but > > regardless the doc doesn't exist. Setting up godoc for the api > > package would be great and so would a new page in the juju > > documentation. I realize there has been some discussion on this point > > of late, from which I expect a doc will take shape in the short term. > > In the meantime, we've actually been telling people to wrap calls to > > the CLI tools rather than using the API. The documentation side of > > things is a somewhat orthogonal, though topically related, issue. > > > > For another, while there has been a pretty good effort to keep the > > `api` package relatively un-entwined from the rest of juju, there were > > a few times when I found it hard to follow what was going on. This > > was particularly true of the underlying state RPC implementation, > > though at this point things make a lot more sense. Having a separate > > repo would help delineate the boundary between the API and juju > > itself, which should in turn help make the API code easier to follow. > > > > So... > > > > In the interest of understanding juju better and of making the API > > more accessible, I took a little time to investigate possible >
Fwd: Splitting out state/api into its own repo
[Hmm, I'm not sure this got through originally, as I sent it from my non-work gmail account] I have a slightly different proposal, inspired by the recent Go proposal for internal imports (http://golang.org/s/go14internal), which currently looks like it will actually be implemented. We move all public facing APIs into top level packages within the juju repo, and move everything else under github.com/juju/juju/internal Packages I would suggest should be exported: github.com/juju/juju/api// moved from state/api github.com/juju/juju/cmd/juju github.com/juju/juju/cmd/jujud ... (all the other commands, but not the cmd package itself) github.com/juju/ github.com/juju/juju/cmd/internal/jujucmd// containing supercommand.go from current cmd github.com/juju/juju/juju // but only the API-related pieces Some packages would want moving, because they're user-facing but currently inside packages that we would probably not want to export: github.com/juju/environs/configstore is the only one I can think of right now. Others would want splitting. For example, the environs package is a mix between user-facing and internal stuff right now. It would be great to take out all the user-facing config file stuff (that might sit well inside the juju package). Still others would benefit from being made available externally. I think the rpc package is probably one of those that would sit well inside its own repo. Some packages could benefit from having their own internal directory - the uniter is one of those, for example. The apiserver too has many sub-packages that should not really be visible to the rest of juju. In the end, we should end up with an API that it might actually be feasible to stabilise. I'd like juju to move under gopkg.in at some point, providing useful stability guarantees for external users that might want to build Go programs based on our code. cheers, rog. On 26 June 2014 17:41, Eric Snow wrote: > (I've put a more structured proposal below, but here's some context.) > > Over the last couple weeks I've been spinning up on the juju code > base, which is large enough to dissipate any hope of understanding it > all quickly. Most of what I've focused on is relative to the > juju tools and the remote API, both for the sake of backup/restore. > > One thing that threw me off is that it has not been obvious that the > code under `state/api` is the public-facing API for juju's state (as > someone recently explained to me). For a while I thought `state/api` > held the state API client code, but from what I understand now it > actually contains all the public facing code for the juju state API > (and [consequently?] for juju as a whole?). In some regard I would > expect a public API to be sit in a top-level package (rather than > nested down like it is). > > A couple of other things got in the way. For one, we basically don't > have any documentation for the API. I expect that it will mirror the > documentation for the juju subcommands/tools pretty closely, but > regardless the doc doesn't exist. Setting up godoc for the api > package would be great and so would a new page in the juju > documentation. I realize there has been some discussion on this point > of late, from which I expect a doc will take shape in the short term. > In the meantime, we've actually been telling people to wrap calls to > the CLI tools rather than using the API. The documentation side of > things is a somewhat orthogonal, though topically related, issue. > > For another, while there has been a pretty good effort to keep the > `api` package relatively un-entwined from the rest of juju, there were > a few times when I found it hard to follow what was going on. This > was particularly true of the underlying state RPC implementation, > though at this point things make a lot more sense. Having a separate > repo would help delineate the boundary between the API and juju > itself, which should in turn help make the API code easier to follow. > > So... > > In the interest of understanding juju better and of making the API > more accessible, I took a little time to investigate possible > improvements. One of the first ones to come to mind was to split > `state/api` into its own repo. That smelled like a heavy lift > especially considering the many interdependencies between `state/api` > and juju proper (though apparently `go` mitigates that somewhat). > > Undaunted, I gave it an initial stab (see `Implementation` below). > The bulk of this effort was fixing all the imports, which I ended up > writing a script to solve. All the tests pass on both sides. I > wouldn't be comfortable with the split as-is (see `Left to do` below) > but it demonstrated to me that it is possible. > > That said, *possible* should not imply *advisable*, and given my > inexperience with the juju project I don't presume to do much more > about splitting out an api repo without feedback. At the very least >
Re: Splitting out state/api into its own repo
On Fri, Jun 27, 2014 at 4:21 PM, John Meinel wrote: > Just my quick thought, I think moving it out from "state/api" into just a > top level "api" would be reasonable and a lot less clumsy than trying to > pull it out into an entirely separate repository. +1 I don't think the api package is useful outside Juju (at this time) and splitting it into another repo just doubles the amount of work. > > I'm not sure if Gustavo realized that "state/apiserver" is the HTTP(-ish, > its really just JSON rpc over a websocket) interface to state. "state/api" > is all go client code (agents or user client code). And it would at least be > really nice for all those things to have an import test that says "does not > depend on anything under "state".) > > It certainly would make it clearer that nothing that is using the api should > import something from, say, apiserver, or state directly. > > John > =:-> > > > On Thu, Jun 26, 2014 at 9:23 PM, roger peppe wrote: >> >> I have a slightly different proposal, inspired by the recent >> Go proposal for internal imports (http://golang.org/s/go14internal), >> which currently looks like it will actually be implemented. >> >> We move all public facing APIs into top level packages within >> the juju repo, and move everything else under >> github.com/juju/juju/internal >> >> Packages I would suggest should be exported: >> >> github.com/juju/juju/api// moved from state/api >> github.com/juju/juju/cmd/juju >> github.com/juju/juju/cmd/jujud >> ... (all the other commands, but not the cmd package itself) >> github.com/juju/ >> github.com/juju/juju/cmd/internal/jujucmd// containing >> supercommand.go from current cmd >> github.com/juju/juju/juju // but only the API-related pieces >> >> Some packages would want moving, because they're user-facing >> but currently inside packages that we would probably not want to >> export: >> >> github.com/juju/environs/configstore >> >> is the only one I can think of right now. >> >> Others would want splitting. For example, the environs package >> is a mix between user-facing and internal stuff right now. >> It would be great to take out all the user-facing config file stuff >> (that might sit >> well inside the juju package). >> >> Still others would benefit from being made available externally. >> I think the rpc package is probably one of those that would >> sit well inside its own repo. >> >> Some packages could benefit from having their own internal >> directory - the uniter is one of those, for example. The >> apiserver too has many sub-packages that should not really >> be visible to the rest of juju. >> >> In the end, we should end up with an API that it might actually be >> feasible to stabilise. I'd like juju to move under gopkg.in at some >> point, providing useful stability guarantees for external users >> that might want to build Go programs based on our code. >> >> cheers, >> rog. >> >> On 26 June 2014 17:41, Eric Snow wrote: >> > (I've put a more structured proposal below, but here's some context.) >> > >> > Over the last couple weeks I've been spinning up on the juju code >> > base, which is large enough to dissipate any hope of understanding it >> > all quickly. Most of what I've focused on is relative to the >> > juju tools and the remote API, both for the sake of backup/restore. >> > >> > One thing that threw me off is that it has not been obvious that the >> > code under `state/api` is the public-facing API for juju's state (as >> > someone recently explained to me). For a while I thought `state/api` >> > held the state API client code, but from what I understand now it >> > actually contains all the public facing code for the juju state API >> > (and [consequently?] for juju as a whole?). In some regard I would >> > expect a public API to be sit in a top-level package (rather than >> > nested down like it is). >> > >> > A couple of other things got in the way. For one, we basically don't >> > have any documentation for the API. I expect that it will mirror the >> > documentation for the juju subcommands/tools pretty closely, but >> > regardless the doc doesn't exist. Setting up godoc for the api >> > package would be great and so would a new page in the juju >> > documentation. I realize there has been some discussion on this point >> > of late, from which I expect a doc will take shape in the short term. >> > In the meantime, we've actually been telling people to wrap calls to >> > the CLI tools rather than using the API. The documentation side of >> > things is a somewhat orthogonal, though topically related, issue. >> > >> > For another, while there has been a pretty good effort to keep the >> > `api` package relatively un-entwined from the rest of juju, there were >> > a few times when I found it hard to follow what was going on. This >> > was particularly true of the underlying state RPC implementation, >> > though at this point things make a lot more sense. Havin
